"For people without disabilities, technology
makes things convenient,
Evaluation Team Activities
The purpose of the formative evaluation component is to obtain user input during the development process through a simple methodology with results that will inform design (i.e. problems, sources of error, barriers to access, learnability). The evaluation team is undertaking an analysis of the entire project and providing relevant feedback to the tools development team. Specific evaluation protocols for evaluating player, editor functionality and user interface are to be utilized. Activities are as follows:
A portion of the evaluation for this project has looked at the addition of an American Sign Language (ASL) video translation of an existing educational video and video setting preferences of individuals who use ASL for communication. The objectives of this evaluation were:
In order to accomplish these objectives, the team presented ASL-speaking participants with two different ASL versions of the same educational video, The Law, from the Room for Five series. One of the videos was a direct interpretation of the law video to ASL while the other video was an acted ASL interpretation of the video. Our goal was to see if learners preferred one format over the other and to see if either format had an effect on their learning.
Our participants were also provided opportunity to arrange the ASL and original video appearance through settings options. These options allowed the viewer to make one video window larger than the other video window, to set the position of the two videos relative to each other (i.e. top, bottom, side, near, far) and to choose to place borders around the video windows.
Participants provided information to the evaluation team through pre and post questionnaires, signed feedback and test scores. Additional sources of information to the team include video tapes of the sessions and key-stroke data that records use of the video controls by the participant.
With the help of experimenters and translators from the Canadian Hearing Society, we have been able to collect the relevant data and are in the process of analyzing the information that has been collected. Test scores show no trend towards better learning with either type of translation, however all of the data must be examined before the potential effect of each option is fully understood. At the completion of the analysis we expect to have a better idea of the kind of ASL adaptation that is most cost-effective and valuable to learners. We also expect that the design of the learner interface will reflect the needs and preferences of learners who are deaf particularly with regards to the degree of control the learner will have over video size and positioning.
Additional Evaluation Initiatives:
An expert survey on distance education practices, recommendations and concern for accessibility of content has been undertaken. This process includes inter-rater reliability of the survey data. A paper based on the findings is being prepared.
In addition, a literature review of distance education practices and accommodations for accessibility has been completed. The evaluation team are continuing to monitor the literature and incorporate new results into current work and recommendations.