Six Facets of Instructional Product Evaluation Review Effectiveness Maintenance Formative Needs Assessment Impact

  • View
    213

  • Download
    0

Embed Size (px)

Text of Six Facets of Instructional Product Evaluation Review Effectiveness Maintenance Formative Needs...

  • Slide 1
  • Slide 2
  • Six Facets of Instructional Product Evaluation Review Effectiveness Maintenance Formative Needs Assessment Impact
  • Slide 3
  • Development Activities Product Conceptualization Design Development Project Re-conceptualization Implementation Institutionalization Evaluation Functions Review Needs Assessment Formative Evaluation Maintenance Evaluation Effectiveness Evaluation Impact Evaluation
  • Slide 4
  • EMCC Design Document Urban Science Course Environment
  • Slide 5
  • Dimensions of effective technology enhanced learning environments: l Task-Oriented l Challenging l Collaborative l Constructionist l Conversational l Responsive l Reflective l Formative
  • Slide 6
  • Task-Oriented The tasks faculty set for students define the essence of the learning environment. If appropriate, tasks should be authentic rather than academic. AcademicAuthentic
  • Slide 7
  • Task-Oriented Example Students in online instructional design courses are tasked with designing interactive modules for real clients.
  • Slide 8
  • Challenging The notion that interactive learning is easy should be dispelled. Learning is difficult and students should not spoon fed simplified versions of their fields of study. SimpleComplex
  • Slide 9
  • Challenging Example In a Masters of Public Health program, students confront problems as complex and difficult as the ones theyll face in the real world.
  • Slide 10
  • Collaborative Web-based tools for group work and collaboration can prepare students for team work in 21st Century work environments. UnsupportedIntegral
  • Slide 11
  • Collaborative Example Art, dance, and music students are collaborating to produce online shows with digital versions of their works and performances for critique by international experts.
  • Slide 12
  • Constructionist Faculty should engage students in creating original knowledge representations that can be shared, critiqued, and revised. ReplicationOrigination
  • Slide 13
  • Constructionist Example Students in fields ranging from aero- engineering to zoo management are producing digital portfolios as integral components of their academic programs.
  • Slide 14
  • Conversational Students must have ample time and secure spaces for in-depth discussions, debates, arguments, and other forms of conversation. One-wayMulti-faceted
  • Slide 15
  • Conversational Example New knowledge and insight are being constructed in conversation spaces such as the e-learning forums found in BlackBoard, WebCT, Desire2Learn, and other online learning authoring platforms.
  • Slide 16
  • Responsive In learning communities, both faculty and students have a mutual responsibility to respond quickly, accurately, and with respect. SuperficialGenuine
  • Slide 17
  • Responsive Example This is an area where R&D are needed. Some universities are seeking to establish supportive online networks that will continue throughout a career, indeed throughout a life.
  • Slide 18
  • Reflective Both faculty and learners must engage in deep reflection and metacognition. These are not instinctive activities, but they can be learned. ShallowDeep
  • Slide 19
  • Reflective Example Teacher preparation students are keeping electronic journals to reflect upon the children they teach, and their roles as advocates for children.
  • Slide 20
  • Formative Learning environments can be designed to allow students to develop prototype solutions over time rather than to find one right answer that someone else has defined. Fixed AssessmentDevelopmental
  • Slide 21
  • Formative Example Faculty should engage their students in ongoing efforts to evaluate and refine their work related to authentic tasks to encourage lifelong learning.
  • Slide 22
  • Task-Oriented Challenging Collaborative Constructionist Conversational Responsive Reflective Formative Traditional Course Online Course
  • Slide 23
  • Heuristic Review
  • Slide 24
  • What is usability? The concern with designing software applications which people find easy to use and personally empowering. Usable computer programs are logical, intuitive, and clear to the people who use them.
  • Slide 25
  • Web Site Usability The most common user action on a Web site is to flee. at least 90% of all commercial Web sites are overly difficult to use..the average outcome of Web usability studies is that test users fail when they try to perform a test task on the Web. Thus, when you try something new on the Web, the expected outcome is failure. Jakob Neilsen Edward Tufte
  • Slide 26
  • Typical Web Usability Problems bloated page design internally focused design obscure site structures lack of navigation support writing style optimized for print Jakob Neilsen http://www.useit.com/
  • Slide 27
  • Key Usability Principles Structure - organize meaningfully Simplicity - make common tasks easy Visibility - all data needed for a task Feedback - keep users informed Tolerance - allow cancel, back Reuse - reduce the users' need to remember
  • Slide 28
  • Nielsens Web Usability Rules Visibility of system status Match between system and real world User control and freedom Consistency and standards Error prevention Recognition rather than recall Flexibility and efficiency of use Help users recognize, diagnose, and recover from errors Help and documentation Aesthetic and minimalist design
  • Slide 29
  • Two Major Ways to Evaluate Usability Heuristic Review quick and relatively inexpensive based on expert analyses no user involvement Usability Testing finds more problems user involvement increases validity when designers see problems live, it has a huge impact
  • Slide 30
  • Heuristic Review Several experts individually compare a product to a set of usability heuristics Typical heuristic: Visibility of system status The system should always keep users informed about what is going on, through appropriate feedback within reasonable time.
  • Slide 31
  • Heuristic Review Violations of the heuristics are evaluated for their severity and extent Severity Scale: 1 Cosmetic: fix if possible. 2 Minor: fixing this should be given low priority. 3 Medium: fixing this should be given medium priority. 4 Major: fixing this should be mandatory before the system is launched. If the problem cannot be fixed before launch, ensure that the documentation clearly shows the user a workaround. 5 Catastrophic: fixing this is mandatory; no workaround possible. Extensiveness Scale: 1 Single case 2 Several places 3 Widespread
  • Slide 32
  • Heuristic Review At a group meeting, violation reports are categorized and assigned Heuristics violated are identified Average severity and extensiveness ratings are compiled Opportunities for improvement are clarified Feasible solutions are recommended
  • Slide 33
  • Heuristic Review Example of Opportunity For Improvement Opportunity 1 (4 reports. Avg. Severity=2.25, Avg. Extent=2.34, Heuristics Used: 1, 3) Consider providing more user feedback about where they are and what they should do next. Examples cited: No Page progress indicator No indication of how to start Suggestions: Provide a page-progress indicator, such as page 3 of 12 Put a Click a section below to start: on the first screen, as a TOC header
  • Slide 34
  • Heuristic Review Disadvantages Advantages Quick: Do not need to find or schedule users Easy to review problem areas many times Inexpensive: No fancy equipment needed Validity: No users involved Finds fewer problems (50% less in some cases) Getting good experts can be challenging Building consensus with experts is sometimes difficult
  • Slide 35
  • Another Weakness Some people believe that heuristic evaluation is too subjective. Human judges are prone to poor judgment at times.
  • Slide 36
  • Usability Standards http://www.astd.org/ASTD/marketplace/ecc/ecc_home ASTD offers certification of e-learning courses, including 8 usability standards: -Navigation -Orientation -Feedback cues -Link cues -Links labeled -Help -Legibility -Text quality
  • Slide 37
  • Heuristics for E-Learning Evaluation 1. Visibility of system status: The e-learning program keeps the learner informed about what is happening, through appropriate feedback within reasonable time. red for a problem yellow for a warning green for OK
  • Slide 38
  • Heuristics for E-Learning Evaluation 2. Match between system and the real world: The e-learning programs interface employs words, phrases and concepts familiar to the learner or appropriate to the content, as opposed to system- oriented terms. Wherever possible, the e-learning program utilizes real-world conventions that make information appear in a natural and logical order.
  • Slide 39
  • Heuristics for E-Learning Evaluation 3. Error Recovery and Exiting: The e- learning program allows the learner to recover from input mistakes and provides a clearly marked exit to leave the program without having to go through an extended dialogue.
  • Slide 40

Recommended

View more >