formative e-assessment developers day

  • View

  • Download

Embed Size (px)



Text of formative e-assessment developers day

  • 1. Scoping a vision for formative e-assessment: Cases and design patterns

2. Problem I: common language Learners? Design Knowledge in TEL Developers Policy makers Teachers Researchers 3. Problem II: The void The Prophets will tell you what should be done The Explorers will tell you what they did Current discussion of learning and technology alternates between the abstract theoretical and the anecdotal. In between there is a shortage of design-level discourse. ? 4. Wanted: a design science of learning

  • A science of design has -
  • A value dimension
  • A functional axis of decomposition
  • Attention to representation
  • (Mor & Winters, 2007)

Herbert Simon (1969): we need a scientific study of the man-made. At its core, the science of design. everyone designs who devises courses of action aimed at changing existing situations into desired ones 5. The Design Knowledge ProblemExpert := one who solves problems in a particular domain Expert :=has domain design knowledgeExperts do, Consultants talk Expertstalk in jargon But.. 6. Solution: sharing stories

  • Stories (narratives) are a fundamental form of generating / sharing knowledge. (Bruner)
  • Thick descriptions of problems & solutions.
  • Everyone likes a good story.

http:// -study-how-to-presentation 7. But..

  • Narratives are not enough:
  • The Aha! Factor
    • How do we identify the key design element in a story?
  • The fantasy factor
    • How do we know its true?
  • The (cognitive) load factor
    • The world is changing too fast for us to take in all the good stories.

8. Design patterns [describe] a problem which occurs over and over again in our environment, and then describes the core of the solution to that problem, in such a way that you can use this solution a million times over, without ever doing it the same way twice (Alexander et al., 1977) C o n t e x t Problem Solution 9. Problem Keep the rain out Context Cold, wet, poor. Method of solution Thatched roof Related Timber frame,Slanted roof, Chimney 10. example: activity nodes Design problem Community facilities scattered individually through the city do nothing for the life of the city. Design solution Create nodes of activity throughout the community, spread about 300 yards apart. 11. Scenarios

  • The ultimate proof of a pattern language is in its effectiveness as a tool for design.
  • Ask participants to tell as fantasy story: a current design challenge as an I wish case story.
  • Apply patterns to derive solution.

12. A few cases

  • Creature of the week
  • CoMo
  • Post 16 String Comparison
  • Open Mentor
  • Academic writing
  • Audio files
  • ...

13. Creature of the week

  • Situation
    • large class (138)
    • first and second year computer science students.
    • assignment: create a virtual pet in Second Life.
  • Task
    • Engage and motivate the students
    • show examples of good work which others could learn from
    • show students their work is valued.
    • build a sense of community.

14. 15. CoMo

  • Situation
    • Royal Vet College.
    • Hospital rotations as part of their training.
  • Task
    • Allow students to capture critical incidents in text and image.
    • Support sharing of clinical experiences and co-reflection.

16. 17. Post 16 string comparison

  • Situation
    • Grammar school been piloting the string comparison approach to language teaching at post-16 for AS and A2 level students.
    • Sixth Form level, grammatical consolidation and whole-sentence translation.
  • Task
    • Allow students to practise written language independently and receive feedback on errors in order to improve their language skills.

18. Solution

  • A bespoke string (sequence) comparator was designed; uses fine-granularity sequence comparison to compare correct language strings to a users answer. Students answer questions and the comparator marks up errors in their input using colour coding (and font style) to highlight the different types of error. If an answer contains errors the student is given a second attempt in which to correct the submission based on the feedback received.

19. Open mentor 20. A few patterns..

  • Try Once, Refine Once
  • Feedback on Feedback
  • Classroom display
  • Use my Stuff
  • Round and Deep
  • Showcase Learning
  • Three Hats

21. 22. Try Once,Refine Once 23. Problem Lack of immediate feedback for students leads to fossilisation of errors and misconceptions providing immediate feedback in an iterative fashion can also hinder effective learning since students are able to "grope their way" step-by-step to a correct solution without necessarily having to think about each answer as a whole. 24. Context

  • Class size
    • Large (30-300)
  • Content
    • Skillsfacts
  • Mode of instruction
    • Blended / on-line. Computer tested.

25. Solution 26. FeedbackonFeedback 27. Problem

  • Good feedback should -
  • Alert learners to their weaknesses.
  • Diagnose the causes and dynamics of these.
  • Include operational suggestions to improve the learning experience.
  • Address socio-emotive factors.

Tutors know this, but are pressed for time. Or not aware of their feedback strategies Large teaching organisations are not equipped to provide tutors with personal feedback on their teaching 28. Context

  • Large scale, technology supported, graded courses
    • many tutors instructing many students.
  • Feedback is mediated by technology that allows it to be captured and processed in real time
  • Topic of study is subject to both grading and formative feedback.

29. Solution

  • Embed a mechanism in the learning and teaching system that regularly captures tutor feedback, analyses it, and presents them with graphical representation of the types of feedback they have given. Ideally, this should also include constructive advice as to how to shift from less to more effective forms.
  • In computer supported environments (e.g. VLEs), this mechanism could be integrated into the system, providing tutors with immediate analysis of their feedback, as well as long-term aggregates.

30. Classroom Display 31. Problem

  • Rewards participation.
  • Relates to learner's personal experiences.
  • Window on student conceptions .

Using learner generated content..

  • Needs to collate works in a single easy to access location.
  • Learners uncomfortable about presenting their work in public
  • Legal or other restrictions on sharing work.

32. Context

  • Class size:
    • Small / medium (6-60)
  • Mode of instruction:
    • Blended (preferable)
  • Time frame
    • Continuous, over a period
  • Pedagogy
    • Involves construction / media production

33. Solution 34. Task 1: highlight concepts Hazard:needs definition Asset:Key common concept 35. Task 2: design game add formative assessment to e-Learning

  • Agree on a scenario (who, what, where)
  • Each team member takes on a role (teacher, student, mgmt, admin)
  • State desires
  • Find appropriate patterns
  • Propose solution

36. A scenario is..

  • Preferably, a real problem you are familiar with.
    • e.g. extending a system you have in development / in use
  • Define the context of use
    • Environmental constraints
    • Define the primary roles and workflows
  • What is the problem to be solved?

37. Coming up.. 38. Deadline: December 23rd 2008 [email_address] 3