formative e-assessment developers day

Formative e-Assessment PED 5, Dec 2008

Scoping a vision for formative e-assessment:

Cases and design patterns

http://creativecommons.org/licenses/by-nc-sa/2.0/uk/


Learners?

Problem I: common language

DesignKnowledge

in TEL

Developers

Policy makers

Teachers

Researchers



Problem II: The void

The Prophetswill tell you what should be done

The Explorerswill tell you what they did

Current discussion of learning and technology alternates between the abstract theoretical and the anecdotal.In between there is a shortage of design-level discourse.

?



Wanted: a design science of learning

A science of design has - A value dimension A functional axis of decomposition Attention to representation

(Mor & Winters, 2007)

Herbert Simon (1969): we need a scientific study of the man-made. At its core, the science of design. “everyone designs who devises courses of action aimed at changing existing situations into desired ones”



The Design Knowledge Problem

Expert := one who solves problems in a

particular domain

Expert := has domain design knowledge

Experts do,Consultants talk

Experts talk in jargon

But..



Solution: sharing stories

Stories (narratives) are a fundamental form of generating / sharing knowledge. (Bruner)

Thick descriptions of problems & solutions. Everyone likes a good story.

http://www.slideshare.net/yish/case-study-how-to-presentation






But..

Narratives are not enough: The Aha! Factor

How do we identify the key design element in a story?

The fantasy factor How do we know its true?

The (cognitive) load factor The world is changing too fast for us to take in all

the good stories.



Design patterns

[describe] a problem which occurs over and over again in our environment, and then describes the core of the solution to that problem, in such a way that you can use this solution a million times over, without ever doing it the same way twice(Alexander et al., 1977)

C o n t e x t

Problem Solution



ProblemKeep the rain out

ContextCold, wet, poor.

Method of solutionThatched roof

RelatedTimber frame, Slanted roof,Chimney



example: activity nodes

Design problemCommunity facilities scattered individually through the city do nothing for the life of the city.

Design solutionCreate nodes of activity throughout the community, spread about 300 yards apart.

http://www.uni-weimar.de/architektur/InfAR/lehre/Entwurf/Patterns/030/ca_030.html



Scenarios

The ultimate proof of a pattern language is in its effectiveness as a tool for design.

Ask participants to tell as “fantasy story”: a current design challenge as an “I wish” case story.

Apply patterns to derive solution.



A few cases

Creature of the week CoMo Post 16 String Comparison Open Mentor Academic writing Audio files CLICK ...



Creature of the week Situation

large class (138) first and second year computer science students. assignment: create a virtual pet in Second Life.

Task Engage and motivate the students show examples of good work which others could

learn from show students their work is valued. build a sense of community.





CoMo

Situation Royal Vet College. Hospital rotations as part of their training.

Task Allow students to capture critical incidents in text

and image. Support sharing of clinical experiences and co-

reflection.





Post 16 string comparison

Situation Grammar school been piloting the ‘string

comparison’ approach to language teaching at post-16 for AS and A2 level students.

Sixth Form level, grammatical consolidation and whole-sentence translation.

Task Allow students to practise written language

independently and receive feedback on errors in order to improve their language skills.



Solution

A bespoke string (sequence) comparator was designed; uses fine-granularity sequence comparison to compare correct language strings to a user’s answer. Students answer questions and the comparator marks up errors in their input using colour coding (and font style) to highlight the different types of error. If an answer contains errors the student is given a second attempt in which to correct the submission based on the feedback received.



Open mentor



A few patterns..

Try Once, Refine Once Feedback on Feedback Classroom display Use my Stuff Round and Deep Showcase Learning Three Hats



where the learner is going where the learner is How to get there

teacher

PeerActivating students as learning resources for one another

LearnerActivating students as owners of their learning

Try once refine once; Three hats; Narrative spaces (***)

Clarify and share learning

intentions

Engineering effective

discussions, tasks and

activities that elicit

evidence of learning

Providing feedback that

moves learners forward

Three hats; Narrative spaces (*)

Feedback on Feedback; Soft Scaffolding

Understand and share

learning intentions

Round and deep; Classroom display; Use my stuff; Showcase learning; Three hats; Objects to talk with;

Narrative spaces (*)

Understanding learning

intentions

Narrative spaces

Wear your skills



Try Once, Refine Once

http://patternlanguagenetwork.myxwiki.org/xwiki/bin/view/Patterns/TryOnceRefineOnce

http://patternlanguagenetwork.myxwiki.org/xwiki/bin/view/Patterns/TryOnceRefineOnce



Problem

Lack of immediate feedback for students leads to fossilisation of errors and misconceptions

providing immediate feedback in an iterative fashion can also hinder effective learning since students are able to "grope their way" step-by-step to a correct solution without necessarily having to think about each answer as a whole.



Context

Class size Large (30-300)

Content Skills \ facts

Mode of instruction Blended / on-line. Computer tested.



Solution



Feedback on

Feedback

http://patternlanguagenetwork.myxwiki.org/xwiki/bin/view/Patterns/FeedbackonFeedback

http://patternlanguagenetwork.myxwiki.org/xwiki/bin/view/Patterns/FeedbackonFeedback



ProblemGood feedback should -

Alert learners to their weaknesses.Diagnose the causes and dynamics of these.Include operational suggestions to improve the learning experience.Address socio-emotive factors.

Tutors know this, but are pressed for time. Or not aware of their feedback strategies

Large teaching organisations are not equipped to provide tutors with personal feedback on their teaching



Context

Large scale, technology supported, graded courses many tutors instructing many students.

Feedback is mediated by technology that allows it to be captured and processed in real time

Topic of study is subject to both grading and formative feedback.



Solution

Embed a mechanism in the learning and teaching system that regularly captures tutor feedback, analyses it, and presents them with graphical representation of the types of feedback they have given. Ideally, this should also include constructive advice as to how to shift from less to more effective forms.

In computer supported environments (e.g. VLEs), this mechanism could be integrated into the system, providing tutors with immediate analysis of their feedback, as well as long-term aggregates.



Classroom Display

http://patternlanguagenetwork.myxwiki.org/xwiki/bin/view/Patterns/Classroomdisplay

http://patternlanguagenetwork.myxwiki.org/xwiki/bin/view/Patterns/Classroomdisplay



Problem

Rewards participation.Relates to learner's personal experiences.Window on student conceptions.

Using learner generated Using learner generated content..content..

Needs to collate works in a single easy to access location.Learners uncomfortable about presenting their work in publicLegal or other restrictions on sharing work.



Context

Class size: Small / medium (6-60)

Mode of instruction: Blended (preferable)

Time frame Continuous, over a period

Pedagogy Involves construction / media production



Solution



Task 1: highlight concepts

Hazard: Hazard: needs definition

Asset: Asset: Key common concept



Task 2: design game – add formative assessment to e-Learning

Agree on a scenario (who, what, where) Each team member takes on a role (teacher,

student, mgmt, admin) State desires Find appropriate patterns Propose solution



A scenario is..

Preferably, a real problem you are familiar with. e.g. extending a system you have in development /

in use Define the context of use

Environmental constraints Define the primary roles and workflows

What is the problem to be solved?



Coming up..



Deadline: December 23rd 2008

[email protected]

http://www.iwm-kmrc.de/workshops/e-learning-patterns/

mailto:[email protected]



Review on formative e-Assessment Paper on the CoMo case study Symposium on pattern methodologies

http://www.cal-conference.elsevier.com/



July 8-12, 2009, Irsee Monastery, Bavaria

Deadline: February 14, 2009

http://www.hillside.net/europlop/

http://flickr.com/photos/ademaraguiar/


Education

formative e-assessment developers day