Professor, dr.philos. Arild Raaheim · Digital assessment –pedagogical possibilities Keynote at: Pedagogical Conference, October 27th 2016 Uppsala University Professor, dr.philos

Digital assessment – pedagogical possibilities

Keynote at:

Pedagogical Conference, October 27th 2016

Uppsala University

Professor, dr.philos. Arild Raaheim

Exams…Assessment…Digitalization…Torgny Roxå:

«It’s NOT OK to just have an opinion about assessment.” “It is a whole lot of difference between

having and opinion and putting forward an informed opinion.” https://www.youtube.com/watch?v=MEXJF1ncNwk

UNIVERSITETET I BERGEN2017-01-17 SIDE 2

We can all learn what we want …

but we have to believe in it…

and others must have faith in us!A. Raaheim, 2011.

Was shocked when I saw

the «examination room».Psychology students were sitting

for 5 hours in a zebra-crossing.

(BT 29.05.13)

https://www.youtube.com/watch?v=MEXJF1ncNwk

Scholarship of teaching and learning

(SoTL):

• Knowledge/information

• What do we know about learning and teaching?

• Planning/reflection

• When, how, what, and what when?

• Communication

• What is effective in different settings?

• Evaluation

• Why did I (not) succeed?


The way we assess students determines:

- What students learn

- How students learn

- Students’ conceptions of knowledge (what does it mean to “know”?)

• Assessment is the most powerful tool we

have to direct students’ learning.

• What is it that we want to assess?


Why should we digitalize?

• Because students demand it of us?

• Because we as teachers wish to develop our

skills or demonstrate that we are up-to-date?

• Because this is the future?

• Because there are good reasons to believe that

this will have a positive effect on students’

learning?


Teaching in the digital age – two routes

Low-threshold route

• More advanced PP-

lectures

• Use of clickers/Kahoot

• Smartboards

• LMS: in/out

• Exam via PC

High threshold route

• Reduce number/length of lectures

• Introduce SMOOC’s

• Open source exam

• Online-conferencing

• Local-interactive «distance» teaching

• Learning labs for students’ (video) productions

• New architecture


Digital exams at the University of Bergen

UiB is Norway’s leading institution when it comes to digital exams:

• Inspera Assessment

– 55% of all school exams are digitalized in spring 2016

– an estimated 61% in autumn 2016

– max 62% with today’s system

Technical needs:

Of the remaining 38%:

• 60% formulas (math, chemical, economic, etc)

• 74% special characters (field specific)

• 47% drawings

• 22% third party integrations (56% of these are drawings of chemical

structures, 37% compiling programming code, 7% other systems).


Exams – historical background

• The examination system was systematized in the Sui Dynasty (581–618) as an official method for recruiting bureaucrats. It was intended to ensure that appointment as a government official was based on merit and not on favoritism or heredity.

• Theoretically, any male adult in China, regardless of his wealth or social status, could become a high-ranking government official by passing the imperial examination.

• Candidates were tested on their knowledge of the Confucian classics, their ability to write, and the "Five Studies:" military strategy, civil law, revenue and taxation, agriculture, and geography. Though only about 5 percent of those who took them passed, the examinations served to maintain cultural unity and consensus on basic values and ensured the identification of the educated elite with national, rather than regional, goals and values.

http://www.newworldencyclopedia.org/entry/Imperial_Examinations_(Keju)


http://www.newworldencyclopedia.org/entry/Imperial_Examinations_(Keju)

The exam – some key features

• Control

• Standardization

• Objectivity

• Confidensiality (external examiner/second opinion)

• Reliability

• Validity– internal - external


Assessment – more than exams

• Summativ assessment

• Formativ assessment

• Peer assessment

• Self assessment

Assessment of, for, and as learning.


The «Big-five» of assessment

1 2 3 4 5

Reliable

Valid

Authentic

Transparent

Fair


Very low Very high

Assessment drives learning

Which rises the following question:

• What is it that we want students to learn?

- recall?

- describe?

- explain?

- discuss

- assess?

- evaluate?

- lead a critical discussion?

- demonstrate?

And: What gives us reason to believe that all of the above

can (best) be assessed by using a traditional exam?

Constructive alignment

An ocean of possibilities.

Variation along the following dimensions:

• Level– Whether students are assessed alone or together with others

• Arena– Where assessment is carried out (at the institution, in natural

situation outside the institution, in a virtual room)

• Form– Whether assessment is based on written material, practical

(artistic) work or an oral presentation

• Format– Summative or formative assessment, or a combination of both

Constructive alignment - Rubrics

A rubric is a scoring tool that lists the criteria for a piece

of work, or “what counts” (for example, purpose,

organization, details, voice, and mechanics are often

what count in a piece of writing); it also articulates

gradations of quality for each criterion, from excellent to

poor.

Learning for an unknown future.Content-focused vs capability-focused curriculum.

Knowledge capability:

• work out the key aspects to be dealt with in each new situation

• relate these aspects to knowledge already acquired and/or to other

knowledge the person knows how to access

• determine what the task or problem in the new situation might be

• design a process or solution to deal with the situation, and then

• have the ability to follow through and complete the task or solve the

problem, either alone or with others.

«…knowledge capability … is achieved through experience of variation,

rather than merely having varied experiences» (s. 233).

Baillie, C., Bowden, J.A. & Meyer, J.H.F. (2013). Threshold capabilities: threshold concepts and knowledge

capability linked through variation theory. Higher Education, 63, 227-246.

If there are 50 ways to leave your

lover….

Ooh, slip out the back, JackMake a new plan, StanYou don’t need to be coy, RoyJust listen to meHop on the bus, GusYou don’t need to discuss muchJust drop off the key, LeeAnd get yourself free

© 1975 Words and Music by Paul Simon

There must be a million ways to assess students…

40 alternatives to the traditional written exam:Eksamensrevolusjonen. Gyldendal Akademisk, 2016.

Skriftlig eksamen

Alternativ 1: Med medbrakt «fuskelapp»

Alternativ 2: Med innlagt mulighet for innhenting av informasjon/diskusjon med medstudenter

Alternativ 3: Med åpenhet til alle typer kilder

Alternativ 4: «Take away» eksamen

Alternativ 5: Individualisert eksamen

Alternativ 6: Objektiv prøve (multiple choice)

Alternativ 7: Omvendt objektiv prøve

Alternativ 8: Studenten som sensor

Alternativ 9: Stasjonseksamen

Alternativ 10: Fagartikkel

Muntlig eksamen

Alternativ 11: Med tid til forberedelse

Alternativ 12: Som forberedt klage/ankemulighet

Alternativ 13: Posterpresentasjon

Alternativ 14: Praktisk muntlig

Alternativ 15: Disputas

Alternativ 16: Foredrag

Alternativ 17: Intervju

Andre vurderingsformer

Alternativ 18: Bidrag på vitenskapelig konferanse

Alternativ 19: Mappevurdering

Alternativ 20: Virtuell konferanse

Alternativ 21: Praktisk oppgave utenfor institusjonen

Alternativ 22: Oppdragsvirksomhet

Alternativ 23: Logg

Alternativ 24: Intervju av fagperson

Alternativ 25: Utplassering

Alternativ 26: Prosjektpresentasjon

Alternativ 27: Vurdering av medstudenter

Alternativ 28: Kronikk

Alternativ 29: Litteraturanmeldelse

Alternativ 30: Kursanmeldelse

Alternativ 31: Planlegging av undervisningsopplegg

Alternativ 32: Blogginnlegg

Alternativ 33: Facebook-gruppe

Alternativ 34: Film

Alternativ 35: TBL-aktivitet

Alternativ 36: Arrangere faglig aktivitet

Alternativ 37: Analyse av vurderingsformen

Alternativ 38: Studenten som underviser

Alternativ 39: Pasienten som sensor

Alternativ 40: Ekstern sensur

Team based learning (TBL)

Four pillars/principles:• Permanent teams (5-7students)

• Readiness assurance process (RAP)

– Prereading

– iRat

– tRat

– Appeals

• 4-S application activities

– S1 – significant problems

– S2 – specific choices/answers

– S3 – Same problems

– S4 – simultaneous reporting

• Peer evaluation

http://www.teambasedlearning.org/

http://www.teambasedlearning.org/

The student as examiner – critical

thinking

The student is provided with a full text (answer)

and asked to assess it:

Is this a good answer/paper?

- why/why not?

- what is lacking?

- which part(s) of the literature is lacking, why?

- which mark would you suggest, why?

Assessment based on a set of explicit criteria.

Full access to all sources

A fully digitalised exam instead of today’s digital

school exam;

- puts high demands on formulation of tasks (as the point is not to

assess students’ ability to reproduce what they have read, but their

critical use of different sources)

- puts high demands on those who assess students

- assessment criteria have to be spelled out

Flipped MCQ-test

Objective tests have limited value; may test lower cognitive skills like memory and understanding (declarative knowledge not procedural knowledge).

a) - provide students with an already finished MCQ

test

- ask students to argue why particular answers are

correct/wrong

b) - students develop an MCQ-test

- write a reflective note in which they argue for

choice of questions/answers

A virtual conference

Use existing LMS:

- conference «here and now» (x no. days)

- group assignment/work

- oral presentation

- accepted paper (supervision account)

- full paper distributed in advance

- conference organised in sessions with moderator

- feedback – external/international expert

- moderator and expert assess (peer assessment?)

Interview an expert

A good interview requires much preparation and insight in topic/field.

And that one has understood the interviewee and is able to refer

him/her correct.

- interview an expert

- write an article (newspaper)

- citation check: the interviewee writes a short note:

- did the student get the message?

- was I correctly referred?

- did the student ask relevant/good questions?

- some questions that was not asked?

- how well had the student prepared him/herself?

Publish the article? Own net page or «public»?

Workplace training: illustrate relevance, promote

motivation and learning, and build identity!

And write individual blog:

http://biopraksis.b.uib.no/


Blog

Workplace controls/accepts attendance (3 weeks of practice). Students complete a portfolio:

– 4 blogs

– Reflective paper

– Oral presentation

• Pass/no pass


Gaute Velle Torstein Nielsen Hole



Teach to Learn – creativity,

communication and learning.

TE2LE

The Teach to Learn (TE2LE) project aims to teach BSc and MSc students in biology cross-disciplinary transferable skills through the application of an innovative pedagogical practice.

– 4 min instructional videos • Biolab (Bio 101)

• bioST@TS (AB 204) co operation with MatRIC (Univ. of Agder)

Jonathan Soulé Anne-Laure Simonelli Lucas M. Jeno


http://teach2learn.b.uib.no/

Documents

Professor, dr.philos. Arild Raaheim · Digital assessment –pedagogical possibilities Keynote at: Pedagogical Conference, October 27th 2016 Uppsala University Professor, dr.philos