Ass

essi

ng S

tude

nt L

earn

ing:

Th

e Q

uest

To

Hol

d H

ighe

r Ed

ucat

ion

Acc

ount

able

Ric

h Sh

avel

son

CA

SBS

Sem

inar

Dec

embe

r 13,

200

0

Ove

rvie

w O

f Tal

k

•M

otiv

atio

n fo

r stu

dyin

g hi

gher

edu

catio

n ac

coun

tabi

lity

•Sk

etch

of e

nvis

ione

d st

udy

•In

stitu

tiona

l and

stat

e ac

coun

tabi

lity

syst

ems (

“rep

ort c

ards

”)•

Crit

eria

for e

valu

atin

g re

port

card

s •

Que

stio

ns fo

r us t

o ad

dres

s

Mot

ivat

ion

for S

tudy

•R

espo

nd to

incr

ease

d de

man

d fo

r acc

ount

abili

ty a

s con

ceiv

ed

by, f

or e

xam

ple:

–N

ew Y

ork’

s Rep

ort C

ard

–V

irgin

ia’s

aud

it–

Dev

elop

men

ts in

Eng

land

, New

Zea

land

, Aus

tralia

, Hon

g K

ong

•A

void

K-1

2’s n

egat

ive

cons

eque

nces

exp

erie

nces

: Ben

efits

&

cost

s–

Ben

efits

incl

ude

incr

ease

d co

nten

t ach

ieve

men

t (pr

imar

ily in

bas

ic

skill

s—e.

g. T

enne

ssee

, Tex

as) a

nd te

ache

r res

pons

iven

ess

–C

osts

incl

ude

narr

owin

g ed

ucat

iona

l goa

ls, r

educ

ed fl

exib

ility

, tea

chin

g to

the

test

, and

che

atin

g

•D

evel

op d

esig

n pr

inci

ples

to re

duce

tran

sfer

of i

napp

ropr

iate

co

ncep

tions

of a

ccou

ntab

ility

to h

ighe

r edu

catio

n--o

utpu

ts a

re

ofte

n di

stal

pro

xies

for d

esir

ed o

utco

mes

The

Que

stio

n:

How

Can

The

Pub

lic G

et C

ontro

l O

ver H

ighe

r Edu

catio

n?R

isin

g co

sts,

part-

time

facu

lty, n

on-tr

aditi

onal

stud

ents

, and

for-

prof

it in

stitu

tions

hav

e fu

eled

con

cern

abo

ut h

ighe

r edu

catio

n:

“The

day

s whe

n m

ost p

ublic

off

icia

ls a

nd th

eir c

onst

ituen

ts

view

ed h

ighe

r edu

catio

n as

a in

nate

goo

d de

serv

ing

of p

ublic

m

oney

s, w

ith o

r with

out m

easu

rabl

e ou

tcom

es, a

re o

ver…

.

And

toda

y it

is n

ot p

ossi

ble

in th

e pu

blic

sect

or in

Sou

th

Car

olin

a an

d in

man

y ot

her s

tate

s to

spen

d ta

xpay

ers’

har

d-ea

rned

mon

ey w

ithou

t acc

ount

ing

for h

ow it

is sp

ent,

som

etim

es in

det

ail…

”

--R

aybu

rn B

arto

n, C

omm

issi

oner

of H

ighe

r Edu

catio

n, S

outh

Car

olin

a

The

Res

pons

e: A

ccou

ntab

ility

“One

of t

he p

rim

e to

ols o

f eff

ectiv

e pr

ivat

e se

ctor

m

anag

emen

t is a

n ac

coun

tabi

lity

syst

em th

at

incl

udes

cle

ar g

oals

, a w

ell-d

esig

ned

ince

ntiv

e st

ruct

ure

and

solid

per

form

ance

mea

sure

s.

Bui

ldin

g th

is k

ind

of sy

stem

into

Am

eric

an

educ

atio

n is

a fi

ne id

ea.

But

we

have

to re

cogn

ize

that

the

deve

lopm

ent o

f acc

urat

e ed

ucat

ion

mea

sure

men

ts re

pres

ents

an

enor

mou

s cha

lleng

e”

--Ji

m T

hom

pson

, Pre

side

nt o

f The

RA

ND

Cor

pora

tion,

just

ifyin

g th

etim

ing

of

polic

y pi

eces

on

the

Texa

s Ass

essm

ent S

yste

m in

the

LA T

imes

Aca

dem

ia’s

Rea

ctio

n: P

robl

ems

•SU

NY

stop

s and

stud

ies p

ropo

sed

syst

em-w

ide

stud

ent

com

preh

ensi

ve a

chie

vem

ent t

est

•Sy

stem

-wid

e Fa

culty

Sen

ate

asse

rts th

at:

–Fa

culty

resp

onsi

ble

for g

ener

al e

duca

tion

desi

gn, i

mpl

emen

tatio

n, &

as

sess

men

t–

Cam

pus d

iffer

entia

tion

fund

amen

tal

–A

sses

smen

t sho

uld

be c

ampu

s cen

tere

d–

Ass

essm

ent d

esig

n re

quire

s fac

ulty

repr

esen

tatio

n

•Sy

stem

-wid

e Fa

culty

Sen

ate

reso

lves

that

:–

Prov

ost s

houl

d su

spen

d hi

s sys

tem

-wid

e un

iform

com

preh

ensi

ve te

st o

f st

uden

t ach

ieve

men

t in

the

first

two

year

s of c

olle

ge–

Facu

lty a

nd o

ther

s sho

uld

deve

lop

a pl

an fo

r a c

ampu

s-ba

sed

stud

ent

outc

omes

ass

essm

ent p

rogr

am

Nut

shel

l Of N

ew S

tudy

•H

isto

rical

, pol

itica

l, so

cial

and

con

cept

ual b

ackg

roun

d fo

r stu

dy–

Prec

olle

ge: b

egin

ning

with

com

mon

scho

ol m

ovem

ent

–C

olle

ge: b

egin

ning

with

land

gra

nt in

stitu

tions

and

pos

t W

WII

edu

catio

n be

nefit

s

•Fr

amew

ork

for a

nd c

ase

stud

ies o

f acc

ount

abili

ty•

Fram

ewor

k fo

r and

cas

e st

udie

s of a

sses

sing

lear

ning

:–

Cog

nitiv

e–

Civ

ic re

spon

sibi

lity

(etc

.)

•O

ptio

ns fo

r alte

rnat

ive

acco

unta

bilit

y sy

stem

s fro

m th

e de

cisi

on m

aker

’s p

ersp

ectiv

e

Acc

ount

abili

ty A

nd A

sses

smen

t

•A

ccou

ntab

ility

is a

pro

cedu

re b

y w

hich

a p

olity

(citi

zen,

pol

itici

an,

publ

ic m

anag

er, o

r clie

nt) a

cts t

o ha

ve p

ublic

age

ncie

s acc

ount

for t

he

reso

urce

s the

y us

e an

d th

e ou

tcom

es th

ey c

reat

e.•

An

acco

unta

bilit

y sy

stem

is a

rout

ine,

syst

emat

ic, “

theo

ry-d

riven

” ef

fort

open

to p

ublic

deb

ate

inte

nded

to:

–C

olle

ct d

ata

on 2

or m

ore

orga

niza

tions

–Tr

ansf

orm

thos

e da

ta in

to in

form

atio

n re

leva

nt to

eva

luat

ing

perf

orm

ance

–Tr

ansm

it th

is in

form

atio

n to

som

e au

dien

ce e

xter

nal t

o th

e or

gani

zatio

ns

thro

ugh

scor

es (o

ften

ratin

gs o

r ran

king

s) a

nd so

met

imes

(cas

e) d

escr

iptio

ns•

Ass

essm

enti

s “th

eory

-driv

en”

mea

sure

men

t (an

d de

scrip

tion)

of

indi

cato

rs th

at c

hara

cter

ize

inpu

ts (r

esou

rces

), p

roce

sses

(use

of

reso

urce

s), o

utpu

ts (p

rodu

cts)

and

out

com

es (v

alue

d co

nseq

uenc

es)

The

Com

plex

Acc

ount

abili

ty S

tage

Acc

ount

abili

tyD

eman

d

Top

Dow

n•

Citi

zens

•Po

litic

ians

•B

urea

ucra

ts

Acc

ount

abili

ty C

onte

xt

Inte

rnat

iona

l*

Econ

. Com

petit

ion

* H

.E. D

eman

d *

Acc

t. C

ases

Nat

iona

l*

Stud

ent A

id *

Res

earc

h *

Acc

redi

tatio

n

Reg

ion/

Stat

e

*A

ccre

dita

tion

* A

ccou

ntab

ility

Non

-Gov

ernm

ent

Acc

ount

abili

ty S

uppl

iers

* U

S N

ews *

Zem

sky

(Mkt

. Seg

.) *

PPH

E

His

toric

al

Soc

ial

E

cono

mic

P

oliti

cal

Ju

dici

al

Hig

her E

duca

tion

Inst

itutio

n M

issi

onIn

puts

-->

Proc

ess-

->

Out

puts

-->

Out

com

es

Hum

aniti

es &

Sci

ence

s--

Gen

eral

Edu

catio

n &

D

epar

tmen

ts/P

rogr

ams

Info

rmat

ion

•St

uden

ts

•Pa

rent

s

•G

over

nmen

t &

Cor

pora

te

Purc

hase

rs

Bot

tom

Up

Acc

ount

abili

ty S

yste

m M

odel

s

•A

bsol

ute

Stan

dard

:−

Perf

orm

ance

of a

syst

em is

mea

sure

d ag

ains

t som

e in

tern

al o

r ext

erna

l sta

ndar

d of

m

inim

ally

acc

epta

ble

(or h

ighl

y re

spec

tabl

e) le

vel o

f per

form

ance

(e.g

., N

AEP

)−

Inte

rnal

Aud

itth

at li

nks a

sses

smen

t of l

earn

ing

with

the

teac

hing

and

lear

ning

mis

sion

of

the

inst

itutio

n, w

ith a

n ex

tern

ally

ver

ifiab

le in

tern

al q

ualit

y-co

ntro

l mec

hani

sm (e

.g.,

Col

orad

o)•

Rel

ativ

e St

anda

rd:

−V

alue

-Add

edw

here

a sy

stem

’s p

erfo

rman

ce in

pro

duci

ng le

arni

ng is

com

pare

d ag

ains

t its

exp

ecte

d pe

rfor

man

ce g

iven

the

natu

re o

f its

inpu

ts (T

enne

ssee

) −

Tim

e-Se

ries t

hat m

onito

rs sy

stem

indi

cato

rs o

ver t

ime

(e.g

., gr

adua

tion

rate

s, ac

hiev

emen

t sco

res)

−Ex

tern

al A

udit

that

ties

a sy

stem

’s fu

ndin

g to

rank

ing

of in

dica

tors

such

as g

radu

atio

n ra

tes,

rete

ntio

n ra

tes,

and

facu

lty te

achi

ng a

nd re

sear

ch p

rodu

ctiv

ity (S

outh

Car

olin

a)•

App

roxi

mat

ion

Stan

dard

:A

mod

elth

at e

valu

ates

a sy

stem

aga

inst

kno

wn

pred

icto

rs o

f a sy

stem

’s o

utco

mes

ove

r tim

e su

ch a

s act

ive

lear

ning

, stu

dent

-fac

ulty

inte

ract

ion,

and

stud

ent t

ime

on ta

sk (N

SSE)

Ran

king

Col

lege

s:

Sout

h C

arol

ina’

s Per

form

ance

Fun

ding

*O

utpu

ts(D

irect

Pro

duct

s)

Inpu

ts(R

esou

rces

)

Proc

esse

s(R

esou

rce

Use

)

Out

com

es(G

oals

)

•C

lass

Siz

e &

Sud

ent/

Facu

lty

Rat

io (A

ve. 3

0-35

in u

nive

rsiti

es; 1

6-21

in te

ch c

olle

ges)

•A

vera

ge h

ours

taug

ht b

y fu

ll-tim

e te

achi

ng fa

culty

(Fin

anci

al

ince

ntiv

es to

incr

ease

)

•Pe

rcen

t of f

ull-t

ime

empl

oyee

s w

ho a

re fa

culty

mem

bers

(29.

6 fo

r 4-y

ear c

olle

ges &

40.

1 fo

r 2-y

ear)

•A

ccre

dita

tion

of d

egre

e pr

ogra

ms b

y re

cogn

ized

bod

ies

•In

st. E

mph

asis

on

Teac

her E

d Q

ualit

y &

Ref

orm

(4-y

ear:

accr

edit.

, stu

dent

per

form

ance

on

nat’l

test

s, %

min

ority

gra

ds in

aca

dem

ic

disc

iplin

es w

. tea

cher

shor

tage

s)

•G

radu

atio

n ra

te (2

yea

r de

gree

s in

3 an

d 4

year

deg

rees

in

6)

•St

uden

tEm

ploy

men

t (A

lum

su

rvey

s & st

ate

empl

oym

ent

data

)

•Em

ploy

er fe

edba

ck(S

tate

wid

e su

rvey

of

satis

fact

ion)

•Pe

rcen

t who

pas

s ce

rtific

atio

n ex

am(d

iffer

s by

sect

or)

•Pe

rcen

t gra

ds w

ho

cont

inue

edu

catio

n(E

nrol

l w

ithin

3 y

ears

)

•C

redi

t hou

rs e

arne

d by

gr

ads(

Avo

id m

ore

cred

its th

an

need

ed fo

r gra

duat

ion)

Facu

lty:

•C

rede

ntia

ls (A

ccre

dit a

genc

y cr

iteria

)

•Pe

rfor

man

ce R

evie

w (L

ose

$ if

not f

ollo

w S

outh

ern

accr

editi

ng a

genc

y st

anda

rds)

•Po

st-te

nure

revi

ew (a

s re

com

men

ded

by a

ccre

ditin

g ag

ency

)

•C

ompe

nsat

ion

(Sal

arie

s =>

nat’l

ave

.)

•A

vaila

bilit

y to

Stu

dent

s (A

non.

eva

lby

stud

ents

)

*In

proc

ess

Inpu

ts(R

esou

rces

)

Proc

esse

s(R

esou

rce

Use

)

Out

puts

(Dire

ct P

rodu

cts)

Out

com

es(G

oals

)

•SA

T/A

CT

Scor

es

•H

S G

PA/C

lass

Ran

k

•Se

lect

ivity

•Fi

nanc

ial R

esou

rces

•Pe

rcen

t Stu

dent

s Out

-of

-Pro

vinc

e

•Pe

r Pup

il Ex

pend

iture

•C

lass

Siz

e

•St

uden

t/Fac

ulty

Rat

io

•Fr

osh

Ret

entio

n

•Fa

culty

Rep

utat

ion

•%

Ful

l-tim

e Fa

culty

•Pe

rcen

t Enr

oll

Part

Tim

e

•R

atio

BA

/BS

to T

otal

U

nder

grad

uate

Enr

oll

•Pe

rcen

t 1st

-Yea

r Cla

sses

Ta

ught

by

Tenu

red

Facu

lty

•$C

N sp

ent o

n St

uden

t Se

rvic

es

•Li

brar

ies

•G

radu

atio

n ra

te

•A

lum

ni G

ivin

g

Non

-Gov

ernm

enta

l Acc

ount

abili

ty S

uppl

iers

:

•U

S N

ews &

Wor

ld R

epor

t

•Ze

msk

y’sM

arke

t Map

ping

•B

oth

US

New

s & Z

emsk

y

•M

acC

lean

s

•Bo

th U

S N

ews a

nd M

acC

lean

s

•Al

l Thr

ee

Ran

king

Col

lege

s:

Inst

itutio

nal R

epor

t Car

ds

New

Con

tend

ers:

Inst

itutio

nal &

Sta

te R

epor

t Car

ds

Out

puts

(Dire

ct P

rodu

cts)

Inpu

ts(R

esou

rces

)

Proc

esse

s(R

esou

rce

Use

)

Out

com

es(G

oals

)

•Per

sist

ence

& C

ompl

etio

n

•St

uden

t Rep

ort o

f:

(1) A

cade

mic

cha

lleng

e

(2) A

ctiv

e &

col

lab

lear

ning

(3) I

nter

actio

n w

ith fa

culty

(4) E

nric

hmen

t (ed

abr

oad)

(5) S

uppo

rt fo

r soc

ial l

ife

•Edu

catio

nal

Gai

ns &

Ret

urns

to

Sta

te

•Stu

dent

lear

ning

lear

ning

•Stu

dent

repo

rted

gain

s tow

ard

pers

onal

goa

ls

and

satis

fact

ion

•Pre

para

tion

•Par

ticip

atio

n

•Aff

orda

bilit

y

•Nat

iona

l Cen

ter f

or P

ublic

Pol

icy

and

Hig

her E

duca

tion’

s Sta

te R

epor

t Car

d

•Nat

iona

l Sur

vey

of S

tude

nt E

ngag

emen

t

Crit

eria

For

Eva

luat

ing

Acc

ount

abili

ty S

yste

ms

•Va

lidity

(Fid

elity

of o

utpu

t ass

essm

ent(s

) to

desi

red

outc

omes

)•

Com

preh

ensi

vene

ss(in

clud

es re

leva

nt v

aria

bles

)•

Com

preh

ensi

bilit

y(to

pot

entia

l use

rs)

•R

elev

ance

(to n

eeds

of p

oten

tial u

ser)

•R

easo

nabl

enes

s (de

man

ds o

n or

gani

zatio

n)•

Func

tiona

lity

(lead

s to

appr

opria

te b

ehav

ior)

Sour

ce: G

rom

ley

& W

eim

er (1

999)

Val

idity

Issu

es: E

spec

ially

With

Le

arni

ng O

utpu

tsO

utpu

ts(D

irect

Pro

duct

s)

Inpu

ts(R

esou

rces

)

Proc

esse

s(R

esou

rce

Use

)

Out

com

es(G

oals

)

•A

ccou

ntab

ility

mus

t be

infe

rred

from

obs

ervi

ng o

utpu

ts in

any

syst

em w

here

al

l act

ions

can

not b

e ob

serv

ed d

irect

ly.

•To

do

this

“in

fere

ncin

g,”

the

perf

orm

ance

mea

sure

is a

n in

dica

tor o

f the

de

sire

d be

havi

or, n

ot th

e be

havi

or it

self.

–In

bus

ines

s, th

e ou

tput

mea

sure

(e.g

., re

venu

e or

stoc

k pr

ice)

is a

ver

y cl

ose

prox

y to

val

ued

outc

omes

. It

guid

es b

usin

ess d

ecis

ions

and

act

ions

. Y

ou c

an’t

man

age

a bu

sine

ss if

you

can

’t m

easu

re it

’s o

utco

me.

–In

edu

catio

n, o

utco

mes

are

man

y an

d de

bate

d. T

he o

utco

me

indi

cato

r, m

ost o

ften

a m

ultip

le-c

hoic

e ac

hiev

emen

t tes

t, is

but

a p

roxy

for t

he d

esire

d ou

tcom

e. W

hen

this

indi

cato

r bec

omes

an

end

in it

self,

and

it d

oes i

n ed

ucat

ion,

wel

l-int

entio

ned

acco

unta

bilit

y m

ay v

ery

wel

l dis

tort

the

syst

em it

was

inte

nded

to im

prov

e.

Sour

ce: M

arch

(199

4)

Hig

her E

duca

tion

Out

com

es%

“A

bsol

utel

y E

ssen

tial”

71 68 63 61 60 57 44 32

Goa

l•

Sens

e of

mat

urity

& m

anag

e on

ow

n•

Abi

lity

get a

long

with

peo

ple

diff

eren

t fro

m se

lf•

Impr

oved

pro

blem

-sol

ving

&

thin

king

abi

lity

•Le

arni

ng h

igh-

tech

skill

s•

Spec

ific

expe

rtise

& k

now

ledg

e in

cho

sen

care

er•

Top-

notc

h w

ritin

g &

spea

king

•R

espo

nsib

ilitie

s of c

itize

nshi

p•

Expo

sure

to g

reat

writ

ers a

nd

thin

kers

Sour

ce: J

. Im

mer

wah

rfor

Nat

iona

l Cen

ter f

or P

ublic

Pol

icy

and

Hig

her E

duca

tion

(Oct

ober

200

0)

Alv

erno

Col

lege

C

riter

ion-

Perf

orm

ance

App

roac

hA

cade

mic

Yea

r/Lev

el

12

34

“Abi

litie

s”C

omm

unic

atio

n

Ana

lysi

s

Prob

lem

Sol

ving

Val

uing

in

Dec

isio

n M

akin

g

Soci

al In

tera

ctio

n

Glo

bal

Pers

pect

ive

Effe

ctiv

e C

itize

nshi

p

Aes

thet

ic

Res

pons

iven

ess

Self-

asse

sses

Ana

lytic

Use

s Com

Tech

niq

Inte

gr C

omm

Abi

l

Obs

erve

s In

fers

R

elat

es

Inte

grat

es

Self-

asse

sses

D

efin

es P

rob

Res

olve

s Pro

b

Impl

emen

ts &

Eva

lSol

ID V

alue

s

Infe

rs Im

pl. V

al.

Rel

ates

val

to te

chno

l

A

pplV

al. P

roc

Self-

asse

sses

Ana

lyze

s Grp

s

Eva

lSel

f & G

rp

P

erfE

ffec

t in

Grp

s

Self-

asse

sses

Exam

cpl

x re

lats

Ex

am M

ult P

ersp

Res

pto

Loc

/Glo

b Is

s

Self-

asse

sses

D

evSt

rats

Info

rm R

esp

ID

Org

StrA

ch G

oal

D

esig

n St

rat

Arti

cPs

nl R

esp

Exp

lain

Psn

l Res

p

Rel

ates

wor

k to

Ctx

t M

akes

/Def

end

Qua

l Jud

g

Trum

an C

olle

ge:

Val

ue-A

dded

App

roac

hW

ho T

akes

It?

Nat

iona

lly N

orm

edSu

rvey

sQ

uala

tativ

e

Fres

hman

• C

olle

geA

sses

smen

t of

Aca

dem

icPr

ofic

ienc

y or

• A

cade

mic

Pro

file

Coo

pera

tive

Inst

itutio

nal R

esea

rch

Prog

ram

(UC

LA)

Fres

hman

Wee

kSu

rvey

• A

cade

mic

Pro

file

• Fr

eshm

anIn

terv

iew

• Pr

ojec

t

Soph

omor

eIn

stitu

tiona

l Stu

dent

Surv

eySo

phom

ore

Writ

ing

Expe

rienc

eJu

nior

• C

AA

P or

• A

PIS

S•

AP

• Ju

nior

Inte

rvie

w•

Proj

ect

Seni

orSe

nior

Tes

t in

Maj

or(G

RE)

Gra

duat

ing

Stud

ent

Que

stio

nnai

re•

Cap

ston

e C

ours

e•

Portf

olio

Alu

mni

• A

lum

ni S

urve

y•

Empl

oyer

Sur

veFa

culty

Facu

lty S

urve

y

Ass

essm

ent:

Fram

ewor

k fo

r Cog

nitiv

e O

utpu

tsD

ecla

rativ

ePr

oced

ural

Stra

tegi

cK

now

ledg

eK

now

ledg

eK

now

ledg

e(K

now

ing

the

“tha

t”)

(Kno

win

g th

e “h

ow”)

(Kno

win

g th

e “w

hich

,”

“whe

n,”

and

“why

”)Pr

ofic

ienc

yLo

wH

igh

Exte

nt(H

ow m

uch?

)

Stru

ctur

e(H

ow is

it o

rgan

ized

?)

Oth

ers

(Pre

cisi

on?

Effic

ienc

y?A

utom

atic

ity?)

Cog

nitiv

eC

ogni

tive

Tool

s:To

ols:

Plan

ning

Plan

ning

Mon

itorin

gM

onito

ring

Dom

ain-

spec

ific

cont

ent:

• fac

ts• c

once

pts

• prin

cipl

es

Dom

ain-

spec

ific

prod

uctio

n sy

stem

s

Prob

lem

sch

emat

a/st

rate

gies

/op

erat

ion

syst

ems

Ass

essm

ent o

f Dec

lara

tive

Kno

wle

dge:

M

ultip

le-C

hoic

e--T

IMSS

Pop

. 2

Air

is m

ade

up o

f man

y ga

ses.

Whi

ch g

as is

fo

und

in th

e gr

eate

st a

mou

nt?

A. N

itrog

enB

. Oxy

gen

C. C

arbo

n D

ioxi

deD

. Hyd

roge

n

Ass

essm

ent o

f Dec

lara

tive

Kno

wle

dge

Stru

ctur

e: E

leve

n-Y

ear-

Old

’s C

once

pt M

ap

wat

erra

in

river

scl

ouds

soil

sun

ocea

ns

is fa

lling

wat

er

com

esfr

om

cont

ain

goes

into

rive

rs

flow

to

cont

ain

shin

es o

n

From

Whi

te &

Gun

ston

e: “

Prob

ing

Und

erst

andi

ng”

(199

2, p

. 16)

Ass

essm

ent o

f Pro

cedu

ral K

now

ledg

e:

Perf

orm

ance

of a

Day

time

Ast

rono

my

Inve

stig

atio

n

Flas

hlig

htSt

icky

To

wer

s

Stud

ent

Not

eboo

ks

and

Penc

ils

Stud

ents

are

ask

ed to

mod

el th

e pa

th o

f the

sun

from

su

nris

e to

sun

set a

nd u

se d

irec

tion,

leng

th, a

nd

angl

es o

f sha

dow

s to

sol

ve lo

catio

n pr

oble

ms.

Ass

essm

ent o

f Stra

tegi

c K

now

ledg

e:

Men

tal M

odel

s•

(A) A

rock

et is

mov

ing

alon

g si

dew

ays i

n de

ep sp

ace,

with

its e

ngin

e of

f, fr

om p

oint

A

to p

oint

B.

It is

not

nea

r any

pla

nets

or o

ther

out

side

forc

es.

Its e

ngin

e is

fire

d at

poi

nt

B a

nd le

ft on

for 2

sec

whi

le th

e ro

cket

trav

els f

rom

poi

nt B

topo

int C

. D

raw

in th

e sh

ape

of th

e pa

th fr

om B

to C

. (S

how

you

r bes

t gue

ss fo

r thi

s pro

blem

eve

n if

you

are

unsu

re o

f the

ans

wer

.)•

(B) S

how

the

path

from

C a

fter t

he e

ngin

e is

turn

ed o

ff o

n th

e sa

me

draw

ing.

B

C

A

C

BA

Cor

rect

Inco

rrec

t

Sour

ce: C

lem

ent,

J. (1

982)

. St

uden

ts’ p

reco

ncep

tions

in in

trodu

ctor

y m

echa

nics

. Am

eric

an Jo

urna

l of P

hysi

cs,

50(1

), 66

-71.

Link

ing

Ass

essm

ents

to

Ach

ieve

men

t Com

pone

nts

Dec

lara

tive

Proc

edur

alSt

rate

gic

Kno

wle

dge

Kno

wle

dge

Kno

wle

dge

Perf

orm

ance

Ass

essm

ents

Con

cept

Map

s

•Pe

rfor

man

ce

Ass

essm

ents

•In

terv

iew

s•

M-C

Tes

ts

•M

ultip

le-C

hoic

e•

Fill-

in

Proc

edur

e M

aps

Mod

els/

Men

tal M

aps

Exte

nt

Stru

ctur

e

Oth

ers

Som

e Em

piric

al E

vide

nce

on L

inks

bet

wee

n K

now

ledg

e an

d M

easu

rem

ent M

etho

ds

Cor

rela

tions

from

Shu

ltz’s

Dis

serta

tion

(N=1

09

6th

Gra

ders

Stu

dyin

g Ec

olog

y):

–R

eadi

ng a

nd M

ultip

le-C

hoic

e: 0

.69

–R

eadi

ng a

nd C

once

pt M

ap:

0.53

–M

-C a

nd C

M:

0.60

–R

eadi

ng a

nd P

erfo

rman

ce

Ass

essm

ent:

0.2

5–

M-C

and

PA

: 0.

33–

CM

and

PA

: 0.

43

Dec

lara

tive

Kno

wle

dge

Dec

lara

tive

vs.

Proc

edur

al

Kno

wle

dge

Man

y Q

uest

ions

... B

ut F

or A

Sta

rt

•W

hat s

ocia

l sci

ence

theo

ries p

rovi

de u

sefu

l len

ses

for t

hink

ing

abou

t acc

ount

abili

ty?

•W

hat k

inds

of i

ncen

tives

nee

d to

be

built

in?

•W

hat e

xem

plar

y as

sess

men

t-of-

achi

evem

ent

prac

tices

mig

ht b

e in

corp

orat

ed?

•W

hat s

houl

d be

avo

ided

to re

duce

neg

ativ

e co

nseq

uenc

es?