Presentation Assessments

Accreditation standards Accreditation standards

Don Curtis, UCSF

Self-study as an institutional self-assessment

Goals to assess his or her own performance

Graduates must be competent in self-assessment

Assessments at all levels of the academic enterprise

and self-assessmentand self-assessment

Outline

•Types of assessments

•Assessments as an educational tool

•Accreditation standards

Assessments• Diagnostic

Identifies student need and knowledge; usually multiple choice

• Formative

Typically during learning experience, multiple choice, portfolio, peer, self-assessment

• Summative

Quantitative grade, threshold, certification, makes judgment about achievement, typically multiple choice

Types of Formative Assessments

Peer assessments

Self-assessments

Portfolio

Self-assessmentSelf-assessment

• Active process of developing an awareness of personal learning

Self-assessments

•A way to share educational responsibility

•A way to develop patterns of reflective learning

•Provide students with a realistic sense of their own strengths and weaknesses

•Guide students towards responsible self-directed learning

Self-assessment

Ultimate outcome of higher education is for students to develop as independent learners and a defining characteristic of independent learners is accuracy in self-assessment.

Assessment in Higher Education, Cassidy, 2007

Self-assessment articles

0

20

40

60

80

1985 1995 2005

year

nu

mb

er

of

art

icle

s

Pub Med

EricEducation Full Text

Purpose of study

• To determine if students could be taught to improve their ability to self-assess

Curtis DA, Lind SL et al. JDE, 2008

• If students that improved the most on successive self-assessments improved the most on successive examinations

Study design

Entire process repeated 5 weeks later

77 studentsComplete3 hr exam

Complete 1 hrself-assessment

Faculty grading

Results and Relevance

• Results showed self assessment skills improved

Curtis, JDE, 2008;

Koivula, Sex Roles, 2001;

Wright, Educ Meas, 2005

• Self-assessment explained 16% (R2=0.164) of the improvement in examination scores

Assessments• Diagnostic

Identifies student need and knowledge; usually multiple choice

• Formative

Typically during learning experience, multiple choice, portfolio, peer, self-assessment

• Summative

Quantitative grade, threshold, certification, makes judgment about achievement, typically multiple choice

Predicting performance

• SAT is limited in ability to predict first year college GPA

2004, California and the SAT, Zwick,R 77,000 UC applicants High school GPA, SAT I, SAT II,

Showed SAT I predicts 8.4% (R2= 0.084) of first year college GPA



2007, Curtis et al, JDE

GPA, SGPA, PAT, DAT, academic rating of college, units/quarter

Weak correlations; r = 0.15 – 0.36

• Pre-admission criteria to dental school do not predict dental school GPA



• Pre-admission criteria to dental school do not predict dental school GPA

• Summative exams in preclinical courses do not predict clinical performance, even in the same discipline

• Summative exams in preclinical courses do not predict clinical performance, even in the same discipline

2007, JDE, Curtis et al

82 students; correlation between knowledge based OSCE , skills based typodont and clinical competency exam

Typodont; r = 0.03 Clinical ExamOSCE, r = 0.21

Predicting Performance

•Not from summative exams

•Not from pre-clinical skills

•Not from simulation assessments; OSCE etc

•Clinical experience???

•Self-assessment????

Outline



•Accreditation standards

Assessments at their best

•Determine if students are learning what youexpect them to learn

•Can help motivate and direct student learning

•Provide feedback on instructional effectiveness

•Can reinforce and stimulate future learning, discovery

Assessments at their worst

• Frustrate students when assessments not matched to the teaching style



• Motivate students towards performance rather than mastery

Motivate students towards performance rather than mastery

Assessments at their worst• Frustrate students when assessments not matched to

the teaching style


• Empower the assessor over the student being assessed




• Empower the assessor over the student being assessed

• Create unprofessional/unethical behavior

Outline



•Assessments and accreditation standards

Predicting Performance

•Not from summative exams

•Not from pre-clinical skills

•Not from simulation assessments; OSCE etc

•Lesson; similiar experiences predict like experiences

•Clinical experience???

•Self-assessment????

Input to accreditation standards

• Needs to be behavioral more than numbers

• We lack the outcome data to state what the educational best practices should be but---

•We do need to establish foundation knowledge

•Clinical experience is the best predictor of future success

Accreditation standards; Medicine, Nursing

• “trial by fire”

• the experimental model theory:

•Most clinical skills in medicine are taught in an apprentice style, and clinical education is primarily based on experimental learning. Reznick, NEJM,2006

Accreditation standards

•

• Trends in California State Board complaints

• Trends in recent graduate reports

• Access to care issues

•

Summary

• Assessments are powerful tools

• Summative assessments have value but are limited in predicting future performance

• Self-assessment is about students learning to learn which is the most important skill students need to succeed

Thank you