-
668 Journal of Dental Education Volume 78, Number 5
The Effects of Student Self-Assessment on Learning in Removable
Prosthodontics Laboratory David W. Chambers, Ed.M., M.B.A., Ph.D.;
Eugene E. LaBarre, D.D.S., M.S. Abstract: It has been consistently
shown that there is a weak association between student
self-assessment and faculty member as-sessment of student projects
in preclinical technique laboratory settings and that students
overestimate their performance. Greater overestimation is observed
among students judged by faculty to be the weakest, and these
students also use a wider range of scores. This study hypothesized
that student self-assessment is a function of capacity to perform,
accuracy of understanding grad-ing standards, and psychological
factors. Further it hypothesized that learning, defined as change
in performance, is a function of ability and self-assessment.
Dental students at one U.S. dental school self-assessed their
performance on two projects in a remov-able prosthodontics
laboratory course separated by a six-month period. Faculty
evaluations of these projects were used to deter-mine students
understanding of the criteria for the projects, and a standardized
psychological test was used to assess the learning orientation of
the students. A statistical correction was made for the artifact of
regression toward the mean. The study found that self-assessment
was a better predictor of future learning under these circumstances
than was evaluation by faculty members.
Dr. Chambers is Professor of Dental Education, University of the
Pacific Arthur A. Dugoni School of Dentistry; and Dr. LaBarre is
Associate Professor of Integrated Reconstructive Dental Sciences,
University of the Pacific Arthur A. Dugoni School of Dentistry.
Direct correspondence to Dr. David W. Chambers, Arthur A. Dugoni
School of Dentistry, University of the Pacific, 2155 Webster
Street, San Francisco, CA 94115; 415-929-6438;
[email protected].
Keywords: technique learning, self-assessment, faculty
consistency, dental education, dental students, assessment
Submitted for publication 4/3/13; accepted 10/16/13
The role self-assessment plays in student learn-ing is still
poorly understood. The hope that students with a greater capacity
for accurately judging the quality of their performance will
im-prove more quickly on similar future tasks has often failed to
be confirmed in research studies. Although there are a number of
studies of this effect in the health science education literature,
progress toward understanding this relationship has been slowed by
not having a common concept of self-assessment or insight into how
it works.
A model will be proposed in this article that works with three
variables as predictors of self-assessed performance: student
ability to perform the type of task in question, student ability to
assess performance against standards, and personality
characteristics of students that cause them to filter their
self-assessments. It is suggested that these three factors
intervene between successive performances and bend the rate at
which learning takes place. The general notion is that if students
cannot reliably recognize discrepancies between their performance
and the expected standard or if they distort such perceptions based
on personal needs, their path to improvement will be slowed. This
model will be tested in the context of a preclinical laboratory
course in removable prosthodontics.
Building a Model of the Effect of Self-Assessment on
Learning
In our model, self-assessment is an indepen-dent variable, and
learning, defined as change in performance, is the dependent
variable of interest. As will be explained in this section,
straightforward correlational tests of this relationship are
subject to methodological challenges.
The relationship between experience or practice and subsequent
learning is accepted. Self-assessment has generally been implicated
as part of this process,1-6 with some exceptions reported.7-9 In
studies in which students repeatedly self-assess themselves across
similar activities over a curricu-lum, the gap between student and
faculty evaluations diminishes. This is especially the case when
the task is specific. Exceptions to this smooth improvement occur
when students move to new contexts, e.g., from the laboratory to
the clinical environment or from school to a clerkship.
Student and faculty member ratings of the same performance are
moderately associated, according to previous research. Projects
that are judged better by
-
May 2014 Journal of Dental Education 669
members as a proxy for this value, but other indexes are
possible, such as average scores on several proj-ects or even grade
point average (GPA) across all related laboratory courses. The term
error in this equation does not mean mistakes. It stands for the
influence of all factors that have not been measured. It is assumed
that this error is random and not sys-tematically associated with
any of the terms that are measured.
The first measurement problem is that faculty assessments are
fuzzy. A particular project may be judged differently by a faculty
member on differ-ent occasions (intrarater inconsistency), or
various faculty members may differ from each other on a specific
occasion (interrater inconsistency). Reports in the literature on
faculty consistency in evaluating dental laboratory projects range
from 0.200 < r < 0.700.49-58 An r-value at the high end of
this range would still mean that barely half of the true
differ-ences reported for student performance are captured in
faculty member assessments. Lack of consistency among evaluators
used as the standard places a limit on what can be learned in any
research study using such measurement. If there is no consistency
at all in the standard, no conclusions can be drawn from the
research. The relationship is given precisely by the formula that
the agreement between a measure-ment of interest (student
self-assessment) and the standard (faculty member assessments) can
never be greater than the square root of the agreement within the
standard.59 Using a blunt instrument masks what can be learned
about how well people can use it. One reason why student
self-assessments are not strongly associated with faculty member
assessments of the same work is the inconsistency in faculty
members judgments.
The second hypothesis to be addressed in this project is whether
accuracy of student self-assess-ment plays a role in learning.
Learning is systematic improvement in performance, which in this
case is hypothesized to be influenced by accuracy of student
self-assessment. Those studies that have looked at this question
before have taken the simple correla-tion between student
self-assessed value on the first project and either self-assessed
or faculty-assessed performance on the second project. These
compari-sons are methodologically inadequate.
Looking at learning as change in performance introduces a new
measurement issue. Performance that is very good on the first
occasion will tend on average to be closer to the mean on a
subsequent per-formance, and performance that appears
especially
faculty members are also judged better by students. Consistency
coefficients have been reported from as low as r=0.100 to as high
as r=0.850, but the center of gravity in such associations is in
the range of 0.250 < r < 0.400.10-17 Students demonstrate a
consistent bias in their favor when compared with faculty member
ratings.18-21 When peer ratings are included in the mix, peer
ratings rank in favorability between faculty member and
self-ratings.14,17,22-27
It has frequently been found that the over-assessment bias is
stronger for students who perform poorly than for those who perform
well.6,28-35 Such an effect requires an explanation beyond saying
that it is a deficient understanding of the criteria on students
parts. Presumably a student with a faulty grasp of standards would
be equally likely to underestimate or overestimate the quality of a
work product. It is necessary to distinguish between the direction
of the poor self-assessment and the absolute magnitude of the
error.
Some personality filter could be assumed to be one factor in a
performance-by-assessment in-teraction. There are studies showing
that students modify self-assessments for personal psychologi-cal
reasons.36-39 One line of reasoning suggests that students
self-handicap in circumstances where they know they will be judged
by authorities.40-44 This is a protective behavior since
overconfidence both increases risks for disappointment and is
re-garded as professionally unseemly. It has also been suggested
that filtering involves stable personality characteristics that
vary across individuals.45-48 Tory Higginss work in achievement
orientation47,48 is such an example. His research team has
demonstrated the existence of two stable personality constructs
that guide performance. Prevention Orientation character-izes
individuals with a life habit of pursuing success by avoiding
errors. The other approachPromotion Orientationis to seek success
in life by looking for opportunities to take risks for rewards.
Hypotheses The expectations suggested by the literature on
student self-assessment on future performance can be summarized
in the following hypothesis:
Self-Assessment = Capacity for Performance + Accuracy of
Understanding the Standards + Psychological Characteristics +
error
Capacity for performance must be estimated. It is traditional to
accept assessments made by faculty
-
670 Journal of Dental Education Volume 78, Number 5
grinding-in minimizes embrasures, tooth length and overlap,
group function, no balancing or protrusive prematurities, anatomy
restored, wax bulk appropri-ate, smooth, pre-formed palate uniform,
gingival and rugae contours reproduced, and teeth clean and ready
to process. In most cases, the overall score was the sum of the
scores for individual criteriabut not always. Both students and
faculty members freely overrode the specific criteria, using half
points and approximating the overall score.
The denture set-up and immediate denture projects were
self-assessed by students on the day they were completed and were
then marked by both a single, randomly assigned faculty member and
by the course director. Overall point scores were con-verted to a
100-point scale (percentage) to facilitate comparison between the
two projects. Students were told that their self-assessments would
not factor in their course grade, and faculty members did not have
access to the students self-assessments. Directly after turning in
the immediate denture in winter quarter, students were randomly
assigned to evaluate another students immediate denture project
using the same criteria they had just used for self-assessment and
which faculty members would use for evaluating the projects. The
peer assessments were anonymous.
Differences between overall faculty and student assessments of
the same project were expressed in two forms. S-F Gap was the term
given to dif-ference in overall score obtained by subtracting the
faculty member score from the student score. A positive gap score
indicated cases in which students rated their work more favorably
than did faculty members. An average gap score of 0.0 would
indicate that students and faculty members, on average, as-signed
the same score to each case. But an average agreement could be
achieved by combining widely optimistic students appraisals with
ones that were overly modest. Thus an S-F Absolute Gap score was
also calculated, using only the absolute value of the differences.
Large values on this variable signify that the student was wide of
the mark, without regard to over- or underestimating the score.
Information was gathered on faculty calibration in two ways. As
part of the marking of each project, faculty members were
instructed to assess several randomly chosen projects that had been
previously marked by other faculty members. These measure-ments are
referred to as Field Consistency Ratings. Additionally, nine
examples from the immediate denture project were chosen by the
course director
poor will generally improve. This is not a phenom-enon of
learning: it is a law of statistics that applies wherever r <
1.00. This is known as regression toward the mean. Frequently,
reports in the literature that remedial programs for poorly
performing students showed improvement on subsequent testing are
noth-ing more than a statistical artifact. The magnitude of the
shrinkage toward the middle increases as the consistency of the
evaluations declines.
The second hypothesis in this study is ex-pressed here in
operational terms. The inclusion of an Ability term is used to
manage the measurement problem posed by regression toward the
mean:
Learning = Ability + Accuracy of Self-Assessment + error
Materials and Methods This project was approved by the
Institutional
Review Board at the University of the Pacific in the expedited
category, Protocol #10-33.4. The dataset for this investigation was
based on two laboratory projects in the preclinical removable
prosthodontics course at the University of the Pacific Arthur A.
Dugoni School of Dentistry. In 2011-12, 135 D.D.S. students took
this course in their second year of the three-year program, and
twenty-two International Dental Studies students took the course
together with the D.D.S. students in the first year of their
two-year program. Data were also gathered from ten faculty members
who served as laboratory instructors and participated in the
grading sessions.
Two practical projects were used. A denture set-up was performed
at the end of summer quarter. This project was evaluated on seven
criteria using a 0-to-3 scale, with a maximum summary score of 20.
The criteria for this project included anterior ar-rangement,
occlusal placement, ridge relationship, centric, wax-up,
articulator settings, and readiness for processing. At the end of
winter quarter (six months later), an immediate denture was
fabricated during multiple laboratory sessions. The maximum point
value was 25, and eighteen dimensions were appraised on the
evaluation sheet, with 0 as the lowest possible score and either 1
or 2 as the highest. The cri-teria considered for the immediate
denture included casts centered in articulator, settings and
incisal guide table, vertical dimension, simulate natural position,
tooth length, stable centric, occlusion, minimum one stop per
tooth, buccal cusp corridor, axial orientation,
-
May 2014 Journal of Dental Education 671
r=0.783 for Prevention Orientation and r=0.641 for Promotion
Orientation. The scores for the two ori-entations were uncorrelated
(r=0.100).
ResultsThe three measures of faculty consistency
Field Consistency, Test Case Consistency, and con-sistency
between faculty members and the course directorwere all modest:
r=0.575, 0.565, and 0.577.
Means, Standard Deviations, and Basic Correlations
The basic values and relationships among the variables measured
in this study are shown in Table 1. There are five, color-coded
groups of measures, each having a structure that tells a story. The
pattern of responses for student and faculty ratings on the denture
set-up project in summer quarter is high-lighted in green in the
upper left. It is apparent that students gave themselves slightly
higher marks than did faculty members (82.20 vs. 80.05, p=0.05).
(The S-F Gap score shows that students overestimated their scores
by 4 percentage points. The reason this number is greater than the
difference between 82.20 and 80.05 is that the S-F Gap score is
calculated only for cases where there were both a student and a
faculty score for each case. Some projects were not self-assessed
during this quarter.) The Absolute Gap value is larger than the S-F
Gap because overesti-mates and underestimates in the S-F Gap
measure are not allowed to cancel out in the absolute measure. The
standard deviations are large. There is only a weak r=0.262 (but
still significant at p
-
672 Journal of Dental Education Volume 78, Number 5
between performance on the two projects as judged by faculty
members was more modest, but still sig-nificant (r=0.313, p
-
May 2014 Journal of Dental Education 673
Tabl
e 1.
Mea
ns, s
tand
ard
devi
atio
ns, a
nd c
orre
lati
ons
amon
g va
riab
les
in a
stu
dy o
f fac
ulty
sco
res
and
stud
ent
self-
asse
ssm
ent
and
peer
ass
essm
ent
on a
den
ture
set
-up
and
imm
edia
te d
entu
re p
roje
ct (
n=15
0 ex
cept
whe
re t
here
are
mis
sing
dat
a)
2
3 4
5 6
7 8
9 10
11
12
13
14
15
16
17
18
M
ean
SD
Cor
rela
tion
Sum
mer
(de
ntur
e se
t-up
)
1. S
tude
nt s
core
82
.20
10.0
7 .2
62
.605
.1
92
.541
.0
35
.445
-.
168
.054
.1
40
-.06
0 .0
29
.212
-.
203
-.13
5 -.
360
.139
.0
68
2. F
acul
ty s
core
80
.05
10.7
5
-.61
0 -.
471
.231
.3
13
-.05
9 -.
230
-.02
2 -.
025
.003
.0
68
.870
-.
741
.294
.0
21
.319
.3
62
3. S
-F G
ap
4.01
12
.19
.546
.2
83
-.19
1 .4
33
.086
.0
68
.196
-.
076
-.05
9 -.
531
.421
-.
096
.287
-.
105
-.20
9
4. S
-F A
bsol
ute
9.84
8.
20
.1
88
-.12
7 .2
90
.157
-.
027
.128
-.
121
-.18
3 -.
397
.323
-.
183
.019
-.
018
-.10
3
Win
ter
(im
med
iate
den
ture
)
5. S
tude
nt s
core
86
.70
8.81
.4
07
.604
-.
232
.003
.1
03
-.07
9 -.
029
.356
.0
44
.108
.1
07
.192
.1
73
6. F
acul
ty s
core
86
.55
8.01
-.48
3 -.
185
.217
-.
032
.198
-.
137
.741
.4
06
.127
.0
21
.212
.2
60
7. S
-F G
ap
0.10
9.
19
-.06
0 -.
234
.135
-.
294
.055
-.
308
-.30
9 .0
02
.092
-.
003
-.06
5
8. S
-F A
bsol
ute
6.91
5.
98
-.
049
.042
-.
072
.066
-.
260
.087
-.
241
-.01
6 -.
195
-.14
9
Peer
rat
ing
(im
med
iate
den
ture
)
9. S
tude
nt s
core
87
.31
8.80
.1
84
.658
.0
41
.106
.1
87
-.00
7 .0
31
-.01
0 .0
07 1
0. F
acul
ty s
core
84
.03
8.43
-.61
9 -.
337
-.04
7 -.
015
-.08
3 .1
76
-.08
4 -.
156
11.
S-F
Gap
3.
28
11.0
1
.2
90
.118
.1
56
.070
-.
132
.058
.1
28 1
2. S
-F A
bsol
ute
9.12
6.
95
-.
010
-.14
3 .1
90
-.12
9 .0
52
.078
Ove
rall
perf
orm
ance
13.
Abi
lity
83.5
7 7.
58
-.31
2 .2
83
.033
.3
37
.366
14.
Lea
rnin
g 6.
55
11.1
3
-.18
4 .0
04
-.16
0 -.
172
Prom
otio
n or
ient
atio
n in
vent
ory
15.
Pro
mot
ion
23.0
5 3.
22
.100
.2
52
.255
16.
Pre
vent
ion
18.0
1 3.
60
.0
07
-.03
2
Lab/
clin
ic G
PA in
all
cour
ses
17.
Sum
mer
3.
10
.39
.895
18.
Win
ter
3.14
.3
7
Not
e: G
reen
sho
ws
patte
rn o
f res
pons
es fo
r st
uden
t and
facu
lty r
atin
gs o
n th
e de
ntur
e se
t-up
pro
ject
in s
umm
er q
uart
er a
nd s
ix m
onth
s la
ter
in th
e ra
tings
of t
he im
med
iate
den
ture
pro
ject
an
d pe
er r
atin
gs. R
ed s
how
s as
soci
atio
ns b
etw
een
scor
es a
cros
s th
e tw
o pr
ojec
ts. P
urpl
e sh
ows
corr
elat
ions
invo
lvin
g av
erag
e sc
ores
acr
oss
both
pro
ject
s an
d ga
in in
sco
res
betw
een
proj
-ec
ts. B
lue
show
s re
latio
nshi
ps in
volv
ing
the
pers
onal
ity in
vent
ory
scal
es P
rom
otio
n O
rien
tatio
n an
d Pr
even
tion
Ori
enta
tion.
-
674 Journal of Dental Education Volume 78, Number 5
highest and the twelve who scored lowest on the first project
(denture set-up) are shown. All students in the bottom group
improved on the immediate denture projectsome dramatically so.
Three-quarters of the students in the top group declined in
performance on the second project. The effect of regression toward
the mean was not the same magnitude for initial high and low
performers.
The columns containing blue numbers in Table 1 show
relationships involving the personality inven-tory scales Promotion
Orientation and Prevention Orientation. Generally, associations
involving these traits and other characteristics of the study were
small. Students reporting an orientation toward seek-ing success
scored higher (in the facultys opinion) on both projects. Those
oriented toward avoiding mistakes were those who gave themselves
low scores on the denture set-up (r=-0.360, p
-
May 2014 Journal of Dental Education 675
Both equations were significant, with R-values of 0.631 in the
first case and 0.629 in the second. Each factor on the right-hand
side of the equation entered the equation at the p
-
676 Journal of Dental Education Volume 78, Number 5
the technical aspects of the dental curriculum. The small but
significant correlation between scores on the Promotion Orientation
of the Higgins instrument and lab/clinic GPA is direct confirmation
of this fact. There was no consistent evidence for an everyone
above the average effect or for a handicapping ef-fect. Student
self-assessments were not lower than faculty member assessments,
and students rated their colleagues work slightly more favorably
than their own. Students did not appear to be manipulating
self-assessments as a means of gaming the system. The small
contribution from the Prevention Orientation scale on the Higgins
instrument suggests a cautionary strategy on the part of some
students, but personality explanations of student technique work
are tenuous at best. Ross and Nisbetts book summarizes the social
psychological literature in this field and con-cludes that
correlations between personality traits and specific performance
typically range between 0.100 < r < 0.300 across an extremely
wide range of cases.61,62 Further compounding the low value of
personality explanations is the availability of an almost endless
assortment of mostly overlapping personality constructs.
Before looking explicitly at the association between faculty
member and student assessments, it should be recalled that a
moderate consistency among faculty members themselves places a
restriction on any conclusions that can be drawn from such a study.
By three methods, consistency among faculty mem-bers was measured
at about r=0.570. This means that associations between faculty
member and student self-assessments were constrained.
Mathematically, none could be greater than r=0.75. Faculty marks
were hardly a gold standard.
The more interesting question is the one raised in Hypothesis 2
concerning the role of self-assessment in learning. A significant
association was found between student self-assessment and
improve-ment from the first to the second project. This was only
observed, however, once the masking effect of regression toward the
mean had been removed statistically. In the straightforward test of
predicting faculty member evaluations of performance on the second
project from both faculty assessment and student assessment of the
first project, both groups got it wrong. The correlations were
small and in the opposite direction, and the faculty members
predic-tions were considerably worse. Correcting for regres-sion by
first removing the effects of student GPA or General Capacity
uncovered positive predictions. The faculty members corrected
predictive associa-
entering at p
-
May 2014 Journal of Dental Education 677
about both the product and the process, thus about likely future
performance. The best question, how-ever, is not how good do you
think the product is? but tell me how you achieved this result and
what you might do differently in the future.
Social psychology researchers have made a case that we should
expect self-assessment to exceed assessments made by those who see
from a distance. Self-assessments are normally made based on
private information that might be relevant; are based on longer
consideration and of choices that were made in sequential
activities; conform to categories that are useful to the
self-assessor; and take into account motivation, attention, and
other factors.64,65
The scattergram picture in Figure 3 showing overestimations by
students generally, greater over-estimation by students when
faculty members think they are relatively poor performers, and the
wider spread of self-assessment at the low performance end is
typical of the literature on student self-assessment. If the
interpretation above is sound, this pattern is not grounds for
cynicism, but rather reflects the potential of students to both
learn from reflection and more accurately predict learning than do
faculty members.
Reflection in Practice The classic exploration of learning to
become a
professional is the work of Donald Schn at MIT.66,67 Schn
studied architects, musicians, city planners, and counseling
psychologists and proposed that professionals combine art and
science in practice, dealing with situations of uncertainty,
instability, and uniqueness where commonly accepted standards of
practice are prevalent among colleagues. Schn was a sometimes
biting critic of the pedantry and dysfunc-tional objectivity of
formal professional education and believed instead in the
practicumlearning by doing in a varied but controlled environment
under the care of a coach.
Schn is best known for his distinction between reflection in
practice and reflection on practice. The former takes place during
practice and is the application of heuristics, techniques, and
standards to guide performance and to signal when a task has been
completed or should be modified. Reflection on practice occurs once
performance stops. It is retrospective and may include review of
the process, the assumptions that were in play, and the context. It
helps consolidate the learning from reflection in practice. There
are many practice occasions where one or the other or neither type
of reflection occurs.
tion was r=0.145. Predicting faculty evaluation of the second
project from students self-assessments was significantly better
(r=0.372, p=0.05 for difference in correlations). It is apparent
that students knew something useful about their performance on
labora-tory projects that the faculty members did not know.
The published research most similar to this study was reported
by Donald Curtis et al. in 2008 in this journal.63 That study also
involved student and faculty assessments of denture set-up
labora-tory project on two occasions. Curtis et al.s findings were
almost identical with ours (realizing that they measured agreement
and we measured disagreement so signs for correlations are
reversed), but the conclu-sions differed. Curtis et al. showed that
many of their correlations were significantly different from zero,
but not that they were different from each other. They also made no
correction for regression. The most comparable comparison between
the two studies is Curtis et al.s report that greater agreement
between students and faculty members on the first project
cor-related r=-0.250 with improvement in performance. In other
words, Curtis et al. and we both showed that students who appear to
be poor judges of their first efforts improved the most when asked
to reflect on what they had done and to try again.
What Do Students Know About Their Performance?
In the basic sense, students have some feeling for whether the
project they completed reflects their true capability, and faculty
members do not know this if they only evaluate the product without
knowing how it was produced. In the extreme case, a student may
turn in a perfect take-home laboratory project and receive a
mistakenly high mark because the work was done by an upper
classmate or by the students parents who are dentists. In the more
benign case, a student might make a mistake on an early step in the
procedure, recognize it, and recover as much as possible, but still
have a flawed project because of an early wrong turn. Such a
student might actually know more after this experience than a
clueless classmate who accidentally did the right thing at the
beginning. If we had full information, we would bet on the future
performance of the first not the second student. Faculty members
who evaluate only the product have less usable information about
students abilities than the students do. Asking students to
self-evaluate their products may provide them with an opportunity
to better inform the faculty members
-
678 Journal of Dental Education Volume 78, Number 5
4. Eva KW, Regehr G. Knowing when to look it up: a new
conception of self-assessment ability. Acad Med
2007;82(10):S81-4.
5. Flannelly LT. Using feedback to reduce students judgment bias
on test questions. J Nurs Educ 2001;40(1):10-6.
6. Itching J, Cassidy S, Eachus P, Hogg P. Creating and
validating self-efficacy scales for students. Radio Technol
2011;83(1):10-9.
7. Miller TM, Geraci L. Unskilled but aware: reinterpreting
overconfidence in low-performing students. J Exp Psychol Learn Mem
Cognit 2011;37(2):502-6.
8. Tomic V, Sporis G, Nizic D, Galinovic I. Self-reported
confidence, attitudes, and skills in practical procedures among
medical students: questionnaire study. Coll Antropol
2007;31(3):683-8.
9. Yang H, Thompson C. Nurses risk assessment judg-ments: a
confidence calibration study. Adv Nurs 2010;66(12):2751-60.
10. Herbert WN, McGaghie WC, Droegemueller W, et al. Student
evaluation in obstetrics and gynecology: self- versus
departmental-assessment. Obstet Gynecol 1990;76(3):458-61.
11. Jahan F, Sadaf S, Bhanji S, et al. Clinical skills
assess-ment: comparison of student and examiner assessment in an
objective structured clinical examination. Educ Health (Abingdon)
2011;24(2):421.
12. Jansen JJ, Tan LH, van der Vleuten CP, et al. Assessment of
competence in technical clinical skill of general prac-titioners.
Med Educ 1995;29(3):247-53.
13. Mattheos N, Nattestad A, Falk-Nilsson E, Attrstrom R. The
interactive examination: assessing students self-assessment
ability. Med Educ 2004;38(4):378-89.
14. Papinczak T, Young L, Groves M, Haynes M. An analysis of
peer, self, and tutor assessment in problem-based learn-ing
tutorials. Med Teach 2007;29(5):122-32.
15. Sicaja M, Romic D, Prka Z. Medical students clinical skills
do not match their teachers expectations: a survey at Zagreb
University School of Medicine, Croatia. Croat Med J
2006;47(1):169-75.
16. Sklar DP, Tandberg D. The value of self-estimated scholastic
standing in residency selection. J Emerg Med 1995;13(5):683-5.
17. Sullivan ME, Hitchcock MA, Dunnington GL. Peer and
self-assessment during problem-based tutorials. Am J Surg
1999;177(3):266-9.
18. Blanch-Hartigan D. Medical students self-assessment of
performance: results from three meta-analyses. Patient Educ Cons
2011;84(1):3-9.
19. Bookhan V, Becher LH, Oostheuizen MP. Criteria ref-erenced
student self-assessment in restorative dentistry. SADJ
2005;60(4):161-6.
20. Cho GC, Chee WW, Tan DT. Dental students ability to evaluate
themselves in fixed prosthodontics. J Dent Educ
2010;74(11):1237-42.
21. Tousignant M, DesMarchais JE. Accuracy of student
self-assessment ability compared to their own performance in a
problem-based learning medical program: a correlation study. Adv
Health Sci Educ Theory Pract 2002;7(1):19-27.
22. Weiss PM, Koller CA, Hess LW, Wasser T. How do medical
student self-assessments compare with their final clerkship grades?
Med Teach 2005;27(5):445-9.
Experts are least likely to reflect. There are two journals that
publish articles based on this approach to understanding
professional learning: Reflective Practice: International and
Multidisciplinary Per-spectives (first published in 1999) and
Journal of Natural Inquiry and Reflective Practice (first
pub-lished nine years earlier).
In a former study of Class II cavity prepara-tions, students who
had just finished their preclinical laboratory course in operative
density, those who were a few weeks shy of graduation, and faculty
members were videotaped.68 The type of motions, the length of each
episode, and their sequence were categorized. Among other findings,
it was noticed that beginners inserted a distinct evaluation
activity in the transitions between each step, while experts used
fewer steps (they did not use the same process) and performed their
evaluations at the same time they were producing the product. That
study would be consistent with the interpretation that, with
ex-perience or higher skill level, reflection on practice becomes
folded into reflection in practice.
In the study reported here, students very likely engaged in
reflection in practice and were required by the research design
also to reflect on their practice. Presumably, effects of
reflection in practice were incorporated by students into their
project by the end of the testing period and prior to the
self-assessment activity. Faculty members had no access to either
type of student reflection. Our research question is whether there
is a stable reflective self-assessment activity that promotes
future performance. Alterna-tively expressed, can students benefit
from critiquing their own performance in ways that lead to improved
future performance? Our study suggests that this may be the case. A
promising means for improving learn-ing in preclinical laboratory
performance would be for faculty members to work collaboratively at
the bench with students as they reflect on their practice.
REFERENCES1. Davis DA, Mazmanian PE, Fordis M, et al.
Accuracy
of physician self-assessment compared with observed measures of
competence: a systematic review. JAMA 2006;296(9):1094-102.
2. Devlin H. Students perception of the prognosis for a
single-surface amalgam restoration. Eur J Dent Educ
2012;16(2):86-90.
3. Eva KW, Cunnington JP, Reiter HI, et al. How can I know what
I dont know? Poor self-assessment in a well-defined domain. Adv
Health Sci Educ Theory Pract 2004;9(3): 211-24.
-
May 2014 Journal of Dental Education 679
41. Huchabay LM, Arndt C. Effect of acquisition of knowl-edge on
self-evaluation and the relationship of self-evaluation to
perception of an ideal self-concept. Nurs Res
1976;25(4):244-51.
42. Josephs RA, Bosson JK, Jacobs CG. Self-esteem main-tenance
processes: why low self-esteem may be resistant to change. Pers Soc
Psychol Bull 2003;29(7):920-33.
43. Thomas CR, Gadbois SA. Academic self-handicapping: the role
of self-concept clarity and students learning strategies. Br J Educ
Psychol 2007;77(1):101-19.
44. Thompson T. Re-examining the effects of noncontingent
success on self-handicapping behavior. Br J Educ Psychol
2004;74(2):239-60.
45. Epley N, Dunning D. Feeling holier than thou: are
self-serving assessments produced by errors in self- or special
prediction? J Pers Soc Psychol 2000;79(6):861-75.
46. Gold RS, Brown MG. Trait valence and the better-than-average
effect. Psychol Rep 2011;109(3):734-8.
47. Higgins ET, Friedman RS, Harlow RE, et al. Achievement
orientation from subjective histories of success: promo-tion pride
versus prevention pride. Eur J Soc Psychol 2001;31:3-23.
48. Higgins ET, Spiegel S. Promotion and prevention strate-gies
for self-regulation. In: Baumeister RF, Vohs KD, eds. Handbook of
self-regulation: research, theory, and applications. New York: The
Guilford Press, 2004.
49. Dhuru VB, Ryple TS, Johnston WM. Criterion-oriented grading
system for preclinical operative dentistry labora-tory course. J
Dent Educ 1978;42(9):528-31.
50. Fuller JL. The effects of training and criterion models on
interjudge reliability. J Dent Educ 1972;36(4):19-22.
51. Gains WG, Bruggers HW, Rasmussen RH. Reliability of ratings
in preclinical fixed prosthodontics: effect of objective scaling. J
Dent Educ 1972;38(12):672-5.
52. Hinkelman KW, Long NK. Method for decreasing subjec-tive
evaluation in preclinical restorative dentistry. J Dent Educ
1973;37(9):13-8.
53. Houpt MI, Kress G. Accuracy of measurement of clinical
performance in dentistry. J Dent Educ 1973;37(7):34-46.
54. Milgrom P, Weinstein P, Ratner P, Morrison K. Dentists
self-evaluations: relationship to clinical performance. J Dent Educ
1978;42(4):180-5.
55. Natkin E, Guild RE. Evaluation of preclinical labora-tory
performance: a systematic study. J Dent Educ 1967;31(2):152-61.
56. Ryge G, Snyder M. Evaluating the clinical quality of
restorations. J Am Dent Assoc 1973;87(8):369-77.
57. Salvendy G, Hinton WM, Ferguson GW, Cunningham PR. Pilot
study on criteria in cavity preparation. J Dent Educ
1973;37(10):27-31.
58. Schiff AJ, Salvendy G, Root CM, et al. Objective evaluation
of quality in cavity preparations. J Dent Educ 1975;39(2):92-6.
59. Cronbach LJ. Essentials of psychological testing. 2nd ed.
New York: Harper and Row, 1949.
60. Chambers DW. Learning curves: what do dental students learn
from repeated practice of clinical procedures? J Dent Educ
2012;76(3):291-302.
61. Chambers DW. Lessons from students in a critical think-ing
course: a case for the third pedagogy. J Dent Educ
2009;73(1):65-82.
23. Bryan RE, Krych AJ, Carmichale SW, et al. Assessing
pro-fessionalism in early medical education: experience with peer
evaluation and self-evaluation in the gross anatomy course. Ann
Acad Med Singapore 2005;34(8):486-91.
24. Hawkins SC, Osborne A, Schofield SJ, et al. Improv-ing the
accuracy of self-assessment of practical clinical skills using
video feedback: the importance of including benchmarks. Med Teach
2012;34(4):279-84.
25. Machado JL, Machado VM, Grec W, et al. Self- and
peer-assessment may not be an accurate measure of PBL tutorial
process. BMC Med Educ 2008;8:55.
26. Morton JB, Macbeth WA. Correlations between staff, peer, and
self-assessment of fourth-year students in surgery. Med Educ
1977;11(3):167-70.
27. Rudy DW, Fejfar MC, Griffith CH 3rd, Wilson JF. Self- and
peer-assessment in a first-year communication and interviewing
course. Eval Health Prof 2001;24(4):436-45.
28. Austin Z, Gregory PA. Evaluating the accuracy of phar-macy
students self-assessment skills. Am J Pharm Educ 2007;71(5):89.
29. Austin Z, Gregory PA, Galli M. I just dont know what Im
supposed to know: evaluation self-assessment skills of
international pharmacy graduates in Canada. Res Social Adm Pharm
2008;4(2):115-24.
30. Colthart I, Bagnall G, Evans A, et al. The effectiveness of
self-assessment on the identification of learner needs, learning
activity, and impact on clinical practice: BEME guide no. 10. Med
Teach 2008;30(2):124-45.
31. Edwards RK, Kellner KR, Sistrom CL, Magyari EJ. Medical
student self-assessment of performance on an obstetrics and
gynecology clerkship. Am J Obstet Gynecol 2003;188(4):1078-82.
32. Evans AW, McKenna C, Oliver M. Self-assessment in medical
practice. J R Soc Med 2002;95:511-3.
33. Lane JL, Gottlieb RP. Improving the interviewing and
self-assessment skills of medical students: is it time to readopt
videotaping as an education tool? Ambul Pediatr
2004;4(3):244-8.
34. Langendyk V. Not knowing that they do not know:
self-assessment accuracy of third-year medical students. Med Educ
2006;40(2):173-9.
35. Root Kustritz MV, Molgaard LK, Rendahl A. Comparison of
student self-assessment with faculty assessment of clinical
competence. J Vet Med Educ 2011;38(2):163-70.
36. Ericson D, Christersson C, Manogue M, Rohlin M. Clini-cal
guidelines and self-assessment in dental education. Eur J Dent Educ
1997;1(3):123-8.
37. Kim YH, Chiu CY, Zou Z. Know thyself: misperceptions of
actual performance undermine achievement motiva-tions, future
performance, and subjective well-being. J Pers Soc Psychol
2010;99(3):395-409.
38. Kobori O, Hayakawa M, Tanno Y. Do perfectionists raise their
standards after success? An experimental examina-tion of the
evaluation of standard setting in perfectionism. J Behav Ther Exp
Psychiatry 2009;40(4):515-21.
39. Svenson O. Are we all less risky and more skillful than our
fellow drivers? Acta Psychol 1981;47:143-8.
40. DLima GM, Pearson MR, Kelley ML. Protective be-havioral
strategies as a mediator and moderator of the relationship between
self-regulation and alcohol-related consequences of first-year
college students. Psychol Ad-dict Behav 2012;26(2):330-7.
-
680 Journal of Dental Education Volume 78, Number 5
66. Schn DA. Educating the reflective practitioner: toward a new
design for teaching and learning in the professions. San Francisco:
Jossey-Bass, 1987.
67. Schn DA. The reflective practitioner: how professionals
think in action. New York: Basic Books, 1983.
68. Chambers DW, Geissberger M. Toward a competency analysis of
operative dentistry technique skills. J Dent Educ
1997;61(10):795-803.
62. Ross L, Nisbett RE. The person and the situation:
per-spectives on social psychology. London: Pinter & Martin,
1991.
63. Curtis DA, Lind SL, Dellinges M, et al. Dental students
self-assessment of preclinical examinations. J Dent Educ
2008;72(3):265-74.
64. Greenwald AG, Albert RD. Acceptance and recall of improvised
arguments. J Pers Soc Psychol 1968;8:31-4.
65. Ross M, Sicoly F. Egocentric biases in availability and
attribution. J Pers Soc Psychol 1979;37:322-36.
