View
31
Download
0
Category
Tags:
Preview:
DESCRIPTION
Is a Mock Best Option MCQ a Valid Measurement of Student Performance in Final Clinical Medicine A. Patterson, M. Shuhaibar, C. O’Morain , M. Hennessy, S.R. McCann University of Dublin, Trinity College, Ireland AMEE CONFERENCE 2009. BACKGROUND: - PowerPoint PPT Presentation
Citation preview
Is a Mock Best Option MCQ a Valid Measurement of Student Performance in Final Clinical Medicine
A. Patterson, M. Shuhaibar, C. O’Morain, M. Hennessy, S.R. McCann University of Dublin, Trinity College, Ireland
AMEE CONFERENCE 2009
BACKGROUND: Validity is one of the core principles on which assessment is based. Validity is the extent to which a test actually measures what it claims to measure1. This study builds on previous work2 examines both predictive and face validity of an examination. Predictive validity examines if the test can predict future performance in a particular area. Face validity is the least scientific measure of validity but it pertains to the students’ perception of the test and if they believe it is a measure of their overall knowledge. If a test has predictive validity it allows students see how they are progressing in the course and highlights if they need to amend their study strategies to perform better.
OBJECTIVES: to compare student performance in a mock best option multiple choice question examination with their performance in a Final Clinical Medicine MCQ examination and their clinical examinations for 2008 and 2009. In addition, the analysis investigated if the mock examination completed by students three months prior to the real examination could predict how students perform in the real examination. Also, to ascertain students’ perceptions of the examination.
DISCUSSION: Students performed better in the real examination versus the mock examination, there was a period of three months between the tests when an intensive course in Clinical Medicine was provided for students. The predictive validity of the mock examination was tested and proved.Students valued the results of the mock examination as a valuable indicator of how they were proceeding in the course and commented that it reduced the anxiety of sitting the real examination.
CONCLUSIONS: The mock examination is a valuable examination that provides students with an opportunity to see how they are progressing in the course. It is under consideration to convert the mock into a real examination. However, as students perform better in the real examination and the tests are examining equivalent knowledge, this study suggest it should be continued as a purely formative examination.
REFERENCES: 1.Brown, G., Bull, J. and Pendlebury, M., 1997. Assessing student learning in higher education. , Routledge, London2.A. Patterson, M. Shuhaibar, Prof. C. O’Morain, The Predictive Validity of a Mock MCQ Examination of Student Performance in a Final Clinical Medicine MCQ examination, INMED Conference, Ireland, 2009
RESULTS: The results for 2008 and 2009 were compared for students who completed both tests using a Pearson correlation coefficient (N= 103, 2008 and N = 116, 2009). There was a strong, positive correlation between both tests, 2008 and 2009 r=0.63, 2009 r = 0.387 respectively. Both correlations were significant. The mock examination results were also compared with students clinical performance and their overall mark. There were significant correlations with all these measurements.A paired t-test was performed to see if there was any significant difference in student performance between the two tests. Students completed an evaluation of the examination.There was no significant difference between student performance on both tests with t<0.001. The means for the mock and the real examinations were 60.6 ± 10.5 and 68.7 ± 12.5, 52.3 ± 8.8 and 67.9 ± 9.5 for 2008 and 2009 respectively.
“It was an insight. Thank
you.” “It has given me
the motivation to study”
2008N= 103
Mock MCQ
Real MCQ
Minor cases
Major cases
Final Mark
Mock MCQ
0.626** 0.433** 0.423** 0.655**
Real MCQ
0.626** 0.563** 0.566** 0.790**
Minor cases
0.433** 0.563** 0.576** 0.819**
Major cases
0.423** 0.566** 0.576** 0.803**
Final Mark
0.655** 0.790** 0.819** 0.803**
2009N= 103
Mock MCQ
Real MCQ
Minor cases
Major cases
Final Mark
Mock MCQ
0.387** 0.415** 0.224** 0.526**
Real MCQ
0.387** 0.354** 0.218* 0.587**
Minor cases
0.415** 0.354** 0.298** 0.789**
Major cases
0.224** 0.218* 0.298** 0.639**
Final Mark
0.526** 0.587** 0.789** 0.639**
**Correlation is significant at the 0.01 level* Correlation is significant at the 0.05 level
Recommended