Five-Year Summary of COMLEX-USA Level 2-PE …jaoa.org/pdfaccess.ashx?url=/data/journals/jaoa/932124/114.pdf · 116 • JAOA • Vol 110 • No 3 • March 2010 of graduation, most

114 • JAOA • Vol 110 • No 3 • March 2010 Langenau et al • Medical Education

The authors present data on examination format and exam-inee demographics, performance, and survey results for theComprehensive Osteopathic Medical Licensing Examina-tion-USA Level 2-Performance Evaluation (COMLEX-USALevel 2-PE) from the first five testing cycles (2004-2005 to2008-2009). First-time examinees in the 2004-2005 testingcycle had a pass rate of 96.1%, compared with a pass rate of94.7% for first-time examinees in the 2008-2009 testing cycle.Pass rates were fairly consistent across all testing cycles.Based on postexamination survey results from all testingcycles, the majority of examinees reported that the cases inCOMLEX-USA Level 2-PE represented appropriate chal-lenges for fourth-year osteopathic medical students. Themajority of examinees also reported that comprehensivestandardized patient-based examinations and exercises wereadministered through their colleges of osteopathic medicine.In addition, survey results indicated overall satisfactionamong examinees with the administration of COMLEX-USA Level 2-PE. J Am Osteopath Assoc. 2010;110(3):114-125

June 25, 2009, marked the end of the fifth testing cycle forthe Comprehensive Osteopathic Medical Licensing Exam-

ination-USA Level 2-Performance Evaluation (COMLEX-USA Level 2-PE), administered by the National Board ofOsteopathic Medical Examiners (NBOME). This examina-tion was developed to assess the fundamental clinical skillcompetence of osteopathic medical students who arepreparing to enter graduate medical education and workingto obtain medical licensure.

Testing medical students for clinical skills competenceusing the objective structured clinical examination format wasfirst described in Scotland in 1975.1 Objective structured clin-ical examinations and standardized patient-based examina-tions—including simulations by standardized patients (SPs)portraying typical medical complaints within an ambulatorysetting—have since become the methods of choice for assessingclinical skills of medical students.2-7

Since 2004, the COMLEX-USA Level 2-PE has been usedto fulfill NBOME’s mission of protecting the public andenhancing patient safety through the evaluation of clinicalskills proficiency of graduates from colleges of osteopathicmedicine (COMs). All students graduating from COMs afterJanuary 1, 2005, have been required to pass COMLEX-USALevel 2-PE in order to complete the entire COMLEX-USAseries.8 In addition, the American Osteopathic Association(AOA) Commission on Osteopathic College Accreditation(COCA) has instituted a policy requiring all osteopathic med-ical students graduating after December 1, 2007, to pass boththe PE and the Cognitive Evaluation (CE, a multiple choice testformat) components of COMLEX-USA Level 2 before gradu-ation.9

In the present report, we use data from the first five testingcycles (2004-2005 to 2008-2009) to describe the format of andrequirements for COMLEX-USA Level 2-PE; the demographiccharacteristics of examinees; the pass/fail rates and trends inexaminee performance on the two domains of COMLEX-USALevel 2-PE (“Humanistic Domain,” “Biomedical/Biome-chanical Domain”); and the survey responses by examineesregarding the examination and their previous exposure tostandardized patient encounters.

FormatStandardized patient-based examinations and objective struc-tured clinical examinations are routinely used for formativeassessments in osteopathic and allopathic medical schools7,10-13and graduate medical education programs,14-17 as well as forsummative assessments in medical schools12,18,19 and licen-sure.4

During COMLEX-USA Level 2-PE, examinees rotatethrough 12 stations, in which they evaluate SPs who havebeen trained to simulate a variety of typical clinical presenta-tions. Each of the 12 stations includes a 14-minute patient-

Five-Year Summary of COMLEX-USA Level 2-PE Examinee Performance and Survey Data

Erik E. Langenau, DO; Caitlin Dyer, MA; William L. Roberts, EdD; Crystal Wilson, MEd; and John Gimpel, DO, MEd

From the National Board of Osteopathic Medical Examiners (NBOME) NationalCenter for Clinical Skills Testing in Conshohocken, Pennsylvania.

Financial Disclosures: None reported. Address correspondence to Erik E. Langenau, DO, Vice President for the

National Center for Clinical Skills Testing, NBOME, 101 W Elm St, Suite 150,Conshohocken, PA 19428-2004.

E-mail: [email protected]

Submitted August 24, 2009; accepted October 7, 2009.

MEDICAL EDUCATION

Downloaded From: http://jaoa.org/pdfaccess.ashx?url=/data/journals/jaoa/932124/ on 07/31/2018

JAOA • Vol 110 • No 3 • March 2010 • 115

Examinee CharacteristicsThe number of COM matriculates and graduates has increasedsince 2004 as a result of expanded class sizes among existingosteopathic medical schools and the opening of new branchcampuses.21 The first-year enrollment for osteopathic med-ical students in 2004 was 3646, increasing to 4528 by 2007.22With this increase in the number of students entering theosteopathic medical profession, the total number of COMLEX-USA Level 2-PE examinees—including first-time examineesand repeat examinees—has also increased with each testingcycle (Figure 1).

The COMLEX-USA Level 2-PE is offered year-round,though the number of examinees tested per month variesthroughout each testing cycle (Figure 2). The number of first-time examinees in the 2004-2005 testing cycle was greatest inmonths near the end of the cycle. During the past four testingcycles, however, there has been a steady increase in the numberof first-time examinees at the beginning of each cycle.

In response to the COCA requirement that students com-plete both COMLEX-USA Level 2-CE and Level 2-PE beforegraduation, the NBOME increased testing capacity so thatstudents could take the COMLEX-USA Level 2-PE before Jan-uary 31 during the 2007-2008 and 2008-2009 testing cycles. Bytaking this examination before January 31 of the expected year

physician encounter (ie, case), followed by a 9-minute periodin which the examinee completes a written note about theencounter—the SOAP (Subjective, Objective, Assessment,Plan) Note. Cases are assigned under blueprint content cate-gories (eg, cardiovascular, gastrointestinal, neuromuscu-loskeletal, respiratory, other clinical presentations) and aredesigned to vary in SP age, sex, and race/ethnicity. Cases alsovary in clinical complaints, which could be acute or chronic innature or provide opportunities for health promotion or dis-ease prevention.

At the start of the examination, students are randomlyassigned to a pretest station, which may contain unscoredmaterial used to evaluate new SPs and cases. Research showsno benefit from the administration of unscored material atany time during the examination.20 Examinee performancemeasures are collected from each of the test stations.

A student is eligible to take the COMLEX-USA Level 2-PE if he or she has passed the COMLEX-USA Level 1, hascompleted his or her second academic year at a COM accred-ited by the COCA, and has been approved to take the exam-ination by the COM’s office of the dean. If an individual hasalready graduated from a COM, that individual must pro-vide a verified copy of his or her diploma to take COMLEX-USA Level 2-PE.8

Langenau et al • Medical Education

MEDICAL EDUCATION

5000

4500

4000

3500

3000

2500

2000

1500

1000

500

0

Exam

inees, No

COMPLEX-USA Level 2-PE Testing Cycle

2004-2005 2005-2006 2006-2007 2007-2008 2008-2009

2742 3005

3261

3751

4698

Figure 1. Number of examinees taking the Comprehensive Osteopathic Medical Licensing Examination-USA Level 2-Per-formance Evaluation (COMLEX-USA Level 2-PE), by testing cycle. The 2008-2009 testing cycle consisted of 13 months(June 1, 2008-June 30, 2009). The other testing cycles consisted of 12 months.


116 • JAOA • Vol 110 • No 3 • March 2010

of graduation, most first-time failing students should havesufficient opportunity to reschedule and retest, receiving asecond score report before graduation. Data from the 2008-2009testing cycle, as shown in Figure 2, suggest that the majority ofexaminees schedule the examination early in the academicyear to take advantage of the opportunity to retest if needed.

Demographic characteristics of first-time examinees acrossall testing cycles are presented in Table 1. The percentage ofexaminees who were women was slightly higher than thepercentage of examinees who were men in the third, fourth,and fifth testing cycles—the opposite of the proportions inthe first two cycles. The primary racial group in all testingcycles was white, followed by Asian and “black orAfrican American.” English was the primary spoken languagefor all testing cycles.

These examinee demographic data are consistent withdata reported by the American Association of Colleges ofOsteopathic Medicine for osteopathic medical student matric-ulates and graduates.23

Competency MeasuresExaminees must pass two domains—the Humanistic Domainand the Biomedical/Biomechanical Domain—to passCOMLEX-USA Level 2-PE. Failure in either domain results infailure of the entire examination. Separately scored compo-nents are used to measure performance in each of the domains(Figure 3).

The Humanistic Domain, as measured by NBOME’sGlobal Patient Assessment (GPA) Tool, consists of the fol-lowing six components: listening skills, respectfulness,empathy, professionalism, ability to elicit information, andability to provide information. Each of these components isevaluated by the SP using the GPA Tool, a holistic Likertscale–based scoring instrument.24

For the 2008-2009 testing cycle, the generalizability coef-ficient for the Humanistic Domain, as measured by theGPA Tool, was 0.85—a coefficient that was similar to thosemeasured in previous testing cycles. This reliability measureis also similar to measures found for other high-stakes clinicalskills examinations.25

The Biomedical/Biomechanical Domain is comprised ofthree weighted component scores that reflect medical and sci-entific knowledge synthesized with clinical skill performanceacross all SP stations. The components are as follows: (1) datagathering, which reflects the examinee’s ability to obtain amedical history and perform a physical examination; (2) osteo-pathic manipulative treatment (OMT), which reflects the exam-inee’s ability to integrate osteopathic principles into clinicalpractice and to use OMT; and (3) SOAP Notes, which reflectthe examinee’s written communication skills and ability tosynthesize information, develop a differential diagnosis, andformulate a diagnostic and treatment plan.

Data gathering, a clinical skill competency measure, isscored on a percent metric based on the number of checklistitems correctly obtained during the medical history and thenumber of correctly performed physical examination maneu-vers recorded by the SP. Although osteopathic principles areassessed throughout the examination, the examinee’s perfor-mance of OMT is specifically assessed in 25% to 40% of theencounters. Performance of OMT is scored by osteopathicphysician examiners who are trained to evaluate OMT skillsusing a holistic Likert scale–based scoring instrument. TheSOAP Notes are also scored by osteopathic physician exam-iners using another Likert scale–based instrument.

For the 2008-2009 testing cycle, the generalizability coef-ficient for the Biomedical/Biomechanical Domain score was0.76. As measures of reliability, other clinical skills perfor-mance assessments show similar generalizability coefficients


MEDICAL EDUCATION

900800700600500400300200100

0

Exam

inees, No. 2004-2005

Jan

Feb

March

April

May

June Ju

ly

Aug

Sept Oc

tNov

Dec

2005-2006

2006-2007

2007-2008

2008-2009

Month of Testing Cycle

Figure 2.Number of first-time examinees taking the Comprehensive Osteopathic Medical Licensing Examination Level-USA 2-PerformanceEvaluation (COMLEX-USA Level 2-PE), by testing cycle and month. The 2008-2009 testing cycle consisted of 13 months (June 1, 2008-June 30, 2009), with 393 first-time examinees in June 2008 and 371 first-time examinees in June 2009. The other testing cycles con-sisted of 12 months.


JAOA • Vol 110 • No 3 • March 2010 • 117

with better performance in another station. After all scores are computed and equated to take into

account the relative leniency or stringency of SPs and physi-cian examiners, the pass/fail decision for COMLEX-USALevel 2-PE is based on standards applied to the cut score ofeach domain. Pass/fail standards are based on minimal com-petency for entry into graduate medical education.

for data gathering and postencounter notes.25The component scores for patient encounters in the

COMLEX-USA Level 2-PE are averaged across all scored sta-tions to compute the examinee’s average performance duringthe testing day. The average performance rating provides acompensatory score within each domain of the examination,allowing poor performance in one station to be compensated


MEDICAL EDUCATION

Table 1Characteristics of First-Time Examinees in COMLEX-USA Level 2-PE Testing Cycles, No. (%)*

Testing Cycle

2004-2005 2005-2006 2006-2007 2007-2008 2008-2009Characteristic N=2720 N=2856 N=3099 N=3476 N=4353

� Sex▫ Men 1422 (52.3) 1485 (52.0) 1529 (49.3) 1715 (49.3) 1888 (43.4)▫ Women 1256 (46.2) 1337 (46.8) 1557 (50.2) 1737 (50.0) 1984 (45.6)� Race▫ American Indian/Alaska Native 26 (1.0) 20 (0.7) 22 (0.7) 17 (0.5) 11 (0.3)▫ Asian 404 (14.9) 390 (13.7) 418 (13.5) 554 (15.9) 588 (13.5)▫ Black or African American 16 (0.6) 14 (0.5) 10 (0.3) 130 (3.7) 139 (3.2)▫ Native Hawaiian/Pacific Islander 67 (2.5) 101 (3.5) 124 (4.0) 19 (0.5) 26 (0.6)▫ White 2073 (76.2) 2201 (77.1) 2384 (76.9) 2561 (73.7) 2664 (61.2)� Ethnicity▫ Hispanic/Latino 83 (3.1) 96 (3.4) 108 (3.5) 125 (3.6) 157 (3.6)▫ Not Hispanic/Latino 2448 (90.0) 2391 (83.7) 2614 (84.3) 2326 (66.9) 2294 (52.7)� English as Primary Language▫ Yes 2451 (90.1) 2626 (91.9) 2853 (92.1) 3175 (91.3) 3548 (81.5)▫ No 211 (7.8) 199 (7.0) 232 (7.5) 271 (7.8) 226 (5.2)

* Percentages are based only on examinees who responded and, therefore, may not total 100.

Abbreviation: COMLEX-USA Level 2-PE, Comprehensive Osteopathic Medical Licensing Examination-USA Level 2-Performance Evaluation.

Figure 3. The Comprehensive Osteopathic Medical Licensing Exami-nation-USA Level 2-Performance Evaluation (COMLEX-USA Level 2-PE)consists of two domains—the Humanistic Domain and the Biomed-ical/Biomechanical Domain. The Humanistic Domain includes six sep-arately scored components (listening skills, respectfulness, empathy,professionalism, ability to elicit information, ability to provide infor-mation), which are measured by the National Board of OsteopathicMedical Examiners’ Global Patient Assessment Tool. The Biomed-

ical/Biomechanical Domain has three separately scored, weightedcomponents: data gathering (reflecting the examinee’s ability toobtain a medical history and perform a physical examination); osteo-pathic manipulative treatment (OMT, reflecting the examinee’s abilityto integrate osteopathic principles into clinical practice and to useOMT); and SOAP Notes (reflecting the examinee’s written communi-cation skills and ability to synthesize information, develop a differentialdiagnosis, and formulate a diagnostic and treatment plan).


118 • JAOA • Vol 110 • No 3 • March 2010

Evaluation of SOAP Note Accuracy Beginning with the 2007-2008 testing cycle, the NBOME offi-cially implemented screening, reviewing, and reporting pro-cesses for SOAP Note fabrication. The 2009-2010 OrientationGuide for COMLEX-USA Level 2-PE26 contains the followingdescription of SOAP Note fabrication:

Falsification of the medical record (written SOAP Notes) inCOMLEX-USA Level 2-PE by documenting medical historythat was not elicited, or physical examination maneuvers ortechniques that were not performed, is considered irregularconduct and will be thoroughly investigated and dealt withas specified in NBOME’s Bulletin of Information.

Examinees are informed that misrepresenting findingsin the SOAP Note is a form of irregular conduct that can resultin a failing score with an annotated score report and tran-script. Additional information about the SOAP Note fabrica-tion process can be obtained not only in NBOME’s OrientationGuide,26 but also in the instructional program video availableat NBOME’s Web site and in various local and national pre-sentations to student groups. This information is also pre-sented in each 50-minute orientation session that occurs imme-diately before the COMLEX-USA Level 2-PE is administered.

Sandella et al27 describes NBOME’s two flagging andscreening procedures for SOAP Note fabrication. In one pro-cedure, physician examiners are instructed to report any incon-sistencies in note documentation. In the other procedure, psy-chometric analysis is used to detect inconsistencies betweendata gathering and note documentation scores.

When SOAP Notes have been flagged for potential fab-rication, a comprehensive review is undertaken by theNBOME. When the NBOME determines that a substantialnumber of SOAP Notes contain information that was notelicited during the patient en counter, the examinee receives afailing score report with the annotation of “irregular conduct.”

Results of SOAP Note fabrication reviews for the 2007-2008 and 2008-2009 testingcycles, listed by flagging proce-dure, are shown in Table 2. Thedata reveal that a total of 17examinees have failed theCOMLEX-USA Level 2-PEbased on SOAP Note fabrica-tion since NBOME’s adoptionof the policy in 2007.

Passing Standards andExaminee PerformanceBased on The Standards for Edu-cational and Psychological Test -ing,28 published by the Amer-ican Educational ResearchAssociation, American Psycho-

logical Association, and the National Council on Measure-ment in Education, pass/fail decisions for the two domains ofthe COMLEX Level 2-PE are reevaluated approximately every3 years.29 The NBOME uses a process of triangulation fordetermining cut points for Level 2-PE, a process that is commonin medical testing organizations and highly recommended inhigh-stakes testing in general.4,30 This process includes stan-dard-setting surveys, standard-setting panel meetings using anexaminee-centered method, and a comprehensive finalreview.29,31,32

In examinee-centered methods, samples of actual per-formance covering the ability continuum of examinees arepresented to standard-setting panelists, who provide expertjudgments about the demonstration of minimal clinical skillsto pass the examination. The standard-setting panels areselected to assure broad representation of the osteopathic med-ical profession in regard to geographic considerations, age,sex, race, and ethnicity. Panelists are selected to include rep-resentation from clinical practice, state medical licensing boards,graduate medical education programs, and COMs.

The numbers and percentages of first-time examineeswho passed or failed the COMLEX-USA Level 2-PE in eachtesting cycle are shown in Table 3. Of the 16,504 first-timeexaminees in all testing cycles, 15,604 (94.5%) passed the exam-ination. Of the 2720 first-time examinees in the 2004-2005testing cycle, 2613 (96.1%) passed. The pass rates for first-timeexaminees in the successive four testing cycles were some-what less than in the 2004-2005 cycle—ranging from 93.4% (in2006-2007) to 95.3% (in 2005-2006). Of the 4353 first-time exam-inees in the most recent testing cycle (2008-2009), 4124 (94.7%)passed COMLEX-USA Level 2-PE.

Domain fail rates shown in Table 3 are based on the totalnumber of first-time examinees tested within each testingcycle. For example, 229 (5.3%) of the 4353 examinees in the2008-2009 testing cycle failed the examination. Because passingeach domain is required to pass the examination, these exam-inees could have failed the Biomedical/Biomechanical Domain,


MEDICAL EDUCATION

Table 2Results of SOAP Note Fabrication Reviews by Flagging Procedure for the 2007-2008

and 2008-2009 COMLEX-USA Level 2-PE Testing Cycles

Psychometric Analysis Physician Examiner Reports

Testing Cycle N Flagged, No. Failures, No. Flagged, No. Failures, No.

2007-2008 3751 64 3 27 52008-2009 4698 96 2 181* 7Total Failures, No. 5 12

* New flagging instructions and procedures were given to physician examiners at the beginning of this testing cycle.

Abbreviations: COMLEX-USA Level 2-PE, Comprehensive Osteopathic Medical Licensing Examination-USA Level 2-Performance Evaluation; SOAP, Subjective, Objective, Assessment, Plan.


JAOA • Vol 110 • No 3 • March 2010 • 119

Standardized Patient SurveyThe standardized patient survey, given to examinees before theexamination, includes questions about examinees’ previousexperience with and exposure to SPs. Each examinee mustinclude his or her name on this survey. Questions address thenumber of SP encounters that the examinees experiencedduring their osteopathic medical school training and the natureof their COMs’ comprehensive SP-based examinations.

As depicted in Table 4, 2984 (63.5%) examinees in the2008-2009 testing cycle reported that SPs were used at theirCOMs for nongraded teaching purposes—a finding consistentwith the rate (63.9%) reported in the 2007-2008 testing cycle.More than 94% of examinees in the 2004-2005, 2005-2006, and2006-2007 testing cycles noted that SPs were used at theirCOMs for teaching purposes. Thus, a noticeable decrease hasoccurred in the percentages of examinees responding that SPsare used for teaching purposes at COMs. This decrease mayreflect a curricular shift in which COMs began using SPs more

the Humanistic Domain, or both. Results show that 81 (1.9%)of all examinees in 2008-2009 failed the Biomedical/Biome-chanical Domain, 115 (2.6%) failed the Humanistic Domain,and 33 (0.8%) failed both domains.

New standards were introduced in the 2007-2008 testingcycle, yielding a technical adjustment to the cut points on eachdomain. Applying new standards resulted in little changebetween the overall fail rate for 2006-2007 and that for 2007-2008(percentage difference=0.1%). However, adjustment to the cutpoints under the new standards resulted in a reversal in failrates for each domain in 2007-2008 (Table 3). Fail rates for theBiomedical/Biomechanical Domain in 2006-2007 and earlierwere consistently higher than those for the HumanisticDomain. Fail rates were estimated for each domain under thenew standards, which predicted a lower fail rate for theBiomedical/Biomechanical Domain and a higher fail rate forthe Humanistic Domain—bringing the fail rates of the twodomains closer together.

As shown in Figure 4, the overall fail ratefor first-time examinees across all testing cycleswas 5.4%. For those taking the examination asecond time, the fail rate was 12.9%. For thosetaking the examination for the third time or more,the fail rate was 21.2%.

Onsite SurveysOn the day of the examination, two onsite sur-veys are administered to examinees—one beforeand one after the examination. The surveys eval-uate examinees’ previous educational experi-ence with SPs and their perceptions of theCOMLEX-USA Level 2-PE and testing center.The surveys are given to both first-time andrepeat examinees.


MEDICAL EDUCATION

Table 3Pass and Fail Rates of First-Time Tested on COMLEX-USA Level 2-PE by Testing Cycles, No. (%)

Testing Cycle

2004-2005 2005-2006 2006-2007 2007-2008 2008-2009 Examination Outcome N=2720 N=2856 N=3099 N=3476 N=4353

� Pass, Overall 2613 (96.1) 2722 (95.3) 2896 (93.4) 3249 (93.5) 4124 (94.7)� Fail, Overall 107 (3.9) 134 (4.7) 203 (6.6) 227 (6.5) 229 (5.3)� Fail, by Domain*

▫ Biomedical/biomechanical 83 (3.1) 112 (3.9) 126 (4.1) 83 (2.4) 81 (1.9)▫ Humanistic 20 (0.7) 15 (0.5) 55 (1.8) 121 (3.5) 115 (2.6)▫ Both 4 (0.1) 7 (0.2) 22 (0.7) 23 (0.7) 33 (0.8)

*Domain sample size and percentages reflect overall fail percentages within each testing cycle.


Exam

inees Who

Failed, %

First take Second take Three or more takes

5.4

12.9

21.2

Figure 4. Percentages of examinees who failed the Comprehensive OsteopathicMedical Licensing Examination Level 2-Performance Evaluation (COMLEX-USA Level 2-PE), by number of takes for all five testing cycles (2004-2005 to 2008-2009).

25

20

15

10

5

0


120 • JAOA • Vol 110 • No 3 • March 2010

for assessment purposes rather than teaching. As shown in Table 5, the median number of school-based

SP encounters experienced by examinees during the first yearof osteopathic medical school has increased from 3 in the 2004-2005 testing cycle to 5 in the 2008-2009 testing cycle. A similartrend is seen for the second year of osteopathic medical school,with the median number of SP encounters increasing from 5in 2004-2005 to 8 in 2008-2009.

A total of 3420 (72.8%) examinees in the 2008-2009 testingcycle agreed that their COMs administered comprehensiveSP examinations that were “similar to [COMLEX-USA] Level 2-PE” (Table 4). This percentage represented a decline from the

81.5% who agreed to that statement in 2006-2007, the firsttesting cycle in which the “similar to Level 2-PE” wordingwas used.

In the 2008-2009 testing cycle, 3430 (73%) examineesresponded that they had completed the comprehensive SP-based examination at their respective COMs, and 2705 (57.6%)responded that they were required to pass the examinationbefore graduation. As shown in Table 4, each of these per-centages has remained relatively consistent over all testingcycles.

Respondents in the 2008-2009 testing cycle who notedthat their COMs administered comprehensive SP-based exam-


MEDICAL EDUCATION

Table 4Examinee Responses to Standardized Patient Survey in All COMLEX-USA Level 2-PE Testing Cycles, No. (%)*

Testing Cycle

2004-2005 2005-2006 2006-2007 2007-2008 2008-2009 Survey Item N=2742 N=3005 N=3261 N=3751 N=4698

� 1. Are SPs used at your schoolfor teaching purposes?†

▫ Yes 2600 (94.8) 2891 (96.2) 3144 (96.4) ... ...▫ No 129 (4.7) 103 (3.4) 86 (2.6) ... ...� 2. Are SPs used at your schoolfor non-graded teaching purposes?†

▫ Yes ... ... ... 2396 (63.9) 2984 (63.5)▫ No ... ... ... 1138 (30.3) 1151 (24.5)� 3. Is performance on SP encountersincorporated into your evaluation/grade?

▫ Yes 2092 (76.3) 2377 (79.1) 2688 (82.4) 3136 (83.6) 3626 (77.2)▫ No 624 (22.8) 611 (20.3) 525 (16.1) 404 (10.8) 515 (11.0)� 4. Does your school administer its ownSP examination?‡

▫ Yes 2343 (85.4) 2622 (87.3) ... ... ...▫ No 373 (13.6) 340 (11.3) ... ... ...� 5. Does your school administer its owncomprehensive SP examination(ie, an assessment similar to Level 2-PE)?‡

▫ Yes ... ... 2658 (81.5) 2908 (77.5) 3420 (72.8)▫ No ... ... 540 (16.6) 494 (13.2) 514 (10.9)– 5A. If yes, are you required to passin order to graduate?

▫ Yes 1642 (59.9) 1761 (58.6) 1819 (55.8) 2278 (60.7) 2705 (57.6)▫ No 766 (27.9) 881 (29.3) 652 (20.0) 659 (17.6) 772 (16.4)– 5B. If yes, have you completedthis [examination]?

▫ Yes ... ... 2516 (77.2) 2890 (77.0) 3430 (73.0)▫ No ... ... 291 (8.9) 222 (5.9) 223 (4.7)

* Percentages are based only on examinees who responded and, therefore, may not total 100.† Question 1 was rephrased as question 2 as of testing cycle 2007-2008.‡ Question 4 was rephrased as question 5 as of testing cycle 2006-2007.

Abbreviations: COMLEX-USA Level 2-PE, Comprehensive Osteopathic Medical Licensing Examination-USA Level 2-Performance Evaluation; SP, standardizedpatient; ... indicates no data available.


JAOA • Vol 110 • No 3 • March 2010 • 121

strongly agreed that the “casesrepresented appropriate chal-lenges for a 4th year student,”3734 (79.4%) examinees agreedor strongly agreed that the“[standardized] patients por-trayed the case in a believablemanner,” and 3557 (75.7%)examinees agreed or stronglyagreed that the “[standardized]patients’ responses to the phys-ical [examination] were real-istic.”

Survey items that elicitedsomewhat lower levels ofagreement among respondentsincluded those related to theadequacy of the SOAP Noteabbreviations list and the suf-ficiency of the time allotted tocomplete SOAP Notes and SPencounters. As illustrated inFigure 5, responses to theseselected survey items—and theitem on whether the cases rep-resented appropriate chal-lenges—remained fairly con-sistent across all testing cycles.

Responses collected duringthe 2008-2009 testing cycle sug-

gest that the National Center for Clinical Skills Testing and theadministration of the COMLEX-USA Level 2-PE were favor-ably received by examinees. In all testing cycles, examinees gen-erally appeared to be satisfied with their testing experiences.

� Written Feedback—Examinees are encouraged to providewritten feedback regarding their experiences with theCOMLEX-USA Level 2-PE. Comments in the surveys ana-lyzed in the present study ranged from congratulatory to per-secutory. The following are samples of selected examinee com-ments from surveys in the 2008-2009 testing cycle:

� “[Patients] were very realistic and cases were challenging butnot overly difficult or confusing. Proctors were very friendlyand helpful. I enjoyed the experience. Test center was incred-ibly easy to find, lots of places to stay [within] walking dis-tance. Lunch was good too. Thank you!

� “Excellent SPs—very believable. I felt like it was a normalrotation.”

� “I feel more time should be allotted for the SP and SOAPNote portions since we are to perform OMT on somepatients. I feel this time limit makes me feel rushed duringnote writing and the physical examination. The proctorswere very helpful and pleasant to be around.”

inations reported that they experienced a median number of5 SP encounters at the COMs. As shown in Table 6, this numberand the numbers from the 2006-2007 and 2007-2008 testingcycles represented decreases in SP encounters at the COMs,compared with the 2004-2005 and 2005-2006 testing cycles.

Postexamination SurveyThe postexamination survey is an anonymous 22-item surveygiven immediately after the examination. It is meant to mon-itor examinee satisfaction with the NBOME’s National Centerfor Clinical Skills Testing and administration of the COMLEX-USA Level 2-PE. For each of the postexamination survey items,examinees are asked to respond using a four-point Likertscale: 1 (strongly disagree), 2 (disagree), 3 (agree), and4 (strongly agree). Written comments are also solicited fromexaminees. The postexamination survey data and writtencomments are used to monitor examinee satisfaction and tomake improvements to the COMLEX-USA Level 2-PE andtesting center.

Overall, the responses of the 4698 examinees from the 2008-2009 testing cycle were favorable regarding the examinationand testing center, as illustrated by a majority of “stronglyagree” or “agree” re-sponses to survey items (Table 7).

As shown in Table 7, 3846 (81.8%) examinees agreed or


MEDICAL EDUCATION

Table 5School-Based Standardized Patient Encounters Experienced by COMLEX-USA Level 2-PE

Examinees During First 2 Years of Osteopathic Medical School, Median No.

Testing Cycle

Osteopathic Medical 2004-2005 2005-2006 2006-2007 2007-2008 2008-2009 School Year N=2742 N=3005 N=3261 N=3751 N=4698

First Year 3.0 4.0 4.0 5.0 5.0Second Year 5.0 6.0 6.0 8.0 8.0


Table 6Standardized Patient (SP) Encounters Experienced by COMLEX-USA Level 2-PE Examinees in

Comprehensive SP-Based Examinations Administered by COMs, Median No.

Testing Cycle

2004-2005 2005-2006 2006-2007 2007-2008 2008-2009 N=2742 N=3005 N=3261 N=3751 N=4698

Individual SP Encounters 8.0 8.0 5.0 6.0 5.0

Abbreviations: COM, college of osteopathic medicine; COMLEX-USA Level 2-PE, Comprehensive Osteopathic MedicalLicensing Examination-USA Level 2-Performance Evaluation.


122 • JAOA • Vol 110 • No 3 • March 2010 Langenau et al • Medical Education

MEDICAL EDUCATION

Table 7Examinee Responses to Post examination Survey in 2008-2009 COMLEX-USA Level 2-PE Testing Cycle, No. (%) (N=4698) *

Survey Item Strongly Disagree Disagree Agree Strongly Agree

1. The cases represented appropriate challenges for a 4th year student. 23 (0.5) 78 (1.7) 1988 (42.3) 1858 (39.5)

2. The equipment in the exam stations was easy to locate. 28 (0.6) 29 (0.6) 1173 (25.0) 2738 (58.3)

3. The list of abbreviations for the SOAP Note was sufficient. 370 (7.9) 893 (19.0) 1471 (31.3) 1230 (26.2)

4. The designated area provided to write the SOAP Note was comfortable. 53 (1.1) 182 (3.9) 1740 (37.0) 1985 (42.2)

5. The designated area provided to writethe SOAP Note was quiet. 30 (0.6) 68 (1.4) 1417 (30.2) 2462 (46.9)

6. The temperature of the test center was comfortable. 39 (0.8) 222 (4.7) 1513 (32.2) 2202 (46.9)

7. The external noise level in the exam rooms was acceptable. 26 (0.6) 66 (1.4) 1455 (31.0) 2424 (51.6)

8. The instructions provided in the orientation were clear. 24 (0.5) 35 (0.7) 1171 (24.9) 2746 (58.4)

9. The orientation provided ample opportunity to ask questions. 31 (0.7) 64 (1.4) 1255 (26.7) 2628 (55.9)

10. The proctors were helpful. 22 (0.5) 27 (0.6) 997 (21.2) 2927 (62.3)

11. The receptionist greeted me in a friendly manner. 73 (1.6) 154 (3.3) 1177 (25.0) 2571 (54.7)

12. The refreshments provided were satisfactory. 61 (1.3) 180 (3.8) 1406 (29.9) 2322 (49.4)

13. It was easy to find my way to the test center in Conshohocken, Pennsylvania. 95 (2.0) 290 (6.2) 1572 (33.5) 2024 (43.1)

14. The patients portrayed the case in a believable manner. 33 (0.7) 196 (4.2) 1778 (37.8) 1956 (41.6)

15. Everything the patients reported was consistent with the case. 34 (0.7) 344 (7.3) 2003 (42.6) 1579 (33.6)

16. The patients’ affect appropriately matched the chief complaint. 42 (0.9) 259 (5.5) 2049 (43.6) 1613 (34.3)

17. The patients’ responses to the physical exam were realistic. 39 (0.8) 369 (7.9) 2105 (44.8) 1452 (30.9)

18. The patients answered my questions completely. 48 (1.0) 282 (6.0) 2012 (42.8) 1627 (34.6)

19. All equipment provided in the exam roomfunctioned properly. 25 (0.5) 38 (0.8) 1313 (27.9) 2593 (55.2)

20. The doorway information sheet made the task (clinical challenge) clear. 27 (0.6) 83 (1.8) 1639 (34.9) 3962 (47.1)

21. The time allowed to complete the SOAP Note was sufficient. 245 (5.2) 633 (13.5) 1655 (35.2) 1420 (30.2)

22. The time allowed to complete the encounter with the SP was sufficient. 247 (5.3) 667 (14.2) 1747 (37.2) 1298 (27.6)

* Percentages are based only on examinees who responded and, therefore, may not total 100%.

Abbreviations: COMLEX-USA Level 2-PE, Comprehensive Osteopathic Medical Licensing Examination Level 2-Performance Evaluation; SOAP, Subjective, Objective,Assessment, Plan; SP, standardized patient.


JAOA • Vol 110 • No 3 • March 2010 • 123

becoming closer in magnitude—with the proportion of exam-inees failing the Humanistic Domain increasing slightly whilethe overall fail rate remained stable. This outcome was pre-dicted under the NBOME policy with application of the newstandards.

Beginning in the 2007-2008 testing cycle, the NBOME for-mally began evaluating SOAP Notes about patient encountersfor accuracy and integrity.27We can only speculate why exam-inees include clinical findings on SOAP Notes that are notelicited or obtained during patient encounters. Examinees’intent or motivation cannot be determined with the currentevaluation processes. Nevertheless, a note containing inaccu-rate information that was not elicited during a patient encountermay lead to dangerous outcomes for the patient. Thus, con-sistent with its mission to protect the public, the NBOME hastaken a strong stance against SOAP Note fabrication and inac-curate documentation.

Examinees overwhelmingly reported that cases presentedin the COMLEX-USA Level 2-PE were realistic and repre-sented an appropriate level of challenge for fourth-year osteo-pathic medical students (Table 7). This finding was consistentwith previous findings.2 However, in contrast to previousreports and pilot studies indicating that 14 minutes is sufficientfor a patient-physician encounter,2,33 19.5% of examinees inthe present study disagreed or strongly disagreed that sufficienttime was allotted for this aspect of the examination. Despitethese survey results, examinees spend an average of 12.1 min-utes with SPs during each clinical encounter for COMLEX-USA

� “I think that the use of OMT should not be a requirement butinstead an additional way for students to gain points. I didnot have any 3rd or 4th year rotations in which it was actu-ally used so I felt ill prepared.”

� “Thank you, I feel a great sense of accomplishment andrealize what I can do better. This has given me more confi-dence in seeing and treating patients. Please thank the SPsfor their time and participation.”

� “The proctors were so helpful and friendly that it felt almostlike being in a real clinic. In other words, no high stress ofbeing watched and monitored. Makes a big difference whenstress level is not too overwhelming. Thank you!”

Such written comments, which are reviewed regularlyby the NBOME, have led to improvements in the adminis-tration of the COMLEX-USA Level 2-PE, including adjustingdirections posted on the Web site, adding a list of hotel accom-modations in the testing area, and streamlining the registrationprocess. Examinees’ comments have also triggered in-depthreviews and analyses of various operational issues, such as theuse of electronic medical records for data entry and the use ofalternative testing sites.

CommentPassing standards and passing rates have remained relativelystable over all five testing cycles for the COMLEX-USA Level 2-PE. The introduction of adjusted standards for the 2007-2008testing cycle resulted in the fail rates for each of the domains


MEDICAL EDUCATION

4

3.5

3

2.5

2

1.5

1

0.5

0

Exam

inees’ M

ean Respon

ses

Survey Item

The casesrepresentedappropriate

challenges for a4th year student

The list of abbreviations for

the SOAP Note wassufficient

The time allowedto complete theSOAP Note was

sufficient

The time allowedto complete the

encounter with theSP was sufficient

2004-2005

2005-2006

2006-2007

2007-2008

2008-2009

Figure 5. Examinee responses to selected postexamination survey items, by Comprehensive Osteopathic MedicalLicensing Examination Level 2-Performance Evaluation (COMLEX-USA Level 2-PE) testing cycle. Examinee responses:1 (strongly disagree), 2 (disagree), 3 (agree), 4 (strongly agree). Abbreviation: SP, standardized patient.


124 • JAOA • Vol 110 • No 3 • March 2010

Level 2-PE—well below the 14 minutes allotted for theencounter.33

A minority of examinees in the present study also reportedinsufficient time for completing SOAP Notes about the encoun-ters (Table 7). In a COMLEX-USA Level 2-PE prototype pilotstudy,2 7 minutes was shown to be sufficient for SOAP notecompletion—well below the 9 minutes allotted for examineesto complete SOAP Notes.

A minority of examinees in the present study reported thatthe list of common abbreviations to use with SOAP Noteswas inadequate (Table 7). Rather than publishing a compre-hensive list of acceptable medical abbreviations, which wouldbe impractical, The Joint Commission—in compliance withNational Patient Safety Goal Requirement 2B—has publisheda list of abbreviations that are not to be used for medical doc-umentation.34 Consistent with this recommendation, theNBOME advises examinees to avoid abbreviations on SOAPNotes that might lead to medical error, and the list providedto examinees is meant as a guide for commonly used abbre-viations.

Between the 2004-2005 and 2008-2009 testing cycles, exam-inees reported increases in the median number of SP encoun-ters in both the first and second years of osteopathic medicalschool (Table 5). This increase implies that osteopathic medicalstudents are being exposed to a greater number of SP encoun-ters. Yet, because the reported number of SPs used for teachingpurposes has decreased (Table 4), this pattern may represent ashift in using SPs for summative, rather than formative, assess-ments at individual COMs.

Although SPs were predominantly used for teaching clin-ical skills several years ago,7 results of the present study are con-sistent with recent reports that COMs are now using SPs morefor assessment rather than teaching.13 Addressing this shiftin SP use more thoroughly requires a more detailed investi-gation of current COM-based SP-based programs.

Examinees reported a reduction in the median number ofencounters used in comprehensive SP examinations at COMs(8 in 2004-2005 vs 5 in 2008-2009), as shown in Table 6. As a mea-sure of reliability, generalizability coefficients decrease sub-stantially when examinations include fewer than 10 encoun-ters.25,35 The selected number of encounters per student maydepend on SP availability, class size, and examination admin-istration costs, as well as the level of reliability that individualCOMs are willing to accept for such an examination.

The present study recorded a decrease over time in thenumber of examinees who reported that their COMs admin-istered comprehensive SP examinations (85.4% in 2004-2005 vs72.8% in 2008-2009), as shown in Table 4. Although this trendmight be explained by a genuine reduction in the use of com-prehensive SP examinations at COMs, it is more likelyexplained by COMs consolidating resources and outsourcingsuch examinations to their affiliated branch campuses. Alter-natively, the trend could be the result of student misinterpre-tation of the survey item or misunderstanding of the school-

sponsored SP experiences. A formal analysis of this trendwould require a detailed survey to be administered directly tothe COMs.

Hauer et al36 demonstrated that allopathic medical schoolshave designed new school-based examinations or modifiedexisting examinations to simulate the United States MedicalLicensing Examination Step 2 Clinical Skills. Gimpel et al13reported that the use of standardized patient programs andmechanical simulators within osteopathic medical schoolsincreased between 2001 and 2005. Data from the present studysupport the finding that SPs are being used by COMs for bothteaching and evaluative purposes. A more detailed school-based analysis would provide additional insight regardinghow clinical skills teaching curricula are designed and how SPsare used for formative and summative assessments.

ConclusionFor the first five testing cycles, examinee performance hasbeen relatively consistent, and examinees have reported gen-eral satisfaction with administration and assessment method-ologies of the COMLEX-USA Level 2-PE. As the osteopathicmedical profession continues to grow, the NBOME looks for-ward to administering examinations for many years to come.These examinations will allow the NBOME to meet its missionof protecting the public by providing the means to assess com-petencies for osteopathic medicine and related healthcare pro-fessions.

AcknowledgmentWe thank Taunya Cossetti, executive assistant for the NBOME, forher assistance in reviewing and editing the information provided inthe present report.

References1.Harden RM, Stevenson M, Downie WW, Wilson GM. Assessment of clinicalcompetence using objective structured examination. Br Med J. 1975;1(5955):447-451. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1672423/?tool=pubmed. Accessed December 3, 2009.

2. Gimpel JR, Boulet JR, Errichetti AM. Evaluating the clinical skills of osteo-pathic medical students. J Am Osteopath Assoc. 2003;103(6):267-279.http://www.jaoa.org/cgi/reprint/103/6/267. Accessed December 3, 2009.

3. Swanson DB, Norman GR, Linn RL. Performance-based assessment: lessonsfrom the health professions. Educ Res. 1995;24(5):5-11.

4. Boulet JR, Smee SM, Dillon GF, Gimpel JR. The use of standardized patientassessments for certification and licensure decisions. Simul Healthc. 2009;4(1):35-42.

5.Adamo G. Simulated and standardized patients in OSCEs: achievements andchallenges 1992-2003. Med Teach. 2003;25(3):262-270.

6. Boulet JR, De Champlain AF, McKinley DW. Setting defensible perfor-mance standards on OSCEs and standardized patient examinations. MedTeach. 2003;25(3):245-249.

7. Errichetti AM, Gimpel JR, Boulet JR. State of the art in standardized patientprograms: a survey of osteopathic medical schools. J Am Osteopath Assoc.2002;102(11):627-631. http://www.jaoa.org/cgi/reprint/102/11/627. AccessedDecember 3, 2009.

8. National Board of Osteopathic Medical Examiners. Bulletin of Informa-


MEDICAL EDUCATION


JAOA • Vol 110 • No 3 • March 2010 • 125

25. Clauser BE, Harik P, Margolis MJ. A multivariate generalizability analysisof data from a performance assessment of physicians’ clinical skills. J Educ Meas.2006;43(3):173-191.

26.National Board of Osteopathic Medical Examiners. 2009-2010 OrientationGuide for COMLEX-USA Level 2-PE. Chicago, IL: National Board of Osteo-pathic Medical Examiners; 2009. http://www.nbome.org/docs/PEOrientationGuide.pdf. Accessed December 21, 2009.

27. Sandella JM, Roberts WL, Gallagher LA, Gimpel JR, Langenau EE, BouletJR. Patient note fabrication and consequences of unprofessional behavior ina high-stakes clinical skills licensing examination. Acad Med. 2009;84(10):S70-S73.

28.American Educational Research Association, American Psychological Asso-ciation, National Council on Measurement in Education. The Standards for Edu-cational and Psychological Testing. Washington, DC: American EducationalResearch Association, American Psychological Association, National Councilon Measurement in Education; 1999.

29. Cohen AS, Kane MT, Crooks TJ. A generalized examinee-centered methodfor setting standards on achievement tests. Appl Meas Educ. 1999;12(4):343-366.

30. Norcini JJ, Shea JA. The credibility and comparability of standards. ApplMeas Educ. 1997;10(1):39-59.

31. Clauser BE, Clyman SG. A contrasting-groups approach to standard settingfor performance assessments of clinical skills. Acad Med. 1994;69(10 suppl):S42-S44.

32. Hambleton RK, Jaeger RM, Plake BS, Mills CN. Setting performance stan-dards on complex educational assessments. Appl Psychol Meas. 2000;24(4):355-366.

33. Sandella JM, Roberts WL, Wilson CD. The relationship between timespent in the encounter and student performance in a high-stakes standard-ized patient assessment. Poster presented at: Meeting of the Society for Sim-ulation in Healthcare; January 2009; Lake Buena Vista, FL.

34. The official “do not use” list of abbreviations; April 1, 2009. The Joint Com-mission Web site. http://www.jointcommission.org/PatientSafety/DoNotUseList/. Accessed December 21, 2009.

35.Margolis MJ, Clauser BE, Swanson DB, Boulet JR. Analysis of the rela-tionship between score components on a standardized patient clinical skillsexamination. Acad Med. 2003;78(10 suppl):S68-S71.

36.Hauer KE, Teherani A, Kerr KM, O’Sullivan PS, Irby DM. Impact of the UnitedStates Medical Licensing Examination Step 2 Clinical Skills exam on medicalschool clinical skills assessment. Acad Med. 2006;81(10 suppl):S13-S16.

tion 2009-2010. Chicago, IL: National Board of Osteopathic Medical Examiners;2009. http://www.nbome.org/docs/comlexBOI.pdf. Accessed July 15, 2009.

9. AOA Commission on Osteopathic College Accreditation. Accreditation ofColleges of Osteopathic Medicine: COM Accreditation Standards and Proce-dures. Chicago, IL: American Osteopathic Association; 2009. http://www.do-online.org/pdf/SB03-Standards%20of%20Accreditation%20July%202009.pdf.Accessed December 21, 2009.

10.Brazeau C, Boyd L, Crosson J. Changing an existing OSCE to a teaching tool:the making of a teaching OSCE. Acad Med. 2002;77(9):932.

11. Barrows HS. An overview of the uses of standardized patients for teachingand evaluating clinical skills. AAMC [review]. Acad Med. 1993;68(6):443-451.

12. Townsend AH, McLlvenny S, Miller CJ, Dunn EV. The use of an objectivestructured clinical examination (OSCE) for formative and summative assess-ment in a general practice clinical attachment and its relationship to final med-ical school examination performance. Med Educ. 2001;35(9):841-846.

13.Gimpel JR, Weidner AC, Boulet JR, Wilson CD, Errichetti AM. Standardizedpatients and mechanical simulators in teaching and assessment at colleges ofosteopathic medicine. J Am Osteopath Assoc. 2007;107(12):557-561.http://www.jaoa.org/cgi/content/full/107/12/557. Accessed December 3, 2009.

14.Altshuler L, Kachur E, Krinshpun S, Sullivan D. Genetics objective structuredclinical exams at the Maimonides Infants & Children’s Hospital of Brooklyn,New York. Acad Med. 2008;83(11):1088-1093.

15. Kligler B, Koithan M, Maizes V, Hayes M, Schneider C, Lebensohn P, et al.Competency-based evaluation tools for integrative medicine training infamily medicine residency: a pilot study. BMC Med Educ. 2007;18(7):7.http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1855050/?tool=pubmed.Accessed December 3, 2009.

16.Aeder L, Altshuler L, Kachur E, Barrett S, Hilfer A, Koepfer S, et al. The “Cul-ture OSCE”—introducing a formative assessment into a postgraduate program[published online ahead of print April 18, 2007]. Educ Health (Abingdon).2007;20(1):11.

17. Cohen R, Reznick RK, Taylor BR, Provan J, Rothman A. Reliability andvalidity of the objective structured clinical examination in assessing surgicalresidents. Am J Surg. 1990;160(3):302-305.

18. Barrows HS, Williams RG, Moy RH. A comprehensive performance-basedassessment of fourth-year students’ clinical skills. J Med Educ. 1987;62(10):805-809.

19. Prislin MD, Fitzpatrick CF, Lie D, Giglio M, Radecki S, Lewis E. Use of anobjective structured clinical examination in evaluating student performance.Fam Med. 1998;30(5):338-344.

20. Roberts WL, McKinley DW, Boulet JR. Effect of first encounter preteston pass/fail rates of a clinical skills medical licensure examination [publishedonline ahead of print September 13, 2009]. Adv Health Sci Educ Theory Pract.

21. Shannon SC, Teitelbaum HS. The status and future of osteopathic med-ical education in the United States. Acad Med. 2009;84(6):707-711.

22. American Association of Colleges of Osteopathic Medicine. Fast FactsAbout Osteopathic Medical Education: Enrollment Growth in the Nation’sCOMs. Chevy Chase, MD: American Association of Colleges of OsteopathicMedicine; 2009. http://publish.aacom.org/about/fastfacts/Documents/Fast-Facts/FF-Enrollment-NationCOMs.pdf. Accessed December 21, 2009.

23. American Association of Colleges of Osteopathic Medicine. 2006 AnnualStatistical Report on Osteopathic Medical Education. Chevy Chase, MD: Amer-ican Association of Colleges of Osteopathic Medicine; 2006.http://www.aacom.org/resources/bookstore/2006statrpt/Documents/ASROME2006.pdf. Accessed December 21, 2009.

24. Weidner AC, Gimpel JR, Boulet JR, Solomon M. Using standardizedpatients to assess the communication skills of graduating physicians for theComprehensive Osteopathic Medical Licensing Examination (COMLEX) Level 2-Performance Evaluation (Level 2-PE). Teaching Learning Med. 2010;22(1):8-15.


MEDICAL EDUCATION
