ExaminerNational Board of Medical Examiners3750 Market StreetPhiladelphia, PA 19104-3102

www.nbme.org

Spring/Summer 2012 Volume 59, Number 1

Members of the NBME gathered inPhiladelphia for the 98th AnnualMeeting. The meeting focused on anemerging business model for theNBME’s future; the history of theFederation of State Medical Boards(FSMB), celebrating its centennialthis year; the USMLE® program, cele-brating its 20th anniversary this year;and the NBME’s upcoming centennialcelebration.

The business architecture focus wasintroduced in the report of thePresident. The proposed businessmodel arising from a year of study anddialog by staff leadership and theExecutive Board places an increasedemphasis on clarity in selecting thedecisions about healthcare profession-als and institutions that the NBME’sassessment-based evidence is targetedat supporting, a more explicit planningprocess to link the desired inferenceswith the design of assessment instru-ments, and a broader view of thekinds of test and real-world data thatmight provide evidence to supportinferences. In small groups, Membersand guests brainstormed around theproposed business model. In plenarysession, the Membership endorsed thebusiness model.

A panel of keynote speakers –Humayun Chaudhry, DO (FSMBPresident/CEO), Bryant Galusha, MD

In This Issue:

1 2012 Annual Meeting of the NBMEMembership

1 Joining Forces

2 Peter Katsufrakis Promoted to Senior Vice President, Assessment Programs

4 From the Public Stakeholders Committee

5 Brian Hodges Receives the 2012 John P. Hubbard Award

7 Stemmler Fund Call for Letters of Intent

8 From the Archives: A Physician Remembers: 1927–1928 National Board Exams

(FSMB Executive Vice President at thecreation of USMLE), Janet Carson, JD(NBME’s general counsel at the cre-ation of USMLE), L. Thompson Bowles,MD, PhD (NBME’s Chair when USMLEwas conceptualized and President dur-ing its implementation), and medicalhistorian and NBME Member KennethM. Ludmerer, MD, MA – guided byHonorary Member Ruth Hoppe, MD,provided a fascinating glimpse of therich interconnections throughout theFSMB and NBME histories, culminatingin the creation of USMLE.

NBME Chair, Dr. Lewis R. First, outlinedNBME achievements in the past year,with a particular focus on theExecutive Board’s work since the 2011Annual Meeting and its attention to

In April 2011, First Lady Michelle Obama and Dr. Jill Bidenlaunched the Joining Forces initiative to recognize and honorAmerica’s veterans and military families. Facilitating and improv-ing education, employment and wellness of America’s troops,veterans, and military families are the major goals of the pro-gram. Among its key accomplishments thus far, Joining Forces

has encouraged associations of many health professions to educate their membershipson diagnosing and treating active military members and veterans and their families,

(continued on page 2)

(continued on page 3)

© 2012 NBME

2012 Annual Meeting of the NBME Membership

Joining Forces

OUT & ABOUT

We are delighted toannounce thepromotion of Peter J. Katsufrakis,MD, MBA toSenior VicePresident,

2

with particular emphasis on post-trau-matic stress disorder (PTSD) and trau-matic brain injury (TBI). According toJoining Forces, one in six veterans andservice members who fought in Iraqand/or Afghanistan is impacted by PTSDor TBI, the “invisible wounds of war andthe signature wounds of our wars inIraq and Afghanistan.”

Over the past year, Mrs. Obama and Dr.Biden have worked with over 30 majorhealth professional and health educationservices to promote improved healthcarefor America’s military. According to theJoining Forces website, over 3 millionhealth professionals, over 130 medicalschools, more than 500 nursing schools,and more than 50 physician assistantprograms have committed, each in theirown disciplines, to improve healthcarefor the military and veterans. NBME isproud to be among these organizations.In connection with the one-year anniver-sary of Joining Forces, NBME made thefollowing commitment:

National Board of Medical Examiners(NBME) will work with other JoiningForces partners to reflect the healthissues affecting service members andveterans in the broad range of exam-inations it prepares for the educationand licensure of medical students,practicing physicians, and otherhealthcare providers.

The National Board’s connection to theAmerican military dates back to its veryorigins. Among the 15 founders of theBoard were Rear Admiral William C.Braisted, Surgeon General, United StatesNavy; Major General William C. Gorgasand Colonel Louis A. LaGarde, repre-senting the United States Army; Dr.Rupert Blue, Surgeon General, and Dr.W. C. Rucker, Assistant Surgeon General,representing the United States PublicHealth Service; and Commander EdwardR. Stitt, representing the United StatesNavy. Today, all branches of the militaryhave representatives in the NBMEMembership.

Dr. Steven Haist, Vice President, TestDevelopment Services, has reached outto the military representative membersof the NBME to seek advice and assis-tance on ways to ensure adequate cov-erage of these important issues onNBME examinations. According to Dr.Haist, “There is a great deal of interestamong the 40-45 physicians from theVA, Uniformed Services University,Army, Navy and Air Force who havebeen identified as content experts insuch areas as TBI, blast injuries, andPTSD in writing questions for USMLE.They are excited to address the health-care issues of veterans, returningdeployed service men and women andfamilies of deployed service men andwomen. I am optimistic that we will beable to convene two task forces towrite a sufficient number of questionsto impact this very important initiative.”

For more information on JoiningForces, please visit their website:http://www.whitehouse.gov/joiningforces.

(Joining Forces, continued from page 1)

INSIDE THE NBME

Assessment Programs. Peter joined theNBME staff in 2007 as Associate VicePresident, Post-Graduate and Develop-mental Programs and, among other ini-tiatives, was instrumental intransitioning the Assessment of Profes-sional Behaviors tool from a Center forInnovation project to an operationalprogram. Most recently, Peter served as

Vice President, Assessment Programs,with major areas of responsibility in-cluding oversight of the implementationof changes to USMLE that resulted fromthe Comprehensive Review of USMLE.In his new role leading the AssessmentPrograms unit at NBME, Peter will over-see the USMLE program, Medical

Peter Katsufrakis Promoted to Senior Vice President, Assessment Programs

Peter J. Katsufrakis, MD, MBA (continued on page 4)

3

five emerging, key areas of strategicengagement: implementing theenhancements of USMLE identifiedthrough the Comprehensive Review ofUSMLE, re-engineering the NBME’sbusiness model and business process-es, expanding our international pro-grams and services, developing supportsystems for implementation ofMaintenance of Licensure by statelicensing authorities, and implementingthe Data Commons and eFolioConnector projects. Dr. Melnick’s writ-ten report provided information onaccomplishment of 2011 NBME goals,the 2012 goals approved by theExecutive Board, and the state of ourcollaborative engagement with sisterorganizations.

The Treasurer, Dr. Lynn M. Cleary,reported on 2011 revenues, expenses,and net gain from operational pro-grams, as well as funding of develop-

mental programs, the state of NBMEreserves, and the external auditor’sunqualified report. Members discussedthe 2011 fiscal performance and the2012 budget with the Treasurer andfinance team in small groups andendorsed the 2012 budget in the ple-nary session.

The NBME Membership participated insmall group discussions on NBME gov-ernance and infrastructure. Followingdiscussion in small groups, theMembership approved a vision state-ment for the NBME:

Improving healthcare around theworld through assessment

In addition to these topics, program-oriented small group discussionsfocused on Medical Education andStudent Services (including theAssessment of Professional Behaviors

project); USMLE (including theComprehensive Review of USMLE) andCSEC; Services for Practicing Physiciansand for Health ProfessionsOrganizations; InternationalCollaborations; and Center forInnovation and other research anddevelopment activities, including DataCommons and eFolio Connector.Members ratified various actions of theExecutive Board during plenary ses-sions, including endorsement of:

• a committee structure for theInternational Foundations ofMedicine examination programand a business plan for interna-tional programs;

• principles for collaboration withthe American Board of MedicalSpecialties and FSMB to supportimplementation of Maintenance ofLicensure;

• principles for collaboration withthe National Board of VeterinaryMedical Examiners in the creationof the North American VeterinaryLicensing Examination;

• a process for review and updatingof contracts governing the USMLEprogram; and

• the next phase of enhancement forUSMLE arising from theComprehensive Review of USMLE.

The 2012 John P. Hubbard Award waspresented to Brian D. Hodges, MD,PhD (see page 5).

(2012 Annual Meeting, continued from page 1)

Mrs. Carson and Drs. Hoppe and Bowles during panel discussion.

In this and futureeditions of theExaminer, we willdevote space tothe voice of thePublic Stakehold-ers. First con-vened in 2008,

the Public Stakeholders are a commit-tee of consumer advocates and healthexperts brought together to advise thepolicies of the NBME and to representthe public voice in NBME’s pursuit toprotect the public health. With eachcolumn, we will be sharing news andissues of interest to consumers ofhealthcare.

The Public Stakeholders have beencharged with identifying and evaluat-ing potential collaborative efforts, mak-ing recommendations for shifts inpriorities in the NBME’s strategic focus,evaluating potential roles for lay partic-ipants in the USMLE process, and con-sidering the optimal NBME structurefor outreach and communication with

4

the public. Recent highlights of thegroup’s activities include the pilot ap-pointment of public members to testdevelopment committees, the develop-ment of a Facebook page to contributeto the increased social media presenceof the NBME, and a “brown bag” pres-entation with NBME staff to discussthe importance of public engagement.The group contributed to the NBMEHealth System Reform Policy docu-ment, promoting the goals of ongoinghealthcare reform to enhance overallhealth system quality and efficiency.The final document is available atwww.nbme.org/about/health-system-reform-policies. Currently, the commit-tee is engaged in the development of astrategic plan, which will guide theNBME’s engagement of and collabora-tion with the public.

In an effort to increase public represen-tation in the NBME Membership andother committees, the NBME will beexpanding the recruitment process forpublic members. In the nomination of

committee members, the NBME isseeking voices outside of the tradi-tional House of Medicine. If you havesuggestions for potential public mem-ber nominees, please contact BaylorHarton at bharton@nbme.org.

At the 2012 Annual Meeting of theNBME, Chair Lewis R. First, MD an-nounced the promotion of the PublicStakeholders from a taskforce to a per-manent committee, demonstratingNBME’s sustained commitment to theobjectives of the committee. The com-mittee is headed by Executive Boardmember Susan Spaulding. Ms. Spauld-ing’s commitment to public protectionwas documented in a recent edition ofthe Examiner. With her support, mem-bers of the committee will work withthe NBME to ensure that the policies ofthe organization are representative ofthe interests of patients and the com-munity. This column will help to ensurethat, as Ms. Spaulding states, “the pub-lic is part of the conversation.”

INSIDE THE NBME

From the Public Stakeholders Committee

PUBLICSTAKEHOLDERS

(Peter Katsufrakis, continued from page 2)

School and Health Professions Services,Post-Licensure Assessment Services andthe Assessment of Professional Behav-iors program.

A board-certified family physician,Peter is a past associate dean for

student affairs at the Keck School ofMedicine of the University of Califor-nia. Active within the Association ofAmerican Medical Colleges Group onStudent Affairs (GSA), Peter served asnational chair on several GSA commit-tees. Peter has done extensive researchin areas of HIV/AIDS, sexually transmit-ted diseases, and physician education

and competence. Peter received his MDdegree from the University of Califor-nia, San Diego and his MBA from theUniversity of California Marshall Schoolof Business.

Please join us in congratulating Peteron his latest achievement.

Dr. Hodges has devoted his profession-al career to the pursuit of excellence inassessment, which is the essence ofthe Hubbard Award. Relentless in hisefforts to enhance the quality of evalu-ation locally, nationally and interna-tionally, Dr. Hodges has shownsupreme mastery of the field of evalua-tion and a willingness to share hisexpertise globally. His academic biog-raphy, including wide-ranging publica-tions in the field of evaluation, demon-strates sustained innovation in assess-ment research. He has presented hun-dreds of talks, keynote addresses, lec-tures, workshops, and posters at schol-arly conferences and has been hon-ored over 30 times with awards for hiseducational scholarship. Today we are

pleased to add the Hubbard Award tothese honors.

“Dr. Hodges brings a restless thought-fulness to his work in assessment andevaluation through a professional life-time of asking challenging questionsand fostering conversation amongmedical educators. Through hisinsightful articulation of the issues, Dr.Hodges brings a new perspective toevaluation and encourages everyoneinvolved in assessment to ask theright questions in order to improvetheir work.

“Dr. Hodges re-examined the ObjectiveStructured Clinical Examination andprovided us with a history of its devel-opment and adoption over 30 years.He has confirmed and refuted our col-lective approach to the OSCE; he hasarticulated our concerns about self-assessment and professionalism andhis research has provided new insightsinto both topics; and he is contributingsignificantly to the conversations aboutcompetence and ‘incompetence’ in thehealth professions. Dr. Hodges’s workreaches beyond the boundaries ofmedicine to include all health profes-sions and it reaches beyond theboundaries of the ‘usual methods’ toexplore new applications and newmethods. Dr. Hodges is a visionary

5

PROFILE

At the Annual Meeting of the NBMEMembership on March 29, 2012, theNBME announced the selection ofBrian David Hodges, MD, PhD as therecipient of the 2012 John P. HubbardAward. The Hubbard Award is given toindividuals who have made outstand-ing contributions to the pursuit ofexcellence in the field of evaluation inmedicine.

In presenting the award, ProfessorDame Lesley Southgate, Chair of the2012 Hubbard Award Committee, stat-ed: “In recognition of his brillianceand exceptional leadership in evalua-tion scholarship, I am pleased toannounce that the recipient of the2012 Hubbard Award is Brian Hodges.

Brian Hodges Receives the 2012 John P. Hubbard Award

(continued on page 6)

Dr. Hodges (holding award) with Drs. First and Melnick and Dame Lesley Southgate.

who has contributed tremendously toour collective understanding of assess-ment and evaluation.

“Through his leadership of theUniversity of Toronto Wilson Centre formany years, and his new role as VicePresident, Education of the UniversityHealth Network, Dr. Hodges has culti-vated a group of researchers whosefocus is research on assessment and theapplication of theory to practice. He fos-ters the application of new methodolo-gies to research in evaluation andassessment and pushes us to thinkbeyond medicine. His work has beenapplied to nursing, social work, veteri-nary medicine, sociology, dentistry, andpharmacy. He challenges us to thinkabout the notion of competence and itsmaintenance through assessment. Dr.Hodges implores us to approach assess-ment and evaluation in a balancedfashion with papers such as ‘Navigatingbetween excessive examination andnaïve reliance on self-assessment.’

“In his work, he continues to explorethe idea of professionalism and itsassessment, and he has recently intro-duced the concept of ‘discourse’ whenconsidering physician competence. Dr.Hodges is expanding the boundaries of

the methodologies used to researchassessment methods, as well as thevalidity and reliability of examinations,and continues to bring fresh insightsand new perspectives to the topics ofprofessionalism, self-assessment, per-formance-based assessment, andexpertise.

“The impact that Dr. Hodges has hadon assessment and evaluation in NorthAmerica and throughout the world isnothing short of profound. He has a20-year history of publication and pre-sentations about assessment; he hasmentored dozens of researchers whocontinue the work he began. In every-thing he does, Dr. Hodges manages toserve as an example, to recognize theeducational possibilities, the ‘teachingmoments,’ and to encourage others tobenefit from these possibilities.

“The Hubbard Award Committeedescribed Dr. Hodges as a dynamicforce in the current trend away fromtraditional time-based approaches tocompetency-based approaches in grad-uate medical education. Dr. Hodgescontextualizes what assessment is,including its meaning and implicationssociologically. He is a philosopher andsocial commentator as well as an out-standing medical education andassessment researcher.”

Dr. Hodges is Richard and ElizabethCurrie Chair in Health ProfessionsEducation Research at the WilsonCentre for Research in Education;Senior Fellow, Massey College;Professor, Department of Psychiatry,University of Toronto; and Vice-President, Education, University HealthNetwork.

The NBME established the HubbardAward in 1983 in special tribute to thelate John P. Hubbard, MD. Honoring Dr. Hubbard as a principal, guidingforce of the NBME, this awardacknowledges his creative and inspiredleadership of the organization duringhis 25-year tenure as its chief execu-tive. Dr. Hodges deservedly joins theranks of the distinguished individualswhom the NBME has honored over theyears with this prestigious award. Hisselection as the recipient followed acall for nominations published widelyin the spring of 2011. The 2012Hubbard Award Committee, chaired byProfessor Dame Lesley Southgate,included as members Michael Jodoin,PhD, Susan R. Johnson, MD, MS,Kenneth M. Ludmerer, MD, MA, John J.Norcini, PhD, and Patricia N. Whitley-Williams, MD.

6

(2012 Hubbard Award, continued from page 5)

7

In support of innovation in medical education assessment,the Edward J. Stemmler, MD Medical Education ResearchFund of the National Board of Medical Examiners ispleased to announce the 2012-2013 Stemmler Fund Callfor Letters of Intent.

Application Due Date: JULY 2, 2012

Applicants may go to http://www.nbme.org/research/appli-cationcenter.html for application instructions and to readthe 2012-2013 Call for Letters of Intent in its entirety.

PURPOSE and GOALS: The purpose of the StemmlerFund is to provide support for research and development ininnovative evaluation methodologies or tech-niques, with the potential to advance assessment in med-ical education or practice. Expected outcomes includeadvances in the theory, knowledge, or practice of assess-ment at any point along the continuum of medical educa-

Stemmler Fund Call For Letters of Intent

INSIDE THE NBME

tion, from undergraduate and graduate education andtraining through practice. Both pilots and more compre-hensive projects are of interest. Collaborative investigationswithin or among institutions are eligible and encouraged,particularly as they strengthen the likelihood of the pro-ject's contribution and success.

ELIGIBILITY: Eligible applicants for Stemmler Fund grantawards are medical schools accredited by the LCME or theAOA. The number of Letters of Intent submitted by any eli-gible school is not limited.

BUDGET INFORMATION: In the 2012-2013 fundingcycle, applicants may request up to $150,000 of NBMEfunding support for a project period of up to two years.

For more information:Web: http://www.nbme.org/research/applicationcenter.html E-mail: stemmlerfund@nbme.org

2011-2012 Stemmler Fund Grantees

The University of IowaPrincipal Investigator: Dr. Donald AndersonGrant Amount/Duration: $150,000 for two yearsProject Title: Simulation Approaches for Training in Fluoroscopically Guided Orthopedic Trauma Surgery

The University of British ColumbiaPrincipal Investigator: Dr. Kevin EvaGrant Amount/Duration: $78,306 for two yearsProject Title: Rater Cognition as Categorical Judgments: Using “Person Models” to Understand Rater Error

The Ohio State UniversityPrincipal Investigator: Dr. Douglas DanforthGrant Amount/Duration: $149,862 for two yearsProject Title: Virtual Patients Simulations to Assess Data-Gathering and Clinical Reasoning

For more information on these Stemmler Fund grantees please visit www.nbme.org.

On March 29, 1990, the NBME held aninvitational conference in Philadelphiaas part of a series of events celebratingits 75th anniversary. Distinguished par-ticipants presented their perspectiveson priorities that the National Boardshould consider as the 21st century ap-peared on the horizon. In the publishedproceedings of the conference, thenSenior Vice President Donald E. Mel-nick, MD discussed “NBME Then andNow-1915-1990.” As part of his re-search, Dr. Melnick interviewed a physi-cian who took the National Boardexams in 1927 and 1928. Dr. Melnickwrote:

Walter MacPherson may not be theoldest living diplomate of the NationalBoard, but he was the oldest livingdiplomate whom I could locate and in-terview. Dr. MacPherson, MD, Collegeof Medical Evangelists, 1924, certifiedby the National Board August 23,1928, is an internist and served asdean of Loma Linda University Schoolof Medicine (then College of MedicalEvangelists) for 32 years. Dr. MacPher-son shared his recollections of the1927 and 1928 National Board exami-nations during a telephone interview:“in those days, the National Board wasrelatively new among us. They were ofgood quality. They weren’t easy, and asyou probably know from the records,unless a person had prepared to takethem, he didn’t pass them. They werehard. They were rather tough. Butwhen the time came for the examina-tions, they were fair. They were of a na-ture that required considerable thoughtand considerable preparation, andeventually everybody around theUnited States began to realize that Na-tional Board examinations weren’t justordinary day by day examinations. Any-body who took them and passed themhad a sort of a feather in his cap forhaving done it. I do remember that

they were essay examinations. I likeessay examinations. The main problemwith essay examinations is that some-body has to read them. To take that ex-amination (Part III) with Verne Mason,who was one of my main examiners, Iliked it; I thought that was fun. I re-member one of the patients particu-larly, a syphilitic aneurysm of the aortathat had eroded clear through the ster-num. I don’t remember some of theothers, but that one at least gave riseto a lot of questions. Each examineewas examined by different people, andso in the post-mortems after the exam-ination, some of the fellows thoughtthat they were hard, some of themthought they were relatively easy. Now,also in those days, we got a gold key towear. It was the style to wear a vestwith a chain and the key in the middleof it that hung on the vest. Those keyswere always visible and people lookedat them. It just made a person sort offeel good to be wearing a NationalBoard key. Also, we commonly used theDNB signatures after our names alongwith other scholastic degrees — theDNB was right in there with them.There was a certain amount of prestigein those early days to have a diplomafrom the National Board.”

8

A Physician Remembers: 1927–1928 National Board Exams

FROM THE ARCHIVES

Diplomate of the National Board (DNB) key.