2007; 29: 711–716

Assessing medical students for non-traditionalcompetencies

STEPHEN R. SMITH, ROBERTA E. GOLDMAN, RICHARD H. DOLLASE & JULIE SCOTT TAYLOR

Brown Medical School, Providence, RI, USA

Abstract

Assessing medical student competence in non-traditional domains can be challenging. Conventional methods of assessment are

generally unsatisfactory. The authors discuss the approach taken at the Brown Medical School, USA to assess students at the

beginner, intermediate, and advanced levels in the social and community contexts of health care – one of the nine abilities that

comprise Brown’s competency-based curriculum. At the beginner level, faculty use weekly field notes written by students about

their experiences in community practice placements as the means for assessment. At the intermediate level, faculty assess students

based on their completion of a project focused on a community health problem. At the advanced level, a screening process is used

in which students first meet individually with a member of the assessment committee who determines if the student’s efforts

appear to meet the expected standards. If success seemed likely, then the student was encouraged to appear before the whole

committee. The authors discuss the common use of subjective judgments at all three levels and the importance of diverse

perspectives in achieving a consensus. Allowing students wide latitude in expressing themselves and their accomplishments helps

to ensure success.

Introduction

As competency-based education has become more wide-

spread among medical schools around the globe, educators

confront challenges in assessing medical students in those

competencies that have not traditionally been a focus of

teaching and learning. These non-traditional competencies

include professionalism, life-long learning, self-awareness and

personal growth, moral reasoning and clinical ethics, and the

social and community contexts of health care, among others.

This article will focus on how one institution, Brown Medical

School, approached the problem for one of those abilities – the

social and community contexts of health care – at three levels

of medical student proficiency.

Assessment of competence is a relatively straightforward

task for more traditional abilities. Early adoption of a

competency-based approach to health professional education

occured in procedure-oriented disciplines such as physical

therapy, medical technology, and dentistry. Assessment

focused on technical procedures that could be broken down

into discreet steps and for which generally agreed upon

standards of proficiency were reasonably easy to define and

measure.

Indeed, it was this early focus on technical procedures

that engendered much of the skepticism toward a

competency-based approach to medical education. Critics

perceived that a competency-based curriculum would be too

narrowly focused and technically oriented (Jessup 1991)

would reduce job competence to reductionistic, ‘atomized,

observable behaviors’ (Saunders 2000) and would not capture

the holism of medical practice (Norris 1991).

These fears proved to be unfounded. Medical schools

avoided these pitfalls by defining the competencies and

outcomes in broad terms that encompass the full spectrum

of the profession of medicine (Harden et al. 1999).

Postgraduate medical education has followed suit, with

the Accreditation Council for Graduate Medical Education

(ACGME) in the United States defining six general

competencies for which all residency programs must assess

their residents (Leach 2002).

The broad scope of competencies in medical education that

characterize the current landscape in curricular planning

do not lend themselves to traditional methods of assessment.

Practice points

. Faculty who assess students’ competence must be

recognized by peers and students as experts in the

field, capable of discerning the quality of students’

efforts – they must be ‘connoisseurs’.

. Subjective judgments may be used, provided that

the faculty have ample opportunity to share their

perspectives with one another in order to arrive at a

consensus.

. Students should be given wide latitude in expressing

themselves and the work they have accomplished –

letting them ‘show their stuff’.

. The boundaries among teaching, learning, and assess-

ment should be blurred, with faculty engaged with

students in an encouraging, iterative process working

towards the goal.

Correspondence: Stephen R. Smith, Brown Medical School Box G-A134, Providence RI 02912, USA. Tel: (401) 863 9781; fax: (401) 863 7411;

email: Stephen_R_Smith@brown.edu

ISSN 0142–159X print/ISSN 1466–187X online/07/070711–6 � 2007 Informa UK Ltd. 711DOI: 10.1080/01421590701316555

Med

Tea

ch D

ownl

oade

d fr

om in

form

ahea

lthca

re.c

om b

y SU

NY

Sta

te U

nive

rsity

of

New

Yor

k at

Sto

ny B

rook

on

10/3

1/14

For

pers

onal

use

onl

y.

And comprehensive elaboration of competency areas brings

with it its own set of problems. A competency domain may

be so broad as to encompass a heterogeneous group of

discreet abilities, for example, in the competency of patient

care. A global assessment of patient care might

overlook deficiencies in any individual ability component

(Torbeck & Wrightson 2005). But should proficiency in

one component, such as diagnostic procedures, be

allowed to compensate for weakness in another area,

such as history-taking? Most medical educators would not

subscribe to such a compensatory assessment approach.

However, disaggregating broad competency areas into a

large number of very specific behaviors leads back to the

original set of criticisms of competency-based education,

namely, that it can become reductionistic and miss the forest

for the trees.

Brown Medical School inaugurated its competency-based

curriculum in 1996 (Smith & Dollase 1999). Among the

nine abilities (see Table 1), Ability VII – The Social and

Community Contexts of Health Care embodies many of

the assessment challenges just described. Traditional

assessment methods, such as multiple-choice question tests,

standard oral examinations, checklists, record reviews,

OSCEs, 360� evaluations, patient satisfaction surveys, or

even portfolios are not appropriate. A qualitative approach

is more suitable, but it creates problems related to

giving students clear and reasonable expectations of

the performance standards required, and then evaluating

achievement of those standards. Medical faculty are

often reluctant to fail students based on qualitative – or

what is more often described in a pejorative tone as

‘subjective’ – standards.

This article describes how different qualitative methods

of assessment are used to assess medical student

competence in the non-traditional ability of the social

and community contexts of health care at three levels of

proficiency. Achievement at the beginner and intermediate

levels is required; advanced competency is optional.

Description of the ability

The educational mission statement of Brown Medical

School emphasizes the goal of producing socially

responsible physicians. The faculty, staff, and students who

were involved in the original design of the competency-based

curriculum sought to capture this concept in Ability VII – The

Social and Community Contexts of Health Care:

The competent graduate provides healing

guidance by responding to the many factors that

influence health, disease and disability, besides those

of a biological nature. These factors include

sociocultural, familial, psychological, economic,

environmental, legal, political and spiritual aspects

of health care seekers and of health care delivery.

Through sensitivity to the interrelationships

of individuals and their communities, the

graduate responds to the broader context of medical

practice.

The criteria for assessing competence in Ability VII are listed in

Table 2. At both the beginner and intermediate levels, the

curriculum contains structured opportunities for students to

demonstrate their competence; advanced competency is

gained through students’ independent initiatives. The levels

of achievement are based on a hierarchy of how students

respond to the values and attitudes embodied in Ability VII,

which correspond to Bloom’s hierarchy of values (Krathwohl

et al. 1964). At the beginner level, students are expected only

to recognize and acknowledge the importance of non-

biological factors in patient care. At the intermediate level,

students are expected to demonstrate their recognition of the

importance of these issues, thus ascribing value to them. At the

advanced level, students respond to these issues sponta-

neously, reflecting an incorporation of the value into their

own world view.

Beginner-level

Beginner-level students meet the criteria for the social and

community contexts of health care in their first two years of

medical education. They do this by recognizing and reflecting

on their own cultural values and traditions, as well as reflecting

on the diverse values and traditions of the patients and health

care providers they encounter.

These goals are met through student participation in our

new two-year Doctoring course which is designed to teach

patient-centered care through acquisition of the knowledge,

skills, attitudes and behaviors necessary to be a competent,

ethical and humane physician. Social medicine themes run

through the entire curriculum. During each semester of

Doctoring, students explore personal beliefs and biases,

while gaining a deepening understanding of the role of

poverty, race, and ethnicity on patient health and health care.

Students spend one afternoon each week working in a

community physician’s office where students are encouraged

to consider how a patient’s illnesses may be affected by such

factors as inadequate and unsafe housing, the lack of health

insurance or societal discrimination. They spend another

afternoon a week participating in a small group, case-based

discussion, facilitated by a physician and a social/behavioral

science instructor.

Table 1. Brown medical school’s nine abilities.

I Effective communication

II Basic clinical Skills

III Using basic science in the practice of medicine

IV Diagnosis, management, and prevention

V Lifelong learning

VI Professional development and personal growth

VII The social and community contexts of health care

VIII Moral reasoning and clinical ethics

IX Problem solving

S. R. Smith et al.

712

Med

Tea

ch D

ownl

oade

d fr

om in

form

ahea

lthca

re.c

om b

y SU

NY

Sta

te U

nive

rsity

of

New

Yor

k at

Sto

ny B

rook

on

10/3

1/14

For

pers

onal

use

onl

y.

Documentation and reflectionthrough field notes

Students in the Doctoring course are required to write weekly

essays – field notes – in response to questions that trigger and

guide their reflection on a topic related to the goals of the

weekly didactic session and students’ participation in the

community physicians’ offices. Students are also encouraged

to reflect on their own questions that arise during their time

at the physicians’ offices. These notes are submitted

electronically to the student’s small-group faculty leader for

review and comment.

The small-group leaders provide timely written feedback to

students by critiquing their field notes by posing questions,

providing alternative interpretations of events, or playing

devil’s advocate. As a form of continual formative evaluation,

the field notes serve to help determine the progress of

individual students in achieving competency as well as

indicating the effectiveness of the Doctoring course in meeting

its goals. This form of assessment is developmentally appro-

priate and individualized to take into account where each

student is at in his or her intellectual and professional growth.

The purpose of the field notes is to promote reflection and

growth that help foster the development of the ‘mindful’

medical student. Field notes become an interactive process as

students receive in-depth, personalized feedback on their

weekly notes from the physician and/or social/behavioral

scientist. The small group leaders critique the field notes

by posing questions and providing alternative interpretations

of events.

The content of the field notes remains confidential between

the teachers and students, which when given the proper

nurturing of the relationship, can create a ‘comfort zone’ for

deeper and more authentic self-reflection. The field notes

document the students’ learning journey as teachers help

students appreciate the cognitive, affective, personal and

professional dimensions of their experiences. The field notes

and the small group discussions provide sufficient data for

faculty to make an informed judgment of whether or

not students can be awarded beginner-level certification for

Ability VII.

First-year students’ reflections

The first year of Doctoring centers on students gaining

mastery of the medical interview and reflecting on the

physician–patient relationship from their perspective as

novices. This initial self-reflection helps medical students

start on the doctoring path by having them articulate their

values, hopes, and aspirations as future physicians.

As the first year progresses, students reflect on their practice

of the medical interview, particularly in regard to taking a

family, social and sexual history. During the medical interview,

students elicit the patient’s perspective and beliefs in regard to

illness and disease, and they inquire about the meaning the

patient attaches to the illness. They assess the impact of the

illness on the patient’s self-image, job and finances. Students

consider such field-note questions as: Have you found it easy

to ask about social factors? What are your impressions about

how shame may impact on the physician-patient encounter?

Does the age, sexual orientation, or gender of the patient make

taking the sexual history more challenging?

Second-year students’ reflections

Year Two of Doctoring builds upon the clinical skills acquired

in the first year with emphasis on mastery of the complete

physical examination and the application of advanced

Table 2. Criteria for assessment for ability VII – the social and community contexts of health care.

1. Appreciates the importance of the many non-biological factors that influence health, disease, disability, and access to care

. obtains information in the patient’s history of these factors.

. attributes proper importance to identified non-biological factors.

. inquires about value systems and lifestyle in a nonjudgmental fashion.

. avoids stereotypical language (e.g., racist, sexist, or homophobic remarks).

. identifies barriers to access of health care resources.

2. Utilizes appropriate resources in the community that may provide support for reducing social causes of disease

. becomes familiar with role of community resources and services provided.

. matches patients’ needs to appropriate community resources.

. communicates the availability of community resources to patients and their families.

. cooperates with community resources through follow-up efforts and support.

3. Acts as an advocate for better health for patients and the community

. assists patients and their families in navigating through bureaucracy.

. supports community activities designed to improve health.

. supports social and political activities to improve access to health care.

. places patients’ and community’s welfare above narrow self-interest.

. encourages actions designed to enhance the total well-being of individuals, families, and communities.

Assessing medical students for non-traditional competencies

713

Med

Tea

ch D

ownl

oade

d fr

om in

form

ahea

lthca

re.c

om b

y SU

NY

Sta

te U

nive

rsity

of

New

Yor

k at

Sto

ny B

rook

on

10/3

1/14

For

pers

onal

use

onl

y.

physician–patient communication skills such as informed

decision-making, end-of-life issues, and cultural competence.

Students write field notes regarding informed consent,

advanced directives, giving bad news and how issues of

spirituality and the end of life are addressed by the community

physician mentor.

As part of a final project, students write a reflective field

note on the patient’s health insurance status. A second-year

student summarized her views this way:

As physicians, we certainly have a responsibility to

advocate for our patient’s best interests, which

includes improving and equalizing their access to

health care. That could mean educating patients on

enrolling their children in the state’s children’s health

care program or working with task forces on

developing newer systems. Either way, some sense

of social responsibility should be here; especially

since it is evident that our current system manages to

provide good care to those who are fortunate

enough to afford it while leaving behind a significant

percentage of equally needy patients. It may be

impractical to enforce advocacy; but by placing it in

the school’s curriculum, rising doctors will hopefully

be better informed and even impregnated with a

desire to take some action, whether that be with a

single patient, community or on a larger scale.

The field note reflections of this second-year student

encapsulate the outcomes we hope that our students obtain

at the beginning level of Ability VII: Recognition of, if

not dedication to, patient-centered health care, social respon-

sibility and advocacy for change in our health care system.

Intermediate level

Students meet the criteria for Ability VII at the intermediate

level within the required family medicine and community

health clerkship rotations. Here we will focus only on the

family medicine experience. In this clerkship, students identify

non-biological factors contributing to health and disease

through history-taking with their patients. They then consider

how to approach these factors at a community-wide level.

Students take personal responsibility for discussing these

issues with patients, assessing their needs and matching

them to appropriate community services, arranging for

referrals and assisting the patient and family in navigating

through any bureaucratic hurdles. Students also consider how

these problems can be approached from a population-based

perspective.

In addition, students choose between designing a hypothe-

tical community intervention project or actively participating

in a service-learning experience at a local Latino social service

agency. Either project is worth 15% of a student’s final grade.

In the last two years, approximately 70% of the class have

chosen the intervention project and 30% have chosen service-

learning. All students give a five-minute oral presentation on

the last day of the clerkship.

The first option, a hypothetical intervention, has six major

components: (1) identify a public health issue relevant to a

patient at an outpatient clinical site; (2) find out more about the

issue by talking with patients and preceptors; (3) locate

existing community resources; (4) compile that information

into a one-page handout for patients; (5) generate a research

hypothesis; and (6) propose a clear, innovative community

intervention that is outside of the doctor’s office. Often

students will choose a topic that is already of medical interest

to them and will use this project to consider the issue from a

community perspective.

The second option, service learning, places students in a

community-based social service agency serving Latinos and

other immigrants. Students learn to broaden their view of

the boundaries of medicine by investigating at the commu-

nity level common problems seen in individual patients.

Guided by a team of agency staff and faculty, students

perform needs assessments and implement activities in

the community designed to meet those or previously

identified needs. Students complete a unique, self-contained

project as a group. These individual projects may be part of

a larger, ongoing endeavor that subsequent groups can

build upon.

The grading process is relatively similar for both options

and the inherently subjective process is made as clear as

possible for students and as objective as possible for faculty

(see Table 3). Two faculty evaluators listen to the presenta-

tions, independently calculate numerical grades and com-

ments and meet to determine the final grade and comments.

Students who participate in the service-learning project tend to

get the highest grades, but students who undertake the other

project also achieve high marks when they follow the

directions explicitly and comprehensively. Students who get

the lowest grades are those who give presentations on a health

topic without completing the inventory of community

resources or producing a patient-education handout. Lower

grades are also assigned if the students’ hypothetical interven-

tions consist of something that happens during an individual

medical encounter rather than the broader community, thus

demonstrating that they failed to appreciate the social and

community contextual elements altogether.

Advanced level

Advanced competency is an optional level of achievement,

and requires independent action in the service of a broad

advocacy goal. As evidence that the student has internalized

the values inherent in Ability VII, students must act out of their

own volition rather than as a prompt from faculty. At this stage

students are pro-active rather than simply reactive to a specific

situation. Students will be able to generalize from individual

and family problems to a broader community context. They

will be able to analyze these community health problems

in the context of political, sociological, cultural, economic and

other factors. Advanced students will go from analysis to

action, taking the initiative themselves to promote solutions to

the identified problems. These are students who demonstrate

commitment to community service that is likely to endure into

their future as medical professionals, and who show great

promise as potential leaders in this area.

S. R. Smith et al.

714

Med

Tea

ch D

ownl

oade

d fr

om in

form

ahea

lthca

re.c

om b

y SU

NY

Sta

te U

nive

rsity

of

New

Yor

k at

Sto

ny B

rook

on

10/3

1/14

For

pers

onal

use

onl

y.

The original committee that developed certification criteria

recognized that these were high standards and that a relatively

small number of students each year would attempt to seek

certification. The committee wanted to ensure that the few

students who received advanced-level certification truly

deserved it, that they really ‘got it’ in terms of internalizing

the concepts underlying this ability. The committee struggled

to clearly operationalize their expectations for a student’s

having ‘gotten it’, and to develop and apply uniform criteria in

the context of diverse student undertakings.

An experience with one of the first students to apply for

advanced certification confirmed the committee’s concerns.

The student had completed a master’s degree in public health

and for this work alone sought advanced-level certification

from the committee. While the student was clearly knowl-

edgeable in the discipline of public health and epidemiology,

it was clear to the committee that he did not ‘get it’ when it

came to the world view of a socially responsible physician.

Despite committee members’ numerous misgivings, the

student was awarded advanced-level certification. The com-

mittee’s reluctance to turn down the student’s application was

due to its own uncertainty whether it had clearly articulated

defensible standards to the students, and because it was fearful

at that early stage of discouraging deserving students from

applying.

The committee then devised a screening process to assure

that students who are encouraged to appear before the

committee are highly likely to have met the standards for

advanced competency in the ability. The approach is a

variation on the idea of ‘connoisseur’ assessment by experts

in the field (Des Marchais & Bordage 1998). A faculty member

of the committee meets individually with potential candidates.

This meeting is not characterized as a formal assessment.

Rather, it provides an excellent mechanism for students to

think critically about the work they have done within the

context of the precepts of the ability, and the exploratory

conversation enables the faculty member to get a better sense

of what the student has undertaken. In this way, faculty

are able to advise the student about expectations for

demonstrating advanced competency, and they discuss the

likelihood that the student will be successful in obtaining this

certification.

The faculty member uses his or her expert judgment as a

‘connoisseur’ to make a determination of whether or not the

student ‘got it’. If the student’s work seems appropriate,

the faculty member encourages the student to appear before

the committee for a formal presentation. The faculty member

provides the student with written guidelines for advanced

competency and offers to serve as the student’s coach in

developing the formal presentation. When necessary faculty

encourage students who have done admirable work to more

specifically explore for themselves how their work relates to

the social and community context of care, thus ensuring that

they are better prepared to address this issue in their

presentation.

If the student’s work does not warrant advanced certifica-

tion or the student’s perspective does not encompass the

underlying precepts of the competency, the faculty member

tells the student why he/she does not recommend pursuing

advanced competency. Sometimes it is helpful for the faculty

member to describe successful projects as a means of

indicating to the student what the committee expects.

Students are then able to see the difference between the

quality or content of their work and that of students who had

achieved advanced competency.

Following a committee member’s recommendation,

students present to the committee by making a formal 20 to

30minute presentation. They are encouraged to structure their

presentation however they prefer to get their points across,

and are required to describe their project and its outcomes

as well as reflect about what they have done and learned.

The committee relies on their expert judgment to come to a

consensus on whether or not the student should be awarded

advanced-level certification.

Most presentations are made during the students’ fourth

year of medical school.

Students are encouraged to view the presentations less as a

required task for competency than as a unique opportunity to

Table 3. Guidelines given to students on how their projects for intermediate-level ability VII–the social and community contexts of health carewill be graded.

Option 1: Hypothetical community intervention

1. Background: what is the issue and why is it important? (2 points)

2. Community resources: what community resources are already available? (3 points)

3. Hypothesis: what impact might an intervention have on this issue? (2 points)

4. Proposed intervention: describe your hypothetical community intervention. (2 points)

5. Benefit to patients: how would your intervention benefit the patient/family? What barriers might prevent your patient from utilizing or benefiting from the

intervention? (2 points)

6. Presentation: clear and simple slides, a concise talk lasting no more than 5 minutes, and a well-formatted resource handout. (4 points)

Option 2: Service-learning project

1. Participation and attitude: interactions with peers, social service agency staff and clients. (3 points)

2. Assessment of the community: needs assessment and how you came up with the project idea. (4 points)

3. Implementing a service: written summary of what was done for the project. (4 points)

4. Presentation: timing, clarity, organization, speaking style, use of visual aides. (4 points)

Assessing medical students for non-traditional competencies

715

Med

Tea

ch D

ownl

oade

d fr

om in

form

ahea

lthca

re.c

om b

y SU

NY

Sta

te U

nive

rsity

of

New

Yor

k at

Sto

ny B

rook

on

10/3

1/14

For

pers

onal

use

onl

y.

discuss and celebrate the culmination of their work with a

group of highly interested faculty and students. The presenta-

tions have therefore become collegial seminars where intense

discussions develop about the students’ projects and how their

experiences will be reflected in how they develop and practice

as a physician during residency and beyond.

Since the screening method and faculty–student prepara-

tion sessions were instituted, all students who have presented

to the committee have received certification for advanced-level

Ability VII with enthusiastic and unanimous decisions by the

committee. At the same time, none of the students who have

been discouraged from pursuing their candidacy have chosen

to do so in spite of the advice. None have appeared

disgruntled; rather, they seem to have appreciated the honesty

of the advice and the resulting avoidance of futile effort and

disappointment. Indeed, the achievements some students

present for consideration are impressive; they are just not in

the area of Ability VII. Those students receive encouragement

to apply for advanced-level certification with the appropriate

ability assessment committee, often with a supportive

referral from us.

The key ingredients in the success of this model are: having

committee members who are recognized as experts by their

peers and the students in the competency; the committee’s

willingness to entrust the initial screening process to individual

members; and the ability of the faculty member to honestly

convey his or her expert judgment to the student in a

supportive and appreciative way, even when that means

discouraging the student from presenting. Additional factors

that are important are the committee’s willingness to have just

a few students each year who are awarded advanced-level

certification and that it is not required for all students to

achieve advanced-level certification in this competency.

Conclusions

Though different in many ways, the approach to assessing

student competence in this non-traditional ability at the

beginner, intermediate, and advanced level has certain key

common elements. First among them is the vital necessity of

selecting faculty who are recognized as experts by their peers

and the students in the competency (they are ‘connoisseurs’).

While sharing the same dedication to the social and commu-

nity contexts of health care, committee members also come

from a diverse range of disciplines and experiences.

Expression of that diversity in energetic dialogue among

the faculty assures that our judgments are not idiosyncratic or

arbitrary. We are not afraid of the subjective nature of our

assessments, since they are open to the critical appraisal of

our colleagues and need to achieve a common consensus.

Creating an opportunity for those kinds of conversations to

occur is essential.

The second common element is the blurring of lines

between teaching, learning, and assessment. Students are

given wide latitude to express themselves in ways that are

important and meaningful to them. Faculty, in turn, respond to

that with encouragement and engagement.

The way in which we assess competencies should mirror

the characteristics we seek to measure. We believe the

approach we have taken to assess the social and community

contexts of health care reflects the very values that we hope

students will themselves demonstrate.

Notes on contributors

STEPHEN R. SMITH, MD, MPH is Associate Dean and Professor of

Family Medicine.

ROBERTA E. GOLDMAN, PhD is Clinical Associate Professor of

Family Medicine.

RICHARD H. DOLLASE, EdD is Director, Office of Curriculum Affairs.

JULIE SCOTT TAYLOR, MD, MSc is Assistant Professor of Family Medicine.

All the authors are from Brown Medical School, Providence,

Rhode Island, USA.

References

Davis MH, Harden RA. 2003. Competency-based assessment: Making it a

reality. Med Teach 25:565–568.

Des Marchais JE, Bordage G. 1998. Sustaining curricular change at

sherbrooke through external, formative program evaluations. Acad

Med 73:494–503.

Harden RM, Crosby JR, Davis MH, Friedman M. 1999. From competency to

meta-competency: A model for the specification of learning outcomes.

Med Teach 21:546–552.

Jessup G. 1991. Outcomes: NVQs and the Emerging Model of Education

and Training (London, Falmer Press), pp. 31–39. As quoted in Davis &

Harden 2003.

Krathwohl DR, Bloom BS, Massia BB. 1964. Taxonomy of Educational.

Objectives: The Classification of Educational Goals, Handbook II:

The Affective Domain (New York, McKay).

Leach DC. 2002. Building and assessing competence: The potential for

evidence-based graduate medical education. Qual Manag Health Care

11:39–44.

Norris N. 1991. The trouble with competence. Cambridge J Ed 21:

331–341. As quoted in Davis & Harden 2003.

Saunders M. 2000. Understanding education and work. In: B. Moon,

M. Ben-Peretz & S. Brown (Eds). Routledge International Companion

to Education, pp. 700–701 (London & New York, Routledge), As

quoted in Davis & Harden 2003.

Smith SR, Dollase R. 1999. AMEE Guide No. 14: Outcome-based education:

Part 2 – Planning, implementing and evaluating a competency-based

curriculum. Med Teach 21:15–22.

Torbeck L, Wrightson A. 2005. A method for defining competency-based

promotion criteria for family medicine residents. Acad Med 80:832–839.

S. R. Smith et al.

716

Med

Tea

ch D

ownl

oade

d fr

om in

form

ahea

lthca

re.c

om b

y SU

NY

Sta

te U

nive

rsity

of

New

Yor

k at

Sto

ny B

rook

on

10/3

1/14

For

pers

onal

use

onl

y.