Illuminating the 'Black Box'A Description of 4464 Patient visits to 138 Family physicians

Kurt C. Stonge, MD, Ph,D; SleTtlætz J. Zyztttzski,, PltD; Caúas R. ,Iat*t, MÐ, PltÐ; þ)d,taarrl,l. Cal|aÌ¿cn"t,PltD; Robert B. Kellg, MD, Ìws; wittiam R. GitLa,nd,ers, MÐ; J. ctuistoph,er stm.nk, MD;Josotz Clzao, MD, MS; Jack H. Medali,e, MD, MPI|; Wi.l,Ii;a:n L. MilIIø., MD, MA;Beniarnin I1 Ct'øbh'ee, PhÐ; Susa,n, A. FLocke, PtlD; Va\eñe,L Gi.ktnist, MD; I)oreen M. Lctnga; cutrlMered,i,th" A. Goadwin, MSCl'eaeLand, Ohi'o; Bulfalo, Naa Yot'k; Saa'annerLto, Ca.Ii.fonti,a,; Oma,h,u, Nebra.slca,;Al.l,entotttn,Pennsylaani,a.; Røo ts f.otun, Olti,o

BACKGROUND. The content and context of family practice outpatîent visits have never been fully described,leaving many aspects of family practice in a "black box," uRseen by policymakers and understood only in isola-tion. This arlicle describes community family practices, physicians, patients, and outpatient visits.

METHODS' Practicing family physicians in northeast Ohio were invited to participate in a multimethod study ofthe content of prímary care practice. Research nurses directly obserued consecutive patient visits, and collectedadditional data using medical record reviews, patient and physician questionnaires, billing data, practice environ-ment checklists, and ethnographic fieldnotes.

HESUUIS. Visits by 4454 patients seeing 138 physicians in 84 practices were obserued. Outpatient visits to fam-ily physícians encompassed a wíde variety of patients, problems, and levels of complexity. The average patientpaid 4.3 visits to the practice within the past year. The mean visit duration was 10 minutes. Fifty-eight percent ofvisits were for acute illness, 24Vo for chroníc illness, and 12t/o for well care. The most common uses of time werehistory-taking, planning treatrnênt, physical examination, health education, feedback, family information, chatting,structuring the interaction, and patient questions.

CONCLUSIONS' Family practice and patient visits are complex, with competing demands and opportunities toaddress a wide range of problems of individuals and families over time and at various stages of health and ill-ness' Multimethod research in practice settings can identify ways to enhance the competing opportunitíes offamily practice to improve the health of their patients.

KEY WORDS. Physician's practice patterns; physícians, family; physicians' offices; preventive heatth services;family practice. (J Fam Pract 1998;46:3T7-389)

amily practice is poorly understood, despite itsrecent resurgence as a comerstone of theAmerican health ca¡e system.r{ Because of thelack of di¡ect data on the patient-physicianencounter and the limited number of resea¡ch

studie.s that assess community practice settings, policy-make.rs view rnany aspects of family practice as obscuredwithin a "black box," Extsting süudies of family practicesand patient visits to family physicians t¡4pically rely onsingle,souess of lrfognation ürcludirrgphysiciâtr tepört,rred¡{rl record levie4 patient sürrrêy, or billing dataEfich of llresessnrces of i¡rf.on¡ation:cån prÐ J¡lO¿r¿$sñrllens with which to view family practice. Ye! each has itsown source of bias.t' .{ mr:ltimethod approach enrphasiz

ing direct oiservation has never been used to describe alarge number of patient visits to fanúly physicians prac-ticing in community settings.

Intemational studies have examined the disease con-tent of general practice.*r" These studies and regishiesestablished important methods for classifying diseases,morbidity, and episodes of ca¡e.

The first mqior description of the content of farnilypractice patient visits in the United States was the 1976Virginia Study."''u This la¡rdmark sbudy involved physi-cians' reports of patient problems &uing 88,000 patientvjsits to 36 practicing family physicians ancl 82 famitypractice residents, By showing tlrc vzutety of problemsseen by family physicians, this study was critically inrpor-

Submìttud,, t'øuísed,, Mørch 10, 19g8.

$om îlæ Ða¡tar:rtl.sl¿t af Fiun:Ílrffied'ÍcÌnë {N.ç;s,, s.J.z,, Ê,Ï!.iK., J.c., J.H.M., S.A.E, D.M.L., M.A.G.), the Depa:t.ttnent ofE_ptttøa:iobglt &,3rfstd¿rb¿{r$ {å'.s-S,, .l,J_ãJi t}tÈ'.Ðêpçr'tffieïtt oJ Socínl.ogy çK.C.S.¡,'Case West¿m Rlserue ui.iaø.sít+t; ihehvlalûd Ç*nrcI{i at'Úa*e ff¿s{*r* 8¿saw {/ni.ual$ly a.xd'-thc Uni.aãsity HaípiøLs oJ Crøuetnnd, (KC.S., ,1.c., s.¡.2.,"Ê-Ë;Jl' "!.fLfuf,, S.eÊ);.llæ De¡ry1'tmenls o¡ îttîtrity Mvrlír,lx¿ ann Sací,at, a,nd, Freuøtttiae ltiedicí,r¿e, ønd,'tlæ Centø- for {h'ban.fiasørtlt i* ll{r¡t¿rå¡ Cw {CUfr,Ê ftq, $tcte {/rriuctsf4r o! Ncto York øt ßI4ffalþ G.n.J.); the Depørtment of Fatwíty and,&mwtat'ítg N*úlr.ina tlnlaersìlg o! Catidot'nía al t auis iA.S.c.¡; thß l[eñItea]th Meó¿co¡ Cenier, Ctøuaatd (R.B.K.); Suttert!9a!.Ut" Sauuntonto,_Caliþmìa ftUA,CS; AatÍan îleqtth. Fd¡tti,Ig Pronti,ce nesiÅørwg hograrn, Ind,Í.urm, Uníuasi.tg ichoot oflvletli'çit¡ts {"f l?'SJ-; Qualßlwíce Heo}t}t /Ptøn, Õløektttl tJ.:t ); lh.e DepùrtnLent of FarníLy Pnzr:ti.ce, Leh,igh Valley Hospitat,Allentaun, P* (W.LM.)' ü1ë.ÐWl!;tìr€tri oÍf*rtÍlg P¡uetí.cq {Jníaersity of Nebru^skn, Omahn (B.EC.); and. the Depai'nnent ofFawtly fruettt¿, Not'thãaal Olyia tJ¿iueßíriÆ.Caltege 4t üyfed,í*í,r¿,e, Rootstßu¡rl, Oh,í.o (VJ.G.). Requests for replints slwuld. bea¿kÞssssrJ ls J{rt¿t C. 8!4age, ùlÐ, PhÐ, tacs Wesla¡rJ *ç{rse¡?rÈ tÌni.aari,tg, logôo nuci¿a eae, CLettet¡wt¿, OII 4+106.

@ 1998 Appleton & IøngelSSN 0094-3509 'Ihe Joumal of Family Practice, Vol. 46, No. 5 (Ivlay), 1998 377

ILLUMINATING THE'BLACK BOX'

tant in defining the disease content of fanúly practice,'n andin setting educational,'? research, r8 and policy'!, prioritiesearly in the course of the discipline.

In asubsequent study using national datafrom multiplesources, samples of general practitioners and family physi.cians reported information about themselves, their prac-tices, and a sample of patient ofitce and hospital visits.",Titis study also had important implications for clinicalcare,rr education,È resea.rch,r' and policy.äs In addition, itdeveloped important new metlrods for clustering the widevariety of diagnoses that describe patient visits to familyphysicians.rtSubsequent ongoing suweys by the AmericanAcademy of Family Physiciand? a¡rd the National Centerfor Health Statistics National Anbulatory Medical CareSurvey (NAMCS)r'g'have continued to use physician self-report information üo describe various aspects ofthe careprovided by family plrysicians.

Despiüe the importance of these tand¡na¡k studies indescribing family practice, a new multimethod study isneeded for several reasons. First, previous resea¡ch waslimited to using nonobservational, physiciar-reportsources ofinformation. Second, the health system eontextof family practice has undergone significant changes in thepast decade.q'r Tlese contexhral changes are affecting thepatient-physician relationshipî¿s and other aspects of prac-tice.s In addition, family physicia¡s themselves are chang-ing; increasing numbers are residency-trained, younger,and female.ã Finally, family physicians are frequently con-fronted with efforts by others to change their approach topractice.æ These attempts at change, although often moti-vated by laudable goals of improving the quality,r cost-effectivenessff or scientific basis of patient ca.re,{,rroftenfail,e€ They fail in part because of a lack of understandingof the core processes and competing demands of realworld community famiþ praetice.€ Therefore, \rye ìrsed amultimethod approach#T ûo describe patient visits to fam-ily physicians in community practice. In addition, wesought to portray the context of these visits with briefdescriptiors of the practice settings, physiciars, andpafients, Ttris article reports selected descriptive quantita-tive data on characteristics of tlre practices, physicians,patients, and patient visits from the Direct Observation ofPrimary Ca¡e (DOPC) study.

ñiÐnr¡rnrStrns ¿,IvD StrBJEcrsTire DOPC methods have been described in det¿il elsewhere.? In the summe¡ of 1994, family physicia¡r membersof the Ohio Academy of Family Physiciars in northeastOhio were invit€d to participate in a study of the contentof family practice, and to become members of a practicebased network designed to seive a.s a laboratory forresearch on primary care practice. Physieians not workingin family practice settings and ñrll-time academic pþsi-cians were excluded with the exeeption of 30 members of

the faculty of the Northeast Olúo Universities Colleges ofMedicine (NEOUCOM), who practice i¡r conununity sitesthat function as training practices for fanily practice resi-dents. These 30 physicians pût¡cipate in the No¡th EastOhio Network (NEON)¡- of conuntnity teaching placticesperforming practice-based research. Basecl on calculationsof the sample size needed to answer specihc stncly qnes-

tions, a sample of 120 physicia¡rs was tatgeted. Of the 531physicians invited to participate, 138 vohurteerecl. Thesephysicia¡s became inaugural membem of the ResealchAssociation of Practicing Physicians (RAPP). This studyand subsequent RAPP studies are guided by a boald ofdi¡ectors of 14 participating physicians.

Corsecutive ou@atients seen by each plìysician cluring2 observation days between Octol:er 1994 and Angust 1995

were en¡olled, if they gave verbal infon.¡red consent. Eacl.rphysician's observation days were sepalated by an averageof 4 months, to maximize seasonal variation in the reasonsfor patient visits.

D¡r¿ Cor,ræcrroN PRocED{;eEsBefore beginning the data collection, the research nurseswere extensively trained in the use of a.ll research i¡rstnr-ments. During the course of the data collecüion, tlreresea¡ch nurses met for t half day every other week toindependently code videotaped patient visibs ald n'redica.lrecords from sites not participating in the larger study Theinterrater reliability of these measures anìong the eightresea¡ch nurses has been previously re¡roúedtald fotndto be good to excellent.

The research nuËes collected data on the content a¡rdcontext of the ouþatient visit, using the following mea-sures: (1) di¡eet observation of the patient visit, using amodified version of the Davis Obsewation Code (DOC)|";(2) adirect observation checklistof services delivered dur.ing the patient visiü (3) a patient exit questionnaire; (4)medical record review; (5) a practice environment check-list; (6) bi[ing data on Current Procedual Tbnninology(CFI) codesú and ICD-94M diagnoses''; (l) a physicianquestioruraire; and (8) etlnographic fi eldnotes.

Each physician rvas visited by a team of two researchnurses dudng 2 patient ca¡e observation days and 2 addi-tional da¡rs during which medical records of the previou+ly obsewed patients were abstracted. During the 2 days ofpatient care observation, one resea¡ch nurse accompaniedthe physician during all visits by consenting patients. Thisnurse recorded her di¡ect observation of the content of thevisit using the DOC a¡rd direct observation checklist. Theother resea¡ch muse obtained verbal informed consentfrom patients in the waiting room, and gave participatingpatients a questiormaire at the end of their visit

Multiple s&ate$es were used to minimize the possibiti-ty of a Hawthorne effec! that is, the chance that the presence of a mrseobsewer wot¡ld alter the phenomenaunder study. Physicians and office staff members weretold to follow thei¡ usual procedures. Tb avoid biasing

378 The Journal of Family Practice, Vol. 46, No. 6 (May), 1998

]LLUMINATING THE'BLACK BOX'

their behavior, physicians were informed that the studywould use multiple methods to examine the content of theambulaüory patient visit, but no specific hypotheses weresha¡ed with the physicians, ofñce staff, or patients. In addi-tion, the observation of consecutive patients made itimpossible for physicians to spend more time or providemore services than their usual rouùine, without severelycomprornising their abitity to slay on schedule. Theresearch nurses asked the physiciars and patients toignore them during the visit. T'hey observed from the leastobtrusive corner of the roonr, from a position that avoidedeye contact with either the physician or the patient. Sincethe presence of a nurse is a nonnal occutl€nce duringmany ouþatient visits to physicians, the vast m4jority ofpatients and physicians repor[ed that fhe presence of thenu¡se observer did not change thei¡ behavior dwing theobserved visils.

Specific patient data were collected using a patientexit questionnaire, which patients completed andreturned to the research nurse in the waiting roorn ormailed to the study research office in a confidential pre-paid envelope. Parents or guardians of children youngerthan 13 years of age were asked to complete the ques-tionnaire for their children. Patients aged 13 to 17 weregiven the option of completing the questionnai¡e them-selves or with help from a parent or guardian. Patientswere sent a reminder postcard within 1 week of theirvisit. Nonrespondents were sent a second questionnairewithin I month of their visit.

The practice environment checklist assessing multipleaspects of the practice organization was completed by theresea¡ch nurse teams on the basis of di¡ect observationand interviews wÍth key office informants, such as theofftce manager, during both the patient care observationand medical record review days. Billing data on theobserved visits were obtained from the responsible offrcepersonnel after the observation day. Ethnog¡aphÍc field-notes were based on brief "field joüüings,"'¿ and were dic-tated by the research nurses immediately after each visit tothe practice. T\¡¡o thousand pages of text were thus dict¿t-ed tc critique the study methods and to provide richerdescriptions ofthe va¡iables under study.

After the first ¡ound of data collection, in which eachphysician was visited once, the resea¡ch instruments we¡eslightly expanded based on the early etlurographic findingsand input from the entire team. Physician questiornaireswere distributed only after each ph5æician had completedthe second observation day to avoid biasing their behaviorduring the study.

llfn¿sunnsPractice cha¡acteristics were determined primariþ fromthe practice environment checklist. Data on the practicetype, location, personnel, and office operations wereobtained by the resea¡ch nurces from direct observationand key informant interviews. Physician cha¡acteristics

'were assessed by questioruraire. Patient characteristicswere measured with the patient exit questiomaire. In addi-tion, some patient characteristics were determined frommedical record review and direct obsewation, thus allow-ing a comparison of questionnaire responders and nonresponders. Information on patients' insurance status tvasobtained from billing data, and confirmed by patient que+tionnaire when possible.

Patient visits were characterized by multiple methods.The di¡ect observation checklist was used to measu¡e thereason for visit, the delivery of services during the visit,and whether a referral was made. Detailed data on preventive services deliverywere obtained, and wiilbe report-ed elsewhere. The medical record provided data onwhetler a drug was prescribed and whether tJre patient$'as a new or established patient. Established patientswere deftned as those who had been seen in the practiceat least once during the previous 3 years.'' The primary andsecondary diagnoses were obtained frorn billing data. Theprimary billing diagnosis was giouped into diagnosis clusterstr to provide infonnation on the most conìnÌon medicalproblems seen.

Finally, time use during patient visits was cha¡acteúzedusing a modihed version of the DOC to classifyvisit timeinto 20 different behavioral categories. The detailed def-initions of these behavioral categories have been previ-ously published.{0 The DOC has shown good interraterreliabiliff.?¡î For this study, the DOC was modified byeliminating the least common catÆgory reported in the ini-tial studies by Calïahan and Bar:takis.s ?lre category of"discussion of trea{¡llênt effe*tsn wål,r,eplaced with "negotiation,n defined aS "Blysiciari eüffnletìts or questionswhich faciìit¿te or invite patient participation in diagnosis,treaünent plaffúng, or problem solving." This modificationwas made to allow additional insight into this particularqualiff of clinician-patient communication.

In recording DOC data, the resea¡ch nurses noted asmany of the 20 behaviors as were observed duing a llsecond obsewation interval. A tape recorder with a¡ ear-phone prompted the research nurse to record these behav-iors during a lsecond recording interval, ând then toobserve for the next l$second interval, and so on. Foreach behavior, the mean number of intewals per visit andthe mean percentage of the total number of intervals pervisit were calculated, This information allows interpreta-tion of the percentage of visit time devoted to each behav-ior. The percentage of visíts for which each behavior wasobserved dwing at lea.st one interr¡al was also calculaf,ed,and the DOC was used to measure the length of the directphysician-patient contact time for each patient visii.

For the direct obsewation checklist the research nurseobserving the office visit checked a box for each sewicethat was performed or ordered during each physician-patient encounter. In addition, for some senrices, thereseaxch nu¡se indicafed whether the service had beenperformed in response to a patient's symptoms or to a

The Journal of Family Practice, Vol. 46, No. 5 ([,Iay), 1998 379

ILLUMINATING THE'BLACK BOX'

cluonic medical condition.Similarly, for the nredical record review, the resea¡ch

nurses inclicated whether particula¡ services were noted

on the chart for the obseled visit. Medical record dat¿were also collected on delivery of services during the past

year or other relevant tirne intervals. The medical recordwas also used to collect data on a number of other vari-ables, inclttding demographics, nrunber of chronic illnesses and medications, number of yeats as a patient of thepractice, and nunber of visits in the past year.

The patient exit questionnaile asked a wide vafiety ofquestions, including whether particular sen¿ices were prùvided during the observed office visit. Dernographic ques-

tions ascertained the patient's age, sex, race, educational

level, and nTarital status. Health stahr was tneasured with5 items (ü,=.81)'{J fr'om the Medical Outcomes Suwey(MOS) &item General Health Survey.il These ifems used a

5-point Likert-type scale to ask about global healttr status'

health limitations in everyday physical activities, emotion-

al problems, limit¿tions i¡r work because of physical oremotional problems, and bodily pain during the 4 weeks

before the visiL Patient satisfaction was assessed withmultipte meâsures, A singte item asked patients to rate the

degree to which their expectatiors for the visit were metusing a 5-point Likert-fype scale, Global satisfaction withthe visit was measured with the 9-item Vìsiting R-ating

Scale f¡om the MOS'5 (u=.88). T\vo subscales were also

created for the fou items assessing patient satisfactionwith the physician (a=.90) and the four items asse*sing

satisfaction with practice operatiors (u=.72).

The reason for the visit was measured with the tJæolcgy from the NAMCS'tr? and was obtained by direct obser-

vation, medical record review, and patient exit question-

naire. CPT codes were assigrred to each visit by theresearch nurses on tlte basis of direct observation a¡td

medical record review using established guidelines.s

A¡s¿r,vsps'Ihe representativeness of the physician sample was cal-

culated by comparing the demographics of participafingphysicians with those of members of the AmericanAcademy of Family Physicians.'z?

Several methods were used to assess the representa-

tiveness of the patient sample. First characteristics ofparticipating patients and visits were compaxed with sim-ilar dat¿ obtained from the NAMCS."P' Second, theresearch nurses recorded observable cha¡acteristics ofpatients who declined to participate, including any rea-

son that patients gave for declining. Third, a subsampleof 12 of the participating physicians reviewed the medicalrecords of their patients who declined participation. Foreach patient, the physician recorded the patient's demo-graphics a¡d number of years as apatient of the practice.The physicians also noted their belief about why thepatient declined to participate, according to the physi-

cian's lstowledge of the patient and the characteristics of

the patient's visit on the obsen'ation day' FinaJly, atllo¡ìg

patients who agreecl to have their outpatient visilsobserved, the chalacteristics of patients who retttrnecl

questionnaires were courparecl rvith ¡roureturnets, usiug

the obseryatiou and medical recorcl clata.

AnaJyses fol this descri¡:tive article involvecl calct¡la-

tion of frequencies, lìleans, standard <leviations, ancl

ranges, depencting on the type of t'ariable. For cottt¡rar'

isons of questionnaire respondets ancl nont'espouclets, I

tests were usecl for cotttinuotts variables, the Wìlcoxott

rank surn test for higlily skewecl ordi¡ral variables, ancl X!tests for categorical va¡iables.

ffi#TiffirTâbte I clepicæ characteústics of the 84 participating pì'ac-

tices. The mqioriüy wet'e siugle-s¡lecialty gror4r ¡:t'actices'with solo practices bcing the rle>.t nlost colìunolì t'ype'

Most were in subut'ban locat'iotls, u'itìr tuoderate l'epreselì-

tation of tural and urban settings. This com¡lales wi{'lt

national dat#? sllowing that 47Vo of futtily pltysicians prac-

tice in single-speciatty gl'oup or partnership settings, 357o

in solo practice, and24o/o in rural settings.

The tnost prevalent' personnel in these practices, afterphysiciarn, were clerical persornel, t¡luses' a¡lcl nteclical

assistants. An average of 2.7 nonclinicia¡s were present

for every clinician, but the ratio of clinicia¡rs to ¡ronclini-

cian staff metnbers varied widety. Tïventy-one perceut ofpractices had either a llurse practitioner or a physician

assistånt among their clinicians, alrd 37o of practices hacl

both physician assistants and nurse ¡rractitioners. Tl-re

roles ñlled by registered nurses, who worked i:. 607o ofpractices, included a variety of clinical and patient educa-

tion and communication tasks.Practices varied considerably in their offrce operations.

Slightly more than half of the practices offered sclteduled

evening or weekend hours. Patient phone calls were pri-

marily returned by nurses or medical assistants in mostpractices, with the physician being the primary percon to

rehnn c¿lls in orüy LIVo of practices. Use of different ty¡res

of reminder systems for patient recall and monitoringwere modestly prerralent. All practices had some type ofwritten pafient educational material available. Avariety ofancillary services were available in these practices, rang-

ing from ptrleboømy inBTVq to flexible sigmoidoscopy in557o, to x-ray facilities in 187o. Most practices expected

payment aJ the time of the patient visit, and the mqiority

did their own billing.Table 2 shows the characteristics of the 138 participat-

ing physicians, Physicians were demographically similar

to active practicing members of the American Academy ofFamily Physiciars (AAFP)" in age (AAFP mean=45 yeaxs)

and number of patients seen per week (AAFP mean=103)'

Our study sample represents recent demographic trends in

family physicians; participafing physicians were more lik+ly to be female (AAFP=Z|Vo) and residency-trained

380 The Journal of Family Praetice, Vol. 46, No' 5 (May)' 1998

TÂ8LE 1

Characteristics of 84 Study Practices

Characteristic Mean or % Bange

Practice type (%)Single-specíalÇ groupSoloMult¡specialty groupResidency training practiceClosed panel health maintenance organization

Practice !ocationSuburbanRuralUrban

PersonnelNumber of personnel

Physicians in the groupClericalMedical AssistantsBNsLPNsNurse practitionersPhysician AssistantsOther

Ratio of nonclinicians to clin¡ciansPractice employs nurse practil¡oners (% yes)Praclice employs physician assistant (% yes)Practice employs registered nurses (7o yes)Role of registered nurses (%)

Returning pat¡ent phone callsTriagePatient health educationGiving shotsHistory{âkingDiet counselingPrenatal teach¡ng

Office Operations (%)Weekend hoursEven¡ng hoursPrimary person to return patient phone calls'

RNMedical AssistantOtherPhysicianLPN

Reminder systemsTelephone recall systemChecklisis/flow chartsPatient rêminder cardsPrevention on problem listOtherPeriodic chart audit within practiceRisk factor chart stickersComputerized recall systemsComputerized provider reminders

Educational mater¡al availableln waiting room, front deskfn examination roomsln hallways

Types of educational mater¡al availablePamphletsPostersMdeosNone

Ancillary services in officePhlebotomyProcedure roomFlexible s¡gmoidoscopyLaboratoryColposcopyRadiograplryConsultants

Ancillary services in buildingBadiographyLaboratoryPhlebotomyConsuhants

BillingPayment expected at time of vis¡tBilling done outside of ofüce

'Total >100% because primary responsibility to return phone calls is shared between cate-gories of personnel ¡n some offices.

53.62S.8&36.02,4

60.221,717.9

3.6âA

2.01.4110.2u.¿0,52,7

11,911.960;5

ô5363.353.149.034.730.614,3

76.260.740.5

81.051.215.50.0

86.S70.254.823.824.217.913.1

36.935.728.627.4

77.47.1

11 ,271(0, 15)(0,7)(0, el{0, 12}(0, 6)(0,2)(0, 4)(.45, 9)

57.153.6

31.016.7'r0.7

10.7

61.927,422.621.48.37.16.06,03.6

(AAFP=73Vo). The mqjority of physiciansprovirled inpatient care (AAFP=97VI haveItospital privileges). Nearly all physicianscared for children ( Nry=9zqo). Familyphysicians in our sample were less likely toprovide prenatal care (34Vo) or deliverbabies (2lo/o) than a national sar.nple of farn-ily physicians, but were slightly more likelyto perform obstetrics tha¡r all family physi-ciãns in Ohio. Of all ÄAFP members, 317o

include obstetrics in sonre fonn in theirpractice, whereas onJy 17 7o of fantily physi-cians in Ohio practice obstetrics.'tr

Physicians described their primaryfocus as taking care of patient needs, withnìanaging chronic illness and providingpreventive services as secondary focusesof their practice energies. Most repõrtedbeing satisfied with their provision of out-patient care, with sonrewhat lower levelsof satisfaction reporùed for other aspectsof practice. The vast mqjority of physi-cians did not sntoke, although lSVo werefonner smokers.

Of the 4994 patients presenting for ca¡eby their family physicians during the 2observation days for each physician, t14ö4

(897o) agreed to have their visits observed.Eleven paLients (2Vo of nonparticipants)were not enrolled because they wereminors who did not have a påxent orguardian present to give verbal informedconsent, and 4 patients (lVo of nonparbici-pants) were not enrolled because languagebarriers inhibited informed consent.

llvelve participating physicians provid-ed information on tleir patients whodecl.ined to participate. Ttris subsample of64 patients was older than participatingpatients (P<.001), but similar in sex, race,and number of years as apatient. The physi-cian athibution of the patients'reasolìs fornonparticipation revealed patient concernsabout privacy as the most common reasion(397o), followed by arxiety (117o), embar-rassment (7Vo), gynecologic reason forvisit(77o), and shyness (6Vo);

Patient characùeristÍcs were simila¡ tocharacteristics of patients coming to seefamily physicians pafticipating in the1994 NAMCS,n in Ee (NAMCS=38 years),sex (N.{MCS=ó9Vo female), and race(NÁMCS=8E% white). Patients in our studywere slightly more likely to be est¿blishedpatients (NAMCS=88?o) and to h¿ve a marr-aged câre type of insulance

The Journal of Famiþ Practice, Vol. 46, No. 5 (May), 1998 381

ILLUM¡NATING THE'BI.ACK BOX'

TABLE 2

Characteristics of f28 Physicians Who Retumed Questionnaire

Charaeteristic % or Mean (SD)

Age {years)

Sex (7o male)'

Marital statusMarriedUnmarriedDivorced

Completed resídency training in family practice

No. of years in current practice

No. of patients seen per week in office settingfTotal no. of patient care hours per weekg

Perform prenatal care (Yo)

Deliver babies (7o)

Provide inpatient care (Yol

Provide care for children under 13 years of age (%)

Self-attribution of focus (1=very little, S=verY much)Taking care of patient needsDoing preventionManaging chronic illnessFamily as the unit of careHandling urgencles, emergenciesKeeping on scheduleBusiness and financial aspects of practiceCommunity / public health

Satisfaction (1 =ver! unsatisfied, s=very sat¡sfìed)Outpatient carelnpatient careManaging practiceMalpractice risks and claimsLeisure and family timeFeelings of control over practice environment

Physician smoking statusNever smokedQuit smokingCurrent smoker

43.1 (7.6)

73.2

BB.14,Q70

89.1

10.5 (7.8)

109.4 (45.5)

42.2 (10.5|

33.6

21.1

80.4

98.4

4.7 (0.6)4.2 (0.71

4.2 (0.71

3.7 (0.e)

3.6 (0.e)

3.5 (0.9)

2.8 (1.2)2.7 (1.1)

4.1 (0.e)

3.4 (1.0)

3.1 (1.0)3.1 (1.2)3.1 (1.'f )2.8 (1.1)

78.0't8.1

3.9

'Physician sex is the only variable based on tota¡ population of 1 3g physi-cians; all other data are from the 128 physicìans who returned the queb-tionnaires.fThis numbor excludes the 30 phys¡cians at residency training sites. Whenthese sites weÞ ¡ncluded, thê meân numb€r of patients seen-per weekwas 91.2, SD=52.7.+This number excludes the 30 physicians praci¡cing at residency trainingsites, When these sítes were included, the mean number of pai¡ent carehours was 36,8, SD=14.7.

(NAMCS=2|7o).Medical records were availâble for review for 4432 of

the MM observed visits (99.õZo). Patient exit question-nafues ìüere retumed by 3283 patients, far a74%o respotìserâte. As shown in'lÞble 3, patients who retumed question-naiÌes were more likely than nonreturners to be oldeq,female, white, ma¡ried, to have a greater number of chron-ic illnesses and a longer relationship with the practice, andto have Medica¡e or fee-for-service i¡sr¡rance. However,

the ntagrútude of these differences is small. Lt addition,smokers and patients presenti,r'rg for an acute ill.lless weresliglr$y less likely to retturr exit questioruraires.

The n4iority of patient visits in this sample were bywomen (62Vo). Established patients accounted far glVo afvisits. The average patient had been with the practice formore than 5 years and had visited the practice 4.3 timesin the past year, with an average of 2.3 additional visits toother physicia¡rs outside the practice during the pastyear. Patients had an averuge of 2.3 probleuìs on theirproblem list.

Visit characteristics are shown in Täble 4, The averagevisit duration was 10 ¡ninutes of direct physicial-patientcontact time. Most visits were for acute Íll¡ess or follow-up of an acute illness, with visits for chronic ilhress a¡rdwell care being the next most colnnon. Dmgs were pre-scribed during nearly two thirds of visits. This is comparedwitlr physician repo$ of prescribing a dmg during ?57o af.visits in the N.AMCS.:? Refen'als to arrother physician wel.emade during 7.60/o of patient visits (NÂMCS=4,6VI).".Patient satisfaction with their physician a.nd with the prac-tice was hþh, as was the degree to which patient expecba-lions for the visit were met.

Tabie 5 sÌlows the rnost comnìon diagnosis clusters forthe observed patient visits, atd compales these with theramk frequency of these clusters among a national sampleof family plrysicians from 1989-1990,5 The most cornmondiagnoses were hypettersion, upper respiratory infection,and general medic¡l examination. Sixtycne percent of vi*its were classihed in these top 25 diagnosis clusters.

Table 6 shows how time is spent during patient visits, asclassifi.ed into the 20 behavioral catÊgories of tlre n¡odifiedDOC. During an average lFsecond interval, 1.9 behaviorswere observed. The most cornrnon use of time involverlhisüory-taking, followed by plaruring treatment, physicalexamination, and health education, in that order. The thirdcolumn of Thble 6 shows the percentage of visits at wNcheach sf the 20 behaviors wa.s observed during at least onelFsecond observation interval. History-taking, plaruringtueafrnent physical examination, provision of feedback onfiadings, and health education occurred during at least907o of patient visits. Structuring the interaction, gatheringfamily information, patient questions, and chattingoccurred dwing more than two thirds of visits. Otherbehaviors, including the next most conìmon, preventiveservices delivery occurred during less than one third ofpatient visits.

ffiThe DOPC shrdy demonstrates the feasibility of carryingout a large multimethod observational shrdy in busy com-munity practice sites. The concurrent use of both quanti-tative and qualitative methodsq?ë holds the promise oftesting a priori hl4rotheses while generating new hypotheses from the study of actual prartices.r¡e The study con-

382 fhe Journal of Family Practice, Vol. 46, No. 5 (May), lg98

ILLUMINAtrING THE'B¡.ACK BOX'

furns, updates, and expandsñndings of previous reportsof the content of fanily prac-tice.'tH In addition, the directobservation data providenew insiglrts on time use clur-ing the patient visit. Forexam¡rle, the directlyobserved length of visit wasshorter than the l&minuteaverage length ofvisit report-ed by physicians in theNAMCS.T' The discrepancymost likely represents physi-cians in the NAMCS sampleestimating tot¿l visit-relatedtime, including time notspent in face-to-face contactwith the patient In contras!our direct obsewation proce.dure measured the time tllephysician spent in directpatient contact

Because of the intersivedata collection methodsinvolved, a re$onal sampleof physicians was the focr¡sof this study. The participat-ing RAPP members aredemographically similar tofamily physicians nationally,but represent recent trendstowa¡d increasing numbersof female and residency-trajned physiciaru practicingin group practice s€ttings.rr,otAt the tirne of the study, capi-tation was ra¡e in oru area;most managed care planspaid discounted fee'for-ser-vice, and managed careMedicare and Medicaid werenot prevalent.ð The percentage of physicians perfonningobstetric care is representative of local and re$onal rztes,and slightly lower than national rafes. These rates showthat local community need and attitudes,Gr as well as per-sonal and other facton,@ determine the scope of locaipractices, The findings also show a substantial minority offamily physicians perf'orming prenatal caxe, a service thathas been recently recommended as a strates/ for main-taining continuous, comprehensive care of ïyomen andinfants by family physicians who do not perform deliver-ies,* In addition, despite recent concems about the rise ofhospitalists,4$ the m4jority of physiciaru in our samplecontinue to provide continuity of care for thei¡ patientswhen theyare hospitatized. This ts similarto findings from

a recent national survey that showed a high level ofinvolvement of family physicians in hospital care.ú

The patient sample appeans representative of patientpopulations visiting family physicians. hn addition, a pre-vious study of patient visits to members of the NEONpractices who participâted in this study showed patientand visit cha¡acterisùics simila¡ to the NAMCS datas? Therearions for patient nonparticipation suggest that thesample may slightly underrepresent counseling and gmecologic visits, However, because of the high patient par-ticipation rate, tfre magnitude of this effect is likely to besmall. Our sampling of patients who came in for caredoes not allow us to a,ssess the frequency with which allpatients in a practice's partel seek care. Other research,

TABLE 3

Characteristics of Patient Population

Characteristic

Entire Sample(N=4454)

% or ltJlean (SD)

Patients t¡l,hoReturned a

Questionnaire(n=3283)

% or Mean (SD)

Age þars)Sex (7o female)

Race (7o nonwhite)

Marital status (% married)

Educational level attained {% >high school)

New vs established patient (% new)

Self-reported health statusOverall healÌh (1 =poor, 5=excellent)Everyday activities limited by health (1=extremely, s=none)Bothered by emotional problems (1=extremely, 5=non€)Amount of bodily pain (1=severe, 5=none)

41.4 (24,21

61.6

11.9

8.6

26.2 (7,s)

2.3 (2.51

1.7 12.2)

5.4 (5.5)

4.3 (2.7)

3.9 (2.6)

0.4 (1.'r)

43.e (23.7r

62.7.

8.7-

54.4

42.O

7.1'

3.4 (1.0)

4,0 {1,2)3.9 (1.1)

3.6 (1.0)

3.e (1.1)

3.8 (0,8)

26.4 (7.31

2.5 (2.6r

1.812.2)

5.6 (5.5r

4.4 (2.6)

4.0 (2.6r

0.4 (1.1).

2.3 (2.9)

2,s {1.5)

25,1.5.1

37.022.8ú. I

6,8

Difficulty doing daily work because of ailments (1=severe, 5=none)Summary

Body mass index.- (kg/m2)

No. of problems on problem list

No. of medications on medication list

No. of years with practice

No. of visits in past year to practice

No. of visits in past year to observed physícian-.

No, of visits with a nurse in past year

No. of visits to physicians outside prâctìce this year

No, of physicians seen in past year

lnsuranceMedicareMedicaidManaged careFee for serviceOlher, undeterm¡nableNone

22.76,7

36.019.97.37.3

' Patients who returned questionnairc differ from those that did not at P <.05.'. Round 2 only.

The Journal of Family Practice, Vol. 46, No. 5 (May), 1998 383

ILLUMINATING THE'BL.ACK BOX'

TABLE 4

Visit Characteristics {N=4454)

Characteristic % or Mean (SD)

Length of visit (min)

Beason lor visit (assessed by nurse observer)Acute, problem

Acute, follow-upChronic, routineChronic, flare-upWeli adulVchild exam¡nat¡onPrenatal careFostnata' care (n=2)

Counseling/acivicelmmunizatio¡rAdministrative purposeOther

Drug prescr¡bed

Refenalsl-o anolher physicianTo a nonphysician in officeTo a nonphysician out of of{ice

Patient satisfaction (1=poor, 5=exceltent)Global measure of satisfaction'Expectations for visit metSatisfaction with physicianfSatisfaction with practice operat¡onsT

10,0 (5.8)

40.317.716.96.4

12.01.1

0.01.4

0.41.02.2

62.2

7.61.62.4

4.s (0.7)

4,4 (0.8)

4.4 (0.71

4.1 (0.8)

'Global satisfaction was measr:red with MOS 9-item Visit RatingScala*t Four item subscale of MOS f-itern Visit ttating Scale,

however, indicates that the avel?ge American sees aphysician 2.8 times per year, witlì 0.8 of those visits beingto family physicia¡s.r

This article's br-ief descriptions of pracüice characteris-tics show a variety of office structures, persormel, andoperatiors. This diversity of approaches indicates individ-ual creativity and adaptation to the unique configurationsof each setting.ß Recent trends toward larger practicesßand centralized management of practices?o are likely toenhance the use of uniform operational systems, such ar¡

flowsheets, self-audits, and computerized reminder sy+tems, that were used at low rates by practices in our stud¡rHoweve4 centralization of managenrent has the potentialto diminish tlte diverse approaches that practices havedeveloped to meet the needs of the particular cliniciars,staff members, and patient populations that they serve.Ongoing resea¡ch is beginning to provide important newinformation on the core processes of family practices thafare offered. by these varied. approas¡es.3aÉ¡.oa"ta

Despite the relatively sm¿ll nu¡nber of perconnel in themqiority of family practices in the study, most provided awide range of ancillary services in the ofñce or the build-

ing. The availability of flexible sigmoidoscopy in nrorethan half the ofñces shows the potential of family prac-tices to t¡se this tool for colorectal ca¡rcer screerring. Thesedat¿ also show a ceiling for efforts to increase its use.Practices that do not have tJre equipment or personneltrained in its use will require extensive traiuing ol will haveto make plans to refer patients, if this procedule is to bewidely used. 1'

Physicians reported that their mqjor focus is caring forpatient needs. Ïte direct ol¡selation data sirow that, for587o of these visits, these patient ¡reeds vvere for acrite iìl-llesses. The low priority given to conuuunity ald ¡tr-rblichealth shows the difficulty of cleveloping a largel poprlla-tion or conrnrunity-oriented prÍmary cale focns,;¡ a-nd

emphasizes the focus of crulent rnedical pmctice ol.r luatì-agirÌg tlìe inunediate demands of acutely ill patients whoconre tluough the door'.tr

The delivery of pleventive sewices, rccoglütion a-nd

treafinent of nrental health problerrs, and managenrent ofchronic drseases present particu.lar challenges, siuce nrostpractices and their operational systenrs are prinrarily setup to care for acute illnesses. Tl'ends toward i-ncleasingcapitation may theoretically inclease the relative value ofprevention and ch¡onic ill:ress ca¡e in prinrary care prac-tice. However, managed care financial caÌve{uts for men-tal health and chronic disease rnay have the opposite effectof devaluing provision of these services within tJ-te contextof an ongoing relationshþ with a family physicial. Thefact that patients in our shrdy saw their physician an aver-age of 4 times a year shows the potential of a longitudinalrelationship between the patient and family physician todeliver ¿ wide range of services over time.

Some changes in approach wül be required if familypractices are to achieve their true potential for adclressingthe entire range of needs thaf patients bring to them. Thekrstitute of Medicine has recognized that despite evidenceabout the abiliff of primary care to provide high,qualitycare at low cost, an expanded vjsion ofthe scope ofpri-mary care practice could result in an even greater impacton the health of Americans.T6 Schergerñ has suggested thatthe optimal role for a family physician may not be as aworkhorse who sees large numbers of patients per day, butas a personal physician?' who r¡ses ongoing relationshipswith patients, families, and communities to serve as ahealth care mânager, providing direct ca¡e for a smallernumber of patients each day during critical events, andorchestrating acute caxe by nonphysician clinicians ardspecialist care of certain problems.

The discrepancy between the percentage of visits forwell care as measrued by direct obsewation and bilingdat¿ shows the additional insights that can be gained fromviewing the same phenomena using multiple methods. Ourdirect observation that well cåxe was the mqior reason forvisit in LZVo ofpaltentvisits corresponds to rates reportedby Luclcnarur and Melvilleæ in a national swvey of familypþsiciars. Yet the 67o of visits in our sample that were

384 The Joumal of Family Pr-actice, Vol, 46, No. 5 (May), 1998

ILLUMINATING THE 'BI.-ACK BOX'

classifiecl as "general medicalexa¡rinations" on the basis ofbilling data are also similar toother national clata.r'This dis-parity between reason forvisit assessed by clirectobservation a¡cl billing datamay be a renìnant of the lackof reimbursement for wellca¡e in traditional indemnityhealth i¡lsurance policies.Since at leas[ one potentiallybillable cliagnosis was uncov-ered during a large percent-age of patient visits initiatedfor welì care, nrary physi-cia¡rs have developed thehabit of using these diag-noses in billing for approxi-mately half of ttrose visits.Recent increases in reim-bursenrent for preventivecare"' and the developmentof specific CPI' codes forsuch care*' may begin to alterphysician billing behaviorover tinre. Nonetheless, it isimportant to realize thatsh¡dies that report the per-centage of visits for well careon the basis of billing datamay seriously underreportthe prevalence of well carevisits.

Patient problenË were handled without referral duringthe large majority of patient visits. Only 7.67o of visitsresulted in a referral to another physician, which is simila¡to the ñndings of other studies.wr The slightly higher rateof referral in this study compared with NAI\{CS may represent the renewal of previous referrals captured by ourdi¡ect observation methods and may not have been report-ed in NAl,f CS .In2.4Vo of patient visÍts, a referral was madeto an outside nonphysician. Tlris shows that family physi-cians exhibit both the comprehensiveness and coordina-tion of ca¡e atfributes of prímary carezby managing thevast mqiority of patient problems themselves and selec-tively refering to other health ca¡e professionals whenindicated by the patient's problem and other factors.

Cornistent with other studies,wë patients in otu sam-ple reported a high degree of satisfaction with their ph¡æi-cian, and reported that their expectatiors were met to ahigh degree during the vast m4iority of visits. Satisfactionwith ofñee operations wa.s also high, but less so than withthe clinical care. Moderate rafes of physician satisfacüiona¡e similar to findings of a recent study of a ha¡rsitionalhealth care market.qe

That nearly 40Vo of patient visits were not classifiedinto the top 25 diagnosis clusters shows the wide variety ofproblems add¡essed by family pþsiciaru. Differences inthe rank order of other diagnosis clusters in the nationalsample and ou¡ sanple may represent temporal ürendsbetween i989-1990 and 1994-1996 or differences in diseasefrequency or diagnosis billing pracùices between the twosamples.

It is tempting to speculate about the reasorìs for higherrates of respiraiory illnesses, musculoskeletal disease,skin infectiors, abdominal pain, headaches, thyroid disease, and peptic dísease seen in our sample of visits, as

compared with a national sample 5 years ea¡lier. These dif-ferences may ¡epresent temporal hends in disease frequencf environmental influences, differences in patientpopulatiors, re$onal variation in diagnostic practices, andchance va¡iatiora.

The time use data represent the ñrst broad-scale pic-ture of the content of the physician-patient interaction dur-ing a large number of visits to physicians in communitypractices. The DOC data show that the patient hisüoryincluding the a.ssassment of family information, represents

TÄBLE 5

The 25 Most Freguent Diagnosis Clusters Among 4454 Patient Visits

Rank Díagnosis Cluster

1 989.90Frequency % of Total NAMCS*of Visits Visits Cluster Rank

1

a

.t45

7Õ

Õ

'f0

11

12131415

lô17{olo2021

¿¿

232425

l-lyperlensionAcute upper respiratory ¡nfectionGeneral medical exanlirìationSinustitis (acute and chronic)Acute lower respiratory infectionOtitis media (acute and chronic)Depression, anxietyDiahetes mellitusAcute sprains and strainsDegenerative joint diseaselschernic heart diseaseAsthmaLow back pain diseases and syndromesLacerations, contusionsFibrositis, myalgia, arlhralgiaNonfungal infections ol skinHeadachesAbdominal pain (excluding pelvic pain)

Bursitis, synovitis, tenosynovitisChronic rhinitisPregnancy careEmphysema, chroníc lrronchitisïlryroicJ diseasesUrinary tract infectionPeptic diseases

3533022611921681651631581r3B2

666564

6261

606059555450444240a7

7.96.85.S4.3J,O

3.73.73.52.51,81.51.51.41.41.41.31.3'1.3

1.21.21 .'1

1.00.90.90.8

r6

1B

10

1ç

4

2A

321

1Q

6Ë

fì

I7

' NAMCS {National Arnbulatory Medical Care Survey) c,ata clustered try: Rosenblatt BA, Hart GL, Gamliel S,

Goldslein B, Mcolendon BJ. ldentiþing pr¡mary care disciplines by ânâlyzing the diagnost¡c content of ambu-lalory care. J Am Board Fam Pract 1995; 203:1-20'

The Journal of Family Practice, Vol. 46, No. 5 (May), 1998 385

ILLUMINATING THE'BI.ACK BOX'

TABLE 6

How Time Was Spent During Patient Visits (N=4401)

Davis Obsenration Code Category

Mean No,

of l5-Secondlntervals

Mean%of %VisitsTotalTime with One orlnteruals* More lnteruafst

History-takíng

Planning treatment

Physicat examinatíon

Health education

Feedback on evaluation results

Family inlormation

Chatting

Structuring the interaction

Patient questions

Preventive seruices

Procedures

Nutrition advice

Counseling

Exercise advice

Compliance assessment

Smoking behavior assessment or advice

Assessing patient's health knowledge

Health promotion

Negotiation

Substance use assessment or advice

16.8

9.1

6.4

5.9

3,8

ó.¿

2.2

2.2

2,O

1,0

'1 .1

0.7

0,6

0.5

o.4

0.4

0.4

0.4

0.3

0.2

55.9

32"0

22.9

19.4

13.9

10.1

7,8

7.8

6.8

3.0

2.7

2.1

l.t

1.5

1.3

1.3

1,2

1.2

1,1

0.5

100

oo

94

90

92

73

69

BO

71

ùù

o

26

21

¿Õ

la

24

'18

21

ô

'Total > 1 00% because more than one behavior could be coded in each intervâ|,

T Davis Observation Code data were not obtâined on 53 vis¡ts'

included in onlY selectedpatient e ncounte rc. Thtat 2It/o

of patient visits involvedsome degree of negotiationshows evidence of a PartÌci-patory sffle in so¡ne i¡rterac-

tions befween Patient aldfamily physicians. Tl.tis palticipatory style has beenfound to be mor'e comtìlonalnûng physiciarrs with Pri-maxy cat'e training,'r and is

associatecl with the durationof the patient-PhYsician rela-

tionship and witlt patient sat-

isfaction.r¿Fanrily physiciatrs have

rnultiple brief contacts witlrpatients, with a great deal ofdemand placecl on diaguos-

ing and treating acute conl-plaints and managing chrotúcillness. In the ctutent, healthcare environttent, thisrequires clinicians Lo be veryselecüive in thei¡ allofrneut oftime to other domains ofcare, such as counseling, Preventive services, and healtlipromotion. A fundamentalchange in the operat'iotralstructure of most Practicesmay be needed if familYphysicians are to focus less

elïort on acute care a¡d nroreeffort on chronic diseasemanagement, Prevention,ment¿I health, and poPula-

tion medicine.ñ Because ofits generalist focus and

the mqjor tool of the practicing clinician. Ttre value of themedical history ha.s been espoused by clinician-teachersfor years.qæ Physical examination is the next most com-mon information gathering æchnique used by family ph¡rsi-

cia¡rs. It has been shown that tn 567o of ouþatient medicalvisits â diagnosis is estabüshed after history-taking, and in73Vo after hisüory-taking a¡rd physical examination.m Thepercentages are likeþ to be even higher for family pþsi-cians who lcrow their patients over time.

The nine most common behaviors occur during morethan two thirds of patient visits, and may be consideredcore activities. Ttrese behaviors involve a mixh¡¡e of infor-mation gathering and information sharing by the physician, as well a.s treatrnent of illness. Other behaviorsassessed by the DOC occu¡red during a minority of patientvisits, and appear to represent discretionary behaviors

patient<entered approach,mfamily practice is likely to be

exEemely robust in its ability to respond to changing

opporhrnities to meet the needs of patients and the health

care system.@ The challenge is to remain üue to the discipline's core values,ffi while adaptin$ to a changing envi-

ronr{rent,

@Many aspects of family practice remain in a black box' Ourresearch used a muttimethod approach including direct

obsewation, patient and physiciær report, medical record

review, and billing datato light several corners of that box'

The findings demonstnate the complexity of farnily prac-

tice on multiple levels, and illustrate the competingdemands of meeting a large pot€ntial agenda of patient

386 The Journal of Famüy Practice, Vol. 46, No. 5 (May), 1998

ILLUMINATING THE 'BI.ACK BOX'

needs duling visits that last an average of 10 minutes.4The diversif¡t of patient nee<Is and practice approaches

represented in family practice shows the need for a broadperspective on efforts to change practices, since a narrowfocus cor¡Id have uni¡rtended effects on other aspects ofpatient ca¡e.'o Adclitional analyses of dat¿ frorn this studyand otl-rers will be needed to further urderst¿¡rd the coreprocesses and struchrres of farnily practice, to assess thei¡effect on inrportå¡t patient outcomes, and to uncoveropporlunities for enhancing the effectiveness of familypractice.

ÄCKNOWLEDGMENTSThis research was srrpport{çl try gr¡rnîs fro¡¡t the NatìsnalCa¡cer hstitute (lROl Cå,ü08ßâ and 2BOl ü4,608ü2J,*nd byRobert lVood Johnson Êe.ne¡,*lls{ Phy,:ician l¡acul{v $:elrclarAwa¡ds to Drs. Stange arrcl.Iadtr.

The authors arf $rrfsful to tlre plrysicitur rnenìtrûrs of tlìeResearch ,Associâlirn *f Practlcirrg Plrysiciaru: (ILÀ.IãP)'and tof.he offrce staffs nnd pâtienls wiülout whose palr.lcipation thisstudy would not luvc been possible,

?ha p*rricipa[ing RÅ.PP physicians wcre: I)olrald Ádanrs,MD, Stanley A¡tdirson, HSr Ëët!,y Saldcll4 Ir{Þ,* üle¡r:rllal't¡etl" MÐ, Kentlrick B*sh$r, MÞ, J.ames B¡y, MI), Tl*tlranBeacl¡y, l'tD, Bri¿n B.ea¡rl, MD, Plúl¡p Bemarcl, MÞ, Pauì:Bimey.It'lü, Klnrlt¿ll Bixerrstine, $lD,T Êobert Blanklield, l{I}, llatr'rciaI}lochorvlak, MD, l{enry Bloom, MÐ, RoLrert l3olstet IifD,l{arold Bowersox, DO, MS, l{en Bråman, DO,t Kenneth tulen,MD,* Briau Cain, MD, Robert üaln, þ1Þ, Philifi Oamvell¿1, lvlÞ,Sr¡k Çhoi, trfD, ¡llrtlrony çÐstâ, MÐr* Nichqtâ.s Ðavjs, ilf&85,Pere¡'ÐeGolia, MÐ, J+nres Ðrrvar, MF,$Joh¡r Ði$abato, MÞ,*V.Iùlwar¿l Ðuïhs¡u, M'I),.Itcderie,k Ðunlea, MÐ. *llrrJJs¡rc Elam,MfJ, üven fimety, MÐ, Rob*rt$vans, Dü, ÐinahFedyrìa, MD,+Matther'¿,Fjnnertn, Mû, N¿rey 3lì*kingeç MD.* Peter f:ianklin-MÐ, Åndrew f"ralko, L{I), Kenrteth fùisoi lúÐ, Micltael Êsr,{r.ItlD, Karl Gefzinger, MD, James Gibbs, IvfD, Valerie Çilchrist,MD,* Rol,¡ert &illette, MD,i Kcnnetlr Goodrnan, i'll), David(lrayson, $10, BarbalaCuånieri, MD,+ Gwen Haas, MI), lvliehaelfTackelt, tr{D, ì{ariha Hackett, MÐ, Jan'rs ilcdin, lr,lD, tiiclrartlHirre*, MD,+ Charles Hugrr.*, ÐO, ÂnW Joliff, MD, Jeffrcy Kase,MD, Patricia Kellner, ÎdD, Philip Kennedy, MD, ,\. Gus Kious,MD, r{lla Kfuxclr, il{D, ?lrornas Xlosierman, MÞ, ïvlark l(omar,MI), ¿\lç¡¡ Kovanko, ü0, Richard Krqiee, MD, lìichard ltratchc,MD, fbank Kraufter, MÐ,* Adarsh Krishen, MÐ,* tlichardKuec¡a, ùTõ, Ðavid 1*s\ MI), $a Koo I¿e¡ MD,'fhomas,Lchnçr,ItIÐ, Conr*d l,intlç{, Ir{Ð, Ðorothyanu tinde*, i{Ð, Ma¡tiuLoflus, ï'lD, thsrles Maetaltun, MÐ, ,ëonstiinee Magqülias,lvl}, Janct lVïnrnqjon,, DÕ,* Dannib McGlusÏ<ey¡ MÞ, K, nMctormick, MD, Michael McGrady, lvlD, Susan Mercer¡ MD,rThomas ïlêttëe, htD, ÅIbert Miller, MD, John Miller, MD, ItiaNassi$ lvfD, Ihonlng Nnu¡noff, MD, Carob'nNemec, il{D, L$ur?Iæeson Novak, MÐ,+.å,llisnn Oprandl, MÐ,* Mugarst Parøner¡MD,* J, Payton, DÕ, John Ffrilfer, MÐ, Philomena Pi¡pzzi, MD,Mark Pluskota, DO, Itichgrd P.res*lsr, MD, MiËlrr1tl Rabovslry.MÐ, Elizabetlr Rana.sing¡he, l{û, 1ll¡nothy Ësed, }il}, ClåreReesey, MD,*Ann Reichsmm, MD, Märk Roininga, MD;* DnnielIìeynolds, ilfD, Randatl Richard, MD,+ Elisabcth Bigllteq MD,Mark Rood, MD, Pamela Rucid, Ili[D, Larry Sandeq MD, CaroteSâvan, MD, Jîm Sdrultn, MD,* Ronalcl Scolt, MD,* RobertSinsheimer, MD, George Smirnoff, MD, Steven, $mitlr, MD,*WillÌåm Smucker, MD,* Komclia Soþmos, MD, Michael Soroka"DÕ, Janicr Spalding, lvfD,r Mark ÐpeclmÐn, i!{t,,Gary $t¿blcr,DÕ,,lrure Stover, MÐ,* ûetrge Stfstiûff, MD, Drrniel Srveeney¡MÐ, Jay Taylor, MÐ, itliaabctlt,llhrlett l{,D, Jtmes Iìuùett, Mü,Duane Wages, IVlÐ, Sâtesh fVrybay, MÞ, Ðevid wtrkulcllik,MD,* lleatlter lVays; MD, Richard Wsinberges À{D, Mn¡lenel9einstein, lvlü, Judlth tffeiss, MQ Davld weldy MD, PhD,Edrvatd White, MD, Robsrt $thiteholrså; MD, /uclúe Wlkinson,MÐ, Jerome llilllams, À,1D, Jay Vfilllamåon, MÐ,* lawrence\Vilso¡L,MD, arrd Mus¿y l{inchell, MÐ,

The Research ,{ssociation of Practicing Physicians Boald ofDirectors contributed greatly to the planniflg, inÌplenÌenlationånd interprelåtion of this study, and provitled additional datafor use in assessing patient nonresponse bias: KinrballBixeustine, MÞ,i llobc* ßlankfield. il{D, l{e¡r¡y lJloo¡lt, Mt),Yaltrk Êilchrist, ltlll), ürven l{a*s, MI}, Itatricia Kellnr:r, À'lü, 5trXoo læe, MË, eor"rrad Lildes, Mü, Dennis McCluskey, MÐ,Tlrolllas lr{ettce, tr{D. åll¡elt Miller, ll{D, John Pfeiffer, Ìr{ll,lt:llchral llabov$lcy" !1,Þ, Tirn Recrl, MD, ¡rrd Ârclrie ll¡ilkjttst¡¡r.tì{0.

The Research Nurses for this study made major contribu-tions to the develoment of the resea¡ch methods and instru-nrents, collected high quality data" devel<rped imporlant rela-tionships with the participating practices, and proved tliat col-lection of high quality multimethod quantitative and qualitativedata is possible in busy comnìunity family practices. Theseresearch nurses were: Lisa Ballou, RN, Cathy Conigan, RN,Luzmaria Jaén, RN, Sherry PaÞke, RN, FYan Powers, RN, KatltySchneeberger, RN, Reily Warner, RN, and Sue Zronek, RN.

Gwen Hass, MD, and Patricia Kellner, MD, provided helpfuisuggesùions on an earlier draft of this manuscript.

+ Denotes physicians who a¡e also members of the Northeast {ilìioNetwork (NEON).t We nrourn the r.rntimely deatli of Kimball Bixenstine, MD, rvho rvas anexcellen[ family physician and a va]ued ¡ueniber of the R.APP lJoa¡d ofDirectors.

REFERENCESl. Rosenblatt RA. Specialists or generalists. On whorn should

we ba^se the American health care system? JAM,{ 1992;267:L665-7.

2. Starfield B, Simpson L. Prirnary care as part of US healthservices refomr. JAMA 1993; 24:3136-9.

3, Geyman JR Hâ¡t LG. FamiÌy practice and the heafth caresystem. Prirnary care at a crossroads: progressl problems,and future prqiections. JAm Boarcl Fam Pract 1994; 7:60-70.

4. Stange KC. Primary care research: barriers and opportuni-ties. J Fam Pract 1996; 42:t923.

5. Gerbert B, Stone G, Stulbarg M, Gullion DS, Greenfield S.

Agreement among physician assessment methods: sea¡ch-ing for the truth among fatlible methods. Med Care igSB;26:619f.2.

6. Montaño DE, Phillips WR, Cancer screening by primarycare physicians: a comparison ofrates obtained from physi-cian self-repor! patient survey, a¡rd chart audÍt. .A'm J PublicHealth 1995; 85:?9S800.

7. Stånge K0,Zyzatrski SJ, Smith T$ et a¡. How vaüd are med-ical records and patient questiorunai¡es for physician profil-ing anct health services research?.{ comparison with tä¡ectobservation of patient visits. Med Ca¡e 1998. In press.

8. Pinsent R.I. Ttre primary obsewer. Ecol Dis 1982; 1:275-9.

9. Cronrbie DL, Pinsent R.f, l,anbert PM, Birch D. Comparsonof the lìrst and second national morbidity suweys. J RoyalColl Gen Pract 1975; 2õ:8744.

10. Radford JG. Morl¡idity recordings in I year of general prac-tice, paxt 1. ^Ann Gen hacü 1963; 8:1347.

11. Lamberts H, Meads S, Wood M, Results of the internationalfield trial with the Reason for Encor¡nter Cla,ssi-ftcation. SozPraventivmed 1985; 30:80-?.

12. L,amberts H, Wood M, Hofmans-Okkes IM. Intetrational pú-mary care classiflcatiors: the effect of fifteen years of evo-lution. Fam Pract 1992; 9:33G9.

13. Lamberts H, Wood M, Hofmars-Okkes L Episode of ca¡e: acore concept in family pracüice. J Fam Pract 1996; 42:i61-9.

14. Marsland D$ Wood M, Mayo E .4, data bank for patienteare, curriculum, and research in farníly practice: 626,196patientproblems. J Fam Pract 1976; 3:2S8.

15, Ma¡sla¡rd DW, lfood M, Mayo E Content of famiþ praetice.Part I. Rånk order of diagnoses by frequeney, Part lI.Diagnoses by disease cstegory and agelsex diskibution. J

The Joumal of Farniþ Practice, Vol. 46, No. 5 (May), 1998 387

]LLUMINATING THE 'BLACK BOX'

Fam Pr¿cl 1976; lì:37{7.16. StervarL WL. Clinic¿l im¡rlications of the Virginia Stu<ly. J

Faur Pract 1976; 3:29-32.

17. Horlgkirr K. Educational inrplications of the Viryiuia Study..l Fam Pract 1976; 3:33-4.

18. McWrinney IR. Researcl¡ implicatious of the Virginia Study.J Fanr Pracl. 1976; 3:35-6,

19. Geyrnan JP. 'Ibward tlte clefinition of family plactice-aqr¡antuuìjunìp. J Fam Plact 1976; 3:23-

20. Rosenblatt RA, Çherkin DC, Sehneeweiss R, et al. 'llteseructule arrcl co¡rtenl of fanrily ¡Iactice: cuffent stâtus arì(lfutnre trencls. J Fanr Pract i982; 15:681-722.

21. Sier+'art !VL. Clinical inrplications of tlre rlational study ofthe content of fäntily practice. J Fanr P¡act L982; 15:723-5.

22. Rakel RB. Ðdrrcationâl irÌpl¡c¿ìlions of lhe national study oftlle content offanrily practice. J Fam Plact 1982; 15:725-Ð.

23. lvletlalie JH, Z,yzanski SI. Researclr implications of thenational study of the content of fanrily practice. J Fanr Pract1982; 15:7304.

24. Glahaut R. Policy iurplications of the national stutly of tlteconte¡lt of fa¡ìrily practÍce. J Fam Pract 1982; 151735-7.

25. Geynran JP. The content of faurily practice: a laucln¡¿rkluatiorìâl stucly. J lranr Pract 1982; 15:679-80.

26. Schneweiss R, Rosenblatt RA, Cherkin DC, Kirkrvooct CR,Hart G. Diagnosis clusters: a new tool for analyzing the con-teni ofambula[ory tredical cale. IVIed Care 1983; 2L:105-22.

27. Ämerican Acaderny of Fanrily Physicians. 1996 facts aboutfamily practice. Ka¡rsas City, Mo: American Acatlemy ofFamily Physicians, 1996.

28. Schappert SM. National Änbulatory Medical Care Survey:1994 sunrrnary. Vitål and Ilealth Statistics. Rockville, Md:National Center for Health St¿tistics 1996; 273;1-20.

29. Rosenblatt RA, Hart GL, Ganrliel S, Goldstein B, McClenclon&I. Icientifying primary cãre disciplines by anaþing thediagnostic content of ambulatory ca¡e. J A¡l Boarel FamPract 1995; 8:3446.

30. Collins KS, Schoen C, Sandlran DR. the Com¡ronwealtJrFurrd Survey of physician experiences with managed care.New York, NY; The Comrnonwealtl.r Fbnd, 1997.

31. Millenson ML. Demanding rnedical excellence. Doctors andaccountability in the information age. Cltícago, Ill: TheUniversity of Chicago Press, 1997.

32. Flocke Sd SÞnge KC, Z,yzanski SI. The impact of irlsurancetype and forced diseontinuity on the delivery of primarycare. J Fam Pract 1997; 4l::L29-85.

33. Emanuel &1, Brett AS. Managed cornpetition and thepatient-physician relationship. JAMA 1995; 273:323-9.

34. Entanuel EI, Dubler NN. Preserving the physicizu.r-patientrelationship in the era of managed ca¡e. N Engl J Med 1993;329:879-82

35. Kletke PR, Emrnons DV/, Gillis I(D. Current trends in physicians'practice arrangements: from owners to employees.J,{MA 1996; 276:555-60.

36, Fra¡ne PS. Process instead of prayen moving toward activernanagement of patient care. J.{m Board Fam Pract 1908;tL:77-8.

37. Chassin Mlì. Quality of he¿lth care. Part 3: improving thequality of care. N Engl J Med 1996; 335:1060-3.

38. McFa¡land BH. Cost+ffectiveness considerâtions for man-aged care systemsr treating depression in primary care. AmJ Med 1994; 97:475-575.

39, McKenney JM, Kinosian B. Econonric benefits of aggressivelipid lowering: a managed care perspective. Am J ManagedUare 1998; 4:6ú74.

40. Evidence-Based Medicine Working Group, Ðvidence-basedmedicine. A new approaeh to teaching the practice of med-icine. ù\MA 1992; 268:24205.

41. Ge¡nnan JP. Evidence-based medicine in primary care: anoverview. J Áln Board Fam Pract 1998; l1:4G56.

388 Tire Journal of Family Practice, Vol. 46, No. 5 (May), l99B

42. Greco PJ, EisentrergJùl, Chnngirtg pltysicians'practicc.s. NEngl J Med 1993; 329:127 L-4.

43. Davis DA, Thomso¡r MA, Oxma¡r ÂD, I{artes B. Cltartgirrgphysician perfoLntarrce: a systeltìatic review ol'thc t'ffect ofcontinuing me<lical crlucation st¡'âtegies. J.A.ítlA 100'':r274:700-5.

44" Jaén CR, Stange KC, Nutting P. The eonrpetirtg dentand-s o[primary care: A nroclel for the delivery of clinic¡l I)te!'crìli\'('sen'ice. J llun Pracf. I004;38:106-?1.

45. Stange KC, Zyzanski SJ. hrteglatirtg qtl¿ultitativc irld quzrli-tatir¡e research ¡netlrods. F¿nt lt'led 1989; 22: lSl'ì-r'i.

46. Stange I(C, Miller WL, Clal:tree BF-, O'Connor IU. ZyzanskiSI. I\{ultimethod rese¿rrch: apptoaches for itttegt'afing qttali-tative arrcl quâutitative nìetlìo(ls. J Gerr hfel'n lrled 1$9'l:9:278-82.

47. Crabtree 8ff, Miller \{¡L. Doing qualitative t'eseat'ch.Newbury Park, Calif: Sage Publications, 1902.

48. Gilchrist VJ, Miller RS, Gillanders WR, ef al. Does f¡nril.vplactice at resiclency teaching sites leflect eollrlìlÌltìity l)r¿ìc-tice? J Fanr Prac0 1993; :17:555-63.

49. Callahan E"l, Bertakis KD. Developnrent antl valirlatir¡rt olthe Davis Obsewation Code. F-ant Merl 1091; 23:19-24.

50. Kirschner CG, Burkett RC, Coy JA, et al. Plrysicinns'Current Procedt¡ral Telnrinologl: CPT '05. Clticago, Ill:fuirerjcar Medican Associ¿tiou, 1994.

5i. St. Antlìony's ICD-9-CM: corie book for physician payrìtelìt,1994 softl¡ound edition. Aiexandria, \¡a: St. .{uthonyPublishing, Inc. 1994; 533;410.

52. Bogdewic SP. Participant obselation. In: Clabtree BllMiller WL, eds. Doing qualitative research: utr.rlti¡tle strate-gies. Newbury Park, Calif: Sage Publications, 1992.

53. Flocke SA. Measuriug attributes of pliutaty¡at'e: develo¡r-ment of a new instntnreut. J FaItr Pract 199ï 45:64-?4-

54. Ware J, Nelson E, Sherbourte C, Stewalt A. Plelirninaratests of a &Item general healtli survey: a patient a¡:¡rlica-tion. In: lVa¡e ASI, ecl. Measuring functionittg attcl well-being. fJuriranr, NC: Duke lJniversity Press, 1992; 291-30?.

55. Rubin HR, Gandek B, Rogers WH, et al. Patient's ratings ofoutpatient visits in different practice settirgs. JAM,A' 1993;270:83Þ40.

56. Schneider D, Appleton L, Mclemore T. A reasol for visitclassification for ambulatory care. Vital a¡rd l{ealtltStatistics. Hyattsville, Md: National Center for HealthStatistics, 1979; 2: t-l l.

57. A¡nerican Academy of !'arnily Pirysiciatts. 1994 facts atioutfamily practice. Kansas City, Mo: Anterican Acadettty ofFamily Physicians, 1994.

õ8. Crabtree B$ Miller WL, Addison RB, Gilcltrist \¡J, Kuzel A.Ex?loring collaborative research in primary eare. ThousandOaks, Calif: Sage Publieatiotu, 1994.

59. Stange KC. Practicebased research nefworks: tlreir currentlevel of validity, generalizability, and potential for widerapplication. Arch Fam Med 1993; 2:921-3.

60. Baxter R.I, Kohn LlI, Omata RIÇ Wllliams C, Health sysùentchange in Cleveland, Ohio: a ease study. Washington, DC:Center for Studying Health System Change, 1997.

61. Ifuise J, PhüIips DM, Wesley R. A comparison of the atti-tudes of obstetricia¡rs and family physicians toward obstet-ric practice, training and hospital privileges of fanily physi-cia¡æ. J Fam Pract 1989;222L0-25.

62. Roberts RG, Bobula JÂ, ï[olkomir MS. Why fantily physicians delivery babies. J Fam Pract 1998; 46:3440.

63. Larimore WL. Sha¡ed antenatal ca¡e: an improved paradignrforwomen's health care leditorialJ. J Fam Pract 1998; 46:31-ù.

64. Wachter RM, Goldman L. The emerging role of "hospital-ists" in the American health care system. N Engl J Med1996;335:5147.

65. Rivo ML. The case for hospitalists: effectiveness or expedi-

ILLUMINATING THE'BLACK BOX,

crìcy? J Anr Iloard Fa¡l pract lgg7; l0:B?g-gl.66 :f:I"-i^.Tt..?1lTki sr, Srange KC, Langa l)M. Farnily

r)¡ìyslclâ.s a^d cu'ren¿ inpatient practice. J A¡a B.¡ar.<i FarirPlac lgg7; l0:85?-62.67. Gilclrrisr \{ Millel RS, .Gillanrlers WR, ct al. Does fanriìypr?ctice at resideucy sitçs r.eflect

"o,ir*,niiy practice? JFaur pract lg0B; 3?;5Bb_6J

65. Mille¡. WL, Crabrlee Bf; lvlcDaniel RA, Stange KC.Unclerstand-ing ¡rr.ilr ary c^a-ric practicc: a corìi p f ex i67 iuoAõiof change, J Fanr pracl tggs; iO:¡eg_iO69. Mitka M. Doctors o.fr.t fo1 employmenl iarger groups:

Manage t I car.c rt rir4 ng rr.ena to cãüJoiiáäil"ii' Åurf ñ.rir,January 20, l0g7.?û l1:,lIS.lIlII',l"ll: DW, Giflis KÐ. Cur.r.enr rrenrls in ptrysi-

crans pracuce anâlìg^emelìts: fronr owners to emplåyãei.JAMA 1996; 276:553-10.71. Flocke S¡.,-Stånge KC, Zyzanski SJ. The a-ssociation ofattli b utes of nrinr arw ca¡'e ivit¡ pr"u*tiue i"ói¿; ãìi;;ry.lr{ecl Carr lgti8. In pi.ess.?2.

9.::ll::..^P_I _y_{l-t w_, Ata v, F,rocke sA, sralge r(c,rnutaty care practice organization: a qualitative anajysis.JFanr placr 1998; 46:403_gi73. \4/ender R. Cancer' ïqt^e^elqtg antl prevention in prinrary

care. Carrcer. lggS; 72,1093_g,l74. 1l{È{r,r, Ill{, *tanln Iü1, Willianrs RL Tools for conuÈurÌity_

o¡.¡eittsrl"Þrln¡äÍy-cårc: â process for tin-tring-practice andcornmtrnity data. J,{¡t Boãrd Fam pract tggã;ìt;ZeSt.-"-75. Gr.eenlick MR. Educating _p,hysicians for population_ba¡ed

cli ni cal practice. JAM,A t ôSZ;2AZ : iA,45€.' -. "'76. Donalclson MS. yordy.KD, Lolrr KN, Vanselow NA, eds.Priurary. care: ÁmcricäC tidattr,- ìi ä^ ri"* äl"w*iri;eidi

I)C: National Àcadenry press, lggé.77. Scherger JFì Does the p_erson¿l physicÍan continue in man-

aged care? J Àm Board F-aur pra'ct iSg6;i:6i-S."

"Ëriili-;ii¡ifiï:'Jlii:'ie"',,,î,îl',îii',î,r'-"îlef Jll;ffåi#írue¡r,ention.s; an no id en¡iotlgjc$ qpüúc,il"crràñä 1lì il,, Nc; tn$rirure for,Roåearctr ¡n SólìãrËi,iaiäål'isil.

70. Luckrnann R, Melvitle SIí periodic health evaluation ofadutts: a survev of farnily physicianr.l¡;irÀ'Þrããi riÉ)Ej40:547-54.

80. Nationwicte Insurance Enterprise. l0Ðg codi¡rs a¡rd reim-bursement update. Columbus, Ohi", ñ;tññ*iå? InsuranccEnterprise, ti?97.81. Kirschner (JG, Davis $J, Jacobson C.A, eù al. physicians,

Currenr proci:durat reúninãiãgi'ipt :öe*btu"ago, I[:Anerican Mertical Ássociation, ìõCZ.-- -

82. ll'anks P, Clancy CIVI., Referrals.of aclult patients ti.orr pd-l,]^,t1,pl!, dernogr.aphic disparities and iheir. rclations'irì¡rto HMU Ílìsurânce. J l'am pracl. l$97;4r;t:47_r,tï.

æ k:,rqt.Ç-,,titt-hrisr V,. McÇort _G, NEON Research Gr.oup.¡¡te cotìsttllalion artd^r.efc¡.r€1. 1:rocess. A r.e¡lort fr.oin¡\Ef)N. J l¡;uü Pr¡ct it¡Og: iO:{i+:i,

84. Rr¡lÍrins JÅ, l3r'{akis I{D, Helnrs lJ, et al. The inl1ue¡lce oflll$_srg!{t-t ptïtf I ir{' behaviors orì ll¿rtient satisfaction. FariiMe<l l9Ð;1; !ñ;17-90,

85. Plobst JO, Gleenhouse DL, Selassie AW. parient ancl physi-cian satisfaction s,ith an oirtpatient cu.u uliü. ¡ Fum'pia..t1997;45:418-2b

86. Schulz R. Scheckler. Wg, Moberg Iì Johnson I)R. Changingnânll'e of phy.siciarr satisfactio¡i with health nraintena"ncËorgalization and fee-fo¡.service practices. J Fanr priii1997;45:321-30.

87. Dean VC- Pìtysicial satisfaction reflecls changes in healthcale lantlscape. J Faur pnlct lgg7; 4b:3lg_20. "

88, Cro¡nt¡ie DL The t{iagnostic prccess. J Coll Gen pract Ig63;6:5?9-8?.

89. Rakel RE. Tlie lanrily ¡:hysician. In: Rakel RE, cd. Esscntialsol^Ialìt¡ty ntedicirre. philartclphia, pa: WB S¿unders Co,1993.

90. Salrdlcr G. .Thc inrportance of the history in the nredicalcnnlc and fltc cost of urìnecessaÌy tesLs. Am Heart J lgg0;100:92&33.

91. Kaplan.Sll,.Greenfreìd S, Gandek B, lìogers WH, Warc JE.unarac¿eris{,ics. o Í physiciarrs rvi th panlcip¡ttory dccision-nìakìng stytes. Anu Intern l\{ecl ¡Ðg6i-lt4:4ú?-504.

92. Ihplan SH, Galdek B, Gmc¡f¡elcl S; Itogers \1'II, lVam JE.Palienr å¡d visit cha¡actcllstjcs rclåi¡jd io-plujiciårs lrãrtiri:parory decision-rnakìng styJe. Med Ce¡e t0li5;$rili$-Si, ---'

93. Sfc_war[ M, Bror.r,u Jß, úes¿on WlV, McWhinney IR,Ivtc\\,illiam CL, F\.eeurari TR. patienr-c'cntèieã rneaicine:[ransforîüng the clirúlal method. Thousand Oats, ðãiîSage Pubhcãtions, lgg5.

94. Rosenblatt RÂ. Çonfronting the millenniunr: family nred!cine in fÀe late 20th century, Farn Med tggO; p2*Ofi.

95. Medalie JH, Faurily nredlcin-e principles and applicafions.f3attiruore, ¡¿rì: Wiriianu & wiiHËõõ-i9ñ." *

96. $tephens GG. The intellectual basis of family practiee.'fucson, Ariz: Wìnter publishing lgSr. -- - -'

97. McWìrinney IR. An introduction to farnily medicine. NewYork, NY: Oxford Univerciff prcss, lggl. -

98. Wúte KL. The t¿sk of medicine. Dialogue at Wickenburg.Menlo Pa¡k, Calif: 'l'lre Henry J. faisei ila¡niiy Foundatic¡fi,1988.

'Ihe Journal of Farnily Practice, VoL 46, No. 5 (ivlay), lggs ggg