Training of a Community Boardto Increase The Effectivenessof a Health Center

SAMUEL J. BOSCH, MDROLANDO MERINO, MDRUTH ZAMBRANA, PhD

SINCE THE ENACTMENT OF MEDICAID AND MEDICARElegislation in the United States, the total national healthexpenditure has risen from about $39 billion in 1965to about $139 billion in 1976, an increase from 5.9 to8.3 percent of the gross national product (1). Duringthis period, consumer demands for change in the healthcare system and, in general, for participation in theinstitutional decisions affecting their lives have esca-lated significantly (2,3). In partial response to thesedemands, Federal legislation has specifically requiredthe integration of consumers into the planning, man-agement, and evaluation activities of federally funded

The authors are in the Department of Community Medicine,Mount Sinai School of Medicine of the City University of NewYork. Dr. Bosch is professor and deputy chairman of the de-partment, Dr. Merino is an assistant professor, and Dr. Zam-brana is a research associate.The study was supported by the Health Services Improve-

ment Fund, Inc., and was described in a paper presented atthe Eighth International Scientific Meeting of the InternationalEpidemiological Association, held in Puerto Rico, September18-23, 1977.

Tearsheet requests to Samuel J. Bosch, MD, Department ofCommunity Medicine, Mount Sinai School of Medicine, FifthAve. and 100th St., New York, N.Y. 10029.

health programs. Among these programs are the Officeof Economic Opportunity's neighborhood health cen-ters of the 1960s and, more recently, the health mainte-nance organizations and the health systems agencies.The laws establishing these programs stipulate highlevels (51-60 percent) of consumer participation inplanning and health policy making activities.

Despite the legislative intent, it is commonly agreedthat effective, broad-based consumer participation gen-erally has not been achieved. As Pecarchik commented,"the presence of consumer representatives on the man-agement board, even though these representativesaccount for 51 percent of the membership, does not initself guarantee meaningful consumer participation indecision making and planning. The most salient factorin meaningful participation by the consumer member-ship is its preparedness" (4).

However, it is precisely in their lack of preparednessthat consumers are at the greatest disadvantage whendealing with health care issues. They are generallyunfamiliar with the framework and vocabulary of healthcare delivery and planning (4). Furthermore, they

May-June 1975, Vol. 94, No. 3 275

frequently lack access to the information they need forrational planning, evaluation, and decision making.If information is available, they are often unable tointerpret it (5). Not surprisingly, the community boardshave had minimal impact on the health care deliverysystem (6). Their lack of preparedness has diminishedtheir capacity to influence change.

In this report we present the results of an educationalproject aimed at improving the health care knowledgeand skills in group planning and policy making of themembers of a community board. The board governs abicultural and bilingual neighborhood health center inthe East Harlem section of New York City.

BackgroundTwo institutions in that community-Mount SinaiSchool of Medicine of the City University of NewYork and the East Harlem Tenants Council Neighbor-hood Health Center-joined in this project. The medi-cal school represents the union of a large voluntaryhospital and a public university. The neighborhoodhealth center, a primary care giver for families inthe area, is governed by the East Harlem TenantsCouncil, an organization of the local Puerto Ricancommunity.

East Harlem is located in the northeast corner ofManhattan, north of 96th Street and east of FifthAvenue. During the early 1900s it was a neighborhoodof Italian and Jewish immigrant workers. Todaypeople of Puerto Rican origin make up 45 percent ofthe population, blacks another 35 percent, whites ofdirect European ancestry 17 percent, and "others"3 percent (unpublished paper, "East Harlem Commu-nity Study," by L. Johnson, A. Lynch, and 0. Rivera,Mount Sinai School of Medicine, City University ofNew York). The area, always inhabited by relativelylow income groups, has deteriorated into the kind ofghetto for which New York City is infamous. Manyhouseholds have no income from employment, andmost persons who work hold low paying jobs. Multiplesocial problems and a high level of morbidity character-ize the population.

From the outset, the founders of iMount Sinai Schoolof Medicine made the identification and solution ofcommunity health problems a clear goal of their insti-tution (7). To implement that commitment they estab-lished a generously endowed Department of Com-munity Medicine (DCM). One of the DCM's principalmandates is to serve as a bridge between the resourcesof the medical center and those of the surroundingcommunity in joint problem solving. The department

wants to contribute to the improvement of health careservices in East Harlem without assuming direct re-sponsibility for the administration or provision of healthcare services. The services of the department entail bothstimulating and responding to requests from commu-nity groups for technical assistance in learning, plan-ning, implementation, and evaluation.

Staff members participate with a variety of consumerand provider groups in identifying and solving prob-lems related to the organization and provision of healthcare, while attempting to foster interorganizational link-ages to encourage rational use of existing health andhealth-related resources and ultimately to attain a logi-cal regionalization of health services in the area (8).The long-range service goal of the DCM is to changethe way health care services are financed and organizedin East Harlem as a means of enhancing the healthstatus and lifestyle of the community.

In 1974, the East Harlem Tenants Council (EHTC)requested from the department technical assistance inthe planning and development of its neighborhoodhealth center. The center has been in operation sinceJuly 1, 1975, and now serves approximately 6,000 per-sons of varying socioeconomic levels. It is planned fora population of 30,000. The EHTC's board is thepolicy-making body for the center. The board is com-posed of 18 persons who are not health care providers.They come from various disciplines and have differenteducational backgrounds. The majority are East Har-lem residents and consumers of services in the healthcenter. Leadership of the board is vested in its execu-tive committee, composed of a chairperson, vice chair-person, treasurer, secretary, and two members at large.

276 Publ Health Reprt

The board's policies are implemented through thehealth center's director (9). The director is responsiblefor planning, implementation, evaluation, and controlof center activities and the performance of the center'stasks. The director is also an ad hoc member of theboard. The DCM, under a contract with the board,maintains a continuing consultative role in evaluatingthe center's programs.

MethodologyIn this project, the department's staff employed aprocess that allowed the learners to be actively involvedin defining educational objectives, selecting the teach-ing methods, and identifying the procedures that wouldbe used in measuring the learning (9). In keeping withthe DCM's participatory approach to change, thelearners paid for the educators' teaching them. Duringthe 12 months of the project, a committee comprisingfour DCM staff members and the director of the centerwere responsible for planning and implementing thetraining. They met weekly; occasionally they werejoined by education consultants in order to organizeand monitor the progressive stages of the process.

In the definition of the educational objectives, twomain procedures were used: (a) a questionnaire wasadministered to board members to identify their per-ception of learning needs and (b) before the formaltraining a Puerto Rican sociologist conducted prelimi-nary observations of three consecutive board meetings.In addition, four interviews were held with the sixmembers of the board's executive committee duringthe analysis of the answers to the questionnaires andthe sociologist's observations. Trainees and trainersagreed that the outcome objectives to be achieved bythe trainees would be acquisition of the ability to:

* Read monthly statistical reports critically (reportscontaining data on the effectiveness and efficiency ofthe health center).* Recognize utilization and productivity trends.

* Use data reports to decide among alternative techni-cal recommendations made by the center's staff.* Critically analyze expenditures in relation to thehealth needs of the community and the health center'sobjectives and sources of income.

In addition to acquiring definable analytical skills,it was agreed that the members should acquire an

understanding of the vocabulary and basic concepts of(a) community health needs, demographic data, and

epidemiologic data such as the social and cultural fac-tors that may affect incidence of disease; (b) demandand use of health services; (c) utilization and produc-tivity trends in the care of a defined population group;(d) budgeting for programs; (e) continuing evalua-tion of the center's performance, with emphasis on theeffectiveness and efficiency of its programs, and (f)definition of health policies and intermediate and long-term goals and objectives.

These objectives were formulated, taking into consid-eration that the skills and knowledge would have to beacquired by the board members by the end of 1 yearand with a maximum of 15 hours available for formaltraining sessions (9).

The DCM team defined the training methods on thebasis of the answers to the questionnaire and thesuggestions of the chairperson and members of theboard's executive committee. It was decided that theformal training sessions would be part of the board'sregular monthly meetings and that the most appropriateteaching procedures would be (a) to present educa-tional materials of incremental complexity in groupdiscussions, (b) in the discussions to analyze actualperformance data on the center (the center's directorwould present the data), and (c) to encourage discus-sions between the board members and the DCM staffduring the training sessions. For example, during thefirst 3 months the discussions were geared toward theunderstanding of terminology and concepts. At the endof the project, an entire session was devoted to theanalysis and approval of the center's program andbudget for 1977-78. At the request of board members,the sessions were initially limited to 1 hour. However,as the project evolved, the sessions were graduallyprolonged from 1 2 hours to 2 hours (again at therequest of the trainees). The DCM technical assistanceteam served as the primary educational resourcethroughout the project.

EvaluationIn evaluating how much board members had learned,

a combination of summative and formative procedureswere utilized (10). The summative procedures includedthe administration of a multiple choice questionnaire toboard members (a) to measure knowledge gain and(b) to obtain feedback. The chief formative procedureto evaluate process, which supplemented the test re-sults, was the sociologist's observations of the board asa work group.

A multiple choice questionnaire was pretested on asample population with background similar to that of

May-June 1079, Vol. 94, No. 3 277

the board members. It was administered to the learnersat the beginning of the first session and at the end ofthe last training session. Sixteen questions addressedissues of medical care organization, financing mecha-nisms, and evaluation of the effectiveness and efficiencyof the center, particularly the analysis of HEW indica-tors.

The purpose of feedback was to provide the trainerswith an assessment of the learners' understanding ofwhich goals had been attained at the end of theproject. Feedback was obtained through a final sessionin which another questionnaire was distributed; it wasfollowed by an informal 45-minute group discussion.

The field observations were focused particularly ondocumenting participation in decision making, on theuse of decision-making skills, and on the promotion ofchange through new policies. In addition, a feedbackquestionnaire administered to the board members dur-ing the project was used to modify and improve thetraining as it evolved. The field observations wereconducted and objectively logged by the sociologistwho participated in the identification of the educationalobjectives. The sociologist's role was to attend, as anobserver, all board meetings and executive sessions, inaddition to the curriculum development meetings, tocollect detailed descriptive data, and to analyze theseobservations in terms of the objectives of the program.

The field data were analyzed qualitatively, particu-larly to examine the following behaviors: level ofparticipation, appropriate use of terminology, under-standing of the relationship between income and ex-penses, and application and use of data to make policydecisions.

Limitations of the MethodThere were several methodological limitations in thisstudy that derived particularly from the nonprofit char-acter of the community organization. First, the "learn-ers" were not a captive audience. Their attendance atboard meetings was voluntary, and most of them hademployment obligations to meet. Consequently, at-tendance ranged between 14 and 6 persons at thedifferent meetings. It should be pointed out, however,that a core of board members, the majority of whombelonged to the executive committee, attended consist-ently. Some members who attended irregularly reportedfeeling "lost" when they did appear.

Second, members had different levels of knowledgeand skills before the training got started. Some wereprofessionals who had graduate and post graduate edu-

cation; the nonprofessionals worked in semiskilled occu-pations. Another important limitation was the fact thatthree new members entered the board half way throughthe evaluation of the project.

ResultsThe data collected in this study demonstrated that themembers of the board gained basic knowledge abouthealth care delivery and acquired decision-making skillsthat they applied in governing and in introducingneeded changes. The data also reflected an activelygrowing participation by the trainees in the educationalsessions.

The multiple-choice pretest was answered by 10 ofthe 14 members and the posttest by 8, including the 3who were new to the board. These numbers are sosmall that the statistical analysis is meaningless. How-ever, a major gain in knowledge was reflected in theresponses to questions on the center's performance,especially as they related to measures of utilization, ef-fectiveness, and productivity.

The analysis of the field observations revealed muchvaluable information in respect to attainment of theeducational objectives that the project was set up toachieve.

All board members definitely increased their level ofparticipation in the group discussions. According tothe field observations, in every meeting at least onequestion was asked by every board member presentand four of the members usually initiated or maintaineddialog with the other trainees and the trainers. Thefour most active participants were members of theexecutive committee. As the project evolved, the formu-lation of pertinent questions, opinions, and suggestionsabout the management of the health center graduallybecame more sophisticated. During the last five ses-sions the trainees primarily raised questions related tofinancing issues, engaged in discussions on sources ofincome to the center, the nature of expenditures, andthe relation of income to expenditures.

Substantial learning occurred in the use of appropri-ate terminology and in the handling of concepts relatedto the operation of the center. For example, clear dis-tinctions were made regarding who is a "registrant"and who is a "user," what is an "encounter," a "utiliza-tion rate," an "office visit" and an "outreach visit." Ofutmost concern was development of members' abilityto use and interpret terminology and concepts em-ployed to evaluate the performance of the center. Forexample, measures of effectiveness were understood as

278 PubIl Health Reports

the comparison between attained and planned objec-tives, activities, and resources. This understanding inturn enabled the members to distinguish between con-cepts of efficiency and effectiveness.

The most outstanding changes were observed in theboard members' ability to extrapolate data from themanagement information system and request additionalspecific information, such as hospital referrals, in orderto recommend changes in the scope of services andprotocols at the center. For example, the presentationof data on no-show rates stimulated active discussionand a request for a preliminary study in order better tounderstand the factors that influence these rates.

The board members appeared to understand the no-tion of limited resources for unlimited health needs. Dur-ing the presentation of budgetary concepts, questionswere raised that reflected knowledge of the relationshipbetween income and expenses and sources of incomeissues such as limited funds and the balancing of thebudget in terms of expenditures and sources of income;cost of home visits versus office visits; and ultimate costsof preventive versus curative medicine. In this meetingboard members exhibited an understanding of the dif-ferences between provision of services and costs thatpositively demonstrated their grasp of new knowledge.They further requested a monthly summary of ex-penditures to keep abreast of this aspect of operations.During the last two learning sessions the board membersbecame involved in complex discussions of priorities asthey are reflected in the formulation of objectives forthe health center. There have also been several dis-cussions on the identification of activities that wouldenable the center to operate more efficiently. For ex-ample, the cost of pediatric care compared with adultcare and the impact of both on the health status ofthe population, as well as the cost of preventive careas opposed to curative care, have been discussed briefly.The need to establish health priorities was recognizedas a fundamental planning concept.

An important outcome of the training has been theacquisition of skills in the interpretation and use ofmonthly statistical data pertaining to the health cen-ter's operation. Following a series of sessions, the boardsuggested standardizing a similar periodic informationsystem for the operation of all programs presently un-der the aegis of the council. How the enhanced knowl-edge and skills of board members were used to effectconstructive change was best demonstrated in the han-dling of the reorganization of the EHTC's housingprogram. At the board meeting of March 3, 1977, afterthe training session, the housing director was scheduledto present her report to the board. The vice chairperson

stated: "In view of our training we will now demandof the project director a systematic quarterly report inorder for us to evaluate what has been done and beginto set policy based on objectives." He continued toexpress the board members' concern that "many timeswe have to make decisions and/or suggestions withoutinformation."

There has been a persistent demand from the boardmembers to discuss topics not originally scheduled inthe curriculum, for example, assessment of the qualityof care. They wanted this topic addressed in a supple-mental training session.

As mentioned previously, the sessions were lengthenedfrom 1 to 2 hours at the request of board members.Even so, it became necessary to set limits on the ques-tions and answers raised by board members, particu-larly when the questions were not related to the mate-rial being covered in the particular session.

Additional training sessions were formally requestedby the chairperson of the board. He requested a specialone for himself and additional sessions for those mem-bers who desired further to expand their knowledgein the areas already presented in the training program.One board member protested, in the feedback ques-tionnaire, that "the time constraints resulted in twoproblems: board members remained with questions un-answered and the trainers were not able to cover theiragenda."

Feedback FindingsSeven of the eight respondents to the feedback ques-tionnaire rated the project as a whole excellent, andthe same number found the seminar discussions in-formative.

In the responses to the feedback questions aboutfulfillment of the cognitive goals, all the learners agreedthat they had acquired (a) a better understanding ofthe technical terminology of health care, (b) skills thatwould enable the board to review health center dataand ask the director more specific "how, where, andwhat questions," and (c) skills that would enable thegoverning body as a whole to be more effective in mak-ing decisions and achieving constructive changes in thecenter. Eighty percent of the board members claimedthat they had gained knowledge enabling them tounderstand data regarding the utilization of the center,and 66 percent responded that they had learned themeaning of the indicators that are requested in thereport of neighborhood health center activities to theDepartment of Health, Education, and Welfare. Only

May-June 1979, Vol. 94, No. 3 279

54 percent stated that they had acquired knowledge infiscal matters. The acquisition of legal knowledge wasgiven the lowest rating. Further training was requestedby 75 percent in understanding epidemiologic data, by50 percent in understanding how to measure quality ofcare, and by 40 percent in acquiring knowledge infiscal matters.

DiscussionIt is difficult to assess an educational project that at-tempts to develop complex knowledge and problem-solving skills in a relatively short period. All who haveengaged in curriculum study and design can understandthe complexity of the task we addressed. Moreover, ininterpreting the results of such projects, it is importantto analyze not only the learning that took place butalso the methodologies used, how they were modifiedby the real-life problems of a community organization,and their applicability to other settings with similarlearning needs.

With regard to curriculum building, the trainingwould probably have confronted serious implementationproblems if the members of the board, as potentiallearners, had not been actively involved in that process.Their participation in the definition of their learningneeds, their approval of the identified educational ob-jectives, and their endorsement of the educationalmethods selected enhanced their motivation to learn.Integration of the scheduling and content of the learn-ing sessions with the agenda of regular board meetingsgave relevance to the subjects.

To acquire qualitative data on the behavior of boardmembers, the collected field notes and the recorded logwere practical, reliable tools, easy to use in any setting.They gave an accurate and objective picture of thelevel of participation, how terminology was employed,and how data were used in making policy. But collect-ing these behavioral data was not without problems.Accepting the observer's presence during the normaldeliberations of the board was at first difficult anddepended heavily on such factors as the observer'sskills, cultural background, and personality. The valid-ity of the collected data depended then on the ob-server's uninhibiting and unobtrusive participation aswell as on the objectivity of the observations.

It was difficult to assess the depth of the knowledgethat was gained. Facts and the understanding of funda-mental concepts were undoubtedly acquired, but theboard members' capacity to analyze data and to makejudgments and decisions could only be determinedthrough qualitative information. At what rate knowl-

edge and skills were retained once the project wascompleted is even more difficult to determine.

The learners themselves were concerned with thisissue and continue to seek ways to retain, in their boardmeetings, some form of technical assistance in orderto "facilitate" their analysis of routine information andtheir capacity to implement planned change. Theirparticular interest now is in learning to measure qualityof care and in developing a mechanism for continuingquality assessment, a direct outcome of their concernwith retaining and enhancing their newly acquiredknowledge and skills.

Beyond the actual learning that took place, an im-portant byproduct of this experience has been the de-velopment of a closer working relationship between thecommunity organization and the department of com-munity medicine. The interaction among members ofthe board and members of the department during thetraining sessions have helped to establish better com-munication and strengthen rapport. The board mem-bers have a much better understanding of the role thatthe medical school is assuming in the community, andthe attitudes of the community organization towardthe school have improved.

References1. Califano, J. A.: Keynote address presented at the Ameri-

can Medical Association, Fairmont Hotel, San Francisco,June 19, 1977, HEW News, U.S. Department of Health,Education, and Welfare.

2. Gartner, A., and Reissman, F.: The service society andthe consumer vanguard. Harper & Row, New York, 1974.

3. Metsch, J. M., and Veney, J. E.: Consumer participationand social accountability. Med Care 14: 283-293 (1976).

4. Pecarchik, R., Ricci, E., and Nelson, B.: Potential contri-bution of consumers to an integrated health care system.Public Health Rep 91: 72-76, January-February 1976.

5. Greer, A. L.: Training board members for health plan-ning agencies. A review of the literature. Public HealthRep 91; 56-61, January-February 1976.

6. Metsch, J., and Rosen, H.: Consumer participation inhealth delivery. N.Y. Univ Educational Q 7: 16-20,winter 1976.

7. Popper, H., and Koffler, D.: The goal. J Mt Sinai Hosp34: 401-407 (1967).

8. Bosch, S. J., and Deuschle, K. W.: The role of a medicalschool in the organization of community health services.Bull NYAcadMed 53: 449-464 (1977).

9. Merino, R., Bosch, S. J., and Zambrana, R.: Evaluationas a tool for community education. In Evaluation of childhealth services: the interface between research and medi-cal practice. Edited by S. J. Bosch and J. Arias. DHEWPublication No. (NIH) 78-1066, U.S. Government Print-ing Office, Washington, D.C., 1978, pp. 257-265.

10. Bloom, B. S., Hastings, J. T., and Madaus, G. F.: Hand-book on formative and summative evaluation of studentlearning. McGraw Hill, New York, 1971.

280 Public Health Reports