developing countries: results of a Delphi survey

Priorities for pharmaceutical policies indeveloping countries: results of a Delphi surveyJ.-D. Rainhorn,' P. Brudon-Jakobowicz,2 & M.R. Reich3

The use of the Delphi method as a systematic and logical approach to establishing consensus amonginternational experts on the priorities for interventions in national drug policies in developing countries isdescribed. The Delphi survey showed a high degree of reliability, as evidenced by the high responserate, the quality of respondents, and the high standard for consensus. In addition to creating consensuson key issues and key components for priority intervention, the study identified six components thatcould constitute a basic framework for designing drug policy in developing countries. The study's con-clusions have important implications for decision-makers within international development agencies andnational governments.

IntroductionIn the 1970s the lack of basic drugs at prices that thepoor could afford became a major concern for healthofficials in the newly independent states of the devel-oping world. In response, the World Health Assem-bly passed a resolution (WHA28.66) in 1975 thatmandated the World Health Organization "to devel-op means by which the Organization can be of great-er direct assistance to Member States in advising onthe selection and procurement, at reasonable cost, ofessential drugs of established quality correspondingto their national needs". WHO published the firstmodel list of essential drugs in 1977 (1). Based onanother World Health Assembly resolution in 1979(WHA32.4 1), the Organization established theAction Programme on Essential Drugs and Vaccinesin 1981 to develop a strategy which embraced allaspects of national drug policies.

In the 1980s, a large number of countries adopt-ed essential drug lists with support from develop-ment agencies and nongovernmental organizations(NGOs) and began operating active programmes. Sev-eral countries, including Bangladesh (2), Philippines(3), and Nigeria (4), developed national policies andcarried out major changes in the pharmaceuticalsector.

I Director, Centre de Recherche et d'Etudes sur le Developpe-ment de la Sante (CREDES), Paris, France.2 Scientist, Action Programme on Essential Drugs, World HealthOrganization, 1211 Geneva 27, Switzerland. Requests forreprints should be sent to this address.3 Professor of International Health Policy and Director of theTakemi Program in International Health, Harvard School of Pub-lic Health, Harvard University, Boston, MA, USA.Reprint No. 5476

Despite these efforts, however, serious problemsremained in many developing countries, due in partto ad hoc, ineffective, and contradictory pharmaceu-tical policies. In 1988, according to WHO, more than1300 million people had little or no regular access tothe most essential drugs (5).

At the international level, the pharmaceuticalindustry and many developed country governmentsvoiced strong opposition to WHO's efforts to pro-mote policies on essential drugs. At the nationallevel, conflicts arose over the design of pharmaceuti-cal policy and such issues as the role of the privatesector and the priority for local production. Conflictsalso emerged among international developmentagencies over where to intervene first in the pharma-ceutical sector and over which strategies were likelyto be cost-effective (6).

Numerous efforts to resolve these controversieshave achieved only limited success. The Conferenceof Experts on the Rational Use of Drugs, held in Nai-robi in 1985, was a concerted attempt by WHO tocreate agreement on what should be done (7). Sincethen, conflicts of opinion have persisted on prioritiesfor action, reflecting the divergent views and inter-ests of the different actors (8). A new approach istherefore required to help formulate drug policiesthat will meet the challenges of the next decade. Thisarticle reports the use of the Delphi technique as asystematic and logical approach to establish consen-sus among international experts on the priorities forinterventions in national drug policies.

MethodsThe Delphi technique is a method for structuringcommunication in a process that allows a group ofindividuals to deal with a complex problem and

Bulletin of the World Health Organization, 1994, 72 (2): 257-264 © World Health Organization 1994 257

J.-D. Rainhorn et al.

reach consensus (9). The process involves the use ofa series of questionnaires designed by a monitorgroup and then sent by mail in several rounds to arespondent group of experts who remain anonymous(10). After each round, the results are summarizedand assessed by the monitor team and used to devel-op a questionnaire for the next round. The assess-ment document and new questionnaire are then sentto all members who responded. A Delphi survey isconsidered complete when a convergence of opinionoccurs or when a point of diminishing retums isreached (11).

A major advantage of the Delphi technique isthat it avoids problems commonly encountered inface-to-face group meetings. These problems includethe influence of key persons on the responses ofother panel members as well as the geographicalconstraints and costs of bringing together a group ofexperts. The anonymity of answers allows Delphiparticipants to express their personal views freely.The method is particularly useful for a subject withstrong differences of opinion or high levels of uncer-tainty. The reliability of the Delphi method dependslargely on the selection of panel members, the size ofthe group, and the number of rounds (11).

The Delphi method was developed as a forecast-ing tool at the RAND Corporation in 1948 (10), andhas been widely used in defence studies and businessstrategy. The first studies in the health field werepublished at the end of the 1960s (12). In the phar-maceutical field, the Delphi technique has been usedfor conceptualizing the future of the pharmacist pro-fession (13), for standardizing terminology (14) andprescription practices (15), and for exploring policyoptions (16). The pharmaceutical industry hasapplied the Delphi technique to forecast the evolu-tion of important markets (17), although most ofthese studies are proprietary and not publicly avail-able. Our review of the literature did not identify anyuse of the Delphi technique to assess or design drugpolicies in developing countries.

Study design. The study was designed by a monitorgroup, set up at the Harvard School of Public Healthin Boston, USA, who received support from aca-demic specialists and public health experts originat-ing from developing countries. The study was basedon a three-step logical approach and was completedover a six-month period. The first step sought todefine the main problems faced by developing coun-tries in the pharmaceutical sector, which have beencalled "key issues". The second step was to identifyfor each of the key issues those elements of thepharmaceutical system that have a major impact onperformance; these have been called "key compo-nents". The third step was to rank both the issuesand the components, in terms of importance forintervention, to establish priorities.

Respondent group

A list of 54 persons with substantial expertise inpharmaceutical policy in developing countries wasprepared to serve as the respondent group. As shownin Table 1, the group included persons from differenttypes of institutions: multilateral donors, such as theWorld Bank and European Economic Community;the United Nations system, such as WHO andUNICEF; nongovernmental organizations; researchand consulting groups; pharmaceutical companies;universities; and consultants. Half were pharmacistsor physicians, and half were economists, managers,policy analysts, anthropologists, or statisticians. Thegroup of experts represented people from 12 coun-tries on four continents.

The questionnairesAs shown in Fig. 1, the first questionnaire was devel-oped after a review of the literature related to phar-maceutical policy in developing countries, includingunpublished reports and documents from more than50 countries. The monitor group proposed a prelimi-nary set of five key issues and 40 key components.

Table 1: Evolution of the Delphi respondent group by categories: initial and final compositionsaDevelopment UN system Drug Nongovernmentalagencies organizations industry organizations Consultants Academics Total

Physicians, (5)/4 (5)/4 (1)/1 (5)/5 (6)/5 (3)/3 (25)/22pharmacists

Other (e.g., economists,managers, policy-analysts, statisticians,anthropologists)

Total of eachsubgroupb

(3)/2

(8)/6

(3)/2

(8)/6

(7)/4

(8)/5

(4)/i

(9)/6

(3)/2

(9)/7

(6)/5 (26)/16

(9)/8 (51)/38

a The initial composition is shown in parentheses. The final composition of respondents in Round No. 3 is shown to the right.b The size of each initial subgroup was nine for a total of 54. Three experts declined the invitation to participate, giving a total of 51 inthe survey.

WHO Bulletin OMS. Vol 72 1994258

Pharmaceutical policies In developing countries

Respondents were asked to review the preliminarylists, indicate their agreement or disagreement,rephrase the proposed formulations, and proposeadditional issues and components. Three members ofthe initial group declined to participate in the study.

In the Round No. 2 questionnaire, respondentswere asked to select the formulation that best definedeach key issue, review additional key issues pro-posed in Round No. 1, rank the key issues accordingto feasibility of intervention and likely impact on theexpected outcomes, and select five priority compo-nents for each key issue.

In the Round No. 3 questionnaire, respondentswere requested to select the best formulation of twokey issues not yet decided, rank the major structuralconstraints confronted by efforts to improve drugsector performance, and rank the key componentsaccording to the priority of intervention for solvingeach key issue.

ResultsKey issues

A strong consensus was obtained in Round No. 1 onthe five key issues proposed by the monitor team:"Structural constraints in the country" (91.1% ofrespondents), "Lack of government commitment"(86.7%), "Structural weakness of the drug public sec-tor" (84.4%), "Limited drug affordability" (82.2%),and "Irrational use of drugs" (100%). The initial for-mulation of two key issues was accepted with strongagreement: "Structural constraints in the country"(61.9%) and "Irrational use of drugs" (73.3%). Newformulations were proposed in Round -No. 2 for theremaining three, taking into account the suggestedrephrasing and comments from respondents.

A total of 40 additional key issues were pro-posed by the respondents, along with 10 commentsregarding alternative key issues. Those most fre-quently mentioned were: the behaviour of the privatesector (17.8%), the role of the multinational pharma-ceutical industry (13.3%), the underestimated role oftraditional medicine (8.9%), the lack of foreignexchange (6.7%), and the poor efficiency of inter-national aid (6.7%). These five were proposed inRound No. 2 as possible new key issues or compo-nents.

In Round No. 2, 59.0% of respondents acceptedthe formulation "Limited affordability of essentialdrugs in both public and private sectors". New for-mulations, reflecting comments from respondents,were proposed for the remaining two key issues inRound No. 3. The proposal to add a sixth key issuewas rejected because none of the proposed additional

key issues received more than 20.5% agreement. Amajority (62.5%) responded that the first key issue toaddress for improving drug sector performance is theproblem of government commitment (with a meanrating of 5.38 out of 6). Respondents ranked theweakness of the public drug sector as the second pri-ority issue (with a mean rating of 4.23) (Table 2).

In Round No. 3 a consensus emerged on the for-mulations that best defined the remaining two keyissues: "The lack of government commitment andcapacity to design and implement a rational drugpolicy" (97.4%), and "The structural weakness andpoor performance of the public drug sector" (57.9%).

Key components

In Round No. 1, the respondent group added another119 items to the preliminary list of 40 componentsprepared by the monitor team. Those mentioned atleast three times were included on a revised list,resulting in a total of 46 key components.

In Round No. 2, respondents were asked toselect five priority components for each key issue.Across key issue categories, six components werementioned as priorities by more than 70% of therespondents: the establishment of appropriate legisla-tion and regulation, the selection of essential drugs,the allocation of sufficient funds for drugs in thehealth budget, the improvement of procurement pro-cedures, the establishment of a drug pricing policy,and the organization of continuing education pro-grammes in drug use (Table 2).

In Round No.3, a consensus emerged that themajor constraints for achieving the objectives ofavailability and affordability of essential drugs forthe majority of the population are the public sector'scapability, the health human resources availability,and the country's socioeconomic level. A consensusalso emerged for each key issue on the three top-ranked key components for intervention (Table 2).

DiscussionThe study's results are discussed below from threeperspectives: the reliability in establishing consen-sus, the priority issues for policy, and the prioritycomponents for action.

Reliability

The high rate of agreement obtained on most ques-tions in this survey indicates that a strong consensusemerged on the priorities for intervention in nationaldrug policies. Systematic application of the Delphi

WHO Bulletin OMS. Vol 72 1994 259


Fig. 1. The Delphi method: summary of major actions taken by the monitor group and the Delphi participants.

ACTIONS BYMONITOR GROUP

ACTIONS BYDELPHI PARTICIPANTS

Round No. 1:

Take position on key issues,propose additional issuesand write comments

Propose additional components

Round No. 2:

Analyse responses

Prepare assessmentdocument No. 1

Prepare secondquestionnaire

Round No. 3:

[45/51 = 88.2% response]

Select key issues

Rephrase and rank key issues

Select key components

[40/45 = 88.9% response]Analyse responses

Prepare assessmentdocument No. 2

Prepare thirdquestionnaire

Rank major structural constraints

Rank key components for each issue

Select priority components for action

[38/40 = 95% response](overall 38/51 = 74.5% response)

Final review:

Analyse responsesfor three rounds

Prepare final paper

--O- Comment on final paper

WHO Bulletin OMS. Vol 72 1994

Propose key issuesand key components

Prepare first questionnaireand background papers onDelphi method

I mi~~~-

op

-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

260

Pharmaceutical policies in developing countries

Table 2: Selection and ranking of key components In Round No. 2 and Round No. 3. All components mentioned as apriority in Round No. 2 by more than 50% of the respondents are listed for each key issue. The top three components areranked according to the results of Round No. 3.

Key issues! Mean rating for ranking % of respondents in agreement with key components:key components of each key issuea <50% 50-59% 60-69% >70%

A. "Lack of government commitment andcapacity to design and implementa rational drug policy" [5.38]

1. Legislation/regulation2. Essential drug selection3. Drug allocation in the health budget-Drug control authority +

B. "Structural weakness and poorperformance of the public drug sector" [4.23]

1. Procurement procedures2. Essential drug selection +3. Health human resources development +-Distribution/logistics +

C. "Irrational use of drugs" [3.61]1. Continuous education (drug use)2. Prescribing practices +3. Essential drug selection +-Pharmaceutical firms marketing +-Public information/education +

D. "Limited affordability foressential drugs in both publicand private sector" [3.23]

1. Essential drug selection +2. Drug pricing policy3. Procurement procedures +-Drug financing policy +-Generic drugs +

E. "Structural constraints inthe country" [2.97]

1. Public sector capability +2. Health human resources availability +3. Country socioeconomic level +-Foreign exchange +- Transport facilities +

a In Round No. 2 respondents ranked the key issues on a scale of 1 to 6, with 6 representing the key issue considered most importantaccording to feasibility of intervention and likely impact on the expected outcomes.** The six key components (that received more than 70% agreement and were top-ranked for key issues A, B, C, and D) could consti-tute a basic framework for designing pharmaceutical policy.

method by the monitor group circumvented contro-versies that have long affected this field. A review ofseveral factors suggests that the consensus obtainedis reliable.

The survey was designed to meet the criteria ofthe classical Delphi method (9). The monitor teamcarried out three rounds of questionnaires, which isstandard in most surveys and considered to be suffi-cient (9). Members of the respondent group were allwell informed on the subject, and the size of thegroup reached acceptable standards for a Delphi sur-vey (18). The literature reports that once an adequate

size is reached (generally around 30 well-informedand motivated experts), few new ideas result fromincreasing the group's size (19). The response ratefor the three rounds was very high (74.5% of theinitial group), compared with figures reported inother Delphi surveys. Finally the survey used a highcutoff rate, compared with other Delphi surveys, tosignify the achievement of consensus.

The respondent group was selected to representdiverse disciplines, various countries and social envi-ronments, and constrasting views on pharmaceuticalpolicy, and the proportions remained reasonably con-



sistent through all three rounds. The group remainingat the survey's end (after removing the 25.5% ofdropouts) had approximately equal representationamong the subgroups, which strengthens thesurvey's reliability. The higher rate of responsesamong physicians and pharmacists could reflect agreater involvement in pharmaceutical questions(Table 1).

The three rounds of questionnaires allowedrespondents to propose their own key issues andcomponents, which were used by the monitor teamfor proposing alternative choices. The commentsfrom respondents and their proposals for 40 addition-al key issues and 119 additional components suggestthat the group used the questionnaires to expressdivergent opinions. These factors, combined with thehigh rate of agreement, suggest that overspecifica-tion of the initial issues was not a serious problem.

The monitor team used the assessment docu-ments to present disagreements and minority viewsexpressed by respondents, and these documents weresent to respondents in Rounds No. 2 and No. 3. Thecontinuing high rate of responses, which reflects thelack of discouraged dissenters who would otherwisehave dropped out, also contributes to the reliabilityof the consensus.

Priority issuesThe key issues identified by the Delphi survey havebeen previously described in the literature on phar-maceutical policy as important problems faced bydeveloping countries. This study makes an importantcontribution in establishing the following five keyissues as priorities.

(1) Lack of government commitment and capacity todesign and implement a rational drug policy. Amajority of respondents ranked this key issue as toppriority for attention. This result emphasizes the crit-ical role of government in defining a drug policy,setting objectives, and implementing strategies.Agreement on this key issue supports the idea that acomprehensive drug policy is necessary to achievesignificant progress. A WHO situation analysis ofmore than 30 African countries in 1990 reached asimilar conclusion.a(2) Structural weakness and poor performance ofthe public drug sector. In the past ten years, manyaid projects have attempted to improve the perfor-mance of the public drug sector. But repeated fail-ures have convinced some agencies to decrease thepublic sector's role and to advocate a greater role for

a Bilan de la situation pharmaceutique des pays africains.Geneva, WHO Action Programme on Essential Drugs, 1990(unpublished report, in French only).

the private sector in the poorest countries (20). Thisstudy's consensus suggests that the public sector stillneeds to be strengthened in order to ensure drugavailability.(3) Irrational use of drugs The survey's 100%agreement on this key issue reflects growing aware-ness that the impact of cost-effective procurementand logistics systems will be significantly under-mined if drugs are not prescribed and used correctly.To date, only limited action has been taken in a fewcountries to address problems of irrational use.

(4) Limited affordability for essential drugs in bothpublic and private sectors. Consensus on this keyissue may reflect two broader concems in the inter-national community: first, that drugs in the privatesector are often unaffordable for poor people indeveloping countries; and second, that recent effortsto promote cost-recovery schemes in the public sec-tor through sales of drugs could similarly restrictaffordability of essential drugs for many poor peopleand could have adverse distributional effects.(5) Structural constraints. A strong consensusemerged that broader structural constraints in thecountry need to be considered, even though pharma-ceutical policy cannot resolve such constraints. Thisconsensus reflects increasing recognition that struc-tural constraints contribute to the failure of manyintemational aid projects (21). For those poor coun-tries that are too weak to sustain an effective drugpolicy on their own, interventions are necessary notonly on pharmaceutical matters but also on nationalconditions outside the drug sector.

Priority componentsSix key components were mentioned as priorities foraction by more than 70% of the respondents inRound No. 2 and were top-ranked as interventionsfor solving each key issue in Round No. 3. In thepast, some countries and donor agencies haveaddressed several of these components but rarely in acomprehensive policy package. This study suggeststhat future efforts to design drug policy and improvepharmaceutical conditions need to include all sixcomponents as a basic framework (with other com-ponents added according to each country's specificconditions).(1) The establishment of appropriate legislation andregulation was ranked as the first priority for gov-emnment action. Most developing countries and thedonor community have underestimated the problemsassociated with implementation of national policies.By ranking this component first, the respondentsacknowledged the importance of translating policyinto legislation and regulatory structures.

262 WHO Bulletin OMS. Vol 72 1994

Pharmaceutical policies in developing countries

(2) The selection of essential drugs was identified asa priority component for four key issues, reflectingstrong support from the respondent group. Essentialdrug lists have been recognized since 1977 as amajor tool for improving the drug situation in devel-oping countries, especially when combined withother policy elements. The results of this survey con-firm that the use of such lists is still a basic elementof appropriate drug policy in developing countries.

(3) The importance of maintaining a significantdrug allocation in the health budget was also empha-sized by the respondent group. In recent years, thecombined consequences of economic crisis andstructural adjustment programmes have frequentlyled to a decrease in drug budgets. The survey sug-gests that the state should remain financiallyinvolved in pharmaceutical provision.

(4) The improvement of procurement procedureswas ranked as an important priority for addressingthe poor performance of the public drug sector andfor improving drug affordability. Small additionalefforts in the area of procurement procedures canyield substantial improvements and savings.b Butmost developing countries have few pharmacistswith an expertise in purchasing low-cost drugs ofgood quality in the international market, and interna-tional aid projects have not generally focused on thisproblem.

(5) The establishment of a drug pricing policy wasselected as a priority component for improving thelimited affordability of essential drugs. This agree-ment suggests that pricing mechanisms should beintroduced to ensure that everybody - even thepoorest - has access in both private and public sec-tors to the drugs they need.

(6) Continuing education programmes have beenrecognized as an important intervention for promot-ing effective drug use (22), although they are oftendifficult to implement. The respondents' selection ofthis component reflects a need for greater efforts tounderstand prescribing behaviour and identify meas-ures to alter it in developing countries.

The strong consensus reached in this surveyargues that decision-makers in both internationaldevelopment agencies and national governmentsshould adhere to the six priority components indesigning and evaluating projects concerned withpharmaceutical policy. It is important to note, how-ever, that this Delphi study did not produce agree-ment on the policy content of each component.

b Financing essential drugs: report of a WHO Workshop, Harare,1988. Unpublished WHO document, WHO/DAP/88.10, 1988.

These often difficult choices need to be made at thenational level. Within individual countries, the Del-phi technique could be used to specify the contentsof national drug policy and create consensus amonglocal interest groups (including consumer associa-tions) involved with the pharmaceutical sector.

AcknowledgementsThe survey was sponsored by the Centre de Recherche etd'Etudes pour le DNveloppement de la Sant6 (CREDES),Paris, France, the WHO Action Programme on EssentialDrugs, and the Takemi Programme in International Health,Harvard School of Public Health, Boston, MA, USA. Theauthors thank the participants of the Delphi group, andalso the late U. Brinkmann, L. Chen, M. Garenne, D. Ross-Degnan, and R. Vaurs for comments on the manuscript.

ResumeEtablissement des prioritespharmaceutiques dans les pays endeveloppement: r6sultats d'une enqueteDelphiDans les ann6es 70, le manque de medicamentsde base a des prix abordables pour les popula-tions les plus d6munies est devenu un grave sujetde pr6occupation pour les nouveaux pays ind6-pendants du monde en developpement. En d6pitdes progres realis6s par certains d'entre eux et dudeveloppement de I'aide internationale, I'accessi-bilite aux medicaments essentiels continue deposer un probleme, notamment dans les pays lesplus pauvres. Les d6saccords qui ont persist6 surces questions au cours des annees 80 et I'absen-ce de consensus international sur les interven-tions prioritaires ont contribu6 a freiner la concep-tion et la mise en ceuvre de politiques pharma-ceutiques appropri6es.

Cet article rend compte de l'utilisation syst6-matique et logique de la technique Delphi pourd6gager un consensus parmi les experts interna-tionaux sur les objectifs prioritaires des politiquespharmaceutiques nationales dans les pays endeveloppement. Le groupe interrog6 initialementcomprenait 54 experts internationaux appartenanta divers types d'institutions et ayant des vues dif-f6rentes sur la politique pharmaceutique. L'enque-te a ete menee selon une approche logique entrois etapes. Elle visait a identifier les principauxproblemes auxquels sont confrontes les pays endeveloppement dans le secteur pharmaceutique("questions cl6s"), a d6terminer, pour chaquequestion cle, les 6l6ments du secteur pharmaceu-



tique qui ont une incidence majeure sur ses per-formances ("el6ments cles") et a classer cesquestions et ces 6l6ments en fonction de l'impor-tance des interventions, de fagon a etablir despriorit6s.

Le taux de r6ponse eleve, la qualite des parti-cipants et le degre de consensus exige ont confe-re aux resultats de l'enquete un haut degre de fia-bilite. L'ampleur de I'accord atteint sur la plupartdes questions montre qu'un consensus tres forts'est degag6 sur les priorites des politiques phar-maceutiques nationales. L'application syst6ma-tique de la technique Delphi a permis d'eviter lescontroverses que l'on observe depuis longtempsdans ce domaine. L'6tude a apport6 une contribu-tion importante en demontrant le caractere priori-taire des cinq questions cl6s suivantes: absencede volont6 et incapacit6 des gouvernements aconcevoir et a mettre en ceuvre une politiquepharmaceutique rationnelle; utilisation irrationnelledes medicaments; difficult6s pour se procurer lesmedicaments essentiels, tant dans le secteurpublic que dans le secteur prive; contraintes struc-turelles. Plus de 70% des participants ont egale-ment mentionn6 six 6l6ments cl6s qui, selon eux,justifiaient une intervention prioritaire: 6tablisse-ment d'une legislation et d'une r6glementationappropri6es; choix des m6dicaments essentiels;maintien d'un poste "m6dicaments" importantdans le budget de la sante; am6lioration des pro-cedures d'achat des m6dicaments; 6tablissementd'une politique de fixation des prix; poursuite desprogrammes d'education.

Les conclusions de l'6tude ont des implica-tions importantes pour les d6cideurs au sein desagences de d6veloppement internationales et desadministrations nationales. Cette 6tude offre eneffet un cadre g6neral pour la conception d'unepolitique pharmaceutique dans les pays en deve-loppement. 11 importe toutefois d'observer qu'ellen'a pas permis d'arriver a un accord sur la placede chaque element dans la politique g6nerale. 11s'agit la d'un choix souvent difficile qui doit etrefait au niveau national. Dans chaque pays, latechnique Delphi pourrait etre utilisee pour speci-fier le contenu de la politique pharmaceutiquenationale et cr6er un consensus parmi lesgroupes d'interets locaux (y compris les associa-tions de consommateurs) qui oeuvrent dans lesecteur pharmaceutique.

References1. The selection of essential drugs. Report of a WHO

Expert Committee. Geneva, World Health Organiza-

tion, 1977 (WHO Technical Report Series, No. 615).2. Report of the Expert Committee for Drugs - Ban-

gladesh. World development, 1983, 11: 251-257.3. The Philippine national drug policy: a primer. Mani-

la, Department of Health, 1987.4. National drug policy for Nigeria. Lagos, Federal Min-

istry of Health, 1990.5. The world drug situation. Geneva, World Health

Organization, 1988.6. Reich MR. Essential drugs: economics and politics

in international health. Health policy, 1987, 8: 39-57.7. The rational use of drugs: Report of the Conference

of Experts, Nairobi. Geneva, World Health Organ-ization, 1987.

8. Chetley A. A healthy business? World health andthe pharmaceutical industry. London, Zed Books,1990.

9. Lindstone HA, Turoff M, eds. Introduction. In: TheDelphi method: techniques and applications. Read-ing, MA, Addison-Wesley, 1975: 3-12.

10 Lindeman CA. Priorities within the health caresystem: a Delphi survey. American Nurse's Associa-tion, 1981, 5: 1-49.

11. Fink A et al. Consensus methods: characteristicsand guidelines for use. Am. j. public health, 1984,74: 979-983.

12. Farrell P, Scherer K. The Delphi technique as amethod for selecting criteria to evaluate nursingcare. Nursing papers, 1983, 15(1): 51-60.

13. Gourley RD et al. ASHP members' concepts ofinstitutional pharmacy in the year 2000. Am. j. hosp.pharm., 1985, 42: 96-101.

14. Rinaldi RC et al. Clarification and standardizationof substance abuse terminology. J. Am. Med.Assoc., 1988, 259: 555-557.

15. Thompson DF, Heflin NR. Frequency and appropri-ateness of drug prescribing for unlabeled uses inpediatric patients. Am. j hosp. pharm., 1987, 44:792-794.

16. Jillson IA. The national drug-abuse policy Delphi:progress report and findings to date. In: LindstoneHA, Turuff M, eds. The Delphi method: techniquesand applications. Reading, MA, Addison-Wesley,1975: 124-159.

17. Rohatgi K, Rohatgi PK. A Delphi study on healthin future India. Journal of scientific and industrialresearch, 1980, 39: 359-363.

18. Pineault R, Daveluy C. L'approche par recherchedu consensus. In: La planification de la sante: con-cepts, m6thodes et strat6gies. Montreal, Agenced'ARC, 1986: 226-246.

19. Delbecq AL, Van de Ven A, Gustafson DH. Grouptechniques for program planning. Glenview, IL,Scott, Foresman, 1975.

20. The World Bank. World development report 1991.The challenge for development. New York, OxfordUniversity Press, 1991.

21. Reich MR, Marui E, eds. International cooperationfor health: problems, prospects and priorities. Dover,MA, Auburn House, 1989.

22. Soumerai SB, McLaughlin TJ, Avorn J. Improvingdrug prescribing in primary care: a critical review ofthe experimental literature. Milbank Memorial Fundquarterly, 1990, 67: 268-317.

264 WHO Bulletin OMS. Vol 72 1994

Documents

developing countries: results of a Delphi survey