Decision Diagram

7/29/2019 Decision Diagram

http://slidepdf.com/reader/full/decision-diagram 1/23

Decision diagram

DETAILED PRESENTATION OF THE TOOL What is a decision diagram?

What does the decision diagram represent?

Where does the decision diagram come from?

What shape can it take?

Why and when should the decision diagram be used? When should the decision diagram be used?

What are its advantages and limitations?

What are the preconditions for its use?

What are the decision diagram's construction stages? Stage 1: determination of the appropriate moment for decision-making

Stage 2: wording of questions

Stage 3: collection of useful information

Stage 4: constructing the temporary diagram

Stage 5: testing of the temporary diagram

Stage 6: constructing the final diagram

How should the decision diagram be used? How many diagrams should be established?

How should the decision diagram be used?

How are the findings presented?

What are the preconditions for the construction of a decision diagram?

Quality control What are the key issues to control?

How should the quality grid be used?

Examples Bibliography

http://ec.europa.eu/europeaid/evaluation/methodology/tools/too_dcs_def_en.htm


http://ec.europa.eu/europeaid/evaluation/methodology/tools/too_dcs_whe_en.htm


http://ec.europa.eu/europeaid/evaluation/methodology/tools/too_dcs_how_etp_en.htm


http://ec.europa.eu/europeaid/evaluation/methodology/tools/too_dcs_how_com_en.htm


http://ec.europa.eu/europeaid/evaluation/methodology/tools/too_dcs_qua_en.htm


http://ec.europa.eu/europeaid/evaluation/methodology/tools/too_dcs_exe_en.htm


http://ec.europa.eu/europeaid/evaluation/methodology/tools/too_dcs_bib_en.htm











What is a decision diagram?

What does the decision diagram represent? o

General definition

o Decision factors Where does the decision diagram come from?

o Various fields of application o Reasons for its use o Can the diagram be combined with other tools?

What shape can it take?

WHAT DOES THE DECISION DIAGRAM

REPRESENT?

General definition The decision diagram shows the process during which the strategic objectives and the overall policies of

cooperation with developing countries, which are defined by the European Union's assistance

agreements, are converted into short-term and medium-term bilateral co-operation decisions.

It illustrates the successive official and informal decisions, resulting in the programming of development

assistance.

Decision factors Two types of decision factors can be found:

Factors included in the decision process, each selected objective justifying the objectives at a

lower level

External factors influencing the decision-making

External factors appear during the decision-making process as a flow of information originating from the

European Union's institutions, or other sources (strategic trends, context of the country under

consideration, governmental policies, position of the Member States and other donors, etc.). They are

called inputs.

WHERE DOES THE DECISION DIAGRAM COMEFROM? Various fields of application Decision-making is an important theme in management theory. Various tools are proposed for the

determination of public policies (such as in the field of defence), medical and managerial decisions.

They usually lead to a series of choices for alternative solutions which have to be optimised by

rationalisation tools, such as cost-effectiveness analysis.

http://ec.europa.eu/europeaid/evaluation/methodology/tools/too_dcs_def_en.htm#01


http://ec.europa.eu/europeaid/evaluation/methodology/tools/too_dcs_def_en.htm#01_01














http://ec.europa.eu/europeaid/evaluation/methodology/tools/too_cef_som_en.htm














In the analysis of the strategic decision-making process, the retrospective use of such a tool was

suggested for the first time (in a simplified version) for the evaluation of the European Commission's

strategic co-operation with Egypt . The tool, however, was not exploited fully during this evaluation.

Reasons for its use In the management field, the role of the decision diagram is to rationalise the decision-making process

and to make it more efficient for the resolution of problems and the achievement of objectives. It

provides an answer to the following question:

Which elements must be taken into account to make the most appropriate operational

decision, with regard to the intended objectives and the available means?

Decision diagrams can be supported by a variety of software (computer devices), providing the user with

guidance throughout the decision-making process. Thus, decisions diagrams are, by nature, a

normative and simplifying tool.

The evaluator needs a tool which describes a complex approach with undefined rationale and severalstakeholders with different aims. Such approaches are more focused on consensus than efficiency, and

do not necessarily rely on any rationalisation of decision-making.

To find out more:

What are the diagram's advantages and limitation?

Can the diagram be combined with other tools? The decision diagram can be directly combined with the objectives diagram and effect diagram (and

indirectly with the problem diagram). It was originally designed to overcome the deficiencies of theobjectives diagram, but it is also useful in itself.

Its construction can be supported by a SWOT analysis and/or a socioanthropological analysis, in which

information about the context of the societies affected by the strategy is provided.

WHAT SHAPE CAN IT TAKE? The fishbone shape can be used to illustrate the three main components included in decision-making.

Thus, a theoretical decision diagram for the European Commission's country development assistance

will present:

In the central axis of the diagram, the planning chain of planning decisions which goes from

the most general orientations to field decisions, including medium-term strategic and political

decisions, and operational decisions

On the left side of the diagram, the flow of inputs originating from the European Union's

institutions(Council, Parliament, Commission), such as decisions, positions, communications,

directives, lessons learned by the implementation of previous strategies and programmes,

review of projects in process or in the portfolio

On the right side of the diagram, the flow of other inputs, such as events or major turning

points in the world situation or a specific region, evolution and need of the country partners,

and the position and interventions of other donors




http://ec.europa.eu/europeaid/evaluation/methodology/examples/exp_too_dcs_one_en.pdf








http://ec.europa.eu/europeaid/evaluation/methodology/tools/too_obj_som_en.htm



http://ec.europa.eu/europeaid/evaluation/methodology/tools/too_dpm_som_en.htm






http://ec.europa.eu/europeaid/evaluation/methodology/tools/too_swo_som_en.htm



http://ec.europa.eu/europeaid/evaluation/methodology/tools/too_asa_som_en.htm


















Standard decision diagram (fishbone shape)

The diagram illustrates the decision-making stages and the main external flow of information supporting

the drafting of the strategic or political decision.

To find out more:

When should the decision diagram be used?






Why and when should the decision diagram beused?

When should the decision diagram be used? o During the design of the strategies o During the evaluation of the strategies

What are its advantages and limitations? o Advantages o Limitations

What are the preconditions for its use? o Existence of an objectives diagram o Existence and relevance of the information o Tight collaboration with the European Commission's services

WHEN SHOULD THE DECISION DIAGRAM BEUSED? During the design of the strategies The European Commission's external intervention strategies result from a complex procedure, taking

into account the contextual elements as well as the position of the stakeholders.

This procedure is usually empirical and lacks transparency, although more rationalisation and

explanation may lead to more appropriate decisions (selection of priorities, adoption of programmes).

These decisions could have a positive influence on the country partners' ownership of the strategies.

Using a decision-making tool such as the decision diagram could facilitate the organisation of the design

process, particularly for the selection of key information, the participation of stakeholders in the process,

and the management of the implementation arrangements.

This methodology, however, focuses on the tool's usage in the particular context of the evaluation

(during the organisation stage). Its use in the strategy design stage requires an adaptation of the

methodology.

During the evaluation of the strategies Illustration of justified choices Co-operation strategies and intervention policies designed by national and multinational donors have

explicit and implicit objectives. The documents which present them usually display a range of objectives

more or less in order (see the complex objectives diagram), and among which one or occasionally more

overall objectives, operational objectives relative to development assistance projects, and a range of

intermediary objectives at various levels can be identified.

Thus, the operational items of a programme (planned projects and/or in process) which depend on the

strategic or political papers, are supported by the objectives system given in these papers.

In an objectives diagram, the selection of a higher-order objective and its division into several

intermediary objectives, down to the operational objectives often results from non-explicit choices.

Unless decisions are deemed arbitrary, the selection carried out by the authors of the strategic and

programming documents depends on various sources:

http://ec.europa.eu/europeaid/evaluation/methodology/tools/too_dcs_whe_en.htm#01


http://ec.europa.eu/europeaid/evaluation/methodology/tools/too_dcs_whe_en.htm#01_01





















http://ec.europa.eu/europeaid/evaluation/methodology/tools/too_obj_def_en.htm

















Recommendations of the European institutions (Council, Parliament, Commission)

Analyses of the external events (major events, country situation)

Intervention of non-European Commission actors (partner governments, Member States, other

donors)

Lessons learned in previous programmes or projects

The decision diagram illustrates the impact of these information flows on the successive choices (i.e. the

progressive elimination of alternative options) made by the decision-makers. Indeed, each box of the

diagram's central column represents a choice (selected and rejected objectives), while the boxes on

each side illustrate the flow of inputs which represents the external justification for these choices.

A strategic sequence and an operational one in the decision

Strategy papers are established for each 7 year budgetary period. They deal with the events,

orientations and changes having a significant and long-lasting impact, which can be worldwide (such as

the attack on the World Trade Centre in 9/11/2001) or regional (the second Intifada in 2000, or the

military invasion of Iraq in 2003). If such events happen during the strategic period, strategy papers can

be revised (for example, in the mid-term revision). However, these documents usually remain

unchanged until the end of their term.

Programme durations are sometimes shorter than strategy papers (usually 2 to 3 years). They can be

affected by substantial changes in the political orientations of the partner government, natural disasters

(typhoons, droughts), the signature of association or co-operation agreements, political shifts in the main

donors' activities, etc. The revision of existing programmes or the drafting of new programmes is

common in the programming activity during the budgetary period.

In addition to a short presentation of the whole decision-making process (in the form of a summary

diagram), the evaluator should divide the theoretical diagram (standard decision diagram) in two. This

should result in a strategic decision diagram on one hand, and an operational decision diagram on the

other hand.

Standard strategic decision diagram







Standard operational decision diagram

Such a separation is interesting because it l imits the number of boxes in each diagram and improves

their readability.

Evaluation of the decision process

The design of the strategy and programming papers is theoretically collaborative. These documents are

the subject of negotiations between the European Commission and the partner governments of the ACP

regions.

In other regions, programmes are the subject of an agreement between partners, whereas the

communication of strategies is for information only.

Member States must be consulted on all strategy and programming documents. Practice shows,

however, that this collaboration is sometimes only formal.

As a consequence, the European Commission insists on an in-depth examination of the decision

procedure which could be part of a country evaluation, or stand as a thematic evaluation on its own. In

both cases, the decision diagram is a very relevant tool.

WHAT ARE ITS ADVANTAGES ANDLIMITATIONS? Advantages Clarification of the strategy

The diagram highlights:



The range of options for the establishment of the objectives system (overall objective, selected

and rejected intermediary objectives, etc.)

The external events influencing the decision-making

Complementing the objectives diagram and effect diagram, the decision diagram facilitates the analysis

of the strategy in terms of internal coherence (logical succession of the choices) and external relevance

(contextual elements and position of the stakeholders).

Illustration of the main partners' interventions and the analysis of the 3 Cs

When the terms of reference of an evaluation require an analysis of the partnership, the diagram is used

to highlight the intervention of the main partners (governments, Member States and other donors) in the

strategy design, the establishment of the programmes and the selection of the projects.

The diagram can perform the same role for the analysis of the 3 Cs (Coherence, Co-ordination,

Complementarity).

Limitations The major limitations in the use of the diagram are:

The availability of information (stage 3)

Uncertainties about causal links

Baseline information

Direct information about the factors influencing the strategic and programming drafting process is

sometimes scarce in the off icial papers which present the strategies and programmes. Other public

documents, such as evaluations, can provide indications but the information they provide is limited. The

restricted documentation (studies, notes, correspondences) is in principle more informative, but often

cannot be consulted, or its access is difficult.

Verbal information has the same limitations concerning availability and accessibility.

Causal interpretation

Even when potentially informative elements are provided (for example, the intervention of other donors,

or the priorities of the government), causal relations between decisions are not always explicit. An

interpretation is sometimes required, which can be the source of a risk of error, particularly in cases

where several causes support the decision-making.

WHAT ARE THE PRECONDITIONS FOR ITS

USE? Existence of an objectives diagram

The presence of an objectives diagram in strategy or programming papers is a precondition for the

construction of a decision diagram. If such is not the case, the evaluator will need to construct an

objectives diagram.

Existence and relevance of the information







http://ec.europa.eu/europeaid/evaluation/methodology/tools/too_dcs_how_etp_en.htm#03












The availability (effective or to be assured) of reliable written or verbal information (stage 3) is also an

important precondition. This information needs to be sufficient and allow verification.

Tight collaboration with the European Commission's services

A trustworthy collaboration between the evaluators and the European Commission's services

(competent services of the DG Development, DG External Relations, and Delegations) would guarantee

the access to the information and its interpretation.







What are the decision diagram's constructionstages?

Stage 1: determination of the points when the decisions were made o General principle o Determination of the period allocated for the selection of objectives

Stage 2: drafting of questions o Identification of the rejected objectives o Wording of questions

Stage 3: collection of useful information o What is useful information? o What specific information is required for country evaluations? o How is useful information collected? o Is the information collected sufficient?

Stage 4: constructing the temporary diagram o Preliminary analysis of the information o Draft of the explanations o Construction of a temporary diagram

Stage 5: testing of the temporary diagram Stage 6: constructing the final diagram

Once the preconditions are fulfilled, the drafting process of the decision diagram continues in two steps:

Determination of the points at which the decision-making took place (selection of the

objectives)

Identifying, collecting and analysing of the relevant information

The process cannot start with the collection of documentation because useful information cannot be

found in one or more identified documents. It has to begin with the definition of the scope of such a

collection.

The determination of the objectives at various levels (overall, intermediate and operational) implies a

series of choices which can be translated into a question: why such an objective and not another one?

The question's wording therefore orients the collection of the information judged to bring answers. The

relevance of these answers is checked, and may result in seeking new information (written or verbal),

and even rewording some of the questions, particularly those concerning the objectives which are

rejected.

STAGE 1: DETERMINATION OF THE POINTSWHEN THE DECISIONS WERE MADE General principle The decision diagram (existing or to be constructed) identifies the points at which the implicit or explicit

selection of the objectives were made.

Whilst the objectives diagram assesses the internal coherence of the objectives system, the decision

diagram illustrates the external factors corresponding to each choice in the decision-making and

resulting in the selection of one objective over another.



http://ec.europa.eu/europeaid/evaluation/methodology/tools/too_dcs_how_etp_en.htm#01_01



































http://ec.europa.eu/europeaid/evaluation/methodology/tools/too_obj_how_en.htm
































Determination of the period allocated for the selection of objectives Usually, the establishment of the decision diagram follows the construction of the objectives diagram.

This sequence should be respected.

The objectives diagram is the basic tool for identifying the points at which the decision-making takes

place. Apart from the definition of the overall objective, each intersection of the diagram represents a

decision-making point. For example, in the objectives diagram, 3 intersections are illustrated, which

stimulates the following questions:

Why have these objectives been selected? Why have others been rejected?

The decision diagram questions the choices concerning:

The initial selection of the overall objective

The subsequent selection of the intermediate and operational objectives

The planning process should be both strategic (implementation of the best means to achieve objectives

of different levels) and empirical (selection of the means likely to achieve specific objectives). Whatever

the planning process, the choices will be made in accordance with the objective diagram.

Each of the decision-making points should be given an identification code (which may include a date), to

facilitate its transfer to analytical tables (see stage 3).

STAGE 2: DRAFTING OF QUESTIONS Identification of the rejected objectives The questions focus on the justification of the selection of objectives. The identification of the rejected

objectives helps the accuracy of the wording of questions. How are these objectives identified? At each

decision-making point, the evaluator may encounter four situations:

Strategy and planning papers explain why certain objectives have been rejected and identify

reasons to support this decision.

Working papers (interim versions, notes and various correspondences) provide elements to

pinpoint rejected objectives and may justify the choices made (in principle, they always include

the positions of Member States and the response of the European Commission). The documentation available does not explain the choices made, but interviews with the

decision-makers provide relevant information.

The evaluators have not collected any written or verbal information relevant to the explanation

of the choices made.

In the first three situations, the evaluators should investigate which of the objectives were planned but

eventually rejected, and the reasons for rejection.

In the last situation, the identification of the objectives will have to be made by the evaluation team, or

with the help of external expertise. The propositions developed this way can be supported by:

Information and analyses stated in the available documentation (for example, about thebeneficiary country's situation or other donor commitments)

Examples from similar countries















Wording of questions Whatever the situation, the outcome is the answer to the following question:

Why such an objective has been rejected?

Several questions of this type may be needed at each decision-making point.

STAGE 3: COLLECTION OF USEFULINFORMATION

Before the construction of the diagram, the evaluation manager or, failing that, the evaluation team

should check the quality of the sources of information. If this verification reveals poor or unreliablesources, the construction of the diagram should be abandoned.

What is useful information? Useful information, written or verbal, should not be identified in a too limited way, nor in a

comprehensive way. Indeed, the identification of appropriate information for any type of evaluation can

only be stated in general.

Written information

Usually, written information can be found in four types of documents.

Strategy papers and programmes can provide information about the lessons learned from

previous implementations, the political, economic and social context, and the interventions of

Member States and the main donors.

The preparatory documentation (meeting reports, notes at the end of preparatory missions,

correspondence, internal notes or notes between services) may provide explanations about the

priorities chosen.

Country/thematic/sector-based and project evaluations can be valuable in terms of lessons

learned.

The European Council and Commission's more general documents (conclusions,

recommendations, reports), as well as the international treaties and agreements often display

the contextual elements, lessons learned and priorities which are known to the designers of

strategies and programmes.

Verbal information

To complement the written information, the evaluator can ask the authors and contributors of the main

documents (strategy papers and programmes) to explain the reasons for their choices.

What specific information is required for country evaluations? Written information

The following list, relating to the kind of available documents and the nature of their information, is not

intended to be comprehensive.

The final version of the Country Strategy Paper (CSP) indicates the priorities and the finalprogramming; sometimes, it describes the lessons learned from previous implementations, the



economic and social problems of the country, and the priorities of the beneficiary government;

it usually includes reference to co-operation agreements of the Member States and main

donors. The successive versions of the CSP may reveal the priorities and programming

intentions which were rejected.

Meeting reports and notes following preparatory missions may provide justification for the

determination of the priorities and programming.

Documents produced during the drafting of the documentation, correspondences, internal

notes, and notes between services may specify and explain the resolution of the priorities and

programming.

Documents describing the reactions of Member States to the drafts addressing the choices

made, and the need for explanation, can highlight the types of questions which were debated.

The European Commission's compulsory reply to the Member States' questions shows the

justification for the choices.

Documents collecting the partner government's reactions to the projects may explain

governmental priorities.

Thematic, sector-based and project evaluations may include lessons learned and

recommendations, which explain the choices made.

Mid-term revisions are supposed to review the analyses underpinning the selection of the

priorities and programming. Specific bilateral and multilateral agreements (co-operation, association) and reports from joint

monitoring yield details about the common or specific priorities of the two partners.

Most of these documents can be found for the strategies established after 2001. Prior to this date, the

situation is more challenging because the documents are not routinely made available, even when the

relationship between the evaluators and the Commission's services is productive.

Verbal information

In addition to - or in substitution for - written information, the authors and contributors to the main

documents (strategy and programming papers), as well as the decision-makers responsible for the

drafts in process, may be valued informants, providing that they are available and in post for a

significant period in the head office (DG Dev and DG Relex, EuropeAid), or in one of the delegations.

How is useful information collected? Collection of written information

Written information should be collected during the preliminary stage of the country evaluation from the

relevant services of the Commission, such as the DG Relex and DG Dev (country and regions), and in

the European Commission's Delegation in the country under consideration.

Collection of verbal information

The main interlocutors are the managers of the services previously described. If they are unavailable,

former managers can also be interviewed if they can be easily contacted.

Is the information collected sufficient? If the evaluation managers have not identified a list of information sources, the evaluation team should

formulate its conclusions about the quality of the sources in terms of quantity, relevance, reliability and

accessibility. This judgement is presented to the managers, who take the f inal decision.

STAGE 4: CONSTRUCTING THE TEMPORARYDIAGRAM Preliminary analysis of the information



Preliminary selection of the relevant information is carried out on the basis of the objectives diagram. It

requires:

The establishment of one or more timelines, describing the successive events and/or

information included in the drafting process of the strategy and the programme

The selection of texts from the documentation collected, relative to the choice of the objectives

and the scheduled assistance process, or relative to the factors influencing such choices

The classification and the construction of an index for the texts

Following the analysis of the texts, additional information resulting from the documentation collected

may be required, or new documents could be requested and investigated (this method yields uncertain

findings).

Drafts of the explanations Provisional answers to the evaluation questions are formulated on the basis of the information collected.

Some of the questions may not be answered at this stage.

A table of questions can illustrate the results of the information analysis and take the following shape:

Table A: results of the collected information analysis Code for the

decision-making

Question(wording of the

choice)

Answer(justification of the

choice)

Identification of theoriginal text

In Table A, the columns gather data concerning:

The decision-making point: determination of a code highlighting the decisions chain and the

successive decision-making point. This code can be the date when the choice has been made.

Questions about the justification of the decision: each decision-making point corresponds to

several questions about the rejected and selected objectives

Answers to questions: justification of the choices determined by the strategies and

programmes

The identification of the original texts refers to the texts which have been selected and coded

Construction of a temporary diagram The objectives diagram supports the decisions chain. It identifies four (sometimes five) decision-making

points dealing with:

The overall objective

The first row intermediary objectives

The secondary row intermediary objectives

Operational objectives

Most of the time the drafting of strategies and programmes is not strictly and exclusively driven by such

a rationale and is, in fact, often empirical. Implicitly or explicitly, the designers of the strategy and

programme begin with an overall objective. Thereafter, they examine which means at the disposal of the

European Commission are able to achieve this objective. The successive choices can be synthesised in

two points:







The choice of the overall objective

The choice of the fields and the intervention processes

Preceding each point, and on both sides of the decisions chain, flows of inputs identified from

the analysisof the written and verbal information collected, can be found.

At the left of the sketch, a magnifier explains the decision's outcome:

Selected objectives

Rejected objectives

A global fishbone shaped diagram is thus completed through detailed diagrams corresponding to each

decision-making point.

Sketch of the moment of the decision-making: example of first row intermediary objectives










STAGE 5: TESTING OF THE TEMPORARYDIAGRAM The decisions and their explanation must be confirmed by the main actors responsible for the drafting of

the strategy and the programming, including the European Commission's services (head office anddelegations), the representatives of the other stakeholders (Member States, NGOs, etc.), the usual

interlocutors in beneficiary countries and/or their government.

The observations of the respondents can be recorded in the following table.

Table B: Results of the testing of the temporary diagram

Date and code forthe decision-

making

Decisions madeaccording to the

temporarydiagram

J ustificationbased on thetemporarydiagram

Observations bythe decision-

makers consulted

In Table B, the columns gather data concerning:

The decision-making point: identification of the decision-making within the decisions chain

(identical code to the first column of Table A)

Decisions made according to the temporary diagram: determination of the selected and

rejected objectives which have been subjected to the questions presented in the second

column of Table A

Justification based on the temporary diagram: answers to the questions justifying the decisions

set out in the strategies and programmes third column of Table A

Observations by the decision-makers consulted: approval (with or without reservations) or

rejection of the decisions and their justification

STAGE 6: CONSTRUCTING THE FINALDIAGRAM If the observations of the decision-makers consulted are not critical, the temporary diagram is corrected

incorporating the observations. This operation produces the final diagram.

The process of testing of the temporary diagram may question some of its parts when the justifications

do not illustrate the real strategic and programming drafting process.

In this case, the information should be reviewed and augmented by another consultation round. The

evaluators and the evaluation managers should decide whether they need to construct a new temporary

diagram and test it. Indeed, the observations collected during the test may be sufficient to avoid another

consultation and to establish the final diagram directly.

The final and temporary diagrams have the same shape (a main diagram and the sketch of the point of

decision-making). The final diagram includes an explanatory table about the analysis of the information

collected.































How should the decision diagram be used?

How many diagrams should be established? o

Diagram of strategic decisions

o Diagram of operational decisions How should the decision diagram be used?

o Recommendations for country evaluations o Evaluation of the relevance of the strategy and planning o Evaluation of the quality of the partnership o Evaluation of the 3Cs o Additional tools

How are the findings presented? o Intermediary documentation o Final report o Presentation of the final study

What are the preconditions for the establishment of a decision diagram? o Human resources and working arrangement o Travelling expenses o Computer devices

HOW MANY DIAGRAMS SHOULD BEESTABLISHED? Diagram of strategic decisions Usually, the number of the diagrams to be constructed should correspond to the number of the strategy

papers drafted during the evaluation period.

Evaluators and evaluation managers may decide to work on the most recent document only, when

required. Indeed, some of the early documentation needed for the establishment of the decision diagram

may be too old to guarantee its availability and that of its authors.

Whatever the case, it is recommended that a decision diagram for each possible medium-term strategic

revision is prepared.

Diagram of operational decisions Such a diagram is usually prepared for each development assistance programme. As strategy papersmay cover several programmes, the evaluators may have to study 4 or 5 programmes for an evaluation

in a twelve-years timeframe.

The construction of such a large number of diagrams represents a significant amount of work and cost,

and yields uncertain results, being dependant on the availability of the information (documents and

informants). Thus, the evaluators and evaluation managers may agree to focus on programmes relating

to the most recent strategy paper.

To find out more:

Simple diagrams: when should the decision diagram be used?

What are its advantages and limitations

http://ec.europa.eu/europeaid/evaluation/methodology/tools/too_dcs_how_com_en.htm#01


http://ec.europa.eu/europeaid/evaluation/methodology/tools/too_dcs_how_com_en.htm#01_01


































































HOW SHOULD THE DECISION DIAGRAM BEUSED? Recommendations for country evaluations The general recommendations about the use of decision diagrams are derived from the experience

drawn from their use in the particular context of country evaluations.

Decision diagrams should be established whenever the evaluation studies the relevance of the

objectives with regard to the European Union's goals, the country specificities and the overall context.

Previous evaluations show that, excluding extreme situations, objectives and interventions are usually

relatively coherent, although the appropriateness of the choices made is not always indisputable. The

decision diagram is likely to alleviate this particular limitation.

The use of decision diagrams in thematic and sector-based evaluations appears possible, but should be

tested, and adapted where necessary.

Evaluation of the relevance of the strategy and planning The explanations and justifications for the choices stated in the documentation and from informants

should be evaluated.

Thanks to their experience or with the support of external experts, evaluators should formulate their own

views about the relevance of the explanations which are provided for each decision-making choice.

They should be able to judge to what extent the rejection of an objective seems justified.

The selection of the overall objective (following the rejection of other possible overall objectives) should

be consistent with the strategic goals formulated by the European Union's institutions (Council,

Parliament, Commission). The justification of the subordinate objectives is based on coherence with the

overall objective or the ones resulting from it, and with the decision chain's inputs.

Evaluation of the quality of the partnership The partnership between the country's authorities and the representatives of the Member States can be

evaluated through the study of specific input flows.

Thus, a judgement can be formulated about the importance given to the following topics during the

strategy and planning decision process:

Member States' strategies, programmes and agreements

Member States' opinions about the European Union's interventions

Priorities and agreements of the partner government

Partner government's wishes about the European Union's interventions

Evaluation of the 3Cs The following can also be analysed:

The coherence of the development assistance strategy with other European policies

(migration, trade, agriculture, etc.)

The coordination between the co-operation policies of the Commission and Member States



















The complementarity of the Commission's programme planning with that of other donors

(including the EIB)

Additional tools These tools can be used when the decision-making factors are not explicit in the documentationcollected, or when their interpretation is uncertain.

They are especially useful for evaluations where the terms of reference explicitly include an analysis of

the decision-making process (thematic evaluations about the decision-making process).

The evaluators can present their successive judgements in a series of tables completing the diagram.

Validation of the diagram by the decision-makers consulted during the test

Table C: Evaluation proposed for the selection of objectives Code for

thedecision-making

Decisionsmadeaccording to

the finaldiagram

Justificationbased on thefinal diagram

Observations bythe decision-

makers consulted

Judgementof the

evaluators

In Table C, the columns gather data concerning:

The decision-making point which is identified in the sequence of the decision-making (the code

is the same as the one chosen in the first column of Table A (stage 4))

Decisions made according to the final diagram: determination of the selected and rejected

objectives

Justification based on the final diagram: includes the findings of the test (reported in the fourth

column of Table B (stage 5))

Observations by the decision-makers consulted: approval (with or without reservation) or

rejection of the decisions and their justification

The judgement of the evaluators: agreement (with or without reservation) or rejection of the

observations made by the decision-makers

Validation of the diagram by an expert panel including the Commission's decision-makers and independent experts

The evaluation may be subjected to a panel composed of representatives of the Commission's services(geographic departments, delegations), and independent and recognised experts. The findings could be

reported in the following table:

Table D: Final evaluation of the selection of objectives Code for the

decision-making

Decisions madeaccording to the

final diagram

Opinions infavour of thefinal diagram

Opinionsagainst the

final diagram

Judgment of the

evaluators







http://ec.europa.eu/europeaid/evaluation/methodology/tools/too_pan_som_en.htm








In Table D, the columns gather data concerning:

The decision-making point, which is identified in the sequence of the decision-making (the code

is the same one chosen the first column of Table A (stage 4))

Decisions made according to the final diagram: determination of the selected and rejected

objectives (second column of Table C)

Opinions in favour of the final diagram: opinions of panel members relating to the final diagram

Opinions against the final diagram: opinions of panel members relating to the final diagram

The judgment of the evaluators: agreement (with or without reservation) or rejection of the

observations made by the decision-makers

To find out more :

When should a decision diagram be used?

What are its advantages and limitations?

HOW ARE THE FINDINGS PRESENTED? Intermediary documentation The intermediary documentation is addressed to the managers of the evaluation and the steering

committee. It must include all the graphs and tables in detail.

Final report The final report is addressed to a wider public, more interested in the findings than the evaluation's

methodology.

It is recommended that analytical graphs and tables should be placed in an annex and a synthesis of the

findings should be shown in the main report under three headings:

Relevance of the strategic and planning decisions

Description of the partnership between Member States and the beneficiary government

Coherence, Co-ordination and Complementarity (3 Cs)

A simplified diagram of the strategic and operational decisions may usefully be included in the main

report.

Presentation of the final study It may be useful for the evaluators to use the standard decision diagram during the presentation of the

evaluation's findings.

To find out more:

How many diagrams should be established?

WHAT ARE THE PRECONDITIONS FOR THE

ESTABLISHMENT OF A DECISION DIAGRAM?
























Human resources and working arrangements

Type of work required for the design of a decision diagram Tasks Types of abilities

Categories of professionals

Number of working

days(estimation)

Identification

Collection of thedocumentation

Examination of thedocumentation

Construction of thediagrams

Knowledge of theEuropeanCommission'sstrategies anddevelopmentprogrammeprocedures

Knowledge of thedocumentation

resulting from theprocedures

Fair knowledge of computer tools

Junior or mediumprofessionals

5-10 days

Analysis of thedecisionsidentified duringthe examination of documentation

Reconstruction of the diagrams

Test of the

diagrams

Establishment of the final diagrams

Logical process of thinking

Experience in thefields covered bythe strategies andprogrammes

Specific knowledgeof the country,

sector or themeunder study

Multidisciplinary teamof experiencedevaluators, whosespecialities shouldcover the thematicscope of theevaluation

5-10 days

Freelance experts inspecific fields

5-10 days

Travelling expenses Strategy papers used to be prepared under the responsibility of the Commission's Directorates-General

(Relex or Dev) which are also responsible for the planning stage.

EuropeAid is in charge of the project design. Thus, the majority of the useful documentation (the whole

baseline documentation) can be found at the Commission's Headquarters in Brussels. The delegations

have taken part in the drafting process of the most recent documents and their responsibilities in this

area will grow.

In addition, decision-makers from ACP countries benefiting from European assistance may in principle

take part in the drafting process of the strategies and programmes which have been co-signed.

Computer devices Specialist software can support decision-making. Yet, these devices do not seem relevant to the

evaluation context, where spreadsheet programs or PowerPoint should be sufficient.



Quality control Check-list for evaluators Check-list for managers

CHECK-LIST FOR EVALUATORS

Questions Answer

Preparation and design

Has a preliminary assessment of the nature and quality of the information been

carried out by the managers of the evaluation?

If not, has the evaluation team carried out such an assessment, which has been

approved by the managers?

Construction of the diagram

Have the choices of objectives been clearly identified in the objectives diagram?

Is the decision-making process clearly established within the decision chain?

Does the available documentation provide information about rejected objectives

and the reasons for these rejections?

Do the interviews with the authors of the documentation compensate the

deficiencies of the written resources?

Do the questions of interviews logically result from the identification of both the

rejected and selected objectives?

Does the temporary diagram explicitly present the results of choices (objectives

selected and rejected)?

Has the test of the temporary diagram been conducted with the main

stakeholders (authors, operators, partners)?

Have the findings of the test been impartially taken into account in the drawing

up of the final diagram?

Has the diagram been used to properly judge the relevance of the strategy and

planning, the quality of the partnership, and the 3Cs?

Has the reviewed judgement been validated by a panel composed of decisions-

makers from the European Commission and independent experts?

Are the panel's conclusions accurately taken into account?

CHECK-LIST FOR MANAGERS

Questions Answer

Does the diagram describe the whole of the objectives and present in an

explicit way the rationale of the choices carried out?

Has the test of the temporary diagram been conducted with the main

stakeholders (authors, operators, partners)?

Has the diagram been used to properly judge the relevance of the strategy and

planning, the quality of the partnership, and the 3Cs?

Has the reviewed judgement been validated by a panel composed of decisions-

http://ec.europa.eu/europeaid/evaluation/methodology/tools/too_dcs_qua_en.htm#01








makers from the European Commission and independent experts? Are the

panel's conclusions accurately taken into account?

Examples

Example 1: decision diagram used in strategy evaluations [EN] (98 kb) Decision diagram taken from the intermediary report of the European Commission's country evaluation in

Egypt, 2002-2006: 'Interim note', MWH, 2003.

Example 2: decision diagram used in strategy evaluations: Tanzania [FR]

(234 kb) Decision diagram elaborated for the testing of the tool during the country evaluation in Tanzania 2000-2007.

Bibliography

'Analyse stratégique de la décision', Carluer F. et Richard A., 2002.

'The anatomy of influence: decision making in international organization', Cox et Jacobson,

1973.

'L'analyse des politiques publiques', Muller P. et Y. Surel. 1998.

'Les approches cognitives des politiques publiques', Muller P. et Y. Surel (eds), Revue

française de science politique, avril 2000.

'Administration et processus de décision', Simon H. A., 1983.



http://ec.europa.eu/europeaid/evaluation/methodology/examples/exp_too_dcs_thr_fr.pdf


