Simulating the EU: Reflections on Module Design
Dr. Anwen Elias
Department of International PoliticsAberystwyth
UniversityPenglaisAberystwythCeredigion SY23 3FEUnited Kingdom
Tel: 01970 621819Email: [email protected]
Abstract
Getting undergraduate students to understand the complex
institutional workings and policies of the European Union (EU) can
be extremely challenging. For this reason, many academics have
turned to simulations as a way of bringing EU politics to life for
students. This article describes and evaluates the design of a new
undergraduate module simulating the Czech Presidency of the EU.
Emphasis is placed on the process of defining learning objectives,
designing appropriate assessment methods, and putting teaching
strategies to support student learning in place. This design
process is too often overlooked in the simulation literature,
although it is key to maximising student learning. As such, the
module offers a template of how to organise a simulation and can
reduce some of the set-up costs involved in such an exercise.
Evaluation of the simulation module finds evidence of student
learning on the module, with support for both the innovative forms
of assessment devised and the support strategy put in place to help
students succeed in meeting the meeting assessment criteria.
Introduction
Getting undergraduate students to understand the complex
institutional workings and policies of the European Union (EU) can
be extremely challenging. An overview of the EU’s basic structures,
decision-making processes and contemporary debates can be provided
in a conventional classroom environment. However, fostering deep
understanding of, and meaningful engagement with, EU politics can
be more difficult.
For this reason, many academics have turned to simulations as a
way of bringing EU politics to life for students (for example Van
Dyke, 1999; Van Dyke et al., 2000; Galatas, 2006; Jones, 2008). As
a form of active learning, simulations can recreate complex,
dynamic political processes which are a feature of EU politics in
particular, and international politics more generally. There is a
substantial literature that asserts the pedagogical value of
simulations for student learning (Greenblat, 1973; Dorn, 1989;
Garard et al., 1998; Petranek et al., 1992; Shellman, 2001). They
give participants the opportunity to examine motivations,
behavioural constraints, resources and interactions among
institutional actors and afford deeper insight into political
processes. Moreover, students who are involved in simulations tend
to be more attentive and proactive in the learning process,
precisely because of the hands-on nature of the exercise and the
perception that they have a personal stake in the outcome of the
class (Smith and Boyer, 1996: 690). By immersing students in a
specific situation that requires their active participation, it has
been argued that simulations provide an immediate opportunity for
students to apply new knowledge and observe the outcomes of their
actions. This contrasts to traditional pedagogical techniques,
where students passively receive information from texts and
lectures, and there is a delay before they can apply this to new
situations (Raymond, 2010: 52; Dorn, 1989: 6). In sum, simulations
- as active, experiential modes of learning - are often held up to
be better than many traditional teaching methods at creating
effective thinkers (Dorn, 1989: 6; Shellman, 2001: 827).
This article outlines the design of an undergraduate module
simulation policy-making in the EU. It focuses in particular on the
different forms of learning that can take place during a
simulation, and how such learning can be assessed in innovative
ways that at the same time meet institutional accreditation
requirements. Such an approach pays close attention to the linkage
between the definition of learning objectives, designing
appropriate assessment methods, and putting teaching strategies to
support student learning in place. Such considerations have been
secondary in much of the simulation literature to date, with a
greater concern being to assess the extent of the simulation
value-added to student learning in comparison to more traditional
teaching methods (see, for example, Gosen and Washbush, 2004; Krain
and Lantis, 2006; Chin et al., 2009; Raymond, 2010).
The article is organised as follows. After providing some
background to the introduction of a new undergraduate module
simulating the EU, the process of formulating expected learning
outcomes and appropriate assessment methods is outlined. These
encompass the acquisition and application of substantive subject
knowledge as well as a range of discipline-relevant skills. The
article then considers the teaching strategy adopted in order to
ensure students arrive at the point where they are able to
undertake the assessment tasks successfully. The success of this
module design is then evaluated. Evidence from formal module
evaluation forms and additional questionnaires suggests that the
simulation was effective in helping students enhance their
understanding of complex process of institutional bargaining and
policy-making within the EU, and facilitated the development of key
skills. Just as importantly, student evaluations of the module
design were also generally positive, with strong support for
innovations in modes of assessment and the teaching support
provided in order to help students satisfy the assessment
criteria.
Reflecting on the simulation experience, however, also
highlights important considerations for educators considering
employing a similar model. In particular, whilst there may be
strong pedagogical reasons for thinking more ambitiously about what
and how much students learn as a result of participating in a
simulation, there is a trade-off in terms of the amount of staff
time required to ensure the simulation is successful. Failing to
appreciate this may compromise the quality of student learning.
Designing an EU Simulation
Background
Aberystwyth University's Department of International Politics
has a long track-record of organising simulations for undergraduate
students (Baylis, 1978). Annual weekend-long 'crisis games' see
students allocated to teams representing countries and
international organisations, and role-playing responses to
specified political crises. However, these events do not form part
of the formal teaching programme, and student participation is not
credited or assessed. In 2002, the department offered an
undergraduate EU Simulation module for one year only. The module's
learning outcomes focused on the substantive knowledge of EU
politics acquired as a result of participating in the simulation,
and was assessed through the standard departmental methods of essay
(50%) and exam (50%).
In 2009, an opportunity was presented to offer a new optional
module in the area of European politics for undergraduates in their
second and third years. Several years of teaching EU politics to
undergraduates using the conventional lecture/seminar format, and
reflection during this time on the difficulties of getting students
to engage with, and understand, distinctive EU institutional and
policy-making dynamics, informed the decision to develop a module
simulating the EU. The aim, as noted in the Introduction, was to
give students a hands-on experience of EU politics as a means of
enhancing student learning.
The module ran during one semester, with one 2-hour seminar each
week for 12 weeks. The theme of the simulation was the 6-month
Czech Presidency of the EU (January-June 2009). The specific policy
areas to be addressed were those identified by the Czech Presidency
itself as priorities, namely the economy, energy and external
relations.[footnoteRef:-1] 60 students registered for the module;
they were divided into teams of 4 representing either the Czech
Presidency, the EU institutions (European Commission, Secretariat
of the Council of Ministers[footnoteRef:0]) or a Member
State.[footnoteRef:1] Within each team, students designated a team
leader (Head of State/Commission President/Head of Secretariat as
appropriate) and distributed the three policy portfolios to
Ministers/Commissioners/Secretariat. The first three sessions were
assigned to preparatory work (see below). The sessions in weeks 4-9
were given over to meetings of the Council of Ministers, and in
week 10 a 2-day European Summit was organised as the final
culmination of the simulation (including a formal dinner where
negotiations were expected to continue). A debriefing session was
held in Week 11, with final coursework being submitted in Week 13.
The timetable for the module (along with the deadlines for
submission of assessed coursework, see below) is outlined in Table
1. [-1: The priorities of the Czech Presidency were summarised in
the document Programme of the Czech Presidency: Europe Without
Barriers (available at http://www.eu2009.cz/en/default.htm). This
document was made available to students at the beginning of the
module. ] [0: It was decided not to include a delegation
representing the European Parliament, as this is a body composed of
multiple country and partisan interests. Trying to reflect these
interests within the simulation would have been extremely complex.
As a result, the main dimension of decision-making enacted in the
simulation was that between the European Commission and the Member
States. ] [1: Student numbers meant that not all Member States of
the EU could be represented in the simulation. In selecting those
to include, a cross section of large vs. small, old vs. new, and
western vs. eastern Member States was sought. 14 Member States were
eventually represented within the simulation: Czech Republic
(Presidency), Bulgaria, Estonia, Finland, France, Germany, Greece,
Ireland, Italy, Poland, Slovenia, Spain, Sweden, United Kingdom.
]
Table 1Organisation of EU simulation module
Timetable
Schedule of assessment
Week 1
- Introduction to the module
- Division of students into teams
- Introduction and discussion of assessment criteria
-
Week 2
- Overview of key EU institutions and policy areas
- Time for teams to discuss their individual and team briefings
(assessed Task 1)
-
Week 3
- Discussion of procedural and decision-making rules for the
simulation
- Teams begin informal negotiations prior to formal commencement
of simulation
Task 1: Individual and team briefings
Week 4
Council of Ministers 1
Task 2: Assessment of participation during simulation
Week 5
Council of Ministers 2
Week 6
Council of Ministers 3
Week 7
Council of Ministers 4
Week 8
Council of Ministers 5
Week 9
Council of Ministers 6
Week 10
2 day European Summit
Week 11
Debriefing session
-
Week 12
-
-
Week 13
-
Submission of final coursework: Task 3: Portfolio of
documentation
Task 4: Final report
Considerations of module design
The process of module design followed Biggs' model of
"constructive alignment", which "aligns teaching method and
assessment to the learning activities stated in the objectives"
(1999: 11). Such an approach requires addressing three key
questions. Firstly, what are students expected to learn as a result
of participating in the simulation? Secondly, how can this learning
be demonstrated and assessed? Thirdly, how can teaching be
structured in order to support student learning? These three
questions are considered in turn.
i. Defining learning outcomes
A starting point for answering the first question was to reflect
on the previous EU simulation module offered in the department. The
learning outcomes, as noted above, focused on enhancing substantive
knowledge of EU politics. Such an objective is a frequently stated
goal of simulations. However, this overlooks the other advantages
of simulations, for example for facilitating the acquisition and
practice of different skills. This has long been recognised in
disciplines with a strong vocational orientation, such as medicine,
engineering and the legal profession. Teachers of international
studies, in contrast, have been slower to appreciate this dimension
of simulations. For students in this area, simulations offer
opportunities to refine debating, argumentation and public speaking
skills (Van Dyke, 1999); there may also be opportunities to gain
experience of negotiation and the drafting of political and legal
texts. Not only can simulations be a useful vehicle for enhancing
skills-based learning in international studies, but incorporating
this into the module design encourages students to take this aspect
seriously.
In the definition of learning objectives for an EU simulation
module, therefore, there is scope for a more expansive
understanding of what students should know and be able to do upon
completion of the course. These are listed in Table 1 below; whilst
the first five relate to knowledge and understanding of European
politics, the last three relate to skills to be used and developed
during the simulation.
Table 2 Learning outcomes for EU simulation module
(Upon completion of this module, students should be able
to:Describe and analyse key dynamics in EU
decision-making;Demonstrate an informed understanding of specified
EU policy areas and actors;Demonstrate a practical understanding of
the key rules governing EU decision-making; Critically assess the
nature of EU decision-making;Show an understanding of the
challenges of engaging in negotiations and bargaining in an
international setting;Employ a range of research skills in order to
gather information on specific actors and policy areas;Work
individually and in a team to identify goals and strategies for
meeting themUtilise oral and written communication skills to
negotiate and seek a consensus on specified policy goals.)
ii. Defining methods of assessment
Assessment is one of the most powerful levers available to
university teachers to change and improve students’ learning
(Gibbs, 1999). In particular, assessment tasks should be designed
in such a way that they will assess directly whether learning
outcomes have been met. With this in mind, four forms of formal
assessment were designed, and these are summarised in Table 3
below.
Table 3 Modes of assessment
Form of assessment
Learning objectives
Task 1. Preparatory briefings (1,500 words, 10% of mark, to be
submitted in Week 3)
- Individual analysis of the role to be played by each student,
based on primary and secondary research. The briefing should
provide details of the individual's background and political views,
and draw on these to reflect on how the individual might be
expected to behave in the simulation scenario. - Team briefing
outlining positions on the policy issues prioritised by the Czech
Presidency. Should include preferred policy outcomes (what must be
achieved and what is negotiable) and the outline of a general
negotiating strategy.
2, 3, 5, 6, 7
Task 2. Assessment of skills employed during the simulation (20%
of mark, on-going during simulation)
- Continuous assessment of four skills: preparation, factual
knowledge, communication skills and negotiation skills. - Feedback
given to teams after each session of the simulation.
3, 5, 6, 7, 8
Task 3. Portfolio of written assignments prepared during the
course of the simulation (20% of mark, to be submitted in Week
12)
- A record of individual and team activities throughout the
simulation. Should include all press releases, statements, draft
policy proposals, agreements with other delegations etc.
5, 6, 7, 8
Task 4. Final report (50% of mark, to be submitted in Week
12)
- Critical reflection on four aspects of EU politics, drawing on
personal experiences during the simulation: i. EU policy-making;
ii. Role of actor/institution played in the simulation; iii. Nature
of international negotiation; iv. future challenges facing the
EU.
1, 2, 3, 4, 5
As well as being closely aligned to the learning outcomes
formulated above, this choice of assessment methods has the
following strengths. Firstly, a range of different assessment tasks
is better able to accommodate different student needs, whilst the
distribution of assessment throughout the module incentivises
on-going student engagement. These are both conditions identified
by Gibbs and Simpson (2004) as encouraging student learning.
Secondly, the design of the assessment encourages deep learning.
This is achieved by requiring students to gather their own
information (on roles to be played, on team strategies) which can
be synthesised and applied to the simulation scenario. Moreover,
upon completion of the simulation students are encouraged to
reflect critically on, analyse, and evaluate their experiences in
order to arrive at a new understanding of the nature of EU
politics. All of these processes employ higher-level cognitive
skills that lead to deep learning (Bloom and Krathwohl, 1956).
iii. Support for student learning
Three aspects of teaching were identified as being necessary in
order to support students throughout the simulation module. The
first related to the provision of appropriate background material
in order to ensure the basic level of knowledge required to
undertake the simulation successfully (see Table 1). This required,
on the one hand, providing students with a brief introduction to
key EU institutions and policy-making processes (it should be noted
that students were allowed to take the module without having taken
any prior modules in European politics). The handbook for the
module provided reading lists for further information, as well as
directions for reading on specific roles/policy areas relevant for
the simulation. On the other hand, key documentation was also
provided relating to the Czech Presidency of the EU. Procedural and
decision-making rules (as are used in the EU) were also provided in
order to make the simulation's negotiations as close as possible to
real-world dynamics. This information was essential in setting the
context for the simulation, with students then given responsibility
for proceeding to negotiate on the key policy areas identified as
priorities.
A second important element of the preparatory work was the
communication of the learning objectives and assessment criteria to
students. Rust (2002: 151) notes that "if the assessment system is
to be as unthreatening as possible, not to mention fair, the
assessment process and criteria should be explicit and transparent
to the students". This is even more important given the novelty of
the simulation model of teaching within the Department of
International Politics, and where the modes of assessment are thus
completely new to students.
Two strategies were employed in order to articulate the
assessment modes and expectations to students. On the one hand, for
each assessed task, detailed criteria were provided in the handbook
for the module. This included the elements that should be addressed
in the task, and specific questions that should be addressed (see
an example in Appendix A). On the other hand, the assessment
criteria for each task were discussed with students during one of
the preparatory sessions. Encouraging students to actively engage
with the assessment criteria can significantly improve students'
performance when they come to undertaking the work themselves
(Price et al., 2001). Hypothetical examples of submitted work
(formulated by the module convenor) were thus discussed in order to
identify good and bad approaches to different tasks, and encourage
a shared understanding of what is expected from students in order
to do well in the module.[footnoteRef:2] [2: In future, past
(anonymised) examples of submitted work can be used as part of this
marking exercise. However, as this was the first time the module
had ran, this was not possible here. ]
A third aspect related to providing feedback to students during
the simulation. The provision of feedback is crucial in enabling
students to evaluate their own performance, identify areas of
improvement, and change their behaviour/thinking in order to meet
the assessment criteria and achieve the learning goal (Juwah et
al., 2004). The continuous assessment of student performance during
the simulation provided one opportunity for formative feedback (see
sample feedback form, Appendix B). This was made available to
students on a weekly basis (after each session of the simulation
during weeks 4-10) and constituted timely advice that could be
acted upon in advance of the next session. Upon completion of the
simulation, time was also dedicated to assist students to reflect
on their experiences and link these with broader questions about EU
politics and decision-making (as required by the assessed Final
Report, see Table 2 above). An exercise was devised to facilitate
this process. Statements about key EU policies and actors from the
academic literature were provided to students (see Table 3a).
Working within their teams, students were asked to respond to these
statements based on their own experiences; this encouraged linkages
to be made between the academic literature and the personal
insights gained during the simulation. Building on this, some key
themes were suggested to students that could be developed as part
of their final reports (see Table 3b). In different groups to those
that took part in the simulation, students were again asked to
respond to, and discuss, these themes based on their experiences.
Changing groups at this point enabled experiences from different
actor perspectives to be shared, with students becoming aware of
different viewpoints.
Table 4Encouraging student reflection on the simulation
a) Statements from the academic literature
"At times, the [Commission]...has provided political direction
to European integration...More recently, it has become almost
accepted wisdom that 'the decline of the
Commission...has...continued and there seems little possibility
that the situation is reversed."
"The Council of Ministers has grown in membership and scope to
the point where decision-making structures are under strain,
ministerial discussions are limited to a few substantive issues,
and agendas are overcrowded."
"The EU was designed as, and remains primarily, a limited
international institution to coordinate national regulation of
trade in goods and services, and the resulting flows of economic
factors."
"The Council Secretariat has earned a reputation for finding
creative EU solutions to deadlocked negotiations and helping defuse
potential political crises."
"In the latter part of the twentieth century for Europe, France
and Germany were at the forefront of the developments that have
shaped both Western and Eastern Europe. Having initiated and
controlled economic and monetary union, the greater goal is now
that of further European integration and the Franco-German dynamic
is likely to be crucial again in the success or failure of
achieving this."
"Scholars of EU governance often emphasise the capacity of the
EU to foster 'deliberation' and 'persuasion' - a model of
policy-making in which actors are open to changing their beliefs
and their preferences; and in which good arguments can matter as
much as, or more than, bargaining power."
b) More themes and issues
· The EU as intergovernmental vs. supranational- balance of
powers between institutions?
· Widening vs. deepening- which has been/is/should be the EU's
priority?- what are the problems associated with each?- can one be
done without the other?
· Small vs. large states- effectiveness in influencing EU
decision-making?- implications for running the EU Presidency?
· Efficiency/transparency/accountability/democracy- which of
these are/should be most important in the EU?- what are the
trade-offs between each of these?
Module Evaluation
A first step in evaluating the EU simulation is to examine
student performance on the module, that is, the extent to which
students have been successful in satisfying the assessment criteria
defined for the four assessment tasks. Overall, student performance
was very strong: of the 64 students registered for the module, 8
were awarded an overall mark of 70% or more, 30 were awarded marks
of 60-69, 20 were in the category 50-59, with only 4 achieving
marks between 40-49.[footnoteRef:3] Four students failed the module
(two due to poor performance, two due to non-submission of work).
When compared to the average distribution of marks across other
undergraduate modules in the Department of International Politics,
these results display a slightly higher than average number of
marks above 70%, with a slightly lower than average number of marks
below 49%. [3: Student performance is evaluated on the British
grading system, with marks being awarded out of 100. Marks under
40% are considered to be fails (with 40% being the equivalent to a
GPA of 2.0). Marks above 40% fall within 4 bands (with their
equivalent in GPA scores): 40-49% (GPA 2.3), 50-59% (GPA 3), 60-69%
(GPA 3.33-3.67), and 70% + (GPA 4.0). ]
These data provide a first indication that students had a clear
understanding of what was required from them to succeed in the
module. This is confirmed by the findings of two other sources of
data. Firstly, standard departmental feedback forms administered at
the end of the module provide data on student perceptions of course
assessment methods and criteria. Secondly, a questionnaire was
distributed to students before and after the simulation. A common
approach to evaluating student learning during simulations (see
Jones, 2008; Klassen and Willougby, 2003), this enables students'
understanding to be evaluated before and after the simulation takes
place. The post-simulation questionnaire also asked questions about
the methods of assessment, and provided space for suggestions on
how to improve the module in future.
With regard to the ways in which the module was assessed, the
post-simulation questionnaire asked students whether they
considered the modes of assessment to be appropriate for the
module. An overwhelming 83% responded that they were, although
several respondents favoured revising the weighting of the
different assessed components (with more given for participation
and less for the final report). Asked in departmental forms whether
"the criteria used in marking have been clear in advance", the
module was given an average score of 4.25 out of 5; on the question
of whether "assessment arrangements and marking have been fair", an
average score of 4.27 out of 5 was given. Student responses also
indicated a high level of satisfaction with the nature and
promptness of feedback on their work.
To what extent did the module design (including the forms of
assessment) also enable students to meet the learning objectives
outlined at the outset? Firstly, the pre- and post-simulation
questionnaires asked students to assess their understanding of the
following aspects of European politics: European current affairs,
the role and responsibilities of different EU institutions, the
role and responsibilities of the actor you will play in the
simulation, EU decision-making processes, and knowledge of what the
EU does in different policy areas. A comparison of the before and
after results showed a general trend whereby students feel that
their understanding of these different aspects has increased as a
result of having participated in the simulation. This increase was
greatest with regard to the specific roles played by individuals in
the simulation. This is not surprising, given that many students
were assigned to teams (member states or EU institutions) and roles
which required them to gather detailed individual- and
policy-specific information which they are unlikely to have
possessed prior to the module. However, the results also indicate
that cognitive learning has occurred more broadly. For example, as
Figure 1 below shows, whilst 40% of students ranked their
understanding of EU decision-making processes as ‘4’ or ‘5’ before
the simulation, this increased to 60% upon completion of the
module. In line with the findings of other simulations, students
thus believed the simulation helped them to gain knowledge related
to the stated learning objectives.
Secondly, the questionnaires also asked students to self-assess
a range of key skills (relevant for meeting the learning
objectives) before and after the simulation. The results are
provided in Table 4 below. A similar upward trend is observed in
the results, whereby students perceive the different skills to have
improved as a result of participating in the simulation. This is
particularly pronounced with working as a team and working
independently. The increase was less marked in other areas, such as
the use of research skills (in the library and on the internet) to
gather information.
Figure 1 - How would you assess your understanding of EU
decision-making processes?
Table 5 - Self-assessment of key skills (1 = Poor, 5 = Good)
1
3
4
2
5
Pre-S. %
Post-S.
%
Pre-S. %
Post-S.
%
Pre-S. %
Post-S.
%
Pre-S. %
Post-S.
%
Pre-S. %
Post-S.
%
1. Ability to argue and debate an issue
0.0
0.0
0.0
3.8
28.3
32.1
43.4
41.5
22.6
26.4
2. Speaking in public
3.8
3.8
9.4
11.3
34.0
28.3
22.6
30.2
26.4
28.3
3. Working with others in a team
0.0
0.0
0.0
7.5
18.9
3.8
35.8
45.3
32.1
50.9
4. Working independently
0.0
0.0
1.9
1.9
9.4
7.5
58.5
43.4
26.4
50.9
5. Use of library
1.9
1.9
3.8
3.8
18.9
13.2
47.2
49.1
24.5
32.1
6. Use of internet
1.9
0.0
0.0
0.0
5.7
7.5
34.0
28.3
56.6
64.2
In their feedback on the module, students also however
identified areas for improvement in the module's organisation.
Firstly, and particularly among students who had not previously
taken any modules on EU politics, the preparatory work required was
deemed to be excessive. This required getting to grips with the
basic institutional, policy and decision-making features of the EU,
in addition to gathering more detailed knowledge relevant to the
individual role being played in the simulation. To a certain
extent, it is unavoidable because of the nature of simulations that
student workload is higher than for other modules organised around
more traditional teaching methods. This fact should be made clearer
to students at the outset, with the option available of changing to
another module (UNSW, 2012). Another options would be to offer the
module only to students who have completed a prior introductory
module on EU politics. The disadvantage of this is that it excludes
students who are attracted by the innovative course design and who
may otherwise not have considered taking a module in EU politics
because of their interest in the subject.[footnoteRef:4] An
alternative would be to provide a more detailed introduction to EU
politics at the beginning of the module, or offer additional
sessions to those without this background prior to commencing the
simulation. This would have time implications for teaching staff.
[4: The pre-simulation questionnaire showed that 54% of students
enrolled on the module were attracted by the different model of
teaching and assessment, rather than the course's focus on EU
politics. ]
A second area for improvement identified is the management of
communication during the module. For example, many students
commented on the excess of paper produced in the form of briefing
documents, press releases, negotiation documents and so forth.
Others noted the difficulty of keeping track of discussions within
their teams and with other delegations. This suggests a future need
to explore the use of e-learning technologies in the organisation
and running of the simulation. For example, the use of on-line
discussion boards could be used to facilitate intra- and inter-team
discussions, and have the added attraction of allowing the module
convenor to monitor students' contributions to such discussions as
the simulation progresses.
Conclusion: Implications for designing simulations
For teachers of European politics, simulations are attractive
because they provide a hands-on experience of negotiation in an
international setting, and can potentially provide new insights
into what are complex and highly distinctive institutional and
policy-making dynamics. But simulations provide no guarantee that
students will learn. For this to happen, careful consideration
needs to be given to what it is students should be able to do upon
completion of the simulation, how this learning can be shown, and
what teachers can do to help students meet these expectations. This
article provides one account of designing a simulation in a way
that tries to maximise student learning. The process of defining
learning outcomes, designing assessment techniques, and developing
strategies for supporting students during the simulation, offers a
practical example of how to think through the challenge of getting
the best out of this mode of active learning. Taking each of these
steps seriously not only improves the student experience of
simulations, but also contributes to ensuring that simulations are
worthwhile exercises in terms of enhancing student learning.
But taking simulation design seriously also has substantial
implications for teaching staff. Simulations are resource
intensive, and costs and available time will constrain what it is
possible to do (UNSW, 2012). The simulation design outlined above
can serve to reduce some of these costs by providing a template for
educators in other institutions as well as disciplines. However,
this model requires a substantial investment of staff time,
especially in terms of providing students with regular and
high-quality feedback, and ensuring that the simulation did not
lose momentum or direction over the 11-week period during which it
ran. An understanding of the practical requirements of designing
and running a simulation in this way is essential if it is to be
successful both in terms of providing students with a positive
learning experience and meet the earning objectives outlined at the
outset. Failing to do so may have serious consequences for student
learning.
Appendix A - Example of guidance students on assessment
Task 1. Preparatory briefings (1500 words, 10% of mark, to be
submitted in Week 2)
a) Individual role analysis (1000 words)
Within each team, each student will be assigned a specific
alter-ego. Each student must prepare an analysis of his/her
individual alter ego, and research the general role that his/her
team plays in European decision-making. The Role Analysis should
address the following questions:
Part I: Individual Profile
- Who holds this position in real life?- What are this person’s
political experiences and views?- What position does this person
take on the issues featuring in the Simulation?- How might this
person behave in an international negotiation setting?
Part II: Institutional Analysis
- In general terms, what role does my actor/institution (i.e.
member state, Presidency, Commission, Council Secretariat) play in
European decision-making?- What are the opportunities and
constraints that my actor/institution faces in European
decision-making?- How is my actor/institution generally understood
in the academic literature?- Based on the answers to the above
questions, are there any important considerations that should guide
my strategy in the simulation, and which could determine how
successfully I play my role?
b) Team negotiating brief (500 words)
Each delegation should prepare a team negotiating brief written
by all members of the team. This should be a response to the
general issues raised by the Czech Presidency, and should form the
basis of your strategy for the simulation. The team negotiating
brief should address the following issues:
- What are the main priorities of the Czech Presidency?- What
are the implications for your member state? (legal, financial,
institutional, other policy linkages)- What is your preferred
outcome? What must you achieve, what may be negotiable, what are
your priority objectives and what are your subsidiary objectives?-
What are the positions of other delegations? Who are your likely
allies and opponents, and which delegations are likely to carry
significant influence? What alliances do you need to form in order
to meet your policy objectives?
Appendix B - Sample feedback form on student performance during
the simulation
EU SIMULATION – TEAM ASSESSMENT SHEET
TEAM: Italy
Excellent Good = Average X Poor XX Unsatisfactory
Preparatory session
Council 1
Council 2
Council 3
1. Preparation
- Research for the simulation
- Intra-team preparation
There is evidence of very good preparation
You could perhaps have done more to inform other teams of
Italy’s views before the meeting (e.g. press releases)
Good use of press releases on the pipelines issue, but could
have done more on other issues (including contributing to
discussion board on Blackboard)
2. Factual knowledge
- Awareness of European affairs
- Awareness of policy issues
- Awareness of specific actor/institutional role and
interests
Evidence of good knowledge
Good knowledge of Slovenia-Croatia dispute, although Italy’s own
position could be more clearly articulated to other delegations
(e.g. statements, press releases)
Generally well informed although less clear what Italian
position is on renewable energy
3. Communication skills
- Nature and appropriateness of written communications
- Oral presentations at formal Council meetings
- Team-work
Good interventions, although team-work could be improved to
ensure more coherence.
Appropriate intervention in the meeting; good team work.
Some good interventions, clear and concise. Scope for more
statements outlining Italy’s position. Energy Minister was left
alone for much of the time whilst other team members negotiating
outside on unrelated matters. Better co-ordination and support
needed.
A few interventions that were appropriate, and team cooperation
improved. Although as noted above, less clear what Italy’s
interests were in some policy areas.
4. Negotiating skills
- Definition of objectives and strategy for achieving defined
goals
- Knowledge of other actors’ positions and objectives
- Interaction with other delegations
- Ability to identify and reach appropriate solutions to
problems
Good efforts to act as a mediator between Slovenia and Croatia;
strategy is to build trust between the two countries and have made
clear to them that ‘this is no former Yugoslav conflict’.
Good mediation skills, although again, there’s possibly more
scope to make sure that Italy gets a good deal out of any
resolution to the border dispute.
=
More could have been done to ensure Italy’s views were heard,
e.g. on diversification of energy sources and renewable energy.
Satisfactory, although more could have been done to defend
Italian interests more visibly.
Council 4
Council 5
Council 6
European Council
1. Preparation
- Research for the simulation
- Intra-team preparation
x
No prior information circulated on Italian views on the issues
under discussion.
Excellent informal negotiations on the tobacco taxation
directive.
Good preparation, with circulation of press release (although
scope for something similar on the credit rating agencies).
x
Disappointing – although there was a press release on the
Italian earthquake, there was no effort to circulate Italian views
on issues to be discussed at the European Council. No response
either to the result of the Irish referendum.
2. Factual knowledge
- Awareness of European affairs
- Awareness of policy issues
- Awareness of specific actor/institutional role and
interests
=
General awareness of main issues, but less evidence of detailed
knowledge of policy positions.
Very well informed about Italian priorities and the country’s
‘green’ car policies.
Good background knowledge and understanding of issues under
discussion.
Some knowledge of Italian position, e.g. on economic situation
and on attitude towards Lisbon Treaty (although less evidence of an
in-depth knowledge on some issues)
3. Communication skills
- Nature and appropriateness of written communications
- Oral presentations at formal Council meetings
- Team-work
/=
Several appropriate interventions, good responses to the
discussion. More effort needed by the team to support the Minister,
as well as engage in informal negotiations. Note-passing to other
delegations should only be used to advance negotiations.
/=
Good interventions, clear and concise. Good responses to other
delegations. Minister seemed to be on top of his brief. However,
little evidence of support from team.
/=
Excellent, kept abreast of debate and interventions timely and
constructive. Still little evidence of team work.
Several clear and concise interventions, with PM seeking
clarification from Ministers when necessary (although team
co-operation could have been improved). But interventions tended to
be reactive rather than proactive.
4. Negotiating skills
- Definition of objectives and strategy for achieving defined
goals
- Knowledge of other actors’ positions and objectives
- Interaction with other delegations
- Ability to identify and reach appropriate solutions to
problems
=
OK, but could be more proactive in pushing issues that are a
priority; for example, could have been more imaginative in getting
the Council to move forward on Montenegro’s accession.
Excellent defence of Italian interests in informal negotiations
on tobacco taxation directive. Also clear about Italian priorities
in formal discussions.
Several critical observations on the credit rating agencies
proposal and suggestions for its improvements
There were good efforts at informal background negotiations, and
a conciliatory/mediating approach within the formal meetings.
However, overall the aims of the team remained vague and
ambiguous.
References
Baylis, J. (1978) 'Crisis gaming: The Aberystwyth experience',
British Journal of International Studies, 4, 233-243.
Biggs, J. (1999) Teaching for Quality Learning at University.
Buckingham: SRHE/OUP.
Bloom, B. and Krathwohl, D. (1956) Taxonomy of Educational
Objectives. Handbook 1: The Cognitive Domain. New York: Longman
Green.
Chin, J., Dukes, R. and Gamson, W. (2009) 'Asessment in
simulation and gaming: A review of the last 40 years', Simulation
and Gaming, 40: 4, 553-567.
Dorn, D. (1989) ‘Simulation games: one more tool on the
pedagogical shelf’, Teaching Sociology, 17: 1, 1-18.
Galatas, S. (2006) ' A Simulation of the Council of the European
Union: Assessment of the Impact on Student Learning', PS: Political
Science and Politics, January 2006, 147-151.
Garard, D., Lippert, L., Hunt, S.K., Paynton, S.T. (1998)
‘Alternatives to traditional instruction: using games and
simulations to increase student learning and motivation’,
Communication Research Reports, 15: 1, 36-44.
Gibbs, G. (1999) 'Using assessment strategically to change the
way students learn', in S. Brown and A. Glasner (eds.), Assessment
Matters in Higher Education. SRHE/Open University Press.
Gibbs, G and Simpson, C. (2004) 'Conditions under which
assessment supports students' learning', Learning and Teaching in
Higher Education, Vol.1, 3-31.
Gosen, J. J. and Washbush, J. (2004) 'A review of scholarship on
assessing experiential learning effectiveness', Simulation &
Gaming, 35, 270-293.
Greenblat, C. (1973) ‘Teaching with simulation games: a review
of claims and evidence’, Teaching Sociology, 17: 1, 1-18.
Jones, R. (2008) ‘Evaluating a cross-continent EU simulation’,
Journal of Political Science Education, 4: 4, 404-434.
Juwah, C., Macfarlane-Dick, D., Matthews, B., Nicol, D., Ross,
D. and Smith, B. (2004) Enhancing Student Learning Through
Effective Formative Feedback. Higher Education Academy (available
at:
http://www.heacademy.ac.uk/assets/documents/resources/resourcedatabase/id353_senlef_guide.pdf).
Klassen, K. and Willoughby, K.A. (2003) 'In-class simulation
games: Assessing student learning', Journal of Information
Technology Education, Vol. 2, 2-13.
Krain, M. and Lantis, J. (2006) 'Building knowledge? Evaluating
the effectiveness of the global problems summit simulation',
International Studies Perspectives, 7, 395-407.
Petranek, C., Corey, S. and Black, R. (1992) ‘Three levels of
learning in simulations: participating, debriefing and journal
writing’, Simulation & Gaming, 23: 2, 174-185.
Price, M., Rust, C. and Donovan, B. (2001) 'Improving students'
learning by developing their understanding of assessment criteria
and processes', Assessment and Evaluation in Higher Education, 28:
2, 147-164.
Raymond, C. (2010) 'Do role-playing simulations generate
measurable and meaningful outcomes? A simulation's effect on exam
scores and teaching evaluations', International Studies
Perspectives, 11, 1, 51-60.
Rust, C. (2002) 'The impact of assessment on student learning',
Active Learning in Higher Education, 3 (2): 145-158.
Shellman, S. M. (2001) 'Active Learning in Comparative Politics:
A Mock German Election and Coalition-Formation Simulation', PS:
Political Science & Politics 34 (4): 827-34.
Smith, E.T. and Boyer, M.A. (1996) ‘Designing in-class
simulations’, PS: Political Science and Politics, 29: 4, 690-4.
University of New South Wales (2012) Assessing with Role Play
and Simulation. Available at:
http://teaching.unsw.edu.au/assessing-role-play-and-simulation
(accessed 12 May 2012).
Van Dyke, G. (1999) 'Assessment and evaluation of the EU
simulation', paper prepared for the Biennial European Union Studies
Association Conference 1999, University of Pittsburgh,
Pennsylvania.
Van Dyke, G., DeClair, E. and Loedl, P.H. (2000) 'Stimulating
simulations: Making the European Union a Classroom reality",
International Studies Perspectives, 1, 145-159.
