arX

iv:1

905.

0332

5v6

[st

at.A

P] 1

2 Fe

b 20

21

Fair tournament design: A flaw of

the UEFA Euro 2020 qualification

Laszlo Csato∗

Institute for Computer Science and Control (SZTAKI)Eotvos Lorand Research Network (ELKH)

Laboratory on Engineering and Management IntelligenceResearch Group of Operations Research and Decision Systems

Corvinus University of Budapest (BCE)Department of Operations Research and Actuarial Sciences

Budapest, Hungary

15th February 2021

Wer, von inneren Kraften angeregt, sich ein solches Werk vorsetzen will, der ruste sich zu demfrommen Unternehmen mit Kraften wie zu einer weiten Pilgerfahrt aus. Er opfere Zeit undscheue keine Anstrengung, er furchte keine zeitliche Gewalt und Große, er erhebe sich uber eigene

Eitelkeit und falsche Scham, um nach dem Ausdruck des franzosischen Kodex die Wahrheit zusagen, nichts als die Wahrheit, die ganze Wahrheit.1

(Carl von Clausewitz: Vom Kriege)

Abstract

The integrity of a sport can be seriously undermined if its rules punish winning as thiscreates incentives for strategic manipulation. Therefore, a sports tournament can be calledunfair if the overall win probabilities are not ordered according to the teams’ rankingbased on their past performances. We present how statistical methods can contributeto choosing a tournament format that is in line with the above axiom. In particular,the qualification for the 2020 UEFA European Championship is shown to violate thisrequirement: being a top team in the lowest-ranked League D of the 2018/19 UEFANations League substantially increases the probability of qualifying compared to being abottom team in the higher-ranked League C. The unfairness can be remarkably reducedor even eliminated with slightly changing the path formation policy of the UEFA Euro2020 qualifying play-offs. The misaligned design has severely punished a team for winninga match years before. Since the deficiency is an inherent feature of the qualifying process,the Union of European Football Associations (UEFA) should reconsider the format offuture tournaments to eliminate the unfair advantage enjoyed by certain teams.

Keywords: fairness; mechanism design; simulation; soccer; UEFA Euro 2020

MSC class: 62F07, 68U20

JEL classification number: C44, C63, Z20

∗ E-mail: laszlo.csato@sztaki.hu1 “Whoever, stirred by ambition, undertakes such a task, let him prepare himself for his pious un-

dertaking as for a long pilgrimage; let him give up his time, spare no sacrifice, fear no temporal rank orpower, and rise above all feelings of personal vanity, of false shame, in order, according to the French code,to speak the Truth, the whole Truth, and nothing but the Truth.” (Source: Carl von Clausewitz: On War,Book 2, Chapter 6 – On Examples. Translated by Colonel James John Graham, London, N. Trubner, 1873.http://clausewitz.com/readings/OnWar1873/TOC.htm)

1 Introduction

Fairness in sports is an increasingly discussed problem in the academic literature, con-cerning various topics such as penalty shootouts (Apesteguia and Palacios-Huerta, 2010;Kocher et al., 2012; Palacios-Huerta, 2014; Brams and Ismail, 2018), referee assignment(Yavuz et al., 2008), scheduling (Chater et al., 2021; Duran et al., 2017; Guyon, 2020;Krumer and Lechner, 2017; Krumer et al., 2020), seeding (Cea et al., 2020; Csato, 2020c;Guyon, 2015; Laliena and Lopez, 2019), team selection (Harville, 2003), or tournamentdesign (Arlegi and Dimitrov, 2020; Guyon, 2018). Nevertheless, since any sporting contestestablishes a hierarchy among the competitors by differentiating winners and losers, per-haps the most important criterion of the fairness of a sports competition is the well-knownAristotelian Justice principle: the winning probabilities should be ordered according tothe players’ ranking. If this property does not hold for a given tournament, players haveperverse incentives to manipulate their ranking (Groh et al., 2012), which means a cleardesign failure (Szymanski, 2003) that can be identified and, ideally, improved by scientificresearch.

This paper aims to demonstrate how statistical methods can contribute to choosingan appropriate tournament format. Similar techniques have been successfully used tocompare the abilities of players from different eras in professional sports (Berry et al.,1999), to model player networks in team sports (Horrace et al., 2020), to forecast bas-ketball results by quantile regression (Koenker and Bassett, 2010), or to predict ultimateworld records in athletics (Einmahl and Magnus, 2008), among others. However, theseapplications usually require extensive and reliable real datasets, which are not necessarilyavailable if the planned tournament is designed in a novel way. Furthermore, most tour-nament formats exhibit a high degree of complexity (Guyon, 2018), thus analytical proofsare often impossible to obtain. Unsurprisingly, computer simulations mean a standardapproach in the evaluation of different tournament designs (Appleton, 1995; Csato, 2019;Dagaev and Rudyak, 2019; Goossens et al., 2012; Scarf et al., 2009).

On the other hand, it is advised to keep the underlying statistical model as simple aspossible because it might help that administrators embrace the results and finally correctthe problematic rule. While using a sophisticated methodology can persuade the reviewersof a prestigious journal, its application might even decrease the impact on decision makerswho may dismiss the intended message as a mere scientific curiosity without clear real-world implications.

Therefore, we scrutinize a competition design in soccer, probably the most popularsport in the world, from the perspective of fairness, without digging deep into the questionof how to predict match results. In particular, the qualification for the 2020 UEFAEuropean Championship is verified to robustly violate a crucial principle in an elementaryprobabilistic model: some national teams that are ranked lower at the beginning of thequalifying process have a considerably higher chance to advance to the final tournamentthan certain higher-ranked teams. The problem is revealed to be caused by the misalignedintegration of the 2020 UEFA European Championship qualifying tournament with theinaugural season of a new competition, the 2018/19 UEFA Nations League.

The root of the shortcoming resides in a novel policy of the Union of European FootballAssociations (UEFA) that aims to increase the diversity of the teams playing in the 2020UEFA European Championship. In this sense, the current work illuminates how thepromotion of small associations can cause unfairness without a careful analysis in advance.Similar ill-devised handicapping systems sometimes make losing a profitable strategy, for

2

instance, in sports using player drafts with the traditional set-up of reverse order whena team might decrease efforts to win after it has no more chance to progress in order toobtain an advantage in the player draft (Taylor and Trogdon, 2002; Price et al., 2010).Fornwagner (2019) is probably the first study verifying that teams apply a concrete losingstrategy. There are several policy proposals to remedy this problem (Banchio and Munro,2020; Lenten, 2016; Lenten et al., 2018; Kazachkov and Vardi, 2020). Analogously, lowerhandicappers in golf have a higher probability of winning in both stroke play and matchplay games (McHale, 2010).

Such opportunities for manipulation may emerge in soccer, too: Lasek et al. (2016)discuss some strategies to improve a team’s position in the “old” official FIFA ranking,which has been used before the reform in June 2018. The situation can be even worseif a team is strictly better off by losing, not only in expected terms (Dagaev and Sonin,2018; Csato, 2020a, 2021). Hence, together with many historical cases when changingsports rules led to unforeseen consequences (Kendall and Lenten, 2017), our paper warnsthe benefit of consultations between sports administrators and the academic community.

The current work is also closely connected to the papers which explicitly check thecondition of whether the order of win probabilities coincides with the contestants’ rankingor not, albeit without a simulation methodology. Hwang (1982) shows that this axiom maynot hold under the traditional seeding method, but reseeding after each round restoresmonotonicity. According to Baumann et al. (2010), the 10th and 11th seeds from theregular season average more wins and statistically progress farther than the 8th and9th teams due to the lack of reseeding in the National Collegiate Athletic Association(NCAA) men’s basketball “March Madness” tournament. The same issue is investigatedby Morris and Bokhari (2012). Groh et al. (2012) study an elimination tournament withfour players and identify all seedings ensuring a higher winning probability to higher-ranked players.

Our main contributions can be summarised as follows: (1) we offer the first detailedand rigorous documentation of unfairness in the qualification for the 2020 UEFA EuropeanChampionship; (2) we provide straightforward solutions to substantially mitigate theproblem of perverse incentives such that the UEFA Nations League does not becomecompletely uninteresting; (3) we present how the misaligned rules have severely punished ateam for winning a match years before. It is worth emphasizing that the most challengingpart of the simulation is coding the algorithm of the qualifying play-offs appropriatelysince the relevant regulation describes only the principles of the team selection and pathformation rules and even contains a contradiction (Csato, 2020b). Before underratingthis achievement, the reader is encouraged to try to understand the corresponding UEFAMedia Briefing, which discusses three possible scenarios over 89 slides (UEFA, 2017).

Admittedly, the strange tournament format has already been criticized for allowingweaker teams to qualify through the Nations League in the media (Dunbar, 2017). ARomanian computer programmer called Eduard Ranghiuc has provided preliminary cal-culations to show that losing could improve the chances of participating in the 2020 UEFAEuropean Championship (Ranghiuc, 2017). Even though his post reports the same qualit-ative findings as our study, the details of the methodology are missing and the calculationscannot be reproduced. Therefore, it has severe shortcomings from a scientific point of viewbut is dutifully acknowledged as a great source of inspiration.

At first sight, the presented example seems to be only an instructive case study forfootball fans. However, as Wright (2014, p. 1) writes, “given that sports are of greatinterest to a high percentage of the world’s population, it could be counter-argued that

3

there is little that could be researched into that is more important”. Our investigation mayinspire similar statistical analyses on the fairness of tournament formats and potentiallycan have an impact on decision makers. There is a recent work presenting the qualific-ation for the 2020 UEFA European Championship as an example of poor tournamentdesign (Haugen and Krumer, 2021), where an entire subsection (Section 3.2) is devotedto discussing our results.

The paper proceeds as follows. The qualification for the 2020 UEFA European Cham-pionship is outlined in Section 2 and the simulation technique is described in Section 3.Section 4 presents our quantitative results with sensitivity analysis and Section 5 proposestwo solutions to reduce the unfairness of the qualification. Section 6 studies to what ex-tent the deficiency of the qualifying process depends on the assumed distribution of teams’strength, while Section 7 turns to some implications for national teams. The paper endswith concluding remarks in Section 8.

2 Qualification for the UEFA Euro 2020

The 2020 UEFA European Football Championship, or shortly the UEFA Euro 2020, isthe 16th international men’s football championship of Europe. For the first time, it willbe spread over 12 cities in 12 countries across the continent, and no national associationreceives an automatic qualifying berth as a host country. Although similarly to the UEFAEuro 2016, 24 teams participate in the final tournament, the qualification is fundamentallydifferent from the previous editions because it is strongly connected to the inaugural seasonof the new competition called the UEFA Nations League.

The whole process of qualification for the UEFA Euro 2020 is regulated by two officialdocuments (UEFA, 2018a,b). It starts with the 2018/19 UEFA Nations League. First,the 55 UEFA national teams are divided into four divisions called leagues. In particular,they are ordered according to their UEFA national team coefficients at the end of the 2018FIFA World Cup qualifiers without the play-offs, and the 12 highest-ranked teams formLeague A, the next 12 form League B, the next 15 form League C, and the remaining16 form League D. The leagues are divided into four groups of three (Leagues A and B),four groups of four (League D), and three groups of four plus one group of three teams(League C) according to the traditional seeding regime: the best four teams are placedinto Pot 1, the second best four teams are placed into Pot 2, the next four into Pot 3, theremaining teams (if any) into Pot 4, and one team from each pot is given to each group.The groups are organized as a home-away (double) round-robin tournament, thereforeeach team plays four or six matches.

After ranking the teams in each group, four league rankings are established, wherethe four group winners are the first four, the four runners-up are the next four, and so on.Teams having the same position are ranked on the basis of the number of points and goaldifferences (in the case of League C, without the results against the fourth-placed teamsbecause there is a group with only three teams), except for positions 1–4 that are allocatedamong the group winners of League A through the 2019 UEFA Nations League Finals.However, this event does not affect the qualification for the UEFA Euro 2020. The leaguerankings are aggregated into the overall UEFA Nations League ranking such that the 12teams of League A occupy positions 1–12 according to the ranking in League A, the 12teams of League B occupy positions 13–24 according to the ranking in League B, the 15teams of League C occupy positions 25–39 according to the ranking in League C, and the16 teams of League D occupy positions 40–55 according to the ranking in League D.

4

Table 1: An overview of the qualification for the UEFA Euro 2020

Rank fromUEFA

nationalteam

coefficient

League inthe UEFANationsLeague

UEFANationsLeague

overall rank

Place in theseeding ofthe UEFAEuro 2020qualifiers

Remark

1–12 A 1–4 (GW) UNL Potdrawn into a group of fiveassured of at least play-offs

1–12 A 5–10 Pot 1 —1–12 A 11–12 Pot 2 —13–24 B 13–16

(GW)Pot 2 assured of at least play-offs

13–24 B 17–20 Pot 2 —13–24 B 21–24 Pot 3 —25–39 C 25–28

(GW)Pot 3 assured of at least play-offs

25–39 C 29–30 Pot 3 —25–39 C 31–39 Pot 4 —40–55 D 40 (GW) Pot 4 assured of at least play-offs40–55 D 41–43

(GW)Pot 5 assured of at least play-offs

40–55 D 44–50 Pot 5 —40–55 D 51–55 Pot 6 —

GW stands for group winner in the UEFA Nations League

The next stage is called the UEFA Euro 2020 qualifiers. The 55 teams are dividedinto five groups of five (Groups A–E) and five groups of six teams (Groups F–J), andthey are seeded according to the overall UEFA Nations League ranking: teams 1–4 (fromLeague A) are placed into the UNL Pot, teams 5–10 (from League A) are placed into Pot1, teams 11–20 (from Leagues A and B) are placed into Pot 2, teams 21–30 (from LeaguesB and C) are placed into Pot 3, teams 31–40 (from Leagues C and D) are placed into Pot4, teams 41–50 (from League D) are placed into Pot 5, and teams 51–55 (from League D)are placed into Pot 6. Then Groups A–D get one team from the UNL Pot each, and oneteam from Pots 2–5 each, Group E gets one team from Pots 1–5 each, while Groups F–Jget one team from Pots 1–6 each. Thus the draw applies a standard procedure but thebest four teams are guaranteed to be in the smaller groups of five. There are also specificrestrictions due to host nations, prohibited team clashes (because of political reasons),winter venue, and excessive travel (UEFA, 2018c). The groups are organized in a home-away (double) round-robin scheme and the first two teams qualify for the UEFA Euro2020.

Table 1 provides a short overview of the 2018/19 UEFA Nations League and the UEFAEuro 2020 qualifiers.

The four remaining berths are filled through the UEFA Euro 2020 qualifying play-offs.Contrary to the previous UEFA European Championships, teams do not advance to theplay-offs from the qualifying group stage, but they are selected on the basis of the overallUEFA Nations League ranking according to the team selection rule (UEFA, 2018a, Art-

5

icle 16.02): the four group winners of each league enter the play-offs unless they havealready qualified when they are substituted by the next best-ranked team in the relevantleague ranking that is available for the play-offs. The 16 teams are divided into fourpaths containing four teams each according to the path formation rule (UEFA, 2018a,Article 16.03). This requires that no group winners face any team from a higher-rankedleague, and leagues with at least four teams in the play-offs form their own path withfour teams from the league. In any play-off path, the highest-ranked team is matchedwith the lowest-ranked team and the two middle teams are matched against each otherin the semifinals, where the higher-ranked teams play at home. The final is contested bythe winners of the semifinals and is hosted by the winner of a semifinal drawn in advance.The winner of the final advances to the UEFA Euro 2020.

Consequently, at least one team from each league participates in the final tournament:if no team qualifies through the qualifying group stage, then the group winners of theleague form an own path. This is especially important for League D where teams havea low chance to advance to the UEFA Euro 2020 through the qualifying group stage.For example, the four group winners in League D of the 2018/19 UEFA Nations League(Georgia, North Macedonia, Kosovo, Belarus) contest the slot available for League D. Thewhole procedure is explained by three possible scenarios in UEFA (2017). Nonetheless,the rules are contradictory and may lead to an unfair formulation of play-off paths (Csato,2020b).

To summarise, the qualification process consists of the following stages:

1. The 55 teams are allocated into Leagues A–D on the basis of the ranking fromthe initial UEFA national team coefficients.

2. Four groups in each league are drawn on the basis of the ranking from the initialUEFA national team coefficients.

3. Matches of the 2018/19 UEFA Nations League are played, and the results de-termine the overall UEFA Nations League ranking.

4. Groups of the UEFA Euro 2020 qualifiers are drawn on the basis of the overallUEFA Nations League ranking.

5. Matches of the UEFA Euro 2020 qualifiers are played, the first two teams fromeach group qualify (altogether 20 teams enter the UEFA Euro 2020).

6. 16 teams that failed to qualify through the UEFA Euro 2020 qualifiers are selectedon the basis of the overall UEFA Nations League ranking by the team selectionrule.

7. The 16 contestants of the UEFA Euro 2020 qualifying play-offs are divided intofour paths of four teams each according to the path formation rule.

8. Matches of the UEFA Euro 2020 qualifying play-offs are played, the winner ofeach path qualify (further four teams enter the UEFA Euro 2020).

3 Methodology

We attempt to quantify the probability of qualification for the UEFA Euro 2020 for eachUEFA member states, which immediately provides the tournament metric to be analyzed.

6

Our computer code closely follows the relevant UEFA regulations (UEFA, 2018a,b) withsome minor differences:

• The first four places 1–4 of the overall UEFA Nations League ranking are determ-ined on the basis of UEFA Nations League group results, and not through the(2019) UEFA Nations League Finals. The draw of the groups in the UEFA Euro2020 qualifiers shows that this does not affect the outcome of the simulation.

• The specific restrictions in the draw of the UEFA Euro 2020 qualifying groupstage (UEFA, 2018c) are ignored because considering all of them would be cum-bersome and supposedly would influence the results only marginally.

• The theoretical contradiction in the rules of the play-offs is avoided through theproposal of Csato (2020b, Section 4.1). This has essentially no effect on thetournament metric because at least 13 teams should qualify from the lowest-ranked associations of League D, which is highly improbable.

Match outcomes are modeled such that the probability with which a given team defeatsits opponent is fixed a priori. That simple approach is chosen because: (1) the nationalteams play few matches in a year; (2) the methodology does not cover our main message;and (3) the qualitative findings are so robust that they should be reliably reproduced byany reasonable prediction model.

The fixed winning probabilities are derived from the World Football Elo Ratings, pub-lished regularly on the website eloratings.net. Although there exists no single nor anyofficial Elo ranking for football teams, Elo-inspired ratings seem to have the highest pre-diction power (Lasek et al., 2013). The overhauled formula for the FIFA/Coca-Cola WorldRanking, introduced in June 2018, also uses the Elo method of calculation. Elo ratingimmediately provides win expectancy We according to the formula:

We =1

1 + 10−d/s, (1)

where d is the difference in the Elo ratings of the two teams, increased by 100 points fora team playing at home, while s = 400 is a scaling parameter. Both the value of thehome advantage and the scaling parameter s comes from the system of World FootballElo Ratings (see http://eloratings.net/about).

For a match between teams i and j, the win probability wi of team i is computed byformula (1), and a random number r between 0 and 1 is generated. If r < wi, then i wins,otherwise, j wins. Note that wi + wj = 1. Draws (ties) in the matches are prohibitedas further arbitrary assumptions would be necessary to obtain the probability of thatoutcome from formula (1). Ties in the group and league rankings are broken randomly.

Fixing home advantage at 100 points has a limited influence on the outcome of thesimulations because both the 2018/19 UEFA Nations League and the UEFA Euro 2020qualifiers are played in double (home-away) round-robin groups. Essentially, only thesemifinals in the UEFA Euro 2020 qualifying play-off paths are affected by this choice,where the two higher-ranked teams are guaranteed to play at home.

We have used Elo ratings as of 6 December 2017 since the seeding pots of the 2018/19UEFA Nations League were announced on 7 December 2017. They are reported in Table 2besides the ranking derived from the UEFA national team coefficients, which provides theseeding of the 2018/19 UEFA Nations League.

7

Table 2: The characteristics of national teams at the beginning of the qualification for the UEFA Euro 2020

Team UEFA rank Elo rating Elo rank Team UEFA rank Elo rating Elo rank

League A League B

Germany 1 2109 1 Austria 13 1710 24Portugal 2 1995 3 Wales 14 1763 16Belgium 3 1927 6 Russia 15 1697 25Spain 4 2031 2 Slovakia 16 1748 17France 5 1989 4 Sweden 17 1825 13England 6 1933 5 Ukraine 18 1737 18Switzerland 7 1866 9 Ireland 19 1732 19Italy 8 1906 7 Bosnia and Herzegovina 20 1723 20Poland 9 1842 11 Northern Ireland 21 1674 27Iceland 10 1811 14 Denmark 22 1842 12Croatia 11 1856 10 Czech Republic 23 1713 22Netherlands 12 1895 8 Turkey 24 1712 23

League C League D

Hungary 25 1611 32 Azerbaijan 40 1400 44Romania 26 1688 26 North Macedonia 41 1520 37Scotland 27 1720 21 Belarus 42 1497 39Slovenia 28 1639 29 Georgia 43 1483 40Greece 29 1661 28 Armenia 44 1480 41Serbia 30 1769 15 Latvia 45 1362 46Albania 31 1609 33 Faroe Islands 46 1281 50Norway 32 1581 35 Luxembourg 47 1321 49Montenegro 33 1614 31 Kazakhstan 48 1340 47Israel 34 1534 36 Moldova 49 1332 48Bulgaria 35 1620 30 Liechtenstein 50 1150 52Finland 36 1595 34 Malta 51 1216 51Cyprus 37 1410 42 Andorra 52 1012 54Estonia 38 1507 38 Kosovo 53 1371 45Lithuania 39 1406 43 San Marino 54 852 55

Gibraltar 55 1079 53

Source of the Elo rating: https://www.international-football.net/elo-ratings-table?year=2017&month=12&day=06&confed=UEFA (6 December 2017)

8

900 1000 1100 1200 1300 1400 1500 1600 1700 1800 1900 2000 21000

0.2

0.4

0.6

0.8

1

Elo rating

League A League B League C League D

Figure 1: The probability of qualification for the UEFA Euro 2020

A simulation run consists of playing all matches in the three subtournaments of thequalification process for the UEFA Euro 2020: the 2018/19 UEFA Nations League, theUEFA Euro 2020 qualifiers, and the UEFA Euro 2020 qualifying play-offs. Since it is onlya single realization of random variables, one million iterations have been considered toobtain exact expected values.

4 Results

Figure 1 plots the probability of qualification for the 55 teams as a function of their Elorating. Intuitively, a team allocated into a weaker league, League B (C) instead of LeagueA (B), has a smaller chance to qualify for the UEFA Euro 2020 ceteris paribus. However,this is completely reversed in the comparison of Leagues C and D.

According to Figure 2, being in a lower-ranked league has two separate effects:

• the probability of qualification through the qualifying group stage is decreasedbecause being in a weaker league reduces (eliminates) the chance of obtaininga place in a stronger pot, hence the team should play against better teams onaverage in the UEFA Euro 2020 qualifiers (Figure 2.a); but

• the probability of qualification through the play-offs is increased because theteam can easier win its Nations League group, which guarantees a place in theplay-off path of its league (Figure 2.b).

When direct qualification through the UEFA Euro 2020 qualifiers has a low probability,that is, in the comparison of the bottom of League C with the top of League D for thegiven distribution of teams’ strength, the second effect dominates the first, hence having

9

(a) Probability of qualification through the UEFA Euro 2020 qualifiers

900 1000 1100 1200 1300 1400 1500 1600 1700 1800 1900 2000 21000

0.2

0.4

0.6

0.8

1

Elo rating

League A League B League C League D

(b) Probability of qualification through the UEFA Euro 2020 qualifying play-offs

900 1000 1100 1200 1300 1400 1500 1600 1700 1800 1900 2000 2100

0

0.05

0.1

0.15

0.2

0.25

0.3

Elo rating

League A League B League C League D

Figure 2: The decomposed probability of qualification for the UEFA Euro 2020

a worse rank at the beginning of the 2018/19 UEFA Nations League remarkably raisesthe probability of qualification for the UEFA Euro 2020.

10

12th 13th0

0.2

0.4

0.6

0.8

1

The Netherlands (A and B)

24th 25th0

0.2

0.4

0.6

Turkey (B and C)

39th 40th0

0.02

0.04

0.06

0.08

Lithuania (C and D)

Direct qualification Qualification through play-offs

Figure 3: The probability of qualification for teams at the boundary of two leagues

Figure 2.b can be somewhat misleading: the probability of qualification through theplay-offs is a non-increasing function of the Elo rating in Leagues A and B only becausethese teams usually qualify through the qualifying group stage and not through the play-offs. Naturally, the conditional probability of qualifying through the play-offs providedthat a team participates in the play-offs depends monotonically on the Elo rating.

To further illustrate the unfairness of the qualification, Figure 3 presents the actualand hypothetical probabilities of qualification for three countries (be aware of the differentindividual scales on the y-axis, applied to increase visibility):

• the Netherlands as the worst team in League A (12th) versus the best team inLeague B (13th);

• Turkey as the worst team in League B (24th) versus the best team in League C(25th);

• Lithuania as the worst team in League C (39th) versus the best team in LeagueD (40th).

Being in a lower-ranked league decreases the probability of qualification by about 2.3%for the Netherlands and by about 8.9% for Turkey, but the same metric increases by morethan seven-fold for Lithuania. While the probability of direct qualification becomes lowerand the probability of qualification through the play-offs becomes higher in all cases, thelatter effect dominates the former in the comparison of Leagues C and D, which maycreate an avenue for tanking.

The sensitivity of the calculations can be checked by modifying the scaling parameters in expression (1). Its original value of 400 may be judged excessive: for example, itimplies that the best team, Germany defeats the 10th (Croatia) with a probability of81.1% at a neutral field, and wins against the 30th (Bulgaria) with a probability of 93.4%.Furthermore, it is worth comparing the worst place of League C (39th) to the fifth placeof League D (44th) as obtaining the 40th position by a strategic manipulation of theUEFA national team coefficient cannot be guaranteed. Note that there is no difference inthe probability of qualification between the 37–39 positions, as well as between the 40–43positions due to the seeding procedure in the UEFA Nations League.

Thus three more competitive distributions of teams’ strength have been consideredand plotted in Figure 4 (again, be aware of the different individual scales on the y-axis).Although the advantage of the 44th over the 39th place decreases when inequality is

11

39th 44th0

0.05

0.1

s = 600

39th 44th0

0.05

0.1

0.15

s = 800

39th 44th0

0.1

0.2

s = 1200

Direct qualification Qualification through play-offs

Figure 4: The probability of qualification for Lithuania: sensitivity analysis

reduced, it remains substantial. Therefore any team has a strong incentive to avoid beingthe 37th, the 38th, and the 39th in the UEFA ranking underlying the construction of the2018/19 UEFA Nations League.

5 How to achieve fairness

We have devised two straightforward proposals to mitigate unfairness that still awardgroup winners in the UEFA Nations League. For the sake of simplicity, only the last stepof the qualification, the path formation of the UEFA Euro 2020 qualifying play-offs ismodified (see item 7 at the end of Section 2). In particular, two alternative policies tothe regular path formation rule are considered:

• Random path formation: the 16 teams of the play-offs are divided randomly intofour paths.

• Seeded path formation: the 16 teams of the play-offs are divided into four pathson the basis of the overall UEFA Nations League ranking under the traditionalseeding regime, that is, the four highest-ranked teams are drawn randomly fromPot 1, the next four from Pot 2, the next four from Pot 3, and the four lowest-ranked from Pot 4.

While the four teams (usually the group winners) of League D always form a separatepath under the regular path formation rule, they can play against higher-ranked teamsif random path formation is applied (however, they still have some chance to avoid thestrongest teams), and they should play against one of the four highest-ranked teams in thesemifinal if seeded path formation is followed. The modification of the path formation ruleaffects only the probability of qualification through the play-offs (i.e. Figure 2.b) but notthe probability of qualification through the qualifying group stage (plotted in Figure 2.a).

Figure 5 shows the overall probability of qualification for the UEFA Euro 2020 underthe proposed random (Figure 5.a) and seeded (Figure 5.b) path formation regimes. Itremains better to be in League A (B) than in League B (C) and the difference betweenthese leagues increases as path formation moves farther from its original concept. Inaddition, unfairness becomes less serious in the case of random path formation and canbe almost wiped out with seeded path formation.

This implication is reinforced by Figure 6, which presents the probability of qualifica-tion for Lithuania, a team at the boundary of Leagues C and D. Since our proposals do

12

(a) Random path formation

900 1000 1100 1200 1300 1400 1500 1600 1700 1800 1900 2000 21000

0.2

0.4

0.6

0.8

1

Elo rating

League A League B League C League D

(b) Seeded path formation

900 1000 1100 1200 1300 1400 1500 1600 1700 1800 1900 2000 21000

0.2

0.4

0.6

0.8

1

Elo rating

League A League B League C League D

Figure 5: The probability of qualification: alternative path formations

not affect the qualifying group stage, the height of the blue columns with vertical lines arethe same but the scale on the vertical axis is changed in order to highlight the differences.

13

39th 40th0

0.02

0.04

0.06

0.08

Regular path formation

39th 40th0

0.01

0.01

0.02

Random path formation

39th 40th0

0.005

0.01

Seeded path formation

Direct qualification Qualification through play-offs

Figure 6: The probability of qualification for Lithuania: various path formations

Playing in League D (40th position) increases the probability of qualification by a factorof 7.14, 1.34, and 1.09 under the three path formation rules called regular, random, andseeded, respectively.

To summarize, the unfairness of the tournament design can be remarkably reduced oreven eliminated with a slight policy change in the path formation of the UEFA Euro 2020qualifying play-offs. Since the modifications do not influence the team selection rule, allcontestants remain interested in performing well in the UEFA Nations League.

6 Theoretical confirmation

One can still say that the above finding of unfairness is mostly driven by the dataset (theactual distribution of teams’ strength) and not by the competition format itself. Therefore,another kind of sensitivity analysis is provided on the basis of win expectancies given byformula (1). In particular, three models called Difference 0, 10, and 20, respectively, areinvestigated: the Elo rating of the 28th ranked middle team is fixed at 1500, while teami has an Elo rating of 1500 + (28 − i)∆, where ∆ ∈ {0, 10, 20} corresponds to the nameof the probabilistic model.

According to Figure 7, the probability of qualification is close to uniform around24/55 ≈ 0.4364 if all teams are equally strong. Four factors imply differences among theteams:

• the teams ranked 51–55 are seeded in groups of six, hence a team from LeagueD has to play against more teams on average in the UEFA Euro 2020 qualifiers;

• Leagues A and B have only 12 teams but League C consists of 15 and LeagueD contains 16 teams, thus lower-ranked teams have a lower chance to win theirNations League group and advance to the qualifying play-offs;

• the team selection rule prefers higher-ranked teams in filling the vacant slots inthe qualifying play-offs; and

• in each play-off path, the semifinals are hosted by the two higher-ranked teams.

In this case, the last two effects are marginal, that is, the teams of Leagues A and Bhave essentially the same probability to qualify. In addition, the probability of direct

14

0 5 10 15 20 25 30 35 40 45 50 550

0.2

0.4

0.6

0.8

1

Team rank

League A League B League C League DDifference 0 Difference 10 Difference 20

Figure 7: Theoretical model: qualifying probability with regular path formation

qualification is the same for the teams of Leagues A, B, and C as only the first pointaffects the UEFA Euro 2020 qualifiers.

Contrarily, with increasing parameter ∆, visible breaks appear at the boundary ofleagues. The teams ranked 21–24, the four weakest in League B, have a considerabledisadvantage compared to the better teams of League B if ∆ = 20 because exactly 20 slotsare allocated in the UEFA Euro 2020 qualifiers. Unsurprisingly, participation in LeagueA is favored to participation in League B, as well as League B is preferred to LeagueC. On the other hand—analogously to the finding with the real data in Section 4—itis better to play in League D compared to League C. Unfairness grows in parallel withcompetitive imbalance. In the model of Difference 20, even the team at bottom of LeagueB is somewhat worse off than the team at the top of League C, which implies that incentiveincompatibility might be a problem at the boundary of the two middle leagues if the profileof teams’ strength changes.

Figure 8 reveals that changing the path formation policy can again improve fair-ness. Both random and seeded path formations result in a remarkable difference betweenLeagues B and C. As expected, seeded path formation favors less the top teams of LeagueD than random path formation, therefore it is closer to fairness. If all teams are equallystrong (∆ = 0), the qualifying probabilities of teams from Leagues A and B essentiallydiffer only under random path formation. That is explained by the rule of hosting thesemifinals in the qualifying play-offs: a play-off path usually consists of teams from thesame league under regular path formation, and it usually contains one team from eachleague under seeded path formation, thus a team from League A has the same chance toplay at home in the semifinals than a team from League B. On the other hand, randompath formation does not guarantee this equality between the teams of Leagues A and B.

To conclude, our theoretical investigation reveals that unfairness is an inherent fea-

15

(a) Random path formation

0 5 10 15 20 25 30 35 40 45 50 550

0.2

0.4

0.6

0.8

1

Team rank

League A League B League C League DDifference 0 Difference 10 Difference 20

(b) Seeded path formation

0 5 10 15 20 25 30 35 40 45 50 550

0.2

0.4

0.6

0.8

1

Team rank

League A League B League C League DDifference 0 Difference 10 Difference 20

Figure 8: Theoretical model: qualifying probability with alternative path formations

16

ture of the UEFA Euro 2020 qualification and its deficiency is not only caused by theunfavorable distribution of the team’s strength.

7 Discussion

According to Section 4, obtaining a worse position in the ranking of the national teamsused for the draw of the 2018/19 UEFA Nations League can considerably increase theprobability of reaching the final tournament. This feature endangers the sport’s credibilityand integrity as certain teams may aim to manipulate the ranking in the future.

The 55 UEFA members have been divided into the four leagues according to theirUEFA national team coefficients after the conclusion of the 2018 FIFA World Cup quali-fiers without the play-offs. The coefficients are calculated as a weighted average:

• 20% of the average ranking points collected in the 2014 FIFA World Cup qualifi-cation and final;

• 40% of the average ranking points collected in the UEFA Euro 2016 qualificationand final;

• 40% of the average ranking points collected in the 2018 FIFA World Cup qualifi-cation.

The allocation of match points is explained in UEFA (2018b, Annex D), while FootballSeeding.com(2017) presents the details of the calculation.

The last nine matches (three matches each in the UEFA groups A, B, and H) thatinfluence this ranking were played at the end of the 2018 FIFA World Cup qualifiers on 10October 2017. In particular, Belgium vs. Cyprus was 4-0, resulting in 19,491.08 points forCyprus, which corresponds to the 37th position. The 40th was Azerbaijan with 17,760.82points. However, Cyprus would have been better off on the 40th place concerning theprobability of qualification for the UEFA Euro 2020. A conceded goal in any match meansminus 500 points, which should be divided by 10 (the number of matches in the 2018 FIFAWorld Cup qualification), and has a weight of 0.4 in the coefficient used for the draw ofthe 2018/19 UEFA Nations League. Hence, Cyprus would have been only the 40th afterkicking 1740/20 = 87 own goals against Belgium because the point difference betweenAzerbaijan and Cyprus was 1730.82.

While a similar tanking can be easily detected and probably sanctioned by the UEFA,a team can achieve the 40–43 positions easily if it is willing to sacrifice a whole FIFA WorldCup qualification, where the probability of its success is marginal anyway. For instance,Lithuania scored only six points in UEFA Group F during the 2018 FIFA World Cupqualification, while 18 was still not enough for the qualification. If Lithuania would haveplayed a draw of 2-2 against Malta on 11 October 2016 instead of winning by 2-0, thenit would have had 18100.74 − 1640.08 + 800.08 = 17260.74 points, guaranteeing the 40thposition in the UEFA national team coefficient ranking used for the draw of the 2018/19UEFA Nations League ceteris paribus. Consequently, Lithuania was severely punished forits win against Malta by the controversial format of the qualification for the UEFA Euro2020.

17

8 Conclusions

The current paper has documented the unfairness of the qualification for the 2020 UEFAEuropean Championship. In particular, being a top team in the lowest-ranked League D ofthe 2018/19 UEFA Nations League can substantially increase the probability of qualifyingcompared to being a bottom team in League C of the 2018/19 UEFA Nations League.Using a more accurate prediction model to forecast match results cannot change thequalitative findings but may hide them behind the details of the statistical methodology.

Two slight modifications have been proposed to mitigate or eliminate the problem ofperverse incentives. Both require only a minor change in the path formation policy of theUEFA Euro 2020 qualifying play-offs. This rule is hidden at the deep of the regulation,is probably understood neither by the public nor by the decision makers, and containstheoretical shortcomings anyway as has already been revealed (Csato, 2020b).

We are afraid that the novel policy of the Union of European Football Associations(UEFA), which aims to increase the diversity of the teams playing in the UEFA Euro2020, has been implemented without a careful analysis in advance. Our work can inspirefurther applications of statistical techniques that identify similar issues of unfairness insports and propose reasonable amendments.

Acknowledgments

This paper could not have been written without my father (also called Laszlo Csato), whohas coded the simulations in Python.We are grateful to Eduard Ranghiuc for inspiration.Alex Krumer, Manuel Mago and Tamas Halm gave useful comments.Two anonymous reviewers provided valuable remarks and suggestions on an earlier draft.We are indebted to the Wikipedia community for contributing to our research by sum-marising the tournaments discussed in the paper.The research was supported by the MTA Premium Postdoctoral Research Program grantPPD2019-9/2019.

References

Apesteguia, J. and Palacios-Huerta, I. (2010). Psychological pressure in competitiveenvironments: Evidence from a randomized natural experiment. American EconomicReview, 100(5):2548–2564.

Appleton, D. R. (1995). May the best man win? Journal of the Royal Statistical Society:Series D (The Statistician), 44(4):529–538.

Arlegi, R. and Dimitrov, D. (2020). Fair elimination-type competitions. European Journalof Operational Research, 287(2):528–535.

Banchio, M. and Munro, E. (2020). Targeting in tournaments with dynamic incentives.Manuscript. https://www.evanmunro.ca/files/targeting.pdf.

Baumann, R., Matheson, V. A., and Howe, C. A. (2010). Anomalies in tournament design:The madness of March Madness. Journal of Quantitative Analysis in Sports, 6(2):4.

18

Berry, S. M., Reese, C. S., and Larkey, P. D. (1999). Bridging different eras in sports.Journal of the American Statistical Association, 94(447):661–676.

Brams, S. J. and Ismail, M. S. (2018). Making the rules of sports fairer. SIAM Review,60(1):181–202.

Cea, S., Duran, G., Guajardo, M., Saure, D., Siebert, J., and Zamorano, G. (2020).An analytics approach to the FIFA ranking procedure and the World Cup final draw.Annals of Operations Research, 286(1-2):119–146.

Chater, M., Arrondel, L., Gayant, J.-P., and Laslier, J.-F. (2021). Fixing match-fixing:Optimal schedules to promote competitiveness. European Journal of Operational Re-search, in press. DOI: 10.1016/j.ejor.2021.02.006.

Csato, L. (2019). A simulation comparison of tournament designs for the World Men’sHandball Championships. International Transactions in Operational Research, in press.DOI: 10.1111/itor.12691.

Csato, L. (2020a). The incentive (in)compatibility of group-based qualification systems.International Journal of General Systems, 49(4):374–399.

Csato, L. (2020b). Two issues of the UEFA Euro 2020 qualifying play-offs. InternationalJournal of Sport Policy and Politics, 12(3):471–484.

Csato, L. (2020c). The UEFA Champions League seeding is not strategy-proof since the2015/16 season. Annals of Operations Research, 292(1):161–169.

Csato, L. (2021). Tournament Design: How Operations Research Can Improve SportsRules. Palgrave Pivots in Sports Economics. Palgrave Macmillan, Cham, Switzerland.

Dagaev, D. and Rudyak, V. (2019). Seeding the UEFA Champions League participants:Evaluation of the reform. Journal of Quantitative Analysis in Sports, 15(2):129–140.

Dagaev, D. and Sonin, K. (2018). Winning by losing: Incentive incompatibility in multiplequalifiers. Journal of Sports Economics, 19(8):1122–1146.

Dunbar, G. (2017). As World Cup hope fades, Europeans turn to Nations League. Asso-ciated Press. 24 March. https://apnews.com/7effa1bd286244a0aebbd9275f3db30d.

Duran, G., Guajardo, M., and Saure, D. (2017). Scheduling the South American Qualifiersto the 2018 FIFA World Cup by integer programming. European Journal of OperationalResearch, 262(3):1109–1115.

Einmahl, J. H. J. and Magnus, J. R. (2008). Records in athletics through extreme-valuetheory. Journal of the American Statistical Association, 103(484):1382–1391.

FootballSeeding.com (2017). National ranking UEFA 2017.https://www.footballseeding.com/national-ranking-uefa/ranking-2017.

Fornwagner, H. (2019). Incentives to lose revisited: The NHL and its tournament incent-ives. Journal of Economic Psychology, 75(Part B):102088.

19

Goossens, D. R., Belien, J., and Spieksma, F. C. R. (2012). Comparing league formatswith respect to match importance in Belgian football. Annals of Operations Research,194(1):223–240.

Groh, C., Moldovanu, B., Sela, A., and Sunde, U. (2012). Optimal seedings in eliminationtournaments. Economic Theory, 49(1):59–80.

Guyon, J. (2015). Rethinking the FIFA World CupTM final draw. Journal of QuantitativeAnalysis in Sports, 11(3):169–182.

Guyon, J. (2018). What a fairer 24 team UEFA Euro could look like. Journal of SportsAnalytics, 4(4):297–317.

Guyon, J. (2020). Risk of collusion: Will groups of 3 ruin the FIFA World Cup? Journalof Sports Analytics, 6(4):259–279.

Harville, D. A. (2003). The selection or seeding of college basketball or football teams forpostseason competition. Journal of the American Statistical Association, 98(461):17–27.

Haugen, K. K. and Krumer, A. (2021). On importance of tournament design in sportsmanagement: Evidence from the UEFA Euro 2020 qualification. In Ratten, V., editor,Innovation and Entrepreneurship in Sport Management, pages 22–35. Edward ElgarPublishing, New York.

Horrace, W. C., Jung, H., and Sanders, S. (2020). Network competition and teamchemistry in the NBA. Journal of Business & Economic Statistics, in press. DOI:10.1080/07350015.2020.1773273.

Hwang, F. K. (1982). New concepts in seeding knockout tournaments. The AmericanMathematical Monthly, 89(4):235–239.

Kazachkov, A. E. and Vardi, S. (2020). On tanking and com-petitive balance: Reconciling conflicting incentives. Manuscript.https://akazachk.github.io/pubs/tanking.pdf.

Kendall, G. and Lenten, L. J. A. (2017). When sports rules go awry. European Journalof Operational Research, 257(2):377–394.

Kocher, M. G., Lenz, M. V., and Sutter, M. (2012). Psychological pressure in competit-ive environments: New evidence from randomized natural experiments. ManagementScience, 58(8):1585–1591.

Koenker, R. and Bassett, J. G. W. (2010). March Madness, quantile regression bracketo-logy, and the Hayek hypothesis. Journal of Business & Economic Statistics, 28(1):26–35.

Krumer, A. and Lechner, M. (2017). First in first win: Evidence on schedule effects inround-robin tournaments in mega-events. European Economic Review, 100:412–427.

Krumer, A., Megidish, R., and Sela, A. (2020). The optimal design of round-robin tour-naments with three players. Journal of Scheduling, 23(3):379–396.

Laliena, P. and Lopez, F. J. (2019). Fair draws for group rounds in sport tournaments.International Transactions in Operational Research, 26(2):439–457.

20

Lasek, J., Szlavik, Z., and Bhulai, S. (2013). The predictive power of ranking systems inassociation football. International Journal of Applied Pattern Recognition, 1(1):27–46.

Lasek, J., Szlavik, Z., Gagolewski, M., and Bhulai, S. (2016). How to improve a team’s pos-ition in the FIFA ranking? A simulation study. Journal of Applied Statistics, 43(7):1349–1368.

Lenten, L. J. A. (2016). Mitigation of perverse incentives in professional sports leagueswith reverse-order drafts. Review of Industrial Organization, 49(1):25–41.

Lenten, L. J. A., Smith, A. C. T., and Boys, N. (2018). Evaluating an alternative draftpick allocation policy to reduce ‘tanking’ in the Australian Football League. EuropeanJournal of Operational Research, 267(1):315–320.

McHale, I. G. (2010). Assessing the fairness of the golf handicapping system in the UK.Journal of Sports Sciences, 28(10):1033–1041.

Morris, T. L. and Bokhari, F. H. (2012). The dreaded middle seeds – Are they the worstseeds in the NCAA basketball tournament? Journal of Quantitative Analysis in Sports,8(2).

Palacios-Huerta, I. (2014). Beautiful Game Theory: How Soccer Can Help Economics.Princeton University Press, Princeton, New York.

Price, J., Soebbing, B. P., Berri, D., and Humphreys, B. R. (2010). Tournament incentives,league policy, and NBA team performance revisited. Journal of Sports Economics,11(2):117–135.

Ranghiuc, E. (2017). UEFA Nations League: Losing could improve your chances. 14 July.http://www.football-rankings.info/2017/07/uefa-nations-league-losing-could.html.

Scarf, P., Yusof, M. M., and Bilbao, M. (2009). A numerical study of designs for sportingcontests. European Journal of Operational Research, 198(1):190–198.

Szymanski, S. (2003). The economic design of sporting contests. Journal of EconomicLiterature, 41(4):1137–1187.

Taylor, B. A. and Trogdon, J. G. (2002). Losing to win: Tournament incentives in theNational Basketball Association. Journal of Labor Economics, 20(1):23–41.

UEFA (2017). UEFA Nations League Media Briefing. Nyon, 20 September.https://web.archive.org/web/20180905214902/http://files.footballseeding.com/200002130-9283c937e4/UNL%20media%20briefing%20PDF%20version.pdf.

UEFA (2018a). Regulations of the UEFA European Football Championship 2018-20.https://web.archive.org/web/20200330123015/https://www.uefa.com/MultimediaFiles/Download/Regulations/uefaorg/Regulations/02/54/36/05/2543605_DOWNLOAD.pdf.

UEFA (2018b). Regulations of the UEFA Nations League 2018/19.https://web.archive.org/web/20201022173359/https://www.uefa.com/MultimediaFiles/Download/Regulations/uefaorg/Regulations/02/50/54/37/2505437_DOWNLOAD.pdf.

UEFA (2018c). UEFA EURO 2020 qualifying draw. 2 December.https://www.uefa.com/european-qualifiers/news/newsid=2573388.html.

Wright, M. (2014). OR analysis of sporting rules – A survey. European Journal ofOperational Research, 232(1):1–8.

21

Yavuz, M., Inan, U. H., and Fıglalı, A. (2008). Fair referee assignments for professionalfootball leagues. Computers & Operations Research, 35(9):2937–2951.

22