Extracted main text — title through conclusion, appendix excluded. This is what our citation measures are computed over, published so the extraction can be checked by eye.
48,165 characters · 11 sections · 46 citation commands
Club coefficients in the UEFA Champions League: Time for shift to an Elo-based formula
\thispagestyle{empty}
{ “Seine Pflicht erkennen und tun, das ist die Hauptsache.”\footnote{ “Recognizing and doing one's duty is the main thing.” \\ Source: \url{https://www.friedrich-der-grosse.net/zitate-friedrich-des-grossen}} }
\flushright (Frederick the Great)
\justify
\restoregeometry
In team sports, assessing the performance of an individual team is clearly impossible without accounting for the strength of opponents: the same number of goals scored or the same probability of successful attacks are usually more worthy if they have been achieved against a stronger team. This is a crucial issue especially for tournaments that are not played in a round-robin format, for example, if the teams are allocated into different groups, and only the best team(s) from each group qualify for the next phase such as in the FIBA Basketball World Cup or the FIFA World Cup.
Thus, there is a strong demand for reliable measures of teams' strengths. However, it is better to use a transparent methodology for this purpose that can be accepted by all stakeholders, including fans, managers, and players with limited mathematical and statistical knowledge. This implies a trade-off between accuracy and complexity: the calculation of the ratings should be relatively simple without requiring the estimation of difficult regression models even if they would have theoretically a higher predictive power.
The UEFA Champions League, organised by the Union of European Football Associations (UEFA), is the most prestigious club football competition in European football. UEFA currently uses a particular rating of the teams, the so-called UEFA club coefficient, in order to guarantee the balancedness of the groups in the Champions League and other competitions. This measure is essentially based on the results in the five previous seasons of the UEFA competitions (Champions League, Europa League, Europa Conference League) UEFA2018g. However, it ignores most matches played by the clubs since it does not depend on the outcomes of matches in the national leagues and cups at all.
Our research question is whether a revised calculation formula for club coefficients can improve performance forecasts in the UEFA Champions League. In particular, we compare the official UEFA club coefficient and the Football Club Elo Rating, a readily available Elo-based measure of team strength. Their explanatory powers are studied via logistic regression models. Success is defined by winning group matches, winning in the knockout stage, and obtaining a higher rank in the group stage. The database contains the 19 Champions League seasons played between 2003/04 and 2021/22.
The main contribution of the current paper resides in comparing a simple alternative rating of the teams with the widely used UEFA club coefficient with respect to predicting future performance in the UEFA Champions League. According to our knowledge, that has never been done before in the literature.
The study is structured as follows. A concise overview of literature is provided in Section (ref). Section (ref) describes the data underlying our empirical investigation, as well as the statistical methods used. The estimations are presented in Section (ref). Section (ref) discusses the main contributions and outlines some directions for future research.
Our work is connected to at least three directions of academic research.
Some previous papers deal with similar topics in the UEFA Champions League. SchokkaertSwinnen2016 empirically examine how the changes in the format of the Champions League have affected uncertainty of outcome. They conclude that qualification in the early rounds has become more predictable but the later stages have become less predictable after 1999. According to the regression discontinuity design of EngistMerkusSchafmeister2021, the UEFA club coefficient itself does not contribute positively to success in this tournament: seeding did not have any effect on the performance of marginally seeded teams. Triguero-RuizAvila-Cano2023 find a notable drop in competitive balance in the group stage of the Champions League over the last two decades. This implies a higher accuracy in predicting performance if one finds a reasonable indicator of strength.
The current study is strongly related to research on rating methods in football, too. HvattumArntzen2010 implement and test two Elo-based prediction methods. LasekSzlavikBhulai2013 and GasquezRoyuela2016 demonstrate that the Elo rating system is quite competitive in predicting the matches played by national football teams. BakerMcHale2015, BakerMcHale2018 suggest time-varying rating models for the English football league and international football teams, respectively. VanEetveldeLey2019 give a survey of the most common ranking methods in football. LeyVandeWieleVanEeetvelde2019 compare several existing and novel statistical models assigning one or more strength parameters to a football team.
SzczecinskiDjebbi2020 and Szczecinski2022 aim to understand and extend the Elo algorithm. CeaDuranGuajardoSureSiebertZamorano2020 and Kaminski2022 discuss the shortcomings of the previous FIFA World Ranking used until 2018. LasekGagolewski2021 apply a popular optimisation heuristic (the gradient descent algorithm) to build interpretable rating systems that can be easily adjusted once new results are observed. According to SzczecinskiRoatis2022, the predictive capacity of the current FIFA World Ranking would considerably improve by incorporating home-field advantage and can be further developed by taking the margin of victory into account. GyarmatiOrban-MihalykoMihalykoVathy-Fogarassy2023 evaluate three methods with respect to their performance in ranking European football teams.
Finally, some studies examine the predictive power of alternative indicators. According to OLeary2017, the Yahoo crowd outperformed experts at predicting the outcomes of matches played in the 2014 FIFA World Cup and was competitive with the accuracy of betting odds. Peeters2018 show that Transfermarkt valuations provide better forecasts for international football matches than standard predictors such as the FIFA ranking and the Elo rating.
The UEFA Champions League has been organised in the same format between the season of 2003/04 and 2021/22. The 32 clubs are divided into eight groups of four to play a home-away round-robin contest. In each group, the top two teams qualify for the knockout stage. The knockout ties are played in a two-legged format except for the final, which is played in a predetermined neutral stadium. Consequently, the group stage consists of $8 \times 12 = 96$ games, and the knockout stage consists of 14 clashes (28 games) without the final.
In the group stage, each group contains one team from each of the four pots. The pots are primarily determined by the ranking of the teams based on their UEFA club coefficients. However, the titleholder and the champions of the strongest associations have been assigned to Pot 1 between the 2015/16 and 2017/18 seasons CoronaForrestTenaWiper2019, DagaevRudyak2019, and the titleholder, the UEFA Europa League titleholder, as well as the champions of the strongest associations, are assigned to Pot 1 since 2018/19 Csato2020a. This allocation rule implies that teams having a similar club coefficient usually do not play against each other in the group stage.
The UEFA club coefficient is either the sum of all points won in the previous five seasons of European competitions (UEFA Champions League, UEFA Europa League, UEFA Europa Conference League) or the association coefficient over the same period, whichever is the higher UEFA2018g. In the Champions League, the first definition usually gives a higher value, although there are a few exceptions such as the German VFL Wolfsburg in the 2021/22 season, which had $14.5$ points but an association coefficient of $14.714$. These data have been collected from the unofficial, but comprehensive website of Bert Kassies Kassies2023a.
The Elo method has been developed by a Hungarian-born American physics, \'Arp\'ad \'El{\H o}, as an improved chess rating system. It is now widely used in many sports, including association football, as shown by the FIFA World Ranking since 2018 FIFA2018c. The method calculates the expected winning probability $W$ before any match according to the formula
where $\Delta$ is the difference between the Elo rating of the home and the away team, while $s$ is a scaling parameter. After the outcome of the match becomes known ($R = 1$ for home win; $R=0.5$ for draw; $R=0$ for away win), the ratings are updated: $E_1 = E_0 + K(R-W)$ with $E_0$ being the old and $E_1$ being the new Elo rating. Parameter $K$ reflects the importance of the match and also controls the speed of convergence; a high $K$ implies a quick convergence but makes the ratings volatile, and a low $K$ provides more stable ratings. Consequently, a win always increases the Elo rating and a loss certainly decreases it. A draw is favourable for the lower-ranked team. The two teams playing each other exchange points, that is, the sum of their ratings remains the same.
The Elo approach has many variants, we have used the Football Club Elo Ratings, which are available at \url{http://clubelo.com/}. Its formula applies the values $k=400$ and $K=20$, and accounts for home advantage and the margin of victory FootballClubEloRatings. This measure has served as the basis of a recent simulation that analysed a recent change in the qualifying system of the UEFA Champions League Csato2022b.
Formula (ref) reveals another advantage of an Elo-based approach over the UEFA club coefficient. In the case of the UEFA club coefficient, all wins (draws) in the group stages of the UEFA Champions League, the UEFA Europa League, and the UEFA Europa Conference League increase the rating by the same amount. Analogously, each round that the clubs reach from the Round of 16 in the UEFA Champions League and the UEFA Europa League means the same number of bonus points. On the other hand, a win against a team with a higher Elo implies a greater update $K(R-W)$ than a win against a team with a lower Elo since $W$ is greater in the second case. Furthermore, the sum of Elo updates for the two opposing teams is always zero. Consequently, the average Elo rating of a national league increases only if its teams perform better than expected in international competitions such as the UEFA Champions League or the UEFA Europa League. The average Elo ratings of domestic leagues are quite different: for example, this was 1,778 for England (the strongest team Manchester City had 2,013) and 1,303 for Hungary (the strongest team Ferencv\'aros had 1,581) on 30 June 2021.
The Elo ratings are continuously updated once a match is played by the teams. On the other hand, the UEFA club coefficient is calculated at the beginning of each season and does not change during the season. Therefore, in order to ensure the fairness of their comparison, we have fixed Football Club Elo Ratings at the level of 30 June each year when most European football leagues, as well as international competitions, have finished. Even though some matches in the first or the preliminary round of the UEFA Champions League qualification have been played before June 30, these teams are missing from our samples since they have never reached the Champions League group stage.
The performance of the UEFA club coefficients and Football Club Elo Ratings are compared by logistic regression models on the basis of the 19 UEFA Champions League seasons played between 2003/04 and 2021/22.
The descriptive statistics of the two ratings are summarised in Table (ref), separately for the group stage and the knockout stage of the UEFA Champions League seasons that are included in our database. The teams qualifying for the knockout stage have a higher value on average.
For the dependent variable, three options are investigated:
Naturally, the COVID-19 pandemic caused some disruptions. Almost all matches were played behind closed doors in the 2020/21 season, which could have affected home advantage BrysonDoltonReadeSchreyerSingleton2021. Furthermore, in the 2019/20 season, neither the UEFA Champions League nor many national leagues were finished by 30 June 2020. Thus, the Elo ratings used for 2020/21 do not contain the results of all matches played in the previous seasons. The 2020/21 season will not be considered in some regressions due to this potential bias.
Since our models contain the difference between the ratings of the two teams for each match as explanatory variable(s), the descriptive statistics of the corresponding independent variables are reported in Table (ref). For group matches, the average difference between the club coefficients and Elo ratings is close to zero. On the other hand, the mean is negative in the knockout stage because the runners-up are guaranteed to host the first game and the group winners are usually stronger teams in the Round of 16. It is worth noting that the highest difference with respect to Elo ratings occurred in the 2021/22 season when Sheriff Tiraspol defeated Real Madrid in Spain, causing one of the biggest shocks in the history of the UEFA Champions League OConnor2021.
The scale of the two measures is not the same, the difference between the Elo ratings is about three-four times higher than the difference between the club coefficients. Nonetheless, the variables will not be standardised since we primarily focus on comparing the performance of the models rather than the interpretation of the estimated parameters.
Some standard metrics will be used to evaluate logistic regression models Allison2013. Naturally, the regression is estimated by maximizing the likelihood function. Denote the value of the likelihood function without predictors by $L_0$, the likelihood of the final model by $L_M$, and the number of observations by $n$. Then Cox & Snell $R^2$ is \[ R_{C \& S}^2 = 1 - \left( \frac{L_0}{L_M} \right)^{2/n}, \] which is a generalisation of the usual $R^2$ for linear regression. However, the upper bound of $R_{C \& S}^2$ is not one but \[ 1 - L_0^{2/n} = 1 - \left[ p^p \left( 1-p \right)^{1-p} \right]^2, \] where $p$ is the ratio of the event to be predicted in the sample. Nagelkerke $R^2$ adjusts $R_{C \& S}^2$ by dividing it with its upper bound in order to get a value between zero and one.
Finally, McFadden $R^2$ is defined as \[ R_{McF}^2 = 1 - \frac{\ln \left( L_M \right)}{\ln \left( L_0 \right)}. \] The idea behind the formula is that $\ln \left( L_0 \right)$ can be regarded as the residual sum of squares in a linear regression.
Another measure to assess the performance of a logistic regression model is the area under the ROC curve. The ROC curve plots the true positive rate (sensitivity) as a function of the false positive rate ($1-$ specificity). The area under the ROC curve is the two-dimensional area below the ROC curve from $(0,0)$ to $(1,1)$. Consequently, the area under ROC is 1 if one has a perfect model, for example, each match is won by the team with a higher UEFA club coefficient. If the outcome is random, the area under ROC equals 0.5. A higher value of this indicator shows a more accurate model.
Section (ref) presents the baseline results for the three samples, and Section (ref) examines some other specifications to verify the robustness of the main findings.
First, in order to motivate the more sophisticated models, the rough prediction accuracy of the UEFA club coefficients and Elo ratings are presented in Table (ref) with the assumption that a higher (or equal) coefficient/Elo rating predicts success. As expected, group ranking is the easiest to predict, followed by group matches and knockout qualification. Crucially, the difference between Elo ratings robustly outperforms the difference between club coefficients.
Table (ref) shows logistic regressions for group matches won by one of the teams. The constant is highly significant, playing on the home field means a substantial advantage. Elo rating is a stronger predictor of success than UEFA club coefficient: model (2) has a higher explanatory power than model (1), and Elo rating is significant in model (3), while the club coefficient has no additional value here.
According to Table (ref), the same conclusions hold for the two-legged clashes played in the knockout stage. Again, model (2) outperforms model (1), and using the club coefficients is not able to improve the predictions based on Elo ratings.
Analogously, Table (ref) reveals that the Elo rating is more useful for predicting group ranking than the UEFA club coefficient as model (2) is more efficient than model (1), although the club coefficients now have a significant contribution when both measures of strengths are considered. In particular, a team with a fixed Elo rating is more likely to finish above another team if it has a higher club coefficient. This makes sense as better performance in previous European competitions can provide some experience for the squads that cannot be obtained by playing against domestic teams.
In the regressions for group ranking (Table (ref)), the definition of the dependent variable is somewhat arbitrary as it “assumes” that the team with a higher club coefficient should be ranked higher. Therefore, the estimations are repeated with an alternative specification when $1$ ($0$) indicates that the club having a higher Elo is ranked higher (lower). Then the dependent variable equals $1$ for 716 observations (78.5%). The results are reported in Table (ref). Again, model (2) has a better fit compared to model (1), but now the UEFA club coefficient is not able to significantly contribute to model (2) as can be seen in model (3).
Table (ref) has shown the results for all group matches without draws, which may contain an inherent distortion if the drawn games are different from the games won by one of the opposing teams. Hence, Table (ref) presents multinomial logistic regressions for the three possible outcomes (home win, draw, away win) with the reference category being away win. The number of observations increases to 1,824 as has been presented in the previous section. A higher club coefficient or a higher Elo rating significantly increases the probability of winning and playing a draw compared to losing. Model (2) clearly outperforms model (1), and the UEFA club coefficient is again insignificant in model (3), thus, the Elo rating remains a better measure of strength. Now there are three alternative interpretations of the area under the ROC curve, which suggest that draws are the most difficult to forecast.
In order to identify potential trends and check the robustness of the previous findings, the sample has been cut into two equal parts of 9-9 seasons together with removing 2020/21, which was affected by the Covid-19 pandemic to a great extent (see Section (ref)). According to Table (ref), there is only a slight difference between the parameters for group matches estimated on the basis of the first and the last nine seasons. Nonetheless, the predictions are somewhat more accurate since 2012, which might imply a worsening competitive balance that is in line with previous research Triguero-RuizAvila-Cano2023. On the other hand, the dominance of model (2) over model (1) is obvious, and the UEFA club coefficient still does not provide additional information to Football Club Elo Rating.
Table (ref) focuses on qualification in the knockout stage for the two subsamples. Again, model (2) outperforms model (1), and the metrics of goodness of fit are higher for the recent Champions League seasons if the model contains the Elo rating. As before, there is no reason to favour the UEFA club coefficient over the Elo rating with respect to predicting qualification.
Finally, Table (ref) presents the estimations for group ranking. The main message does not change: (a) the Champions League has been more predictable between 2012 and 2022 than between 2003 and 2012; (b) the Elo rating gives more accurate forecasts compared to the UEFA club coefficient. Furthermore, similar to Table (ref), adding the club coefficient is able to improve the model as its parameter is significant in equation (3).
Ensuring the balancedness of competitions is a fundamental issue of tournament design in order to avoid that a strong team has a lower chance to qualify than a weak team merely because of the outcome of the draw Csato2021a, Guyon2015a, LaprePalazzolo2022, LaprePalazzolo2023. According to our findings presented above, the Football Club Elo Rating robustly outperforms the currently used UEFA club coefficient in terms of predictive accuracy. This conclusion does not depend on the sample (group matches, knockout qualification, group ranking). Similarly, both pseudo-$R^2$ values and the areas under ROC support the use of Elo ratings instead of club coefficients.
In the following, it will be highlighted why this result is especially important for the UEFA Champions League.
The format of the UEFA Champions League has essentially not changed between the 2003/04 and 2023/24 seasons, even though there have been some reforms in the entry rules Csato2019c, in the use of the away goals rule Bahamonde-BirkeBahamonde-Birke2023, Jost2021, in the seeding policy CoronaForrestTenaWiper2019, Csato2020a, DagaevRudyak2019, as well as in the design of the qualification system Csato2022b.
However, UEFA will introduce a fundamentally new competition format in the 2024/25 season. In particular, the 36 teams will compete in one league where each team plays four matches at home and four matches away instead of the previous six matches against three teams, played on a home-and-away basis UEFA2022a. The top eight clubs of the league will qualify for the Round of 16, while the teams ranked between the 9th and 24th places will go to the knockout round play-offs to play two-legged clashes for the remaining eight places in the Round of 16. A similar design will be used in the other two European competitions, the UEFA Europa League and the UEFA Europa Conference League.
The novel competition structure is officially called the “Swiss system” UEFA2022a. The name has been inspired by a non-eliminating tournament format containing a fixed number of rounds that is widely used in chess Csato2013a, DongRibeiroXuZamoraMaJing2023. It is usually applied when the high number of participants allows only to play a considerably fewer number of rounds than required by a round-robin contest. In the original Swiss-system, the pairing of players in each round is determined by the results of previous rounds, ensuring that both opponents have an equal or similar score CsehFuhrlichLenzner2023, FuhrlichCsehLenzner2021. This is feasible in chess and some other sports, where the matches can be played at a given location or at least in the same city. However, dynamic scheduling is hardly an option in a football tournament played across the continent since the teams and the fans want to know at the moment of the draw the opponents and the field of all games.
Therefore, the schedule of the UEFA Champions League will be determined according to the following rules UEFA2023:
Obviously, it is crucial in this system to reliably estimate the performance of the teams in advance, which can only be achieved by a relatively accurate rating of the clubs. Otherwise, the league phase has a high probability of becoming unbalanced, meaning that a particular team mostly plays against opponents with a high number of wins, while another team mainly plays against opponents with a low number of wins. This will certainly be regarded as unfair, analogously to traditional Swiss-system tournaments Csato2017a. The early elimination of strong clubs can also have serious financial consequences as they attract the most attention from both the media and the fans.
This fundamental change in the UEFA Champions League offers a unique opportunity to modify the calculation of the UEFA club coefficient, too. Two recent reforms, approved by the UEFA and the F\'ed\'eration Internationale de Football Association (International Association Football Federation, FIFA), reinforce that our recommendation has a reasonable chance to be implemented in practice:
The current algorithm of the FIFA World Ranking “is not only intuitive, easy to understand and improves overall accuracy of the formula, but also addresses feedback received about the previous model and provides fair and equal opportunities for all teams across all confederations to ascend the FIFA World Ranking” FIFA2018c. Consequently, using the Elo method to quantify the strength of European football clubs seems to be a promising recommendation in order to guarantee fairer schedules in the novel tournament design of the UEFA Champions League.
This study has investigated the ability of two measures of strength---the official UEFA club coefficient and the alternative Football Club Elo Rating (\url{http://clubelo.com/})---to predict the performance of the teams playing in the UEFA Champions League, a highly prestigious and popular football competition. For the sake of comparability, the Elo ratings have also been fixed at the beginning of each season. Since the club coefficient does not take the games played in the national leagues and cups into account, it is not surprising that it is outperformed by the Elo rating, which contains this information.
Our findings can be interesting for tournament organisers, especially for administrators who are responsible for the methodology of coefficients used for ranking, seeding, and distributing prize money in sports competitions. In particular, we have a clear message for the Union of European Football Association (UEFA): it is time to reform the calculation of club coefficients used for seeding and distributing prize money Csato2023f, UEFA2022c in European club football. This would be especially important because the new tournament format of the Champions League, to be introduced in the 2024/25 season, requires an accurate measurement of teams' strength in order to create a fair schedule.
Naturally, the Football Club Elo Rating is not necessarily the best possible predictor. The Elo algorithm is able to incorporate several characteristics of the matches and preferences of the decision-makers SzczecinskiRoatis2022; for instance, games played in European competitions could have a higher weight compared to games played in the national leagues. Hopefully, testing and comparing these variants will be the topic of future papers. In addition, simulations may uncover the sporting effects of using an inaccurate measure of strength such as the current UEFA club coefficient, and the importance of finding the hidden ranking of the teams in various competition formats.
\addcontentsline{toc}{section}{Acknowledgements} This paper could not have been written without Gergely Bodn\'ar, who has helped with data collection. \\ Kolos Csaba \'Agoston, Andr\'as Gyimesi, and D\'ora Gr\'eta Petr\'oczy have provided valuable comments and suggestions. \\ Three anonymous reviewers gave useful remarks on earlier drafts.