Is Performance in Close Matches Luck or Skill? (Goal Diff and Points - Part 3)

Note: Here are parts one and two of this series. I strongly recommend reading those before reading this, though I give a summary at the beginning if you just want to get to the business of which teams have gotten lucky and/or been clutch over the last several years.

In the first article I discussed how well a team's number of league points at the end of the season corresponds to their goal differential. The answer was extremely well. In the second article I discussed why a team would over or under perform relative to their goal differential. This comes down to how good their results were in close matches compared to teams that are similarly skilled, at least as judged by their goal differential. I then discussed why some teams would getter better results than others.

The question at hand is whether there is a "close-match skill" that causes some teams to play better when it matters most compared to teams of the same skill level otherwise.

Correlation

To test this, I used the same data set as before. It includes every season of the Spanish Primera Division from 1987-1988 to 1994-1995 and from 1997-1998 to 2008-2009. I also have the English Premier League for all seasons since 1995-1996 and the Serie A since 2004-2005. I believe this to be all seasons for the three leagues in which they had 20 teams. For the first 8 seasons of La Liga I converted the points so that a win would be worth 3 points as it is today instead of 2 as it was then.

Using this data I looked at the correlation between how many points above or below goal-differential expectation each team gets a given season and the next season. In footballing terms this statistic will tell us if a team's performance in close matches is consistent from one season to the next. A large positive figure would indicate that teams doing particularly well in important situations this year will tend to also do so next year. A large negative value would mean that teams that do well for their skill level this year will tend to be bad at that next year. A value close to zero would tell us that performance in close matches this year means nothing as far as next year goes. In other words, values close to zero mean that there is no evidence that performing especially well in crucial situations is anything other than luck.

Here is the correlation coefficient for each of the three leagues:
English Premier League: -0.0218 (sample size 240)
Spanish Primera Division: 0.05579 (sample size 357)
Italian Serie A: 0.02312 (sample size 73)

As you can see these values are all very close to zero. In fact, the correlation coefficient for the Premiership is negative! For the fellow nerds out there, the p-values for the one-sided t-test were 0.147 for the Spanish league and 0.423 for the Serie A. The correlation for the Spanish league is in the ballpark of being statistically significant but is not at the usual 5% level, or even at 10%. Given the large sample size, it's safe to conclude that the correlation is effectively 0. There is no evidence that teams overperforming their goal-differential expectation can be attributed to anything other than luck, or at least anything that would carry over from one season to another.

A Look At Individual Clubs

While that is very conclusive, let's look at how different teams have done compared to their goal-differential expectation. For all teams in England with at least 10 seasons in the sample, here is a chart with the average difference between actual points and goal-differential expectation each season.



Liverpool have gotten an average of 1.5 fewer points than their goal-differential suggests that they should. At the other end, West Ham have managed to outperform their expectation by nearly 3 points a season. That Manchester United have done better than expectation might be the least shocking thing I have uncovered in all of my research for this blog.

Looking at the numbers, I think the chart is more evidence that there isn't skill involved in outperforming your goal-differential - that it's just luck. One reason for this is that the top teams are all over the place. Admittedly, Manchester United are near the top and are easily the best club in England over the sample. Other than them though, Chelsea and Arsenal have run close to average, while Liverpool are well below average.

On the flip side, West Ham United are the team that has been most successful compared to their skill level! While they haven't been the worst club to see the top flight, a look at them over the years of the sample gives one no reason to think that they have something that makes them the best team in close matches compared to their quality. They went through 4 managers during that time and were relegated. The relegation isn't really relevant as absolute level isn't what we're looking at but how well a team does in close games compared to their skill level. To me going through four managers is important. It would be easy to point to Alex Ferguson as the reason for Manchester United outperforming their goal-differential expectation, but then why would a club that sacked four managers in that time do even better compared to their expectation?

Here is the list for La Liga. I included all teams with at least 10 seasons in the sample:



Once again, we have a similar pattern. Strong teams such as Athletic Bilbao and Real Madrid are near the top but then again there are big clubs like Barcelona and Atletico de Madrid at the other end. Sporting Gijon is similar to West Ham. Again, just glancing at the list of teams that over and underperformed, it seems to be mostly luck, if not all.

For completeness here's the chart for the Serie A:



Just like the others it features a huge club at or near the top and bottom. Reggina plays the role of club that has seriously over performed in close matches for no obvious reason other than luck. Looking at both this and the Premieship data, I found it interesting that Liverpool and AC Milan have under performed in their leagues. Both have a reputation for grinding out wins in cup competitions, most notably the Champions League, even when playing against superior opponents. Apparently those reputations didn't help them any in those spots in league fixtures.

Conclusion

Looking at points earned compared to the goal-differential expectation from one season to the next there is no correlation. Teams that over perform in close matches this season are no more likely to do so next year than those that under performed. In other words, there is no evidence that some teams are able to step up in important situations better than others. Looking at which teams have historically done better or worse than their goal-differential expectation helps to confirm this. While for each of the three leagues there is a team or two near the top that one might think could have this ability to step up, there are other teams at the bottom that it seems should also have it. As importantly, there are teams that have overperformed that have lacked consistency in management and squad so it seems they couldn't have done so if the ability to rise to the occasion were real, or at least an important factor.


Scoring and being scored on are based on a combination of skill and luck, but having the goals come at the right time seems to be all luck.

Goal Differential and Points - Part Deux

In a previous article I discussed goal differential and how well it explains how many points a team gets in a season. I would highly recommend reading that article for those who haven't. The short of it is that goal differential almost perfectly explains how many points a team gets. Other than goal differential, how well a team does in close matches compared to blowouts determines how many points they will get. If a team wins a lot of games by a single goal and then gets beat several times by 3 or more goals then they will over perform. In other words, they will get more points than there goal differential indicates that they "should" according to their goal differential.

The question at hand is whether this is based on luck, skill or perhaps something else entirely. Here are some possible explanations.

Luck

The main argument for luck is that it plays a role in every single goal that is scored. The difference between a perfectly played ball and one that goes wildly into touch is probably less than a millimeter in where the foot strikes the ball or a tiny change in the amount of force applied. The team aspect of the game also throws an element of luck into the build up and scoring chances. Sometimes your perfect through ball will be run onto by your teammate, other times he won't get there because he was expecting you to play it directly to him. The referee also adds some variance. Sometimes the linesman will miss a player just offside, other times he'll get it right and on occasion he'll rule that a player that was onside was off.

Over the course of a season, these things will even out so that the number of goals scored and allowed will be pretty close to representing how much skill teams have at each end of the field. On the other hand, luck still will have played a big role in the timing of those goals. Some matches it will all be clicking and you'll beat a team by 3 goals, netting two you didn't really need. Other days you'll play an inferior opponent but will be held to a 0-0 draw because the bounces weren't going your way. If things happen to go your way when the score is close more often than they do when you are well ahead or behind then you will do better than teams with the same goal differential and vice-versa.

Luck certainly plays some role, I don't think anyone could reasonably argue otherwise. Whether it's a small factor or the only thing that matters is impossible to prove as it is the null hypothesis. In other words, we can only show that luck is the only factor by showing that nothing else could explain it.

Player Skill

The argument for player skill is that in addition to football skills there is a mental skill that allows some players to play better than they usually do (or not as much worse as usual than others) in more important game situations. In American sports, this is often referred to as a player being clutch. In soccer this could appear a number of ways. The most obvious example is penalty kicks. When you're watching it certainly seems like some players are better than others at not letting nerves get the best of them and just burying it as they should. In the run of play, you could see this if an attacking player is more likely than others of the same skill level to score in the dying minutes when his team needs a goal to even the score or go ahead.

Unfortunately, because football is a team sport evidence of this is going to be hard to come by. Perhaps one could look at attacking players' scoring rates in various situations. That doesn't seem like it would work though as there would be serious sample-size problems and it would be hard to untangle other possible explanations like tactics and the play of teammates.

Manager Skill

Differences in tactics could also play a role. Suppose a manager has a tactic that drastically improves his team defensively, but that also makes them unlikely to score. If he is ahead by a goal near the end then he will use it and as a result his team will give up fewer equalizing goals when they are ahead. They will also score fewer goals in that spot. This has a double effect. They will perform better than the average team of their skill level in close games and also score fewer useless goals making their goal differential less than other teams of their skill level. This will cause them to over perform relative to their goal differential. Similarly, if a manager has a tactic that drastically increases his teams chances of scoring but also being scored on then the same will hold because they'll draw even more often when behind in a one-goal match and will also give up more extra goals.

This and player skill is difficult to untangle, but the argument could be made that a manager skill is picking players who are able to step up in important situations. To look for this, I will compare how well different teams perform compared to their goal differential through various seasons. If the manager or group of players is important then some teams should consistently outperform those with the same goal differential.

Other Factors

The above explanations focused on doing better or worse than teams of a similar skill level in close games. It could also be the case that some teams do better or worse than those of the same skill level when they are well ahead or behind another team in a match. On the positive side of things, if a team has a lot of competition for starting spots then the players might be more willing to play with full effort even when the result has been decided in order to impress the manager. If this is the case, then they will get more goals on average when they are already ahead by two or three goals and will underachieve compared to their goal difference. Another way to think about it is that their goal differential would be higher than their actual skill level so they would get fewer points than those with a similar difference in goals. On the negative side, a team may give up more easily than others. This could be due to a lack of mental toughness or players smartly backing off a bit in order to conserve energy for the long season and prevent injury. To clarify, it not only has to be the case that a team does this, but they would have to do so more than the average team for it to matter.

Future Work and Discussion

The next article in the series I will look into these explanations and examine whether some teams seem to perform better in important situations than others with similar skills.

In the meantime, I'd love to know your opinion. Do you think it's all luck or is player and manager skill important? Do you think it might be something else that I didn't mention? Please give your thoughts below.

How Well Does Goal Differential Explain Points?

While championships, continental-cup qualification, promotion, relegation and so on are based on league points, much of my analysis is based on goal differential. The Poisson model effectively uses goal differential as its inputs are goals for and goals conceded. Any player analysis, whether based on statistics or watching the game, will overwhelmingly be related to how much a player contributes to his team scoring or preventing goals. An important question is how related goal differential and league points are. More specifically, how much are points based on goal differential and how much are they a result of other factors like luck, tactics and the ability for some players to play particularly well in situations where it matters more?


How Related Are Goal Differential and League Points?

In short: very.

To study this question, I took data from the Spanish Primera Division, English Premier League and Italian Serie A. Because I wanted to only look at seasons in which the given league had 20 teams, the number of years I could use varies by league. For Spain, I looked at all seasons between 1987-1988 and 1994-1995 as well as those from 1997-1998 to last season. From England I include all seasons from 1996-1997 to 2008-2009. The Serie A had 18 teams for some time, so I only looked at 2004-2005 to last year. For the first 8 La Liga seasons, only 2 points were given for a win. To keep things the same, I converted the number of points for each team to what they would have gotten under the current system in which 3 points are awarded for a win and 1 for a draw. I considered removing these seasons from the sample, as well as the 2005-2006 Serie A season due to the match-fixing scandal, but they didn't appear to be different so I left them in.

Here is a scatter plot with the combined results from all three leagues. The horizontal axix represents the team's goal differential while the vertical axis gives the number of league points they attained.



As you can see, there is a clear relationship between points and goal differential. In fact, the correlation coefficient is an incredible 0.955. The line in the graph is the result of what is called ordinary least squares regression. OLS takes the data and comes up with an equation for the line that minimizes the sum of the square of how far each point is away from the line. In other words, it derives an equation for the line that best fits the data. In this case, the equation is Points = 51.863 + 0.654*Goal Differential. So a team that scores the same number of goals as they conceded would be estimated to finish the league with about 52 points. A team that scores 10 more goals than they allow would finish with around 58 points. On the other hand, a team that scores 20 fewer goals than they conceded would be expected to finish the season with about 39 points.

For anyone who has worked with data before, something particularly impressive about this OLS regression is the R-squared (hereafter R2). R2, also called the coefficient of determination, is a measure of how much of the variation in the response variable is due to the variation in the explanatory variable. Applying this to goal differential and points, it tells us how much differences in points for teams in a given season are due to differences in goal differential. In more simple terms, it says how well points are explained by goal difference alone. For the data given in the graph above, the R2 is .9117. This means that about 91% of how many points a team gets at the end of the season is due to their goal differential. Less than 9% of it is explained by anything else!

What is going on here?

The incredibly high correlation and good fit of the linear equation is an indication that the timing of goals doesn't have much impact when it comes to how many points a team gets in a season. This seems odd since the (marginal) value of a goal is quite related to the score. If a team is ahead by 2 or more goals in the dying moments then scoring or conceding a goal makes no difference - they'll get 3 points either way. If they are tied in that situation then a goal for them is worth 2 points and letting one in would cost them a point. If they are down a goal then scoring one is worth a point and conceding one doesn't negatively affect them. On the other hand, no matter what the score is, scoring a goal increases the team's goal differential by 1 and letting one in decreases it the same amount. Despite the value of a goal changing a lot depending on the situation, the number of points a team gets in a season is almost completely explained by a stat that assumes all goals are equally important.

It must be the case that, for the most part, things tend to even out over the course of a season. One game your team may win 3-0, but later in the season they'll lose a couple matches by 2 goals. There is apparently little variation across teams when it comes to how many superfluous goals they score or concede compared to important goals.

Implication for future work.

This result is very good news for analyzing the game. For teams it implies that tools like the Poisson model, which are based on either goal differential or goals scored as well as goals conceded, can be very good at determining how well a team gets points. For player analysis, it is helpful because it means that if we can somehow sort out a player's contribution to his team scoring and being scored on then it would pretty directly determine his value. Overall it will simplify things a lot.

What about the other 9%?

This I will discuss in a future article.

The question is essentially whether anything but luck determines how well a team does compared to their goal differential. Is there a skill that causes some players or managers to perform disproportionally better or worse in close games? More succinctly, does clutch exist in football?