Thursday, December 3, 2009

Stat Series: Fouls

I'll now continue the stat series by looking at fouls. Like corners and time of possession, about which I'll write in the future, fouls are often used as an indicator that one team was a lot better than the other in a match. A defender usually fouls a player with the ball because he cannot otherwise make a tackle. It is often because the attacker is making a run past the defender or receiving the ball in a dangerous area. Other than a few exceptions like fouling the keeper on a corner, fouls tend to be called on you when your team is getting outplayed, at least for the couple seconds before they occur. But are fouls really a strong indicator of performance in a match?

If that is the case then we should see a link between the number of fouls committed by each team and the match result. If getting fouled a lot more often than fouling your opponents indicates that you are playing better, and it's a given that playing better leads to winning, then getting fouled more often should be well correlated with getting results.

Looking at the data, I was quite surprised to find that there is practically no connection between the number of fouls called for each team and the results of matches. In other words, foul statistics seem to not indicate anything at all in terms of which team was better on a given day. My data set is all matches from the previous two seasons in the English, Spanish, Italian, German and French top flight. I may expand the dataset for this study but frankly I didn't see the point after seeing what the results were for the last two years.

I looked at it a couple different ways. The first was just to look at the correlation between foul differential (home fouls minus away fouls) and goal differential (home goals minus away goals). A positive correlation would indicate that fouling more often than your opponent tends to lead to outscoring them. A negative correlation would be more in line with what you would expect; if you get fouled more often than your opponent you will tend to outscore them. If the correlation coefficient, which varies between -1 and 1, is very close to zero then that indicates that there is little relationship between the two. As it turned out that was the case. The correlation coefficient was -0.01278. In other words, the data indicates that there is essentially no link between number of fouls committed by each team and the goal difference in a match.

To look into this further, I broke the matches up into three groups: those where the home team committed more fouls than the away team, those where the away team fouled more and those where the two teams committed the same number of fouls. In the sample there were 1,582 matches where the home side committed more fouls. In those, the home team won 729 times, got a draw in 415 and lost 438. When the home team committed more fouls they averaged 1.64 points per match. The away team fouled more in 1824 matches. In those, the home team went 853-484-487 for an average of 1.66 points per match. There were 246 matches where the two teams were called for the same number of fouls. In those the home side went 122-55-69 for 1.71 points per match.

For the fellow nerds out there, there is no statistical significance when comparing 1.66 points per match and 1.64. More importantly though, there is no actual significance! "Winning the fouls battle" was only worth 2 hundredths of a league point. That is absolutely nothing. If you gave that up over an entire season, it would still cost you less than one point on average: 0.76 for a 20-team league. In a previous article on goal differential and points, I showed that there is a very strong link between goal differential over a season and points. Using the formula from that article, giving up .02 points per match every match for the entire season is about the equivalent of conceding one more goal overall. The stats indicate that the difference in points from getting fouled more or fewer times in a match almost certainly is due to random noise and not a real effect. My point is that even if it were due to something real, the difference is so small that it isn't relevant.

To put the final nail in the coffin, I looked only at matches where one team or the other committed a lot more fouls. In those where the away team committed 5 or more additional fouls than the home team, the home side went 442-273-260 for an average of 1.64 points. When the home side committed 5+ fouls more than the away team they went 330-196-208 for an average of 1.62 points per match. So the difference is the same as in the other case. I find it curious, though it doesn't matter at all, that the home teams did better in matches where the number of fouls committed by the two teams were close than either extreme. So even in the extreme case, it's extremely likely that the difference is just due to noise and even if it isn't the difference is so small that it doesn't matter.

There is no evidence whatsoever of a connection between the number of fouls committed by each team in a match and the result. If in the future you see me using the difference in fouls to make the case that one team was better than the other despite the result, please yell at me in the comments.

In the future I will look at full-season information for different teams as well as different leagues. It is possible that there is a link between fouls and results in individual matches, but it is restricted to certain leagues due to style of play or officiating.

Wednesday, December 2, 2009

FIFA Announces Pots for Draw

FIFA has announced the pots for the World Cup draw, with one major surprise. Instead of using the same type of formula they used last time, which I used in a past article to calculate the 7 seeded teams along with South Africa, they just used the FIFA rankings from last month. This meant that France dropped out and the Netherlands became seeded. That's a pretty big break from tradition as they've used roughly the same seeding formula for the last handful of World Cups.

Here are the pots:

Seeded:
Argentina
Brazil
England
Germany
Italy
Netherlands
South Africa
Spain

Pot 2:
Australia
Japan
North Korea
South Korea
New Zealand
Honduras
Mexico
United States

Pot 3:
Algeria
Cameroon
Côte d’Ivoir
Ghana
Nigeria
Chile
Paraguay
Uruguay

Pot 4:
Denmark
France
Greece
Portugal
Serbia
Slovakia
Slovenia
Switzerland

Here are a few thoughts:
- The second pot contains no teams from the same confederation as the seeded pot. This means that for those teams, each of the seeded teams are equally likely.

- If you are like most people and think that South Africa is by far the easiest draw from the seeded pot then the unseeded South American sides have an edge because each has a 1 in 3 chance of playing in South Africa's group. This point and the last one are because no two teams from the same confederation other than UEFA can be put into the same group.

- If you're French or a fan you obviously aren't happy. If you are Dutch then I'm sure you are. For the rest of the world, the only change as far as the seeding goes is that France replaced the Netherlands as the big team you'd like to avoid. In my view, and probably that of most others, France, Portugal and the Ivory Coast are clearly the best three teams that aren't seeded. I think it's a lock that one of those three will be in what is considered the group of death.

- The United States and Mexico are also very likely to be in the group of death. Keep in mind that this doesn't mean the draw will be tougher. The US and Honduras have the exact same probability of drawing any given set of opponents, but because the US and Mexico are the best two teams in their pot they are likely to be in the group of death.

- Perhaps because of South Africa or maybe just some kind of recency bias, it seems like there is more on the line in terms of how tough the groups could be. A possible group is Brazil, the United States, Ivory Coast and France. Another is South Africa, North Korea, Uruguay and Slovenia. For countries like the US, Denmark, Chile and Ghana there are possible groups where it would be very unlikely to get out, maybe a 10% chance, and others where it would be a near certainty.

Tuesday, December 1, 2009

Spanish League Post-Clasico Update

Barcelona went back on top by virtue of their 1-0 win over Real Madrid. To be honest, Madrid far exceeded my expectations in the match. They actually outplayed Barcelona for long stretches, especially in the first half and Barça were fortunate in that Zlatan put away their only clear chance of the match. I expected it to be like last year when Barça played Real Madrid off the park both times. Elsewhere, Sevilla got a disappointing result with a draw versus their Andalusia rival Malaga. The result didn't feel particularly fair as Sevilla dominated, including the first half even though the score didn't indicate that; they went into the break down 0-2. Valencia also failed to win at home, picking up just a draw against fellow Catalan speakers Mallorca. Atletico de Madrid perhaps are getting it back as they crushed Espanyol 4-0.

Here are the rankings:


The last column is the expected goal differential if the teams played a new season at the level the results so far indicate. O and D are offensive and defensive ranks which are based on how many goals scored and conceded as well as the opponent strength.

Barcelona remain best at both ends of the pitch. Sevilla slipped a bit due to giving up two goals at home to Malaga, who are pretty weak when it comes to scoring.

Monday, November 30, 2009

EPL Update - Chelsea Clear Top

With the big win over Arsenal at Emirates, Chelsea cemented their place at the top. The blues remain 5 points clear of Manchester United and are now 11 points ahead of the Gunners, with Arsenal having a match in hand. Looking ahead to December, Chelsea have Man City next and then a pretty comfortable list of fixtures: Everton, Portsmouth, West Ham, Birmingham and Fulham. They have a great shot at being in a dominant position come the break. They will have to be good though since Manchester United has a pretty similarly easy schedule for that stretch. I thought it would be different but at this point it's tough to see anybody but those two winning it, with Chelsea the favorite at the moment.

Here's the updated rankings:

Again, the last column is the expected goal differential if these teams played out a full season at the level shown so far.

Someone asked me when I expect these numbers to start to converge. I don't have a good answer to that and it's something I'll be looking into. They will certainly converge at the end of the season which is a nice feature of the model. Most teams have played 14 matches, which by the way is right around the number of matches for a World Cup qualifying campaign. I think things should be pretty good, but because right now the model only uses goals and no other statistics, it is still subject to randomness.

When I was checking on the numbers, I discovered something interesting. I've said before that teams like Arsenal and even Chelsea are overrated when it comes to scoring as they've likely been running over expectation. The reason for that is that the model predicts an expected goals of 108 for Arsenal and 97 for Chelsea when both clubs scored 68 goals last season with similar rosters. The model accounts for schedule so it's a bit different, but both of those clubs are on pace for about as many goals as the model says. When I compared other clubs' expected goals with last season, I discovered that 14 of the 17 teams that played last season in the EPL have a higher expected goals scored.

I thought this might be a problem with the model, but for all matches thus far the average number of combined goals is 3.04. Last season it was 2.21. If scoring stays at this pace, there will be about 318 more goals scored this year than last year in the entire league. I have no explanation for why this would be the case, but scoring seems to be up in a big way. I'm going to write another article or two for the stats series but this raised another interesting question which I'll write about. I wonder if scoring is often higher in matches in the early part of the season compared to later.

Saturday, November 28, 2009

EPL Mini Preview

Sorry for the lack of posts this week. I've been visiting family for Thanksgiving, a major holiday in the US. So as not to leave you completely hanging, here are the model predictions for the EPL this week. I should have a more-detailed preview for Arsenal - Chelsea and the Clasico tomorrow. I know it's late but perhaps you can read it the morning of the matches to get in the mood.

Saturday:

Blackburn - Stoke
Blackburn: 43.5%
Stoke: 27.4%
Draw: 29.1%

Fulham - Bolton
Fulham: 65.9%
Bolton: 11.7%
Draw: 22.4%

Man City - Hull City
Manchester: 75.6%
Hull: 7.1%
Draw: 17.3%

Portsmouth - Manchester United
Portsmouth: 22.5%
Man United: 47.9%

West Ham - Burnley
West Ham: 56.5%
Burnley: 18.2%
Draw: 25.2%

Wigan - Sunderland
Wigan: 29.8%
Sunderland: 42.1%
Draw: 28.1%

Aston Villa - Tottenham
Aston Villa: 39.8%
Tottenham: 32.3%
Draw: 27.9%


Sunday:

Wolves - Birmingham
Wolves: 36.9%
Birmingham: 31.9%
Draw: 31.1%

Everton - Liverpool
Everton: 26.1%
Liverpool: 47.5%
Draw: 26.4%

Arsenal - Chelsea
Arsenal: 49.1%
Chelsea: 25.7%
Draw: 25.2%

Monday, November 23, 2009

Liga Update

This week is obviously a big week for the Spanish League with the Clásico coming up on Sunday. The two biggest stories there are that Barcelona were held to a draw, putting Real Madrid on top going into their match, and that Cristiano Ronaldo is expected to play in the Champions League in the midweek and then in the Clásico. Other than Athletic de Bilbao holding Barcelona to a 1-1 draw, there weren't any surprises.

Here's the rankings table:

Premier League Rankings Update

Tottenham threw a nice wrench into things with a 9-goal win. That's unusual enough that I suspect my models will overrate Spurs by a bit for the rest of the year. That will be reduced over time as the total number of matches goes up but I don't think it will go away entirely. After getting just a draw, Liverpool have just about completely left the title race. I thought the big four would all be in it 6 weeks from now so I'll just call that prediction dead. With Arsenal hosting Chelsea this weekend, it could go anywhere from a wide-open three-club race to Chelsea in great position well clear.