The Numbers Game
by Andrew Godden
There are lies, damn lies, and statistics – but that’s enough about the Editor’s love life! This explanation of the increasingly popular xG statistical system by Andrew Godden is far more interesting…..
One benefit of the Swans being in the Championship is that we’re saved the debacle that is the way VAR has been implemented in the Premier League. As a fan of increased use of technology in decision-making (yes, still bitter about THAT FA Cup QF against Man City) I had high hopes VAR would prove the sort of benefits we’ve seen in rugby (both codes) and cricket. But no, it’s been used very poorly, to the point where even its backers are fondly reminiscing about the pre-VAR days. So, thanks for that!
This isn’t an article about VAR though, it’s about football’s relationship with technological advancements and how we use them. This is an article primarily about statistics. If you click onto the next article at this point, I will totally understand, but please bear with me!
Many of you will have seen Moneyball, a film about a baseball team built entirely on the use of statistics and how it led to decisions being taken that other teams simply wouldn’t make using more traditional methods. It’s a film, so not entirely accurate, but it gives a good feel for how statistics are extensively used in American sports.
The same can also be said for rugby union, where the concept of marginal gains are everything. If you want a nap, watch a Six Nations game involving England, and see how they’ll always kick the ball from their own half to limit the potential to concede kickable penalties.
Football, on the other hand, has been very slow to see the merits of statistics in the professional game. This is curious as games such as Football Manager have developed a massive base of devotees particularly around its database of player related statistics. The latest iteration of FM has almost 40 player characteristics associated with each player which, given the number of players around the world, is quite a phenomenal dataset.
There are some great stories about agents attempting to influence or even bribe directors at Sports International to give their players better ratings (note: they didn’t succeed). How these values are obtained is an interesting debate, previously it was all done by researchers (usually fans) giving their views, so data quality could be of variable, but it’s potentially very useful data and clearly better than nothing.
Statistics are becoming increasingly used within football clubs for player recruitment and performance analysis purposes.
We recently heard Andy Scott talking about how he’d been tapping into the DC United database of South American players as we look to pick up bargains in new markets.
A current favourite metric for analysing match performance, amongst a certain section of fans and commentators, is Expected Goals (xG).
This is a metric that, to directly quote Opta, “measures the quality of a shot based on several variables such as assist type, shot angle and distance from goal. Adding up a team’s expected goals can give an indication of how many goals a player or team should have scored on average, given the shots they have taken”.
Opta used 300,000 shots across the top football leagues to develop this metric, which is a very considerable statistical sample.
So far, so interesting. We’ve all seen games where a team has battered the opposition, hit the woodwork six times before being sucker punched in the 94th minute to lose 1-0. Such a metric can help quantify just how big a robbery that result was, and how unlikely it is to occur in general.
If we look over a longer period, such metrics are useful guides to predict how a team will perform over the season. I’m reminded about the Roberto Martinez quote (no, not that one) that, in the long-term, if you get the performance right then results will take care of themselves. I couldn’t agree more.
If you’re a devotee of xG then you may want to look away now as, whisper it quietly, it does have some weaknesses.
Firstly, it simply ignores player quality. Let’s take a simple example – penalty kicks. An xG value has been calculated for penalties scored based on all those that have been included in the sample set. Let’s say 60% were scored, so the xG for a penalty is 0.6. How does that fit the Swans? Rather poorly as it goes, as Andre Ayew doesn’t tend to miss, and Freddie Woodman saves them more often than not.
Let’s branch out a little further and look at free kicks outside the box, which may have an xG of 0.1, if 10% of them are scored. With all due respect to Matty Grimes, who I’m a big fan of, that’s wildly optimistic given we didn’t score a direct free kick since Gylfi’s time here. It woefully underestimates Conor Hourihane, who has scored two in a short period of time.
Moving to shots outside the box, you can guarantee that goal against Norwich had a very low xG score too.
How a team defends, or attacks, is also not factored in. Using xG, a shot taken from 12 yards out in the centre of the goal will be assigned an xG of, say, 0.2. What this doesn’t consider is a) how much space the player has in the box and b) how many defenders are in the vicinity. It tells you that someone scores from this position 20% of the time, but it doesn’t really distinguish in terms of the true quality of the chance.
It thus inherently prioritises quantity over quality, both in terms of the chance and the individual.
Another weakness of xG is game management. What typically happens when a team goes a goal up? They sit back. The urgency to score another goal has gone. Flip that around, and if you’re a goal behind you’re chasing the game, attacking more and thus inevitably racking up the xG points even if the true chance of you scoring from any one of those chances isn’t particularly high. As we know, it’s harder to score if the other team is focused entirely on defending and parking the bus.
Lots of the xG resources are paid for, so I’m relying on the free ones such as Infogol here, but some of the individual game scores highlight the issue.
Let’s take the South Wales derby in December. One-way traffic, right? The phrase Men vs Boys was chucked out on both sides of the divide, and it was. What does xG say? Cardiff 0.67 vs Swansea 1.19. That made it look like a narrow win. It wasn’t.
It works both ways of course. The recent Huddersfield game resulted in a 4-1 defeat but, from an xG perspective it was a relatively narrow 0.95 to 0.42. Based on xG, we’re expected to draw that game once Hourihane scores that free kick.
If you want one that really doesn’t make any sense, consider the Watford game at the Liberty. That was a dominant 2-1 home win in the opinion of most observers. However, we ‘lost’ the xG 1.34 to 1.64.
I’m sure this will be read as a dismissal of xG and it really isn’t. I think it’s a really interesting stat and, when used in association with other things, I’m sure it’s a useful metric in the analyst’s toolkit. It will be a good fit for some teams and some situations.
However, like any statistical model, it has its flaws and clear areas where the next iteration of xG, or more likely an entirely new statistical model, will look to improve. If a team is consistently defying xG over the course of the season, then maybe it’s a weakness in the model rather than an indication that the team will revert to the mean.
It is still important to trust your eyes when assessing a game. None of these games were close contests in the eyes of both sets of fans, so this isn’t our own perception bias. The beauty, and unfairness, of football is that the only stats that ever truly matters are the scoreboard and the end of season league table.
It is important to remember that there are still other, less intangible, things to consider, particularly at a time when statistical analysis is in its infancy. It was great to hear Andy Scott talking about the due diligence that we do to make sure the player we recruit is the right fit, which goes far beyond bare stats.
It’s important we understand those limitations and that is something its disciples, usually fans, don’t always appreciate.
The irony is that the people that develop these statistical models will appreciate that better than anyone. They’ll have faith in their models, but they won’t believe they’re the best that could ever be achieved. It’s in their interests to develop the best models they can, otherwise they ultimately become worthless as people realise these limitations and other mathematicians develop more accurate models.
That’s been the history of mathematics for millennia and I am sure they are constantly questioning their own assumptions. If they’re not, they’re not really doing their jobs.
It’s early days for the use of statistics in football, but it is here to stay and already adds considerable value.
While I doubt we’ll ever come to a unifying formula for predicting future results based on past performance, I’m sure these models will become more powerful, smarter and generally more useful.
They have in other sports and industries so it’s inevitable. Just don’t blindly believe they’re the answer for everything. They aren’t. Not yet anyway.
