In previous posts in this series I looked into some of the basics of PPG and what that meant for recent seasons, and proposed a more useful interpretation of it for putting player performance into context. Now to let loose somewhat on some of my reservations with its usage.
It gets skewed by small sample sizes
In our earlier assessments we set a qualifying criteria for the number of league minutes played by a player to enter consideration in a PPG table – 1,035 minutes, which is enough to play in all of a quarter of the games (or a quarter of all of the games, or anywhere in between). For the last four completed seasons that gave us a pool of between 14-17 qualifying players per season, enough for 64 data points in total. Over those four seasons those 64 most used players played an average of 2,554 minutes, which equates to 28.4 games.
So then, let’s consider a working example with some ‘averagely used’ players. Let’s say that Player A and Player B – both central midfielders – play in the first 28 games of a season together, with their side amassing a decent 45 points (and 1.61 ppg) for their efforts. For game 29 Player C, the club captain, comes into the side after a long layoff from injury and supplants Player B in the team, but they lose. Player A’s PPG drops to 1.55 and he’s subsequently dropped for the next game and Player B reinstated. The team go on to win game 30 and Player B has a new PPG of 1.66! The natural implication given only their PPG stats alone would be that over the course of a full season the team would be five points better off (76 instead of 71) with Player B in the side than with Player A, probably sneaking into the playoffs as a result. Yet, we can’t reliably measure the effect Player C has had after only two games (PPG of 1.50!), and so it should be equally unreasonable to jump to conclusions regarding Players A and B, even though they now have an apparent gulf between their relative contributions.
It doesn’t account for difficulty of schedule
Thanks to the Comet’s recent observation on player deployment home and away, I looked at those players who appear to have been played more often either at home or on the road and observed more than a few outliers. It’s generally considered (though by no means a truism) that teams earn more points at home than away, but we make no allowance for a player who’s been played 25% more often at home (e.g. Stacy Long) or away (e.g. Chris Beardsley). But why not? This presumably isn’t their choice, after all!
Similarly, only once the full season has panned out do we get a broad impression of how good each team in a league is. If it turns out that one player featured much more against the sides in the top half of the table, whilst another did so against lesser opponents, shouldn’t they be compensated and penalised accordingly?
It crudely disregards narrative
You might be forgiven for thinking that for someone banging on about statistics that narrative is the evil arch-nemesis, but harking back again to our concerns with sample size, we have to question the difference between what’s real and what’s noise.
Consider a game only a week ago. It’s the 56th minute at Fratton Park and Boro’ are contesting a tightly fought game, poised at 1-1. Chris Whelpdale receives the ball in the opposition’s half and makes towards goal, unleashing an apparently unstoppable drive from 25 yards. It cruelly crashes against the right hand post and darts across the face of the goal. Not more than a minute and a half later, Pompey break at the other end and score. They lead 2-1 where they were only inches from being 2-1 down. Now let’s reconsider this scenario, but instead during injury time, where there is no opportunity for Boro’ to recover from that setback. The nine players playing the full 90 minutes are credited with 0 points, yet they were just an inch away from getting all three. Yes, these things are evident throughout the season, and ought to balance out with enough games but in this example we can see just how volatile the process is, and how easily things can swing. There simply aren’t enough games in the season for these things to reliably balance out.
As it played out Boro’ lost 3-2 in Portsmouth and most players are credited with 0 points for their contribution (except for Parrett who went off at 2-2 and Clarke who came on at 3-2; they both get 1 point for their time on the field). Compare with only two games prior, at the other end of the country, when Boro’ went down heavily 3-0 at Carlisle. Every one of the 14 players used that day was on the pitch whilst at least one goal was conceded, and each is also attributed with the loss. But these defeats have been treated equally when, by all accounts, the performances deserved different outcomes. If the purpose of measuring these events is to depict with accuracy, then why are we being so crude in treating them equally?
It lets you down when games are abnormal
Here we boil the situation down to some real intricacies. Since entering the Football League, Boro’ players have been sent off on 23 occasions (let’s quickly call that six per season) in league games. Six of those 23 were in the first half of a game. How are we to fairly attribute a result here? Let’s say the score is 0-0 at the point of dismissal – if the team goes on to lose, is it fair to attribute that loss to the other 10 players left to fight an uphill battle? Shouldn’t the dismissed player take an appropriate penalty for the position they’ve put the rest of the team in? Should they overcome the odds and win, isn’t that deserving of even more credit? Or should we disregard the entire result because it isn’t comparable with the majority of the games in the season?
If the boot is on the other foot and an opponent is sent off, what should we do to account for this when Boro’ go on to win? Is a win over 10 men equal to one over a full strength opponent?
Clearly we should be doing something to adjust for these scenarios, but what? We could analyse seasons and seasons of data to work out how often teams win, draw or lose against opponents with more or fewer players, but we’d need to do that for each possible permutation of scoreline and time during the game the red card was shown, and there is definitely not sufficient available data to begin to perform these calculations. We can setup a crude method ourselves, but at this point it becomes more than a little subjective.
It doesn’t account for how much a player contributes
This is definitely a grey area, but not an insignificant one, and I’d be fascinated to hear opinions on it. There are 11 players on the field at any time, but in any system there have to be imbalances – they don’t all contribute equally. Better players find themselves on the ball more often, and are often able to contribute more to the game. Meanwhile others are able to contribute more as a result of their position – a full back in a 4-4-2 might often (and I have definitely not researched this!) have a lot less to do than a centre midfielder or a winger. If the team wins, should they get equal credit? If so, then what do we do with players who change position during a game?
Putting all of this together serves to undermine the reliability of Points Per Game as a mechanism for ranking and rating players. The more flaws we identify, the less credence we can put on it, and the fewer inferences and decisions we should make from having studied it. That’s not to say that it is entirely redundant, but the holes are huge and gaping, and it should be used very sparingly indeed, and a healthy slab of caution applied whenever it is considered.
Why is this so important to flag up now? Because Graham Westley evidently puts a lot of faith in it…
I lost my two full backs — Scott Laird and Keith Keane — and they were my biggest points-per-game players. I think those stats are really important.
Every manager has his dashboard of indices that he looks at and points per game is the indicator I use most.
…and it seems thoroughly misguided.