[Cross-posted at Signal/Noise]
One of the biggest problems with standard pre-draft rankings, particularly those of the big fantasy hosting sites (e.g. ESPN, CBS, etc) is that the rankings are based solely on aggregate measures of performance such as total projected points for the upcoming season. Now, of course the goal is to assemble a team with players that end up scoring lots of points throughout the season, but total points scored ignores the fact that teams compete head-to-head, week-to-week. In order to make the playoffs a team has to outscore opponents on a consistent basis in order to accumulate wins, not just points. That means drafting players that not only score a lot of points, but score a lot of points week in and week out. When it comes to deciding between which players to draft, managers would be better off selecting consistent scorers versus boom-or-bust players (at least, that is my hypothesis).
Let’s take a look at two hypothetical players:
Over the course of four weeks both players score the same amount of total points. However, Player A is clearly a boom-or-bust player while Player B is more consistent week-to-week. Player A gives you a great chance to win Weeks 1 and 4, but makes it much hard to win in Weeks 2 and 3. On the other hand, Player B is the model of consistency, giving you a great chance to win each week. On most pre-draft rankings, Players A and B will look like equally valuable picks, but this is misleading.
This year, I decided to see whether a player’s penchant for boom-or-bust performances was at all consistent and predictable. The initial answer seems to be yes.
I developed two metrics; one to evaluate high scoring consistency and one that takes predicted points and combines them with scoring consistency. The first, ConBoom, measures, weights, and then combines the number of times a player scored >=20 points, >=15 points, and <10 points per game over the course of a season. This is the foundation of the consistency metric. ConRank combines the ConBoom score for a player with a weighted measure of that player's predicted total points for the upcoming season. (How am I weighting each component of the measures? Well now I can't reveal the entire secret sauce, now can I?) I validated the measures against the past three years of actual player data and found that ConBoom scores from one year were highly correlated with ConBoom scores the next year (.70). I am going to use the new metric to guide my drafts in all three leagues and essentially test how my teams fair against other teams over the course of the season. With data and predictions for every player I’ll be able to test the method over three league scoring systems, 32 teams, and 256 games as well as validate the measures predictive attributes over another year. Draft number one is tonight. Let the games begin!