NOTE (1/29/24): this blog needs to be validated, as there are potential biases introduced into the data, due to how I hand selected the stats for the metrics and then used the same data for building the model. Additionally, some data pollution may have occured, during model building.
Additionally, the wide receiver modeling blog has been impacted by the same potential bias.
I will be revisiting this subject once more, after my compute has been upgraded. Expect updates within the next month. Amendment blogs will be linked here.
Hi everyone! Welcome back to another fantasy analysis post!
In this episode, we will be building a model that predicts the fantasy performance of running backs. This work will utilize the information discovered in the previous blog post, A View at Running Back Metrics, which essentially showed that fantasy relevant backs are predictable, if receiving work is taken into account. Moreover, it showed that receiving stats for backs had strong correlations with fantasy performance in the next season.
While the inclusion of receiving work may seem obvious to some, I would like to point out that using only rushing stats to find metrics that serve as solid indicators of fantasy performance, is rather futile. In fact, it's far more likely to be accurate with predictions by just using receiving work. My first glance into how sticky rushing stats are displayed how poor rushing stats were as indicators of fantasy performance, unfortunately.
So, given the discovery of the strong correlations from the previous work, the goal for the research here is to accomplish the following: first find an 'optimal' metric and second use it to create a useful predictive model.
Before moving onto the exciting stuff, I must mention that the data used for this analysis is from 2012-2022.
Note: for stat and term definitions in this section that you are unfamiliar with, please check out the glossary below.
First, we need to choose some stat(s) to create a metric that would be useful for predictions. In our previous work, we explored two potential metrics. The first was built on wopr_y, tgt_sh, and receptions:
The second included these three stats and carries:
For more info, including how to interact with the graphs, the Fantasy Relevance section from that work should be referenced. In this version of running back metric research, I attempted to find a more optimal metric.
Before moving on, I want to go into how I went about this. Continuing off the idea of sticky stats, I created many metrics based off stats from the previous season (or season X - 1) and used the metric to build a model that would inference fantasy points - ppr flavored - in the next season (or season X). The assumption here is that strong correlations should provide us models that are capable at predicting fantasy performance in the future, or at least the direction of change (i.e. improved or declined season upcoming). This may not take into account potential non-linear patterns of the data, but it again feels like a reasonable direction to at least start with.
One final note: the combinations created were limited by my compute power, so I was not able to calculate all permutations of the stats. However, I was still be able to run correlations for many potential metrics. The results from the last piece seemed to be more promising than real. It was difficult to find anything better than the metric built on wopr_y, tgt_sh, receptions and carries.
The metric we used to build these models was based on the following stats: rushing_fumbles_lost, target_share, tgt_sh, dom, yptmpa, and ppr_sh. It was quite an improvement, and this is what it looks like when graphed with fantasy points ppr against age:
Again, I encourage you to interact with the graph. Specifically, display only new-metric and fantasy_points_ppr_normalized_mean on the graph, by first double-clicking on one of the lines in the legend, and then single-click on the other. Double-click on any selected line to reset the graph.
Now, I want to note that building a model on the individual data points, instead of age buckets, resulted in correlation with fantasy relevant backs to decrease.
This is still a strong correlation, and serves as a much more useful indicator than any of my previous work on backs. Furthermore, on the general running back data, the correlation is actually stronger, at about 0.85.
We are cooking with gas here, even if it might not feel like it, given the previous research preparing this blog post.
I do plan on revisiting this work once my compute situation improves, which should be in about a month or so, with the hope that I'll be able to compute more metric options that can beat the current one.
The moment we've all been waiting for! What kind of damage can a model built on our new-metric do?
Well, if we start with a multiple linear regression (MLR) model, we get the following result:
The green in the graph above indicates that the predicted fantasy production yielded the same delta as what actually occurred. In other words, if the model predicts an increase in fantasy points and the player actually increased their fantasy production, the point is marked as green on the graph.
Similarly, if the model incorrectly predicts an increase in fantasy points, then the point is marked as red.
Unfortunately, our test set of data points is small - only 36 samples. So, it might be hard to strongly believe in the results, but the model has a 69.39% accuracy when it comes to identifying the correct direction of change. I would say this matches the strong, but not spectacular correlation from our new-metric and fantasy points ppr. Furthermore, since the correlation of the new-metric was out of this world, at 0.85, this model, or at least how it was built, should perform even better on all backs.
I also trained a random forest regression (RFR) model with the same methodology as the MLR model above. It yielded the following result:
This performed better than the MLR model, with an accuracy of 77.8% when it comes to predicting the correct direction of change for fantasy production. That's exciting, but again, it is a small test set, so I would take these results with a grain of salt.
The correlation value we calculated and the results of these models do bode well for running back predictability. I think this shows that it isn't quite the desolate wasteland that I had originally thought, and that models can out perform random chance.
In this section we will train these models on all running back data from 2012-2022, to see how they perform on the generalized data. Furthermore, by doing this, the test set will be far larger, with 237 samples. The same metric used for the models above will be used here.
Our generalized MLR model:
This is an improvement on our niche version, as it has an accuracy of 74.7%. I also feel more confident about the results here, as we have both a larger test and train set. Additionally, the correlation value is super strong for the generalized version, at 0.85.
Now, looking at the RFR model:
Turns out the performance of the RFR model is slightly diminished when using the generalized version of the data, falling to an accuracy of 74.7%. Still, not bad.
It is encouraging to see the models perform well on the generalized level too.
Thanks to all for making it this far!
Before wrapping up with the TL;DR, I want to mention that the progress here is quite exciting, even if I was unable to find an optimal metric. The performance of these models beats that of the wide receiver ones with trinity scoring and alternative metrics.
If you were to tell me beforehand that back performance could actually be more accurately predicted than wide receiver performance, I wouldn't have believed you. So, while the work here is not over, I think this piece serves as a strong proof of concept that backs can be accurately predicted.
Additionally, this makes me want to revisit the wide receiver numbers to find something even better for modeling that position group, but more on that in another piece.
On that note, here's the recap for this one:
Thanks for reading!
Cheers,
Alex
Term definitions below:
For stat definitions on the graphs, please check out this previous glossary and these vignettes.