I mentioned in my last post on calculating adjusted plus/minus that the next thing I wanted to do was split it into offensive and defensive adjusted plus/minus. Lior and Cherokee_ACB had some good suggestions about how to do that in the comments, but first I wanted to see if I could replicate Dan Rosenbaum’s original methodology. This was a little tricky because Dan didn’t spell out his process in detail, but after some trial and error I think I’ve been able to duplicate what he did. As a result, I’m able to calculate 2007-08 player rankings for offensive and defensive adjusted plus/minus, metrics that have not been available publicly since Rosenbaum last presented them in 2005.
Below are all the posts from the "Advanced Stats" category. Click here to view all posts.
June 3, 2008
June 1, 2008
Adjusted plus/minus is a way of rating players first developed by Wayne Winston and Jeff Sagarin in the form of their WINVAL system (more here). The basic idea is simple. For each player, it starts with the team’s average point differential for each possession when they are on the court (sometimes referred to as the player’s on-court plus/minus). This gives a number showing how effective the player’s team was when they were in the game. The problem with using this to evaluate individual players is that it is biased in favor of players who play alongside great teammates (and players who play against weak opponents). This can be seen by looking at the 2007-08 leaders in on-court plus/minus, which can be seen here (the Overall Rtg, On column) or here (the On Court/Off Court, On column). Kendrick Perkins rode his teammates’ coattails to the second highest mark in the league. Adjusted plus/minus uses regression analysis to control for these biases by controlling for the quality of the teammates a player played with and the opponents he played against.
That sounds easy enough, but it’s actually kind of complicated, and the specifics of WINVAL were never made public (Mark Cuban reportedly was paying a handsome sum to use the system for the Mavs). Thankfully, in 2004 Dan Rosenbaum spelled out the details of the methodology in an article. He called his version adjusted plus/minus, and released a series of analyses using the metric (here and here). Eventually Dan was hired to consult for the Cleveland Cavaliers, but because he had spelled out the methodology others were able to duplicate his work for future seasons. David Lewin published rankings for the 2004-05 and 2005-06 seasons, and Steve Ilardi and Aaron Barzilai have done the same for the 2006-07 and 2007-08 seasons (up-to-date ratings can be found here).
I always wanted to try to calculate adjusted plus/minus on my own, but I was intimidated. I figured that I didn’t know enough about running regressions and that I didn’t have the data, software, or computing power to run such a large analysis. But I finally sat down and tried to do it a few days ago, and I discovered that it’s not that difficult. Using Dan Rosenbaum’s description of his method, publicly available data from BasketballValue, Excel 2007, and the free statistics program R, I was able to set up and run the whole thing in less than an hour. Here’s how I did it.
May 28, 2008
I’m still working on that follow-up post on regression to the mean, but in the meantime I wanted to put up a post comparing various player rating systems. For the most part this will be a subjective rather than objective evaluation of the metrics, along the lines of Dean Oliver’s “laugh test” (as in, “a rating system that thinks Dennis Rodman was better than Michael Jordan doesn’t pass the laugh test”). I think looking at how players are rated differently in various systems can tell us a lot about both those players and those rating systems.
The Player Ratings
I took a look at seven popular player ratings. Two basic linear weights metrics based on boxscore stats - John Hollinger’s Player Efficiency Rating (PER), and Dave Berri’s Wins Produced (WP). Two metrics built on Dean Oliver’s individual offensive and defensive ratings - Justin Kubatko’s Win Shares (WS), and Davis21wylie’s Wins Above Replacement Player (WARP). And three plus/minus metrics based on team point differential while the player is on the court - Roland Beech’s Net Plus/Minus (Net +/-), Dan Rosenbaum’s Adjusted Plus/Minus (Adj +/-), and Dan Rosenbaum’s Statistical Plus/Minus (Stat +/-). For the purposes of comparison I looked at the per-minute (or per-possession) versions of all these metrics (e.g. WP48 instead of WP, WSAA/48 instead of WSAA, WARPr instead of WARP).
May 19, 2008
In my next few posts I’m going to cover the topic of regression to the mean, and how it applies to basketball statistics. This is a complex issue and this first post is pretty heavy on the math, but I plan on following it up with more practical examples showing how you can do the calculations in Excel and looking at what the results tell us about different areas of the game.
Most of the equations in this post are not my original work but instead were taken from various sources. I’ve tried to compile them all into one place and in a fairly logical order that can benefit both a newcomer to the topic as well as those with more advanced knowledge looking for a refresher. The main sources are various posts and comments by Tangotiger, MGL, and others on The Book blog, Andy Dolphin’s appendix to The Book, and the Social Research Methods site. Throughout this post I will link to several specific pages that are of relevance. I would also recommend two excellent introductions to regression to the mean by Ed Küpfer and Sal Baxamusa.
True Score Theory
Regression to the mean is rooted in true score theory (aka classical test theory). The basic idea is that a player’s observed performance over some period of time (as measured by a statistic like field-goal percentage) is a function of  the player’s true ability or talent in that area and  a random error component. It should not be forgotten that this is a simplified model, and it leaves a lot of stuff out (team context, for one).
Observed measure = true ability + random error
A player’s true ability can never be known, it can only be estimated. A player’s observed rate is the typical estimate that is used (i.e. we assume a player with a 40% three-point percentage is a “40% three-point shooter”), but by using regression to the mean we can get a better estimate. This is done by combining what we know about how the individual fares in a particular metric with what we know about how players generally fare in that metric.
The first step is to convert the true score model from the individual level to the group level by looking at the spread (or variance) of the distribution of many players’ stats:
var(obs) = var(true + rand) ...but since the errors are by definition random, they aren't correlated with true ability, so... var(obs) = var(true) + var(rand)
If you look at the field-goal percentages of a group of players, some of the variation would be from the differing shooting abilities among the players, and some would come from the differing amounts of random luck each player had. As the equation shows, the overall variance (the standard deviation squared) of players’ observed rates is equal to the sum of the variance of their true rates and the variance of the random errors.
April 25, 2008
Following up on my previous post, I thought it might be interesting to look at the strength of the opposing lineups that individual players faced, rather than looking at five-man units as a whole. The basic idea is the same. I used lineup data from BasketballValue, and for each player I calculated a weighted average of the season defensive ratings of the opposing lineups that they faced, weighted by the number of possessions they played against each opposing lineup. In the tables below I excluded players who were on the court for less than 1000 offensive possessions.
Players who faced the weakest defenses:
Player Team(s) Poss oppDRtg ------------------- ------- ---- ------- Linas Kleiza DEN 3943 110.8 Sasha Vujacic LAL 2482 110.5 Vladimir Radmanovic LAL 2981 110.3 J.R. Smith DEN 2994 109.9 Kelenna Azubuike GSW 3574 109.8 Carl Landry HOU 1341 109.7 Andris Biedrins GSW 4278 109.6 Carlos Boozer UTA 5584 109.5 Stephen Jackson GSW 5914 109.5 Al Harrington GSW 4528 109.5 Carlos Arroyo ORL 2425 109.5 Jordan Farmar LAL 3330 109.4 Baron Davis GSW 6635 109.4 Dikembe Mutombo HOU 1152 109.3 Andrei Kirilenko UTA 4386 109.3 Steve Nash PHX 5641 109.3 Shelden Williams ATL/SAC 1508 109.3 Maurice Evans LAL/ORL 3329 109.2 Deron Williams UTA 6002 109.0 Hedo Turkoglu ORL 5910 109.0
That’s a pretty interesting list. There are a lot of players from great offensive teams. Maybe this is saying that those offenses weren’t so much great as they were lucky - they had the good fortune of facing weaker defensive lineups than other teams faced. But I don’t think this conclusion is warranted. I can think of a few other theories to explain some of the entries on this list.
April 24, 2008
Recently, 82games put up some pages listing the top five-man lineups from this past regular season in terms of plus/minus and points scored and allowed per possession. You can find similar rankings on BasketballValue. I wanted to go a step further and adjust each lineup’s ranking based on the quality of the opposing lineups that it faced during the season.
To do this I started with lineup data from BasketballValue. To adjust each lineup’s offensive rating, I calculated a weighted average of the season defensive ratings of all the opposing lineups that that lineup faced. These defensive ratings were weighted by the number of possessions the original lineup played against that defensive lineup. This meant that for each lineup I had its offensive rating and its average opponents’ defensive rating. I subtracted the second from the first to get an adjusted measure of the lineup’s offensive production. So if a lineup had a good offensive rating but played against poor defensive lineups, its rating was decreased, while if a lineup had a poor offensive rating but played against good defensive lineups, its rating was increased.
The adjustments I made were only one level deep. In college football ranking systems you sometimes see similar multi-level adjustments for strength of schedule that take into account a team’s record, its opponents’ records, and its opponents’ opponents’ records. The same thing could be done here - I’m adjusting each team’s offensive ratings for their opponents’ defensive ratings, but I could first adjust the opponents’ defensive ratings for their opponents’ offensive ratings. Theoretically, one could do this infinitely, and I think the results would ultimately be similar to what you’d get from a regression-based method like Dan Rosenbaum uses for his adjusted plus/minus. But I’m just going to do one level of adjusting, partly because it can be calculated pretty quickly with some pivot tables in Excel, and partly because you just don’t gain that much the deeper you go. This is because over the course of a season, things tend to even out, and most lineups end up facing a similar mix of good and bad opposing lineups. The variance in opponents’ defensive ratings is a lot less than the variance in lineup offensive ratings, and the variance in opponents’ opponents’ offensive ratings would be even smaller.
Below are the adjusted rankings for offensive rating, defensive rating, and point differential. I excluded lineups that played together for less than 200 offensive possessions (or 200 defensive possessions). “ORtg” is the lineup’s offensive rating (points per 100 possessions), “oppDRtg” is the weighted average of the defensive ratings of the opposing lineups faced. “offDiff” is the additional points scored per 100 possessions over what would be expected based on the quality of the defenses faced. “DRtg”, “oppORtg”, and “defDiff” are the defensive counterparts to those stats. “totDiff” is the sum of “offDiff” and “defDiff”, which represents the additional point differential per 100 possessions over what would be expected based on the quality of the offenses and defenses faced.
Best and Worst Offensive Lineups:
February 5, 2008
There has been a lot of discussion in recent months about the importance of rebounding on the player level. Much of this debate has been in reaction to the high value that Dave Berri’s Wins Produced player rating puts on rebounds. On Berri’s blog there have several posts with long, insightful debates in the comments about the issue (that is, if you ignore the unfortunate mudslinging often directed at those with differing points of view). In particular, I would recommend the comments sections of “The Best One-Two Punch in the Association”, “Chris Paul vs. Deron Williams, Again”, and “How Has Texas Survived the Loss of Kevin Durant?”. There have also been some good debates on the topic in the APBRmetrics threads, “Current season Win Scores/Wins Produced” and “Can some one explain the ‘possession cost’ scheme?”.
These are wide-ranging debates, involving such issues as the relative value of rebounding versus scoring and the apportioning of credit for a defensive stop between the defensive rebounder and his teammates. The issue that I want to pick up on is the extent to which the law of diminishing returns applies to rebounding.
December 17, 2007
It’s taking longer than I anticipated to compile and analyze the context-dependency of various player stats by the method I outlined in my last post, so in the meantime I would like to shift gears and introduce a method that uses team stats to try to understand whether the offensive or defensive team controls various aspects of the game.
There’s an old saying in baseball that “good pitching always beats good hitting.” I want to examine what a claim like this is trying to get at, look at a method that attempts to objectively analyze whether it’s true, and then apply that method to many areas of basketball and see what we can learn.
December 4, 2007
There has been a lot of debate recently about comprehensive player ratings such as John Hollinger’s PER, Dave Berri’s Wins Produced, and Dan Rosenbaum’s Adjusted Plus/Minus. Is one of these rating systems better than the others? What methods can be used to make such an assessment? One approach is to analyze and critique the theory behind each measure - does the way it was constructed make basketball (and statistical) sense? An alternative approach is to analyze them empirically - what happens when we actually start applying the ratings to players? Dean Oliver, the author of Basketball on Paper, has suggested two such empirical methods by which to evaluate player ratings: