I’ve got good news and bad news. The good news is that I’ve accepted an offer to work for an NBA team doing statistical analysis along the lines of the work I’ve posted on this site. The bad news is that the future research I do will be just for that team and won’t be something that I can publish and share with everyone (even Amir Johnson himself!).
So for the foreseeable future, I won’t be posting any of my original basketball research on this blog. My old posts will still be here, the links page will still be here (I’ll try to keep it updated), and I still may make occasional blog posts on other topics (perhaps just linking to other people’s work).
Thanks to everyone who has posted comments or sent me emails regarding past posts. And going forward if anyone has any thoughts or questions about old posts feel free to comment or email and I’ll try to respond.
I mentioned in my last post on calculating adjusted plus/minus that the next thing I wanted to do was split it into offensive and defensive adjusted plus/minus. Lior and Cherokee_ACB had some good suggestions about how to do that in the comments, but first I wanted to see if I could replicate Dan Rosenbaum’s original methodology. This was a little tricky because Dan didn’t spell out his process in detail, but after some trial and error I think I’ve been able to duplicate what he did. As a result, I’m able to calculate 2007-08 player rankings for offensive and defensive adjusted plus/minus, metrics that have not been available publicly since Rosenbaum last presented them in 2005.
Adjusted plus/minus is a way of rating players first developed by Wayne Winston and Jeff Sagarin in the form of their WINVAL system (more here). The basic idea is simple. For each player, it starts with the team’s average point differential for each possession when they are on the court (sometimes referred to as the player’s on-court plus/minus). This gives a number showing how effective the player’s team was when they were in the game. The problem with using this to evaluate individual players is that it is biased in favor of players who play alongside great teammates (and players who play against weak opponents). This can be seen by looking at the 2007-08 leaders in on-court plus/minus, which can be seen here (the Overall Rtg, On column) or here (the On Court/Off Court, On column). Kendrick Perkins rode his teammates’ coattails to the second highest mark in the league. Adjusted plus/minus uses regression analysis to control for these biases by controlling for the quality of the teammates a player played with and the opponents he played against.
That sounds easy enough, but it’s actually kind of complicated, and the specifics of WINVAL were never made public (Mark Cuban reportedly was paying a handsome sum to use the system for the Mavs). Thankfully, in 2004 Dan Rosenbaum spelled out the details of the methodology in an article. He called his version adjusted plus/minus, and released a series of analyses using the metric (here and here). Eventually Dan was hired to consult for the Cleveland Cavaliers, but because he had spelled out the methodology others were able to duplicate his work for future seasons. David Lewin published rankings for the 2004-05 and 2005-06 seasons, and Steve Ilardi and Aaron Barzilai have done the same for the 2006-07 and 2007-08 seasons (up-to-date ratings can be found here).
I always wanted to try to calculate adjusted plus/minus on my own, but I was intimidated. I figured that I didn’t know enough about running regressions and that I didn’t have the data, software, or computing power to run such a large analysis. But I finally sat down and tried to do it a few days ago, and I discovered that it’s not that difficult. Using Dan Rosenbaum’s description of his method, publicly available data from BasketballValue, Excel 2007, and the free statistics program R, I was able to set up and run the whole thing in less than an hour. Here’s how I did it.
I’m still working on that follow-up post on regression to the mean, but in the meantime I wanted to put up a post comparing various player rating systems. For the most part this will be a subjective rather than objective evaluation of the metrics, along the lines of Dean Oliver’s “laugh test” (as in, “a rating system that thinks Dennis Rodman was better than Michael Jordan doesn’t pass the laugh test”). I think looking at how players are rated differently in various systems can tell us a lot about both those players and those rating systems.
The Player Ratings
I took a look at seven popular player ratings. Two basic linear weights metrics based on boxscore stats - John Hollinger’s Player Efficiency Rating (PER), and Dave Berri’s Wins Produced (WP). Two metrics built on Dean Oliver’s individual offensive and defensive ratings - Justin Kubatko’s Win Shares (WS), and Davis21wylie’s Wins Above Replacement Player (WARP). And three plus/minus metrics based on team point differential while the player is on the court - Roland Beech’s Net Plus/Minus (Net +/-), Dan Rosenbaum’s Adjusted Plus/Minus (Adj +/-), and Dan Rosenbaum’s Statistical Plus/Minus (Stat +/-). For the purposes of comparison I looked at the per-minute (or per-possession) versions of all these metrics (e.g. WP48 instead of WP, WSAA/48 instead of WSAA, WARPr instead of WARP).
Jason from BallHype has a new post on Chris Paul with some really nice dynamic charts thanks to the Google Chart API.
In my next few posts I’m going to cover the topic of regression to the mean, and how it applies to basketball statistics. This is a complex issue and this first post is pretty heavy on the math, but I plan on following it up with more practical examples showing how you can do the calculations in Excel and looking at what the results tell us about different areas of the game.
Most of the equations in this post are not my original work but instead were taken from various sources. I’ve tried to compile them all into one place and in a fairly logical order that can benefit both a newcomer to the topic as well as those with more advanced knowledge looking for a refresher. The main sources are various posts and comments by Tangotiger, MGL, and others on The Book blog, Andy Dolphin’s appendix to The Book, and the Social Research Methods site. Throughout this post I will link to several specific pages that are of relevance. I would also recommend two excellent introductions to regression to the mean by Ed Küpfer and Sal Baxamusa.
True Score Theory
Regression to the mean is rooted in true score theory (aka classical test theory). The basic idea is that a player’s observed performance over some period of time (as measured by a statistic like field-goal percentage) is a function of  the player’s true ability or talent in that area and  a random error component. It should not be forgotten that this is a simplified model, and it leaves a lot of stuff out (team context, for one).
Observed measure = true ability + random error
A player’s true ability can never be known, it can only be estimated. A player’s observed rate is the typical estimate that is used (i.e. we assume a player with a 40% three-point percentage is a “40% three-point shooter”), but by using regression to the mean we can get a better estimate. This is done by combining what we know about how the individual fares in a particular metric with what we know about how players generally fare in that metric.
The first step is to convert the true score model from the individual level to the group level by looking at the spread (or variance) of the distribution of many players’ stats:
var(obs) = var(true + rand)
...but since the errors are by definition random, they aren't correlated with true ability, so...
var(obs) = var(true) + var(rand)
If you look at the field-goal percentages of a group of players, some of the variation would be from the differing shooting abilities among the players, and some would come from the differing amounts of random luck each player had. As the equation shows, the overall variance (the standard deviation squared) of players’ observed rates is equal to the sum of the variance of their true rates and the variance of the random errors.
Following up on my previous post, I thought it might be interesting to look at the strength of the opposing lineups that individual players faced, rather than looking at five-man units as a whole. The basic idea is the same. I used lineup data from BasketballValue, and for each player I calculated a weighted average of the season defensive ratings of the opposing lineups that they faced, weighted by the number of possessions they played against each opposing lineup. In the tables below I excluded players who were on the court for less than 1000 offensive possessions.
Players who faced the weakest defenses:
Player Team(s) Poss oppDRtg
------------------- ------- ---- -------
Linas Kleiza DEN 3943 110.8
Sasha Vujacic LAL 2482 110.5
Vladimir Radmanovic LAL 2981 110.3
J.R. Smith DEN 2994 109.9
Kelenna Azubuike GSW 3574 109.8
Carl Landry HOU 1341 109.7
Andris Biedrins GSW 4278 109.6
Carlos Boozer UTA 5584 109.5
Stephen Jackson GSW 5914 109.5
Al Harrington GSW 4528 109.5
Carlos Arroyo ORL 2425 109.5
Jordan Farmar LAL 3330 109.4
Baron Davis GSW 6635 109.4
Dikembe Mutombo HOU 1152 109.3
Andrei Kirilenko UTA 4386 109.3
Steve Nash PHX 5641 109.3
Shelden Williams ATL/SAC 1508 109.3
Maurice Evans LAL/ORL 3329 109.2
Deron Williams UTA 6002 109.0
Hedo Turkoglu ORL 5910 109.0
That’s a pretty interesting list. There are a lot of players from great offensive teams. Maybe this is saying that those offenses weren’t so much great as they were lucky - they had the good fortune of facing weaker defensive lineups than other teams faced. But I don’t think this conclusion is warranted. I can think of a few other theories to explain some of the entries on this list.
Recently, 82games put up some pages listing the top five-man lineups from this past regular season in terms of plus/minus and points scored and allowed per possession. You can find similar rankings on BasketballValue. I wanted to go a step further and adjust each lineup’s ranking based on the quality of the opposing lineups that it faced during the season.
To do this I started with lineup data from BasketballValue. To adjust each lineup’s offensive rating, I calculated a weighted average of the season defensive ratings of all the opposing lineups that that lineup faced. These defensive ratings were weighted by the number of possessions the original lineup played against that defensive lineup. This meant that for each lineup I had its offensive rating and its average opponents’ defensive rating. I subtracted the second from the first to get an adjusted measure of the lineup’s offensive production. So if a lineup had a good offensive rating but played against poor defensive lineups, its rating was decreased, while if a lineup had a poor offensive rating but played against good defensive lineups, its rating was increased.
The adjustments I made were only one level deep. In college football ranking systems you sometimes see similar multi-level adjustments for strength of schedule that take into account a team’s record, its opponents’ records, and its opponents’ opponents’ records. The same thing could be done here - I’m adjusting each team’s offensive ratings for their opponents’ defensive ratings, but I could first adjust the opponents’ defensive ratings for their opponents’ offensive ratings. Theoretically, one could do this infinitely, and I think the results would ultimately be similar to what you’d get from a regression-based method like Dan Rosenbaum uses for his adjusted plus/minus. But I’m just going to do one level of adjusting, partly because it can be calculated pretty quickly with some pivot tables in Excel, and partly because you just don’t gain that much the deeper you go. This is because over the course of a season, things tend to even out, and most lineups end up facing a similar mix of good and bad opposing lineups. The variance in opponents’ defensive ratings is a lot less than the variance in lineup offensive ratings, and the variance in opponents’ opponents’ offensive ratings would be even smaller.
Below are the adjusted rankings for offensive rating, defensive rating, and point differential. I excluded lineups that played together for less than 200 offensive possessions (or 200 defensive possessions). “ORtg” is the lineup’s offensive rating (points per 100 possessions), “oppDRtg” is the weighted average of the defensive ratings of the opposing lineups faced. “offDiff” is the additional points scored per 100 possessions over what would be expected based on the quality of the defenses faced. “DRtg”, “oppORtg”, and “defDiff” are the defensive counterparts to those stats. “totDiff” is the sum of “offDiff” and “defDiff”, which represents the additional point differential per 100 possessions over what would be expected based on the quality of the offenses and defenses faced.
Best and Worst Offensive Lineups:
I’m working on a longer study, but for now here are a few links.
Last week, the New York Times had an article on using Chernoff faces to visualize data about baseball managers. The Arbitrarian followed that up with a post that used Chernoff faces to compare some star players in the NBA. Chernoff faces are a way to display data by mapping it onto simplified human faces - you can read more about them here and here. The main reason I’m linking to these is because this method of visualizing data was invented by my great-uncle, Herman Chernoff. He’s made a lot of contributions to the field of statistics in his career, but most of them (like this) aren’t as fun as the faces that he came up with thirty-five years ago.
Here’s a long Q and A with Bill James that’s well worth reading. This response in particular caught my eye:
Q: Generally, who should have a larger role in evaluating college and minor league players: scouts or stat guys?
A: Ninety-five percent scouts, five percent stats. The thing is that — with the exception of a very few players like Ryan Braun — college players are so far away from the major leagues that even the best of them will have to improve tremendously in order to survive as major league players — thus, the knowledge of who will improve is vastly more important than the knowledge of who is good. Stats can tell you who is good, but they’re almost 100 percent useless when it comes to who will improve.
In addition to that, college baseball is substantially different from pro baseball, because of the non-wooden bats and because of the scheduling of games. So … you have to pretty much let the scouts do that.
These issues seem to me to be important in basketball as well, and I think they are a good starting point for thinking about the statistical analysis of sports. Taking them in reverse order, here’s one way of framing James’ points:
Here’s a chart of offensive rebounding percentage by shot location, as suggested by Eric in the comments to my last post. The chart doesn’t show where players grab rebounds on the court (I don’t have that data), instead it shows which shot locations result in more offensive rebounds.
This took a little more work to construct because rebounds are listed in separate lines of the play-by-play than shot attempts (and not always on the very next line), and because the league credits a rebound for all missed shots, even half-court heaves at the end of a quarter where the clock expires while the ball is in the air. I tried to eliminate as many of these bookkeeping rebounds from the data as I could (though the red squares on the longest threes suggest that some still remain, as the offensive team is typically credited with the bookkeeping rebound).
I smoothed out the data as I did on the FG% chart, so each 1×1 ft square actually represents the offensive rebounding percentage on shots taken from the surrounding three-foot by three-foot square on the court. The color scale runs from blue (low ORB%) to yellow (average ORB%) to red (high ORB%). The data is from the 03-04 to 06-07 seasons.
Which Shots are Offensive Rebounded - ORB% by Shot Location:
The conventional wisdom on this topic that one usually hears from broadcasters is that it’s easier to get offensive rebounds on three-pointers because of the long caroms. However, Dean Oliver, John Hollinger and others have done research suggesting that there is actually a higher offensive rebounding percentage on twos than threes (see this thread and this study).
This chart suggests that both sides are onto something - longer three-point jumpers are rebounded by the offense more often than shorter two-point jumpers, but shots in the paint are rebounded by the offense even more frequently than three-pointers. Interestingly, it also looks like jumpers from the right baseline are offensive rebounded more frequently than jumpers from the left baseline, but I’d have to take a closer look at the data to confirm that.
Here are a few more shot charts along the lines of those from my previous post.
First, a chart of effective field-goal percentage by shot location, which some commenters requested. eFG% is basically a measure of points per shot (excluding points from free throws) that is different from FG% in that it gives extra credit for made threes. The formula is 0.5*(2*2PM + 3*3PM)/(2PA + 3PA). I didn’t smooth out the data like I did in the FG% chart, so each one-foot by one-foot square contains the eFG% on shots from that spot. The color scale runs from blue (low eFG%) to yellow (average eFG%) to red (high eFG%).
Where Players Score Points - eFG% by Location:
As expected, three point shots get a big boost by looking at eFG% rather than just FG%. Shots right around the basket (dunks and layups mainly) also rate well, as do elbow jumpers, anything down the center line of the court, and small areas on the left and right blocks. The red squares just inside the three-point line are artifacts of three-point makes that were mistakenly coded as being taken from inside the three-point line.
The other new chart shows in which locations shots are most likely to get blocked. Each one-foot by one-foot square represents the percent of field goal attempts from that spot that were blocked. The color scale runs from blue (few blocks/FGA) to yellow (some blocks/FGA) to red (many blocks/FGA). The results are basically as expected - most blocks occur close to the hoop.
Where Shots Get Blocked - Blocks/FGA by Location:
If anyone has any other suggestions let me know.
I haven’t posted in a while, but I have a good excuse. I’ve been assembling a database of play-by-plays from the last five seasons. This will make it much easier to investigate many areas of the game. It’s not completely finished yet, but already there’s a lot of interesting stuff that I can pull from the database. For example, I can quickly calculate that in the four seasons from 2003-04 to 2006-07 (I’m missing a few games, but I have over 99% of games from that time period), there were 14,216 hook shots taken. 45.5% of those shots were made, 54% of those makes were assisted, and 4% of all hook shots were blocked. But that’s not what I really want to examine in this post. Instead I’m going to look at shot location data.
During all NBA games, diligent game charters sit courtside and mark the location of every shot taken on the court down to the square foot. I believe this is done by using a stylus on a touchscreen monitor with a replica of the court on it. This is the source of the shot charts you can see after a game on ESPN.com or CBS Sportsline, and of the compiled shot charts of NBA.com’s Hot Spots (which I discussed in my first post). With the play-by-play database I’ve constructed it’s possible to look at this shot location data in a number of ways.
First, below is a picture showing where players shoot from. Again it’s based on data from 03-04 to 06-07. The court dimensions are to scale, and each one-foot by one-foot square on the court is color-coded based on the number of field goal attempts from that spot. The color scale runs from blue (few FGA) to yellow (some FGA) to red (many FGA). Obviously the charted locations aren’t 100% accurate, but there are still some very interesting patterns that emerge.
Where Players Take Shots - FGA by Location:
In the wake of my last few posts on diminishing returns in rebounding, a lot of people have suggested looking at how diminishing returns applies to scoring. This is a more complex issue, but I think some of the same methods can be used to try to understand what’s going on in this part of the game of basketball. For rebounding, we were just looking at the relationship of player rebounding to team rebounding. For scoring, we have to look at the relationship of player efficiency and player usage to team efficiency. Diminishing returns for scoring is really just another way of framing the usage vs. efficiency debate which has been going on in the stats community for years. Does efficiency decrease as usage increases? By how much? What, if any, value should be placed on shot creation? Are coaches using anything near to the optimal strategies in distributing shot attempts among their players? Is Allen Iverson wildly overrated? Was Fred Hoiberg criminally underutilized? The big names in basketball stats like Dean Oliver, Bob Chaikin, John Hollinger, Dan Rosenbaum, and Dave Berri have all staked out positions in this debate. For some background, see here and here and here and here and here and so on and so on. A lot of words have been written on this topic.
The major difficulty in studying the usage vs. efficiency tradeoff is the chicken-and-egg problem - does a positive correlation between usage and efficiency mean that players’ efficiencies aren’t hurt as they attempt to up their usage, or just that in seasons/games/matchups where players are more efficient (for whatever reason) they use more possessions? For instance, if a player is facing a poor defender (which will increase his efficiency) he (or his coach) might increase his usage. But it could be that this positive correlation is drowning out the presence of a real diminishing returns effect. If players go from low-usage, low-efficiency against a good defenders to high-usage, high-efficiency against poor defenders, it still could be the case that if they tried to increase their usage against average defenders their efficiency would decrease. Defender strength is just one of the factors that can cloud things - another confound comes from game-to-game or season-to-season variation in a player’s abilities (e.g. a player being “hot” or having an “off game”, a player being injured or tired, or a player using more possessions as his skills improve from year to year).
By using the method from my last study on diminishing returns for rebounding, it’s possible to largely avoid this chicken-and-egg problem. This method looks at situations in which some or all of the players on the court were forced to increase their usage (relative to their average usage on the season). And on the other side, it looks at lineups in which some or all of the players on the court were forced to decrease their typical usage. By looking at these forced cases the method minimizes the confounds from players increasing or decreasing their usage by choice in favorable situations.
Following up on my last post, I’m going to look at the issue of diminishing returns for rebounding from a different angle. The new method I’m going to use has several advantages over the previous one (and some disadvantages). What I like best about it is that it does a great job of presenting the effect of diminishing returns visually, rather than just through a table of numbers.
The approach I will use was first suggested to me by Ben F. from the APBRmetrics forum. But before I got a chance to try it out, another poster, Cherokee_ACB, presented results of his own using a similar method. So this post can be seen as building on the ideas of both of these posters.
Instead of comparing individual players’ rebounding percentages to the rebounding percentages of the lineups they played in, this method takes into account the rebounding of all five players on the court for a team. Instead of just speculating about how well a team would rebound if it put five strong rebounders on the court together (or five poor rebounders), it looks at what has actually happened in such situations in the past.