Hollinger Argues ATL and POR ISO Offenses in Playoffs = FAIL

[AHF]

The truncation bias isn't because you have a smaller sample size, its because the data you removed isn't representative of your population. Small size is not a bias. If I try to estimate a population with 500 observations as opposed to 1000 observations, but they are both representative of the population then I will get unbiased estimates. We do not say we have a bias because 500<1000, thats a little silly if they are representative. Now if we have 1000 observations that all have a similar characteristic (say teams that win > 40 games) while a different 500 observation is representative of the population we do have a bias. The bias is not with the 500 observation (smaller sample size), it is with the 1000 observations.

That is why I said both sample size and the limited number of teams faced (in different proportions) in the playoffs were issues. Unless I am reading what you are saying the wrong way, the fact that we have only played four teams over 19 games 7 for Miami, 7 for Mil, 4 for CLE, and 1 for ORL, the past two offseasons is a truncation bias problem.

My solution is just for the truncation bias. I am not accounting for other flaws within Hollinger's analysis that you may or may not be seeing. But what I believe you want to see is a diff-n-diff estimation of isolation basketball and the effects it has in the playoffs. If we want to actually estimate this what we need is a listing of teams that play in both the playoffs and regular season (see ya "out of 30" stats which Hollinger uses like the dope he is), and a list of teams that we can attach a dummy variable of 1 = "iso" and 0 = "not-iso". This alone doesn't guarantee that we can estimate the effects, we also need to have teams that switch between playoffs and regular season. Meaning, we need a team to be 0 (not iso) in the regular season and 1 (iso) in the playoffs. Also the other way around. We also need teams that are 0 in the regular season and 0 in the playoffs (same with 1). Is this feasible? No, so you are right to say we cannot test this. Does Hollinger attempt to test this and pass it off as "statistics"? Yes, and thats why he is a dope.

Based on your parameters, we clearly cannot test for whether offenses are more or less successful in the post-season than in the regular season. That just leaves me saying "Hollinger is once again claiming to have proven something with bad math," but "Nevertheless, the ranking statistics are very interesting and it doesn't surprise me in the least that Atlanta has ranked 14th of 16 and 11th of 16 in offensive efficiency the last two years given their predictable iso offense."

Edited May 6, 2010 by AHF

[Guest]

That is why I said both sample size and the limited number of teams faced (in different proportions) in the playoffs were issues. Unless I am reading what you are saying the wrong way, the fact that we have only played four teams over 19 games 7 for Miami, 7 for Mil, 4 for CLE, and 1 for ORL, the past two offseasons is a truncation bias problem.

Truncation here has nothing to do with sample size, its about the biased sampling. So when you talk about sample size that may be a problem with the power of your test but it is not a bias. The truncation bias also does not deal with the issue of disproportionate team play. That is a completely separate issue (which is a bigger problem, but for the sake of argument I ignore it). Disproportionate game play is an issue with the playoffs, but this doesn't result from truncation.

First, Hollinger looks at all 30 teams and throws up some rankings. Then he truncates the data (i.e. he throws out the bottom 14 teams) and looks at the data again. He truncates the data with the playoffs, looks at the rankings and then screams "Look the rankings are different!" Well no s*** they are different, there is a truncation bias when comparing. One could simply look at this in the context of the regular season. You look at Atlanta verse the other 29 teams (and all other teams in this context) and you will see their ranking is 3rd offensively. OK, but their ranking will differ if you only look Atlanta verse the other 15 playoff teams (and all other teams in this context) for the regular season. You do this, and you will see that Atlanta during the regular season against the truncated data will not have the 3rd ranked offensive team (nor the 1.6th ranked). This is because you have truncated the data, so Atlanta verse the other 15 playoff teams will give you differing results than Atlanta verse the league as a whole. This is what we call a truncation bias.

I believe you are thinking that I am referring to a strict difference between playoffs and regular season. This isn't true, I am actually just referring to the technique of truncating data that it will result in a bias. I am ignoring so many other flaws in Hollinger's techniques and just focusing on Hollinger truncating data and then screaming "Look at the difference!" Again, there is a truncation bias in what he is doing. If you don't want this bias, then compare the same 16 playoff teams during the regular season and playoffs in order to correct this one bias that he has. To correct all the other problems (NOT necessarily biases) would take too long.

Based on your parameters, we clearly cannot test for whether offenses are more or less successful in the post-season than in the regular season.

Are you referring to the actual data that we have or my hypothetical regression? In the hypothetical, we will certainly attain a value for how much better/worse iso offenses are in the post-season. Of course, the problem is we don't actually have this data. Or maybe the new Synergy website actually has it, but I haven't had time to check it out.

[AHF]

Truncation here has nothing to do with sample size, its about the biased sampling. So when you talk about sample size that may be a problem with the power of your test but it is not a bias. The truncation bias also does not deal with the issue of disproportionate team play. That is a completely separate issue (which is a bigger problem, but for the sake of argument I ignore it). Disproportionate game play is an issue with the playoffs, but this doesn't result from truncation.

First, Hollinger looks at all 30 teams and throws up some rankings. Then he truncates the data (i.e. he throws out the bottom 14 teams) and looks at the data again. He truncates the data with the playoffs, looks at the rankings and then screams "Look the rankings are different!" Well no s*** they are different, there is a truncation bias when comparing. One could simply look at this in the context of the regular season. You look at Atlanta verse the other 29 teams (and all other teams in this context) and you will see their ranking is 3rd offensively. OK, but their ranking will differ if you only look Atlanta verse the other 15 playoff teams (and all other teams in this context) for the regular season. You do this, and you will see that Atlanta during the regular season against the truncated data will not have the 3rd ranked offensive team (nor the 1.6th ranked). This is because you have truncated the data, so Atlanta verse the other 15 playoff teams will give you differing results than Atlanta verse the league as a whole. This is what we call a truncation bias.

I believe you are thinking that I am referring to a strict difference between playoffs and regular season. This isn't true, I am actually just referring to the technique of truncating data that it will result in a bias. I am ignoring so many other flaws in Hollinger's techniques and just focusing on Hollinger truncating data and then screaming "Look at the difference!" Again, there is a truncation bias in what he is doing. If you don't want this bias, then compare the same 16 playoff teams during the regular season and playoffs in order to correct this one bias that he has. To correct all the other problems (NOT necessarily biases) would take too long.

That makes sense. Thanks for the explanation.

Are you referring to the actual data that we have or my hypothetical regression? In the hypothetical, we will certainly attain a value for how much better/worse iso offenses are in the post-season. Of course, the problem is we don't actually have this data. Or maybe the new Synergy website actually has it, but I haven't had time to check it out.

I am referring to your proposed test:

But what I believe you want to see is a diff-n-diff estimation of isolation basketball and the effects it has in the playoffs. If we want to actually estimate this what we need is a listing of teams that play in both the playoffs and regular season (see ya "out of 30" stats which Hollinger uses like the dope he is), and a list of teams that we can attach a dummy variable of 1 = "iso" and 0 = "not-iso". This alone doesn't guarantee that we can estimate the effects, we also need to have teams that switch between playoffs and regular season. Meaning, we need a team to be 0 (not iso) in the regular season and 1 (iso) in the playoffs. Also the other way around. We also need teams that are 0 in the regular season and 0 in the playoffs (same with 1). Is this feasible? No, so you are right to say we cannot test this.

Identifying the teams that switch both from non-iso offenses to iso-offenses and vice versa seems extremely unlikely. The other problem I see in addition to what you ID above, is that it seems very difficult to ID which teams are "iso" and which teams are not if you are looking for a TRUE/FALSE sort of variable since all teams run isolations in widely varying degrees. For example, in the Synergy statistics they have Orlando running a lot of isos when their offense is clearly more akin to the Hakeem era Rockets (inside/outside) style than the Hawks perimeter isolation offense. So I have some concerns about us even defining the terms here in a meaningful way.

A team that runs a lot of iso but that can go for long stretches to pick and roll or some other offense to exploit matchups is really different than ours. I'm not sure how the type of 1=iso, 0=non-iso format could address a situation where most teams are a shade of gray rather than black or white.

[Guest]

Identifying the teams that switch both from non-iso offenses to iso-offenses and vice versa seems extremely unlikely. The other problem I see in addition to what you ID above, is that it seems very difficult to ID which teams are "iso" and which teams are not if you are looking for a TRUE/FALSE sort of variable since all teams run isolations in widely varying degrees. For example, in the Synergy statistics they have Orlando running a lot of isos when their offense is clearly more akin to the Hakeem era Rockets (inside/outside) style than the Hawks perimeter isolation offense. So I have some concerns about us even defining the terms here in a meaningful way.

A team that runs a lot of iso but that can go for long stretches to pick and roll or some other offense to exploit matchups is really different than ours. I'm not sure how the type of 1=iso, 0=non-iso format could address a situation where most teams are a shade of gray rather than black or white.

I see what you are saying, and it actually gives me a better idea.

Yes, it is very hard to determine whether or not a team is an Iso or not. It also doesn't make a lot of sense, I mean what classifies us as an Iso offense? Well one way we could get around this issue would be to define an Iso team as one that runs Iso plays more than 33% of the time (or some different percentage if you want). Thats one solution.

But another cool thing that one could do is use Synergy. Look at the offensive production for teams when using an Iso offense in the regular season verse the playoffs. Then look at the offensive production for teams when not using an Iso offense in the regular season verse the playoffs. This will actually test whether an iso offense is more or less productive in the playoffs than the regular season. I haven't used Synergy, but if its at all what I imagine it is then we should be able to estimate this.

[AHF]

I see what you are saying, and it actually gives me a better idea.

Yes, it is very hard to determine whether or not a team is an Iso or not. It also doesn't make a lot of sense, I mean what classifies us as an Iso offense? Well one way we could get around this issue would be to define an Iso team as one that runs Iso plays more than 33% of the time (or some different percentage if you want). Thats one solution.

But another cool thing that one could do is use Synergy. Look at the offensive production for teams when using an Iso offense in the regular season verse the playoffs. Then look at the offensive production for teams when not using an Iso offense in the regular season verse the playoffs. This will actually test whether an iso offense is more or less productive in the playoffs than the regular season. I haven't used Synergy, but if its at all what I imagine it is then we should be able to estimate this.

Assuming they are accurately and consistently labeling plays, I am with you. That would be very interesting.