Why PER Ain't no good

[Sir John]

I just don't gravitate to wins produced. Has it changed since the old days when the mysterious "team adjustment' was needed to get wins produced to make individual wins produced match teams wins produced? Haven't followed it's development thru the years

Win Shares seems much more reasonable: at least it's transparent

edit: looks like they were able to save some of the apbrmetrics threads. For those you enjoy nerdfights, here is one where Dan Rosenbaum just goes off on Dave Berri
http://godismyjudgeok.com/DStats/AP...t=1232&postdays=0&postorder=asc&start=15.html

Back in town. Allow me to share with you a few things in response:

1. It's great that you prefer Win Shares to Wins Produced. What you may not realize it they are very, very similar in how they are calculated. Dave Berri actually worked with Dean Oliver (Win Shares) to create Wins Produced which is basically just a refinement of Win Shares. Berri and Oliver are friends who have worked together. Dean Oliver consulted with Dave Berri on creating and refining Wins Produced. These two systems aren't really competitors to each other. The underlying principles are the same. The only differences are slight changes in how certain variables are rated. It's not possible to like one but dislike the other. You can slightly prefer one to the other but that's it. In fact, by conceding that you like Win Shares, you just stated that you like Wins Produced too.

2. Wins Produced is the far more transparent of the two. What is being called Dean Oliver's Win Shares is not actually calculated by Dean Oliver himself. Kubatko took Oliver's formulas from his book Basketball on Paper and tried to translate them into his approximation of what Dean Oliver might create if Dean Oliver was into publicly making available his stats. In fact, Dean Oliver works as a private consultant and does not and will not publicly debate / discuss or explain Win Shares.

Understand: I like Win Shares a lot (I use it / refer to it as much as any of them) and Dean Oliver even more (he's a pioneer in the field) but Win Shares is significantly less transparent than Wins Produced by every measure. This isn't even up for debate. On the other hand, Dave Berri has published Wins Produced formula in full both on the web and in two books. He routinely publishes academic papers and submits his ideas and math to official peer review. He also engages critics on his blog, responds to comments, and generally behaves like an open academic and not a proprietary consultant trying to keep things secret so he can sell his services. So you have actually got it opposite. Dave Berri and Wins Produced is the most transparent of the publicly known metrics. What Dan Rosenbaum and John Hollinger do is among the least transparent.

3. Dave Berri has in fact publicly responded to Dan Rosenbaum's criticism in multiple places at multiple times, in both blogs, books, and peer reviewed journals. Here's but one example among many: http://wagesofwins.com/2006/11/26/answering-a-critic/
Conversely, Dan Rosenbaum has NEVER responded publicly to any criticisms of his ideas / systems. This is the difference between a true academic and someone who is trying to make money as a proprietary consultant.

4. There never was a "mysterious" team adjustment or anything else "mysterious" in Wins Produced. Yes, there is a publicly acknowledged and published factor that includes team defense (which is entirely reasonable and well explained by Berri all over the place -- and I might add not in the least unique to Wins Produced). Everything that Berri and Wins Produced publishes including the entire formula and all the calculations are available for free to read on his website. If you know math, there's nothing mysterious about it. It's all written out and explained in black and white. (I supposed that if someone don't know math, then I guess it might seem "mysterious" but then they wouldn't be qualified to have much of an opinion on any of this.)

4. Another difference is that people like Dave Berri can and do admit they are wrong and make changes something that those hawking proprietary formulas and trying to get rich from them seldom if ever do. Berri is happy to admit that he's not a perfect all-knowing god. Here's an interesting example: after years of discussion, debate, input from qualified people (including Dean Oliver of Win Shares), in 2011, Berri changed how Wins Produced is calculated. Specifically, he decided to lessen how much weight he was giving to rebounding.

This is how open-minded and academic people behave. They don't dogmatically lock themselves into one position. They strive to continually improve their understanding and their output. They have dialogue with other experts. As their information changes, they allow the evidence to take them where it leads, not blindly argue against or ignore anything that contradicts their worldview. Science, math, statistics is an open-ended process. Sincere practitioners constantly try to do better, improve, and understand more.

These are the kind of people I prefer to listen to not the dogmatic fundamentalists. But hey, to each his own.

 

[Sir John]

Re: Advanced Stats, the Cavs, and Player Evaluation

Yes, I think we just did. As it is widely known, Hollinger has stated his metric is used to gauge offense, not defense.

No he has not. I've written this before and you ignored it, so let me make this as clear as possible: I CHALLENGE YOU TO FIND A STATEMENT FROM HOLLINGER TO THIS EFFECT.

Until you find it from Hollinger and quote it with a reference that the rest of us can see and confirm for ourselves, you don't have a leg to stand on. You are trying to fabricate your way out of a false claim that you made.

 

[Sir John]

Re: Advanced Stats, the Cavs, and Player Evaluation

Does that not settle the matter? Hollinger has admitted numerous times that PER does not measure defensive qualities because they are not deduced from the box score. Therefore, it has very limited usefulness when discussing a player's defensive qualities. However, it is very useful when gauging their offensive attributes, as evidenced by the PER output for NBA players year-to-year when compared with any standard analysis. PER does generally work to rank the offensive talents of players (in the NBA), does it not?



Yes, I think we just did. As it is widely known, Hollinger has stated his metric is used to gauge offense, not defense.

No PER absolutely does not accurately rank the offensive talents of players.

In fact, it's time to clearly addresses the even bigger issue with PER which so far not a single pro-PER person has mathematically discussed: even as an offensive metric, PER is bad. It does NOT do a good job of pinpointing who is a good offensive player. It routinely gives inflated PER numbers to players whose offensive play is preventing the team from winning because they are so terribly inefficient at scoring.

That is the heart and soul of the critique of PER and not a single one of you pro-PER has said much about it. Curious.

 

[Douglar]

Re: Advanced Stats, the Cavs, and Player Evaluation

even as an offensive metric, PER is bad.

People are bad. PER is a math function.

Math functions can be used to extrapolate relationships between basketball stats and subjective attributes.

Those relationships can be tightly correlated , loosely corrolated, not corrolated, or inversely corrolated.

If I find PER to be a reasonably effective, easy to find, and easy to read tool for creating loosely corrolated groupings for NBA players.

How can that be bad?

 

[KI4MVP]

I guess this got pulled out of another thread because the first post has nothing to do with PER.

There is something about it being a good thing that the people who come up with these formulas are willing to adjust their formula. First, I'm not sure how that's better than getting it "right" the first time. Second, Hollinger make a significant addition a few years ago. He didn't adjust the PER formula itself, he came up with two new calculations based on PER. PER*minutes played is Value added. Value Added divided by 48 is Estimated Wins Added. Using either of these numbers to rate a players impact gets rid of the issue that a guy like Durmmond could be 17th in the league in PER while just playing 19 minutes/game. Clearly he's not having the impact Kyrie has at #18, playing 34.7 minutes/game. And neither of them had the impact of Curry at 19 on the season, who played almost 20 more games. VA and EWA makes these adjustments.

No statistical analysis is going to be perfect because everything isn't tracked. For example, a charge drawn should be worth as much as a steal, but it isn't tracked. Contesting a shot has value, but isn't tracked. But EWA is gives a pretty close estimation of who the most important players in the league were this year when games played is factored in.

http://insider.espn.go.com/nba/hollinger/statistics/_/sort/VORPe

The main change I would make is rank Duncan higher. It also pretty accuratly shows the gap between LeBron and Durant and the larger gap between them and the rest of the league.

And Hollinger himself used his own tools to determine it wouldn't hurt Memphis to trade Rudy Gay, who had a reputation and salary of a near all star, but who's advanced stats said he was simply a high volume average player. They've already gotten further this year without him than they did last year with him facing the same team in the first round.

 

[Randolphkeys]

I'm really hoping the Hollinger-philes give the Memphis project longer than one playoff series to assess his system. His system will have nothing to do with all the injuries to the expected top teams in the Western Conference.

 

[KI4MVP]

^^ what do injuries have to do with beating the clippers? When Memphis made the trade, people thought Hollinger was crazy.

 

[Randolphkeys]

^^ what do injuries have to do with beating the clippers? When Memphis made the trade, people thought Hollinger was crazy.

I think it is fair to say they beat the Clippers straight up, but my post had more to do with success they should find against a Westbrookless OKC.

 

[gourimoko]

Re: Advanced Stats, the Cavs, and Player Evaluation

No PER absolutely does not accurately rank the offensive talents of players.

In fact, it's time to clearly addresses the even bigger issue with PER which so far not a single pro-PER person has mathematically discussed: even as an offensive metric, PER is bad. It does NOT do a good job of pinpointing who is a good offensive player. It routinely gives inflated PER numbers to players whose offensive play is preventing the team from winning because they are so terribly inefficient at scoring.

That is the heart and soul of the critique of PER and not a single one of you pro-PER has said much about it. Curious.

Now we're getting somewhere..

You're saying that PER is not useful to gauge offensive production, I contend it is, could you provide evidence to your statement. Again, with respect to individual performance, not that of the team.

 

[Ohio]

PER might be my favorite advanced statistic. I find it pretty accurate, at least with guards, anyway.

 

[Tornicade]

Can we move this back to the advanced stats thread so i dont have to retype all my post. i find myself tiring of the op's bush league forum tactics

 

[Tornicade]

Back in town. Allow me to share with you a few things in response:

1. It's great that you prefer Win Shares to Wins Produced. What you may not realize it they are very, very similar in how they are calculated. Dave Berri actually worked with Dean Oliver (Win Shares) to create Wins Produced which is basically just a refinement of Win Shares. Berri and Oliver are friends who have worked together. Dean Oliver consulted with Dave Berri on creating and refining Wins Produced. These two systems aren't really competitors to each other. The underlying principles are the same. The only differences are slight changes in how certain variables are rated. It's not possible to like one but dislike the other. You can slightly prefer one to the other but that's it. In fact, by conceding that you like Win Shares, you just stated that you like Wins Produced too.

2. Wins Produced is the far more transparent of the two. What is being called Dean Oliver's Win Shares is not actually calculated by Dean Oliver himself. Kubatko took Oliver's formulas from his book Basketball on Paper and tried to translate them into his approximation of what Dean Oliver might create if Dean Oliver was into publicly making available his stats. In fact, Dean Oliver works as a private consultant and does not and will not publicly debate / discuss or explain Win Shares.

Understand: I like Win Shares a lot (I use it / refer to it as much as any of them) and Dean Oliver even more (he's a pioneer in the field) but Win Shares is significantly less transparent than Wins Produced by every measure. This isn't even up for debate. On the other hand, Dave Berri has published Wins Produced formula in full both on the web and in two books. He routinely publishes academic papers and submits his ideas and math to official peer review. He also engages critics on his blog, responds to comments, and generally behaves like an open academic and not a proprietary consultant trying to keep things secret so he can sell his services. So you have actually got it opposite. Dave Berri and Wins Produced is the most transparent of the publicly known metrics. What Dan Rosenbaum and John Hollinger do is among the least transparent.

3. Dave Berri has in fact publicly responded to Dan Rosenbaum's criticism in multiple places at multiple times, in both blogs, books, and peer reviewed journals. Here's but one example among many: http://wagesofwins.com/2006/11/26/answering-a-critic/
Conversely, Dan Rosenbaum has NEVER responded publicly to any criticisms of his ideas / systems. This is the difference between a true academic and someone who is trying to make money as a proprietary consultant.

4. There never was a "mysterious" team adjustment or anything else "mysterious" in Wins Produced. Yes, there is a publicly acknowledged and published factor that includes team defense (which is entirely reasonable and well explained by Berri all over the place -- and I might add not in the least unique to Wins Produced). Everything that Berri and Wins Produced publishes including the entire formula and all the calculations are available for free to read on his website. If you know math, there's nothing mysterious about it. It's all written out and explained in black and white. (I supposed that if someone don't know math, then I guess it might seem "mysterious" but then they wouldn't be qualified to have much of an opinion on any of this.)

4. Another difference is that people like Dave Berri can and do admit they are wrong and make changes something that those hawking proprietary formulas and trying to get rich from them seldom if ever do. Berri is happy to admit that he's not a perfect all-knowing god. Here's an interesting example: after years of discussion, debate, input from qualified people (including Dean Oliver of Win Shares), in 2011, Berri changed how Wins Produced is calculated. Specifically, he decided to lessen how much weight he was giving to rebounding.

This is how open-minded and academic people behave. They don't dogmatically lock themselves into one position. They strive to continually improve their understanding and their output. They have dialogue with other experts. As their information changes, they allow the evidence to take them where it leads, not blindly argue against or ignore anything that contradicts their worldview. Science, math, statistics is an open-ended process. Sincere practitioners constantly try to do better, improve, and understand more.

These are the kind of people I prefer to listen to not the dogmatic fundamentalists. But hey, to each his own.

No he has not. I've written this before and you ignored it, so let me make this as clear as possible: I CHALLENGE YOU TO FIND A STATEMENT FROM HOLLINGER TO THIS EFFECT.

Until you find it from Hollinger and quote it with a reference that the rest of us can see and confirm for ourselves, you don't have a leg to stand on. You are trying to fabricate your way out of a false claim that you made.

No PER absolutely does not accurately rank the offensive talents of players.

In fact, it's time to clearly addresses the even bigger issue with PER which so far not a single pro-PER person has mathematically discussed: even as an offensive metric, PER is bad. It does NOT do a good job of pinpointing who is a good offensive player. It routinely gives inflated PER numbers to players whose offensive play is preventing the team from winning because they are so terribly inefficient at scoring.

That is the heart and soul of the critique of PER and not a single one of you pro-PER has said much about it. Curious.

It doesn't actually matter or relevant what hollinger says about per but I quoted the offensive reference on the thread you ditched to reiterate your hyperbole.

What is relevant is the actual equation itself and its overall purpose. Its main purpose is to sort and separated what players are playing at a superstar level, Star level, okay level and below average level. of course there are exception and hollinger never hesitated being upfront about that systems drawbacks. If one has an understanding of those drawbacks and limitations it allows for a useful tool when analyzing basketball. you want one source or magic system to provide a complete analysis when the only things even close are a culmination of multiple types of statistical analysis combined and set to an equation.

Yes hollinger has modifiers. but so do the other systems. in fact I haven't yet to see a system that at some point doesnt have arbitrary weightings have a bearing on their overall outcome. even the ones that use equations to produce those arbitrary weightings.


noones being dogmatic about anything. your on this crusade to stop the usage of PER. PEr works for what its supposed to. its a general snapshot of players production.people dont use it exclusively and typically a variety of different metrics will give one a better overall view of a player statistically.

+/-, Per, simple ratngs. offensive and defensive rating, win shares. wages of wins. all have ther good ponts and drawbacks. ultimately its not he stat but how some might use the stats that become problematic.

 

[Douglar]

Anyone want to provide some examples were PER is bad but another box score aggregate is better?

These pages might help locate some: http://www.82games.com/1213/1213CLE.HTM

They are also very helpful for locating players where OppPer just looks silly.

 

[Pioneer10]

I wish Meshuggener would spend his time trying to demolish Opp PER. That would be something I would jump on w/o hesitation :chuckles:

 

[col63onel]

Re: Advanced Stats, the Cavs, and Player Evaluation

No he has not. I've written this before and you ignored it, so let me make this as clear as possible: I CHALLENGE YOU TO FIND A STATEMENT FROM HOLLINGER TO THIS EFFECT.

Until you find it from Hollinger and quote it with a reference that the rest of us can see and confirm for ourselves, you don't have a leg to stand on. You are trying to fabricate your way out of a false claim that you made.

"Bear in mind that this rating is not the final, once-and-for-all answer for a player's accomplishments during the season. This is especially true for players such as Bruce Bowen and Trenton Hassell who are defensive specialists but don't get many blocks or steals."

https://en.wikipedia.org/wiki/Player_efficiency_rating

I always use PER as a baseline, and combine with other stats. If you are a bigman, I look at REB% as well. If you are a PG, AST%. And always look at TS%.