The Lake Wobegon effect, named after the fictional town from Garrison Keillor’s A Prairie Home Companion, is the idea that … aw, screw it, let’s just quote Wikipedia:
The Lake Wobegon effect is the human tendency to overestimate one’s achievements and capabilities in relation to others. It is named for the fictional town of Lake Wobegon from the radio series A Prairie Home Companion, where, according to Garrison Keillor, “all the women are strong, all the men are good-looking, and all the children are above average”. In a similar way, a large majority of people claim to be above average; this phenomenon has been observed among drivers, CEOs, stock market analysts, college students, police officers and state education officials, among others. Experiments and surveys have repeatedly shown that most people believe that they possess attributes that are better or more desirable than average.
Basically, the Lake Wobegon effect is the idea that everybody is above-average. This is something that any projection system needs to be mindful of. People often comment about a projection system along the lines of: “this set of projections is too optimistic”, or, “the PECOTAs are too pessimistic”; usually this just means that they don’t like the projection for a couple of players on their favorite team.
But there’s one fairly simple test that can be applied to see whether a projection system is worth its weight: put together depth charts for all 30 MLB teams, plug in the projections, extrapolate wins and losses, and see how close you get to 81 wins per team. Okay, so the mechanics of doing this are not so simple. But the idea behind it is simple: if you plug in all the numbers, and the average team is projected to win 90 games, then something is seriously wrong with your projection system — the Lake Wobegon effect.
It’s important to emphasize that avoiding the Lake Wobegon effect is a necessary but not sufficient condition of a good projection system. If you wanted to end up with projections wherein every team ended up with exactly 81 wins, you could simply project every player to perform at league average, and … voilà!. But that projection system would be completely useless. It’s a tougher task to keep your projection system balanced while projecting an appropriate array of variance in the individual player forecasts.
You’re probably waiting for me to tell you that PECOTA performs perfectly in this regard, but it doesn’t. When I put together a beta set of depth charts (to launch shortly on our Fantasy page), the average major league team came out with 83-84 wins. That’s a pretty good result — there are competing systems that would give you an average of 90 wins or more. But there are still a few excess wins in the system. Where do these come from?
As it happens, PECOTA does make some Lake Wobegon assumptions, not with respect to performance but rather with respect to playing time. Basically, the weighted mean forecast lines somewhat discount the risk of catastrophic injuries, the idea being that until we learn about little bit more about how to predict injuries, who pulls a Derrek Lee is something of a crap shoot. It isn’t a large fudge — maybe 3-5% per player — but if you take 3-5% of the playing time out of the depth charts and swap it with replacement level talent, you wind up very close to 81 wins per team.
p.s. We hope to have another build of the PECOTA cards ready to go by the end of the week. These will both update some transactions that we missed the first time around — Darin Erstad as a White Sock and so forth — and build in better assumptions for team-dependent stats like RBI, W-L, and ERA based on the depth charts. (Yes, ERA is team-dependent).
0 comments have been left for this post.
