There have been a lot of questions about PECOTA in 2010. Here’s what we’re going to do to resolve them.
There are two concerns that we are currently working to address, and they apply to most of the questions people have asked. The first regards the fitness of the 2010 PECOTA projections. Clay Davenport posted some analysis that he did of them in an Unfiltered post on February 13. To summarize Clay’s work, there do not appear to be problems with the 2010 PECOTAs. The PECOTAs have changed since then, and we want to provide comprehensive analysis, so Colin Wyers will be repeating and expanding the tests that Clay did and we will report his findings.
The second concern is of the fitness of the ten-year projections which were released as part of the beta PECOTA hitter cards late Friday. As several commenters on the beta release announcement have noted, there are problems with the ten-year projections. Most of us saw those for the first time at about the same time you did, in our rush to get the cards released, and we’ve not yet gotten to the bottom of the issue. We will be getting help from our old friend Nate Silver on these, and will provide updates as we have more information. In the meantime, we’ll be prominently displaying a link to this post on the beta PECOTA cards.
PECOTA is currently in a state of flux as it moves from Nate Silver’s old Frankensteinian STATA/monster Excel spreadsheet process to a more automated, easier to maintain, easier to execute environment. We’ve pushed this along as fast as we were able, but we’ve plainly still got some work to do. I apologize again for the issues we’ve had with this transition, and I hope you will bear with us a little longer on this. I fully expect us to give you the best long-term projections we have ever generated soon.
We’ll have much more on these topics, but I wanted to get this information out as soon as I could. Thank you for your patience.