Nate introduces this year's PECOTA-based look at ranking prospects. Today, he lays out the methodology, which includes a few key changes to how he approached this project last year.
Last year, we ran our first-ever series of PECOTA-based prospect rankings. This wasn’t necessarily intended to be an annual feature, but it proved to generate a lot of good discussion, so here we are again.
Kevin tempers the enthusiasm we all have about our team's minor league system.
Prior to the 2001 season, Baseball America published their first Prospect Handbook, and it was a groundbreaking work. With rankings of the top 30 prospects for each team, and a write-up for each one, fans were suddenly able to get an in-depth look at their favorite team's system far more extensively than ever before. But somewhere between there and now, something happened. That something involved how fans interpreted these lists, as opposed to how they were compiled. With the growth in prospect interest over the past few years (due to a variety of reasons), rankings are now everywhere. Baseball America still does them, but plenty of other publications, Web sites, blogs, etc. have entered the fray. One of the key things I believe nearly all prospect rankers have failed to do, however, is to manage expectations. Many fans tend to believe that when they look at a top 10 list, they are looking at 10 future big league players, or even future stars. Although it's no fault of the rankers, the reality couldn't be further from the truth.
The rest of this article is restricted to Baseball Prospectus Subscribers.
Not a subscriber?
Click here for more information on Baseball Prospectus subscriptions or use the buttons to the right to subscribe and get access to the best baseball content on the web.
We take a look inside the selection criteria for assembling our Top 50 Prospects list.
For example, BP's Top 50 from 2005, while not without flaws, was better than most. Sure there were pitchers ranked highly (Richie Gardner and Adam Miller) who succumbed to arm injuries. We had thought that Willy Aybar (#34) would develop some power by now and that Edwin Jackson (#45) would improve from his sub-par 2004. Despite those missteps, Baseball Prospectus is proud of the work that went into that list as well as the 2006 version.
With a verifiable cornucopia of prospect lists out in cyberspace, there of course exists a vast array of philosophies governing the compilation of these lists. The king of prospect sites, Baseball America, ranks prospects based on scouting reports, tools, upside, age vs. level of competition and performance. Other sites lean heavily on a player's walk rate. Take, for example, the case of second baseman Travis Denker, in the Dodgers' system. After Denker hit .310/.417/.556 in Low A as a 20-year-old, many sites had him among their top 50 and, in one case, much higher. With a BB/PA rate of .147, Denker has exhibited unusual plate discipline for a young prospect. However, what these lofty rankings ignored were his stone hands, iron glove, .155 EqA upon his promotion to High-A that year, and his PECOTA projections. When different ranking systems rate some pieces of the puzzle higher than other systems, wildly differing outcomes will result.