Lies, Damned Lies: PECOTA Takes on Prospects, Wrap-up

One last meta-argument to make, and then we’ll get into the fun stuff. Indulge me these 600 words.

Since the All-Star Game was first played in 1933, there have been 519 players who have been named to participate in the event at least three times. This constitutes a reasonable, if slightly arbitrary, definition of what we might consider to be a “star” performer; our apologies in advance to relatives of Tim Salmon and Bobby Abreu. If we divide that 519 figure by the number of seasons covered during this time period (73), we come up with 7.1 players that “graduate” to star status per year. That is, if we took a typical amateur draft class, we’d expect–give or take–seven players from that draft class to develop into stars.

Things become slightly more complicated if we look at a “prospect class” instead of a draft class, since a player can make a prospect list multiple times before he reaches (or doesn’t reach) the majors. But it’s safe to say that most of the players on any Top 100 list you look at are not going to be stars, perhaps with ten or twelve or fifteen exceptions. If hypothetically you had a prospect, let’s call him Pustin Dendrophilia, who was quite certain to be an above-average major league performer, perhaps finding his way onto an All-Star squad or two in his best seasons, but just as certainly wouldn’t be much better than that, he would still deserve to rate quite highly on a top prospects list, certainly no lower than #20 or so. Just which twenty prospects you’d rank ahead of him is a matter of some debate. But a list that rates prospects based solely on “ceiling” risks suffering from the Lake Wobegon effect: every prospect is above average.

There’s a distinction to be made, in other words, between “ceiling” and “upside.” “Ceiling,” as I think of it, is the best that player could plausibly turn out to be within some non-trivial (though possibly quite small) degree of probability. “Upside,” on the other hand, incorporates (or ought to incorporate) at least some estimate of the probability that the player actually reaches his ceiling. For example, something like this:

Runs Above Average        Pustin Dendrophilia        Milledge Fillmore
50+                                0%                       5%
40-50                             10%                      10%
30-40                             25%                      10%
20-30                             35%                      15%
10-20                             15%                      15%
0-10                              10%                      20%
<0                                 5%                      25%

"Milledge Fillmore" has a nominally higher ceiling than Pustin Dendrophilia; he has a 5% chance of being a 50 RAA player at his peak, whereas Dendropilia has none. But the overall upside profile of Dendrophilia is much more attractive. Looking solely at "ceiling," to invoke the Lies, Damned Lies Poker Analogy of the Week, is like concluding that 3-2 of clubs is a better poker hand than Ace-King offsuit because 3-2 suited can make a straight flush (using both cards), while the Ace-King cannot. There is a natural cognitive bias toward overestimating the probability that a plausible, though unlikely event will actually occur--winning a lot of money in your NCAA pool, meeting the woman of your dreams at the corner bar, the player who is struggling to hit Double-A pitching morphing into a superstar.

The Top 100

These are the top 100 prospects in baseball, rated purely by PECOTA. For the sake of comparison, I've included the player's ranking in the BP consensus Top 50 and the Baseball America Top 100 lists, respectively.

Keep in mind that PECOTA cannot rank players who have little or no professional track record at the Pioneer/Appalachian league levels or higher. This includes Justin Upton, Alex Gordon, Elvis Andrus, and the like. It will also underrate players with very limited professional playing time, like Troy Tulowitzki and Jeff Clement. Finally, PECOTA has trouble with players with significant injury issues, or players who are about to undergo position switches.

Rank     Player                              Score        BP50     BA100
------------------------------------------------------------------------
1        Howie Kendrick, 2B, LAA (22)        512.9         5        12
2        Ryan Zimmerman, 3B, WAS (21)        492.9         3        15
3        Delmon Young, RF, TBA (20)          483.1         1         1
4        Dustin Pedroia, 2B, BOS (22)        472.7        11        77
5        Chris Young, CF, ARI (22)           437.8         8        23
6        Andy Marte, 3B, CLE (22)            426.2         7        14
7        Prince Fielder, 1B, MIL (22)        409.6         9        11
8        Jeremy Hermida, RF, FLO (22)        378.7         2         4
9        Joel Guzman, SS, LAN (21)           374.3        14        26
10       Eric Aybar, SS, LAA (22)            369.2        50        46

11       Brandon Wood, SS, LAA (21)          347.8         6         3
12       Eduardo Nunez, SS, NYA (19)         345.7        --        --
13       Joshua Barfield, 2B, SDN (23)       327.0        49        --
14       Ian Kinsler, 2B, TEX (24)           326.9        39        --
15       Francisco Liriano, SP, MIN (22)     318.2         4         6
16       Corey Hart, RF, MIL (24)            315.2        HM        --
17       Yusmeiro Petit, SP, FLO (21)        314.4        33        69
18       Philip Hughes, SP, NYA (20)         310.8        40        39
19       Jarrod Saltalamacchia, C, ATL (21)  306.4        23        18
20       Adam Jones, SS, SEA (20)            295.4        44        64

21       Donald Murphy, 2B, KCA (23)         295.1        --        --
22       Mike Jacobs, 1B, FLO (25)           288.9        42        --
23       Alexander Romero, LF, MIN (22)      288.1        --        --
24       Josh Willingham, C, FLO (27)        279.3        48        --
25       Jason Stokes, 1B, FLO (24)          270.6        --        --
26       Justin Huber, 1B, KCA (23)          269.3        34        84
27       Justin Verlander, SP, DET (23)      268.3        15         8
28       Kevin Frandsen, 2B, SFN (24)        267.6        --        --
29       Brian Anderson, CF, CHA (24)        265.7        31        51
30       Delwyn Young, 2B, LAN (24)          263.4        --        --

31       Ryan Garko, 1B, CLE (25)            257.2        --        --
32       Anthony Reyes, SP, SLN (24)         254.3        25        41
33       Matt Cain, SP, SFN (21)             253.3        12        10
34       Rob Cosby, 3B, TOR (25)             252.0        --        --
35       Eric Patterson, 2B, CHN (23)        250.9        HM        --
36       Franklin Gutierrez, CF, CLE (23)    246.8        --        --
37       Hanley Ramirez, SS, FLO (22)        246.7        HM        30
38       Conor Jackson, 1B, ARI (24)         246.2        HM        17
39       Nathan McLouth, LF, PIT (24)        241.8        --        --
40       Daric Barton, 1B, OAK (20)          239.1        18        28

41       Carlos Quentin, RF, ARI (23)        238.2        27        20
42       Stephen Drew, SS, ARI (23)          237.8        19         5
43       Kenji Jojima, C, SEA (30)           234.9        21        66
44       Felix Pie, CF, CHN (21)             234.5        29        27
45       Kendry Morales, 1B, LAA (23)        233.6        26        78
46       Lastings Milledge, CF, NYN (21)     228.4        13         9
47       Jeffrey Mathis, C, LAA (23)         226.1        --        60
48       Michael Napoli, C, LAA (24)         224.7        --        --
49       Willy Aybar, 3B, LAN (23)           224.6        --        --
50       Billy Butler, LF, KCA (20)          221.8        20        29

Rank     Player                              Score        BP50     BA100
------------------------------------------------------------------------
51       Jason Botts, LF, TEX (25)           221.2        --        --
52       Ron Paulino, C, PIT (25)            220.4        --        --
53       Ryan Shealy, 1B, COL (26)           219.7        --        --
54       Andy Laroche, 3B, LAN (22)          218.3        22        19
55       Russell Martin, C, LAN (23)         215.4        HM        42
56       Hunter Pence, CF, HOU (23)          214.2        HM        --
57       Jeremy Sowers, SP, CLE (23)         213.6        17        53
58       Craig Stansberry, 2B, PIT (24)      212.3        --        --
59       Joel Zumaya, SP, DET (21)           211.8        28        35
60       Jeff Baker, 3B, COL (25)            210.5        --        --

61       Brandon Moss, RF, BOS (22)          206.5        --        --
62       Matthew Kemp, RF, LAN (21)          206.4        HM        96
63       Cole Hamels, SP, PHI (22)           204.7        HM        68
64       Melky Cabrera, CF, NYA (21)         203.4        --        --
65       Kevin Melillo, 2B, OAK (24)         203.3        --        --
66       Brad Snyder, CF, CLE (24)           200.9        --        71
67       Neil Walker, C, PIT (20)            200.6        --        43
68       Elijah Dukes, CF, TBA (22)          200.0        --        --
69       Chuck James, SP, ATL (24)           199.0        --        98
70       Andrew McCutchen, CF, PIT (19)      198.8        --        50

71       Kevin Thompson, LF, NYA (26)        195.9        --        --
72       Ian Stewart, 3B, COL (21)           192.7        16        16
73       Chin-Lung Hu, SS, LAN (22)          192.4        --        --
74       Chris Snelling, RF, SEA (24)        191.4        46        --
75       Yunel Escobar, SS, ATL (23)         190.9        --        --
76       Alberto Callaspo, 2B, LAA (23)      188.8        --        --
77       Aarom Baldiris, 3B, NYN (23)        187.1        --        --
78       Anderson Hernandez, SS, NYN (23)    187.0        --        --
79       Chad Billingsley, SP, LAN (21)      186.6        24         7
80       Marcus Sanders, SS, SFN (20)        185.4        --        65

81       Ben Johnson, RF, SDN (25)           184.3        --        --
82       Tony Giarratano, SS, DET (23)       184.0        --        --
83       Joaquin Arias, SS, TEX (21)         183.1        --        --
84       Ryan Braun, 3B, MIL (22)            182.4        37        49
85       Brian Barton, CF, CLE (24)          182.2        --        --
86       Wesley Bankston, 1B, TBA (22)       181.3        --        --
87       Alejandro Machado, 2B, BOS (24)     180.9        --        --
88       Val Majewski, CF, BAL (25)          180.4        --        --
89       Chris Denorfia, CF, CIN (25)        178.2        --        --
90       Jeffrey Keppinger, 2B, NYN (26)     177.6        --        --

91       Sean Rodriguez, SS, LAA (21)        177.4        --        --
92       Jason Kubel, RF, MIN (24)           177.4        --        58
93       Ryan Raburn, 2B, DET (25)           176.9        --        --
94       Eric Duncan, 3B, NYA (21)           176.0        --        86
95       Eider Torres, 2B, CLE (23)          174.9        --        --
96       Miguel Montero, C, ARI (22)         174.5        --        --
97       Nelson Cruz, RF, MIL (25)           171.9        --        --
98       Josh Wilson, SS, COL (25)           171.7        --        --
99       Carlos Gonzalez, RF, ARI (20)       171.3        HM        32
100      Hernan Iribarren, 2B, MIL (22)      170.4        --        --

It's pretty clear where each of the three lists falls on the relevant Venn Diagram. Considering only the Top 50s (excluding honorable mentions), there are 33 players common to the BP and PECOTA lists, 32 common to the BP and BA lists, but just 24 common to the BA and PECOTA lists. We can form "teams" if we like, consisting of the highest ranking players in the BA and PECOTA Top 50s that do not rank in the opposing Top 50 at all.

Team PECOTA

• Dustin Pedroia
• Eduardo Nunez
• Josh Barfield
• Ian Kinsler
• Corey Hart
• Adam Jones
• Yusmeiro Petit
• Donald Murphy
• Mike Jacobs
• Alexander Romero

Team BA

• Chad Billingsley
• Ian Stewart
• Andy Laroche
• Nick Markakis
• Jon Lester
• Carlos Gonzalez
• Scott Olsen
• Jon Papelbon
• Homer Bailey
• Anibel Sanchez

I feel like I've just added a keeper team for the next ten, twenty years. Go Dendrophiliacs!

Almost Prospects

We can also do a combined list featuring true prospects, "almost" prospects, and 24-and-under players. We can think of these players as the best pre-arbitration eligible commodities in baseball--which necessarily makes them the most valuable commodities in baseball, period.

Rank     Player                             Prospect?        Score
------------------------------------------------------------------
1        Miguel Cabrera, LF, FLO (23)                        824.1
2        David Wright, 3B, NYN (23)                          784.4
3        Joe Mauer, C, MIN (23)                              622.3
4        Felix Hernandez, SP, SEA (20)                       590.9
5        B.J. Upton, SS, TBA (21)                            563.3
6        Howie Kendrick, 2B, LAA (22)        **              512.9
7        Jhonny Peralta, SS, CLE (24)                        512.1
8        Grady Sizemore, CF, CLE (23)                        496.7
9        Ryan Zimmerman, 3B, WAS (21)        **              492.9
10       Ryan Howard, 1B, PHI (26)                           487.0

11       Delmon Young, RF, TBA (20)          **              483.1
12       Dustin Pedroia, 2B, BOS (22)        **              472.7
13       Brian Mccann, C, ATL (22)                           457.5
14       Rickie Weeks, 2B, MIL (23)                          449.6
15       Francisco Rodriguez, RP, LAA (24)                   446.9
16       Chris Young, CF, ARI (22)           **              437.8
17       Dontrelle Willis, SP, FLO (24)                      432.6
18       Andy Marte, 3B, CLE (22)            **              426.2
19       Rich Harden, SP, OAK (24)                           413.0
20       Prince Fielder, 1B, MIL (22)        **              409.6

21       Wily Pena, RF, CIN (24)                             401.4
22       Jose Reyes, SS, NYN (23)                            400.6
23       Edwin Encarnacion, 3B, CIN (23)                     396.6
24       Curtis Granderson, CF, DET (25)                     389.0
25       Jeremy Hermida, RF, FLO (22)        **              378.7
26       Joel Guzman, SS, LAN (21)           **              374.3
27       Eric Aybar, SS, LAA (22)            **              369.2
28       Jonny Gomes, RF, TBA (25)                           368.6
29       Dallas McPherson, 3B, LAA (25)                      358.4
30       Jeremy Bonderman, SP, DET (23)                      357.2

31       Jorge Cantu, 2B, TBA (24)                           355.7
32       Zach Duke, SP, PIT (23)                             350.9
33       Brandon Wood, SS, LAA (21)          **              347.8
34       Eduardo Nunez, SS, NYA (19)         **              345.7
35       James Hardy, SS, MIL (23)                           344.0
36       Jeffrey Francoeur, RF, ATL (22)                     331.3
37       Chris Shelton, 1B, DET (26)                         327.7
38       Joshua Barfield, 2B, SDN (23)       **              327.0
39       Ian Kinsler, 2B, TEX (24)           **              326.9
40       Jose Lopez, 2B, SEA (22)                            323.7

41       Dioner Navarro, C, LAN (22)                         320.2
42       Robinson Cano, 2B, NYA (23)                         319.6
43       Francisco Liriano, SP, MIN (22)     **              318.2
44       Carl Crawford, LF, TBA (24)                         316.4
45       Corey Hart, RF, MIL (24)            **              315.2
46       Yusmeiro Petit, SP, FLO (21)        **              314.4
47       Philip Hughes, SP, NYA (20)         **              310.8
48       Jarrod Saltalamacchia, C, ATL (21)  **              306.4
49       Adam Jones, SS, SEA (20)            **              295.4
50       Donald Murphy, 2B, KCA (23)         **              295.1

Lastly, here are the fifty highest PECOTA five-year peak ratings with no restrictions on eligibility--roughly how the players might go in some sort of super fantasy draft. Notice the relative dearth of true prospects on this list.

Rank     Player                            Prospect?        Score
-----------------------------------------------------------------
1        Albert Pujols, 1B, SLN (26)                       1030.6
2        Miguel Cabrera, LF, FLO (23)                       824.1
3        Alex Rodriguez, 3B, NYA (30)                       797.7
4        David Wright, 3B, NYN (23)                         784.4
5        Adam Dunn, LF, CIN (26)                            662.1
6        Joe Mauer, C, MIN (23)                             622.3
7        Jake Peavy, SP, SDN (25)                           609.1
8        Johan Santana, SP, MIN (27)                        594.9
9        Felix Hernandez, SP, SEA (20)                      590.9
10       Jason Bay, LF, PIT (27)                            569.2

11       Mark Teixeira, 1B, TEX (26)                        568.9
12       B.J. Upton, SS, TBA (21)                           563.3
13       David Ortiz, DH, BOS (30)                          557.6
14       Vladimir Guerrero, RF, LAA (30)                    542.3
15       Carlos Zambrano, SP, CHN (25)                      541.2
16       Pedro Martinez, SP, NYN (34)                       539.6
17       Miguel Tejada, SS, BAL (30)                        524.5
18       Howie Kendrick, 2B, LAA (22)        **             512.9
19       Jhonny Peralta, SS, CLE (24)                       512.1
20       Grady Sizemore, CF, CLE (23)                       496.7

21       Lance Berkman, LF, HOU (30)                        494.2
22       Ryan Zimmerman, 3B, WAS (21)        **             492.9
23       Derrek Lee, 1B, CHN (30)                           492.3
24       Ryan Howard, 1B, PHI (26)                          487.0
25       Delmon Young, RF, TBA (20)          **             483.1
26       Jim Edmonds, CF, SLN (36)                          482.9
27       Dustin Pedroia, 2B, BOS (22)        **             472.7
28       Ben Sheets, SP, MIL (27)                           472.4
29       Derek Jeter, SS, NYA (32)                          463.2
30       Andruw Jones, CF, ATL (29)                         461.0

31       Brandon Webb, SP, ARI (27)                         460.9
32       Travis Hafner, DH, CLE (29)                        459.1
33       Brian Mccann, C, ATL (22)                          457.5
34       Mark Prior, SP, CHN (25)                           451.7
35       Rickie Weeks, 2B, MIL (23)                         449.6
36       Francisco Rodriguez, RP, LAA (24)                  446.9
37       Carlos Beltran, CF, NYN (29)                       444.1
38       Chris Young, CF, ARI (22)                          437.8
39       Marcus Giles, 2B, ATL (28)                         436.6
40       Hank Blalock, 3B, TEX (25)                         435.0

41       Dontrelle Willis, SP, FLO (24)                     432.6
42       Eric Chavez, 3B, OAK (28)                          428.7
43       Todd Helton, 1B, COL (32)                          428.0
44       Aramis Ramirez, 3B, CHN (28)                       427.8
45       Roy Oswalt, SP, HOU (28)                           427.5
46       Andy Marte, 3B, CLE (22)                           426.2
47       Roy Halladay, SP, TOR (29)                         422.2
48       Mike Young, SS, TEX (29)                           421.0
49       Vernon Wells, CF, TOR (27)                         420.4
50       Troy Glaus, 3B, TOR (29)                           420.1

Organization Ratings

It's a straightforward application of the individual player rankings to evaluate the relative strength of different farm systems. Here's what we get by summing the ratings within each organization for all "true" (pre-MLB) prospects with a combined PECOTA peak score of 100 or higher.

Best Farm Systems

1. Dodgers        2820.0
2. Indians        2608.7
3. Angels         2280.6
4. Marlins        2097.5
5. Brewers        2008.7
6. Twins          1970.7
7. Diamondbacks   1873.0
8. Devil Rays     1747.5
9. Tigers         1674.7
10. Pirates       1609.8
11. Yankees       1580.7
12. Rockies       1423.2
13. A's           1407.4
14. Giants        1399.2
15. Braves        1365.2
16. Red Sox       1338.7
17. Royals        1331.7
18. Rangers       1321.6
19. Mets          1225.2
20. Mariners      1223.9
21. Phillies      1212.6
22. Nationals     1189.6
23. Orioles       1180.6
24. Cardinals     1152.7
25. Cubs          1024.1
26. Blue Jays      954.6
27. Astros         784.6
28. Padres         734.3
29. White Sox      680.3
30. Reds           653.3

These numbers are really quite consistent with the BA organizational talent ratings, and the rest of the expert consensus on the depth of different systems. A lot of the noise cancels out when you're dealing with entire systems, rather than individual prospects; PECOTA might like Andy LaRoche a bit less than BA and Joel Guzman a bit more, but either way it recognizes that the Dodgers are loaded. One exception is that pitching-rich organizations like the Red Sox and Cubs do notably poorer by PECOTA's reckoning.

We can also look at the 24-and-under talent within each organization, without regard to prospect status. For a number of reasons, this is probably the fairer way to do things.

Best 24-and-Under Talent

1. Devil Rays     3893.2
2. Marlins        3601.1
3. Dodgers        3554.5
4. Angels         3503.3
5. Braves         3362.5
6. Mariners       3250.7
7. Indians        3173.7
8. Brewers        3100.2
9. Twins          3098.8
10. Royals        2979.1
11. Diamondbacks  2606.1
12. Mets          2582.5
13. Tigers        2202.6
14. A's           2074.4
15. Red Sox       1985.6
16. Pirates       1938.7
17. Cubs          1857.9
18. Yankees       1857.7
19. Rangers       1739.0
20. Reds          1734.9
21. Blue Jays     1695.1
22. Phillies      1610.6
23. Orioles       1530.9
24. Cardinals     1518.9
25. Nationals     1512.6
26. Giants        1405.5
27. Rockies       1360.2
28. White Sox     1334.5
29. Astros        1322.6
30. Padres         914.0

Don't wait: book your tickets now for the 2012 Trans-Everglades Classic. Hurricane-safe lodging still available.

Finally, we can disregard the age requirement entirely, to get a sense for present medium-to-long-term organizational health.

Overall Talent Stock

1. A's            8015.3
2. Dodgers        7609.6
3. Angels         7515.5
4. Indians        7488.5
5. Twins          7275.6
6. Yankees        7194.5
7. Braves         7047.1
8. Cardinals      7029.9
9. Red Sox        6904.1
10. Brewers       6670.8
11. Devil Rays    6660.4
12. Mets          6554.8
13. Pirates       6544.0
14. Phillies      6543.5
15. Blue Jays     6525.1
16. Mariners      6502.5
17. Cubs          6420.5
18. Diamondbacks  6390.0
19. Rangers       6378.9
20. Tigers        6322.0
21. Reds          5969.9
22. Rockies       5785.6
23. Royals        5726.6
24. Orioles       5623.3
25. Astros        5475.3
26. Marlins       5302.3
27. White Sox     5278.6
28. Giants        4873.8
29. Nationals     4843.7
30. Padres        4581.1

The biggest surprise may be the poor ranking of the two World Series combatants from last season, each of which couple pitching-heavy talent stocks with relatively barren farm systems. This is more proof that the White Sox got it right by understanding that their future is now. Other organizations would do well to recognize that true major-league-plus talent is rare, and that when you have a reasonable assemblage of it, it will usually be better to take advantage of your window of opportunity than to default into another ill-conceived Five-Year Plan.