CSS Button No Image Css3Menu.com

Baseball Prospectus home
  
  
Click here to log in Click here for forgotten password Click here to subscribe

No Previous Article
No Next Article

April 18, 2001

Top 40 Prospects In Review: Part Seven

Epilogue

by Rany Jazayerli

Part One
Part Two
Part Three
Part Four
Part Five
Part Six

First, let's recap the complete Top 40 lists for each publication, along with the grade for each player:

Baseball Prospectus Baseball America John Sickels
1. Nick Johnson 1 1. Rick Ankiel 6 1. Corey Patterson 5
2. Rick Ankiel 6 2. Pat Burrell 5 2. Rick Ankiel 6
3. Pat Burrell 5 3. Corey Patterson 5 3. Pat Burrell 5
4. Sean Burroughs 4 4. Vernon Wells 3 4. Vernon Wells 3
5. Vernon Wells 3 5. Nick Johnson 1 5. Nick Johnson 1
6. Rafael Furcal 7 6. Ruben Mateo 4 6. Dee Brown 3
7. Ruben Mateo 4 7. Sean Burroughs 4 7. Kip Wells 2
8. Ben Petrick 5 8. Rafael Furcal 7 8. Ben Petrick 5
9. Corey Patterson 5 9. Ryan Anderson 5 9. Sean Burroughs 4
10. Michael Cuddyer 3 10. John Patterson 1 10. Michael Cuddyer 3
11. Dee Brown 3 11. Dee Brown 3 11. D'Angelo Jimenez INC
12. Kip Wells 2 12. Mark Mulder 4 12. Ruben Mateo 4
13. Jack Cust 3 13. Josh Hamilton 5 13. Brad Penny 5
14. Tony Armas Jr. 5 14. Kip Wells 2 14. Chin-Feng Chen 2
15. D'Angelo Jimenez INC 15. Matt Riley 1 15. Rafael Furcal 7
16. Peter Bergeron 3 16. Alfonso Soriano 3* 16. Eric Gagne 3
17. Chin-Feng Chen 2 17. Chin-Feng Chen 2 17. Matt Riley 1
18. Esteban German 2 18. Michael Cuddyer 3 18. Hee Seop Choi 6
19. Matt Riley 1 19. Josh Beckett 4 19. Wilfredo Rodriguez 2
20. Lance Berkman 6 20. A.J. Burnett 5 20. Ed Yarnall 2
21. Brad Penny 5 21. Travis Dawkins 2 21. Ramon Ortiz 5
22. Milton Bradley 4 22. Brad Penny 5 22. Ryan Anderson 5
23. Chad Hermansen 1 23. Eric Munson 3 23. Alfonso Soriano 3*
24. Ryan Anderson 5 24. Drew Henson 4 24. John Patterson 1
25. Eric Gagne 3 25. Wilfredo Rodriguez 2 25. Milton Bradley 4
26. Aubrey Huff 6 26. Michael Restovich 2 26. Jesus Colome 5
27. Matt LeCroy 2 27. Tony Armas Jr. 5 27. Ramon Hernandez NA
28. Ed Yarnall 2 28. Ramon Ortiz 5 28. Peter Bergeron 3
29. Marcus Giles 4 29. Francisco Cordero 2 29. Abraham Nunez 3
30. Michael Restovich 2 30. Abraham Nunez 3 30. Matt LeCroy 2
31. Jon Garland 6 31. Jack Cust 3 31. Mike Lamb 3
32. Adam Piatt 5 32. Jon Garland 6 32. Josh Hamilton 5
33. Alfonso Soriano 3* 33. Chad Hermansen 1 33. Tony Armas Jr. 5
34. Mike Meyers 3 34. Alex Escobar 6 34. Aaron Myette 3
35. Jayson Werth 2 35. Ben Petrick 5 35. Jack Cust 3
36. Jason Romano 3 36. Milton Bradley 4 36. Francisco Cordero 2
37. Ramon Ortiz 5 37. Lance Berkman 6 37. A.J. Burnett 5
38. Dernell Stenson 3 38. Felipe Lopez 4 38. Luis Rivera 2
39. Travis Dawkins 2 39. Danys Baez 3 39. Junior Guerrero 2
40. Mario Encarnacion 2 40. Chris George 5 40. Drew Henson 4

You'll note that Alfonso Soriano has an asterisk next to his grade. We originally awarded Soriano a grade 2 when we first started this retrospective, back in February, and events since then would suggest that perhaps Soriano's stock hasn't fallen as much as I had thought. I'm still not a believer in his long-term future, but a grade 3 now seems more appropriate.

Let's go ahead and get the rankings out of the way. Using a weighted average (the grade of the #1 prospect is worth 40 points, the #2 prospect 39 points, etc., with the #40 prospect just 1 point), here is the average score for each publication:

Publication Average
Baseball Prospectus 3.72
Baseball America 3.69
John Sickels 3.67

There hasn't been a race that close since the 1973 NL East. While we'd like to claim victory by a whisker, the race is so close that if we give D'Angelo Jimenez a grade 1 instead of an incomplete, and give Ramon Hernandez a grade 4, the scores would be:

Publication Average
Baseball America 3.69
John Sickels 3.638
Baseball Prospectus 3.637

The point is not simply to prove who's the best at making predictions--if it was, I could have left Soriano's grade as a 2 and engineered a clear BP victory! It should be pretty clear that this was damn well near a dead heat, and none of the publications can claim a better prospect list than the others. In itself, this is very revealing. It appears that traditional scouting methods do not evaluate prospects any more accurately than newer sabermetric methods. That doesn't mean that scouts are useless; it does mean that objective analysis has as much a place in the evaluation of prospects as the scouts do.

The scores hover around 3.7, which (given that a grade 4 is our equivalent of the .500 pitcher) suggests that none of us did a particularly good job ranking prospects last year. That only adds to the argument that last year was a very tough year in which to evaluate prospects. As this is the first time we've evaluated top prospects lists in this way, we can't tell if it was just a bad year or if projecting prospects is simply more difficult than we thought. Hopefully, we'll have our answer next season when we evaluate our prospect lists again.

Let's dig a little deeper. It's no secret that we are much more conservative when it comes to evaluating pitching prospects than their hitting brethren. This next chart illustrates how much more conservative we are compared to Baseball America and Sickels.

Publication Hitters Points Pitchers Points
Baseball America 24 506 16 314
John Sickels 23 527 17 293
Baseball Prospectus 29 616 11 204

Only 11 of the 40 players on our Top 40 list were pitchers, and the number of points awarded for pitchers on our list is less than 25% of the total, compared to 38% for BA and 36% for Sickels.

Is that conservatism warranted? Let's split up the pitchers and hitters on each list, and compare what the weighted grades for each group were:

Publication Grade (Pitchers) Grade (Hitters)
Baseball America 3.73 3.66
John Sickels 3.46 3.79
Baseball Prospectus 3.97 3.64

This data is not particularly compelling, in that the worst projections for pitchers--and the best projections for hitters--came from John Sickels, whose philosophy on pitchers vs. hitters takes the middle ground between us and Baseball America. However, you'll note that the best grade by any publication in either category was for our projection of pitchers. We may be biased, but we'll take that to mean that by restricting inclusion on our list to only the very finest pitching prospects in the land, we can minimize the danger that we'll get burned by an injury to a John Patterson or make a mistake projecting an A-ball hurler like Wilfredo Rodriguez or believe in the myth of the minor-league closer with a Francisco Cordero.

The extra hitters that stock our list did not drag our grade for hitters down by much; our grade for hitters was just two-hundreths of a point lower than that of Baseball America, even while our grade for pitchers was nearly a quarter-point higher.

It's possible we swung too far the other way, however. If you look at just the bottom 10 prospects on our list, three of them are pitchers (Jon Garland, Mike Meyers, and Ramon Ortiz), who had respective scores of 6, 3, and 5. Of the seven hitters in our bottom 10, only one (Adam Piatt) had a grade higher than 3. It's reasonable to suggest that some of those extra hitters at the bottom of our Top 40 might be less deserving than the two or three best pitching prospects who didn't make the list.

If that's the case, hopefully we found a better balance this year. The Top 40 list in Baseball Prospectus 2001 includes 13 pitchers (up from 11) totalling 275 points (up from 204). Then again, 40 of those points were wasted on Ryan Anderson.

Let's look at how each publication fared with their "reaches," those players who didn't appear on either of the others' Top 40 lists. Both BA and Sickels had seven players unique to their Top 40 list, although Sickels only had six when you eliminate Ramon Hernandez (who wasn't a rookie last season, and so wasn't eligible for any other list). We had nine unique players, which isn't surprising given the lip service we sometimes pay to conventional wisdom. Here are those players:

Baseball Prospectus Baseball America John Sickels

18. Esteban German 2 12. Mark Mulder 4 18. Hee Seop Choi 6
26. Aubrey Huff 6 19. Josh Beckett 4 26. Jesus Colome 5
29. Marcus Giles 4 23. Eric Munson 3 31. Mike Lamb 3
32. Adam Piatt 5 34. Alex Escobar 6 34. Aaron Myette 3
34. Mike Meyers 3 38. Felipe Lopez 4 38. Luis Rivera 2
35. Jayson Werth 2 39. Danys Baez 3 39. Junior Guerrero 2
36. Jason Romano 3 40. Chris George 5
38. Dernell Stenson 3
40. Mario Encarnacion 2
Average: 3.33 Average: 4.14 Average: 3.50
Weighted Average: 3.56 Weighted Average: 3.94 Weighted Average: 4.57

It's clear that we did very poorly here. Six of our nine unique players, including all five ranked #34 or lower, scored a 3 or less. By comparison, Baseball America did very well with their unique players, and Sickels--thanks to big hits with Hee Seop Choi and Jesus Colome--had the highest weighted average in the group.

You'll also note that almost all of the poor grades in this category went to players at the bottom of the lists. Players that one organization gave a mention to at the bottom of their Top 40 list tended to crash and burn. By comparison, of the eight unique players who were ranked higher than #30--players that one publication was very optimistic about despite the lack of attention from the others--only two of them had grades below 4, Esteban German and Eric Munson.

So what's the lesson for the reader to take from this? Well, you could sum up the "points" for each player on all three lists and sum them together, creating a composite Top 40 list from the three publications. Such a list would look like this:

1. Rick Ankiel 6
2. Pat Burrell 5
3. Nick Johnson 1
4. Corey Patterson 5
4. Vernon Wells 3
6. Sean Burroughs 4
7. Ruben Mateo 4
8. Dee Brown 3
9. Rafael Furcal 7
10. Kip Wells 2
11. Michael Cuddyer 3
12. Chin-Feng Chen 2
13. Ben Petrick 5
13. Matt Riley 1
15. Ryan Anderson 5
16. Brad Penny 5
17. D'Angelo Jimenez I
18. Alfonso Soriano 3
19. Tony Armas Jr. 5
20. John Patterson 1
21. Jack Cust 3
22. Eric Gagne 3
23. Milton Bradley 4
24. Peter Bergeron 3
24. Wilfredo Rodriguez 2
26. Ramon Ortiz 5
26. Josh Hamilton 5
28. Ed Yarnall 2
29. Mark Mulder 4
30. Michael Restovich 2
30. Chad Hermansen 1
32. A.J. Burnett 5
32. Matt LeCroy 2
32. Lance Berkman 6
35. Abraham Nunez 3
36. Esteban German 2
36. Hee Seop Choi 6
38. Josh Beckett 4
38. Travis Dawkins 2
40. Jon Garland 6

The only problem is that the anticipated synergy doesn't emerge; the weighted grade (sans Jimenez) of this Top 40 list is 3.64, which is lower than any of the three lists on their own!

So the lesson to take from all this is that there are no lessons. There are no hard and fast rules. There is not even an advantage to forming a consensus opinion. In short, the task of prognosticating minor leaguers is both a science and an art, and the most important step any evaluator needs to take is to recognize the inherent limitations in predicting the future. God is omnipotent; the rest of us just have to do the best that we can with the information that's available, and recognize that even the surest of prospects--like Nick Johnson--is no sure thing.

Rany Jazayerli is an author of Baseball Prospectus. Contact him by clicking here.

Rany Jazayerli is an author of Baseball Prospectus. 
Click here to see Rany's other articles. You can contact Rany by clicking here

0 comments have been left for this article.

No Previous Article
No Next Article

RECENTLY AT BASEBALL PROSPECTUS
Fantasy Article Fantasy Freestyle: Offseason Trades and Resp...
Fantasy Article Fantasy Freestyle: 40-Man Additions to Know
Premium Article Rumor Roundup: Revenge is a Dish Best Served...
Premium Article Baseball Therapy: The Timeshare DH
Premium Article Transaction Analysis: Mariners Eager To Lock...
Premium Article Transaction Analysis: Red Sox Do Whatever Th...
Before They Were Prospects

MORE FROM APRIL 18, 2001
Transaction Analysis: April 16-17, 2001
The Daily Prospectus: Even More on Park Fact...

MORE BY RANY JAZAYERLI
2001-05-23 - Doctoring The Numbers: Alex Gonzalez
2001-05-18 - Doctoring Ichiro
2001-05-16 - Doctoring The Numbers: Ballardesque Strikeou...
2001-04-18 - Top 40 Prospects In Review: Part Seven
2001-04-11 - Top 40 Prospects In Review: Part Six
2001-04-03 - Top 40 Prospects In Review: Part Five
2001-03-27 - Top 40 Prospects In Review: Part Four
More...