keyboard_arrow_uptop

Part One
Part Two
Part Three
Part Four
Part Five
Part Six

First, let’s recap the complete Top 40 lists for each publication, along
with the grade for each player:


Baseball Prospectus

Baseball America

John Sickels

1. Nick Johnson 1

1. Rick Ankiel 6

1. Corey Patterson 5

2. Rick Ankiel 6

2. Pat Burrell 5

2. Rick Ankiel 6

3. Pat Burrell 5

3. Corey Patterson 5

3. Pat Burrell 5

4. Sean Burroughs 4

4. Vernon Wells 3

4. Vernon Wells 3

5. Vernon Wells 3

5. Nick Johnson 1

5. Nick Johnson 1

6. Rafael Furcal 7

6. Ruben Mateo 4

6. Dee Brown 3

7. Ruben Mateo 4

7. Sean Burroughs 4

7. Kip Wells 2

8. Ben Petrick 5

8. Rafael Furcal 7

8. Ben Petrick 5

9. Corey Patterson 5

9. Ryan Anderson 5

9. Sean Burroughs 4

10. Michael Cuddyer 3

10. John Patterson 1

10. Michael Cuddyer 3

11. Dee Brown 3

11. Dee Brown 3

11. D’Angelo Jimenez INC

12. Kip Wells 2

12. Mark Mulder 4

12. Ruben Mateo 4

13. Jack Cust 3

13. Josh Hamilton 5

13. Brad Penny 5

14. Tony Armas Jr. 5

14. Kip Wells 2

14. Chin-Feng Chen 2

15. D’Angelo Jimenez INC

15. Matt Riley 1

15. Rafael Furcal 7

16. Peter Bergeron 3

16. Alfonso Soriano 3*

16. Eric Gagne 3

17. Chin-Feng Chen 2

17. Chin-Feng Chen 2

17. Matt Riley 1

18. Esteban German 2

18. Michael Cuddyer 3

18. Hee Seop Choi 6

19. Matt Riley 1

19. Josh Beckett 4

19. Wilfredo Rodriguez 2

20. Lance Berkman 6

20. A.J. Burnett 5

20. Ed Yarnall 2

21. Brad Penny 5

21. Travis Dawkins 2

21. Ramon Ortiz 5

22. Milton Bradley 4

22. Brad Penny 5

22. Ryan Anderson 5

23. Chad Hermansen 1

23. Eric Munson 3

23. Alfonso Soriano 3*

24. Ryan Anderson 5

24. Drew Henson 4

24. John Patterson 1

25. Eric Gagne 3

25. Wilfredo Rodriguez 2

25. Milton Bradley 4

26. Aubrey Huff 6

26. Michael Restovich 2

26. Jesus Colome 5

27. Matt LeCroy 2

27. Tony Armas Jr. 5

27. Ramon Hernandez NA

28. Ed Yarnall 2

28. Ramon Ortiz 5

28. Peter Bergeron 3

29. Marcus Giles 4

29. Francisco Cordero 2

29. Abraham Nunez 3

30. Michael Restovich 2

30. Abraham Nunez 3

30. Matt LeCroy 2

31. Jon Garland 6

31. Jack Cust 3

31. Mike Lamb 3

32. Adam Piatt 5

32. Jon Garland 6

32. Josh Hamilton 5

33. Alfonso Soriano 3*

33. Chad Hermansen 1

33. Tony Armas Jr. 5

34. Mike Meyers 3

34. Alex Escobar 6

34. Aaron Myette 3

35. Jayson Werth 2

35. Ben Petrick 5

35. Jack Cust 3

36. Jason Romano 3

36. Milton Bradley 4

36. Francisco Cordero 2

37. Ramon Ortiz 5

37. Lance Berkman 6

37. A.J. Burnett 5

38. Dernell Stenson 3

38. Felipe Lopez 4

38. Luis Rivera 2

39. Travis Dawkins 2

39. Danys Baez 3

39. Junior Guerrero 2

40. Mario Encarnacion 2

40. Chris George 5

40. Drew Henson 4

You’ll note that Alfonso Soriano has an asterisk next to his grade.
We originally awarded Soriano a grade 2 when we first started this
retrospective, back in February, and events since then would suggest that
perhaps Soriano’s stock hasn’t fallen as much as I had thought. I’m still
not a believer in his long-term future, but a grade 3 now seems more
appropriate.

Let’s go ahead and get the rankings out of the way. Using a weighted
average (the grade of the #1 prospect is worth 40 points, the #2 prospect
39 points, etc., with the #40 prospect just 1 point), here is the average
score for each publication:


Publication

Average

Baseball Prospectus

3.72

Baseball America

3.69

John Sickels

3.67

There hasn’t been a race that close since the 1973 NL East. While we’d like
to claim victory by a whisker, the race is so close that if we give
D’Angelo Jimenez a grade 1 instead of an incomplete, and give
Ramon Hernandez a grade 4, the scores would be:


Publication

Average

Baseball America

3.69

John Sickels

3.638

Baseball Prospectus

3.637

The point is not simply to prove who’s the best at making predictions–if
it was, I could have left Soriano’s grade as a 2 and engineered a clear BP
victory! It should be pretty clear that this was damn well near a dead
heat, and none of the publications can claim a better prospect list than
the others. In itself, this is very revealing. It appears that traditional
scouting methods do not evaluate prospects any more accurately than newer
sabermetric methods. That doesn’t mean that scouts are useless; it does
mean that objective analysis has as much a place in the evaluation of
prospects as the scouts do.

The scores hover around 3.7, which (given that a grade 4 is our equivalent
of the .500 pitcher) suggests that none of us did a particularly good job
ranking prospects last year. That only adds to the argument that last year
was a very tough year in which to evaluate prospects. As this is the first
time we’ve evaluated top prospects lists in this way, we can’t tell if it
was just a bad year or if projecting prospects is simply more difficult
than we thought. Hopefully, we’ll have our answer next season when we
evaluate our prospect lists again.

Let’s dig a little deeper. It’s no secret that we are much more
conservative when it comes to evaluating pitching prospects than their
hitting brethren. This next chart illustrates how much more conservative we
are compared to Baseball America and Sickels.


Publication

Hitters

Points

Pitchers

Points

Baseball America

24

506

16

314

John Sickels

23

527

17

293

Baseball Prospectus

29

616

11

204

Only 11 of the 40 players on our Top 40 list were pitchers, and the number
of points awarded for pitchers on our list is less than 25% of the total,
compared to 38% for BA and 36% for Sickels.

Is that conservatism warranted? Let’s split up the pitchers and hitters on
each list, and compare what the weighted grades for each group were:


Publication

Grade (Pitchers)

Grade (Hitters)

Baseball America

3.73

3.66

John Sickels

3.46

3.79

Baseball Prospectus

3.97

3.64

This data is not particularly compelling, in that the worst projections for
pitchers–and the best projections for hitters–came from John Sickels,
whose philosophy on pitchers vs. hitters takes the middle ground between us
and Baseball America. However, you’ll note that the best grade by
any publication in either category was for our projection of pitchers. We
may be biased, but we’ll take that to mean that by restricting inclusion on
our list to only the very finest pitching prospects in the land, we can
minimize the danger that we’ll get burned by an injury to a John
Patterson
or make a mistake projecting an A-ball hurler like
Wilfredo Rodriguez or believe in the myth of the minor-league closer
with a Francisco Cordero.

The extra hitters that stock our list did not drag our grade for hitters
down by much; our grade for hitters was just two-hundreths of a point lower
than that of Baseball America, even while our grade for pitchers was
nearly a quarter-point higher.

It’s possible we swung too far the other way, however. If you look at just
the bottom 10 prospects on our list, three of them are pitchers (Jon
Garland
, Mike Meyers, and Ramon Ortiz), who had
respective scores of 6, 3, and 5. Of the seven hitters in our bottom 10,
only one (Adam Piatt) had a grade higher than 3. It’s reasonable to
suggest that some of those extra hitters at the bottom of our Top 40 might
be less deserving than the two or three best pitching prospects who didn’t
make the list.

If that’s the case, hopefully we found a better balance this year. The Top
40 list in Baseball Prospectus 2001 includes 13 pitchers (up from
11) totalling 275 points (up from 204). Then again, 40 of those points were
wasted on Ryan Anderson.

Let’s look at how each publication fared with their "reaches,"
those players who didn’t appear on either of the others’ Top 40 lists. Both
BA and Sickels had seven players unique to their Top 40 list, although
Sickels only had six when you eliminate Ramon Hernandez (who wasn’t a
rookie last season, and so wasn’t eligible for any other list). We had nine
unique players, which isn’t surprising given the lip service we sometimes
pay to conventional wisdom. Here are those players:


Baseball Prospectus

Baseball America

John Sickels


18. Esteban German 2

12. Mark Mulder 4

18. Hee Seop Choi 6

26. Aubrey Huff 6

19. Josh Beckett 4

26. Jesus Colome 5

29. Marcus Giles 4

23. Eric Munson 3

31. Mike Lamb 3

32. Adam Piatt 5

34. Alex Escobar 6

34. Aaron Myette 3

34. Mike Meyers 3

38. Felipe Lopez 4

38. Luis Rivera 2

35. Jayson Werth 2

39. Danys Baez 3

39. Junior Guerrero 2

36. Jason Romano 3

40. Chris George 5

38. Dernell Stenson 3

40. Mario Encarnacion 2

Average: 3.33

Average: 4.14

Average: 3.50

Weighted Average: 3.56

Weighted Average: 3.94

Weighted Average: 4.57

It’s clear that we did very poorly here. Six of our nine unique players,
including all five ranked #34 or lower, scored a 3 or less. By comparison,
Baseball America did very well with their unique players, and
Sickels–thanks to big hits with Hee Seop Choi and Jesus
Colome
–had the highest weighted average in the group.

You’ll also note that almost all of the poor grades in this category went
to players at the bottom of the lists. Players that one organization gave a
mention to at the bottom of their Top 40 list tended to crash and burn. By
comparison, of the eight unique players who were ranked higher than
#30–players that one publication was very optimistic about despite the
lack of attention from the others–only two of them had grades below 4,
Esteban German and Eric Munson.

So what’s the lesson for the reader to take from this? Well, you could sum
up the "points" for each player on all three lists and sum them together,
creating a composite Top 40 list from the three publications. Such a list
would look like this:


1. Rick Ankiel

6

2. Pat Burrell

5

3. Nick Johnson

1

4. Corey Patterson

5

4. Vernon Wells

3

6. Sean Burroughs

4

7. Ruben Mateo

4

8. Dee Brown

3

9. Rafael Furcal

7

10. Kip Wells

2

11. Michael Cuddyer

3

12. Chin-Feng Chen

2

13. Ben Petrick

5

13. Matt Riley

1

15. Ryan Anderson

5

16. Brad Penny

5

17. D’Angelo Jimenez

I

18. Alfonso Soriano

3

19. Tony Armas Jr.

5

20. John Patterson

1

21. Jack Cust

3

22. Eric Gagne

3

23. Milton Bradley

4

24. Peter Bergeron

3

24. Wilfredo Rodriguez

2

26. Ramon Ortiz

5

26. Josh Hamilton

5

28. Ed Yarnall

2

29. Mark Mulder

4

30. Michael Restovich

2

30. Chad Hermansen

1

32. A.J. Burnett

5

32. Matt LeCroy

2

32. Lance Berkman

6

35. Abraham Nunez

3

36. Esteban German

2

36. Hee Seop Choi

6

38. Josh Beckett

4

38. Travis Dawkins

2

40. Jon Garland

6

The only problem is that the anticipated synergy doesn’t emerge; the
weighted grade (sans Jimenez) of this Top 40 list is 3.64, which is lower
than any of the three lists on their own!

So the lesson to take from all this is that there are no lessons. There are
no hard and fast rules. There is not even an advantage to forming a
consensus opinion. In short, the task of prognosticating minor leaguers is
both a science and an art, and the most important step any evaluator needs
to take is to recognize the inherent limitations in predicting the future.
God is omnipotent; the rest of us just have to do the best that we can with
the information that’s available, and recognize that even the surest of
prospects–like Nick Johnson–is no sure thing.

Rany Jazayerli is an author of Baseball Prospectus. Contact him by
clicking here.

You need to be logged in to comment. Login or Subscribe