If you’re the sort of fan with only enough mental storage space to keep track of the top 101 or so prospects, you may officially quit paying attention to Mike Olt, Trevor Story, Rymer Liriano, Kaleb Cowart, Taylor Guerrieri, Bubba Starling, Danny Hultzen, Mason Williams, Luis Heredia, Arodys Vizcaino, J.R. Graham, Alen Hanson, Allen Webster, Jake Marisnick, Justin Nicolino, Hak-Ju Lee, Casey Kelly, Tyrell Jenkins, Clayton Blackburn, Dorssys Paulino, Sean Nolin, Michael Fulmer, Lewis Brinson, Delino DeShields. All were top 101 prospects last year, but things happen to prospects (they break your heart) and none is this year. You can just rewrite over those brain cells.

If you’re the sort of fan, though, who keeps a few post-hype sleepers in the back of your mind, those are your boys. Each has already demonstrated some combination of tools and traits—and, usually, performance—that was worth getting excited about, so, you figure, even after a bad year, or a bad injury, there’s at least something exciting buried under each player’s recent disappointment. To the collector of post-hype sleepers, the only thing better than a prospect is a guy who used to be a prospect.

So what this will attempt to answer is whether that’s true. Or, because it’s the question that got me wondering: Is Gary Brown still worth paying attention to?

Brown, you’ll recall, was a top 101 prospect two years ago, and after a torrid season in High-A we considered his arrival imminent: in a perfect world “he’s a dynamic, All-Star center fielder,” according Kevin Goldstein at the time. A year later, he was off the top 101—though still one of the top prospects in a (relatively weak) Giants system. Jason Parks’ write-up of him that year: “The adjustment issue and lack of growth is a red flag, but the overall athleticism and glove-work give him a high floor. Even if the bat is only average, the defensive profile will have use on a major-league bench (eventually).” This year, Brown tumbled further, off the Giants’ top 10 entirely. Parks again: “It’s highly unlikely that his bat improves to the point where it plays as a major-league regular. But the defensive profile and speed should be enough to carry him to the highest level… however, when you have the physical tools of Brown—a former first round pick—the bar for success is set much higher than most players.”

So, Gary Brown. Worth the brain cells or not?

In 2007, Kevin Goldstein wrote his first top 101 for this site. In 2008, he wrote his second, predictably. Twenty-six players disappeared in the meantime, not counting players who became ineligible for prospect lists due to service time. The 26:

Jeff Niemann, Philip Humber, Brandon Erbe, Chuck Lofgren, Donnie Veal, Billy Rowell, Troy Patton, Trevor Crowe, Sean West, Humberto Sanchez, Ryan Tucker, Ryan Sweeney, Will Inman, Matt Harrison, Glen Perkins, Sean Gallagher, Brad Lincoln, Sean Rodriguez, Cesar Carrillo, Chris Parmelee, Dellin Betances, Brett Sinkbeil, Cedric Hunter, Pedro Beato, Javier Herrera, Kyle Drabek.

What do you think of that list? Nobody on there who is a star; one closer (Perkins) and one Opening Day starter (Harrison), some solid role players, but if you’d ignored these 26 in your keeper league you’d be okay right now. Of the 26, only 10 have produced a positive career WARP (two of them with just 0.1). Nine are inactive, just six years later. Overall, they produced a cumulative 28.3 WARP, about one win per player. Ryan Sweeney, by WARP, has had the best career in the group, at 8.3. Ryan Sweeney, by WARP, has had the best career in the group. Ryan Sweeney, by WARP, has had the best…

On the other hand, relative to the nearly 6,000 minor leaguers outside the top 101, these guys did pretty well. The point of this exercise is to figure out whether players who drop on prospect lists are underrated assets, but to know whether something is underrated you have to know how he is rated. And once a player drops off the top 101, it’s hard for the uninformed among us to know exactly how he was rated—was he the 102nd best prospect in the game by consensus, or the 1,500th best?

So step away from the guys who fell off the list entirely and look at players who merely dropped lower on the list—we still know, after all, how they were rated. I divided the list into tiers, admittedly arbitrary tiers but something to let me clearly define whether a player truly dropped in the eyes of the assessor or was merely shuffled about. Tiers:

  • 1 to 5
  • 6 to 15
  • 16 to 24
  • 25 to 39
  • 40 to 59
  • 60 to 79
  • 80 to 101

And I looked only for players who dropped two tiers or more. (Going from 15 to 16 would drop the player a tier but wouldn’t signify any real change. Hence two tiers or more.) In Goldstein’s first two lists, the following players dropped two tiers or more:

You’ll note that a lot of these players, save Fowler and Gomez, have something in common: disappointment. But are they disappointing relative to the other bottom-half prospects they were ranked alongside? Let’s compare their career WARPs to the prospects who were most similarly ranked in 2008, the post-drop year for each of these players. Pitchers are compared to pitchers only; hitters to hitters. Each is compared to the two prospects directly above him and the two directly below:

Player Career WARP Comparable prospects (average)
Wood (no. 5 to no. 38) -3.5 6.6
Brignac (no. 11 to no. 25) -0.5 14.9
Martinez (no. 18 to no. 51) -0.4 2.4
Adam MIller (no. 19 to no. 52) 0.0 7.1
Luke Hochevar (no. 23 to no. 72) 5.6 0.7
Scott Elbert (no. 32 to no. 66) 0.0 3.1
Carlos Gomez (no. 34 to no. 65) 13.0 2.5
Carlos Carrasco (no. 37 to no. 68) 0.8 1.9
Dexter Fowler (no. 39 to no. 92) 11.5 2.4
Total 26.5 41.6

On average, the prospect who had dropped produced about two fewer WARP over the next six years than he “should” have produced based on his new prospect ranking. There is so much noise here, though, so let’s try another year. Moving further back, and switching to Baseball America rankings, we’ll repeat with three more seasons’ dropped prospects:

2003 to 2004 fallers: Jason Stokes, Adam Wainwright, Brad Nelson, Sean Burnett, Jose Lopez, Bryan Bullington, Mike Jones.
Produced: 4.7 WARP per player
Comparable prospects: 5.7 WARP per player

2000 to 2001 fallers: Dee Brown, Chin-Feng Chen, Michael Cuddyer, Eric Munson, Abraham Nunez, Danys Baez.
Produced: 2.6 WARP per player
Comparable prospects: 8.9 WARP per player

1995 to 1996 fallers: Ben Grieve, Todd Hollandsworth, Dustin Hermanson, Doug Million, Josh Booty, LaTroy Hawkins, Julio Santana, Trey Beamon, John Wasdin
Produced: 4.1 WARP per player
Comparable prospects: 11.1 WARP per player

There’s nothing conclusive about this very imprecise and very arbitrary way of looking at things, but what we can say is this: In all four years I looked at, the players who dropped down prospect lists produced fewer WARP than similar ranked players. In the four years overall, the difference is large—falling prospects produce around half of what non-falling prospects do. (You might keep that in mind for this year's two-tier fallers: Billy Hamilton, ​​Travis d'Arnaud, Matt Barnes, David Dahl, Gary Sanchez.) If we were to hypothesize anything from this, for further research, it is that prospects who fall are actually overrated, not underrated.

In which case, the idea of the post-hype sleeper becomes less compelling overall. That’s not to say everybody in the opening paragraph is equally hopeless; Jake Marisnick might still have a case to be on a top 101 prospect list, while Mike Olt took the Gary Brown path all the way off his team’s top 10. Jason Parks frequently notes that prospect rankings are a snapshot of a particular moment in time. Snapshots are amazing: they stop the world for an instant and let us study them in detail, to really explore a scene in a way that we never can when life is moving forward, as it does. Snapshots give us insight that we could otherwise never have. If there is one thing a snapshot struggles to depict, though, it’s momentum. Is this man walking backward, forward, or standing still? It’s hard to know. Two snapshots, though. Two snapshots often do the trick. When we’re talking about prospect rankings, the two snapshots might be telling us something about the third.

Thank you for reading

This is a free article. If you enjoyed it, consider subscribing to Baseball Prospectus. Subscriptions support ongoing public baseball research and analysis in an increasingly proprietary environment.

Subscribe now
You need to be logged in to comment. Login or Subscribe
In your original list of prospects not on top 101 lists for this year (at the top of this page), you list Kyle Crick. Is that a mistake? He's #38 on BP's list this year.
Great article. I've always felt that people are too conservative in upgrading and downgrading prospects. Nice to see some numbers behind it. Hope you can do something similar for "rising" prospects.
This is very interesting. Baseball writers and institution have long been compiling archives of knowledge, and this is a great example of putting them to good use.

I'm curious about David Dahl, and because of him injuries in general. He's still nineteen and only played ten games last year due to a torn hamstring. Were any of the other fallers similarly hurt by being...hurt? The article implies performance.
Adam Miller offhand fell completely due to injuries. That 0.0 WARP overstates his value; he never made the majors.
Interesting . . . could be on to something, but my instinct tells me we need a fair bit more data to be conclusive.

But then, Jason Parks and Baseball America could be reading this and make bolder adjustments next year. . . How would we know?

Anyway, for those of us who don't blindly go by prospect rankings, this was a useful article. Thanks.
I recall the "endowment effect" from psych 101. Experiment: professor randomly hands out either a mug or $1 to each student. On the way out the door, the students get a chance to exchange one for the other. But regardless of mug or $1, the students will have a strong preference to hold on to what they got.

I bet there's an endowment effect with prospects. Once you rank them, you "own" them, overvalue them, and it's hard to let them go. Good data in this article supporting it.
The endowment effect is cited often in fantasy circles. It is the only possible explanation I can think of as to why someone in one of my leagues is holding on to a six dollar Dee Gordon.
This is also why I am still holding on to Gary Brown on my reserve for the third year. Well, maybe not this year. :)
Who wouldn't rather have a mug than a dollar? At least you weren't studying the Kitty Genovese effect. There must be a baseball corollary. The Marlins maybe
Good news! Through the magic of thrift stores, you can exchange your dollars for mugs to your heart's content!
Bogaerts is a prime example. People were very slow to upgrade, despite the fact that he was putting up a historic age-adjusted year in single-A.
Great article. I would not have predicted this. Somehow, I'd have guessed there would be "regression to the mean" of the previous and current rankings.
Sam, great work as usual. This is very interesting. I wonder if it might be useful to look at this in a slightly different way. Instead of looking at the average of the comp groups as whole, maybe it would be more informative to analyze the comp groups to find the expected likelihood of reaching certain levels of achievement.

For example, Brandon Wood has obviously been a huge disappointment with -3.5 WARP vs an average of 6.6 in his comp group. But looking at that group in more detail there's a huge range of performance hiding behind the 6.6 WARP average.

Group 1: Cargo 18, Heyward 15.3,
Group 2: Soto 12, Kennedy 6.8, Feliz 3
Group 3: Detwiler 1.9
Group X: Nick Adenhart .1
Group 4: Villalona, Antonelli, Brignac, Hu, Marrero, Clement, LaPorta; -6.3 as a group

Group 4 makes up half of the comp group. So rather than seeing it as Wood grossly underperformed the group, we might instead say Wood had a 50% chance of turning out just like he did. I wonder if expanding this through the entire sample might reduce or even eliminate the disparity between the fallers and the comps, especially if an increased sample size finds even just a small handful of fallers that went on to find some moderate MLB success.
I wonder what a look at median WARPs for nearby prospects might show rather than averages, though I'd expect the trend to be similar but far less pronounced. And likely would require a larger comparative sample.

A problem with Average WARP I've seen in a lot of analysis is that the average is heavily skewed by a few values.

In your samples above, imagine how bad the decliners would look were it not for Gomez and Fowler delivering over 90% of the group WARP? So it cuts both ways.

Fundamentally - and you are obviously aware of this - it is the problem with small sample averages. And this comparison will inherently be based on small samples.

But this is very interesting anyway, and may be as fair an effort as is practicable. The overall impression seems more than substantiated looking at the broad set of "decliners" though.