CSS Button No Image Css3Menu.com

Baseball Prospectus home
  
  
Click here to log in Click here for forgotten password Click here to subscribe

<< Previous Article
Premium Article What You Need to Know:... (05/27)
<< Previous Column
Premium Article Baseball Therapy: Bewa... (05/20)
Next Column >>
Premium Article Baseball Therapy: The ... (05/30)
Next Article >>
Painting the Black: Re... (05/27)

May 27, 2014

Baseball Therapy

The Annual Amateur Draft Guessing Game

by Russell A. Carleton


Next week is going to be a big one for all 30 teams in Major League Baseball. It’s draft week! The Rule 4 Draft (which is the fancy name for the amateur draft) will take place from June 5th through June 7th. There will be pageantry (which is a fancy name for people trying to make a boring administrative event into a less-boring administrative event). There will be Hall of Famers representing teams. And the end result of a year of hard work by your favorite team’s scouting staff will come to fruition in the form of 30 teams making a bunch of wild guesses.

Every general manager has to deal with at least one “you could have had this guy, but you drafted the other guy and he never made it!” complaint. I think they all involve the 2009 Draft and Mike Trout. The draft is an inexact science. Unlike the NFL and NBA, where draftees are put directly into the starting lineup, it’s going to be a while before a team sees the fruits, whether luscious or rotten, of their draft. You have to project what a guy who only recently attained the right to vote will look like at age 27. Every year, there are can’t-miss prospects who end up missing and “he’s a nice org guy” picks who turn into really good players.

But how good are teams at predicting the future? All 30 teams have a scouting department filled with people who are experts at evaluating amateur talent, many whom measure their experience in decades. They get access to all sorts of extra information that is not public. They have cross-checkers and big secret meetings. They have every incentive to get this right because, at the end of the day, someone will be cutting checks with a lot of zeroes in them. They’d better get it right.

In a perfect world, the team that picks first should take the player who will eventually provide the most major-league value. The team picking second should take the guy who will provide the second-most value. But then, in the past, the draft was a game that was half proper draft and half auction. Teams with high picks would pass on players whom they believed to be more talented, but whom they saw as wanting too much money. For example, in 2001, the Minnesota Twins drafted Joe Mauer no. 1 overall and gave him a signing bonus of $4 million. The no. 2 overall pick was Mark Prior, who actually got more money ($4.6 million), as did no. 4 pick Gavin Floyd at $4.2 million. The Tampa Bay (Devil) Rays got Dewon Brazelton for the bargain price of $2.5 million at no. 3. (I guess you get what you pay for.) Then again, for that same $2.5M figure, the Rangers got Mark Teixeira two spots later. The market might be efficient, but the shoppers might be fools.

It’s been said that if you want to know how much a team really values a draft pick, don’t look at his overall position, but instead look at his signing bonus. Like everything else in life, follow the money. But if that price tag is the end result of months of research and high-level discussion among the best experts available, how good is that process?

Warning! Gory Mathematical Details Ahead!
I obtained data on signing bonuses here. Data were fairly complete beginning in the year 2003, and there were generally data for the first 10 rounds. I examined how well teams performed in the 2003 to 2008 drafts, as the players picked in later drafts haven’t yet had a full chance to develop and to make their major-league selves known yet. I got career data from Baseball-Reference, who had their draft data set up so nicely for my purposes.

The rest of this article is restricted to Baseball Prospectus Subscribers.

Not a subscriber?

Click here for more information on Baseball Prospectus subscriptions or use the buttons to the right to subscribe and get access to the best baseball content on the web.


Cancel anytime.


That's a 33% savings over the monthly price!


That's a 33% savings over the monthly price!

Already a subscriber? Click here and use the blue login bar to log in.

Related Content:  Scouting,  Drafting,  Amateur Draft,  Rule 4 Draft

13 comments have been left for this article. (Click to hide comments)

BP Comment Quick Links

Behemoth

Just a thought, but would the high school versus college comparisons not be impacted by the fact that high school players will have had less time to accumulate WAR. For example, a high school pick from 2008 might reasonably have reached the bigs in 2012, and would have to have done pretty well from the start to have reached 5WAR by now. A college guy might have reached MLB in 2010, and would have that much more time to start WAR collecting.

May 27, 2014 05:28 AM
rating: 2
 
BP staff member Russell A. Carleton
BP staff

In those comparison, I'm comparing HS to HS and college to college. So, there will be some 2003 HS grads and 2008 HS grads in there, same as there will be 2003 college and 2008 college grads. Sure, there's bias in that a 2003 HS grad would be 29 now, while a 2008 HS grad would only be 24. It's why I sliced things a few other ways. What amazed me was that the correlations stayed so consistent. And moderate.

May 27, 2014 05:57 AM
 
Geoff Young

"The market might be efficient, but the shoppers might be fools." This might be my new favorite phrase.

May 27, 2014 08:48 AM
rating: 2
 
BP staff member Russell A. Carleton
BP staff

I was happy when that one popped into my head.

May 27, 2014 10:13 AM
 
J. Eldred

Particularly as (a fellow) and long suffering Friars fan.

May 27, 2014 10:29 AM
rating: 1
 
MGL

"In 2013, the correlation between salary and WAR among those making more than $1,000,000 was .23. I played around with various filters, but the result was always similar. So, from that point of view, teams are doing a better job of understanding the market for wins that prospects will eventually put up (years in the future) than they are of understanding market for actual MLB free agents."

In one, you are using career WAR and in the other (2013 salary and WAR), you are using one-year WAR, so of course the correlation for the one-year WAR is going to be less. That does not mean that teams are doing a better job in the draft than in valuing FA.

Back to the article in general:

When you look at individual rounds and find virtually no correlation after the first round, all that tells us is that teams cannot distinguish among the talent in each of those rounds, not that they are necessarily doing a bad job in drafting in those rounds.

For example, let's say that all players in the first round accumulated an average of 20 career WAR and all players in the second round accumulated an average of 15 career WAR. That might be a good result, I don't know. Let's say that it is. Well, the correlation of zero (or negative) in the second round simply means that teams cannot distinguish between one 15 WAR player and another, which doesn't mean that they are doing a bad job at drafting in the second round. Maybe all the talent is bunched together after the first round?

In fact, if the talent is normally distributed, then there are more and more players bunched together in talent after each round so that it becomes harder and harder for teams to distinguish among all that talent PLUS the actual spread of talent between the 30 players in each subsequent round is getting smaller and smaller so we expect that the correlation between WAR and signing bonus will also get smaller and smaller.

Even in the first round, the relatively small correlation doesn't real tell us much. What if teams are very good at identifying the best 30 or 40 players, but they are not so good at distinguishing among them? And again, what if the talent even in the first round does not have such a large spread?

All the correlations in each round tells us, from a team perspective, is that it doesn't really matter that much what position in each round you select from and you are probably better off with a lower number especially after the first round since you can save yourself some money in signing bonuses.

Also, isn't there a lot of noise in the signing bonuses depending on the player's agent, his family's financial situation, his personal preferences, etc.? Why wouldn't draft pick number be a better proxy for evaluating talent? with a few exceptions, aren't teams trying to pick the best available talent for each pick, regardless of what the eventual signing bonus is?

May 27, 2014 09:07 AM
rating: 1
 
BP staff member Russell A. Carleton
BP staff

I threw in the single year (2013) MLB salary stuff as a quick benchmark. The message there is that the correlation ain't .80 even when we have MLB data to work from and we're not projecting into the future.

I actually ran just about everything in here with draft position as the predictor. Signing bonus was a better predictor consistently. These drafts (03-08) are back in the era before slotting (or even "slotting") and teams made "signability" picks in the top ten, so there were likely players whom the teams believed weren't as good who got picked ahead of better players who wanted bigger bonuses.

It's true that the talent may bunch in different ways, but in an efficient market, where everyone has perfect information (obviously not actually the case), the signing bonuses should sort that out as well. What strikes me about all of this is that the market is a long way away from efficient, which I take as "prospectin' is hard."

May 27, 2014 10:13 AM
 
MGL

Thank you for the explanation about why signing bonus is a better predictor than actual pick number, at least before "slotting." It makes sense.

" The message there is that the correlation ain't .80 even when we have MLB data to work from and we're not projecting into the future."

It can't be that high even with perfect information, because there is too much random variance in one season of performance. Remember that even with perfect information (if we knew every player's exact true talent and payed them fair market value for that), the smaller the time frame, the lower the correlation because correlation includes random variance. That was MY point. Comparing correlations tells you nothing about how much information you have or how efficient you are at evaluating talent, when one correlation is based on one year of performance and the other is based on a career (or at least several years).

When I worked for the Cardinals in 2004, I studied the draft. I found that after the first round, using nothing but college MLE's would do better than the actual draft. In other words, after the first round, scouting is very inefficient, as your analysis suggests. Of course, drafting is probably better now than it was 10 years ago, as many teams are using sophisticated analytics to project ML performance from high school and college performance. There is also more emphasis on defense and positional adjustments now then there used to be. In the old days, teams would much rather draft the slugging outfielder or first baseman than the mediocre hitting excellent SS. Not so anymore, or at least it shouldn't be, although I suspect that most teams still overvalue hitting at the amateur level at the expense of defense and positional adjustments (and probably base running too). Everyone wants the next great hitter and not the next great fielder.

May 27, 2014 18:42 PM
rating: 4
 
mitchiapet

Question in regards to the Gory Math: Can you provide the c statistics for the logistic regressions you ran?

May 27, 2014 09:22 AM
rating: 0
 
jfranco77

Isn't there a little bit of bias built in? Teams are more likely to give their first rounder some time in the majors to justify their investment to the fans. I don't know how to account for that but I suspect it is there somewhere.

May 27, 2014 09:34 AM
rating: 1
 
BP staff member Russell A. Carleton
BP staff

Yeah, that's probably in there somewhere. I think there's been explicit work done on this in the NBA. It's why I also considered career WAR so that we had something to balance against that threat.

May 27, 2014 09:56 AM
 
schlicht

Are there enough data to compare teams' performances?

May 27, 2014 14:42 PM
rating: 1
 
doncoffin
(422)

This comes up (implicitly) right near the end of the current batch of comments. But which major sport does *best* in their drafts--MLB, NFL, or NBA? I would guess the NBA does, but that's all it is--a guess.

May 29, 2014 17:37 PM
rating: 0
 
You must be a Premium subscriber to post a comment.
Not a subscriber? Sign up today!
<< Previous Article
Premium Article What You Need to Know:... (05/27)
<< Previous Column
Premium Article Baseball Therapy: Bewa... (05/20)
Next Column >>
Premium Article Baseball Therapy: The ... (05/30)
Next Article >>
Painting the Black: Re... (05/27)

RECENTLY AT BASEBALL PROSPECTUS
Premium Article Painting the Black: Pedro Alvarez's Trip Fro...
Premium Article Minor League Update: Games of Thursday, Apri...
Premium Article What You Need to Know: April 17, 2015
Going Yard: How Bryant Can Crush
The Call-Up: Kris Bryant
Premium Article Rubbing Mud: So You Want to Trade Your Draft...
BP Boston

MORE FROM MAY 27, 2014
Premium Article Perfect Game Presents: 2014 MLB Mock Draft, ...
This is Not Your Father's Baseball Road Trip...
Premium Article Minor League Update: Games of May 23-26
The Week in Quotes: May 19-26
Fantasy Article Dynasty Dynamics: The U25 Top 150, Part Thre...
Fantasy Article Interleague Report: Week Nine
Daily League Strategy: Get Your Phil

MORE BY RUSSELL A. CARLETON
2014-06-10 - Premium Article Baseball Therapy: Can Draft Lightning Be Bot...
2014-06-03 - Premium Article Baseball Therapy: Introducing My Daughter to...
2014-05-30 - Premium Article Baseball Therapy: The Hard Part About Preven...
2014-05-27 - Premium Article Baseball Therapy: The Annual Amateur Draft G...
2014-05-20 - Premium Article Baseball Therapy: Beware the Genius Tag for ...
2014-05-13 - Baseball Therapy: Analytical Master or Leade...
2014-05-07 - BP Unfiltered: Baseball Prospectus at Fox Sp...
More...

MORE BASEBALL THERAPY
2014-06-10 - Premium Article Baseball Therapy: Can Draft Lightning Be Bot...
2014-06-03 - Premium Article Baseball Therapy: Introducing My Daughter to...
2014-05-30 - Premium Article Baseball Therapy: The Hard Part About Preven...
2014-05-27 - Premium Article Baseball Therapy: The Annual Amateur Draft G...
2014-05-20 - Premium Article Baseball Therapy: Beware the Genius Tag for ...
2014-05-13 - Baseball Therapy: Analytical Master or Leade...
2014-05-06 - Baseball Therapy: Is Oakland's Run Different...
More...

INCOMING ARTICLE LINKS
2014-10-08 - BP Unfiltered: The Cardinals' Flickering Mag...
2014-07-15 - Premium Article Baseball Therapy: Why Are We Playing Hunger ...
2014-06-24 - Premium Article Baseball Therapy: Is it Really Harder to Sco...
2014-06-10 - Premium Article Baseball Therapy: Can Draft Lightning Be Bot...