CSS Button No Image Css3Menu.com

Baseball Prospectus home
  
  
Click here to log in Click here for forgotten password Click here to subscribe

<< Previous Article
Premium Article Another Look: Bob Fell... (12/22)
<< Previous Column
Checking the Numbers: ... (12/15)
Next Column >>
Checking the Numbers: ... (01/06)
Next Article >>
Premium Article Prospectus Perspective... (12/22)

December 22, 2010

Checking the Numbers

Relievers and the Value of Perfect Information

by Eric Seidman

the archives are now free.

All Baseball Prospectus Premium and Fantasy articles more than a year old are now free as a thank you to the entire Internet for making our work possible.

Not a subscriber? Get exclusive content like this delivered hot to your inbox every weekday. Click here for more information on Baseball Prospectus subscriptions or use the buttons to the right to subscribe and get instant access to the best baseball content on the web.

Subscribe for $4.95 per month
Recurring subscription - cancel anytime.


a 33% savings over the monthly price!

Purchase a $39.95 gift subscription
a 33% savings over the monthly price!

Already a subscriber? Click here and use the blue login bar to log in.

One of the most interesting aspects of baseball transactions is that every move carries a certain level of risk. Without perfect information akin to an extreme level of accurate prognostication, teams can never be 100 percent certain how a player will perform. The best investment is a player who will produce at a very high level, but who comes as close to guaranteeing that high level of production as possible. Albert Pujols personifies this idea, as a bad season for him still involves above-average defense and a TAv north of .310. I will say right now, with as much confidence as I have in my mind and body, that Pujols will hit, at worst, .310/.370/.530 next season. He has proven himself capable of production far beyond that slash line and is as close to a sure thing as we have.

Signing Jeff Francoeur to play every day would be an example of the opposite effect, as there is ample evidence that he cannot handle such a role. We are virtually certain he cannot handle that type of role in a productive manner, so the risk associated with signing him extends beyond the normal types inherent in signing a player because of the increased level of certainty that he will not live up to expectations. The majority of players traded for or signed falls into the area in between both of these extremes, which makes evaluating transactions quite the tall, but fun, task. This is before even getting into the idea of evaluating moves on their merits at the time as opposed to retrospectively, which is a topic for another day.

Relief pitchers, however, muddy the waters, as they are as fickle in performance as teenage girls are with their crushes. In spite of this, teams often place a high premium on their services. Too often is a reliever signed to a multi-year deal off of 55 solid innings, only to prove that his success was a direct result of small sample size trickery. In other cases, the same hypothetical reliever might pitch well in the year following his signing, while his production erodes over the remainder of the deal. If signing a player who has proven himself unfit for an everyday role represents a terrible investment, signing one to a lucrative contract based on numbers derived in minute samples is not too far off.

When Scott Downs signed a three-year deal worth $15 million with the Angels, I got to thinking about how relievers signed to multi-year deals typically perform. I could at least somewhat understand it in Downs’ case, because he has performed very well for an extended period of time. When Jesse Crain and Matt Guerrier signed three-year deals, my head began to spin and I started wondering about how relievers perform over the lives of their multi-year deals. Do they live up to expectations? Does their production match, or exceed, the value of the contract? Are they signed to long-term deals so the team can reap the benefits of that first season through an added push in playoff contention?

On a more basic level, does a reliever signed to a three-year deal perform better than three relievers signed to one-year deals over the same time span? And are the relievers inked to one-year contracts given less money, on average, than those signing multi-year deals? If the answers to the last two questions are no and yes, respectively, then it would behoove teams to avoid doling out significant contracts to relievers. These are important questions to research because some teams have allocated the majority of their off-season spending to shoring up a bullpen (cough, 2007 Orioles, cough). If taking such a course of action is suboptimal in both the short- and long-term, then teams should spend their money elsewhere. However, my goal here is not to definitively answer these questions, but rather to open the discussion and explore why conducting studies along these lines are not as clear cut as they may seem.

Downs seems to be in the elite reliever mold, as his ERAs since 2007 have been 2.17, 1.78, 3.09, and 2.64. In a grand total of 236 2/3 innings over the same four-year span he has produced 8.8 WARP, a very good mark for such a small sample of production. But is he worth committing three years and $15 million to? There are a few ways to attack a question like this, but I am not interested in projecting his performance and figuring out if the translation to financial value exceeds the dollars paid out in the contract. Instead, I am looking to see how multi-year deals for relievers generally work out relative to their one-year counterparts.

To that end, I used the ESPN Free Agent Tracker and went back to the 2006 offseason, binning relievers based on whether they signed multi-year deals or deals for just one season. Multi-year relievers were recorded for the 2006 and 2007 offseasons, while I added all one-year relievers from the 2006-09 offseasons to my spreadsheet. For the multi-year relievers I recorded their WARP totals in each of the two years preceding their contract year, as well as over the life of the deal. The same process was repeated for the one-year wonders, though the lives of their deals were just one season.

We could repeat this process with WXRL, FRA, SIERA, or even ERA, but I will stick with WARP for now, as the more important idea is how to actually begin a study like this—too many articles have attempted to answer the same question, either starting with the wrong question or failing to use a control group, or even neglecting to include very important variables discussed later on. The table below shows the pertinent information for the multi-year relievers signed prior to the 2007 season, with the PY and Y columns representing WARP totals in the second preceding year, the prior year, and each of the years in the new contract. Note, too, that I excluded Mariano Rivera from the tables as he is a curve-buster.

NAME

AGE

DEAL

PY-2

PY

Y1

Y2

Y3

Danys Baez

33

3/$19

3.9

0.1

-0.1

0.0

1.7

Miguel Batista

39

3/$25

0.4

2.3

2.0

-2.3

0.7

Chad Bradford

36

3/$10.5

0.6

1.7

2.1

1.7

0.1

Alan Embree

40

2/$5.5

-1.6

0.9

1.4

0.0

N/A

Steve Kline

38

2/$3.5

0.8

0.8

0.0

0.0

N/A

Guillermo Mota

37

2/$5

0.3

1.1

0.2

0.8

N/A

Hideki Okajima

34

2/$2.5

N/A

N/A

2.5

1.9

N/A

Scott Schoeneweis

37

3/$10.8

1.6

1.3

-0.1

0.9

-0.6

Justin Speier

37

4/$18

1.8

0.9

1.5

0.2

0.4

Mike Stanton

43

2/$5.5

-0.1

2.0

0.1

0.0

N/A

Jamie Walker

39

3/$12

0.7

0.7

1.5

-0.4

0.1

David Weathers

41

2/$5

1.4

1.8

2.9

2.2

0.8

Francisco Cordero

35

4/$46

3.5

3.3

3.0

3.7

2.1

Octavio Dotel

37

2/$11

-0.3

0.7

1.2

1.3

N/A

Scott Linebrink

34

4/$19

1.8

1.0

1.1

0.5

0.8

Ron Mahay

39

2/$8

0.3

1.8

1.4

0.6

N/A

Troy Percival

41

2/$8

N/A

1.5

-0.1

-0.3

N/A

David Riske

34

3/$13

0.6

2.7

-0.2

-0.1

0.1

Luis Vizcaino

36

2/$7.5

2.3

1.3

0.4

0.0

N/A

AVERAGE

37.3

2.6/$12.4

1.1

1.4

1.1

0.6

0.6

 

Summarizing the table above, the pitchers averaged 37.3 years of age and signed deals to an average of 2.6 years and $12.4 million. In the second preceding year they averaged 1.1 WARP; in the preceding year they averaged 1.4 WARP. In the first year of their shiny new contracts, they reverted to the level of production two years before signing. And during the second and third years of their deals this group performed worse than any of the other years tabled above. The sample is small, of course, so nothing definitive can be drawn from this table. But this is a starting point.

The relievers appeared to be worth approximately $5-6 million leading into the deal and then saw their production plummet. Over the lives of the various deals, the multi-year relievers provided, on average, 2.3 wins. How did their one-year deal counterparts fare over the same time span? Well, there are far too many relievers to list in a table, so I will instead just summarize the information in the table below:

Year

Age

Deal

PY-2

PY

Y1

2007

37.3

1/$2.6

0.6

0.1

0.5

2008

36.1

1/$2.9

0.6

0.3

0.4

2009

37.3

1/$2.2

1.3

0.9

1.0

2010

35.4

1/$2.6

1.1

0.8

1.7

Average

36.6

1/$2.6

0.9

0.5

0.8

What is striking at first is how the ages are so similar across both groups. It isn’t as if the younger relievers are signing multi-year deals and the elderly are stuck going to year-to-year with teams. Additionally, the salaries go hand-in-hand with the deals, as a reliever is not going to sign a three-year deal for $5 million, just like it is somewhat rare to see a one-year deal worth $9 million or more for a reliever. The latter situation happens, but not all that often, while the former virtually never occurs in this day and age. Translating dollars to wins it appears that those signing multi-year deals during the 2006 and 2007 offseasons cost approximately $5.6 million/win in salary. The relievers signing one-year deals cost approximately $3.3 million/win in salary. And as the numbers above suggest, the relievers signed to one-year deals were not that far off of the multi-year relievers during their shorter contracts.

Given that, on average, a win is worth somewhere around $5 million per the current incarnation of MORP, multi-year relievers were slightly overpaid, while relievers signing one-year contracts were veritable bargains. However, and this is a big however, the data above is strictly salary-based. It does not include free-agent compensation for Type-A or Type-B free agents. The cost of signing a Type-A free agent is not merely the actual salary paid to the player, but that contract plus the value of the draft picks surrendered. According to this article by Matt Swartz, the cost of surrendering a first-round pick is $8 million, and the cost of giving up a second-round pick is $5 million. These values are derived from the historical value provided by players selected with those picks.

When a team signs a Type-A free agent they surrender a first-round pick, unless they finish in the bottom half of the sport, in which case they surrender a second-round pick. This is, of course, assuming that the lame-duck team offers the player arbitration. So if we have a pitcher who was offered arbitration, turned it down, was classified as a Type-A free agent, and signed a 3-year/$19 million deal with a team finishing in the upper half of the league, the new team actually spent $27 million when the cost of a surrendered first-round pick is factored into the equation. Here it becomes much more difficult to fully recoup the value of the contract with production on the field. In cases like this, teams may look to keep the reliever in the fold for an extra year or two as a means of trying to recoup more of the overall value of the contract and the compensatory picks.

If the current free-agent system receives a much-needed makeover and the compensation structure was updated, relievers would arguably benefit the most. Teams could sign relievers to one-year deals without having to give up a first-round pick, and they would not feel the need to add extra years to the deal just to potentially make up for the loss of the pick. Given that relievers are so fickle in their performance, doesn’t it make sense that the vast majority of them would not be offered multi-year deals if teams didn’t have to give anything extra up to bring them in?

There is a lot more to look into with a topic like this, but it is very important to understand that determining whether relievers are worth multi-year deals is nowhere near as easy as it might seem. Because of the structure of the free-agent compensation system, there is often a greater cost associated with signing relievers to long-term deals. Given their minute sample sizes of production it often becomes difficult to recoup the full value of the deal, which includes the value of any picks surrendered. Some teams like to make big splashes with their bullpens in the offseason, while others search the scrap heap for America’s Next Great Reliever.

Either way, it would be wise to constantly evaluate how these deals work out from an overall point of view, taking all of these factors into account, because if a strategy does not work it makes no sense to stick with it. With such a small sample it is impossible to concretely determine if multi-year relievers are poor investments, because with a larger pool of relievers the numbers may sway in the positive direction. Regardless, there are several variables that must be accounted for to even come close to accurately answering the question, and hopefully this is a good starting point. 

Eric Seidman is an author of Baseball Prospectus. 
Click here to see Eric's other articles. You can contact Eric by clicking here

Related Content:  Relievers,  A's,  Deal,  The Who,  Scott Downs,  The Process

14 comments have been left for this article. (Click to hide comments)

BP Comment Quick Links

BP staff member Eric Seidman
BP staff

Note that as of 6:50 AM EST the multi-year table is just showing the 2006 offseason guys. The 7 guys signing during the 2007 offseason will be added shortly. Apologies.

Dec 22, 2010 03:49 AM
 
BP staff member Eric Seidman
BP staff

Fixed!

Dec 22, 2010 04:02 AM
 
TangoTiger

Eric, good job! Can you tell me the total number of 1-yr relievers, and the total number of dollars paid and total number of WARP in Y1?

Dec 22, 2010 07:32 AM
rating: 0
 
BP staff member Eric Seidman
BP staff

Gracias! My spreadsheet is at home -- will take a look when I get in later tonight. There were like 45 in just 2007-08 alone, so it's probably around like 65 for 2007-10.

Dec 22, 2010 08:01 AM
 
TangoTiger

Presuming 65, and they averaged 0.8 WARP and 2.2MM$ salary above the minimum, that's 52 wins and 143MM$ above minimum. Add to that the 38 wins from the multi-year relievers and their 215MM$ above minimum gets you to 90 wins and 358MM$ above minimum. That's an average of 4.0MM$ per win.

Just last week, I posted that I estimated that the average marginal $ per marginal WARP win was 4MM$ per win.

From this standpoint, the amount of dollars allocated to relievers is just right, but that there is an obvious inefficiency between single-year and multi-year.

And this is true for all positions, not just relievers.

Dec 22, 2010 08:35 AM
rating: 0
 
Richie

Looks to me like there's enough there to conclude that multi-year contracts for relievers are a bad, bad idea.

Dec 22, 2010 09:19 AM
rating: 0
 
BP staff member Eric Seidman
BP staff

To me what it boils down to is that if it's a free agent who will cost compensation picks in return, you really need to be sure of the level of production you can get. And that is just virtually impossible with relievers. And then the guys who do have established levels of performance are going to be costly anyways and likely expensive enough to the point that value would be unlikely to be recouped.

Dec 22, 2010 10:30 AM
 
R.A.Wagman

Before judging the Angels' signing of Type A FA Scott Downs including the idea that it would cost them a pick, I thnk it should also be taken into consideration that they plan/planned to sign another, higher ranked Type A in Adrian Beltre, so that Downs would only cost them a 3rd rounder. How much would that add to the opportunity cost of the signing?

Dec 22, 2010 13:52 PM
rating: 1
 
BP staff member Eric Seidman
BP staff

Well see this is where the entire idea of evaluating these contracts gets tricky, and I'm glad you brought it up. It is just not anywhere near as easy as saying Player X produced Y WARP which is worth Z Money. He was paid B Dollars, so deal is (good/bad) depending on if B is greater or less than Z. Using the strategy you mentioned above would certainly have an impact, because the pick associated with Beltre would actually lessen that which is associated with Downs. If we say the 3rd rounder is worth around $1 million, then Downs would have a contract worth $20 million to the Angels, not the $27 million (his $19 mil plus $8 mil for the first rounder).

Dec 22, 2010 14:02 PM
 
R.A.Wagman

Even if they fail to land Beltre, the comp pick for Downs would only be a 2nd rounder at most - their 1st round pick is protected.

Dec 22, 2010 14:35 PM
rating: 0
 
BP staff member Eric Seidman
BP staff

Well Downs really isn't worth talking about all that much, though, given that he seems to be in that elite reliever mold. But if it's a 2nd rounder, we have that valued at $5 million. So the 3/$15 is 3/$20. At around $5 mil/win through MORP, Downs would need to produce four wins over the life of the contract to be a breakeven. From 2007-10 he produced 8.8 wins. Barring an injury, this should be a good deal. It's when guys like Crain sign 3-yr deals that get screwy.

Dec 22, 2010 16:09 PM
 
BP staff member Matt Swartz
BP staff

FYI When recreating MORP, I used the actual pick surrendered. The $8/5/14MM I talk about for Surrender 1st/Surrender 2nd/Resign Type A is just an approximation of the output. I literally went through each and every draft the last few years and wrote down which picks were surrendered. It was boring!

Dec 22, 2010 18:13 PM
 
BP staff member Eric Seidman
BP staff

Yeah, well you're a nerd.

Dec 22, 2010 18:14 PM
 
onegameref

I guess Kenny Williams still hasn't learned about the pitfalls of multi-year deals to relievers. How many times must a GM get burned before he finally gets the message. At least he is not alone in this affliction.

Dec 22, 2010 14:57 PM
rating: 0
 
You must be a Premium subscriber to post a comment.
Not a subscriber? Sign up today!
<< Previous Article
Premium Article Another Look: Bob Fell... (12/22)
<< Previous Column
Checking the Numbers: ... (12/15)
Next Column >>
Checking the Numbers: ... (01/06)
Next Article >>
Premium Article Prospectus Perspective... (12/22)

RECENTLY AT BASEBALL PROSPECTUS
Premium Article League Preview Series
Every Team's Moneyball: Minnesota Twins: Reb...
Premium Article Skewed Left: History Repeats Itself
Premium Article League Preview Series
Premium Article Pitching Backward: Why Relievers Get A Free ...
Premium Article Spring Training Notebook: Cactus League
Prospectus Feature: How the Astros do Spring...

MORE FROM DECEMBER 22, 2010
Future Shock: New York Mets Top 11 Prospects
Premium Article Prospectus Perspective: Athletic Ambitions
Premium Article Another Look: Bob Feller

MORE BY ERIC SEIDMAN
2011-01-06 - Checking the Numbers: The Jock Tax
2010-12-30 - Seidnotes: WARP Speed
2010-12-23 - Premium Article Seidnotes: The Rodrigo Lopez All-Stars
2010-12-22 - Premium Article Checking the Numbers: Relievers and the Valu...
2010-12-15 - Checking the Numbers: Ranking R2C2
2010-12-08 - Premium Article Checking the Numbers: A Snake, a Padre, and ...
2010-12-01 - Premium Article Checking the Numbers: Upton and Away
More...

MORE CHECKING THE NUMBERS
2011-01-26 - Checking the Numbers: Withholding Aplenty
2011-01-13 - Premium Article Checking the Numbers: The Questionable Pursu...
2011-01-06 - Checking the Numbers: The Jock Tax
2010-12-22 - Premium Article Checking the Numbers: Relievers and the Valu...
2010-12-15 - Checking the Numbers: Ranking R2C2
2010-12-08 - Premium Article Checking the Numbers: A Snake, a Padre, and ...
2010-12-01 - Premium Article Checking the Numbers: Upton and Away
More...

INCOMING ARTICLE LINKS
2013-11-25 - Premium Article Transaction Analysis: Dipoto Abandons His Pr...
2013-10-23 - The Lineup Card: 12 Moves That Helped the Ca...
2012-11-01 - Premium Article Transaction Analysis: The League of Extraord...
2012-07-10 - Premium Article Overthinking It: The Brief Wondrous Life of ...
2012-01-12 - Premium Article Transaction Analysis: Off-Brand Bargains