CSS Button No Image Css3Menu.com

Baseball Prospectus home
  
  
Click here to log in Click here for forgotten password Click here to subscribe

<< Previous Article
Premium Article Another Look: Bob Fell... (12/22)
<< Previous Column
Checking the Numbers: ... (12/15)
Next Column >>
Checking the Numbers: ... (01/06)
Next Article >>
Premium Article Prospectus Perspective... (12/22)

December 22, 2010

Checking the Numbers

Relievers and the Value of Perfect Information

by Eric Seidman

the archives are now free.

All Baseball Prospectus Premium and Fantasy articles more than a year old are now free as a thank you to the entire Internet for making our work possible.

Not a subscriber? Get exclusive content like this delivered hot to your inbox every weekday. Click here for more information on Baseball Prospectus subscriptions or use the buttons to the right to subscribe and get instant access to the best baseball content on the web.

Subscribe for $4.95 per month
Recurring subscription - cancel anytime.


a 33% savings over the monthly price!

Purchase a $39.95 gift subscription
a 33% savings over the monthly price!

Already a subscriber? Click here and use the blue login bar to log in.

One of the most interesting aspects of baseball transactions is that every move carries a certain level of risk. Without perfect information akin to an extreme level of accurate prognostication, teams can never be 100 percent certain how a player will perform. The best investment is a player who will produce at a very high level, but who comes as close to guaranteeing that high level of production as possible. Albert Pujols personifies this idea, as a bad season for him still involves above-average defense and a TAv north of .310. I will say right now, with as much confidence as I have in my mind and body, that Pujols will hit, at worst, .310/.370/.530 next season. He has proven himself capable of production far beyond that slash line and is as close to a sure thing as we have.

Signing Jeff Francoeur to play every day would be an example of the opposite effect, as there is ample evidence that he cannot handle such a role. We are virtually certain he cannot handle that type of role in a productive manner, so the risk associated with signing him extends beyond the normal types inherent in signing a player because of the increased level of certainty that he will not live up to expectations. The majority of players traded for or signed falls into the area in between both of these extremes, which makes evaluating transactions quite the tall, but fun, task. This is before even getting into the idea of evaluating moves on their merits at the time as opposed to retrospectively, which is a topic for another day.

Relief pitchers, however, muddy the waters, as they are as fickle in performance as teenage girls are with their crushes. In spite of this, teams often place a high premium on their services. Too often is a reliever signed to a multi-year deal off of 55 solid innings, only to prove that his success was a direct result of small sample size trickery. In other cases, the same hypothetical reliever might pitch well in the year following his signing, while his production erodes over the remainder of the deal. If signing a player who has proven himself unfit for an everyday role represents a terrible investment, signing one to a lucrative contract based on numbers derived in minute samples is not too far off.

When Scott Downs signed a three-year deal worth $15 million with the Angels, I got to thinking about how relievers signed to multi-year deals typically perform. I could at least somewhat understand it in Downs’ case, because he has performed very well for an extended period of time. When Jesse Crain and Matt Guerrier signed three-year deals, my head began to spin and I started wondering about how relievers perform over the lives of their multi-year deals. Do they live up to expectations? Does their production match, or exceed, the value of the contract? Are they signed to long-term deals so the team can reap the benefits of that first season through an added push in playoff contention?

On a more basic level, does a reliever signed to a three-year deal perform better than three relievers signed to one-year deals over the same time span? And are the relievers inked to one-year contracts given less money, on average, than those signing multi-year deals? If the answers to the last two questions are no and yes, respectively, then it would behoove teams to avoid doling out significant contracts to relievers. These are important questions to research because some teams have allocated the majority of their off-season spending to shoring up a bullpen (cough, 2007 Orioles, cough). If taking such a course of action is suboptimal in both the short- and long-term, then teams should spend their money elsewhere. However, my goal here is not to definitively answer these questions, but rather to open the discussion and explore why conducting studies along these lines are not as clear cut as they may seem.

Downs seems to be in the elite reliever mold, as his ERAs since 2007 have been 2.17, 1.78, 3.09, and 2.64. In a grand total of 236 2/3 innings over the same four-year span he has produced 8.8 WARP, a very good mark for such a small sample of production. But is he worth committing three years and $15 million to? There are a few ways to attack a question like this, but I am not interested in projecting his performance and figuring out if the translation to financial value exceeds the dollars paid out in the contract. Instead, I am looking to see how multi-year deals for relievers generally work out relative to their one-year counterparts.

To that end, I used the ESPN Free Agent Tracker and went back to the 2006 offseason, binning relievers based on whether they signed multi-year deals or deals for just one season. Multi-year relievers were recorded for the 2006 and 2007 offseasons, while I added all one-year relievers from the 2006-09 offseasons to my spreadsheet. For the multi-year relievers I recorded their WARP totals in each of the two years preceding their contract year, as well as over the life of the deal. The same process was repeated for the one-year wonders, though the lives of their deals were just one season.

We could repeat this process with WXRL, FRA, SIERA, or even ERA, but I will stick with WARP for now, as the more important idea is how to actually begin a study like this—too many articles have attempted to answer the same question, either starting with the wrong question or failing to use a control group, or even neglecting to include very important variables discussed later on. The table below shows the pertinent information for the multi-year relievers signed prior to the 2007 season, with the PY and Y columns representing WARP totals in the second preceding year, the prior year, and each of the years in the new contract. Note, too, that I excluded Mariano Rivera from the tables as he is a curve-buster.

NAME

AGE

DEAL

PY-2

PY

Y1

Y2

Y3

Danys Baez

33

3/$19

3.9

0.1

-0.1

0.0

1.7

Miguel Batista

39

3/$25

0.4

2.3

2.0

-2.3

0.7

Chad Bradford

36

3/$10.5

0.6

1.7

2.1

1.7

0.1

Alan Embree

40

2/$5.5

-1.6

0.9

1.4

0.0

N/A

Steve Kline

38

2/$3.5

0.8

0.8

0.0

0.0

N/A

Guillermo Mota

37

2/$5

0.3

1.1

0.2

0.8

N/A

Hideki Okajima

34

2/$2.5

N/A

N/A

2.5

1.9

N/A

Scott Schoeneweis

37

3/$10.8

1.6

1.3

-0.1

0.9

-0.6

Justin Speier

37

4/$18

1.8

0.9

1.5

0.2

0.4

Mike Stanton

43

2/$5.5

-0.1

2.0

0.1

0.0

N/A

Jamie Walker

39

3/$12

0.7

0.7

1.5

-0.4

0.1

David Weathers

41

2/$5

1.4

1.8

2.9

2.2

0.8

Francisco Cordero

35

4/$46

3.5

3.3

3.0

3.7

2.1

Octavio Dotel

37

2/$11

-0.3

0.7

1.2

1.3

N/A

Scott Linebrink

34

4/$19

1.8

1.0

1.1

0.5

0.8

Ron Mahay

39

2/$8

0.3

1.8

1.4

0.6

N/A

Troy Percival

41

2/$8

N/A

1.5

-0.1

-0.3

N/A

David Riske

34

3/$13

0.6

2.7

-0.2

-0.1

0.1

Luis Vizcaino

36

2/$7.5

2.3

1.3

0.4

0.0

N/A

AVERAGE

37.3

2.6/$12.4

1.1

1.4

1.1

0.6

0.6

 

Summarizing the table above, the pitchers averaged 37.3 years of age and signed deals to an average of 2.6 years and $12.4 million. In the second preceding year they averaged 1.1 WARP; in the preceding year they averaged 1.4 WARP. In the first year of their shiny new contracts, they reverted to the level of production two years before signing. And during the second and third years of their deals this group performed worse than any of the other years tabled above. The sample is small, of course, so nothing definitive can be drawn from this table. But this is a starting point.

The relievers appeared to be worth approximately $5-6 million leading into the deal and then saw their production plummet. Over the lives of the various deals, the multi-year relievers provided, on average, 2.3 wins. How did their one-year deal counterparts fare over the same time span? Well, there are far too many relievers to list in a table, so I will instead just summarize the information in the table below:

Year

Age

Deal

PY-2

PY

Y1

2007

37.3

1/$2.6

0.6

0.1

0.5

2008

36.1

1/$2.9

0.6

0.3

0.4

2009

37.3

1/$2.2

1.3

0.9

1.0

2010

35.4

1/$2.6

1.1

0.8

1.7

Average

36.6

1/$2.6

0.9

0.5

0.8

What is striking at first is how the ages are so similar across both groups. It isn’t as if the younger relievers are signing multi-year deals and the elderly are stuck going to year-to-year with teams. Additionally, the salaries go hand-in-hand with the deals, as a reliever is not going to sign a three-year deal for $5 million, just like it is somewhat rare to see a one-year deal worth $9 million or more for a reliever. The latter situation happens, but not all that often, while the former virtually never occurs in this day and age. Translating dollars to wins it appears that those signing multi-year deals during the 2006 and 2007 offseasons cost approximately $5.6 million/win in salary. The relievers signing one-year deals cost approximately $3.3 million/win in salary. And as the numbers above suggest, the relievers signed to one-year deals were not that far off of the multi-year relievers during their shorter contracts.

Given that, on average, a win is worth somewhere around $5 million per the current incarnation of MORP, multi-year relievers were slightly overpaid, while relievers signing one-year contracts were veritable bargains. However, and this is a big however, the data above is strictly salary-based. It does not include free-agent compensation for Type-A or Type-B free agents. The cost of signing a Type-A free agent is not merely the actual salary paid to the player, but that contract plus the value of the draft picks surrendered. According to this article by Matt Swartz, the cost of surrendering a first-round pick is $8 million, and the cost of giving up a second-round pick is $5 million. These values are derived from the historical value provided by players selected with those picks.

When a team signs a Type-A free agent they surrender a first-round pick, unless they finish in the bottom half of the sport, in which case they surrender a second-round pick. This is, of course, assuming that the lame-duck team offers the player arbitration. So if we have a pitcher who was offered arbitration, turned it down, was classified as a Type-A free agent, and signed a 3-year/$19 million deal with a team finishing in the upper half of the league, the new team actually spent $27 million when the cost of a surrendered first-round pick is factored into the equation. Here it becomes much more difficult to fully recoup the value of the contract with production on the field. In cases like this, teams may look to keep the reliever in the fold for an extra year or two as a means of trying to recoup more of the overall value of the contract and the compensatory picks.

If the current free-agent system receives a much-needed makeover and the compensation structure was updated, relievers would arguably benefit the most. Teams could sign relievers to one-year deals without having to give up a first-round pick, and they would not feel the need to add extra years to the deal just to potentially make up for the loss of the pick. Given that relievers are so fickle in their performance, doesn’t it make sense that the vast majority of them would not be offered multi-year deals if teams didn’t have to give anything extra up to bring them in?

There is a lot more to look into with a topic like this, but it is very important to understand that determining whether relievers are worth multi-year deals is nowhere near as easy as it might seem. Because of the structure of the free-agent compensation system, there is often a greater cost associated with signing relievers to long-term deals. Given their minute sample sizes of production it often becomes difficult to recoup the full value of the deal, which includes the value of any picks surrendered. Some teams like to make big splashes with their bullpens in the offseason, while others search the scrap heap for America’s Next Great Reliever.

Either way, it would be wise to constantly evaluate how these deals work out from an overall point of view, taking all of these factors into account, because if a strategy does not work it makes no sense to stick with it. With such a small sample it is impossible to concretely determine if multi-year relievers are poor investments, because with a larger pool of relievers the numbers may sway in the positive direction. Regardless, there are several variables that must be accounted for to even come close to accurately answering the question, and hopefully this is a good starting point. 

Eric Seidman is an author of Baseball Prospectus. 
Click here to see Eric's other articles. You can contact Eric by clicking here

Related Content:  Relievers,  A's,  Deal,  The Who,  Scott Downs,  The Process

14 comments have been left for this article.

<< Previous Article
Premium Article Another Look: Bob Fell... (12/22)
<< Previous Column
Checking the Numbers: ... (12/15)
Next Column >>
Checking the Numbers: ... (01/06)
Next Article >>
Premium Article Prospectus Perspective... (12/22)

RECENTLY AT BASEBALL PROSPECTUS
Premium Article Transaction Analysis: Live That Fantasy
Premium Article Pitching Backward: Brandon McCarthy and the ...
Premium Article Transaction Analysis: Bringing the Band Back...
Premium Article Raising Aces: Best and Worst Mechanics: NL W...
Premium Article Transaction Analysis: Catchin' Relief
Premium Article Transaction Analysis: Martin in Miami, Nate ...
Premium Article Transaction Analysis: Padres Wish Upton a St...

MORE FROM DECEMBER 22, 2010
Future Shock: New York Mets Top 11 Prospects
Premium Article Prospectus Perspective: Athletic Ambitions
Premium Article Another Look: Bob Feller

MORE BY ERIC SEIDMAN
2011-01-06 - Checking the Numbers: The Jock Tax
2010-12-30 - Seidnotes: WARP Speed
2010-12-23 - Premium Article Seidnotes: The Rodrigo Lopez All-Stars
2010-12-22 - Premium Article Checking the Numbers: Relievers and the Valu...
2010-12-15 - Checking the Numbers: Ranking R2C2
2010-12-08 - Premium Article Checking the Numbers: A Snake, a Padre, and ...
2010-12-01 - Premium Article Checking the Numbers: Upton and Away
More...

MORE CHECKING THE NUMBERS
2011-01-26 - Checking the Numbers: Withholding Aplenty
2011-01-13 - Premium Article Checking the Numbers: The Questionable Pursu...
2011-01-06 - Checking the Numbers: The Jock Tax
2010-12-22 - Premium Article Checking the Numbers: Relievers and the Valu...
2010-12-15 - Checking the Numbers: Ranking R2C2
2010-12-08 - Premium Article Checking the Numbers: A Snake, a Padre, and ...
2010-12-01 - Premium Article Checking the Numbers: Upton and Away
More...

INCOMING ARTICLE LINKS
2013-11-25 - Premium Article Transaction Analysis: Dipoto Abandons His Pr...
2013-10-23 - The Lineup Card: 12 Moves That Helped the Ca...
2012-11-01 - Premium Article Transaction Analysis: The League of Extraord...
2012-07-10 - Premium Article Overthinking It: The Brief Wondrous Life of ...
2012-01-12 - Premium Article Transaction Analysis: Off-Brand Bargains