CSS Button No Image Css3Menu.com

Baseball Prospectus home
  
  
Click here to log in Click here for forgotten password Click here to subscribe

Premium and Super Premium Subscribers Get a 20% Discount at MLB.tv!

<< Previous Article
Prospectus Notebook: B... (12/01)
<< Previous Column
Premium Article Crooked Numbers: What ... (11/17)
Next Column >>
Premium Article Crooked Numbers: Devia... (12/15)
Next Article >>
Prospectus Notebook: R... (12/02)

December 1, 2005

Crooked Numbers

Plop Plop Fizz Fizz

by James Click

the archives are now free.

All Baseball Prospectus Premium and Fantasy articles more than a year old are now free as a thank you to the entire Internet for making our work possible.

Not a subscriber? Get exclusive content like this delivered hot to your inbox every weekday. Click here for more information on Baseball Prospectus subscriptions or use the buttons to the right to subscribe and get instant access to the best baseball content on the web.

Subscribe for $4.95 per month
Recurring subscription - cancel anytime.


a 33% savings over the monthly price!

Purchase a $39.95 gift subscription
a 33% savings over the monthly price!

Already a subscriber? Click here and use the blue login bar to log in.

It's a good time to be a free-agent relief pitcher. So far this offseason, Billy Wagner and B.J. Ryan turned their closer roles into $43 and $47 million, respectively. The Yankees appear to have inked Kyle Farnsworth for over $17 million. To boot, the Cubs have signed Bobby Howry and Scott Eyre for a combined $23 million, indicating they either know something the rest of us don't about the seventh and eighth innings or there's some weird reenactment of "Brewster's Millions" going on in Chicago. Only $7 million more to go before the Cubs have spent it all and gotten no tangible assets in return.

Putting aside the dollar values on these contracts for a moment, it's important to consider just how consistent and predictable reliever performances are. There are a multitude of factors that routinely influence reliever performance more than that of starting pitchers or batters; primarily those are small sample size and the prevailing usage patterns of modern bullpens. The sample size issue is obvious--most relievers top out around 60 or 70 innings, roughly 1/3 of a typical starting pitcher's innings--but the way modern bullpens are managed (bringing in relievers in the middle of innings, for example) often means that a reliever's performance, as measured by ERA, is as much a reflection of those pitching before and after him than his own contributions. Whereas starters often get to work into and out of their own jams, relievers don't have that luxury.

The second problem is more easily corrected than the first. We can use Fair Run Average (FRA), a BP stat that removes the problems of appropriately placing responsibility for inherited or bequeathed runners. As a first pass, just to see how bad the small sample size is, let's see how consistent a variety of pitching statistics are for both starters and relievers. To do so, we'll only use significant consecutive seasons, in this case defined as a minimum of 150 innings in consecutive seasons for starters and 50 innings for relievers.


              WXRL /
GROUP   FRA   SNLVAR    RA      ARP     SO%     BB%     HR%
SP     0.146   0.113   0.132    N/A    0.654   0.482   0.245
RP     0.040   0.094   0.039   0.052   0.585   0.353   0.092

(The second column "WXRL / SNLVAR" is SNLVAR for SP and WXRL for RP.) What we have here are the coefficient of determination (r-squared) for each metric, on a scale of 0-to-1 with 1 indicating a perfect correlation and 0 indicating total randomness. For example, 14.6% of FRA for starters is explained by the FRA in the previous season; for relievers, it's 4.0%. As expected, relievers see significantly lower correlations across the board, most likely due to their smaller sample sizes.

Of course, teams aren't limited to looking at only the previous season when determining how they think relievers are going to perform in the future and, thus, how much to pay them. Eyre's FRAs the last three years have been 3.82, 3.98 and 1.01 Wait, maybe that's not the best example. Farnsworth checks in with 3.37, 5.52 and 2.01. Okay, not so great. Howry has been good the last two years, but missed nearly all of 2003. Only Wagner and Ryan have been anywhere near consistently dominant the last three years, posting FRAs of 1.61, 2.76, 1.83 and 3.17, 1.10, 2.44, respectively. But notice that even their last three seasons show dramatic changes. There's a large difference between 2.76 and 1.61 or 3.17 and 1.10, anywhere between around eight and 15 runs depending on workload.

Instead, let's see how well a pitcher's three previous seasons project a fourth, again broken up by starters and relievers.


              WXRL /
GROUP   FRA   SNLVAR    RA      ARP     SO%     BB%     HR%
SP     0.192   0.177   0.179    N/A    0.683   0.541   0.287
RP     0.064   0.104   0.068   0.054   0.631   0.450   0.161

While any prediction system will tell you that using the previous three seasons is going to be more accurate than using just a single year, the difference isn't as great as perhaps we would hope. Reliever FRA and RA are still nearly three times as random as that of starting pitchers.

To get an idea of what that means in a practical sense, take a look at the distribution of the difference in FRA from the actual value to the predicted value in the previous three seasons. Remember that FRA has already accounted for bullpen support, so the change in values isn't a result of a pitcher suddenly having better or worse pitchers around him.

chart 1

The large pink spike in the middle is the starters, while the wider, blue curve is the relievers. While there is considerable overlap over the two series, the relievers are significantly more disbursed, particularly when it comes to pitchers whose FRA suddenly jumps by around two runs. The standard deviation for the starters is 0.81; for relievers it's 1.46. This means two key things: First, relievers are nearly twice as unpredictable as starting pitchers. Second, in any given season, over a third of relievers will have an FRA more than 1.46 runs more or less than their previous three year average.

Even when separating starters and relievers into three groups, the elite relievers do not suddenly become surer bets. Relievers who post low FRA numbers--in this case, 2.50 or below--in any three-year stretch have a .008 r-squared to their next season and the difference between their previous three year stretch and the next season has a standard deviation of 1.15. Similar starters have a consistency of .100 and a standard deviation of 0.67. Once again, relievers show a distribution of performance just less than twice that of starters, but while relievers were about a third as consistent as starters (as measured by r-squared), the elite relievers show almost no consistent ability to remain as such. For every Mariano Rivera in 1999--posting a 1.37 FRA after a weighted 1.34 the previous three seasons--there's a Mariano Rivera in 2000: a 3.23 after 1.32. For every Keith Foulke in 2004, there's Keith Foulke in 2005 (who doesn't even make this study because he didn't pitch enough innings).

So what does this mean for teams like the Cubs, Yankees, Blue Jays and Mets? Of the five relievers they signed, it's likely that two of them will post an FRA a run-and-a-half or more from their established levels. Some of this variance is the natural change in player performance; after all, starting pitchers, while more consistent than relievers, are certainly no models of consistency. But even when comparing three-year groups of relief performance--attempting to remove the small-sample-size issue--relievers never approach the consistency of starting pitchers. Over the next three years, it's likely that two of them will post a total FRA more than a run off of their established levels over the past three seasons.

There are a multitude of reasons why those signings are a mistake--overpaying for such a small number of innings pitched and the availability of respectable relief arms at the league minimum via other sources to name two--but the main reason is that relief pitchers are significantly less consistent than even volatile starting pitchers. Toronto GM J.P. Riccardi has publicly justified his decision to sign Ryan by comparing him to Trevor Hoffman, but it's just as likely the Jays are now the proud owners of Troy Percival (1995-97: 1.36; 1998-2000: 4.33), Jose Mesa (1995-97: 2.69; 1998-2000: 5.45), or Rod Beck (1992-94: 1.68; 1995-97: 3.89).

By the way, Hoffman's FRA from 1997-99 was 1.65; it jumped to 3.12 from 2000-02.

Related Content:  Relievers,  Starters,  Sample Size,  FRA

0 comments have been left for this article.

<< Previous Article
Prospectus Notebook: B... (12/01)
<< Previous Column
Premium Article Crooked Numbers: What ... (11/17)
Next Column >>
Premium Article Crooked Numbers: Devia... (12/15)
Next Article >>
Prospectus Notebook: R... (12/02)

RECENTLY AT BASEBALL PROSPECTUS
Premium Article What You Need to Know: September 2, 2014
Premium Article The Call-Up: Daniel Norris
Premium Article The Call-Up: Maikel Franco
Premium Article Transaction Analysis: Bo Gone
The Future
Premium Article The Call-Up: Joc Pederson
Premium Article Monday Morning Ten Pack: September 2, 2014

MORE FROM DECEMBER 1, 2005
Prospectus Notebook: Braves, Athletics

MORE BY JAMES CLICK
2005-12-29 - Premium Article Crooked Numbers: The Numbers You Saw
2005-12-22 - Premium Article Crooked Numbers: Value Over Replacement Colu...
2005-12-15 - Premium Article Crooked Numbers: Deviation from Standards
2005-12-01 - Premium Article Crooked Numbers: Plop Plop Fizz Fizz
2005-11-17 - Premium Article Crooked Numbers: What Were They Thinking?
2005-11-10 - Premium Article Crooked Numbers: Today's Oxymoron Is Free Ag...
2005-11-03 - Premium Article Crooked Numbers: Homeland Defense
More...

MORE CROOKED NUMBERS
2005-12-29 - Premium Article Crooked Numbers: The Numbers You Saw
2005-12-22 - Premium Article Crooked Numbers: Value Over Replacement Colu...
2005-12-15 - Premium Article Crooked Numbers: Deviation from Standards
2005-12-01 - Premium Article Crooked Numbers: Plop Plop Fizz Fizz
2005-11-17 - Premium Article Crooked Numbers: What Were They Thinking?
2005-11-10 - Premium Article Crooked Numbers: Today's Oxymoron Is Free Ag...
2005-11-03 - Premium Article Crooked Numbers: Homeland Defense
More...

INCOMING ARTICLE LINKS
2006-02-27 - Prospectus Notebook: Red Sox, Cubs