Printer-friendly

February 6, 2008

Building a Better Bullpen

The Rays' 2008 Relief Corps

by Caleb Peiffer

The Tampa Bay Devil Rays existed for 10 seasons—the club exorcised the Devil this past November—and have been plagued for most of that decade by an inability to put a decent bullpen together. Consider for a moment that in five out of 10 years, Tampa Bay's firemen combined for a negative Adjusted Runs Prevented (ARP) total.

What exactly is ARP, and why is it used here rather than another bullpen metric, such as WXRL? ARP is a pure context-free measure of pitcher effectiveness that doesn't take into account the leverage of the situation; a counting stat that compares a reliever's performance to how an average (not replacement-level) relief pitcher would have performed in the same situations. In other words, if you are looking simply for how many more runs a bullpen prevented or allowed than average, regardless of the timing of the relief work or how it impacted the game, then ARP is your stat. It boils away luck and any statistical advantage (or disadvantage) attained from pitching well (or poorly) in more important situations to get at the bare-bones underlying performance, which is what we want to evaluate in looking at past bullpen work. As Keith Woolner explained in a 2005 mailbag:

WXRL is a way to assess past performance, usage, and game importance. But it is not independent of how and where a pitcher is used. A better measure for that would be Adjusted Runs Prevented (ARP), which takes into account only how a pitcher performed, not the leverage of the situation, but does consider inherited and bequeathed runners fairly in doing so.

Derek Jacques did a great job of breaking down the differences between WXRL and ARP in this Toolbox last year, and if you want to get down to the real nuts and bolts of the ARP formula, take a trip back to the 20^th century with this article by the founder of the metric, Michael Wolverton.

Getting back to the Rays' bullpen, the last three years have been especially gruesome—in that span the Rays relief has had aggregate ARPs of -40, -33.2, and -95.4, respectively. Last year's figure was the worst in the Baseball Prospectus database, which dates back to 1959. Using the sabermetrically-standard conversion of 10 runs equals a win, Devil Rays relievers cost the team nearly 10 wins compared to what an average pen would have produced pitching in the same situations. Twenty relievers moved through the bullpen for the Devil Rays last year, and exactly two of them were more than one run better than average. In fact, it can be argued that the best Devil Rays reliever was utility infielder Josh Wilson, on the basis of his pitching a scoreless inning in the team's 14-8 loss to the Marlins on June 8, and then never taking the mound again.

There hasn't been a bullpen that's come within 15 runs of last year's Tropicana disaster; the next-worst team by ARP is the 1990 Braves, whose relievers compiled a -78.4 figure. The 2007 Tampa Bay pen had a WXRL that ran into the red, as well, clocking in collectively at -1.75. In other words, if a journeyman assemblage of Triple-A relievers had taken over the home bullpen in the Trop, the Devil Rays would have won almost two more games than they did.