September 7, 2006
Lies, Damned Lies
What to Do With Papelbon
Since 1946, there have been exactly 100 pitchers who pitched at least 80% of their innings in relief after having pitched at least 80% of their innings as a starter in the previous season (minimum 50 IP in year n and 100 IP in year n-1). Sometimes, these were pitchers like
The good majority of these pitchers improved their ERA, as well as their peripheral statistics, upon making the jump to the bullpen. But this should be no surprise. As I described last week, if you take a starter whom you know nothing else about, you can expect him to knock about 25% off his ERA when he pitches in relief. The question, then, is whether there are any similarities among the pitchers that improved their ERA by more than 25% once joining the bullpen.
In order to tackle this question, we’ll compare two things:
These ten pitchers took the most advantage of their new role:
Pitcher Year EqERA Role-Adjusted Improvement -PECOTA EqERA ------------------------------------------------------------------- Goose Gossage 1977 3.96 1.89 210% Eric Gagne 2002 4.42 2.39 185% Mike Remlinger 1999 4.38 2.53 173% Jeff Russell 1989 4.75 2.89 164% Dave Leonhard 1969 4.92 3.05 161% Wayne Twitchell 1976 4.43 2.76 160% Tom Gordon 1998 4.04 2.53 160% Hal Woodeschik 1963 4.74 3.07 154% Jose Jimenez 2000 4.32 2.89 150% LaTroy Hawkins 2000 4.96 3.32 150%
Many of these names will come as no surprise, and the list includes some famously successful bullpen conversions like those of
Conversely, here is the rather less interesting list of unsuccessful conversions:
Pitcher Year EqERA Role-Adjusted Improvement -PECOTA EqERA ------------------------------------------------------------------- Ray Fontenot 1986 4.27 7.34 58% Bill Greif 1975 4.63 7.69 60% Dennis Ribant 1968 4.83 8.00 60% Scott Terry 1990 4.41 6.99 63% Shawn Chacon 2004 4.42 6.95 64% Paul Calvert 1950 4.97 7.78 64% Lee Stange 1968 4.28 6.66 64% Duke Maas 1960 4.77 7.31 65% Don Kirkwood 1977 4.94 7.55 65% Dick Trout 1949 3.81 5.74 66%
What do the successes (and failures) have in common? Well, I don’t know that there’s anything obvious. It’s easier to say that the successes had better stuff, and while there’s some truth to that, there’s also an element of selective memory: Gagne’s change-up wasn’t nearly as feared until he became a closer.
We can also examine the components of the PECOTA baseline forecast-things like a pitcher’s strikeout and walk rates--and compare them with a pitcher’s performance as a reliever. Like the baseline forecasts, these rates are adjusted for park and league effects, and determined based on a pitcher’s weighted average performance in the three years before he converted to the bullpen.
Following are the correlations between performance in these departments and 'improvement', i.e. the success that the pitcher had in adapting to the bullpen.
A higher strikeout rate, for example, is correlated with a higher comparative advantage when working in relief. Note that this is above and beyond the advantage that a higher strikeout rate always provides, which is already reflected within a pitcher’s baseline PECOTA. That is perhaps intuitive; if you’re going to work to only a few hitters at a time, you damned well better have a pitch that can get some of them out.
Interestingly, however, a higher walk rate is also associated with relief success. In fact, nine of the top ten pitchers on our 'success' list were wilder than average as starting pitchers. (
Another important relationship involves isolated power. Pitchers that allow a high ISO--that is, pitchers that give up a lot of home runs--have a lot of trouble if they try and convert to relief. Note that there is no relationship between groundball rate and relief success. It is not keeping the ball down that makes one a successful reliever--in fact, a number of outstanding relievers like
Are these relationships statistically significant? It turns out that two of them are, according to a regression analysis. Specifically, the relationships with walk rate and isolated power are statistically significant; strikeout rate drops out once we account for the effects of isolated power, which has a strong inverse correlation with strikeout rate. Together, walk rate and isolated power account for about 10% of the variance between forecasted and actual EqERA. That is not large enough to make them some sort of "secret sauce" for predicting successful relief transitions, but considering how much noise there is in ERA from season to season, it is a relatively impressive result.
Why do isolated power and walk rate have this special relationship with a pitcher’s role? I think it has less to do with the nature of the stats themselves, and more to do with what they tell us about a pitcher:
A relief role emphasizes high-impact pitching and deemphasizes consistency and durability. A low ISO is a good proxy for high-impact pitching, a pitcher who can take control of the at-bat with one or two great pitches. Meanwhile, a low BB rate is a good proxy for mechanical consistency. Thus, we’re left with something like this:
All of this seems intuitive enough. Nevertheless, it’s routine to see teams employing pitchers in starting roles even after they’ve demonstrated time and time again that they lack the mechanical foothold to throw six or seven innings without giving something up.
Which current starters might stand the most to gain by converting to relief? Let’s do this Olbermann-style:
Would Papelbon have made this list a year ago? Probably not. Nevertheless, the question boils down to his durability. Papelbon was a reliever in college. He’ll be 26 next year, and has never thrown more than 148.2 innings in a professional season. His walk rate has risen and fallen with the proportion of innings he’s pitched as a starter. And he’s made more effective use of his splitter this season, a pitch that is generally very taxing on a starter’s arm. I understand full well that a great starter is more valuable than a great reliever. But in Papelbon’s case, the risk is too high to justify the reward.