Baseball Prospectus is looking for a Public Data Services Director. Read the description here.

Always a Bridesmaid, Never a Bride

For a certain type of completist, memorizing all the relievers that get roster spots by virtue of 13-man pitching staffs is the troublesome last mile. Any fantasy player worth his salt can rattle off nearly every team’s closer, even ones whose job security is questionable. But closers aren’t always selected purely on merit, and there is usually a flamethrower waiting in the wings to become the anointed. Those relievers who, for whatever reason, do not close ballgames but that nevertheless mow down hitters are fascinating to me. Ask most fans who the best non-closer reliever in baseball is, they might stumble a bit, but you’ll get a pretty interesting discussion going.

Like Janus, Looking Forward and Backwards

Any statistical attempts to answer the question will be highly sensitive to the metrics chosen. We know ERA is unacceptable for relievers, but most rate stats suffer similar problems. We can look at component stats, but even then small sample size problems still persist. Besides, there seems to be a stronger edict for asking how a player actually did, rather than how he ought to have done, or could have done), when it comes to relievers. On the other hand, we cannot simply look backwards to evaluate talent, since we would inevitably single out guys who were merely lucky to be there.

To solve these twin problems, I came up with a toy to try to rank relievers. It’s not a complicated toy. In fact, it only has two moving parts: SIERA and WXRL/LEV. I hope by now you’ve had a chance to peruse Eric Seidman & Matt Swartz‘s series on SIERA, but the gist is that it gives an estimation of a pitcher’s controllable skills (fly ball rate, strikeout rate, ground ball rate, and walk rate) and considers how they interact with one another. Put simply, it’s a way to evaluate the totality of a pitcher’s skills while looking beyond contingent (or luck-based) factors. WXRL, on the other hand, is a metric based on win expectancy. It simply measures, compared to replacement and adjusted for quality of opposing lineup, how the likelihood of the reliever’s team winning changed from when he entered the game to when he left. Because it is dependent on the situations in which a pitcher is deployed (and thus would favor closers especially), I have divided it by LEV, which is a measure of the leverage of the average situation the pitcher faced. (I realize there have been criticisms of LEV, but as a rough and ready way to de-leverage WXRL, it should do fine.) Remember, with SIERA, lower is better, but with WXRL/LEV, higher is better (because it is denominated in wins).

Damn the Torpedoes

I’ve taken these two measures of reliever performance, and ranked the best relievers (with more than 25 IP) in each measure for the 2009 season. I’ve assigned a number rank based on position in each list. I’ve then added the ranks on the two lists together, and re-sorted that list from lowest to highest. In other words, to rank high on my hybrid list, a reliever must have done well in both WXRL/LEV and SIERA in 2009. Here are the top three overall (SIERA and WXRL/LEV in parentheses):

1. Jonathan Broxton (1.98 SIERA, 2.84 WXRL/LEV)
2. Joe Nathan (2.31, 3.18)
3. Mariano Rivera (2.44, 3.25)
3. Michael Wuertz (2.21, 3.13)

Wuertz snuck in to tie Rivera, placing ahead of closers Huston Street (tied for fifth), Jonathan Papelbon (seventh), Joakim Soria (eight), Andrew Bailey (ninth), and Rafael Soriano (10th). Obviously, he’s No. 1 on the non-closer list, presented below.

1. Michael Wuertz (2.21, 3.13)

Wuertz’s slider is truly a sight to behold, and many hitters have walked red-faced back to the dugout after beholding it dart through the zone for a backwards K. Though pitchF/X data tell us his fastball sits around 91 mph, the slider is more than enough to make him among the most dangerous non-closer relievers in baseball. Because he plays in a pitcher-friendly park and generates plenty of ground balls, he’s likely to continue to succeed in 2010.

2. Matt Thornton (2.56, 2.81)
Thornton is a perfect example of how dominant relievers can emerge slowly and quietly. He’s always had electric stuff (including a fastball that sits 95-96 mph and touches the high 90s), but in the last two years, Thornton has become nearly unhittable. Last season in 72.1 innings, Thornton struck out 87 and walked just 20. Did I mention his ground ball tendencies? Those are the kinds of peripherals that SIERA notices, and his mark was lower than his still-low ERA (which was 2.74 in 2009).

3. Luke Gregerson (2.91, 1.96)
Acquired prior to last season in the deal that sent Khalil Greene to the Cardinals, Gregerson flourished in his Padres‘ debut. Though there are worse gigs to help a reliever’s value than pitching in Petco Park, Gregerson was not a product of his circumstances. In 75 innings, he struck out an astounding 93 batters. He issued just 22 unintentional walks and allowed just three home runs. His low-90 mph fastball doesn’t blow guys away, but he supplements it with a nifty slider that also generates plenty of ground balls.

4. Pedro Feliciano (3.07, 2.04)
One byproduct of a list generated like this is that the WXRL aspect tends to favor high-workload LOOGYs like Feliciano. Nevertheless, he’s earned his way on the list, and SIERA is especially good at evaluating extreme groundballers like him.

5. Mike Adams (2.09, 1.58)
Another byproduct of this list is that it can value guys like Adams, who pitched just 37 innings in ’09, but was utterly dominant. Like Feliciano, Adams doesn’t have one dominant out pitch. Instead, he combines his low-90s fastball with a kitchen sink approach that baffled hitters in limited time. The result was a 45/7 K/UBB. It’s around this point that you start wondering where the Padres find all these guys.

6. Jeremy Affeldt (3.46, 2.68)
The Old Man and the Bay may sport the highest SIERA of any guy on the list, but his extreme ground ball tendencies last season (64-percent ground ball rate) helped him rack up the WXRL. A sinkerballer who throws mid-90s can kill plenty of rallies, but Affeldt may be the guy on this list I most doubt going forward.

7. Neftali Feliz (1.99, 1.35)
Like Adams, Feliz just didn’t have the reps to accumulate WXRL, but, boy howdy, did he dominate the SIERA leader board. Only Broxton bested him, and Feliz’s stuff is probably better than anyone else’s on this list. His high-90s fastball is enough to make grown men (or Kevin Goldstein, at least) weep. Whether he remains a reliever in the short term is unclear, but his long term future is in the rotation.

8. Trever Miller (2.88, 1.60)
Miller’s presence on this list is a lot like Feliciano’s. He’s a LOOGY who strikes out plenty of batters and managed to avoid yielding hits last year. He’s been solid for several years now, so his formula may be more or less replicable.

9. Sergio Romo (2.64, 1.41)
Romo may be the biggest mystery on this list. Although his fastball sits right around 90 mph, his slider is his best weapon. He hasn’t been used too much by the Giants in the last two seasons (just 68 innings combined in ’08 and ’09), but he has a 74/19 K/BB in that time. The Giants and Padres have cornered this list.

10. Nick Masset (3.32, 1.92)
White Sox fans, look away. Masset was part of the deal that sent Griffey, Jr. to the South Side, and has emerged as a very valuable reliever. He complements a mid-90s fastball with a curveball and cutter that is capable of generating ground balls. He benefited from a fluky-low BABIP and high LOB percentage last year, but his peripherals and stuff suggest he will be above average for several years.

Question of the Day

I limited myself to the methodology of my toy. Which deserving relievers did such a method leave out? Which included relievers are undeserving?

Thank you for reading

This is a free article. If you enjoyed it, consider subscribing to Baseball Prospectus. Subscriptions support ongoing public baseball research and analysis in an increasingly proprietary environment.

Subscribe now
You need to be logged in to comment. Login or Subscribe
As a Padres fan, I've always felt that Kevin Towers (KT) was the best at finding live arms to stock a deep bullpen. Guess we are about the find out if it was KT or something organizational.
I agree-KT is a genius at finding "scrap heap" relievers. I'll be interested to see if the Yankees suddenly develop the ability to find those under-valued relievers now that he's on staff.
KT did have some talent for this in the past, but I wouldn't be surprised if DePodesta had his fingerprints on some of the more recent ones. It's pretty well accepted that he's the one who pushed to get Heath Bell. It wouldn't surprise me if Adams or Mujica were his guys as well. I'd lean towards Gregorson being a KT guy though.

You also have to add in that Darren Balsley is absolutely fantastic at helping guys iron out some of their issues and making sure they are used appropriately.
Padres unite! I hope you are right that DePodesta was involved, because KT did seem amazing at finding those arms. Sludge merchant: I think that was a self-description at one point.

Regarding Balsley, anyone ever see a "pitching coach lift" analysis, kind of like the ones that see if catchers improve pitcher performance (usually not)? Duncan in STL has the reputation, and I always hoped that Perez would completely flame out and SDP pick him up on a minor league deal only to get him back to prior glory.
I can't imagine that Masset will continue his good fortune. I guess Moylan missed the list because he only had a handful of appearances in 2008 and had a week start to 2009, but he's a severe groundball releiver in a pitcher's park. Admittedly he's unlikely to go two years without allowing a HR, but surely he's one of the best non-closers in the game.
Moylan didn't do TERRIBLY by this metric last year, but his SIERA (3.66) wasn't great and his WXRL/LEV actually ranked still lower among his peers.
Re: the Q of the day... off the top of my head, I'd think Phil Hughes deserved a mention.
Yeah, Hughes would actually qualify for this if I included both starter innings and reliever innings. But he loses out in WXRL when you look at just the reliever innings, so he slipped past Romo in my methodology. If he were a reliever for a full year, he might very well qualify.
*Romo and Masset, that should be.
Robertson is hurt by his low WXRL, but he's a good bet going forward, provided he limits the walks.
Hughes or Chamberlain - whichever one ends up in the pen.
Sergio Romo throws several different pitches for a lot of different arm angles, and manages to get them into the strike zone. He looks 'different' enough that guys don't have time to figure him out when he only goes through the lineup one time (or less). He has had a couple of minor injuries which is the primary reason he hasn't pitched much. I love watching Romo pitch, but he is kind of risky bet because he doesn't throw that hard and needs to hit his spots more than most guys to be effective.
Wouldn't it be White Sox fans that need to look away and not Seattle fans? Masset went from CWS to CIN in the Griffey deal...
Yep, that's a typo! Thanks for the good catch.
White Sox fans such as myself are too stunned by the inclusion of Nick Masset on a good anything list to turn away. Seriously? Nick Masset? How easy is it to pitch in the NL??
When healthy, Hong-Chih Kuo has looked very good the last two seasons.
No mention of Angel Guzman? He has been pretty great.
His 4.09 SIERA basically disqualified him.
Mujica seemed to have solid year for the Padres when he worked out of the pen. I would assume that his numbers as a starter later in the year complicate things.
Is Ryan Madson considered a closer?
I wouldn't have counted him as the closer, but he ranked 30th overall on the list (which includes closers). That means he just missed this list.
Thank you for finally giving some love to Mike Adams. He put up nintendo numbers the last two seasons. I've rooted for him a long time-he's had knee surgeries (both knees) and shoulder surgery for good measure. He does nothing but dominate in between the DL stints. If he's healthy, I'd think the Padres will make Heath Bell available, and slide Adams into the 9th. I know I would.
When will pitching metrics advance to the point where quality of opposition faced is used to calculate an opponent-neutral ERA? A key component of the opp. "quality" calculation would be handedness of the batter rel. to the pitcher. Continuing to evaluate pitchers without doing this is like using a college football power rating that doesn't incorporate strength of schedule.
Thats why he used SIERA and WXRL/LEV instead of ERA...ANY ERA(opponent neutral or not) is basically garbage. There is more to a pitcher than just the batters he faces(Lefty or Righty) and the actual outcome of his efforts. (ie. Defense, Park factors, BABIP, etc.)
Anybody from the Rays a threat to make this list? They always seem to have a deep bullpen.
Be curious to know how Matt Guerrier would compare. The Twins bullpen in general seems solid. Guerrier has led the league in appearances for the last two years. Impressive that he's held up pretty well despite pitching in 70+ games three years running.
I think it would be interesting to see how these guys rank on the complete list. How many non-closers outpitched the worst closer?
From the rebuilding Blue Jays you might want to consider Jason Frasor. He had 11 saves last year and is definately in the mix for the closer's job this year. He also has an improved change up which made its appearance last year as well. From the various interviews it seems that he's ready for the position and with no incumbant he may very well get the job.
A few other names to toss out there, based on a scan of Strat-O-Matic ratings, not SIERRA or WNXL/LEV (for which I don't have the whole list):

Claudio Vargas
Darren O'Day
Brian Sanches
Fu Te Ni
Kiko Calero
Joe Thatcher
Alfredo Aceves
Sean Burnett

All would be solid non-closer Strat cards in the 2009 set.

Meant WXRL/LEV...sorry