Andrew Bailey couldn't even wait until Opening Day to get hurt. Before the Red Sox’ most high-profile off-season acquisition could even take the mound during the regular season, the team discovered that he had torn the ulnar collateral ligament of his thumb, requiring surgery that could sideline him until at least the All-Star break. General manager Ben Cherington and manager Bobby Valentine resisted the call to push converted set-up man Daniel Bard back to the bullpen, instead naming Alfredo Aceves—another reliever who spent the spring vying for a rotation spot—the interim closer.

The move bucked some conventional wisdom, given that the 29-year-old righty had accumulated just four saves during his four previous major-league seasons, three of them by pitching more than three innings in games that weren't necessarily close. Bypassed by the Boston brass was Mark Melancon, a 27-year-old righty who saved 20 games for the Astros last year in his first shot at closing in the majors.

Both pitchers have started the 2012 season in inauspicious fashion. On Opening Day (III) against the Tigers in Detroit, Melancon was summoned into a tie ballgame in the ninth inning. After getting one out, he yielded a pair of singles and was pulled. In came Aceves, who hit a batter to load the bases, then gave up a game-winning single to Austin Jackson. It wasn't a save opportunity, and it wasn't a blown save, but it wasn't anybody's finest hour, either.

On Sunday, the two pitchers struggled even more mightily. In a game in which the Sox rallied from a 4-0 deficit to carry a 10-7 lead into the ninth, Aceves gave up consecutive singles to Jackson and Brennan Boesch, then served up a game-tying three-run homer to Miguel Cabrera, thereby blowing his first save opportunity of the year. The Sox would battle back to score twice in the top of the 11th, but Melancon couldn't hold the lead; after retiring Boesch, he surrendered singles to Cabrera and Prince Fielder, who both advanced on a wild pitch. Delmon Young hit a sacrifice fly, and then Melancon completed the bullpen's miserable day by surrendering a walk-off homer to Alex Avila.

Valentine, who after the loss called his late-game bullpen setup "a work in progress," isn't the only manager to go out on a limb in terms of who's getting first crack at ninth-inning chores. On Saturday in Texas, rookie White Sox manager Robin Ventura called upon Hector Santiago, a 24-year-old rookie with just two major-league appearances under his belt, the first time his team had a ninth-inning lead to protect. In doing so, Ventura bypassed Matt Thornton (20 career saves, but never more than eight in one season) and Addison Reed, a 2010 third-round pick who closed at San Diego State but who has yet to save a major-league game. Santiago downed the Rangers 1-2-3 for his first career save.

Around the majors, a handful of other teams have turned to relatively inexperienced closers. In St. Louis, the world champion Cardinals have begun the year under new manager Mike Matheny with Jason Motte as their ninth-inning man; though he notched five saves last October, Motte had just 12 regular-season saves under his belt coming into the year. In Oakland, Bob Melvin anointed Grant Balfour (10 career saves coming into the year) his main man before the end of spring training. In Washington, where closer Drew Storen has begun the season on the disabled list, Davey Johnson chose to split the chores between Henry Rodriguez (two career saves) and Brad Lidge (223 career saves). In Cincinnati, the loss of Ryan Madson to Tommy John surgery led Dusty Baker to tab Sean Marshall (seven career saves) as his closer. In Colorado, longtime setup man Rafael Betancourt (27 career saves coming into the year, never more than eight in one season) finally has a ninth-inning job to call his own. Likewise in Baltimore, Jim Johnson (21 career saves, never more than 10 in a season) wears the crown.

Leaving aside the larger debate about when a manager should deploy his best reliever and for how long, or whether the save rule should even exist given managers' slavish devotion to it, does it matter how much ninth-inning experience a closer has? The way many managers, mainstream media types, and even fans discuss the role, it would seem to, and to a limited extent, the data backs that up.

I asked our stats team to help find out if the success rate of closers increases with experience, tracking both the conversion rates and the runs allowed rates for pitchers across their first 30 save opportunities in the ninth inning or later. I qualified the opportunities by inning because an official blown save (and a save opportunity) is charged to a reliever who gives up leads in the eighth inning or earlier even when his manager has no intention of using him to close out the game. Last year, Aaron Crow was charged with seven blown saves, none of which occurred during the ninth inning, and thus none of which are included in this survey. All further references to opportunities and conversion rates in this article refer only to those in the ninth inning or later.

From the beginning of 1993—the year the majors expanded to 28 teams, and a point when the game had already shifted into the Dennis Eckersley-driven, one-inning closer mode—through Friday, 767 pitchers received their first ninth-inning save opportunity. Some of them (95 to be exact) were replaced mid-inning, before they could register either a save or a blown save; for their troubles, they received that most cherished statistical marker of all, the hold. Of the other 672 pitchers, 518 shut the door successfully, a conversion rate of 77.1 percent. That's well below the period's 85.0 percent overall rate, regardless of experience. The conversion rate of the novice closers tends to increase with experience, but it still comes out to just 82.4 percent for the period over the course of those first 30 opportunities. It's hardly a smooth progression as the level of experience increases:

Furthermore, the performance of those pitchers in terms of runs allowed per nine innings given each number of save opportunities starts out higher (3.79) than the period rate for all such pitchers under those conditions (3.56) varies considerably as well. Again, the pitchers removed mid-inning who received holds have been excluded from the calculations. The overall average for the novices over the course of opportunities 1-30 is 3.68, 3.3 percent higher than the rate for all such pitchers under those conditions:

The data on both counts is more easily digested by aggregating the number of opportunities into equal-sized buckets. For convention's sake, let's group them by fives:

Number of Opportunities

Opportunities (No Holds)


































Here we can see a progression that makes somewhat more sense, at least in terms of the conversion rate; on the other hand, the 21-25 and 26-30 buckets have the two most extreme RA/9s of the set. If we aggregate once more while instead using a bucket size of seven saves, the trend takes on a striking clarity:

Number of Opportunities

Opportunities (No Holds)
























The run prevention rates don't exactly line up, but they converge to within a very narrow range. The conversion rate is what's more interesting: It's virtually identical in the buckets beyond the first one, differing by just 0.06 percentage points between the largest (8-14, 84.03 percent) and the smallest (15-21, 83.97 percent). That rate is just one percentage point behind the overall 85.0 percent rate for the period. In other words, beyond the seventh save opportunity, there's very little difference in the conversion rate between novice closers and experienced ones. The 4.1 percentage point gain from the 1-7 bucket to the rest is roughly the equivalent of one extra save for every 25 opportunities. As it turns out, that gap is also identical to the one between pitchers facing their first opportunity (77.1 percent) and those facing their second (81.2 percent)—again, one extra save out of every 25 chances. That's not nothing, but it's not much, either.

Beyond those conversion rates, it’s worth noting the level of attrition by the number of opportunities. Where 767 pitchers got one chance during the period, and 95 were pulled mid-inning without blowing the save—either due to being on a particularly short leash or being part of a more piecemeal strategy (possibly matchup-based) in the absence of an established closer—just 532 (69.4 percent) of those pitchers got a second chance, 445 (58.0 percent) a third chance, and 381 (49.6 percent) a fourth chance. In other words, slightly over half the pitchers under consideration received three or fewer ninth-or-later chances. Furthermore, just 275 (35.9 percent) received more than seven opportunities, enough to reach the point where their conversion rate wasn't appreciably worse than their more experienced peers. The graph of this data shows the start of a strikingly smooth asymptotic progression:

The number of pitchers pulled mid-inning decreases rapidly as well; just 70 were pulled amid their second opportunity, just 40 in their third, and down to 21 by their sixth, with levels skirting single digits as the number of opportunities rises. Again this lends credence to the idea that in situations without an established closer, managers might attempt a more piecemeal approach.

This quick-and-dirty study raises additional questions that I can't answer at this time because the data kitchen had closed for the night. For example, I was not able to separate out the performances of the pitchers who fell by the wayside from those who continued to receive opportunities. Some may well have pitched their ways out of those roles, as unfair as it might have been for a manager to judge them given the limited number of opportunities. Others may have simply been stepping in due to the unavailability of the regular closer, whether due to workload concerns or injuries, and thus weren't needed for more than brief stretches of a few save chances. It would be interesting to know how many of those who go, say, 0-for-3, get a fourth chance, how the opportunity and conversion rates vary given leads of one, two, or three runs, or how the total amount of major-league experience a pitcher has under his belt relates to the save conversion rate, but those questions must wait for another day.

As must those for Aceves, Melancon, and Valentine. Though Sunday's loss was a gut-wrenching one, it's too early to deem Boston's makeshift bullpen arrangement a failure. It is nonetheless appropriate to note that Aceves does fall within the lower-yielding 1-7 opportunity bucket, as do Chicago's Santiago and Washington's Rodriguez. Such pitchers, though only slightly less successful than their more experienced counterparts, appear far more vulnerable to being bumped out of the closing role for whatever reason—Rodriguez is on the clock as Storen's return looms—and so it shouldn't surprise us if their managers backtrack on their unconventional decisions in relatively short order.

Thank you for reading

This is a free article. If you enjoyed it, consider subscribing to Baseball Prospectus. Subscriptions support ongoing public baseball research and analysis in an increasingly proprietary environment.

Subscribe now
You need to be logged in to comment. Login or Subscribe
Isn't this just common sense? Ie the first few opportunities should weed out the unsuccessful, thereby increasing the success rate of the "proven" closers?
Only if a "few" opportunities is enough to distinguish successful from unsuccessful.
To a certain extent, the results jibe with common sense, sure. But the goal of sabermetrics isn't just to nod heads with common sense and received wisdom. It's to attempt to grasp the extent to which something we "know" is actually true, and/or to discover the limitations of what we know.
One goal of sabermetics is to understand basic statistical concepts, like selection bias. I'm stunned by this article.
While admittedly Cleveland-o-centric, I am relatively flabbered that this article mentions none of "Manny Acta," "Chris Perez," or "Horrifying Orthodoxy."
If I read this correctly, those with more "9th inning experience" are given more opportunities to pitch the 9th, but are no more successful (by RA) than their less experienced counterparts?
Jay, correct me if I am wrong, but I assume that the pool of pitchers in each group are not the same. In other words, some of the pitchers who got 1-5 or 1-7 saves did not go on to have any more opportunities and hence they were not in any other groups. Obviously anyone in a larger group was also in a smaller group.

If my assumption is correct, then your results are simply due to selective sampling, no? Some of the pitchers in group 1 pitches badly (blew a good proportion of their save opportunities) and were not allowed to have a save opportunity anymore, right? So, of course, the lower groups will have a lower save percentage. This tells us nothing about their talent withe respect to being able to close out a game. Nothing whatsoever.

So, if you are trying to find some evidence that experience matters, you are not going to, using this methodology. I'd have to say that this statement you made:

"The way many managers, mainstream media types, and even fans discuss the role, it would seem to, and to a limited extent, the data backs that up."

is false. The last section at least.

In fact, if you make sure that you use the same pitches in each group - in other words, if you worked backwards by only using the pitchers in the last group, you will likely see the opposite result - that these pitchers, as a group, who went on the get at least 22 or 26 save opportunities, were lucky in the beginning. You should find their save percentage in the first 5 or 10 games (group 1) to be higher than later on (the latter groups).

Maybe the purpose here is not a great new finding but in measuring where (7 opps) a new closer can be said to still be under the gun. Those who pass that gauntlet are quite likely to keep it going.
I suspect MGL is correct, though I'd really like to see that study done.

Another thing that might be interesting to look at, though it might run into some sample-size issues, is the effect of overall major league experience. I'd think that closers who had many innings of experience before their first opportunity would do better in their early opportunities, and be given more, than rookies thrust into the role (presumably mostly middling prospects/suspects for crummy teams out of the race in September, though that might be changing). It'd be nice to see the data, though.