“Doc, I had that dream again.”

“The one about the bullpen?”

“Yeah, I’m walking out of the dugout to go get my starter. We’re up 3-2 in the seventh and I’m waving for Andrew Miller.”

“I see …”

“The door opens up out in right field, but this time they start playing "Dark Star" by the Grateful Dead and Harry Pavlidis starts jogging out from the bullpen.”

“Who is Harr …”

“He’s not even left-handed, Doc! He’s not even left-handed!”

“OK …”

“And I’m yelling at the scoreboard operator guy to play something else, as if that will get Miller to come out of the ‘pen.”

“How are you feeling at this point?”

“Panicked. I spent all this time calculating the proper way to have a leveraged bullpen and all of a sudden, it’s undone because they send the wrong guy out. And I’m standing in front of 40,000 people in the middle of the ballpark and they’re all laughing at me.”

“That sounds unsettling.”

“What do you think it means, Doc?”

“Well, was Miller warmed up this time?”

“Beg pardon?”

“Was Miller warmed up?”

“Well, I don’t know, the dream started with me walking out to the mound. I mean if I’m waving for him, it makes sense that I would have warmed him.”

“Tell me what happened this time.”

“Well, Doc, Harry Pavlidis comes in and throws some warmup pitches and then on the first pitch, he gets the batter to roll the ball to the second baseman and we get out of the inning.”

“So, despite your stress, it all worked out.”

“Yeah, I guess.”

“What do you think the dream is trying to tell you?”

“Well, it’s not finished, Doc.”


“After the ground out to second, I went down the dugout stairs and I just had to figure out what was going on. I meant to go to my office, but I ended up walking and walking down this dark hallway. And I was trying to talk myself through it. How did I mess up the leveraged bullpen thing?”

“Did you figure it out?”

“No, Doc. I walked down the hallway a while, and I came to a door that had a sign on it.”

“Interesting. Could you read the sign?”

“Yes, which is weird for me. I don’t usually read things in dreams. It said …"

Warning! Gory Mathematical Details Ahead!

This has puzzled sabermetricians for years. Why don’t managers use their bullpens more optimally? We know how to quantify game leverage. We know that runners at second and third with two outs in the seventh inning with a one-run lead is a more important spot than a clean three-run lead at the start of the ninth inning, so why doesn’t the closer (or whoever is the bullpen's best reliever) come into the tight spot in the seventh?

Modern bullpen strategy has tended to be inning-based. There’s a closer for those ninth-inning close games. There’s a setup guy for the eighth. There are guys who pitch in the sixth and seventh. And there’s the LOOGY. During the playoffs, managers tend to get a little more creative, mostly borne of the fact that they start mistrusting everyone but two or three guys and they just ask those three guys to pitch all of the time.

But why not make a change? When it’s getting hairy, why not bring in the closer? Managers have embraced the shift. They are making better lineups. They’ve stopped bunting so much. Is it really too much to ask them to do this?

I think the leveraged bullpen is a fantastic theory. It makes mathematical sense. What I’ve come to realize is that it's a perfect example of something that works great in a video game and not in reality. And frankly, I blame the video games. They never make you warm the pitcher up! (And the ones that do usually have that option turned off as the default.)

But before we get to that, let’s ponder another problem. Time is linear. The seventh inning unfolds before the eighth inning presents itself, which is in turn followed by the ninth inning (and if necessary, the 10th). We want the closer to come into the point of highest leverage, but this whole linearity thing presents a problem.

I looked for the highest leverage index point that happened in the seventh inning for each team in all games from 2012-2016, with the criteria that it needed to be above 1.5. Any games that were already blowouts were discarded. If a leverage index value of at least 1.5 appears in the seventh inning, I found that this was also the highest leverage point in the game (from that point forward) in 53 percent of cases. So, yes, our manager could bring his closer in here to pitch the seventh inning, but if he did and the goal is to try to get him into the highest leverage point of the game, there’s a 47 percent chance that this isn’t it.

So, if our manager hits the “closer” button now, he very much runs the risk of having a bunch of people like me whining that he hit it too soon. If he gets into the eighth inning without using the closer and then a situation with a leverage of 1.5 or greater appears, there’s still a 40 percent chance that yet another moment will appear in a later inning (either in the ninth or in extras) that will be a greater inflection point.

Should the manager bring in the closer at the first sign of trouble because the game might be won or lost right here or should he hold off a bit and maybe bring in his third-best reliever (if he’s not already out there)? The data points to the fact that this is essentially a coin flip decision. You could make the case that probabilitstically, the managers should favor pushing the “closer button” more quickly because in the seventh inning, he’d be right 53 percent of the time, but he’s going to be wrong a lot.

Plus, this kinda assumes that our manager has psychic powers. We’re assuming that the manager could hit the “insert closer” button at this point. When I do a retrospective look at what the highest leverage point was in the seventh inning, I have the advantage of hindsight to know that a relatively big moment happened. In other words, I can whine because I get to know what happened after the decision was/wasn’t made. I gotta say, all managers should manage that way. They’d be so much better at their jobs.

The problem is that when the seventh inning dawns in these games that eventually feature that high-leverage point in the seventh, there’s no way to tell that it’s anything other than a pedestrian medium-leverage seventh inning. But if our manager wants to have the “closer button” ready for pushing, that’s when he has to make the decision to get the closer warming up. Let’s prove that.

Warmup time varies from pitcher to pitcher. Some guys need more time than others, but for our purposes, we’re going to assume that our reliever needs three batters of lead time to get fully ready. (And yes, I know that at-bats are not a unit of time.) I looked at all cases in which the seventh inning ended up holding the highest point of leverage in the game for a team and where that point had a leverage index over 1.5. I then looked back in time to see what the leverage index was three batters earlier. In 47 percent of cases, the leverage index was below 1.0 at that decision point, and in 60 percent, it was below 1.2. There are a bunch of these seventh-inning, tight-spot scenarios in which the manager couldn’t reasonably have seen it coming.

Running that analysis the other way, I looked at all cases where, at the beginning of the seventh inning, the leverage index was somewhere between 0.8 and 1.2. Three batters later, I looked at the leverage index. In more than three-quarters of the cases, the leverage index was still below 1.5.

At the time a manager needs to make the decision to have his closer ready (i.e., tell him to warm up), we’re talking about a 75 percent chance that there’s not going to be a particularly high-leverage spot for him to work by the time he's ready. Even if one appears, it’s a 50-50 shot as to whether that’s the one spot in the game where we want him to be.

So yeah, our manager could have his closer warming at the beginning of the seventh inning if it looks like it might get a little hairy. And maybe he’ll be right one of those times, but most of the time the data tell us that he’ll be wrong. And if he’s wrong, he has a closer who’s gotten up and either has to be sat back down or perhaps brought into the eighth inning of a game that isn’t all that much in dispute, which is funny because the reason that we started yelling at the manager in the first place was that he was bringing his closer into a situation that wasn’t all that important.

Why The Modern Bullpen Will Persist

Using a bullpen strategy based on pitchers being assigned certain innings is tremendously inefficient and should be gotten rid of, except for the fact that the alternative is even more horribly inefficient and should be avoided at all costs. The leveraged bullpen is a great dream, but dreams don’t always come true. There will be times when our valiant manager will sense that something vile might be happening and have a high-leverage guy ready in a spot where he normally doesn’t have a high-leverage guy ready. That’s great on the day it happens, and I’m sure we’ll all take it as evidence that he’s surely capable of such feats (so why doesn’t he do it regularly!).

The reason that inning-based or matchup-based strategies persist is that managers have a pretty good idea of when the eighth inning is going to happen. If you want to play matchups, there is a handy, reliable guide on the dugout wall that details who is coming up to bat and when. There are some smaller tweaks that managers could make to the prevailing strategy that are still inning-based, including using the closer in tie games on the road (or tie games in general), and bringing him into the eighth or ninth inning, depending on what matchups might present themselves. However, when you do the #GoryMath, they actually wouldn’t add very much value.

Instead, I think that we need a small re-think on how we evaluate managers and their bullpen usage. We’re holding them to an ideal that isn’t really achievable. It’s time to put the idea of a leverage-based bullpen strategy to bed.

Thank you for reading

This is a free article. If you enjoyed it, consider subscribing to Baseball Prospectus. Subscriptions support ongoing public baseball research and analysis in an increasingly proprietary environment.

Subscribe now
You need to be logged in to comment. Login or Subscribe
Awesome article. I wish I had the math the last time I got into this argument, because it turns out I was dead wrong.
Great job. One of the fantasy baseball games I play, Dynasty League Baseball requires two batters, not three, to warm-up relievers with additional rules on the number of times a reliever can warm-up in a game.

I wonder how much a two batter lead time affects the analysis. I'd suspect it is still very difficult to get your best relievers into the highest leverage spots.
I've thought about the two batter rule in Dynasty and wondered if it's right. Maybe Mike has built in the "stalling" factor that sometimes happens in big league games. Send the catcher out to the mound to chat. Throw over to first a few times.

On the other hand, he also counts "between innings" as a batter and I bet that's typically longer than an at bat, so there's some rounding off going on there.
You had me at "Dark Star."
You can actually thank Harry for that one. When I ran by him that I wanted to use him as a bit of comic foil, I figured that I'd at least give him the right to pick his own entrance music. (Awesome pick on his part.)
If this Pavlidis can bring some heat to the hill, then perhaps entering to "Fire on the Mountain" would be more appropriate? :)
So the real market inefficiency is finding relievers who can warm up fast?
And who can pitch like Andrew Miller.
There's a small bias in your calculations. Doesn't the 47% incidence of a higher leverage point after the 7th inning have something to do with inferior relievers pitching in the 7th?

It would also be interesting to ask the question a little more broadly: what are the situations in which a high leverage situation (defined as, say, at least 90% of the highest point in the game)is most likely to appear 3 batters hence? Are there any good predictors?
I had that thought too, although didn't write about it for length concerns. I agree that it is a bias and that it is a small bias.

We've seen in the past that "save situation" conversion rates aren't all that different for teams in the seventh inning as the ninth. The ninth inning guys are better, but not by enough to really move the needle on this one.
A first-order predictor might be where you are in the batting order, platoon advantages, etc. If you're facing the leadoff hitter, the chance of trouble three batters hence is higher than if you're facing the #7 guy. It's my understanding that Leverage Index doesn't incorporate the abilities of individual hitters and pitchers.
You are correct that LI doesn't consider any of that, but rarely do our analyses where we whine about managers include that either!
Wha dad you mean? Video games always have to pitcher warm up.
What do*
Some of them don't. Some of them, you just click on Andrew Miller and suddenly, he appears on the mound ready to go!
Ohhh, I don't think I've seen an article on what games the staff are playing for fun and what they love about each one. Of course this could be a dangerous article because I don't need to get addicted to more baseball!
Your piece makes me wonder if there are *some* occasions where it would be reasonable for managers to be a bit more aggressive with warming up their best relievers mid-game. It seems that the main reason against against aggressive bullpen use is that warming up takes a while, and the bullets you spend in the bullpen count just the same whether you get into a game or not; you can't just have relievers warming up 130 times a year.

But what about certain games where the bullpen is fresh and the opportunity cost for warming a reliever is relatively low? Let's say that you got a CG from your starter on Friday, used only the LOOGY and the mop up guys on Saturday, and have an off-day Monday: do you think managers should be looking at that Sunday game as an opportunity to take a playoff mentality with their 'pen? Or is the risk of alienating your seventh inning guy not really worth it?

Great stuff Russell.
I wouldn't rule it out. One thing that I thought of was that maybe having the closer start getting ready at the beginning of the eighth inning in a one-run game might make sense. If the situation gets hairy, he's ready. If not, he's coming into the ninth anyway.
I guess I'm thinking about it a different way. 47 vs 53 might not seem like a huge difference but it's playing those kinds of odds that turn bad teams into good teams. Platoon decisions are made based on similar splits.

Also, by putting in your best reliever i.e. closer in that 7th inning, you're guaranteeing your best pitchers did get into the game.

There's also the fringe benefit that if the game becomes a blowout after the closer put out the fire, that a mid-long reliever can be used for multiple innings, saving the other short relievers in the bullpen for games later on in the week.
It isn't just the 47 vs. 53 here that we have to contend with. That's just "is a critical seventh inning situation likely to be the most important situation of the game?" You also have to consider the fact that the manager has to sniff out that there might be a key situation that's coming up, in advance of when it appears, and we've seen that those don't usually give a lot of warning.
Just great stuff. Proper use of the best relief pitcher has been my favorite topic since forever. I go back to the days of Hoyt Wilhelm, Joe Page, four man rotations and 8 or 9 man staffs. Nobody pitched one inning and the super reliever was Hall of Fame bound. Wilhelm, Gossage, Fingers, Sutter were all expected to face the high leverage situation in the later middle innings then finish the game. For this reason, Francona's brilliant usage of Miller, not just in the post-season, but immediately upon his arrival in Cleveland was right up my alley. Of course it helps if you can call on Cody Allen in the 9th and not Joe Kelly.
Bringing the closer on at a tight spot in the 7th used to be typical. Firpo Marberry, one of the first real closers often came in in the 5th or 6th. Terry Francona's usage of Andrew Miller was a creative attempt to return to those golden days of yesteryear.

What changed this was the save rule. The current system is designed to maximize the number of saves. Saves mean money in the bank for closers and their agents. To get to the leverages bullpen, we need to redefine the save rule so it goes to the pitcher who gets the highest leverage out, whether or not he finishes the game.
"The current system is designed to maximize the number of saves. Saves mean money in the bank for closers and their agents."

You'd think, then, that there would be instructions from the front office *not* to manage for saves, so as to reduce arbitration/free agent expenditures...
Not necessarily so. More saves means more wins, and more wins means money in the bank for owners. Most teams have one closer who garners most of the saves. For general managers, this allows them to justify a big raise for the closer, but keeps the other relievers cheap. So the save rule has some benefits for management, too.
It also behooves you to keep your best players happy. Or create value for future trades if you are Billy Beane (see Koch, Billy).
This was a fantastic article.
Dire Straits
One potential exception to this relates to something I've been thinking about Otani. This is all theoretical, BUT:
There are many questions about Otani but one of the most interesting is how teams may use him. This plays into which team (and even which league, due to the DH- I won't go into those arguments) he will end up with.

Anyhoo, this is one of the things I think about as I'm driving around... thought I would burden you guys with some thoughts and hope you might actually enjoy either reading or responding or both...
He's an outfielder, throws right bats left.

It seems to me like the best use, despite all of the AL-because-the-DH boosters, would be on an NL roster.

Make him your starting LF. He plays every day. He could relieve for one batter as needed by switching places with the pitcher. This would be especially helpful with 2 outs and/or the pitcher due up soon.

A top short reliever appears in around 70 games and pitches to about 250 batters per year. Most of those are high-leverage at-bats, late inning close game, etc.

However, high-leverage at bats, unlike "closer" appearances, come at various times throughout the game, and present really the most difficult decisions. In a late-game situation, especially with a lead, the manager just reaches for the closer. In the 5th, down 1 with 2 on and 1 out, it's a lot tougher. Having a top pitcher that you could use without costing you a move would come in handy many times in a successful team's year.

Of course, there might be other situations where the simple handedness might come into play. Lefty pitching, Righty coming up, send Otani to the mound. If the other manager pinch-hits a lefty, no problem! Lefty comes back to the mound and Otani back to LF. If no pinch-hitter, Otani stays until a lefty comes to bat then goes back to LF. The manager can either bring the pitcher back in from LF or bring in relief.

Batter quality might also be a factor. If a mediocre starter or long reliever is in, and the 3-4-5 is coming up, maybe you bring Otani to the mound just for that sequence. Obviously no guarantee that he gets them out but no moves have been spent regardless.

This would get a team pretty much full value on the offensive side, and possibly higher value on the pitching side than even a good starter or short reliever provides, given the leverage effects. PLUS you could carry one less pitcher than most teams, giving you even more offensive flexibility.