keyboard_arrow_uptop
Baseball Prospectus is looking for a Public Data Services Director. Read the description here.

“If Jose Valverde is not better in the ninth inning than Rick Porcello after eight innings, and Jose Valverde's a top closer, then we might as well not have a closer.”—Jim Leyland

 “Never is (Colon) your best chance, because on his best day, he’s never as good as Rivera.”—Mike Francesa

Here’s the situation: your starter has gone eight scoreless innings, but it’s a save situation. Do you trot him back out there for the ninth, or do you bring in your closer for the final three outs?

Presented with that very scenario last Wednesday and Sunday, respectively, both Jim Leyland and Joe Girardi removed their starters in favor of their closers. Mariano Rivera blew his save opportunity, while Jose Valverde converted his. Both managers ultimately presided over victories, but both drew criticism for not sticking with the starters who put their teams in a position to win.

So what's the correct decision? You go with your closer. It’s not really a question. If you, the reader, think otherwise, then you are going to be wrong more often than not. I am going to assume that you do think otherwise. That way, I can join the company of Francesa and Leyland, just as I’ve always dreamed of, and engage in a rant of my own directed against you.

Who is the better pitcher, in general, on a per-inning basis—the closer or the starter? Right, it’s the closer. Just look at their ERAs. What’s more, the starter is probably tired, and he’s probably run out of ways to get batters out. In the cases of Colon and Porcello, the starter wasn’t even that great to begin with.

Sure, you say, but the starter’s been throwing great. He’s maintained his velocity, he hasn’t made any mistakes, he’s getting stronger as the game goes on, he’s retired the guys due up each time he’s faced them. Meanwhile, the closer is an unknown commodity, and you can always bring him in if the starter gets into trouble.

And here’s what I say: How awesome would it be if I had that level of perception? If that were the case, and I could just tell from my living room that at any particular moment, a pitcher were about to pitch better than he usually does—or better than Mariano Rivera, for that matter—I would be a manager in the Major Leagues of Baseball, or at least a bookmaker or a doctor (the U.S. could always use more doctors). Because normally, pitchers who go eight scoreless are not that good in the ninth. It’s true, and I’ll get to that data in a minute.

I’m not saying that the starter hasn’t been excellent. The excellence is not what’s so hard to observe. (Mind you, I often have trouble observing it.) The starter has given up no runs in eight innings. Sure, we’ll give him that. We’ll allow that he’s been excellent. But you need a lot more than that to say that he will continue to be excellent.

Between 2000 and 2010, there were over 1700 instances in which a starting pitcher completed eight scoreless innings. In nearly two-thirds of those instances, the starter came back out for the ninth. The team allowed an average of 0.45 runs in those innings. Conversely, when the starter was pulled, his team allowed an average of 0.35 runs in the ninth.

There are sampling caveats. Teams that went to the bullpen might have been afforded better relievers. Managers like to allow starters their shot at a complete game, and therefore have motives other than run prevention.

Whatever the case, you would need your starter to be over half a run in ERA better than your best reliever to leave the starter in. Again, if you are watching a game where a pitcher has thrown eight scoreless, most of the time, bringing in a reliever is the better option in the ninth. Pinch-hitting for the pitcher is always the better option at that stage of the game, given that the objective is to win the game.

Why? Why is it that these pitchers who have pitched excellently do not outperform relievers?

Not all pitchers lose their stuff as the game wears on. The more universal phenomenon is that batters gain an advantage with each time they face the same pitcher by making “adjustments.” This is why relievers can survive with fewer pitchers than starters, because they have to face batters only once, whereas starters need to find a way to mix their pitches more as the game goes on.

This ground was covered in depth in The Book. By projecting the outcome of each plate appearance and comparing it to the actual outcome based on the time through the order, one can observe the magnitude of the times-through-the-order effect. I chose to come at this from a different angle, mainly because I didn’t want to take the time to project a million plate appearances.

The easiest mistake that people make in studying the times-through-the-order effect is looking at it on a league-wide basis and seeing that pitchers have better numbers the third and fourth time through the order than they do overall. This is obviously a sampling problem, as only the best pitchers make it through the third and fourth time through the order.

To control for this, I ran a fixed-effect regression to predict each pitcher’s linear weights based on his time facing a batter in a game. Essentially, the output showed that the average starting pitcher loses a half-a-run-per-nine in effectiveness each time through the order.

Some pitchers, like Zack Greinke, who expertly mixes his pitches, seemingly get more effective as the game goes on. Some, like CC Sabathia, who maintains his velocity throughout, show no observed split. The best—Pedro Martinez and Johan Santana and Roy Halladay and Felix Hernandez and Tim Lincecum—go through the order a third time, and even if they’ve lost something, they’re still as good as an average reliever. But those are the exceptions.

The rule is that Bartolo Colon and Rick Porcello are not the answers in the ninth inning. The rule is that we know how good Mariano Rivera and Jose Valverde are and that our powers of perception are not such that we can spot the rare starter who is better after eight scoreless than those two.

 I can’t put it better than these two did:

“You’re telling me that Joe Girardi should worry that Mariano can’t close the game…It’s the classic second-guess.”—Francesa

 “I’m glad I did it. And I’ll do it again tonight if we have the same (expletive) situation if I think it gives us the best (expletive) chance to win the (expletive) game.”—Leyland

Thank you for reading

This is a free article. If you enjoyed it, consider subscribing to Baseball Prospectus. Subscriptions support ongoing public baseball research and analysis in an increasingly proprietary environment.

Subscribe now
You need to be logged in to comment. Login or Subscribe
MHaywood1025
5/26
I think an argument against this logic would be that it's better to put yourself in a situation to win multiple games rather than one game. I'm not saying you are wrong, or that I disagree necessarily--only that I think different games may call for different courses of action. If my starter has cruised through eight and is leading by 4 or 5, I'll most likely leave him in and keep my best reliever fresh for tomorrow's game, in which a higher leverage situation may present itself.

Again, I'm not saying you're wrong, I think it's certainly something to consider, but I always am weary about making a statement like "there is only one answer".
ScottBehson
5/26
The article clearly deals only with close games ("save situations")
jgreenhouse
5/27
Sorry if I didn't make this clear. This analysis only included a team's runs allowed. Ten-run games were treated the same as tie games.

I also tried to make it clear that the objective was to win that one game. I am surprised that so many people seem to think that using a single reliever in one game will have any sort of significant impact on a team's win expectancy in following games.
MHaywood1025
5/26
^^ That should say "wary". And also, I understand a very big lead would not be a save situation, but even with a 2 or 3 run lead I might still leave in the starter.
greggborgeson
5/26
MHaywood, good points, and there are other considerations as well. Did the closer pitch yesterday? Who are the starting pitchers tomorrow (i.e., is it likely to be a close game)? How many pitches has the starter thrown today and in his last start? Will the starter get an extra day of rest before his next start because of an off-day? Has the starter been truly dominant, or have there been a lot of well-hit balls that were caught.

I don't disagree with your conclusion as a rule-of-thumb, but the dismissive tone of the article isn't appropriate, nor is the decision as simple as you present it.
bravejason
5/26
I'm still not convinced that bringing in the closer is an automatic decision. Don't I need to consider how using a reliever today will affect my ability to win a game tomorrow? Or do I simply say that winning tomorrow is of sufficiently less importance than winning today that I can just ignore tomorrow?

I think you need to go in more detail with respect to how much the lead (1, 2, or 3 runs) plays into the decision. With a three run lead, do I really need a closer? Wouldn't a lesser reliver suffice? Isn't the ideal bullpen usage all about matching the situation the quality of the reliever? If not, then why do we bother the idea of leverage? Or is a three run lead not large enough and a higher caliber pitcher is still needed? If that's the case, then why do I read so many complaints by BP authors about managers automatically bringing in the closer to finish out ballgames?

Also, I still don't understand why the starter needs to be a half run better than the reliever. Why isn't one-tenth of a run better? I know you mentioned that average starting pitcher loses a half-a-run-per-nine in effectiveness each time through the order, but if the starter will be facing the middle of the order in the 9th, then haven't we already factored in the loss of effectiveness since the we've already started that next trip through the lineup?
jgreenhouse
5/27
Sure, you can consider it, but using a reliever today won't have much of an affect on tomorrow.

Good point that I should control for the pitching team's lead.

The quality of pitcher brought back out for the ninth certainly depends on that lead. Running the numbers:

Runs Pull% Instances Pull-RA Stay-RA
0 35% 133 3.14 4.21
1 45% 174 3.41 3.30
2 60% 209 1.82 5.33
3 59% 240 3.58 3.55
4 66% 242 2.17 4.81
5 70% 202 3.10 4.98
6 68% 165 3.81 3.66
7 70% 121 5.00 2.22
8 68% 95 3.00 3.60
9 79% 66 5.79 3.29
10 86% 42 1.50 6.25
>10 75% 65 5.06 2.94

Another way to put the half-run thing, which isn't a hard rule, and I'm not even sure if this line of thinking is correct, is that the starter's talent level to start the game had to be around 0.5 runs-per-nine better than the closer's, because after eight scoreless, the average starter is half-a-run worse than the average alternative. That might not make sense.
jgreenhouse
5/27
Pull% is actually 1 minus the number listed. The bigger the lead, the more likely the starter remains in the game.
rawagman
5/26
All good points, but I think the author's main point is that just because the starter has been solid, does not mean he should be automatically left in to finish it off. Other factors should be considered, and more often than not (with many exceptions), those factors lead to the bullpen.
jgreenhouse
5/27
Cool.
ScottBehson
5/26
I'd love to see a companion piece examining whether staying with a reliever who looks great 8th vs. going to a closer for the 9th.
sebajoe1
5/26
Scoot B-You make a great point. However, as long as saves are a statistic that makes releif pitchers large sums of money, managers will never manage their bulpens in that manner.
NYYanks826
5/26
You have to take pitch counts into account also.

Both pitchers were under 90 pitches. I believe that Colon was at at 87, and Porcello was at 84 when they got taken out of their outings.

It'd be one thing if they were both over 100 pitches, and fatigue might be a bigger issue, but those are extremely low pitch counts, so that has to have some impact on how effective they could be expected to be in the ninth.
jgreenhouse
5/27
Don't think the data supports you here.
escapeNihlism
5/26
devoid of context the answer is obviously very clear. in Game 6 of the ALCS there is no question. over the course of a 162-game regular season certain variables can pop up that at least make the question competitive.
jgreenhouse
5/27
Very fair.
Oleoay
5/26
Besides pitch counts, I think it's important to look at the quality of the game pitched so far, especially from a K/9 perspective. If a starter's gone 8 innings but given up a lot of hits and/or walks without a lot of strikeouts, I'd lean more in favor of bringing in the closer. If he's gone 8 innings and had a WHIP around 1 and struck out 6 or more batters over those 8 innings, I'd be more likely to keep him in. I'd also consider the schedule and offdays too before making a decision.
KaiserD2
5/27
I think this makes perfect sense. The closer is usually a better pitcher than the starter. However, I think that also illustrates the insanity of using such a good pitcher for a maximum of one or occasionally two innings a game. You will never convince me that Mariano Rivera could not have contributed more wins to the Yankees as a starter. I would be interested in your comments on this.
jgreenhouse
5/27
I have no idea how good Mo would have been as a starter. He failed as a starter, which is why, he, and mostly all other relievers, are moved to the pen. He probably could not have survived with the same one-pitch repertoire as a starter. The best starters are more valuable than the best closers, but it's tough to know if Mo would have been one of the best starters.
Deadheadbrewer
5/27
Maybe I'm missing the tongue-in-cheek aspects of this piece (if they exist), but the tone toward the reader seems rather harsh at the beginning, especially since I'm paying to be scolded.
jgreenhouse
5/27
Just a writing device, forming an imaginary argument against the reader. No offense intended.
bravejason
5/27
Since my browser always loads the webpage with errors, I can't reply to your reply to my earlier comment, so I'll do it here.

In response to my inquiry about why the starter had to be a half-run better, not, say, one-tenth of a run better than the reliever, you replied "Another way to put the half-run thing, which isn't a hard rule, and I'm not even sure if this line of thinking is correct, is that the starter's talent level to start the game had to be around 0.5 runs-per-nine better than the closer's, because after eight scoreless, the average starter is half-a-run worse than the average alternative. That might not make sense."

What you say here makes sense in that it tries to account for the loss of starter effectiveness over time. What I read your article to say was that going into the last inning the starter needed to be a half-run better. That's what I didn't understand.