March 13, 2012
On Runs, Wins, and Two Types of Leverage Index
Believe it or not, most of our writers didn't enter the world sporting an @baseballprospectus.com address; with a few exceptions, they started out somewhere else. In an effort to up your reading pleasure while tipping our caps to some of the most illuminating work being done elsewhere on the internet, we'll be yielding the stage once a week to the best and brightest baseball writers, researchers and thinkers from outside of the BP umbrella. If you'd like to nominate a guest contributor (including yourself), please drop us a line.
Adam Dorhauer is a third-generation Cardinal fan from Missouri now living in Arizona. He has contributed to the Hardball Times Annual and occasionally blogs about mathematics and baseball at 3-DBaseball.net.
Baseball, at its core, is about runs. The offense scores runs, the defense prevents them. Whoever scores more of them wins. Of course, runs are shorthand for wins (which is what we really care about), but wins are a team measure, a combination of the efforts of the pitchers, fielders, and hitters. Runs scored and allowed, however, divide cleanly between the offense and defense, making them an ideal currency for measuring production in baseball. Runs are direct measures of team offense and defense in a way that wins are not.
Not all runs are created equal, however. A run scored late in a tie game has a larger impact on a team’s chances of winning than a run scored in a blowout, for example. As a result, teams try to maximize not only the number of runs they score and prevent, but also the impact of each run.
This is quantified by Leverage Index. Leverage Index is a measure of how much an average play will change the team’s win expectancy given the game-state (typically the inning, score, men on base, and number of outs), and it is an important tool for getting the most value out of players. You want your best players and performances in high-leverage situations if you can help it. A team can’t control what situations its starters see over the course of a game, but substitutions can be tailored to the game-state. As such, LI becomes an important consideration for the usage of relief pitchers and pinch hitters.
A typical closer, for example, doesn't pitch more innings than most other full-time relievers. However, a closer can see an LI of about 2.0, meaning he is pitching in situations where an average play will swing the team's win expectancy twice as much as in an average situation. By using their best relievers in high-leverage situations, teams can turn more of those large swings in their favor and magnify the ace reliever’s impact.
Likewise, when a pinch-hitting opportunity comes up, a team must decide whether to use its best bench bat or save him for a potentially more crucial situation later. Just as teams don’t want to waste their closers in the fifth inning when they could have double the impact in the ninth, teams don’t necessarily want to go to their best pinch hitters in low-leverage situations.
There are actually two types of Leverage Index. In addition to LI, which is based on swings in win expectancy, there is base-out Leverage Index (boLI), which is based on swings in run expectancy (RE depends only on the base-out state, hence the name). In general, high boLI situations will tend to be above-average LI situations, but that is not always the case, since the inning and score can have a larger impact on LI than the base-out state.
When the two disagree, which should teams care more about? The obvious answer would be LI, since wins are a more important currency than runs. If you could choose between allowing fewer runs in close games but more runs overall and allowing fewer runs overall but more runs in close games, you would probably choose the former. That does not necessarily make LI the better choice, however.
It is important to realize that LI does not attempt to put a play’s impact in the context of the entire game, only in the context of the information known about the game at the time. The more likely the game is to end up close, the more important each run, and the higher the LI. Since not much is known about the final score early in the game, LI stays within a fairly tight range (that is, LI won’t commit to saying a play is especially important or unimportant without more information). The later in the game, the more information is known, so the more LI diverges based on the situation.
We can see this by looking at the maximum possible LI as the game progresses. In the third inning, LI tops out at about 3.5; in the sixth, 5.0; and in the ninth, 11.0 (those maximum values are pretty rare, but you also see more LI north of 2.0 or 3.0 in the late innings). Similarly, you get a lot more values close to zero late in the game as it becomes more apparent that the final score will not be close. The extremes that occur late in games cancel out so that, on average, the late innings have the same LI as the early innings, just spread out over a wider range.
Once a game is complete, however, you don’t care whether your runs came in the first inning or the ninth inning. If a game is lopsided early but ends up being close, the early runs are still just as important as the late runs, even though the LI for the later plays is higher. By relying only on the information known at the time of the play, LI undersells the impact of plays that happen early in close games, or at points in close games where the score was not close.
On average, a play in the first inning is not going to end up being particularly important or unimportant, given what is known at the time. Some first-inning plays will end up being very important in the context of the whole game, though, and some inconsequential. There is no way to distinguish between those eventual contexts given the limitations of LI, so LI just assigns them all the same importance.
boLI, on the other hand, does not depend on the inning or score, so it behaves the same way throughout the game. This is the crux of the difference between boLI and LI. To decide which you should use, you need to decide whether or not you care how much information you have about the final score of the game.
Consider a simplified one-game situation where you only care about your chances of winning that one game and not how your decisions affect other games. It’s the bottom of the fifth, bases loaded, nobody out. You are down by two already and want to pull your starting pitcher. Should you consider using your closer?
This is a high-boLI situation (2.3 boLI) but a below-average LI situation (.9 LI). From this situation, two things can happen. One, your offense can fail to score the runs necessary to make up the two-run deficit, in which case it doesn’t matter how many runs you prevent or how you use your closer. Two, your offense can make up the runs, in which case keeping the deficit as small as possible gives you the best chance of winning. The below-average LI reflects both of these possibilities. That is, the importance of keeping the opponent’s run expectancy down in case you do come back (which would lead to a high-LI situation) is offset by the likelihood that you won’t come back and it won’t matter anyway (which would lead to a low-LI situation).
If you wait until later in the game, the balance of these possibilities will shift. The later in the game you get without coming back, the smaller the chance of coming back gets, and the lower LI will drop. If you do come back, then the chance of not coming back disappears, and you are left with a high-LI situation.
The typical usage of closers would be to wait and use the closer if you do come back and find yourself in a high-LI situation, and otherwise simply not use him. This is important because teams are limited in how often they can use their closers. If today's game is not particularly close, then it doesn't make much difference whether he pitches an inning or not, and you want him available tomorrow in case that game is close.
Using your closer in games where each run has a large impact is more important than maximizing the number of runs he actually prevents, so teams trade giving up a few more runs (by passing up high-boLI situations early in games) for ensuring that they prevent more runs in close games (by waiting for high-LI opportunities to present themselves). Incorporating information about the score and how many innings are left for the score to change allows teams to identify the games where the closer's impact is most likely to affect the outcome of the game.
In our simplified scenario, though, you don’t care about how your decisions today affect other games. You only care about winning today’s game. How does that change how you should approach the high-boLI, low-LI situation?
Recall that there are two possibilities: your offense fails to come back and it makes no difference how you use your closer, or your offense does come back and keeping the deficit as small as possible is imperative. Since your decision is irrelevant in the first scenario, let’s focus on the second.
Using your closer in the high-boLI situation maximizes your chance of coming back, because that will minimize the deficit in runs. If your offense comes back with two runs, you need to hold the opponent to zero to have a chance. You have a better chance of doing that using your closer in the hi-boLI situation and a lesser reliever in a lower-boLI situation, even if the lower-boLI situation ends up having a higher LI. Same thing if your offense comes back with three runs and you have to hold the opponent to no more than one run, etc.
As a result, if all you care about is winning today’s game, you are actually better off just going by boLI to decide when to use your closer. This boils down to the simple requirement that in order to win, you need to score more runs than your opponent. If we assume that when you use your closer doesn’t change the expected number of runs scored by your offense or the expected performance of your other pitchers, then that means maximizing the closer’s impact on run prevention (at least relative to other relievers) gives you the best chance of outscoring your opponent and winning the game.
This is actually reflected in Leverage Index, as long as you concede that you are going to use the closer in the game; on average, you don’t get higher-LI opportunities to use your closer by waiting than you do by using him in a high-boLI situation in the fifth. You do get a higher LI if you wait and only use your closer if you come back, but those games are offset by all the low-LI outcomes where you would normally just not use your closer.
Of course, teams can’t use their closers every day, so they have a good reason to prefer LI to boLI in deciding when to use the closer. The extra information provided by the inning and score goes a long way toward making sure the games in which they do use their closers are the ones where each run prevented has a large impact. There might be some cases where a team could capitalize on the advantage of boLI over LI, such as when the closer is underworked and scheduled to pitch no matter what, or in critical playoff games where you have more days off, but even then any gains you make could be diminished by whatever cost there is to breaking the pitcher's routine.
This restriction does not apply to pinch hitters, however. You can use a pinch hitter every day without the adverse effects pitchers suffer from usage. As a result, teams are better off choosing which pinch hitter to use based on boLI rather than LI.
The key difference in using LI and boLI to make decisions about substitutions is whether you care about the context of other games, or the context of the current game alone. If you care about which games you use a player in and which ones you don’t (as in the case of a reliever who can pitch only a limited number of games), then LI includes important context that ensures that you are choosing the best games for him to make an impact. If you care only about how your decision affects the current game, however (as is usually the case with pinch hitters), then you are better off maximizing the player’s run impact using boLI.