“Give me a lever long enough and a fulcrum on which to place it, and I shall move the world”

There seems to be one baseball topic where there is agreement between the “old school” and the “new school” bullpen management. Frequently, former players-those who haven’t played in 20 or more years-or color commentators talk about the demise of the fireman and the rise of the closer, and bemoan the fact that you don’t see the likes of a Rich Gossage or Dan Quisenberry coming into the game at a critical juncture in the seventh inning any more, or only occasionally in the eighth. Similarly, the sabermetric community has shown mathematically (see Keith Woolner‘s piece in Baseball Between the Numbers), that a manager willing to break from the current mold could garner a few more wins per year by bringing in his “closer” in crucial seventh- and eighth-inning situations.

Bullpen management, however, goes well beyond simple closer/fireman usage. It’s the gamesmanship of knowing what the other manager has in his lineup and on his bench. It’s about resource management and getting pitchers the right amount of work. If we solely focus on a manager’s tactical decisions throughout the course of a game and a season, most would rank the handling of the bullpen as most important compared to things like lineup construction, managing the running game, or deciding whether or not to bunt in a given situation.

How can we measure a manager’s effectiveness at bullpen management? Simply taking an overall statistic like WXRL is a start, but within that statistic is the overall talent level of the pitcher as well. There isn’t a distinction between the manager’s decision and the performance of the pitcher. What we are trying to get is the value that the manager brings to the table given the talent that he has in the bullpen. Jeremy Greenhouse (a BP Idol entrant) did some work aligning Win Probability Added of a reliever with the average Leverage Index that they faced in the game. There’s one thing that Greenhouse’s analysis misses, and that’s the ability to exploit the lefty-righty matchup. Some managers, like Tony La Russa, are very good (or obsessive) at exploiting the lefty-righty matchup, but does he go too far and leave himself vulnerable by minimizing his available resources later in the game, or even the next day?

In an ideal world, a manager would like to have their best pitcher pitch in the highest leverage situations. One step deeper, he would like his best pitcher against right-handed batters face the highest leverage situations where a right-handed batter is up. To that end, we will reward the manager who did the best job of aligning his best pitchers to the highest leverage situations with the award named for Archimedes, the Greek mathematician who first rigorously explained the principles of the lever.

The Approach

First, to define a pitcher’s effectiveness, I calculated his wOBA (or for those sabr-ing at home, you can use your favorite statistic, such as EqA or even OPS) against both left-handed and right-handed batters. Second, for each relief situation, we assign a Leverage Index, which is based on the inning, the outs, the runners, and the score differential. For each plate appearance, we multiply the Leverage Index by the pitcher’s wOBA against that-handed batter and sum these. Dividing by the sum of the Leverage Index over all relief plate appearances gives us an Effective wOBA.

If we do the same exercise, but instead of using the given pitcher’s wOBA, we use the overall team’s wOBA against the same-handed batter. This serves as a baseline if the manager essentially drew names out of a hat based on who would be the next reliever. So if we subtract the Effective wOBA from the “random” wOBA, we get a statistic that we will call BMAR for Bullpen Management Above Random. Essentially, a BMAR of 15 says that by putting the best pitchers in the highest leverage situations, the Effective wOBA of the opponent’s hitter is 15 points worse than if the manager chose his relievers at random.

The leaders in BMAR in 2009 in each league were:

  American League           National League
Manager       BMAR         Manager       BMAR
Gardenhire    20.8         Macha         17.9
Scioscia      13.9         Bochy         16.8
Maddon        13.8         Black         13.2
Francona      13.7         La Russa      12.8
AL Average    10.3         NL Average     6.6 

For the more astute readers, you may have realized that in some ways the BMAR metric isn’t completely fair in the following two ways:

  • The greater the spread in terms of performance of a team’s relief corps, the greater the opportunity is for having a greater impact on bullpen management.

  • The more times a team is in high-leverage situations (tighter games or more extra innings), the greater the potential BMAR.

To adjust for this, we will develop an upper bound bullpen management (UBBM) that is a measure of how much potential BMAR there could be. But how do we do this?

First, we sort each of the situations against right-handed batters by the Leverage Index, then assign the best pitcher against right-handers to that highest-leverage situation. We move to the next highest-leverage situation and keep assigning our best pitcher against right-handers until we have reached the number of right-handed batters faced for that pitcher. Once we have exhausted a pitcher’s situations, we move to the next pitcher, etc. If we keep on doing this, we get the potential maximum improvement in Effective wOBA that could be achieved, assuming that each pitcher faces exactly the same number of righties and lefties. Obviously, this upper bound could likely never be achieved, as it may require numerous and sometimes illegal pitching changes (i.e., a pitcher coming out of a ballgame, then coming back in later) or forcing a pitcher to pitch in a situation when he was on the DL, but it can serve as a useful benchmark. If we then look at the ratio of BMAR/UBBM (which will be a percentage), it gives us an idea of the managers who made the most of their potential. At the end of the day, it didn’t change the standings very much in 2009:

        American League                       National League
Manager       BMAR    BMAR/UBBM       Manager       BMAR   BMAR/UBBM
Gardenhire    20.8       36.8%        Macha         17.9     30.2%
Girardi       11.5       23.5%        Bochy         16.8     27.7%
Scioscia      13.9       23.3%        LaRussa       12.8     19.6%
Wakamatsu     13.0       21.2%        Tracy          9.4     17.2%
AL Average    10.3       17.9%        NL Average     6.6     11.3%

If we look at the managers at the bottom of these rankings (let’s call them the Sisyphus Awards, continuing our allusion to the Greeks and someone who definitely did not use a lever), we witness what we will call the “Lidge Effect”:

Manager       BMAR    BMAR/UBBM
C. Manuel     -17.0     -41.0% 
Pinnella       -2.9      -5.8%
Russell        -0.1      -0.3% 
Gaston          2.4       4.5% 

A manager who has a closer whose performance immediately falls off the cliff in a given year (Brad Lidge in Philadelphia, Matt Capps in Pittsburgh in 2009) will cause a manager’s BMAR to be quite poor as the closer is continually brought out in high-leverage situations despite his poor performances.

The Award Winners

Here are the Archimedes Award winners (based on BMAR) over the last five years in both leagues:

Year  American League      BMAR     National League  BMAR
2009  Ron Gardenhire       20.8     Ken Macha        17.9
2008  Gardenhire/Hillman   22.4     Lou Piniella     17.6
2007  Ozzie Guillen        24.7     Bob Melvin       19.0
2006  Mike Scioscia        20.8     Bruce Bochy      18.5
2005  Joe Torre            30.7     Frank Robinson   21.5

Over the time frame of 2005-2009, the table below lists the managers (limited to those that have managed at least four of the five years) ranked by best BMAR. Ron Gardenhire of Minnesota takes the award for best bullpen manager by a wide margin, never finishing below eighth in the majors in any year, and having the best BMAR in each of the last two years.

Manager          BMAR   BMAR/UBBM
Ron Gardenhire   20.0     34.5%        
Mike Scioscia    17.3     32.0%
Terry Francona   16.0     25.1%
Joe Torre        14.2     24.8%
Bruce Bochy      13.6     23.3%
Ozzie Guillen    13.0     20.3%
Jim Tracy        12.5     22.6%