CSS Button No Image Css3Menu.com

Baseball Prospectus home
  
  
Click here to log in Click here for forgotten password Click here to subscribe

Premium and Super Premium Subscribers Get a 20% Discount at MLB.tv!

<< Previous Article
Premium Article Prospectus Matchups: M... (06/05)
<< Previous Column
Prospectus Toolbox: To... (05/22)
Next Column >>
Prospectus Toolbox: Ne... (06/12)
Next Article >>
Premium Article Wait 'Til Next Year: M... (06/05)

June 5, 2007

Prospectus Toolbox

Reliever Tools

by Derek Jacques

Welcome back to Prospectus Toolbox. I spent the first two editions talking about the replacement level, focusing on two stats-VORP and WARP-primarily as they apply to position players. Both statistics can be and are also used to measure pitching performance, and while this time out we're going to focus on pitching metrics, we'll be particularly looking at the ones that deal with relief pitchers.

Relievers have played an increasingly large role in the game. This year alone, relievers have pitched 33.8 percent of the innings played in Major League Baseball. Back at the dawn of our play-by-play database, 1959, relievers only contributed 27.7 percent of the total innings pitched. By 1979, the figure had risen to 28.9 percent, and by 1999 it was 33.7 percent, or around where we're at today. Despite the growing emphasis on relief pitching, the mainstream statistics used to describe relievers' success remain rather crude. On Sunday, John Perrotto bemoaned the lack of consensus as to what number of saves should open the gates to Cooperstown. Part of the reason that there is no consensus is because the save is a severely flawed statistic. Despite the statistic's name, saves are not necessarily awarded to the reliever who contributes to his team when it needs the most help, but rather to the reliever who finishes the game with a small lead. The statistic's influence has resulted in managers sometimes using their best relievers in accordance to the rules governing saves, rather than the rules governing common sense.

If saves are such a bad statistic, then what tools should we use to evaluate relievers' performances? When evaluating offense, we tend to emphasize components (hits, walks, home runs, outs made) over results (runs, RBI). That's because we're usually trying to isolate one hitter's performance from those of the batter before him and the batter behind him. The pitcher, on the other hand, stands alone on the mound, and for better or for worse, we tend to blame him for anything that happens in the game while he's up there. The best-known pitching stat, earned run average, is results-oriented: it doesn't matter if a pitcher's earned runs come on solo homers or in a soft barrage of singles and walks, all that matters is how many earned runs have scored. However, ERA isn't the ideal statistic for relievers, either, in large part because relievers often come into the game with another pitcher's runners already on base, and they sometimes leave runners of their own on base for the next reliever on their team to deal with. Any good reliever evaluation stat would have to account for how well the reliever prevents those inherited runs from scoring, and what sort of situation he tends to leave for the next reliever on his team to clean up.

The advanced relief statistics we use here take an extremely result-oriented approach to a relief pitcher's contributions. Using different methodologies, these statistics look at the game situation when the reliever enters the game, and compare that to the game situation when the reliever leaves the game (or the game ends), without much concern over how we got from Point A to Point B. That comparison creates a positive or negative value that represents how well (or poorly) the reliever helped his team by preventing runs from scoring.

One reliever statistic which you might be familiar with because it's used extensively on this site is WXRL. WXRL is a metric developed by Keith Woolner which is based on a Win Expectation framework. Win Expectation has its own statistical report (the Win Expectancy Matrix), and is a pretty complicated topic, so I'll give you a bare-bones, no-math explanation. Win Expectation breaks down each game situation-inning, score, number of outs, number of runners on base, and which bases they're on-that occurs in the major leagues, all to measure how the transition from one situation to another alters a team's chance of winning the game. So, within this framework, a pitcher who enters a game in a classic "save situation"-ninth inning, three-run lead, bases empty-increases his team's win expectancy if he makes it through the ninth without giving up the lead, albeit by a small amount, since the chances of victory with a three-run lead were pretty good to begin with. If the pitcher comes into the ninth with a one-run lead, no outs, and two men on, and gets out of the jam, his contribution to the team's win expectancy is considerably greater.

In WXRL the raw change in Win Expectation (the "WX" in WXRL) from the reliever's entrance to his departure is adjusted for the replacement level (the "R" in WXRL) and for strength of the opposing lineup (the "L"). Here are the current WXRL leaders:


Pitcher           W   L  SV  BS  IP     RA+    WXRL    LEV
Takashi Saito     1   0  16   0  24.0   3.07   2.904   1.86
Francisco Cordero 0   0  21   0  23.1   2.11   2.702   1.49
Al Reyes          1   0  14   0  25.2   2.21   2.627   1.73
Rafael Soriano    1   0   5   0  25.0   1.85   2.600   1.53
Scott Linebrink   1   1   1   1  25.2   2.14   2.268   2.23

The qualities of WXRL to keep in mind:

  • WXRL is a counting stat, measured in wins.
  • All WXRL values are above replacement level-any below replacement-level performances have negative values.
  • WXRL accounts for inherited and bequeathed runners.
  • Leverage matters. A reliever who comes into high-leverage situations will accumulate more WXRL than a player who pitches just as well in garbage time.
  • WXRL can be found in the Relievers Expected Wins Added Report, or in the Pitcher Season, Pitcher Team Year, or Team Pitching custom reports.

Now, WXRL is quite popular around here, so much so that it has overshadowed a previously favored in-house relief pitcher statistic, Adjusted Runs Prevented (ARP). Originally developed by Michael Wolverton, ARP measures a reliever's contribution using the Run Expectancy Matrix. For each base-out situation, the Run Expectancy Matrix tells us how many runs are expected to score in that inning. So far in 2007, if you have runners on first and second and no outs, it's expected that the team on offense will score 1.45354 runs. If the team on offense successfully sacrifices, that creates a situation with men on second and third and one out; at that point, the number of runs expected to score increases slightly, to 1.45951. A relief pitcher who takes over that second and third, one-out situation, and only allows one run to score, would be considered to have prevented 0.45951 runs from scoring over what an average relief pitcher would have done. Those prevented runs are adjusted for park and league to make ARP.

How is this different from WXRL? The Win Expectancy and Run Expectancy matrices are calculated in completely different ways, so the two statistics are measuring different things. The biggest difference is that in a Run Expectancy context, every inning the reliever works is created equal-it doesn't matter if the pitcher's team is tied or trailing by ten runs, the value of the reliever's performance is based on the base runners and outs when he takes the mound that inning.

You can see the difference by checking out the league leaders in ARP. While the top of the WXRL list is dominated by closers, the ARP list is a little more diverse:


Pitcher         W   L  SV  BS  IP     RA+    ARP    LEV
Pat Neshek      3   0   0   0  27.2   3.56   16.0   1.30
Heath Bell      0   2   0   1  33.1   4.16   15.2   1.34
Matt Guerrier   1   2   0   0  33.2   2.90   14.4   1.10
Jeremy Accardo  1   0   7   1  25.1   4.39   13.9   1.42
J.J. Putz       0   0  14   0  25.1   3.16   13.6   1.14

As you can see in the last column, marked "LEV," all the ARP leaders have been used in lower-leverage situations than the WXRL leaders. Some qualities of ARP to remember:

  • ARP is a counting stat, measured in runs prevented from scoring.
  • ARP is measured against the performance of an average pitcher, not a replacement level pitcher.
  • ARP accounts for inherited and bequeathed base runners.
  • The score of the ballgame and lateness of the inning are irrelevant.
  • ARP can be found in the Relievers Expected Runs sortable report, or in the Pitcher Season, Pitcher Team Year, or Team Pitching custom reports.
  • Further Reading

    Michael Wolverton, What the "R" Column Doesn't Tell You: Evaluating Relievers by Prevention of Expected Runs: This article sets out the technical details of ARP and reliever run expectancy.

    Keith Woolner, "An Analytical Framework for Win Expectancy," in Baseball Prospectus 2005 (Workman, 2005): This article introduces the Win Expectancy framework used by Baseball Prospectus, and the sets out the technical aspects of the WXRL statistic.

    Keith Woolner, "Are Teams Letting Their Closers Go To Waste?" in Baseball Between the Numbers (Basic Books, 2006): This article discusses both the run expectation and win expectation frameworks, putting both in the context of the optimal time for teams to use their ace relievers.

    Derek Jacques is an author of Baseball Prospectus. 
    Click here to see Derek's other articles. You can contact Derek by clicking here

    0 comments have been left for this article.

    << Previous Article
    Premium Article Prospectus Matchups: M... (06/05)
    << Previous Column
    Prospectus Toolbox: To... (05/22)
    Next Column >>
    Prospectus Toolbox: Ne... (06/12)
    Next Article >>
    Premium Article Wait 'Til Next Year: M... (06/05)

    RECENTLY AT BASEBALL PROSPECTUS
    Premium Article What You Need to Know: September 2, 2014
    Premium Article The Call-Up: Daniel Norris
    Premium Article The Call-Up: Maikel Franco
    Premium Article Transaction Analysis: Bo Gone
    The Future
    Premium Article The Call-Up: Joc Pederson
    Premium Article Monday Morning Ten Pack: September 2, 2014

    MORE FROM JUNE 5, 2007
    Premium Article Future Shock: Top 50 Draft Prospects
    Premium Article Prospectus Today: AL All-Stars
    Premium Article Wait 'Til Next Year: My Guys on Draft Day
    Premium Article Prospectus Matchups: Midweek Miscellany

    MORE BY DEREK JACQUES
    2007-06-26 - Prospectus Toolbox: A Secret Affair
    2007-06-19 - Prospectus Toolbox: Dying Quails and Pitcher...
    2007-06-12 - Prospectus Toolbox: Neutral—Like Switzerland...
    2007-06-05 - Prospectus Toolbox: Reliever Tools
    2007-05-22 - Prospectus Toolbox: Toolbox at the Mailbox
    2007-05-15 - Prospectus Toolbox: Value Over Replacement P...
    2007-04-03 - Prospectus Game of the Week: Tampa Bay Devil...
    More...

    MORE PROSPECTUS TOOLBOX
    2007-06-26 - Prospectus Toolbox: A Secret Affair
    2007-06-19 - Prospectus Toolbox: Dying Quails and Pitcher...
    2007-06-12 - Prospectus Toolbox: Neutral—Like Switzerland...
    2007-06-05 - Prospectus Toolbox: Reliever Tools
    2007-05-22 - Prospectus Toolbox: Toolbox at the Mailbox
    2007-05-15 - Prospectus Toolbox: Value Over Replacement P...
    More...

    INCOMING ARTICLE LINKS
    2008-06-24 - Prospectus Toolbox: Translating Scales: Reli...
    2008-02-06 - Premium Article Building a Better Bullpen
    2007-10-09 - Prospectus Toolbox: On Awards and Statistic...
    2007-07-24 - Premium Article Prospectus Hit and Run: Debut
    2007-07-17 - Prospectus Toolbox: Even Further Reading