Assessing Productive Outs: Little Things Mean a Little

This past April, ESPN.com's Buster Olney introduced a new statistic, Productive Out Percentage, to the baseball public. Working with the Elias Sports Bureau, Olney attempted to create a metric that would support the idea that productive outs were a key element in winning baseball. While the sabermetric community swiftly debunked Olney's creation as flawed–there's no relationship between the quality of a team's offense and its tendency to make productive outs–one question remained unanswered: how valuable are productive outs relative to other offensive events?

Productive outs, such as ground balls that advance runners, have a small benefit relative to outs that do not, such as strikeouts and pop-ups. Certainly, moving a runner over is preferable to not doing so, and over the course of 162 games, occasional bases gained can add up. What they add up to has never been quantified, but thanks to the new widespread availability of play-by-play data, however, we now have the opportunity to do so.

Defined simply, a productive out is any out on which a runner advances. Its run value, then, can be discerned from Baseball Prospectus' Expected Runs Matrix. Consider the most common bunting situation–runner on first with no outs–using 2002 data:

Base/Out    Exp. Runs
1st/0 out    .8998
2nd/1 out    .6904
1st/1 out    .5407

As its value lies in runs added above an unproductive out, the productive out here has a run value of .1497.

We can now look at the play-by-play data for the most recently available seasons, 1999-2002, using Ray Kerby's Astros Statistical Software. We need the following pieces of information for each batter/team:

• Runners on each base when an out was made
• Runner advancement on the play
• Outs in the inning
• GIDP

The first team mentioned in any discussion of productive outs is the 2002 Angels, so let's use them as an example. That lineup made the first out of the inning 322 times with a runner on first. 91 of those runners (28.3%) advanced, versus the league average of 20.4%, meaning the Angels advanced about 25 runners more than an average team would have. The run value of moving a runner from first to second was .1497, which equates to 3.8 runs.

Applying the various run values to other bases and double plays, the Angels come out as 11.5 runs better than average with no outs, and +13.1 runs with one out. Last step: the Angels had 1,522 opportunities to make productive outs, compared to the league average of 1,433. This brings their adjusted total to +23.2 runs, more than ten runs ahead of any other team that year (Pirates at +12.8), and 37 runs ahead of the bottom-dwelling Brewers (-14.1).

Here are the ten best and worst team totals from 1999 to 2002:

Team      Year   Adjusted Runs Above Average
Angels    2002            +23.2
Mariners  2001            +21.7
Royals    2000            +19.1
Rockies   2000            +14.7
Mariners  2000            +13.3
Pirates   2002            +12.8
Brewers   1999            +12.2
Phillies  2001            +12.2
Pirates   1999            +10.6
Expos     2002            +10.4

Team      Year   Adjusted Runs Above Average
Indians    2000           -19.5
Red Sox    2001           -18.8
Devil Rays 2001           -17.6
Devil Rays 1999           -16.6
Expos      2001           -16.3
Astros     2000           -15.6
Yankees    2000           -15.5
Padres     1999           -14.7
Brewers    2002           -14.1
Tigers     2000           -12.4

While each list has a World Series winner (2002 Angels and 2000 Yankees), the overall group quality differs greatly. Our best productive outmakers have a combined .531 winning percentage, with three 90-win teams and no 90-loss teams. The worst productive outmakers have a .457 winning percentage, one 90-win team, and five 90-loss teams. A striking difference, but we must be wary of the small sample size. Across all four years, the correlation between runs from productive outs and winning percentage is just .16, not even close to being significant.

How about the players? Here are the top and bottom five in adjusted runs for each year (minimum 100 opportunities):

1999
          Best                          Worst
Player            Adj. Runs    Player            Adj. Runs
Matt Lawton         +8.8       Ivan Rodriguez      -6.5
Cristian Guzman     +7.2       Reggie Sanders      -5.4
Joe McEwing         +7.1       Magglio Ordonez     -4.9
Neifi Perez         +6.6       Darryl Hamilton     -4.8
Carl Everett        +6.1       Charles Johnson     -4.1

2000
          Best                          Worst
Player            Adj. Runs    Player            Adj. Runs
Mark Grace           +8.0      Brad Ausmus         -8.8
Johnny Damon         +7.4      Scott Brosius       -8.3
Chris Singleton      +7.3      Tim Salmon          -7.9
Warren Morris        +6.7      Fernando Tatis      -7.4
Darin Erstad         +6.6      Juan Gonzalez       -7.2

2001
          Best                          Worst
Player            Adj. Runs    Player            Adj. Runs
Roger Cedeno        +8.5       Dante Bichette      -8.9
Adam Kennedy        +7.2       Matt Williams       -8.4
Ricky Gutierrez     +5.7       Placido Polanco     -7.2
Bret Boone          +5.6       Vinny Castilla      -6.6
Julio Lugo          +5.5       Derek Jeter         -6.6

2002
          Best                          Worst
Player            Adj. Runs    Player            Adj. Runs
Gary Matthews       +7.8       Scott Rolen         -7.3
Jack Wilson         +7.2       Brad Ausmus         -6.7
Johnny Damon        +6.4       Torii Hunter        -6.4
Edgar Martinez      +5.9       Jorge Posada        -6.2
Jose Valentin       +5.0       Mike Piazza         -5.9

• Derek Jeter? Yes, Derek Jeter. Don't tell Tim McCarver, but Jeter's totals for the four years were +2.8, -0.4, -6.6, -1.2.
• Brad Ausmus leads the way as catchers perform particularly poorly at making their outs productive. And you didn't think Ausmus could look any worse.
• Barry Bonds: +3.0, +1.5, +2.6, +6.4. He would have been on the leaderboard for 2002 if he only made outs more frequently.

To analyze whether making productive outs is a repeatable skill, I compared a player's adjusted runs with no outs to his adjusted runs with one out (minimum 50 total opportunities), giving us 1,336 player-seasons. Here is a scatter graph of those tandems:

Can you detect any pattern? I sure can't. The graph yields an r-squared of .01, which is, well, meaningless. A player who thrives at making productive outs with nobody out is just as likely to bomb miserably with one out as he is to excel.

So what does this mean? The overall impact of productive outs still places it firmly in the "little things" category. But the fact that the Angels picked up two wins in 2002 due entirely to their outs being less bad than everyone else's is pretty impressive. That's the equivalent of picking up a six-win player at the trading deadline.

That said, there is simply no evidence to support the notion that making productive outs is a legitimate, repeatable skill. Nor is their any reason to believe that they are the key to winning games. The productive out is to baseball what a lab fee is to a college term bill: sure it's there, but those big numbers at the top still trump everything else.

—

Anthony Passaretti can be reached here.