November 23, 2004
Assessing Productive Outs
Little Things Mean a Little
This past April, ESPN.com's Buster Olney introduced a new statistic, Productive Out Percentage, to the baseball public. Working with the Elias Sports Bureau, Olney attempted to create a metric that would support the idea that productive outs were a key element in winning baseball. While the sabermetric community swiftly debunked Olney's creation as flawed--there's no relationship between the quality of a team's offense and its tendency to make productive outs--one question remained unanswered: how valuable are productive outs relative to other offensive events?
Productive outs, such as ground balls that advance runners, have a small benefit relative to outs that do not, such as strikeouts and pop-ups. Certainly, moving a runner over is preferable to not doing so, and over the course of 162 games, occasional bases gained can add up. What they add up to has never been quantified, but thanks to the new widespread availability of play-by-play data, however, we now have the opportunity to do so.
Defined simply, a productive out is any out on which a runner advances. Its run value, then, can be discerned from Baseball Prospectus' Expected Runs Matrix. Consider the most common bunting situation--runner on first with no outs--using 2002 data:
Base/Out Exp. Runs 1st/0 out .8998 2nd/1 out .6904 1st/1 out .5407
As its value lies in runs added above an unproductive out, the productive out here has a run value of .1497.
We can now look at the play-by-play data for the most recently available seasons, 1999-2002, using Ray Kerby's Astros Statistical Software. We need the following pieces of information for each batter/team:
The first team mentioned in any discussion of productive outs is the 2002 Angels, so let's use them as an example. That lineup made the first out of the inning 322 times with a runner on first. 91 of those runners (28.3%) advanced, versus the league average of 20.4%, meaning the Angels advanced about 25 runners more than an average team would have. The run value of moving a runner from first to second was .1497, which equates to 3.8 runs.
Applying the various run values to other bases and double plays, the Angels come out as 11.5 runs better than average with no outs, and +13.1 runs with one out. Last step: the Angels had 1,522 opportunities to make productive outs, compared to the league average of 1,433. This brings their adjusted total to +23.2 runs, more than ten runs ahead of any other team that year (Pirates at +12.8), and 37 runs ahead of the bottom-dwelling Brewers (-14.1).
Here are the ten best and worst team totals from 1999 to 2002:
Team Year Adjusted Runs Above Average Angels 2002 +23.2 Mariners 2001 +21.7 Royals 2000 +19.1 Rockies 2000 +14.7 Mariners 2000 +13.3 Pirates 2002 +12.8 Brewers 1999 +12.2 Phillies 2001 +12.2 Pirates 1999 +10.6 Expos 2002 +10.4 Team Year Adjusted Runs Above Average Indians 2000 -19.5 Red Sox 2001 -18.8 Devil Rays 2001 -17.6 Devil Rays 1999 -16.6 Expos 2001 -16.3 Astros 2000 -15.6 Yankees 2000 -15.5 Padres 1999 -14.7 Brewers 2002 -14.1 Tigers 2000 -12.4
While each list has a World Series winner (2002 Angels and 2000 Yankees), the overall group quality differs greatly. Our best productive outmakers have a combined .531 winning percentage, with three 90-win teams and no 90-loss teams. The worst productive outmakers have a .457 winning percentage, one 90-win team, and five 90-loss teams. A striking difference, but we must be wary of the small sample size. Across all four years, the correlation between runs from productive outs and winning percentage is just .16, not even close to being significant.
How about the players? Here are the top and bottom five in adjusted runs for each year (minimum 100 opportunities):
1999 Best Worst Player Adj. Runs Player Adj. Runs Matt Lawton +8.8 Ivan Rodriguez -6.5 Cristian Guzman +7.2 Reggie Sanders -5.4 Joe McEwing +7.1 Magglio Ordonez -4.9 Neifi Perez +6.6 Darryl Hamilton -4.8 Carl Everett +6.1 Charles Johnson -4.1 2000 Best Worst Player Adj. Runs Player Adj. Runs Mark Grace +8.0 Brad Ausmus -8.8 Johnny Damon +7.4 Scott Brosius -8.3 Chris Singleton +7.3 Tim Salmon -7.9 Warren Morris +6.7 Fernando Tatis -7.4 Darin Erstad +6.6 Juan Gonzalez -7.2 2001 Best Worst Player Adj. Runs Player Adj. Runs Roger Cedeno +8.5 Dante Bichette -8.9 Adam Kennedy +7.2 Matt Williams -8.4 Ricky Gutierrez +5.7 Placido Polanco -7.2 Bret Boone +5.6 Vinny Castilla -6.6 Julio Lugo +5.5 Derek Jeter -6.6 2002 Best Worst Player Adj. Runs Player Adj. Runs Gary Matthews +7.8 Scott Rolen -7.3 Jack Wilson +7.2 Brad Ausmus -6.7 Johnny Damon +6.4 Torii Hunter -6.4 Edgar Martinez +5.9 Jorge Posada -6.2 Jose Valentin +5.0 Mike Piazza -5.9
To analyze whether making productive outs is a repeatable skill, I compared a player's adjusted runs with no outs to his adjusted runs with one out (minimum 50 total opportunities), giving us 1,336 player-seasons. Here is a scatter graph of those tandems:
Can you detect any pattern? I sure can't. The graph yields an r-squared of .01, which is, well, meaningless. A player who thrives at making productive outs with nobody out is just as likely to bomb miserably with one out as he is to excel.
So what does this mean? The overall impact of productive outs still places it firmly in the "little things" category. But the fact that the Angels picked up two wins in 2002 due entirely to their outs being less bad than everyone else's is pretty impressive. That's the equivalent of picking up a six-win player at the trading deadline.
That said, there is simply no evidence to support the notion that making productive outs is a legitimate, repeatable skill. Nor is their any reason to believe that they are the key to winning games. The productive out is to baseball what a lab fee is to a college term bill: sure it's there, but those big numbers at the top still trump everything else.
Anthony Passaretti can be reached here.