August 6, 2003
Lies, Damned Lies
Up until this season, my clearest memory of Jose Guillen is as the object of some very unflattering jeering in the right field bleachers at Wrigley Field. The bleacher bums are never kind to opposing outfielders, but Guillen, being young, bad, and foreign, was a particularly vulnerable target. Guillen reacted to the taunts by alternately appearing hopelessly dejected and demonstratively angry, only making matters worse. Though he got his revenge that day--hitting a home run off crowd-favorite/headcase Turk Wendell--I've always had trouble watching him play without the phrase Jo-se-do-you-suck! running warbled, drunken, Francis Scott Off-Key through my head.
However cruel, the taunting had proved prescient. Back in 1997, Guillen had time and an abundance of raw talent on his side. Bouncing between four organizations and failing to demonstrate any development, Guillen had regressed to the level of benchwarmer; his career .239 EqA entering the season was below replacement level for a corner outfielder. If not for his powerful right arm (an impressive tool, but overrated in its importance) and his much-tarnished Topps All-Rookie Team trophy, Guillen might have been riding shuttles between Louisville and Chattanooga or selling real estate instead of holding down a fourth outfielder job in the bigs.
This season, of course, Guillen has had the last laugh. Easily the most productive hitter on the Reds this year, Guillen filled in admirably for Ken Griffey Jr. Now traded to the A's, he's been charged with the Herculean task of trying to make up for an entire outfield's worth of mediocrity, salvaging Billy Beane's reputation as a deadline dealer nonpareil in the process.
But what if Guillen turns back into a pumpkin? Writes BP reader Gavin Williams:
I was just having an argument with a friend regarding Jose Guillen (I'm a big A's fan by the way). He was saying that no way is a guy with a 6-year 0.702 OPS average going to keep much of his 1.013 OPS season. He thinks Guillen is having a fluke year and expects him to be back to his low 0.700s next year. I believe that Guillen is young enough that his improvements might stick, to some extent.
Glad you asked, Gavin. In the spirit of your suggestion, I ran a search for mid-career "breakout" players who met the following criteria:
In other words, we're looking for players that were successful for at least a full season following a considerable history of failure, the pattern that Guillen has exhibited.
In order to measure productivity, we'll introduce a metric called EqRG--as you might have guessed, it's a member of the Equivalent Average family--which is calculated simply enough as:
EqRG is an analog to RC/27--it estimates the number of runs that a lineup composed of nine facsimiles of a given player would score in an average game, adjusted for park and league effects. Replacement-level players sustain an EqRG of around three. An EqRG of between four and five is average. Anything above seven is very good, and anything above 10 is fantastic.
Why the need to introduce a new metric? We're looking for a rate stat, so using straight Equivalent Runs would be problematic--we want to focus on changes in ability, not changes in playing time. On the other hand, we have EqA, which is a rate stat, but it isn't set up well for measuring change in the way that we're interested in. We could say that a player who improves his EqA by 40 points from .260 (league average) to .300 (All-Star) has become 15% more productive (40/260), but that would be misleading; the improvement that the player has experienced in terms of his ability to create runs is much greater than that. EqRG, a rate stat that stays grounded in the language of run scoring, serves both of our purposes.
In order to measure the player's improvement, we also need to establish a baseline for EqRG entering season n. This equation that follows is going to look complicated, but all it's doing is weighting a player's EqRG in the previous three seasons by two factors: his playing time, and the recentness of the season in question (the three most recent seasons are weighted in a ratio of 5:3:2). This is similar to the approach that PECOTA and most other projection systems use to create a baseline.
Remember, we're looking for players who improved their productivity by at least 50%--that is, players whose EqRG improved by at least 50% over its baseline. Since 1950, there have been 29 such players that meet our qualifications, a little less than one every other season.
Single-Season Productivity Improvement of at least 50%, Age 26-28
EqRG Name Year Age Baseline Breakout Improvement ----------------------------------------------------------- Easley, Damion 1997 27 3.2 5.9 83% Alou, Matty 1966 27 3.2 5.8 80% Christopher, J. 1964 28 3.7 6.4 74% Petrocelli, R. 1969 26 4.4 7.7 74% LeFlore, Ron 1976 28 3.7 6.3 68% Cash, Norm 1961 26 6.5 10.8 66% Mitchell, K. 1989 27 5.6 9.2 64% Rodriguez, He. 1996 28 3.5 5.8 63% Daulton, Dar. 1990 28 3.5 5.7 62% Yount, Robin 1982 26 5.2 8.4 61% Erstad, Darin 2000 26 4.7 7.5 58% Brett, George 1980 27 6.8 10.8 58% McGee, Willie 1985 26 4.7 7.4 56% Cerone, Rick 1980 26 3.1 4.9 56% Helton, Todd 2000 26 5.4 8.5 56% Karros, Eric 1995 27 4.3 6.7 55% Anderson, Mar. 2001 27 2.8 4.3 54% Busby, Jim 1953 26 3.3 5.1 54% Gruber, Kelly 1988 26 3.6 5.6 54% Kluszewski, T. 1952 27 4.5 6.9 54% Reynolds, Har. 1987 26 2.8 4.3 54% Epstein, Mike 1969 26 5.3 8.1 53% Bass, Kevin 1985 26 3.3 5.1 52% Yastrzemski, C. 1967 27 6.1 9.3 52% Lieberthal, M. 1999 27 3.9 6.0 52% Wilkins, Rick 1993 26 4.6 7.0 52% Harrah, Toby 1975 26 4.8 7.3 50% Groat, Dick 1957 26 3.6 5.4 50% McGwire, Mark 1992 28 6.2 9.3 50% Guillen, Jose 2003 27 3.5 7.0 98%
That list includes both some famous breakout seasons--the years that George Brett, Robin Yount and Todd Helton really established themselves as stars--and some famous one-year flukes like Norm Cash's 1961 and Rick Wilkins' 1993. Guillen is on pace to top them all. Only Matty Alou and Damion Easley improved their productivity in mid-career by at least 80% in a single season; Guillen is on track to improve his by nearly 100%. By this measure at least, Guillen's 2003 has been among the most remarkable seasons in modern history.
That's interesting trivia, but it hasn't really addressed Gavin's question--what we're really interested in is how much of their improvement those breakout players are able to hold onto in the future. Does the breakout season indicate that a new level of ability has been attained, or merely that a player has strung together a few hundred very good plate appearances?
To answer that, we need to introduce a companion to Baseline EqRG, which for lack of a better term we'll call Future EqRG. Future EqRG is the exact mirror image of Baseline EqRG, measuring a player's productivity in the three seasons following the breakout year, weighted based on playing time and time elapsed since season n. Specifically:
By comparing a player's productivity in seasons past, present, and future, we can see how much of the breakout is retained going forward. For example, a player that improves from four EqRG to six EqRG in his breakout season, then plateaus at five EqRG in the three seasons following, has held onto 50% of the gains that he made.
Last equation, I promise:
Here is the same group of players that we looked at before, arranged in descending order of their breakout retention percentage (my apologies if that sounds like something you'd read about in a tampon commercial).
EqRG Name Year Baseline Breakout Future Retain% -------------------------------------------------------- Bass, Kevin 1985 3.3 5.1 5.8 141% McGwire, Mark 1992 6.2 9.3 10.3 134% Reynolds, Har. 1987 2.8 4.3 4.8 127% Daulton, Dar. 1990 3.5 5.7 6.1 120% Kluszewski, T. 1952 4.5 6.9 7.3 117% Gruber, Kelly 1988 3.6 5.6 5.8 111% Alou, Matty 1966 3.2 5.8 6.0 107% Helton, Todd 2000 5.4 8.5 8.1 87% LeFlore, Ron 1976 3.7 6.3 5.8 82% Anderson, Mar. 2001 2.8 4.3 3.9 75% Easley, Damion 1997 3.2 5.9 5.1 71% Rodriguez, He. 1996 3.5 5.8 5.0 67% Yastrzemski, C. 1967 6.1 9.3 7.8 52% Groat, Dick 1957 3.6 5.4 4.5 50% Harrah, Toby 1975 4.8 7.3 6.1 50% Yount, Robin 1982 5.2 8.4 6.8 49% Busby, Jim 1953 3.3 5.1 4.2 45% Karros, Eric 1995 4.3 6.7 5.3 40% Epstein, Mike 1969 5.3 8.1 6.4 39% Lieberthal, M. 1999 3.9 6.0 4.7 36% Mitchell, K. 1989 5.6 9.2 6.6 29% Petrocelli, R. 1969 4.4 7.7 5.3 25% Christopher, J. 1964 3.7 6.4 4.0 10% Brett, George 1980 6.8 10.8 7.1 8% Cerone, Rick 1980 3.1 4.9 3.3 7% Cash, Norm 1961 6.5 10.8 6.4 -3% McGee, Willie 1985 4.7 7.4 4.3 -15% Wilkins, Rick 1993 4.6 7.0 4.1 -22% Erstad, Darin 2000 4.7 7.5 3.9 -31%Note: Figures for Anderson, Erstad and Helton include partial-season data from 2003
As you might expect, we have a full spectrum of results here, ranging from players like Mark McGwire and Darren Daulton, who continued to improve after their breakout season, to others like Wilkins and Darin Erstad, who gave back the gains they had made and then some. The median retention rate was exactly 50%. If you know nothing else, you'll do pretty well by guessing that the player will settle into a level of performance halfway between his current and previously established levels.
Looking at the data in a bit more detail:
It might be objected that this isn't really a representative sample. The breakouts of Alou, a line-drive hitter who learned improbably at age 27 to hit 'em where they ain't, and Ron LeFlore, a speed player whose major league debut was delayed by time spent in a penal institution, might seem to have little to do with that of Guillen--a hacker who suddenly started connecting with considerably greater frequency.
Ignoring the usual age-matching requirement, I ran PECOTA similarity scores for each of the breakout players against Guillen's numbers entering 2003 (to clarify, we're looking at how similar Guillen was to the other players before their respective breakouts). While there weren't many great matches, a few players turned out to be at least somewhat comparable to Guillen:
Top PECOTA similarity scores to Jose Guillen, Breakout players only
Rodriguez 43 Lieberthal 29 Petrocelli 29 Gruber 21 ------------------ Karros 16 Busby 11 Easley 11
Four players received a score of 20 or higher--those guys are highlighted in the chart below.
There's no clear pattern there, which leads us to the inevitable but frustrating conclusion: though the range of outcomes following a breakout season is very wide, we don't really know where Guillen is going to fall within it. There are many factors that motivate a sudden and dramatic shift in performance level--a change in personal habits, a change in coaching, off-season LASIK surgery, whatever--that a statistical model won't do a very good job of picking up. By definition, these players are special cases.
PECOTA, in fact, punts on the question of single-year breakouts entirely. My research in developing PECOTA indicated that--assuming that performance in previous years is weighted, calibrated, and age-adjusted properly--there is no additional accuracy gain from accounting for single-season trends in performance (for offensive players, that is--pitchers are another matter). In fact, this finding has been confirmed by other analysts.
That doesn't mean that we're completely in the dark with respect to a player like Guillen, just that his case is one in which statistically based methods of analysis hit a wall. Guillen's profile hasn't changed very much--he's still a toolsy player with a poor strikeout-to-walk ratio--it's just that he's become substantially more effective at leveraging those skills into results. The best advice for a team looking to acquire Guillen is to scout him carefully, and to sign him to a contract heavy on incentive clauses.