“Species arise in a geological moment-the punctuation (slow by our standards, abrupt by the planet’s). They then persist as large and stable populations on substantial geological watches, usually changing little (if at all) and in an aimless fashion about an unaltered average-the equilibrium.”
–Stephen Jay Gould, “Opus 200”, Natural History, August 1991
In Part 1 of this two-part column we looked at three interesting research presentations given at the 36th annual SABR convention. In review, those included a study evaluating managers by Chris Jaffe, a look at the performance of players in the “walk year” of their contract by Phil Birnbaum, and Sean Forman’s quantitative look at a catcher’s ability to stop wild pitches and passed balls.
To continue that theme we’ll look at three more presentations, and we’ll conclude with a few corrections to my column from last week related to variation in salaries within teams.
Punctuating the Closer’s Equilibrium
Probably the presentation with the longest and most interesting title was given by Jeff Angus, titled “Punctuated Equilibrium in the Bullpen: The 2005 World Champion Chicago White Sox Blend Sabermetrics & Sociology to Deliver a Successful Innovation.” Some readers may be familiar with the work of Angus from his popular Management by Baseball blog and the publication of his new book of the same title which he signed for convention attendees. He also has done some writing for BP.
In his presentation, Angus took a two-track approach, and first presented a history of the evolution of the bullpen since the late 1970s, and then discussed how that evolution played into the 2005 White Sox’ use of their bullpen and their successful “Closer by Situation” strategy.
In the first section, Angus takes a look at the role of closer as it has developed over the last 30 years and, interestingly, takes issue with the most frequent theory of its origin. That “origin myth,” as Angus calls it, involves Herman Franks’ use of Bruce Sutter in the late ’70s as the founding combination, followed by Tony LaRussa’s deployment of Dennis Eckersley in the late 1980s.
That theory is only “partially true”; Angus views the development of the role as akin to Gould’s biological evolutionary theory of punctuated equilibrium (PE). Under that view, there was much experimentation initially, followed by establishment of the dominant model and, finally, intensification where the model’s limitations become apparent. That path from initiation to dominance for the “Clean 9th” closer in Angus’ terminology (meaning a closer who is brought in almost exclusively to start the ninth inning in a save situation) runs as follows: experimentation in the models used by Franks and LaRussa, to establishment with Jeff Torborg‘s use of Bobby Thigpen in 1990, to intensification as Jim Fregosi took it to the extreme in his use of Mitch Williams in 1993. That blind adherence to the dominant model ultimately cost the Phillies dearly. Williams struggled in the NLCS and in Game Four of the World Series before imploding in Game Six. Fregosi could only watch, not having developed any other options.
To back up this idea, Angus provides a variety of interesting tables in his written version of the presentation (which you can obtain by contacting him directly). For example, he shows that both in 1979–after Franks supposedly had the idea for using Sutter as modern closer–and 1984, Sutter was still pitching just one inning in a minority of games.
Appearances by IP <1 1 1-1/3 1-2/3 2 >2 --------------------------------------------------- 1979 7 14 5 10 17 10 1984 5 15 10 29 6 6
Interestingly, Angus’ view contradicts that of another presenter, Mike Carminati. Carminati’s presentation was titled “Welcome to the Halls of Relief: An Historical Review of Relief Pitching,” and it pinpointed the origin of the modern closer to August 19, 1978 when Sutter was hit hard, and when afterwards Franks is said to have changed his usage pattern. In fact, Angus shows that Sutter was not used any differently after late 1978, as shown above.
To support the pattern of (PE) he included several interesting tables which I’ll condense in the following table.
Percentage of Appearances
Name Year Not as Exactly 1 IP Used Before
A Closer As a Closer 9th Innning
----------------------------------------------------------
Eckersley 1988 23% 46% 35%
Eckersley 1989 10% 54% 33%
Thigpen 1990 5% 71% 15%
Williams 1993 2% 78% 0%
In the second part of the presentation, Angus looks at how the White Sox overcame the dominant model and perhaps became the vanguards of a new model by employing a “Closer by Situation” model that succeeded where the Red Sox of 2003 and Cubs of 2005 had failed. Angus credits the Sox with being willing to make changes in their bullpen usage and, perhaps more importantly, create an environment where those changes were accepted in a world where hierarchy and the psychological comfort of established roles are king. In Angus’ words from his paper:
Their rational-in-design, revolutionary-in-its-sabermetric-underpinnings idea was to make each reliever understand the truth that he was equally-responsible for the team’s success as the reliever who appeared in an inning named for a different number. As Guillén said, ‘If I put you in there in the seventh, close the seventh. If it’s the eighth, close the eighth. If it’s the ninth, close the ninth.'”
This idea underlines the key notion discussed in Keith Woolner’s chapter “Are Teams Letting Their Closers Go to Waste?” in Baseball Between the Numbers (BBTN) which basically is that the most highly leveraged situations often occur as early as the 6th inning and often times “closing” those earlier innings will provide greater benefit to the team in the long run. In Woolner’s study this equates to about 1.6 wins on average over the course of a season, and as many as 4.5.
Angus then illustrates the four phases of the White Sox application of the closer role in 2005 by showing how, over the course of the season, Shingo Takatsu, Dustin Hermanson, Bobby Jenks and Hermanson, and finally Jenks were phased through the 9th-inning role (although never exclusively). What I found interesting was that these transitions were not done from a state of panic or absolute necessity, but appeared planned and therefore instilled confidence.
Heaping on A-Rod
The final two presentations we’ll review together. They include Vince Gennaro’s talk titled “The Dollar Value of the Last Piece of the Puzzle” and Jonah Keri’s presentation of the chapter “Is Alex Rodriguez Overpaid?” from BBTN.
What both of these presentations had in common was the idea that in order to get at the question of whether a player is overpaid or what their compensation should be, one must examine the context, meaning the particular team and their performance expectations in the upcoming season or seasons. Some readers may be familiar with Gennaro’s work where he wrote about much of this in his article “Player Value: The Last Piece of the Puzzle”. He’ll also have a book out in early 2007 that includes these types of analyses.
What I found most interesting about both of these presentations is that the central idea they capture is different from our usual static analysis where we equate player performance with a specific number of wins, and therefore a distinct quantifiable value. These sorts of analyses are inherently more dynamic, and may help explain why it is that teams appear to sometimes pay players in excess of what their on-field contributions might indicate at first glance.
To put it succinctly, Gennaro uses WARP to estimate player value, and then applies that to the last x number of wins for a team. He then uses his model to calculate how much those last wins are worth to a particular team in terms of attendance and other revenue streams including the post season. Using this methodology, he can make a case that, for a team like the Toronto Blue Jays, a player with a projected WARP of 6.0 like B.J. Ryan (WARP3 6.3 in 2005 and projected at 8.0 in 2006) or A.J. Burnett (5.7 in 2005) would be worth $12M if the Jays win 93 games, stepping down to $4M at 83 wins. This is shown in the following chart that Gennaro used in the presentation:
This provides support for the notion that perhaps Ryan’s five-year $47M and Burnett’s five-year $55M contracts are not that out of line if the Blue Jays viewed the pair as the “last piece of the puzzle.”
As you might expect, from his team-specific win revenue models, which include a variety of factors, he finds that the sweet spot for wins is between 70 and 98, where the marginal value of each win increases dramatically. However, there is significant variation in the value of a win by market and team, with team competitiveness having the greatest impact. For example, he showed that for the Angels in 2005 the value of the three wins between 75 and 78 was $2M while three wins at 89-92 was $6.1M. He also includes post-season effects in his model (customized for team and market, of course), and noted that reaching the playoffs was indeed the “holy grail” of revenue generation, but winning a World Championship has a significant additional impact. That impact, however, is largely felt in future revenue through increased season ticket sales, increased ticket prices (prices increase an average of 10% for playoff teams), more demand for luxury suites, and additional broadcast revenue. In fact, Gennaro estimates that the 2005 World Championship was worth about $46M in future revenues for the White Sox.
He then applied this model to the 2006 postseason and concluded that the Mets and Cubs (yeah, like that’s going to happen) had the most to gain from reaching the playoffs this season at $36M and $35M respectively, while Boston and Atlanta at roughly $8M and $3M respectively would benefit the least. These estimates stem from his analysis of the Chicago market that the Cubs, unlike the Red Sox, still have room to maximize their revenue while the Braves–having reached the postseason 14 straight seasons–would see relatively little in terms of additional revenue.
In the end, the takeaway is that although it’s very difficult, teams need to understand where they are on the win-revenue curve when thinking about signing players. That same free agent can be worth vastly different amounts to different teams.
All of this should be sounding familiar to readers of BBTN. Gennaro’s talk overlaps with Keri’s in many ways, including where A-Rod is concerned. If you’ve read BBTN you’ll know that Nate Silver (the author of the study) spends most of the chapter developing three models for player valuation: the linear, the market price, and the two-tiered model, the latter of which includes both regular and post-season revenue. Like Gennaro, Silver looked at a variety of revenue streams and used regression analysis to discover the relative impact of the variables within each stream.
When all is said and done, he uses the two-tiered model to produce a general win-revenue curve very similar to the team-specific curves discussed previously, where the sweet spot is between 86 and 93 wins. His conclusions then are basically the same: the amount a team offers a free agent should coincide with the expected revenue impact he’ll have, and that impact is largely determined by where the team is (or projects to be) on the win-revenue curve. As Keri pointed out, Silver also makes allowances that signing a free agent may be viewed as an additional revenue enhancer; think the real estate boom around The Ballpark at Arlington predicted by Rangers owner Tom Hicks on the heels of the A-Rod contract, or the creation of a regional television network.
The result of applying the various models to A-Rod’s performance and salary indicate that only in 2005 when he was worth $32M did his value to the Yankees exceed his salary of $25M. Taking his last six seasons as a whole, when he was paid a total of $122.3M, his performance was worth around $72M, so Keri concluded that by any standard A-Rod is indeed overpaid. At the end of Gennaro’s talk he showed a slide with the top players in terms of their dollar value to their teams. Perhaps coincidentally (but perhaps not) A-Rod comes in first at $25.2M, almost exactly matching his contract.
Rethinking Variation
It seems to me that if you’re going to critique the work of others you had better be able to critique yourself. It is in that vein that the remainder of this column will be devoted to rethinking a few of the methods and conclusions from my column last week, Variations on a Monetary Theme.
The Small Stuff
First, the small stuff. Several readers have pointed out that the first table in the column contains but 17 players and not 20 as advertised. In the interest of completeness, here are numbers 18 through 20.
Pct Div
Year Team Name Sal Pyrll Pyrll W L Pct Rank WC Div Lg WS
1999 CHA Frank Thomas $7.0 $25.6 27.3% 75 86 0.466 2 N N N N
2003 KCA Mike Sweeney $11.0 $40.5 27.1% 83 79 0.512 3 N N N N
1996 MIL Greg Vaughn $5.9 $21.7 27.0% 80 82 0.494 3 N N N N
Of course, adding these players also changes the calculations presented in the following paragraph. In fact, the teams represented on the list did not play almost .500 ball, but were instead pretty mediocre, posting a winning percentage of just .438.
The other small problem with that table is that not all the players listed had their salaries paid entirely by the team listed. For example, Rey Ordonez‘s 2003 salary of $6.5M was not paid entirely by the Devil Rays–$4.25M was picked up by the Mets. That means of course the Ordonez did not consume 33% of the Devil Rays payroll, but more like 11.5%. This problem also occurs with players whose salaries are deferred. For example, Ivan Rodriguez‘s $10M 2003 salary listed in the second table was actually set up so that the Marlins paid $3M in 2003, $3M in 2004, and $2M each in 2005 and 2006. In both cases the data at my disposal did not contain enough granularity to catch these issues. So Peter Gammons’ statement that no team had won the World Series with a single playing consuming as much as 16% of the payroll still stands, although it should be noted that it could easily have been overturned with the 2005 Astros with Jeff Bagwell and Roger Clemens.
As an aside, to calculate this correctly one needs access to data contained in the MLB/MLBPA Joint Exhibit One as colorfully described by Maury Brown earlier this week. Now if only Maury would get that in a database…
A Case of Multiple Regression
In the second half of the column I presented an analysis of salary variation and its correlation with winning percentage, and concluded that there was a weak negative correlation (-0.337) between salary variation as measured by the coefficient of variation (CV) and winning percentage. Further, that correlation was stronger than that between total payroll, average salary, or maximum salary and winning percentage.
There are two errors with that analysis, however.
The first: rather than compare the correlation of CV with that of total payroll, I should have normalized the payrolls of each team based on the league average. This is the case since average payrolls have risen sharply over the course of the last 15 years; under the original method, the 1990 Royals at $23.4M come out as one of the lowest payroll teams when, in fact, they were the highest payroll team of 1990. Correcting for that by using Normalized Payroll (NP), we find a correlation coefficient (r) of .414, instead of the .297 shown in the previous article. As a result, payroll relative to league can be interpreted as having a larger impact on winning percentage than variation within payroll.
But more problematic for the study is the fact that CV and total payroll (or CV and NP) are themselves correlated. Although I didn’t appreciate it at the time–and attempted to control for it by introducing CV in the first place–the variation in a team’s payroll even when accounting for their average salary (as CV does) will still tend to decrease as team payroll rises. The underlying reason is that player salaries have a limited range set by the market; as a result, a high payroll team like the Yankees simply can’t have as much of their payroll consumed by one or two individuals unless they were to start offering salaries of upwards of $60M.
The end result is that CV does not remove the dependence of these two variables. This can be seen to some degree by the graph I offered in the original article that plots CV versus payroll for 2005, where lower payroll teams are more likely to have a higher CV.
To see how the two variables (CV and NP) together predict winning percentage we can run a multiple regression with both variables included. While the r for NP alone remember was .414, the r when both are included jumps slightly to .451. In other words, adding CV to the model does not appreciably strengthen the correlation, which is an indication that either CV simply doesn’t matter or that most of the difference CV makes is smaller but bound up with NP. Since the correlation with CV alone was -0.337 and the test for statistical significance passed in the multiple regression, we can conclude that the latter is the case.
Always Learning
The great thing about baseball is that there are a seemingly endless number of things to learn. To that end I’m grateful to the folks at SABR36 for sharing their thinking with the rest of us.
Thank you for reading
This is a free article. If you enjoyed it, consider subscribing to Baseball Prospectus. Subscriptions support ongoing public baseball research and analysis in an increasingly proprietary environment.
Subscribe now