Baseball Prospectus is looking for a Public Data Services Director. Read the description here.

So far this winter we’ve seen a few teams acquiring players despite not having openings at those positions; we’ve seen this particularly with pitchers. The first major instances of acquiring surplus talent were the Red Sox’s acquisition of Josh Beckett and the Toronto Blue Jays’ signing of Beckett’s former teammate, A.J. Burnett. At the time they traded for Beckett, the Red Sox already had a full five man rotation under contract for 2006 (Curt Schilling, Matt Clement, David Wells, Tim Wakefield, and Bronson Arroyo) to say nothing of the presence of Jon Papelbon and several other throwers in the high minors. Likewise, the Blue Jays were at the time sitting on five starting pitchers (Roy Halladay, Ted Lilly, Gustavo Chacin, Josh Towers, and the newly traded David Bush) along with several other options (Scott Downs and a multitude of arms in the high-minors or bullpen including Brandon League, Dustin McGowan, Zach Jackson, and Chad Gaudin).

When judging the value of these acquisitions, it’s very easy to fall back on metrics like VORP or WARP to estimate how much a player will add to a team in terms of runs or wins. Then, by comparing those totals to estimated economic incentives for making the playoffs and the increased revenue of more wins, an estimate of the player’s economic value can be made as well. For example, those of you who are subscribers to Sports Illustrated may have noticed that Daniel G. Habib called upon PECOTA to estimate the value of the players signed to some of the more recent contracts this winter. Though it was not published, PECOTA’s estimate for Burnett was $8.3 million per year for a five year deal. By that number, the Blue Jays appear to have overpaid by $2.7 million per season.

However, the problems with relying on numbers like these to value players are two-fold. First, those numbers do not consider some of the more recent valuation analysis that accounts for the varying value of victories and the odds of reaching the playoffs. As highlighted by Nate Silver, the same player is going to be worth vastly different amounts to teams in different markets and stages of competition. For example, if the Royals determine that Burnett is a five win player and enter the bidding, their return on those five extra wins–going from 56 to 61–isn’t anywhere close to the Blue Jays’ increase in expected revenue both because they’ve increased from 80 to 85 wins and because they’ve helped their chances to secure a playoff spot.

The second reason is the one I want to delve into today: the application of replacement level. It’s hard to overstate the value of the work done by Keith Woolner and others to establish a numerical value for replacement level and to develop metrics that value a player’s contribution over that level rather than league average or zero. But the theoretical replacement level involved in metrics like VORP and WARP is just that: theoretical. It’s extremely handy when comparing players across teams or leagues, but when it comes to specific player acquisitions, it’s possible to better analyze the situation by using the actual options available to the team than the theoretical threshold, at least in the short term.

Getting back to the Red Sox for a minute, let’s take a simple example and say that they simply remove Arroyo from the rotation and replace him with Beckett. Arroyo posted SNLVARs of 4.0 and 3.0 in the last two seasons, respectively while Beckett was good for 3.8 and 5.3. Now, rather than acquiring an extra 5.3 wins–Beckett’s value over replacement–the Sox have only added 2.3, assuming both players repeat their performances from 2005. If Beckett can stay healthy and pitch more than his 178.2 innings from last year, perhaps the Sox might net a little more in the deal. But rather than netting Josh Beckett, the Red Sox have effectively netted Brett Tomko or Woody Williams.

Likewise, in Toronto, Burnett has been worth 3.5 and 4.6 wins over replacement the last two seasons. However, the Jays’ worst starter last year was Lilly, who posted a 1.7 SNLVAR in 126.3 innings (after a 5.6 in 2004). Likewise, Bush posted a 2.6 last year in 134.0 innings, meaning the Jays have only increased their team total by about 3.1 wins, again assuming a consistent performance from all parties. In the bullpen, things aren’t quite as bad. B.J. Ryan notched 3.4 WXRL last year to Miguel Batista‘s 0.9 or Pete Walker‘s 0.5. Once again, the improvement to the team isn’t above the theoretical replacement level, but rather a group of players slightly above our baseline. As such, basing valuations of these players is overvaluing their likely contributions to the team going forward.

Of course, the Jays largely solved their rotation overload with the trade for Lyle Overbay, a player worth 33.3 VORP last year, a total that would have led the Jays. Here again, Overbay doesn’t fill a position of need for the Jays at the moment. He would largely be replacing the production of Eric Hinske or Shea Hillenbrand, players who totaled 17.1 and 32.5 VORP last year, respectively. Assuming that Overbay eventually replaces Hinske, and disregarding a slight discrepancy in playing time, the Jays have gained 16.2 runs rather than 33.3. Add that up and swap out Bush and Hinske for Burnett and Overbay, the Jays have gained themselves only about 4-5 wins while blocking Hinske and shipping out one of their better pitching prospects.

Admittedly, viewing these decisions solely based on 2005 performances and disregarding injuries leaves several other factors to be considered, but the application of a team-specific replacement level to decision-making can change the value of players as much as a team’s situation with regards to the competition and the number of games they’re likely to win with and without the player.

At the bottom of this piece, there are three charts. The first is the average VORP of all bench players by team last year, an attempt to estimate each team’s replacement level. Note that most teams are in the positive in this regard, not necessarily because the theoretical replacement level is too low, but because platoon players and players like backup catchers–who get significant playing time–are included. But regardless of the crudeness of this estimate, it’s still easy to see that the same player with the same production is going to be significantly more valuable to the Yankees than the Brewers–to the tune of nine runs or nearly one win–assuming they are both looking at him to fill a starting role.

The next two charts are the estimations for replacement level in both the rotation and the bullpen. The sample groups for these charts are all players who ranked sixth or lower in GS for the rotation or seventh or lower in IP (max 3 GS) for the bullpen. A few quick notes on these two charts: 1) Nearly every team requires at least 10 starts from a pitcher outside its top five, so the value of having quality pitchers past the rotation is key. 2) Most teams are going to require at least 100 innings from bullpen pitchers outside their top six, or more than any individual reliever. 3) The Cubs–a team that has so aggressively spent money to acquire setup men for its bullpen–actually had the highest estimated replacement level in the pen last year. This finding indicates that of all 30 major league teams, the Cubs are the team that should value additional relief help the least because their marginal returns are the lowest.

It should be obvious that the charts below fail to compensate for player movement both during and since the 2005 season. More importantly, they can largely overlook teams that may have a number of capable backups who combine to contribute respectable value. The ability to combine equal value into a single roster spot is both overlooked and highlighted by this analysis: the charts below will overvalue teams who have a strong bench top to bottom, but they also emphasize the fact that if a team can sign one player to replace two, the newly available roster spot can likely be filled by a player who is above theoretical replacement level, thereby adding value to the team above and beyond the actual value contributed by the newly acquired superstar. Furthermore, a team that is able to raise its replacement level will find less value in new players than those who neglect their bottom line. This can be both a blessing (the ability to let a team’s free agents walk because their value would be less to their current team than they will demand on the open market) and a curse (a capable bench may encourage teams to underbid on free agents, and while the bench may provide more bang for the buck, the only playoffs they’ll reach will be of the Triple-A variety).

On a micro level, this kind of analysis can be very enlightening, but by no means does it indicate a need to reevaluate the league-wide replacement-level thresholds currently in use. Those levels of performance established for replacement level are accurate on a macro level and across seasons and are still very valuable when comparing players. But when determining a player’s value to a specific team, comparing the player to the available options can reveal what opposing teams will be willing to spend. Of course, that assumes that teams are all valuing player performance accurately and similarly which, if it were true, wouldn’t leave the rest of us very much to write about.

           Batter Replacement Level
Year  Team   Batters   PA    VORP  VORP/Batter
2005   MIL      9     1603   59.9      6.7
2005   CIN     12     1618   60.8      5.1
2005   NYN     11     1566   49.6      4.5
2005   CHN      8     1226   34.1      4.3
2005   PHI      7     1086   29.8      4.3
2005   SDN     14     1999   47.9      3.4
2005   ATL      9     1564   28.8      3.2
2005   PIT     16     1820   48.7      3.0
2005   LAN     14     2200   40.5      2.9
2005   TBA     12     1393   33.3      2.8
2005   SLN      9     1617   24.9      2.8
2005   SFN     13     1415   35.4      2.7
2005   WAS     17     1604   36.6      2.2
2005   COL     17     2269   36.1      2.1
2005   TOR     10     1398   19.8      2.0
2005   DET     13     2119   24.0      1.8
2005   BOS     17     1183   29.6      1.7
2005   ANA     14     1644   22.9      1.6
2005   MIN     13     1960   17.8      1.4
2005   FLO      9      681    8.7      1.0
2005   ARI      7     1150    6.3      0.9
2005   OAK     11     1486    8.3      0.8
2005   HOU      9     1138    0.0      0.0
2005   TEX     11      984   -0.2      0.0
2005   BAL     17     1655   -1.6     -0.1
2005   CHA     11     1033   -8.6     -0.8
2005   KCA     15     1571  -12.7     -0.8
2005   CLE     12      859  -12.6     -1.1
2005   SEA     16     1327  -20.1     -1.3
2005   NYA     14     1003  -32.6     -2.3

     Rotation Replacement Level
Year  Team  Pitchers   GS   SNLVAR
2005   PIT      4      26     5.9
2005   NYN      3      27     5.1
2005   ATL      4      31     4.6
2005   NYA      8      38     4.6
2005   COL      6      44     3.9
2005   SEA      5      23     3.8
2005   WAS     10      32     3.3
2005   MIL      3      26     3.2
2005   SFN      7      31     3.1
2005   ARI      3      27     2.7
2005   FLO      5      26     2.4
2005   MIN      3      15     1.8
2005   SDN      6      35     1.8
2005   CHN      6      31     1.6
2005   TEX      9      57     1.6
2005   CHA      1      10     1.5
2005   TOR      4      27     1.2
2005   ANA      4      13     1.0
2005   TBA      5      29     0.8
2005   PHI      2       9     0.8
2005   KCA      6      33     0.6
2005   BOS      5      19     0.5
2005   LAN      5      26     0.5
2005   CLE      1       4     0.4
2005   SLN      2       2     0.4
2005   BAL      3      19     0.3
2005   OAK      3      14     0.3
2005   HOU      2      16     0.0
2005   DET      5      18    -0.4
2005   CIN      4      22    -1.0

     Bullpen Replacement Level
Year  Team  Pitchers    IP    WXRL
2005   CHN     13     197.3    4.6
2005   ATL     15     164.3    2.8
2005   PIT      7      82.3    2.7
2005   WAS     12     177.0    2.3
2005   ARI     14     158.7    2.1
2005   CLE      6      91.7    1.7
2005   LAN     10     149.0    1.5
2005   DET     11     163.3    1.5
2005   TEX     19     272.0    1.4
2005   NYN     11     150.0    1.4
2005   ANA      6      67.7    1.3
2005   BAL      9     152.0    1.3
2005   SEA     10      94.0    1.2
2005   TOR      8     118.3    0.9
2005   SFN     14     226.0    0.9
2005   TBA     11     174.0    0.6
2005   CIN     11     209.3    0.4
2005   OAK     10     101.0    0.3
2005   CHA      7      62.0    0.2
2005   SLN     10      73.7   -0.2
2005   SDN     14     163.0   -0.2
2005   KCA     10     190.7   -0.2
2005   HOU      8      62.0   -0.3
2005   MIN      5      24.3   -0.3
2005   MIL     10     154.0   -0.4
2005   FLO     17     189.0   -0.9
2005   BOS     16     144.7   -0.9
2005   PHI      8      83.3   -1.5
2005   NYA     15     128.0   -1.8
2005   COL     16     202.7   -3.4

Thank you for reading

This is a free article. If you enjoyed it, consider subscribing to Baseball Prospectus. Subscriptions support ongoing public baseball research and analysis in an increasingly proprietary environment.

Subscribe now
You need to be logged in to comment. Login or Subscribe