Every year, one of the first steps in the free agent dance is the ranking of players who finished the year on major league rosters for purposes of compensation. Under baseball’s Collective Bargaining Agreement (CBA), teams that lose a free agent may be entitled to additional picks in the next year’s Rule 4 amateur draft, depending on how good the free agent is.

The quality of free agents in each year’s class is determined by a statistical formula which grades players in position-based categories, per league, with the top 40 percent in each category being eligible for draft pick compensation. Players from the top 20 percent are designated Type A free agents, which net the team losing the free agent two first round draft picks; players from the next 20 percent are designated Type B free agents, making the team that loses one of them eligible for a supplemental first-round draft pick (a “sandwich pick”) between the first and second rounds. This is a change from the past twenty-five years, when the top 60 percent in each category were eligible for compensation, with three compensation types: A, B, and C; the 2006 CBA eliminated the Type C free agent compensation level.

These rankings are very important to the personnel decisions teams will make this offseason. In order for a team to receive compensation, the departing free agent must be offered (and refuse) salary arbitration. Because Type A free agents can cost the signing team their first-round draft pick, they’re sometimes avoided by teams successful enough to pick in the second half of the draft (the top 15 picks in the draft are protected). On the other side of the coin, a team may offer arbitration to free agents-to-be they have no intention of re-signing, in hopes of receiving compensation picks when they depart.

The details of the system used to create the rankings are shrouded in secrecy. The rankings are tabulated each year by the Elias Sports Bureau, but to be fair to Elias, you can’t place the system’s faults at their feet-Elias is merely executing a design formulated by the MLB and the Player’s Association in the early ’80s. Although previous editions of baseball’s Collective Bargaining Agreement have been made available to the public, the document detailing the ranking system, imaginatively entitled “A Statistical System for the Ranking of Players” has never been published along with the rest of the agreement. However, we do know a few things:

  • It’s not just free agents who are ranked. The system aims to rank the top players in the game overall, considering over 800 players total in this latest iteration. The list includes rookies and other players who are not yet eligible for arbitration, much less free agency. In theory, you could have a situation where none of the Type A or B players are actually eligible for free agency.
  • Starting and relief pitchers are considered separately from each other. Position players are divided into three categories: catchers, infielders (shortstops, second, and third basemen), and outfield/first base/designated hitter.
  • The rankings are based on the past two years’ data. Published reports and regression analysis have indicated that the ranking system is a 100-point scale, and is done by averaging the player’s ranks within their categories in certain statistical categories, such as plate appearances, home runs, and RBI for hitters, and innings pitched, strikeouts, and ERA for pitchers. It seems that each category is measured based upon a slightly different set of statistics from the others.

Now, let’s take a look at the 2007 Rankings for the AL and NL, courtesy of USA Today. The most spit-take-inducing ranking is in the AL Infielders group, which produce results as glaring and obvious as a slap across Scott Boras’s face:

Rk  Player          Grade    AVG  OBP   SLG   RBI   HR  Fielding Pct.
1   Michael Young   89.286  .314  .360  .439   197   23   .9768
2   Miguel Tejada   87.500  .315  .370  .473   181   42   .9716
3   Robinson Cano   85.714  .322  .358  .504   175   34   .9843
4   Derek Jeter     85.119  .333  .402  .468   170   26   .9729
5   Mike Lowell     84.857  .305  .359  .488   200   41   .9752
6   Alex Rodriguez  84.000  .302  .407  .584   277   89   .9508

The man who reportedly won’t come to the bargaining table for less than $350 million is nestled snugly behind fellow free agent third baseman Mike Lowell in the rankings. Since both men are comfortably Type A free agents, it doesn’t make much real-world difference, but it must be irksome for a man who’s being marketed as a historic talent, and who’s coming off a great year, to see that he doesn’t even crack the top five infielders here.

While the instinct is to use this result to decry the rankings as a farce, this result tells us quite a bit about the CBA’s secret formula. Lowell’s presence ahead of Rodriguez indicates that there is no VORP-style positional adjustment being applied, nor are the numbers adjusted for ballpark. The rankings also suggest that the two-year data isn’t weighted in favor of the most recent year, and that a qualitative judgment is being made about defense-because that’s the most significant place where Rodriguez’s performance has lagged over the last two years. Even though most advanced metrics agree that Rodriguez hasn’t been an exemplary defender at third, the most likely culprit in the CBA formula is its probable use of simple fielding percentage.

Looking at the infielders in the National League, you can see other places where the rankings fall afoul of more advanced metrics. Tadahito Iguchi is a Type A free agent this winter, but compare his statistics with those of the next player down the list, a man who if he were eligible for free agency this year, would be a Type B free agent:

Player            Grade    AVG   OBP   SLG   WARP   VORP
Tadahito Iguchi   72.222  .275  .347  .412    6.6   40.5
Rafael Furcal     70.879  .286  .350  .402   12.9   58.7

Over the last two years, we have the difference between Iguchi and Furcal as more than 18 runs on offense, and six wins by WARP.

Now, the point of this isn’t really to play gotcha or to say that the CBA rankings are horrible. In the end, those rankings are an industry-generated tool, and if the players and owners are happy to use it, why should we object? But looking at how such a system operates, we can hopefully gain some insight on how we want our own tools to work. For example, the “averaged ranking” approach that the free agent compensation rankings use can be applied to the various tools we use to look at NL relievers, producing a ranking system of our own:

2006-2007 NL Relievers
Rk   PITCHER           WARP1   VORP    WXR   ARP
 1   Takashi Saito     14.3    65.6   10.7   50.9
 2   Billy Wagner      11.4    48.1    8.3   31.6
 3   Chad Qualls        8.3    47.0    5.9   38.4
 4   Trevor Hoffman    11.5    41.3    8.1   26.9
 5   Bob Howry          9.1    46.5    4.7   34.6
 6   Jonathan Broxton   7.2    49.8    4.4   36.8
 7   Chad Cordero      10.2    38.8    6.4   25.8
 8   Matt Capps         9.5    47.0    4.1   23.0
 9   David Weathers     8.9    37.4    4.9   23.5
10   Heath Bell         5.8    39.0    5.5   32.1

William Burke contributed research to this article.

Further Reading

Thomas Gorman, The BP Guide to Transaction Rules, “Free Agent Compensation Draft Picks”: The Guide to Transaction Rules series is a must-read for anyone who wants to understand off-season personnel maneuverings. Keep in mind, however, that this installment hasn’t been updated for the 2006 changes to the CBA.

Rany Jazayerli, Doctoring the Numbers, “The Hidden Market Boost”: In this 2007 article, the good doctor details the changes to the free agent compensation rules and the repercussions to the amateur draft.

Nate Silver, Lies, Damned Lies, “Valuing Draft Picks”: This 2005 study takes a look at the value of a draft pick, translating draft pick compensation into dollars and cents.

Thomas Gorman, From the Mailbag: “JAWS, Revenue Sharing, and Sandwich Picks”: The last letter in this edition of the mailbag column addresses the mysteries of the sandwich pick system.