CSS Button No Image Css3Menu.com

Baseball Prospectus home
  
  
Click here to log in Click here for forgotten password Click here to subscribe

<< Previous Article
Premium Article Prospectus Today: Post... (09/24)
<< Previous Column
Premium Article The Biz Beat: The Wint... (09/11)
Next Column >>
Premium Article The Biz Beat: Another ... (10/25)
Next Article >>
Premium Article Under The Knife: Doubt... (09/25)

September 24, 2009

The Biz Beat

Fixing the GM Rankings

by Shawn Hoffman

the archives are now free.

All Baseball Prospectus Premium and Fantasy articles more than a year old are now free as a thank you to the entire Internet for making our work possible.

Not a subscriber? Get exclusive content like this delivered hot to your inbox every weekday. Click here for more information on Baseball Prospectus subscriptions or use the buttons to the right to subscribe and get instant access to the best baseball content on the web.

Subscribe for $4.95 per month
Recurring subscription - cancel anytime.


a 33% savings over the monthly price!

Purchase a $39.95 gift subscription
a 33% savings over the monthly price!

Already a subscriber? Click here and use the blue login bar to log in.

Way back in August of 2009, I tried to evaluate major league front offices on how efficiently they were spending their payroll dollars, building on the work of Doug Pappas and Nate Silver. I called it MR3/ExpMR-which was horrible. So from now on, we're going with Payroll Efficiency Rating, or PER. So there.

Anyway, I think we nailed the high end. Teams like the Rays and Red Sox dominated the top of the charts, while the previously overrated Marlins and underrated Yankees converged in the middle. But on the bottom end, there was still a major bugaboo, as I mentioned:

If there's still a piece missing, it's the value that comes with finishing last. The first pick in the draft is worth a lot more than the fifth or the tenth or the fifteenth, so a team that wins 59 games, as the Nationals did last year, should have that factored into its marginal revenue figures.

So there's still some work to do, but the process is actually pretty straightforward: figure out which pick each team should expect to get, based on their third-order win total, and about how much that pick is normally worth, in terms of future value. (For this, I used Sky Andreychuk's draft-pick value calculator, which holds up pretty well under heavy testing.)

Before we get to the numbers, let's acknowledge the five-hundred pound elephant that's about to walk into the room: this adjustment rewards teams for sucking. In fact, the worse a team is, below somewhere around 75-80 wins, the better they'll come out in the final rankings. So maybe we're rewarding incompetence, in some cases. But the reality is that teams are usually better off trying to win 60 games than 81-81 is a moral victory, while 60 could get you Justin Upton.

So with that said, here's the bottom ten with the old system (but updated for current third-order winning percentage), followed by the new one. Also, a big hat tip to Eric Seidman, who ran simulations to find expected draft positions, based on teams' win totals.


Bottom 10, Old Way
Team        PER
Padres     0.90
Nationals  0.90
Cubs       0.89
Brewers    0.88
Orioles    0.84
Royals     0.80
Mets       0.76
Pirates    0.76
Astros     0.75
Reds       0.75

Bottom 10, New Way
Team        PER
Orioles    0.96
Reds       0.96
Indians    0.95
White Sox  0.95
Mariners   0.94
Brewers    0.93
Tigers     0.84
Astros     0.82
Cubs       0.80
Mets       0.72

For the curious, here's the full data. The Pirates (15th in the new rankings), Padres (17th), Nationals (19th), and Royals (20th) move out of the bottom ten, which shouldn't be a big surprise, given their expected draft positions. Meanwhile, the Mets, Cubs, and Tigers all fall a few places, with the Mets now bringing up the rear. If you're surprised to see the Tigers there, just check out their third-order winning percentage, which would have them winning 78 games. Also-and are you allowed to do two parentheticals in a row? Whatever-you'll notice that the numbers are higher in the new version, but this is actually just a function of the top and bottom converging. The 30-team average is actually down, from about 1.17 to 1.09.)

At the top, the Rays keep the top spot, despite their troubles lately (the third-order standings still have them ahead of the Red Sox), while the Dodgers, Rockies, Red Sox, and Cardinals round out the top five. The top eight teams in terms of third-order winning percentage are all in the top half of the league, and all but the Phillies are ahead of the no-cost Marlins.

So integrating draft-pick values didn't rock the boat at the top of the chart, which I think is a good thing. But there are a lot more questions we can now try to answer with this data. For example, where is that key cutoff point where your team is actually better off losing? We can take the expected pick value curve, add the totals to Nate Silver's marginal revenue curve, and see where the dips are:

Some notes:

  • This isn't perfect. It probably overstates the value of winning 75-80 games, for example. But remember something: this doesn't take into account the expense side. Using the 60 versus 81 example from above, the two are very close on the chart. But all other things being equal, an 81-win team should cost you more than a 60-win team. So while the "revenue" side might be similar, the ROI should be much higher for the 60-win team, and it's obviously far less risky. 

  • There's a steady drop between 60 and 70 wins, because a team's chances at the number-one pick start going down very quickly in that range. In Eric's simulations, teams that won around 60 games had about a fifty percent shot; teams that won 70 didn't get it a single time, in over 65,000 tries.

  • It's still a lot better to win 100 games than 50.

If there's another obvious practical application with this, it's to show how much better it is for a small-market team to win than to simply fold up the tent and go for the number-one pick. Even with draft picks factored in, the Rays are still an easy number one, and the Pirates, Nationals, A's, et al are in the middle of the pack. In terms of pure return on investment, a winning team always justifies a few more payroll dollars.

However, it's also important to remember the constraints that these teams are working under. For example, if the Pirates were on pace to win 92 games instead of 62, we would expect them to bring in about $35 million more in revenue; that's a decent chunk of money, but try improving your team by thirty wins in free agency-where a win usually goes for $3-5 million-with only $35 million. The only way that small-market teams can build economically viable contenders is through player development or trades; we already knew that, of course, but it's an important reminder for Pirates fans who are about ready to hang themselves after watching the team go 3-22 over the past few weeks. (For what it's worth, the Yankees' expected revenue would go up by $140 million if the team went from 62 to 92, which makes their strategy actually seem somewhat reasonable.)

There are plenty of other potential applications of this, which I'm sure I'll get to at some point. In the meantime, if anyone has ideas, fire away in the comments section.

Shawn Hoffman is an author of Baseball Prospectus. 
Click here to see Shawn's other articles. You can contact Shawn by clicking here

Related Content:  Win Curve

27 comments have been left for this article. (Click to hide comments)

BP Comment Quick Links

kcboomer

Any ranking system that contends there are ten front offices worse than the Royals is seriously flawed.

Sep 24, 2009 09:57 AM
rating: 9
 
Wyomissing

Agreed. And, ditto for the Pirates!

Sep 24, 2009 16:26 PM
rating: -3
 
awayish

not so

Sep 25, 2009 06:36 AM
rating: 0
 
oira61

Shawn: Maybe you covered this earlier, but is it really true that a GM is better off winning 60 than 79? Attendance matters more to the bottom line than a draft pick. Is the attendance that similar for mediocre and bad teams?

Sep 24, 2009 10:35 AM
rating: 2
 
Peter Hood

While I realize that winning the top draft pick does confer value. it's a p___poor way of evaluating the front-office and GMs. It strikes me that a better approach would be to factor in some "quasi-rigourous" estimate of the top 20 or 30 prospects in the team's system. That way at least, the ability of the FO to develop talent internally can be assessed and included as part of the evaulation.

Sep 24, 2009 10:40 AM
rating: 1
 
baserip4

Tell those same Pirates fans that the Orioles went 4-32 to close the 2002 season, going from 63-63 (!) to 67-95. And look what we have to show for it 7 years later! Wait...

Sep 24, 2009 10:46 AM
rating: 1
 
twac00

You guys have a team that should be great for years to come.

Sep 24, 2009 12:25 PM
rating: 0
 
BP staff member Shawn Hoffman
BP staff

Is it just me or has that happened to the Orioles 3 or 4 more times since then...

Sep 24, 2009 12:36 PM
 
TGisriel

The Orioles have been terrible late in the season virtually every year this century.

The difference this year for the Orioles is that, although they are losing, they have used the year productively to introduce significant new talent to the major leagues, including, Matt Wieters, Nolan Reimold, Chis Tillman, Brian Matusz, Brad Bergesen, Jason Berken, David Hernandez. They have also shown substantial improvement in several of their young players who were not rookies: Adam Jones, Felix Pie.

It's been a losing year, but a productive year.

Sep 24, 2009 13:12 PM
rating: 3
 
amazin_mess

Again, the system does not take into consideration the amount of payroll on the DL.

Sep 24, 2009 12:49 PM
rating: 0
 
BP staff member Shawn Hoffman
BP staff

There are a couple different ways to do this, and I don't think either really works. One is to give teams credit for projected performance, which is a very slippery slope (if Carl Pavano is on the DL, do you give full credit for his projected output?). The other is to rebate teams for the actual payroll dollars on the DL, but this would give teams credit for crappy contracts that they've simply shelved.

If there's a smart way to do this, I'd up for adding it. But I actually like not having it factored in, b/c it gives teams credit for having depth.

Sep 24, 2009 13:01 PM
 
awayish

it does, by according that payroll the performance it generated, in the case of dl'd players something like 0. your argument is that dl time should be excused as natural accidents. this is a debatable question, but like shawn said, injuries are a real problem that teams have to deal with, and whether that is dealt with successfully or not is a part of the gm's job. it is really hard to draw the line between aberrant and excusable injury plague and normal wear and tear.

Sep 25, 2009 14:07 PM
rating: 0
 
Scot

So the Nationals GM sets out to win as many games as he could in 2009, and by being absolutely terrible as a GM and fielding an awful team he gets a higher GM ranking than if he did a good job of assembling a team, winning more games?

Unless a GM SETS OUT to lose a bunch of games and get a good draft pick, I don't think it makes any sense to reward a GM whose team loses a bunch of games and gets a high draft pick.

If you want to include the draft in GM ratings, why not compare how a teams draft picks do relative to expectations for the spot, rather than just rewarding a GM for running a team that gets a high draft pick.

Sep 24, 2009 14:39 PM
rating: 4
 
Seth

I think this depends on what the purpose of this stat is. If it is to attempt to objectively determine the smartest or best GM in baseball, then you're probably right. Of course, if that's the case, then it would almost make sense to remove any actual results, since they include a good amount of luck.

It seems to me that the purpose of the exercise is to identify the GMs who have contributed the greatest possible (quantifiable) value to their franchises. And while it is the case that a GM may or may not be intentionally setting out to finish in the bottom of the standings, it is nearly indisputable that a team that receives the number 1 draft pick has obtained greater added value than the team that receives the number 15 draft pick.

Including draft pick value goes along with a long-held principal among analysts that it is better to rebuild than to tread water.

Sep 26, 2009 12:34 PM
rating: 0
 
amazin_mess

I see what you are saying Shawn. My beef is that, even though as a Mets fan I know they are not efficiently run, I don't think they're as bad as say the Royals or Pirates.

Sep 24, 2009 16:13 PM
rating: 0
 
sbnirish77

Where else except at BP could you find a methodology that would actually reward a GM for finishing last.

"If there's still a piece missing, it's the value that comes with finishing last."

By that standard the GMs of the Tampa Bay DEVIL Rays must have been geniuses for 10 years.

Did they make some good draft picks? Yes. Did they build their brand with ten years of losing? Look at all the Yankee and Red Sox fans in their stands.

I think there was an excellent series of articles on the depreciating value of draft picks in the first round based on lifetime VORP. The article did point out, from a historical analysis, that the talent was front-loaded early in the draft.

But not only was the annual VORP number small (due to a large number of flunkies) but the teams on the cheap would only capture a fraction of the lifetime VORP (due to retaining the players for only six years on their careers).

Sep 24, 2009 19:18 PM
rating: -2
 
BP staff member Shawn Hoffman
BP staff

Everything you're saying is pretty well-reflected in the chart. There's a bump between 55-65, because of the value of getting the first pick. But it's always a lot better to win 90+ games, and might even be better to win 85, assuming equal costs.

Sep 24, 2009 19:58 PM
 
Sharky

Shawn, can you PLEASE send this to Fred Wilpon? How much evidence does he need to see?

Sep 25, 2009 06:44 AM
rating: 0
 
Dan W.

One thing that you might need to account for (suggested by the thread about the Orioles, above) is how this fluctuates over the course of a season. Say there was a season in which, hypothetically, Baltimore started out, say, 0-21, and went .500 the rest of the way and finished with 70-75 wins. I'm going to assume that their marginal revenues would be lower than in a season where they started out 30-15, drew fans to the stadium through the all-star break hoping that the early season showing wasn't a mirage, and then faded down the stretch.

Sep 25, 2009 06:47 AM
rating: 1
 
awayish

while fluctuations like that are significant when they do exist, you have to show that there is a way to control the fluctuation to show a need to account for them in gm evaluation. otherwise you are crediting people for luck.

Sep 25, 2009 13:44 PM
rating: 0
 
nicholj

Should we not evaluate GM's in the same manner that we evaluate CEO's? Simply look at the percentage change in franchise value from the date the GM started to finished and compare against the average franchise value percentage change during the same period. GM's do not just spend money on player payrolls but they build (or destroy) franchise brands.

Sep 25, 2009 12:19 PM
rating: 0
 
awayish

that's fine from an owners' point of view, but this is a site written for fans, presumably, and fans are interested in wins.

Sep 25, 2009 13:42 PM
rating: 0
 
Seth

Shawn,

Are you still using a linear model for expected wins? IIRC, last time I and a few other people suggested a logarithmic model, to account for the diminishing returns of dollars tacked onto larger payrolls. Any thoughts on that?

Sep 26, 2009 12:36 PM
rating: 0
 
Richard Bergstrom

Why are we assuming that having the first pick is valuable? Many of those don't pan out and few of the ones that do become superstar players... and besides, there is usually a consensus among the mass media on who the #1 pick is so why overly reward a GM for making the obvious choice?

It's kind of like saying people who got ARM balloon mortgages, foreclosed, then got bailed out by the government so they could keep their house at a cheaply refinanced rate (because interest rates are so low currently) are better at managing their finances than blokes like me who got a fixed rate mortgage I can afford...

Sep 26, 2009 18:10 PM
rating: 1
 
awayish

it's not having the first pick, but having a higher pick than is normal

Sep 27, 2009 08:15 AM
rating: 0
 
BrewersTT

What time frame is covered in this study? Thanks.

Sep 27, 2009 18:08 PM
rating: 0
 
radm605

I admit up front to skimming and typing quickly but I think you have this all wrong. Presumably, the goal is to win as many games as possible. Teams that get the first pick more often do so out of incompetence than strategy or skill. Further, you should count getting the first pick of the draft AGAINST the front office because they've been rewarded for their incompetence. (And the 2nd pick should be counted against them to, though not as heavily -- and so on.) You absolutely should put in something that accounts for draft order, but it should be giving more credit to the teams at the bottom of the draft who have still created successful farm systems that produces major league talent (Red Sox, Dodgers, Yankees - lesser extent: Braves, Rockies, A's, etc.) and less credit to the Rays of the world who could only build a successful team by failing miserably for years. (This is not to say that the Rays haven't done a great job -- they should get a heck of a lot more credit than the Pirates and Royals, who blow their top draft picks every year.) While there is the problem that the Red Sox (and a few others) spend more in the draft that other teams, their success prevents them from ever getting the top pick that nets a Longoria or Upton. The teams picking low in the draft should be rewarded, not punished, for their draft slot vs. their MLB performance.

Sep 28, 2009 13:37 PM
rating: 0
 
You must be a Premium subscriber to post a comment.
Not a subscriber? Sign up today!
<< Previous Article
Premium Article Prospectus Today: Post... (09/24)
<< Previous Column
Premium Article The Biz Beat: The Wint... (09/11)
Next Column >>
Premium Article The Biz Beat: Another ... (10/25)
Next Article >>
Premium Article Under The Knife: Doubt... (09/25)

RECENTLY AT BASEBALL PROSPECTUS
Fantasy Article Expert League Auction Recap: CBS AL-Only
Fantasy Article State of the Position: Relief Pitcher
Fantasy Article The Quinton: Market Corrections and Underval...
The Week in Quotes: February 23-March 1, 201...
Premium Article Rumor Roundup: You Can't Predict Padres
Premium Article Transaction Analysis: It's the Latest Johan ...
Fantasy Article Fantasy Players to Target: Relief Pitchers

MORE FROM SEPTEMBER 24, 2009
Premium Article Prospectus Today: Post-season Bullpens
Premium Article Kiss'Em Goodbye: Oakland A's
Premium Article Fab Fours

MORE BY SHAWN HOFFMAN
2010-01-14 - Premium Article Squawking Baseball: Waiting for the Winter o...
2009-10-25 - Premium Article The Biz Beat: Another Chilly Winter?
2009-10-08 - Premium Article Relative League Quality
2009-09-24 - Premium Article The Biz Beat: Fixing the GM Rankings
2009-09-11 - Premium Article The Biz Beat: The Winter Market
2009-08-20 - Premium Article The Biz Beat: A New Way to Rank the GMs
2009-08-12 - Premium Article The Biz Beat: A Revenue-Sharing Re-Think
More...

MORE THE BIZ BEAT
2009-10-25 - Premium Article The Biz Beat: Another Chilly Winter?
2009-09-24 - Premium Article The Biz Beat: Fixing the GM Rankings
2009-09-11 - Premium Article The Biz Beat: The Winter Market
2009-08-20 - Premium Article The Biz Beat: A New Way to Rank the GMs
2009-08-12 - Premium Article The Biz Beat: A Revenue-Sharing Re-Think
More...

INCOMING ARTICLE LINKS
2010-03-09 - Premium Article Prospectus Hit and Run: NL Central Competiti...
2010-02-23 - Premium Article Prospectus Hit and Run: AL Central Competiti...
2010-02-12 - Premium Article Prospectus Hit and Run: AL East Competitive ...
2010-02-04 - Premium Article Squawking Baseball: The Best and Worst GMs o...
2010-02-03 - Premium Article Prospectus Hit and Run: The Lay of the Land