As the recent scrum between supporters of the candidacies of Joe Mauer and Mark Teixeira reminds us, nearly every Most Valuable Player Award is capable of producing controversy. Not only do the voters from the Baseball Writers Association of America rarely elect the player who, via some objective formula, is worth the most wins to his team, they appear to shift their standards from year to year, instead constructing narratives to fit whatever loosely-gathered facts are at hand. Particularly in recent years, defensive value is often minimized or entirely ignored in favor of heavy hitters with big Triple Crown stats, almost invariably from successful teams.
The question is whether the voters’ behavior can be predicted. To that end, I was tasked with building an MVP predictor in the spirit of a system such as Bill James’ Hall of Fame Monitor, one that awards points for various levels of achievement in an attempt to identify who will win, as opposed to who should win. My initial bursts of enthusiasm for the assignment were soon followed by endless hours of cowering in the fetal position before a massive spreadsheet, but in the end I emerged with a system-Jaffe’s Ugly MVP Predictor (JUMP)-which correctly identified 14 of the 28 winners during the Wild Card Era (1995 onward), and put 27 of those winners among the league’s top three in its point totals.
I limited the scope of the system to that post-strike timeframe for three main reasons: none of the 28 winners were pitchers, only one played for a team that finished below .500 (Alex Rodriguez in 2003), and 22 of them played on teams that qualified for the expanded postseason-extremely strong tendencies that could help separate seemingly equal candidates. Instead of focusing on round-numbered benchmarks like James did (a .300 batting average, 100 RBI), I chose to dispense with actual stat totals and rates and focus on league rankings among batting title qualifiers (3.1 plate appearances per game) in 12 key offensive categories: batting average, on-base percentage, slugging percentage, OPS, hits, homers, total bases, runs, RBI, walks, intentional walks, and steals. Through much study, trial, and error-indeed, every single step of the process involved this-I eventually settled upon a 10-7-5-3-2-1-1-1-1-1 point system in each category, which produces a slight scoring bonus for leading the league or finishing in the top three, and some acknowledgement of a top-ten finish.
Surprisingly enough, it’s not a strong showing in RBI or even home runs which is most common among the award winners of this era:
Category Lead Top 5 Top 10 Total Bases 7 19 25 Slugging Pct. 10 18 23 Runs 9 16 23 OPS 8 17 22 Batting Avg. 3 11 21 Home Runs 7 19 21 Runs Batted In 6 18 20 Intentional BB 7 13 20 On-Base Pct. 6 11 18 Hits 2 10 16 Bases on Balls 6 8 12 Stolen Bases 1 3 5
As you’ll see below, the lack of a correlation between the getting-on-base stats and the eventual hardware had consequences that needed to be taken into account.
Because team performance has such an overwhelming effect on the voters’ perceptions of players’ candidacies, I recorded each team’s record and route to the playoffs, fixing upon a system that awarded a maximum of three “Team Success Points”: one for finishing at or above .500, another for winning the Wild Card, and two for winning the division. Those points were then multiplied by the team’s win total and divided by nine; a player on a 99-win division winner thus received 33 points, one on a 90-win Wild Card team received 20 points, and one on an 81-win team received nine points. These team points, which can outweigh the points of any individual categories, do much to winnow the field.
At that point, various iterations of the system-some of which included weighting the stat categories according to the frequency with which past winners had placed in the top 10-correctly identified anywhere from nine to 12 winners out of 28, not a terribly impressive result. From there, JUMP became an exercise in careful gerrymandering, not only to increase the direct hits but to push as many winners as possible into the league’s top three, a concession to the fact that at some point subjective elements take over for a number of voters. The point totals in a few categories-OBP, OPS, hits and walks-were dropped entirely from the scoring once it was determined that excluding them made no difference; simplicity was given the priority. Intentional walks were reduced to a 0.5 weight, stolen bases to 0.55. I introduced a positional adjustment, adding 3.33 points for middle infielders and penalizing 13 points for designated hitters, and an anti-Rockies adjustment, penalizing 10 points for high-altitude residence. All of these values were arrived at only after tedious trial and error.
Here’s how the actual award winners fared in JUMP, along with the players it flagged as the likely winners in years where they differed from the voting:
Year AL Winner Rank System Winner 1995 Mo Vaughn 3 Albert Belle 1996 Juan Gonzalez 2 Albert Belle 1997 Ken Griffey 1 1998 Juan Gonzalez 1 1999 Ivan Rodriguez 10 Manny Ramirez 2000 Jason Giambi 1 2001 Ichiro Suzuki 2 Bret Boone 2002 Miguel Tejada 2 Alfonso Soriano 2003 Alex Rodriguez 1 2004 Vladimir Guerrero 1 2005 Alex Rodriguez 1 2006 Justin Morneau 3 Derek Jeter 2007 Alex Rodriguez 1 2008 Dustin Pedroia 1 Year NL Winner Rank System Winner 1995 Barry Larkin 3 Dante Bichette 1996 Ken Caminiti 1 1997 Larry Walker 2 Jeff Bagwell 1998 Sammy Sosa 1 1999 Chipper Jones 1 2000 Jeff Kent 3 Barry Bonds 2001 Barry Bonds 3 Sammy Sosa 2002 Barry Bonds 1 2003 Barry Bonds 1 2004 Barry Bonds 3 Albert Pujols 2005 Albert Pujols 1 2006 Ryan Howard 2 Albert Pujols 2007 Jimmy Rollins 3 Matt Holliday 2008 Albert Pujols 2 Ryan Howard
Memo to Mauer fans: the only catcher to win the award during this era is the one whose result sticks out like a sore thumb. In my gerrymandering efforts, no amount of positional bonus given to Pudge could offset the consequence on boosting Mike Piazza into the top three a few times or creating other problems. The system does correctly nail a few of the curveballs thrown by the voters, including Sosa over McGwire despite the latter’s record-setting home run total in 1998 (one reason OBP was dropped), A-Rod with the 71-win Rangers in 2003, and Pedroia last year, and it gets pretty close on players like Larkin, Kent, Suzuki and Rollins, who won the award despite not finishing in the top five in homers, RBI, or slugging percentage-three of the four most common categories populated by MVP winners.
As for the various discrepancies, it doesn’t take too much to recall some of the subjective elements which may have played a part in the voting, particularly in the AL. Take the writers’ loathing of Belle, who in 1995 led the league in six point-accumulating categories and outdistanced Vaughn 84-47 here (Edgar Martinez was second at 52 points). Recall their fascination with the novelty of Ichiro and that 116-win, post Griffey/Rodriguez/Randy Johnson team. Note their tendency to avoid voting for Yankees when at all possible, particularly during the Torre dynasty years; Rodriguez’s win in 2005 was the first for someone in pinstripes since Don Mattingly in 1985. Remember the way the wholesome Midwestern clutch goodness of Morneau’s 130 RBI carried the day over a fine all-around year from Jeter (.343/.417/.483 with 118 runs and 97 RBI), not to mention the season turned in by teammate and batting-title winner Mauer.
Jeter’s 2006 plight only serves to reopen the wounds of 1999, a monster year in which he hit .349/.438/.552 with 24 homers, 134 runs and 102 RBI, all career highs; while he ranks second in JUMP, he could do no better than sixth in the actual vote. Pedro Martinez, who won the Cy Young and the pitchers’ Triple Crown by going 23-4 with a 2.07 ERA and 313 strikeouts, received the most first-place votes that year but wound up a tight second, the highest by a pitcher during this span. JUMP leader Ramirez wound up tied with teammate Roberto Alomar (who ranked third here) for a close third in the actual voting. Pudge overcame the strong candidacy of teammate Rafael Palmeiro, who ranked fourth here and finished fifth in the voting, leapfrogging a very strong, very unusual field.
There are actually more discrepancies here on the NL side, though the “wrong” winners are often players who wound up winning in other years-namely Bonds, Sosa, Pujols and Howard-softening the blows of those “injustices.” Rollins won it as the sparkplug leadoff man who led the league in runs and finished second in total bases, and while the Phillies only made the playoffs on the regular season’s final day, who knows how many already-sent votes might have turned Holliday’s way given his Game 163 heroics.
As to what the system says about this year’s MVP races, the names in the NL race are no surprise. Pujols, who leads the league in five of these categories and is in the top five in two others, is the overwhelming leader with 84 points, followed by Howard with 49 and Chase Utley with 40, repeating last year’s third-place ranking. The system has flip-flopped on Pujols and Howard twice, getting the wrong answer both times, but neither time was the gap separating the two so wide as this year.
As for the AL, Teixeira leads in RBI and is second in homers, and ranks first with 55 points; he’s followed by Miguel Cabrera with 47 points. Surprisingly, Chone Figgins is third with 44 points thanks to the league lead in runs and a third-place showing in steals; teammate Kendry Morales, who’s second in the league in slugging, is just a fraction of a point behind him. As for Mauer, he’s currently 28th, a consequence of playing for a team that through Saturday was a game under .500; Sunday’s Twins win vaults him to 15th. While he leads the league in all three triple-slash categories, OBP doesn’t score points here, and he currently cracks the top 10 only in intentional walks. He is 11th in homers and total bases, and 13th in RBI, so if the guy could just snap out of his August funk (.402/.462/.654 with seven homers and 22 RBI), he may yet JUMP into the fray.
A few years back, Rob Neyer and Bill James introduced a Cy Young Predictor formula in the Neyer/James Guide to Pitchers, a formula made possible by the relatively smaller number of statistical inputs which go into consideration for that award, and one that produced a much higher level of accuracy (around 80 percent) than JUMP does. In the end, a sharper mind than mine might well have produced an MVP prediction system with more direct hits, perhaps even a simpler, more elegant system altogether. Nonetheless, JUMP underscores both the wider variety of inputs that can come into play in a single MVP vote and the fact that nearly any given year produces at least a few candidates with strong enough statistical resumés and team backgrounds for a voter to attach to a narrative which rationalizes their vote. As with any season’s actual voting results, I strongly suspect we haven’t heard the last word on this topic.
A version of this story originally appeared on ESPN Insider .
Thank you for reading
This is a free article. If you enjoyed it, consider subscribing to Baseball Prospectus. Subscriptions support ongoing public baseball research and analysis in an increasingly proprietary environment.
Subscribe now
For the Yankees problem (ie, voters are reluctant to vote for players who have multiple excellent teammates), why not introduce a penalty for players in the system with a teammate (or two) who also score above a certain point level?
Another thought--i'd be curious to see if, in a close mvp race, voters are more likely to give votes to a guy who narrowly missed out on winning the award in a recent year...so give some bonus for second-through-fifth place finishes in the past three years maybe?
Great analysis! This was a pleasure to read.
Also, really cool article.
As it now stands, there is *no* positional adjustment for catcher (only the plus for middle IF and the minus for DH)? I'm still not seeing why the boost for catchers messes things up. It would boost Piazza into the top 3 several times? Shouldn't Piazza *be* in the top 3 several times? It seems like he was a perennial MVP candidate if not a perennial MVP. And it's not like it isn't messed up now without that adjustment, given the absured miss on Mauer and the bad miss on Pudge.
I certainly appreciate the effort, though. I suspect, as you suggested, that this is a bit quixotic. Good narratives drive the vote, and the statistical case is cobbled together later to verify whatever conclusion they'd already reached.
Every positional adjustment made in this process is a global one, so if I adjust for one catcher, I adjust for them all. With only one catcher winning during this time span, I'm not looking to create bonus that boosts Piazza or any other catcher into the top three, because that threatens other results that were right. Pudge is so far behind in 1999 that to get that single outlying vote right, I create an adjustment that gets four other votes wrong (Geovanny Soto for 2008 NL MVP, anyone?) and results in just 23 of the 28 MVP winners winding up in the JUMP top three.
I guess I did misunderstand you to be saying that your system says that Mauer is currently projected to come in 15th or so in voting. So the idea is whether you're 15th or 50th in points, if you're well outside the top three, you have almost no chance? OK.
When I said the system seems to badly miss on Mauer I wasn't talking about 2006, when of course it correctly predicted that he wouldn't win. I meant that it looks way off for 2009, since the idea that Mauer has little chance in 2009 seems incredible. Maybe time will prove you right, but it's hard for me to believe that. Especially since the lack of the catcher positional adjustment is so curiously defended.
To answer your question regarding Piazza, I didn't try to concern myself with the actual voting results beyond the winner. I make no guarantee that the player JUMP identifies as top three actually finished in the top three in the voting (though it wouldn't surprise me if a good number of them did) just that one of the players in the top three would actually win.
Without a positional adjustment, Piazza comes in 2nd in 1995, 9th in 1996, 7th in 1997 and 21st in 2000 - the years he finished in the top five of the actual voting. Jorge Posada finishes 11th in the rankings in 2003, when he placed third in the actual voting. Mauer finishes 2nd in the rankings in 2006 (sixth in the voting) and 20th in 2008. Taken together, the data suggests that the system as constructed is undervaluing catchers, and should have a positional adjustment, but without another data point for a catcher winning it's hard to figure out just how big that adjustment should be.
Also, Jay, for years you have been semi-apologizing for having your own last name in the JAWS name but yet you just duplicate the same thing with JUMP. It seems apparent to me that you like including your name in this acronyms so you can drop the bashful routine. :-)
Narrative FAIL.
If you designed two models - one for each league - I suspect you could be far more accurate with your predictions going forward.
Since your model seems to be more descriptive than proscriptive, I would hesitate to use words like "wrong" when someone not predicted by the model wins. It seems more like a failing on the part of the model to describe reality than a fault of reality to conform to the model.
You used quotes around the words I would take issue with, so perhaps you're cognizant of the issue but chose to use less rigorous language.
On a more positive note, I enjoyed the article and appreciate the effort. This dispels the myth that the MVP is all about RBIs from the perspective of the voters. Far more important in the eyes of the voters, apparently, is a team's record. I also found the emphasis on slugging stats (total bases, etc.) relative to on-base stats (walks, etc.) very telling (if not surprising). It seems baseball writers also dig the long ball.
I thought voters tend to vote for the best player on a team that *just* gets into the playoffs- so, for example, you are more likely to win MVP if your team gets into the playoofs in the last few days of a season versus being a great player on a 103 winteam.
I also thought that voters tend to vote for players with BIG late-season performances- like Jimmy Rollins, Ryan Howard or Chipper Jones- versus those with more consistent production, especially if that team gets into the playoffs late.
I know you can't throw everything into your system, but these made sense to me.
As for the late-season performances, I recall that Vlad's 2004 MVP had much to do with a fantastic September (.363/.424/.726), and the system got that vote right as well as Chipper's. I don't have any split breakdowns for these, so I'm not sure it's something I can easily incorporate.
Some of these narratives are mentioned in the previous comments. One criterion might be best player on a team that barely made the playoffs. Worth more points still would be best player who also performed well down the stretch on a team that finished strong to narrowly secure a playoff spot (as opposed to a team that stumbles to the finish and barely hangs onto a playoff spot). And maybe points in those cases should be scaled by how much better the player is than his teammates (a rough way of describing the "he carried his team to the playoffs singlehandedly" narrative). You would also get points for being by far the best player in the league (Mauer), fewer for being at least in the argument, and zero for anything else even if you had a really good season. You should also get points for doing something really spectacular for awhile, again like some of Mauer's stretches this year. The basic idea is that you need to do something to rise into the public's conscience and then sustain that awareness through the end of the season. Being really good over the course of a season but never jumping off the charts seems to only have a chance in the "toss-up" years like the AL in '08.
Those are just a few examples among many that I can think of, but the point is the MVP is driven by narratives so reverse engineering those narratives and their preference among voters is probably the best way to predict MVPs.
http://www.baseballprospectus.com/article.php?articleid=1306
http://www.baseballprospectus.com/article.php?articleid=1307
http://www.baseballprospectus.com/article.php?articleid=1312
I plugged the system into my spreadsheet and got 12 direct hits (it split two tiebreakers, getting 1998 NL right but missing on NL 2005) and 21 top threes.
Bernstein's work certainly lends some credence to the idea that different models may be needed for AL and NL voting.
I mess with who "should" get the MVP award though. What I like to do is take runs created from individual players and divide that number by a team's total runs...thus showing what percentage of runs a player creates for his team.
For purpose of my research in the past I had put a .550 winning percentage or a spot in the playoffs as a requirement for the award as I believe value comes in wins. Thus a run created for a competitive team would be more valuable in my opinion than a run created by a non-competitive team.
Using this strategy takes ballpark factors out of the equation. Obviously some players will have better numbers due to playing in Colorado or Philadelphia and some players numbers won't be as good because they play in San Diego or Oakland.
Fun stuff..
What I would suggest to you (and anyone else listening) is to use your preferred performance metric(s) (in your case, the percentage of runs a player creates for his team; for others, VORP, the WARPs, traditional slashes, whatever) and apply them solely to games which the candidate's team won.
That is, what was the player's percentage of runs created IN THE GAMES WHICH HIS TEAM WON? (Or what was the player's AVG/OBP/SLG/OPS in the games which his team won? Etc.)
Applying performance metrics to ALL GAMES a player plays in gives you a rough idea of how good (comparatively) a player he is. Using the same performance metrics only from GAMES WON may give a (rough) indication of how valuable the player was in those wins.
Imo, no player is valuable in a team's loss.
I know the ballot says Most Valuable Player to his team, but I can't help suspecting that the actual voting is by Most Valuable Player to the writer. That is, players that generate good copy get votes.
Has anyone done an analysis of correlation between articles written by the voters and resulting vote totals?
As an aside, I think any analysis concerning MVP voting would be made stronger by looking at vote totals of non winners rather than just trying to get hits on winners as this analysis is doing. While I understand that 2nd place and 10th place get exactly the same thing (nothing) and the tool is trying to predict winners, not project finish order, I think it promotes better understanding of the voter mindset if we look at vote totals rather than just the winner.
There, that's my 2 cents in. Thanks for your patience.
Perhaps that concern is simply beyond the scope of your efforts here, but I'm left with the question -- Why?
Why would you want to create an MVP predictor when you know that the actual MVP selection process is itself flawed?
Just asking. Nice work, by the way.
Second, digging into this stuff and figuring out how to gather and incorporate additional data is a huge process. It's certainly something that I may revisit, particularly with an eye on the award announcement in November, but in the context of filling a daily deadline here at BP, not every question is going to be answered in a first attempt.
Pure, not Applied, Research.
;-)
The other way is to just simply listen to the various media outlets. You can see them stumping various guys well before the end of the year.
Which is why I can see Mauer winning this year.
Teixeira is leading the AL in RBI's. But his OPS is LESS than last year, or the year before.
Any MVP discussion in the NL that doesn't begin and end with Pujols isn't really worth it.
I'm surprised because I thought that at least fans considered the post-season the post-season without regard to how you got there.