April 26, 2006
Lies, Damned Lies
Underrating Mark Buehrle
There's nothing like doing something as presumptuous as ranking the 50 most valuable players in baseball to generate a lot of hate mail. Curiously, I received very few messages asking me why I had included a certain player. But perhaps 50 different players were mentioned as unjust exclusions, including everyone from Jorge Posada to Hanley Ramirez to Jonathan Papelbon.
We should keep in mind that baseball talent forms a pyramid or bell curve; the difference between the 30th-best player and the 70th-best player is probably less than the difference between the #1 player (Albert Pujols) and the #10 (Vladimir Guerrero). There is a lot of room for subjectivity in the back half of the list. That said, two particular names came up over and over again: Chase Utley and Mark Buehrle. Although the e-mails were often somewhere to the left of reasonable--one citizen of SportsNation compared me to Jay Mariotti--it is worth a re-examination to see whether there is anything to the wisdom of crowds.
I won't deal with Utley at length here. He was close to making the cut, and very well might have passed if I had purchased The Fielding Bible prior to writing, which rates Utley as an excellent defensive player rather than a merely adequate one.
But the Buehrle e-mails hit a bit closer to home, since he's one of my favorite baseball players. Like Utley, Buehrle was a late cut--if I had found room for one more starting pitcher, it would either have been Buehrle or Francisco Liriano.
Buehrle wouldn't have made the list based on a pure PECOTA version of the rankings. He'd have rated about 75th overall, and that's before accounting for the fact that the PECOTA version of these rankings looks at a five-year window, whereas I was considering six years for the ESPN version of the analysis. One player rated lower than Buehrle in the PECOTA rankings but made the list--Justin Verlander. (Justin Upton and Daisuke Matsuzaka also made the list, but don't have PECOTA scores.)
So the operative question is: is PECOTA underrating Buehrle? I made upward adjustments for other players, like Rich Harden and Victor Martinez, that I thought PECOTA was giving the short end of the stick. Should I have done the same for Buehrle? Why would PECOTA underrate a pitcher like Buehrle?
- PECOTA has calibrated Buehrle's key peripheral statistics incorrectly. I don't think this is likely to be the case. For one thing, this is the easiest part of a pitcher's forecast to calculate, particularly for a veteran who has had relatively stable statistics. For another, Buehrle's projected strikeout, walk, and groundball totals almost exactly match his three-year averages. The home run rate is just a tick higher, but it isn't a lot higher, and Buehrle is facing an awful lot of regression to the mean in that ballpark of his. Nor are there scouting factors that call Buehrle's numbers into question--he's been in the league for a long time, and has been pretty much the same pitcher all along.
- PECOTA regresses Buehrle's BABIP too much to the mean. One statistic I stumbled across in this process is the difference in BABIP based on the count. In 2005, the league had a BABIP of .306 when ahead in the count, and .291 when behind. This is a fairly substantial difference as BABIP things go, although it may be partly the result of a selection effect--better hitters tend to be ahead in the count more often. In any event, I thought it could be relevant to Buehrle, because he's a pitcher who loves to get the first strike. But it doesn't have much influence here--in fact, PECOTA is projecting Buehrle's BABIP to be slightly lower than it has been in the recent past. (As an aside, Buehrle is undeniably Madduxian in the way that he approaches each at bat. The first strike is not intended to set up a big breaking ball or splitter, as it is for many other pitchers, but rather to enable him to work the corners. After getting that first strike, Buehrle often works the count for pretty much the rest of the at bat, including if the count goes to 2-1 or even 3-1).
- PECOTA isn't determining Buehrle's projected ERA correctly from his peripheral stats. Trust me when I say that this isn't a problem with PECOTA in general. I've worked tirelessly to ensure that PECOTA projected ERAs accurately match the peripherals.
PECOTA also allows a pitcher's ERA to diverge from his peripherals based on his past history, or the performance of his comparables, although it still regresses to the mean quite a bit. I thought this might be an issue for Buehrle, since he's a little bit like Tom Glavine
in the way that he pitches to the situation. Glavine, during his prime, routinely posted ERAs that were well lower than his PERAs. Most of this wasn't luck--it was because Glavine was skilled enough to tend to give up walks when walks hurt him least, home runs when home runs hurt him least, and so forth. Here is Buehrle's performance by base state over the period 2003-2005:
Bases BB/HBP | Single | XBH | Out (not K)| K
Empty 4.8% | 18.2% | 7.2% | 53.7% | 16.0%
Runner on 1st Only 5.1% | 18.9% | 6.8% | 55.9% | 13.3%
Runner(s) in Scoring Position 9.3% | 17.0% | 8.4% | 52.7% | 12.6%
Note that Buehrle gives up a lot more walks with runners in scoring position. This is natural and effective strategy. With the bases empty, a walk is every bit as good as a single. With runners on, it can be either a little bit less valuable (if it forces a runner one base when a base hit might have advanced him two), or a lot less valuable (if there's an open base at first). This is indicative of a pitcher who has good command of the at bat, as well as good command of his pitches.
On the other hand, Buehrle realizes the benefit to this trade-off in the form of allowing fewer singles, and not allowing fewer extra base hits. In fact, he gives up quite a few extra base hits with runners in scoring position. Because of this, his ERAs have historically been very close to his PERAs, and PECOTA is expecting the same going forward.
One small factor to consider is that Buehrle has an excellent pickoff move, which is something else that PECOTA doesn't account for. Over the course of his career, runners have stolen just 26 bags against Buehrle, and been caught 35 times. This is almost certainly Buehrle's doing--the White Sox have had catchers with average or poor throwing arms over most of the past five or six years. This ability is worth about two runs per season to Buehrle, or 8-10 points worth of ERA.
- PECOTA has selected inappropriate comparables for Buehrle. Buehrle comes from a relatively common pitcher family--the strike-throwing, groundball-inducing lefty--and his comparables are mostly in this vein. Glavine, and Jim Kaat make his list. So do Ken Holtzman and Jim Abbott--Abbott fell off a cliff at 28, and Holtzman at 29. But that's part of the game, so to speak, and Buehrle's overall attrition rates aren't especially high. That said, Buehrle's numbers would probably hold up a little bit better if PECOTA were allowed to consider Buehrle's (completely pristine) injury history in more detail, rather than just making inferences about it. PECOTA is treating Buehrle as pretty darn durable, when it should be treating him as really damn durable.
- PECOTA is getting the ERA and stuff about right, but it's valuing Buehrle improperly given these numbers. There are two elements to consider here, one of which works against Buehrle and the other works in his favor. The unfavorable factor is that Buehrle gives up a lot of unearned runs--his RAs have been .41 points higher than his ERAs over the course of his career. This isn't an accident. Pitchers give up unearned runs more frequently when they induce a lot of groundballs, and when they put the ball in play. Buehrle does both of these things in spades, which makes him an unearned run magnet. Our valuation metrics, which are based on projected RA rather than ERA, appropriately ding him for this characteristic.
The other factor is that the PECOTA long-term rankings are based partly on Upside
, and PECOTA doesn't consider Buehrle to have a lot of Upside. That is, PECOTA thinks that Buehrle is very likely to be very good, but that he's very unlikely to ever have a Cy Young type of season on account of his middling strikeout rates--he isn't likely to turn it up a notch as pitchers like Ben Sheets
and Chris Carpenter
did. Taken in a vacuum, that seems like a fair enough assessment. But there's a question about whether upside (capitalized or not) is something that we should really be thinking about for a veteran pitcher. Subjectively, it seems to me that a league average performance from a starting pitcher is worth quite a bit more than a league average performance from a position player (and therefore that a
somewhat-above-average performance is also worth more). In the near future, I hope to apply the Freely Available Talent approach
to pitchers, as I did for the position players. In the meantime, it might be proper to give Buehrle a little bit of "extra credit" for this ambiguity (but only a little bit).
So we've identified two or three small things that might tend to underrate Buehrle--the pickoff move, the clean injury history, and the treatment of his risk/reward profile. Are these things enough to warrant a ranking in the Top 50? I think that they probably are, considering how closely bunched together everybody is, and that the #50 slot should probably have gone to Buehrle or Utley, rather than the "cuter" selection of Matsuzaka. (I'm going to be more stubborn about Justin Verlander, as that is a scouting judgment of mine; I really honest-to-god would not trade Verlander for Mark Buehrle.) That said, we should remember that for a pitcher with a below average strikeout rate to be so much as considered for a Top 50 list is an achievement unto itself, and says a lot about all the other things that Mark Buehrle does so well.
Nate Silver is an author of Baseball Prospectus.
Click here to see Nate's other articles.
You can contact Nate by clicking here