April 20, 2005
Lies, Damned Lies
It was only minutes after I submitted last week's column to Joe Sheehan that I received the following e-mail from him:
So, here's the first question that popped into my head...if there's something of a self-selection process going on here--you get on these lists by being good for a long time, and low hit deltas are one way to be good for a long time--wouldn't a necessary test be to compare this research with pitchers known for other pitches?Joe's a sharp guy, as are all the readers who submitted similar questions, but this is something I've got little excuse for not catching myself. About the first rule of any hypothesis test is that you need to include a control group, and especially so if you're going to spout off a pretentious line or two about the scientific method. I didn't include a control group.
So let's look at lifetime Delta-H numbers for curveball pitchers; a curveball, after all, is often slower than a change-up. As with last week's article, I've taken the names of excellent curveball pitchers from two sources: the Neyer/James Guide to Pitchers--both the top ten and honorable mentions are included--and a series of USA Today scouting articles from last spring training. Technically speaking, the USA Today describes "best breaking ball" rather than "best curve"; I excluded a couple of pitchers like Pedro Martinez who are better known for their slider.
Mordecai Brown -168 Tommy Bridges -140 Sandy Koufax -99 Barry Zito -83 Sal Maglie -77 Vic Aldridge -76 Johnny Morrison -71 Mike Mussina -57 Kerry Wood -32 David Wells -29 Guy Morton -29 A.J. Burnett -26 Dave McNally -25 C.C. Sabathia -19 Tom Gordon -16 Clem Labine -15 Eric Gagne -14 Sam Jones -12 Kaz Ishii -12 Roy Halladay -4 Gregg Olson -1 Rosy Ryan +4 Josh Beckett +9 Darryl Kile +11 Roy Oswalt +11 Jeremy Affeldt +12 Brad Penny +16 Bert Blyleven +17 Matt Morris +18 Johnny Schmitz +24 Earl Whitehill +34 Johnny Sain +35 Mike Witt +37 Dwight Gooden +55 Camilo Pascual +65 Herb Pennock +123 AVERAGE -15Twenty-one of 36 curveball pitchers had Delta-H numbers below par. The group average was -15 career Delta-H, as compared with -40 for the change-up pitchers. If you exclude Mordecai Brown, who had possibly the best defense in history working behind him, the average works out to -11. Of course, my methodology is a little bit sloppy--a career total means more for Bert Blyleven than it does for Josh Beckett--but the sample size is larger here, and it looks like great curveball pitchers also do a little bit better than the norm.
However, it's the fastball pitchers who are probably going to decide this. Neyer and James have several lists of fastball pitchers; I included their top ten overall starters with honorable mentions, as well as their top ten relievers, but didn't look at the separate era-by-era lists. As before, I also included pitchers from the USA Today lists.
Cy Young -372 Walter Johnson -298 Amos Rusie -237 Lefty Grove -120 Virgil Trucks -106 Sandy Koufax -99 Nolan Ryan -95 Firpo Marberry -86 Roger Clemens -83 Troy Percival -81 John Smoltz -81 Jim Maloney -80 Bob Feller -79 Joe Wood -72 Bob Turley -65 Herb Score -60 Mariano Rivera -56 Curt Schilling -46 Robin Roberts -45 Bartolo Colon -43 Goose Gossage -40 John Wetteland -40 Billy Wagner -34 Kerry Wood -32 Tom Henke -28 Dick Radatz -20 Randy Johnson -20 C.C. Sabathia -19 Eric Gagne -14 Jason Schmidt -9 Rafael Soriano -5 Roy Halladay -4 Dazzy Vance +1 Jouett Meekin +3 Kyle Farnsworth +4 Rob Dibble +6 Mike MacDougal +6 Josh Beckett +9 Sam McDowell +10 Jeremy Bonderman +11 Joe Page +18 Lee Smith +19 Mark Prior +23 Rube Waddell +51 AVERAGE -52Well, that's pretty overwhelming. The group average is -52 career Delta-H, which is a fair bit better than the change-up pitchers. It isn't all Cy Young and Walter Johnson, either: just 12 of 44 fastball pitchers had a Delta-H above zero, and only one of those by a statistically significant margin.
Perhaps it's something about having one particularly great pitch that leads to a superior BABIP? This one is a little bit harder to test, but we'll try. So far as I can tell, there are 11 pitchers who have been elected to the Hall of Fame by the writers (that is, Veteran's Committee selections are excluded) who did not appear prominently on any of the Neyer/James "best of" lists, including a top five placement on the era-by-era fastball lists:
Pete Alexader -270 Jim Palmer -251 Tom Seaver -250 Catfish Hunter -237 Fergie Jenkins -225 Juan Marichal -160 Don Sutton -152 Warren Spahn -126 Dennis Eckersley -116 Whitey Ford -25 Rollie Fingers +18Now, most of these guys had great repertoires, even if they didn't have any one absolutely world-class pitch. Juan Marichal, for example, threw as many as five very good pitches at certain points of his career; it's hard to sustain an argument that Marichal didn't have great "stuff." Nevertheless, it's pretty clear which way the arrow points.
It seems to me that we have two interrelated problems here, both of which are very sticky in terms of research design. The first is that it is difficult to distinguish pitchers with great "stuff" from great pitchers, period. About the only guys who thrive in the majors for any length of time with below average "stuff" are folks like Tommy John and Dan Quisenberry, who are on the absolute left end of the bell curve in terms of both command and home run prevention, and I don't know that there are enough of those guys to constitute a reasonable sample. You'll sometimes hear the term "stuff" used as a euphemism for throwing fast--as in "Jamie Moyer succeeds without having great stuff"--but Moyer has an outstanding change-up, as do folks like Greg Maddux and Kirk Rueter.
Did Steve Dalkowski have a great fastball if he couldn't throw it for strikes? Does Jon Garland have a great sinker if he can't get anybody out with it? Except perhaps for radar-gun readings, there is really no objective way to measure "stuff." A scout might describe a prospect as having a great curveball, but the pitcher is not likely to make any post-facto "best of" lists if he goes 6-14 in his major-league career.
Which leads into our second chicken-and-egg problem: this is a highly self-selected list. Being considered to have great stuff has an awful lot to do with being a great pitcher, but what does being a great pitcher have to do with? Well, getting folks out, preventing runs and winning games. And, so the skeptical viewpoint goes, if you're just plain old getting lucky on the outcome of balls in play, you're also going to get lucky in those departments. Perhaps Catfish Hunter or somebody put together an entire Hall of Fame career simply by being very, very lucky on his BABIP.
I don't really buy it. While there's a lot of noise in the data, there's also a lot of data, and the data points toward better pitchers having a small, but statistically significant ability to prevent hits on balls in play. I won't belabor this point too much, since Tom Tippett has already done such an effective job of doing so.
What I will talk about a little bit more is my experience with PECOTA. PECOTA needs to predict BABIP, just as it predicts everything else, and it uses all the tools at its disposal in order to do so. This includes previous BABIP rates, strikeout and walk rates, groundball and home-run rates (groundballers allow a higher BABIP), and even hit-by-pitch rates--it turns out that guys who hit a lot of batters are just a little bit better at preventing hits. Throw all that stuff into a regression equation, including multi-year averages for the various predictors, and sophisticated adjustments for park, league and defense effects, and it turns out that you're able to explain around seven percent of variance in BABIP based on a pitcher's prior statistical track record.
Seven percent. Meaning that 93 percent is a matter of defense and luck.
Whether we could improve upon that seven percent by including pitch-type data remains an open question. My guess is that we probably could, at least a little bit, given a sufficient sample of data, but it would be a matter of different pitches being especially effective for different pitchers--perhaps the curveball really is a great "out pitch" for Barry Zito--rather than any one variety of pitch being more effective across the board, save perhaps for the knuckleball.
At the very least, it seems that I have to revise the claim that I made last week. Great pitchers have lower BABIPs, if only by a little bit, and great pitchers throw great pitches, but change-ups in particular don't appear to have anything to do with it.