September 4, 2003
Back to the Drawing Board
Many of you reading this have been reading my work for years, and you know that I'm not a capital-S sabermetrician. I've been fortunate to work with people who are, and I've leaned on them heavily over the years, but my own research and math skills are limited to keeping Keith Woolner's phone number current and among the one-touch dials on my cell phone.
Yesterday, though, I presented a small study that looked at the differences found when you evaluate a team's starting pitching not by its aggregate numbers, but by the performance of those players currently in the rotation. It stemmed from some thinking I'd be doing about a gap in my analytical methods, and I was fairly happy with the way it crystallized the ideas for me. We can't just say "Team X is Nth in the league with a 7.2 SNVA" and call that evaluating their rotation, because the pitchers in that rotation may be much different from the ones who compiled that figure.
I promised a second part to the study in which I would analyze team bullpens in the same manner, and I spent a good chunk of Wednesday doing the research and preparing the data. I used Michael Wolverton's Adjusted Runs Prevented, and separated team bullpens into current core relievers (five or six per team) and everyone else.
Now, even as I was doing the work I kind of thought ARP might not be the best tool for the job, because it's not a pure rate stat. It is a value metric that has performance, context and playing time components, the latter two of which make it a poor analogue for Support-Neutral Winning Percentage. Nevertheless, I went ahead with the research because I thought using ARP would still be useful while being a much simpler calculation than Runs Responsible Average, the rate stat from which ARP is derived. (Calculating RRAs for the core relievers and the others is a manual task, and no small one.)
I was wrong. The playing-time effect dominates everything, so much so that using ARP in this manner only really tells you which teams are using pitchers who they haven't used all season. It's a worthless data set that clouds, rather than illuminates, the issue of which teams have the best bullpens right now.
So what do you do when you've completed a study and you realize you haven't done it properly? I don't know what the professionals do, but I'm starting over. It'll take me a couple of days, but I'm going to re-run this research using RRA, which is the proper metric for this task.
As long as I'm learning lessons this week, I can take from this experience the idea that you can't cut corners in performance analysis, or make a statistic do something it's not designed to do. We see this all the time in mainstream media, where accounting notes such as pitcher wins and runs batted in are used to evaluate performance and even character, tasks for which they're ill-suited. My wasted Wednesday is the same thing with a different set of tools.
Sports Illustrated has nothing on me.
You might recall that in both 2000 and 2001, I wrote fairly fawning articles about the Padres just in advance of tailspins by the team. They dropped four straight to fall out of fringe contention in August 2000, and lost 11 of 13 after the May 2001 piece. Craig Elsten still hasn't forgiven me for the second one.
Last Wednesday, afflicted with Expos Fever--hey, you try working with Jonah Keri--I wrote an article that discussed their wild-card chances. They managed to tie for the top slot in the wild-card race on Thursday, but have lost six in a row since then to fall five games behind the Marlins and Phillies and all but end their postseason hopes.
I also pointed out Friday that the Pirates, or what's left of them, had crawled to within seven games of the NL Central lead going into Labor Day weekend. Maybe they had no chance to leapfrog three teams and win the division, but it seemed like something worth pointing out. The Bucs have gone 2-3 since, falling to 8 1/2 games behind whichever .525 team leads the Central today.
With all this in mind, I should point out that the UCLA football team looks like it has a real chance to be a national story this year. With new coach Karl Dorrell at the helm, and sophomore Matt Moore looking great in grabbing the starting quarterback job, the Bruins could be on their way to an 11-1 season, perhaps even a spot in the Sugar Bowl. I'd be shocked if they weren't the Pac-10's representative in the BCS.