keyboard_arrow_uptop

Many of you reading this have been reading my work for years, and you know
that I’m not a capital-S sabermetrician. I’ve been fortunate to work with
people who are, and I’ve leaned on them heavily over the years, but my own
research and math skills are limited to keeping Keith Woolner’s phone number
current and among the one-touch dials on my cell phone.

Yesterday, though, I presented a small
study
that looked at the differences found when you evaluate a team’s
starting pitching not by its aggregate numbers, but by the performance of
those players currently in the rotation. It stemmed from some thinking I’d be
doing about a gap in my analytical methods, and I was fairly happy with the
way it crystallized the ideas for me. We can’t just say “Team X is Nth in
the league with a 7.2 SNVA” and call that evaluating their rotation,
because the pitchers in that rotation may be much different from the ones who
compiled that figure.

I promised a second part to the study in which I would analyze team bullpens
in the same manner, and I spent a good chunk of Wednesday doing the research
and preparing the data. I used Michael Wolverton’s Adjusted Runs
Prevented
, and separated team bullpens into current core relievers (five
or six per team) and everyone else.

Now, even as I was doing the work I kind of thought ARP might not be the best
tool for the job, because it’s not a pure rate stat. It is a value metric that
has performance, context and playing time components, the latter two of which
make it a poor analogue for Support-Neutral
Winning Percentage
. Nevertheless, I went ahead with the research because I
thought using ARP would still be useful while being a much simpler calculation
than Runs Responsible Average, the rate stat from which ARP is derived.
(Calculating RRAs for the core relievers and the others is a manual task, and
no small one.)

I was wrong. The playing-time effect dominates everything, so much so that
using ARP in this manner only really tells you which teams are using pitchers
who they haven’t used all season. It’s a worthless data set that clouds,
rather than illuminates, the issue of which teams have the best bullpens right
now.

So what do you do when you’ve completed a study and you realize you haven’t
done it properly? I don’t know what the professionals do, but I’m starting
over. It’ll take me a couple of days, but I’m going to re-run this research
using RRA, which is the proper metric for this task.

As long as I’m learning lessons this week, I can take from this experience the
idea that you can’t cut corners in performance analysis, or make a statistic
do something it’s not designed to do. We see this all the time in mainstream
media, where accounting notes such as pitcher wins and runs batted in are used
to evaluate performance and even character, tasks for which they’re
ill-suited. My wasted Wednesday is the same thing with a different set of
tools.

Sports Illustrated has nothing on me.

You might recall that in both 2000 and 2001, I wrote fairly fawning articles
about the
Padres
just in advance of tailspins by the team. They dropped four
straight to fall out of fringe contention in August 2000, and lost 11 of 13
after the May 2001 piece. Craig Elsten still hasn’t forgiven me for the second
one.

Last Wednesday, afflicted with Expos Fever–hey, you try working with Jonah
Keri
–I wrote an article
that discussed their wild-card chances
. They managed to tie for the top
slot in the wild-card race on Thursday, but have lost six in a row since then
to fall five games behind the Marlins and Phillies and all but end their
postseason hopes.

I also pointed out Friday
that the Pirates, or what’s left of them, had crawled to within seven games of
the NL Central lead going into Labor Day weekend. Maybe they had no chance to
leapfrog three teams and win the division, but it seemed like something worth
pointing out. The Bucs have gone 2-3 since, falling to 8 1/2 games behind
whichever .525 team leads the Central today.

With all this in mind, I should point out that the UCLA football team looks
like it has a real chance to be a national story this year. With new coach
Karl Dorrell at the helm, and sophomore Matt Moore looking great in grabbing
the starting quarterback job, the Bruins could be on their way to an 11-1
season, perhaps even a spot in the Sugar Bowl. I’d be shocked if they weren’t
the Pac-10’s representative in the BCS.