I’ve been thinking about the way I use advanced performance metrics such as Michael Wolverton’s Support-Neutral pitching statistics. In evaluating a team, I’ll often quote aggregate data such as their relievers’ Adjusted Runs Prevented or their starters’ Support-Neutral Value Added as a way of showing how effective they’ve been, and as shorthand for what we can expect them to do going forward.
This can be deceiving. Of course I know that past performance isn’t an exact predictor of future results, and that a group of players can be over or under their heads in the short- or medium-term. That’s not necessarily the problem; the problem is that the aggregate totals have been compiled by a group of players who are not necessarily representative of the team at the present time.
For example, just last week I mentioned that the Expos’ starters ranked sixth in the National League in SNVA. That’s true, but it’s past performance, and it includes six wretched starts by four pitchers who aren’t going to be used again, as well as the good pitching of Claudio Vargas, who may not take the mound again this year due to a shoulder injury. The SNVA figure is useful in letting us know how the Expos’ rotation has performed to date, but to get an idea of its current quality, it’s best to look at the pitchers taking the ball every fifth day. Isolating the performance of the current five Expos starters yields an SNPct of .569, 20 points higher than the team’s seasonal .549 mark.
The Expos aren’t special in this regard. Twenty-five of the 30 major league teams have current rotations with better SNPcts than their seasonal numbers, which you’d expect in a game so fiercely Darwinian. The pitchers whose stats have soiled the aggregate numbers are back in the minors, on the disabled list or running for governor, and they’ve been replaced by better performers. The only notable exceptions to this are two teams who each lost their best pitcher: the A’s, with Mark Mulder‘s hip injury, and the Orioles, who traded Sidney Ponson. The Angels have virtually identical seasonal and current SNPcts, while the Braves have gotten just three extra-rotation starts and the Mariners none.
Who looks the best once you adjust for this? The following chart ranks teams by SNPct based on the five pitchers currently being used. In a few cases, four starters were used in calculating current rotations, usually because it was unclear who #5 is. (The perils of early-September performance analysis.)
Through September 1 SNPct Rank Season SNPct Rank ------------------------------------------------------------ Diamondbacks .632 1 .596 2 Giants .617 2 .539 9 Cubs .600 3 .542 8 Royals .590 4 .549 5 A's .589 5 .598 1 Dodgers .579 6 .551 4 Yankees .578 7 .548 7 White Sox .578 8 .562 3 Expos .569 9 .549 6 Indians .561 10 .501 14 Marlins .560 11 .537 10 Astros .536 12 .519 13 Mariners .526 13 .526 12 Braves .526 14 .527 11 Phillies .524 15 .501 15 Red Sox .508 16 .489 17 Mets .508 17 .456 23 Cardinals .506 18 .487 18 Pirates .501 19 .495 16 Reds .495 20 .393 29 Blue Jays .493 21 .469 21 Twins .483 22 .481 20 Orioles .479 23 .482 19 Devil Rays .471 24 .405 27 Angels .463 25 .463 22 Padres .461 26 .415 25 Rockies .460 27 .450 24 Brewers .456 28 .413 26 Rangers .433 29 .382 30 Tigers .427 30 .404 28
As I mentioned, everyone looks better this way, but look at the relative changes. The Giants, having added Sidney Ponson and Jerome Williams in-season, have the second-best rotation in the game. I cheated a little with the Cubs, who make their leap on the basis of Juan Cruz effective Thursday, a move that has yet to be confirmed. The A’s slip without Mark Mulder, the Orioles slide without Ponson and the Pirates move down without Jeff Suppan. The White Sox are third overall, but they haven’t improved the back of their rotation, while the teams around them are no longer using the likes of Chris George, Andy Ashby and Jeff Weaver.
I think looking down the list can provide some interesting information. The Indians have used the 2003 season to sort through their young starters, and appear to have been successful in finding ones who can pitch in the major leagues. The Mets, Reds, and Devil Rays are all getting better work from their rotations than their overall numbers would indicate. I wonder if this information–the teams whose rotations improved the most during the season–might be an indicator of future success, of teams who made the transition from using warm bodies to starting pitchers capable of contributing to winning teams.
This method isn’t without its problems. For one thing, the Royals’ placement on this list is out of whack, largely because their rotation for these purposes consists of three guys with 10 starts for them combined, and the good work of Darrell May. (I couldn’t discern their fifth starter, and I’m not sure Tony Pena himself knows.) Take that information with a grain of salt, but also consider that it illustrates a key point about the Royals: it’s hard to say what they’ll get out of their starters over the next four weeks. The data for the Reds, Indians, Rockies and Orioles all have a similar problem, as all those teams have made significant changes to their rotations in the past month.
I also ran into the question of what to do with multi-team pitchers. While it largely comes out in the wash–only Jeff Suppan‘s performance has been significantly different across two teams–whether to credit a pitcher’s current team with his past performance, to increase the sample size and more accurately reflect ability, was an issue. I elected not to, so as to avoid double-counting two-team pitchers, but I can see an argument both ways.
The key point, however, is that the aggregate data I’ve been using can be misleading. It’s not enough to cite a team’s rank in some metric–even well-crafted ones like Support-Neutral stats–as evidence of that team’s level of ability, not when rosters and roles are so fluid. The Cubs, with a huge difference between their rotation with and without Shawn Estes, are perhaps the best example, but the list above provides plenty of other reasons to do more than just run a finger down a list in doing analyses. It’s an important lesson for me, one of the biggest things I’ve learned this year.
Tomorrow, I’ll run this same analysis for bullpens. I think the differences will be even greater because of the damage a few bad outings by staff fodder can do to overall numbers, and therefore, the evaluation of a team’s relief corps.
Thank you for reading
This is a free article. If you enjoyed it, consider subscribing to Baseball Prospectus. Subscriptions support ongoing public baseball research and analysis in an increasingly proprietary environment.
Subscribe now