One of the reasons we started Baseball Prospectus was to point out the biases
within the baseball industry that were affecting player evaluation. We’ve
worked hard to establish the ideas that great athletes don’t necessarily make
great baseball players, that command is as important to pitching as throwing
hard is, and that hitters tend to follow a predictable career path.

We traded infallibility for a package of draft picks, though, so along the way
damaging biases have crept into our analyses, the same way that they did in
traditional evaluation. If performance analysis is going to continue to make
inroads as both a perspective for covering baseball and a decision-making tool
for management, its practitioners will have to understand these biases and how
they corrupt the process.

The best-known aspect of the modern school of baseball analysis is its
emphasis on how often a hitter reaches base. A shift from the old way of
evaluating hitters just based on their batting average, or how often they got
a hit, the new mindset values walks and the high OBPs they help to create.
Where BP has gotten into trouble is in getting overly excited about players
with exceptional walk rates who bring little else to the table. Over the
years, we’ve touted players such as Jackie Rexrode and
Mark Johnson who had big seasons, featuring high walk totals,
in the minors. None of these players were able to replicate that performance
at higher levels, with their batting and slugging averages dropping as
pitchers learned to challenge them.

Kevin Youkilis, called up last month by the Red Sox, is just
the latest example. Youkilis, referred to in Michael Lewis’ Moneyball
as “The Greek God of Walks,” garnered attention for his high batting
averages and doubles power, as well as his amazing eye at the plate. Promoted
to Triple-A last year, Youkilis was overmatched, batting .165 and slugging
.248, and he opened 2004 putting up similar numbers for Pawtucket. He
eventually adjusted and was hitting .258 with a .406 slugging average when he
was promoted to the majors. With the Sox, Youkilis is at .273 with a .434
slugging. For the first time in his career, Youkilis is striking out more
often than he walks. His eye is a great asset, but he’ll have to keep hitting
at least .270 and drive the ball for doubles to be a viable contributor.

These players make it clear that a high walk rate isn’t enough for success. A
prospect with a great batting eye has to be able to hit the ball hard enough,
often enough, to both have value beyond the walks and to avoid seeing a steady
diet of strikes as he ascends the ladder.

At the major-league level, you can see this misplaced emphasis on plate
discipline in the A’s signing of Scott Hatteberg to a
three-year contract. Hatteberg, to whom Lewis devoted an entire chapter,
personified the patient approach that the A’s want from their hitters.
However, he doesn’t do enough else at the the plate to be valuable. Despite
his plate discipline, Hatteberg was one of the worst regular first basemen in
the American League last year. If he hits .319 with power–as he is at this
moment–he can be an asset, but that’s the point: he has to do more than walk
to be a productive player.

Our tools for evaluating offensive performance are well-developed. Our tools
for measuring defense are not nearly so precise. Because we have a better idea
of what a player contributes at the plate than in the field, we have tended to
overvalue offense at the expense of defense. Over the years, BP has advocated
that teams should find spots for players who could hit and do virtually
nothing else.

Defense matters, though. It’s possible to be so bad with the glove, so awkward
on your feet, that you’re unable to play a position on the field without
hurting your team. Jack Cust is a heck of a hitter, but a
disaster with the glove at first base and in the outfield. Jeremy
has a career OBP of .377, yet he is so bad defensively that he
can’t hold a job.

This blind spot is already being fixed. As our defensive metrics improve, we
get better at evaluating a player’s contributions with the glove. We’re
getting better at balancing offense and defense in our work, and finding the
line where a good hitter gives back too much with his glove to warrant a
lineup slot. Moreover, teams on the cutting edge of performance analysis are
moving beyond the work of outsiders and developing proprietary systems for
gauging defensive value. The A’s, who moved from a slugging team to a
pitching-and-defense team over three seasons–ditching the same Jeremy Giambi
along the way–are at the top of the list of teams working to apply advanced
techniques to defense. Their belief in their system motivated their trade for
Mark Kotsay and their pursuit of Mike
over the winter.

In the 1980s, Bill James investigated the career paths of baseball players and
discovered that hitters had a bell-curve shaped path with a peak around the
age of 27. This notion is one of his most important discoveries, and drives
much of what performance analysts do, particularly in prospect projection.
Performing well while young relative to the other players in the league is a
hallmark of a top prospect.

It’s also occasionally the hallmark of a guy who just peaked before he was old
enough to buy a drink. Javier Valentin hit .324 wth 19 home
runs in 383 at-bats in the Midwest League as a 20-year-old. Baseball
Prospectus 1997
dubbed him the best catching prospect in baseball. Nine
years later, Valentin has 616 major-league at-bats and a career slugging
average of .344. Wilson Betemit was our #5 prospect in 2002,
coming off his .355 batting average as a 20-year-old in the Southern League.
Two seasons later, he has a 48/14 strikeout-to-walk ratio in Triple-A, and
growing questions about whether he’ll ever hit in the majors.

With the recent spate of players caught falsifying their ages, it makes sense
to be skeptical of players who put up great seasons in their teens. They might
have great talent, or they might just have their cousin’s birth certificate.
In the above cases, though, there’s no three-birthdays-in-one explanation; the
players just didn’t develop, and they serve as a reminder that prospect
projection remains inexact. Individuals occasionally surprise and disappoint
us, which is great. If they didn’t, this would be a very boring gig.

It’s not just hitters we can get wrong. The emphasis on performance and
control of the strike zone that leads to infatuations with certain hitters
also leads to the touting of pitching prospects with great strikeout-to-walk
ratios. In some cases, though, those pitchers succeed because their superior
command allows them to baffle inexperienced hitters. Painting the corners with
88-mph fastballs can work in the majors, but with rare exception, a pitcher
has to be able to miss bats to succeed in The Show.

The scouting bias against players of this type is real, and so is the fact
that BP has, perhaps in reaction to that bias, been too enthusiastic about
people like John Stephens, Ed Yarnall and
Mike Meyers. They all put up amazing control numbers in the
minor leagues, and all saw their careers peter out well before reaching
stardom. The Orioles produced Stephens, Josh Towers and
John Maine (currently at Triple-A) in a three-year span, and
none can be expected to have more than a token major-league career.

Steve Woodard has a career major-league strikeout-to-walk
ratio of better than three-to-one, and better than five-to-one at Triple-A over the past
three years. He’s found himself in the Red Sox and A’s organizations the past
two seasons, two teams who use performance analysis to make decisions, and yet
he still can’t get a major-league job. Minimizing walks is great, but it takes
more than command to pitch well in the major leagues. Woodard’s
“stuff”–that scouting touchstone–isn’t good enough to keep
major-league hitters from pounding him into submission.

All things considered, performance analysis brings a lot to the table, and a
successful team has to incorporate it into its player evaluation. It’s not
perfect, however, and any application of the method has to include an
understanding of the biases it can introduce. Just as an affection for young
men who are strong and fast can lead to a system full of decathletes that
can’t hit a baseball, evaluating players based on walk rates and power can
lead to having too many prospects who can’t catch one. The best approach, the
one that’s going to put the most wins on the board, takes the best information
from each method.