CSS Button No Image Css3Menu.com

Baseball Prospectus home
  
  
Click here to log in Click here for forgotten password Click here to subscribe

<< Previous Article
Premium Article Prospectus Perspective... (11/09)
<< Previous Column
Manufactured Runs: Rep... (11/03)
Next Column >>
Premium Article Manufactured Runs: Der... (11/11)
Next Article >>
Internet Baseball Awar... (11/09)

November 9, 2010

Manufactured Runs

How Do You Solve a Problem Like Derek Jeter?

by Colin Wyers

One of the things about being a baseball analyst as a writer is that I have the luxury of taking the broad view. If I am right about most players, I’m doing a pretty good job. If I miss on a player here or there…I don’t enjoy it, and I try and learn from it, but it’s not devastating.

A general manager, on the other hand, is responsible for about 25 players or so—more if you include everyone who could conceivably play on a team in one season, fewer if you limit it to players who will end up having (or missing) enough playing time to have a real impact. Missing on one player can, in fact, be devastating.

All of which is to say—I don’t envy Brian Cashman right now. He has a lot of tough choices to make about Derek Jeter—and if he slips up, Yankees fans aren’t likely to be forgiving. Even if he does the right thing, Yankees fans may not be understanding.

While none of us shares Cashman’s unique burden, the outright refusal of MLB to play even a single game between now and March means a lot of us will satisfy our desire for baseball by following the Hot Stove League, and Jeter’s contract negotiation figures to be the star attraction. So, let’s ask, how much is Jeter worth?

Figuring out how much Jeter’s bat has been worth is relatively trivial; it’s also pretty uncontroversial. Here’s runs above replacement player for Jeter over the past five seasons from us, Baseball Reference, and FanGraphs, excluding defense:

Year

RARP

rRAR

fRAR

2006

61

67

71.7

2007

43

61

53.6

2008

29

32

37.2

2009

55

62

64.7

2010

25

24

28.4

Total

213

246

255.6

Everyone seems to be using subtly different definitions of replacement-level offense, but past that there seems to be solid agreement between the three as to what Jeter has been worth with the bat.

The question the Yankees have to answer, at least in terms of batting, is this: was 2010 simply an aberration for an otherwise exceptional hitter, or was 2009 an Indian summer masking Jeter’s age taking its toll? What our projections can tell us is the most likely answer (or, more accurately, the answer with the least presumed error), but there’s still uncertainty around that forecast, and that uncertainty compounds the more years you tack onto the contract.

(The Yankees also need to come to grips with where in the lineup Jeter’s bat belongs. Barring a substantial bounce-back, it’s hard to make the case that the Yankees are best served with the Captain leading off every game).

But still, looking simply at Jeter’s bat and his position, he seems like an average to above-average player over the next few seasons, and the Yankees seem to lack decent alternatives (either internally or available through free agency). It seems like a simple decision to retain Jeter in pinstripes, doesn’t it?

The trouble is fielding.

Let’s look at several common fielding stats, all expressed in terms of runs saved compared to the average player at the position—Defensive Runs Saved as published by Baseball Info Solutions, Ultimate Zone Rating (based on the same BIS data as DRS), Sean Smith’s TotalZone, BP’s current implementation of FRAA, and our forthcoming overhaul of FRAA. Looking at years for which all of those metrics are available:

Year

DRS

UZR

TZ

FRAA1

nFRAA

2003

-13

-4

-14

-23

-38

2004

-12

-0.7

5

-9

-17

2005

-28

-14.9

-5

-5

5

2006

-18

-7.3

-3

-11

-30

2007

-23

-17.9

-14

-19

-27

2008

-9

-0.3

-5

-9

-33

2009

2

6.4

4

-11

-34

2010

-13

-4.7

-10

-4

-22

Total

-114

-43.4

-42

-91

-196

Someone relying on UZR or TotalZone might reasonably conclude that Jeter’s defense was subpar, but unlikely to deter the Yankees from wanting him to return. Looking at DRS and FRAA1, one might be more skeptical.

New FRAA, however, casts more substantial doubts on the question.

I’ve discussed what new FRAA does before, but a refresher is probably in order. Simply put, we count how many plays a player made, as well as expected plays for the average player at that position based upon a pitcher’s estimated ground-ball tendencies and the handedness of the batter. There are also adjustments for park and the base-out situations; depending on whether there are runners on base, as well as the number of outs, the shortstop may position himself differently, and we account for that in the average baselines.

The other metrics use other data to come to their estimate of expected outs—in the cases of UZR and DRS, it’s batted-ball and hit location data measured by BIS video scouts. In the cases of TZ and FRAA, it’s data collected by press box stringers working for MLB’s Gameday product.

(TotalZone and FRAA1 both incorporate some batted-ball data from MLB’s Gameday product from 2005 on; this means that what we’re calling TZ and FRAA1 aren’t exactly the same thing depending on season. GuyM has explained why range bias seems to exist in pre-hit location metrics, which is probably beyond the scope of this discussion).

So let’s examine nFRAA for all seasons Jeter played in the majors:

Year

CH

PM

AVG_PM

PAA

MOE_PM

RAA

MOE_RUN

1995

385

33

36.1

-3.1

6.0

-2.7

5.0

1996

4069

403

402.2

0.8

19.0

0.7

16.3

1997

4149

407

424.3

-17.3

19.3

-14.5

16.1

1998

3837

349

374.4

-25.4

18.3

-21.3

15.3

1999

4081

335

386.1

-51.1

18.4

-44.2

15.9

2000

3900

300

353.7

-53.7

18.1

-46.8

15.8

2001

3749

308

350.2

-42.2

18.0

-35.1

15.0

2002

4135

339

394.8

-55.8

19.0

-45.9

15.6

2003

1878

173

218.1

-45.1

14.0

-37.6

11.6

2004

4100

344

364.5

-20.5

18.3

-17.1

15.3

2005

4173

400

394.1

5.9

18.7

4.9

15.3

2006

3941

338

373.8

-35.8

18.3

-30.2

15.5

2007

4046

335

366.5

-31.5

18.4

-26.5

15.5

2008

3738

307

347.4

-40.4

17.5

-33.4

14.5

2009

3563

301

342.2

-41.2

17.4

-33.8

14.3

2010

3703

315

342.8

-27.8

17.7

-22.3

14.2

Total

57447

4987

5471.3

-484.3

70.2

-405.7

58.8

Over the course of his career, Jeter has made nearly 500 fewer plays than we would expect a shortstop to have made. MOE_PM represents the margin of error around our estimate of an average shortstop’s plays. What I want to re-emphasize is that the margin of error doesn’t scale linearly—the margin of error for three seasons is smaller than the margin of error for each of those seasons added together.

Looking only at 2003-2010, the cumulative MOE (expressed in runs, not plays) for those seasons is 41.209. In other words, 68 percent of shortstops with that number of chances will have their “actual” value within 41 runs of the value estimated by FRAA—that’s one standard deviation. Assuming a normal distribution, 99.73 percent of players will have their actual value within three standard deviations of the estimate.

The estimates of Jeter’s defense provided by UZR and TZ are about 3.7 standard deviations away from what nFRAA says. It is staggeringly unlikely that Jeter would end up with a batted-ball distribution that cost him so many opportunities based upon random chance alone. DRS is just under two SDs away (roughly, the 95 percent confidence interval)—still very, very unlikely.

If not random chance, then what else might it be? Is there something else that could be so consistent across Jeter’s career? We’ve already controlled for park factors, ground-ball tendencies of the pitchers, and the handedness of opposing hitters. He’s played in nearly 40 ballparks (including two home parks), behind nearly 80 starting pitchers, with two managers (one of whom was still catching for the Yankees when he started playing). Since Jeter became the full-time shortstop for the Yankees in 1996, the team has had six different starting second basemen and five different starting third basemen (not to mention the numerous backups to each). The single-greatest constant to Derek Jeter’s career has been, well, Derek Jeter.

The appropriate response is that the probabilities assume we’ve plucked Jeter at random; of course we haven’t. But we haven’t cherry-picked Jeter because he’s an isolated case. He’s an example of a systemic problem with fielding metrics based on observational data—range bias.

It’s a fairly straightforward matter to determine whether a play was made or not—primarily, you figure out if the batter was safe or out after putting the ball in play. It’s nearly as simple to figure out who made the play; typically the idea of “first touch” is used—who was the first fielder to contact the ball? So "plays made" is mainly an assertion of fact. Expected outs, derived from observation of where the ball was hit and how it travelled there, is more difficult. Here’s a still frame grabbed from a highlight of Jeter making a play:

Jeter about to field a ground ball.

What I want to emphasize is that, other than Alex Rodriguez, the only real reference point to where the ball is when it reaches Jeter is Jeter himself. Because of the way baseball telecasts are shot, this isn’t an isolated incident—this is how baseball looks on TV. And companies like BIS get the same video feeds the rest of us get.

So a player’s range seems to influence his expected outs. How can we tell this? The first data point we have is the spread of observed fielding performance—the spread of observed performance in metrics like UZR and DRS is much, much smaller than that of metrics like nFRAA (or Tom Tango’s With Or Without You system, which is similarly down on Jeter’s fielding ability).

The other thing we see is that a player’s expected outs as a share of team balls in play (or “balls in zone,” a proxy measure for expected outs) persists from season to season, even when looking only at players who switch teams. In other words, the identity of the player seems to change the recorded batted-ball distribution (We have no mechanism that would allow us to explain how a fielder could dramatically change the actual distribution of batted balls; it seems much easier for him to be impacting the estimates).

All of the available evidence seems to suggest that Jeter is a worse fielder than most defensive metrics indicate, perhaps on the order of 20 to 30 runs below the average shortstop. This makes it possible that Jeter, in 2010, was performing at roughly the same level as a typical replacement—in other words, his ability to hit like something resembling an average shortstop doesn’t offset his inability to field like one. And while Jeter’s bat may improve next year from a disappointing 2010, over the next three years it’s a fair bet that his hitting will continue to erode. That’s what happens to baseball players as they get closer to 40.

No, I don’t envy Brian Cashman at all.

Colin Wyers is an author of Baseball Prospectus. 
Click here to see Colin's other articles. You can contact Colin by clicking here

Related Content:  Derek Jeter,  The Who,  Managers Of The Year,  DRS,  TotalZone

40 comments have been left for this article.

<< Previous Article
Premium Article Prospectus Perspective... (11/09)
<< Previous Column
Manufactured Runs: Rep... (11/03)
Next Column >>
Premium Article Manufactured Runs: Der... (11/11)
Next Article >>
Internet Baseball Awar... (11/09)

RECENTLY AT BASEBALL PROSPECTUS
Premium Article The View from the Loge Level: The Alex Jacks...
Fantasy Article Fantasy Freestyle: Power Outage
Premium Article What You Need to Know: September 19, 2014
Premium Article Raising Aces: Carlos in Charge
Notes About Baseball: The Second-Hardest Par...
Premium Article Pebble Hunting: Cueto's Quirks
Premium Article Moonshot: Detecting the Best Medicine

MORE FROM NOVEMBER 9, 2010
Internet Baseball Awards: National League
Premium Article Prospectus Perspective: Five Deals That Must...
Premium Article So You Need: Catchers
Premium Article GM for a Day: Colorado Rockies
Another Look: The Joe Morgan Trade
Premium Article On the Beat: Scioscia's Tree is Branching Ou...

MORE BY COLIN WYERS
2010-12-03 - BP Unfiltered: Remembering Santo
2010-11-24 - Premium Article Manufactured Runs: Expanded Coverage
2010-11-11 - Premium Article Manufactured Runs: Derek Jeter And The Philo...
2010-11-09 - Manufactured Runs: How Do You Solve a Proble...
2010-11-09 - Between The Numbers: Glove checks
2010-11-03 - Manufactured Runs: Replacing Replacement
2010-10-25 - Premium Article BP Unfiltered: Making sense of ratings
More...

MORE MANUFACTURED RUNS
2011-01-19 - Manufactured Runs: The Twilight of the Gods
2010-11-24 - Premium Article Manufactured Runs: Expanded Coverage
2010-11-11 - Premium Article Manufactured Runs: Derek Jeter And The Philo...
2010-11-09 - Manufactured Runs: How Do You Solve a Proble...
2010-11-03 - Manufactured Runs: Replacing Replacement
2010-10-14 - Manufactured Runs: Just a Bit Outside
2010-09-22 - Manufactured Runs: A Walk in the Park
More...

INCOMING ARTICLE LINKS
2011-05-10 - Manufactured Runs: The Deconstruction of Fal...
2010-11-24 - Premium Article GM for a Day: New York Yankees
2010-11-11 - Premium Article Manufactured Runs: Derek Jeter And The Philo...
2010-11-11 - Premium Article So You Need: Middle Infielders
2010-11-09 - Between The Numbers: Glove checks