CSS Button No Image Css3Menu.com

Baseball Prospectus home
  
  
Click here to log in Click here for forgotten password Click here to subscribe

Premium and Super Premium Subscribers Get a 20% Discount at MLB.tv!

<< Previous Article
Premium Article Daily Roundup: Around ... (04/28)
<< Previous Column
BP Unfiltered: Pitcher... (04/25)
Next Column >>
BP Unfiltered: The Wee... (04/29)
Next Article >>
Fantasy Article The Stash List: Seeing... (04/29)

April 29, 2013

BP Unfiltered

Starling Marte, Bryce Harper, and the Limits of WARP

by Russell A. Carleton

Earlier today, CBS baseball writer Jon Heyman asked a rather innocuous question on Twitter.

It's a hardly a clown question, bro.  
 
Heyman is citing the Baseball-Reference version of WAR, and as of this writing, the two men are rated as equal in their performance so far in 2013. Our own WARP has Harper at 1.9 WARP, with Marte at 0.8. That's not a knock on Marte. Harper has just been that good so far this year. The two systems use different inputs, and in this case, they spit out different results.
 
The answer to Heyman's question is that while Harper is the clearly superior offensive player (.361/.444/.756 for Harper; .331/.393/.444 for Marte as of Monday afternoon), Marte rates rather high on the defensive metric used by Baseball Reference (he is tied with Norichika Aoki for the best defensive player in MLB), while Harper rates as merely average. On FRAA (Fielding Runs Above Average), which we use at BP, Marte doesn't do as well, and as such, doesn't rate as highly as Harper overall.
 
One advantage of the WARP framework is that it incorporates defense into the overall understanding of a player's value. Defense matters. But how good are we at measuring defense?
 
There are two parts to that answer. One is whether our defensive metrics have construct validity. Are they measuring what we think they are? According to Baseball Reference (not to pick on them, the same question could and should be asked of all metrics), Marte has saved 8 runs above average in the field.  Has he really? That's construct validity. The truth is that fielding data are imprecise, often low resolution, subject to a lot of biases, and in some cases, not publicly available for inspection. It's not to say that all fielding metrics are garbage, but they aren't gospel either.
 
The other issue is whether our defensive metrics have good reliability. If we assume that we really are accurately measuring fielding, the next question is whether one month of a season is enough to tell whether it accurately predicts what's going to happen over the rest of the season. I'd argue that the answer is no, particularly for outfielders. Starling Marte very likely has gotten lucky over a small sample size.  That's what happens with metrics that have low reliability. He may very well be a good outfielder, but we just can't trust that his early season performance accurately reflects his overall talent level at this point.
 
Heyman points out a real issue with WARP, and with any measure. It is subject to the quality of the data that back it up, the assumptions that underlie it, and the inherent randomness in the game of baseball. WARP is a fantastically useful tool, but it has flaws.  Being imperfect isn't the same thing as being pointless. It just means that, despite the fact that it's one of the best ways of measuring player value that we have available, there's a lot more work to do.

Russell A. Carleton is an author of Baseball Prospectus. 
Click here to see Russell's other articles. You can contact Russell by clicking here

Related Content:  Starling Marte

2 comments have been left for this article.

<< Previous Article
Premium Article Daily Roundup: Around ... (04/28)
<< Previous Column
BP Unfiltered: Pitcher... (04/25)
Next Column >>
BP Unfiltered: The Wee... (04/29)
Next Article >>
Fantasy Article The Stash List: Seeing... (04/29)

RECENTLY AT BASEBALL PROSPECTUS
Premium Article Minor League Update: Games of Wednesday, Aug...
Fantasy Article Fantasy Freestyle: Week 21
Premium Article What You Need to Know: August 21, 2014
Pebble Hunting: You Lie!
Eyewitness Accounts: August 21, 2014
Premium Article Skewed Left: Saber Seminar One Year From Now
Fantasy Article Fantasy Freestyle: Information, Humans, and ...

MORE FROM APRIL 29, 2013
Premium Article The Prospectus Hit List: Monday, April 29
The Week in Quotes: April 22-28
Premium Article Bizball: Baseball's Marketing Problem Isn't ...
Fantasy Article The Stash List: Seeing What You Want to See
Premium Article Daily Roundup: Around the League: April 29, ...
Premium Article The Call-Up: Nolan Arenado
Fantasy Article Fantasy Freestyle: The Art of Trading

MORE BY RUSSELL A. CARLETON
2013-05-09 - Baseball Therapy: Should I Worry About My Fa...
2013-05-06 - Premium Article Baseball Therapy: What is a Good Hitting Coa...
2013-04-29 - Premium Article Baseball Therapy: On the Evolution of the Pa...
2013-04-29 - BP Unfiltered: Starling Marte, Bryce Harper,...
2013-04-23 - BP Unfiltered: Should Players Get Preventati...
2013-04-22 - Premium Article Baseball Therapy: What is a Good Pitching Co...
2013-04-15 - Premium Article Baseball Therapy: Boys Will Be Boys?: The Ca...
More...

MORE BP UNFILTERED
2013-05-02 - BP Unfiltered: Dissecting the Draft: Clint F...
2013-04-30 - BP Unfiltered: The Love Song Of P. Scott Pr...
2013-04-29 - BP Unfiltered: The Week In Albert Pujols Pla...
2013-04-29 - BP Unfiltered: Starling Marte, Bryce Harper,...
2013-04-25 - BP Unfiltered: Pitcher BABIP and Age
2013-04-24 - BP Unfiltered: More on Shutouts
2013-04-23 - BP Unfiltered: Should Players Get Preventati...
More...

INCOMING ARTICLE LINKS
2014-01-27 - Premium Article Baseball Therapy: Why Are Smart Teams Spendi...
2013-05-14 - Manufactured Runs: Listen to What the Heyman...
2013-05-14 - Premium Article Baseball Therapy: How Reliable Are Our Field...