BP Comment Quick Links


June 8, 2011 Between The NumbersAnother look at ZiPSYesterday I claimed that the restofseason projections at Fangraphs are wrong. Again, I am well aware this is a statement from interest, as I made this claim as I was announcing our own restofseason PECOTA projections. Some people were skeptical of my illustrations, so I’ll go into the weeds here and explain why I think the methodology is incorrect, as opposed to simply disagreeable. First, I’ll lay out the methodology in neutral terms, then I will provide my own commentary. What I’ve done is recreate at least a portion of the methodology used to compute restofseason ZiPS, as based on this spreadsheet, particularly, the portion having to do with HR. (If you want to look at the methods yourself, just download that spreadsheet, and copy the batter worksheet into a blank sheet so you can unhide the locked columns in the sheet.) First, you need to take the number of games a player’s team has played, and divide by 162. This is called G% in the original spreadsheet. For clarity I will call this G_RT. Now you take that, along with a player’s preseason projected HR (which I’ll abbreviate ProjHR) and his season to date HR. To figure his projected restofseason HR according to ZiPS, you do this:
The first term of this is simply a weighted average, where a player’s preseason and seasontodate HR totals are averaged based upon weights of 11 and a value that floats along with team games. After 60 games, the weight is about 3, after 120 games the weight is about 6, after a full season it is of course 8. Note that these are total home runs, not home run rates. The second term simply prorates the weighted average out to the remaining games of the season. The method in the ZiPS restofseason spreadsheet uses park adjusted data in this step, and then converts to a player’s particular park; I did not have access to the specific park factors being used. And as a shortcut, I used a constant value of G_RT for all players, rather than compute the correct value for each individual player. And yet, even without those steps, I get a very good agreement with the values at Fangraphs – using the same data as I did for yesterday’s post, I come up with 23.09 projected HR for Bautista; the listed total with that data was 23. If I do this for all players, I get an average absolute error of .29; if I round the projected HR totals I did to whole numbers (as is done for the values I downloaded from Fangraphs), I get an average absolute error of .19. (If I limit myself to players with at least 100 projected AB restofseason, I get an average error of .19 as well.) So even with excluding the park factors, I get a very good agreement with the values currently being published; if Fangraphs has changed the methodology originally used in the spreadsheet I have, it’s been an extremely modest change. Everything is computed by that same formula: AB, H, 2B, 3B, BB, SO, etc. And once you’ve done that, you have restofseason ZiPS. Now, my thoughts: First, the weight for a player’s previous seasons is based on two factorsa constant of 11, and a player’s projected playing time. There is going to be a modest correlation between a player’s projected playing time and his previous playing time, of course, so in this sense I suppose there is some accounting for the reliability of a past projection, but it seems to occur almost entirely by accident. And for young players who are expected to hold down a job for a whole season (your Jason Heyward types, for instance) it treats those forecasts exactly the same as it does those for an experienced veteran with the same expected playing time. Secondly, it treats all events the same way. If you take two players with the same hit totals and atbats, one with a lot of HR and one with a lot of singles, the restofseason forecast at Fangraphs will treat those exactly the same when it comes to projecting their batting average going foward. The restofseason forecast also does not distinguish between a player whose low batting average is driven by a low BABIP, versus one whose low BABIP is driven by a high strikeout rate. And it uses the same arbitrary 11 and 8 weights for HR as it does for triples. Writing at ESPN today (how serendipitious!) Dan Szymborski, the creator of ZiPS and the inseason projection methodology, says: Not all bad seasons are created alike, though. For example, changes in walk and strikeout rate are far more likely to be retained going forward than changes in batting average on balls in play (BABIP). A homer outage for a home run hitter is a greater cause of concern than a singles hitter having a single outage. And he’s absolutely right; it’s just that the restofseason ZiPS projections don’t know those things. It treats changes in walks rates just the same as changes in hit rates. The pitcher projections are essentially the same as the batter projections. There is a reduced set of stats to project, as one might expect, but ERA, has some peculiarities of its own. A component ERA is used in place of standard ERA for projecting future performance, which would be comforting except:
Tthe pitching methodology means that a pitcher whose peripherals suggest he’s pitching in line with his preseason projection can still see his numbers change significantly because of ball in play outcomes, or bases empty/men on splits, or relievers allowing a greater or lesser number of inherited runners to score. Am I suggesting you shouldn’t use the restofseason ZiPS projections? Of course I am; but that is clearly a biased statement. So let me suggest instead, when looking at a player’s rest of season ZiPS forecast, ask yourself how much of the change is due to the player’s ability being better measured, and how much is due to the peculiarities in how the restofseason adjustments are being done.
Colin Wyers is an author of Baseball Prospectus. Follow @cwyers
17 comments have been left for this article. (Click to hide comments) BP Comment Quick Links RedsManRick (23592) Thank for the followup Colin. This answered my specific question. Jun 08, 2011 14:45 PM Brian Cartwright (4519) OK, I'll agree that I wouldn't use the ZiPS method, for many of the reasons you discuss. Jun 08, 2011 22:01 PM evo34 (33584) I also am more interested in what PECOTA is doing *right* than what Fangraphs is doing wrong. I.e., what weighting per stat is inseason PECOTA using and more importantly, what was the method used to arrive at these weightings? Jun 09, 2011 02:48 AM TangoTiger (57181) (Crossposted from my blog.) Jun 09, 2011 10:37 AM evo34 (33584) I don't think anyone is questioning how the theory of weighting by PA/IP works, or how one would implement it. The question is what are the optimal weights to use to provide the most accurate inseason forecasts? It's an interesting problem and one I would like to hear from Colin on. Jun 09, 2011 13:12 PM TangoTiger (57181) I've already provided that in the past. The weights is as follows: Jun 09, 2011 13:29 PM evo34 (33584) If the optimal weighting scheme is that well known and accepted, I'm not totally sure why none of the three inseason projection systems (ZIPs, Oliver and PECOTA) are using it. Or is PECOTA now using that weighting scheme? Jun 09, 2011 20:02 PM TangoTiger (57181) daysAgo is human days, because that's how a player ages, based on realtime. Jun 10, 2011 04:21 AM evo34 (33584) That's if you assume that whatever phenomenon causing past data to lose its predictive value over time is generated solely by a ticking clock. The alternative hypothesis is that a player is more likely to implement/undergo fundamental changes in the offseason (new delivery/stance, fitness change, nagging injury healing, etc.), and so the existence of the offseason would cause more information value to evaporate from the past performance than would the passage of the same amount of time within a continuous season. Jun 10, 2011 09:12 AM What I'm doing is treating the preseason forecast as a set of Bayesian priors, similar to what Nate Silver has done in the past: Jun 10, 2011 07:38 AM Brian Cartwright (4519) I'm well aware of Tango's daily decay weighting system, where days ago refers to calendar days. The 1.0/0.8/0.6 seasonal weights for batters, and 1.0/0.7/0.5 for pitchers conforms to those numbers. Jun 10, 2011 00:00 AM Brian Cartwright (4519) Discussion of this article at "The Book Blog" Jun 10, 2011 01:32 AM Not a subscriber? Sign up today!

I was one of the people who questioned you yesterday, and think this is a great follow up. tx.
Me too. As expected, I don't really understand this article, but that is an issue with my math skills, not the writing or methodology. Appreciate you going "into the weeds" to back up your statement.