CSS Button No Image Css3Menu.com

Baseball Prospectus home
  
  
Click here to log in Click here for forgotten password Click here to subscribe

<< Previous Article
Premium Article Future Shock: Yankees ... (03/08)
<< Previous Column
Reintroducing PECOTA: ... (02/08)
No Next Column
Next Article >>
Fantasy Article Everyone's Perfect: Th... (03/09)

March 8, 2012

Reintroducing PECOTA

House of Cards

by Colin Wyers

Without further ado, we present to you the PECOTA cards. Debuting on the cards are the 10-year forecasts and the percentile forecasts.

The 10-year projection process breaks down like so:

  • First, a player’s performance is broken down into a series of components. These are more detailed than the official stats breakdown, featuring everything required to build a full batting (or batting against) line, including reach on error, infield single rates, etc.
  • Single-season aging curves for all of the components are built upon comparing adjacent seasons. There is a selection bias in adjacent seasons, so we use a back-weighted sample of the player’s past performance (using the same process we use in the normal PECOTA forecasts), regressed to the mean, to build the single-season aging curves.
  • We use a curve-fitting process to “smooth” the aging curve, to make sure the progression is orderly and to diminish the effect of random variance in the aging curves (especially for very young or very old players, where the pools are much smaller).
  • To come up with multi-year forecasts, we “chain” the smoothed single-year age adjustments.
  • Each player has a custom aging curve built using the comparables, and that curve is tested for reliability and then regressed to a generic aging curve based on that reliability assessment.

Peak ages will vary based upon a player’s comparables and his skill set (as each component gets its own aging curve). But typically for hitters, we see peak ages in the 10-year forecasts at around age 28, a year later than the conventional wisdom. This is somewhat offset by a decline in a player’s defensive value, which doesn’t really peak at all but starts to decline almost immediately upon his debut in the majors. (You can see the effects in this aggregation of various 10-year forecast components by age, weighted by DC playing time, here.)

Pitchers are more interesting—there seems to be an earlier peak, at age 26, for pitchers, in terms of ERA. But for pitchers who manage to survive beyond age 26, there seems to be a much later peak. Pitchers who pitch past ages 27–29 seem to peak around age 30, with some pitchers peaking even later. So pitching seems to be essentially bimodal in aging, where some pitchers peak early and others peak late. This is even true if we restrict our analysis only to pitchers who work primarily as starters their entire career. (Again, a breakdown is available here.)

Playing time in the 10-year forecasts is a reflection of what we expect a player’s playing time to be at his peak production, rather than starting with his expected 2012 playing time; this results in more sensible long-term forecasts for young prospects who aren’t quite ready for MLB yet but are expected to have productive careers when they do make it to MLB. Playing forecasts for off-peak years are then adjusted from the peak-year playing time forecast.

Now, projecting the future is more difficult the further out it goes. So how reliable are the 10-year forecasts? Looking at the root mean square error of projected 10-year True Averages for “backcasts” of historic players:

Year

  RMSE

1

0.031

2

0.032

3

0.033

4

0.035

5

0.037

6

0.039

7

0.041

8

0.043

9

0.046

10

0.050

This is about what we would expect; a player’s performance 10 years down the road is substantially more difficult to project than his performance one year down the road. But especially through the first several seasons, the reliability of the forecasts is not substantially different.

Similarly, for pitchers:

Year

RMSE

1

1.18

2

1.21

3

1.23

4

1.26

5

1.29

6

1.30

7

1.35

8

1.41

9

1.49

10

1.56

Again, results in the first several seasons are very close, with results becoming harder to project the further out you get. (Bear in mind that these are forecasts for a neutral park and league context and thus will exhibit higher RMSEs than regular PECOTA forecasts.)

The percentiles are based on three primary variables:

  • The reliability of a player’s forecast, estimated based on the playing time (weighted) that went into the forecast,
  • The projected playing time, which affects the amount of random variance expected from the forecast, and
  • The population tendencies (a player is more likely to underperform his forecast than overperform it if he is projected to be above the league average, for instance).

Again, playing time is based upon a pitcher’s expected performance—the better the performance, the more playing time we expect for that player.

Keep in mind that the percentiles key off of the primary value component—TAv for batters, and ERA for pitchers (although the ERA is a component ERA with less variance than actual ERA, as it does not account for random variance in sequencing around the component lines). Component stats are meant to illustrate the key value stats only—a pitcher’s 90th-percentile home run forecast, for instance, is not his maximum home run potential but the most likely home run total to accompany his 90th-percentile TAv.

How well do the percentiles do on historic data? Looking at back-forecasts from 1950 on, we see that 79 percent of observed TAvs fall within the 10th and 90th percentiles, and 60 percent fall between the 20th and 80th percentiles, exactly what we should expect.

As a reminder, PECOTA on the cards is restricted to subscribers only. To those of you who already subscribe, thank you for your patronage, and I hope you find the PECOTA cards useful, informative, and (employers of America, forgive me) a veritable time sink. Enjoy.


Additionally, you can look at progressions for different age groups over time, weighted by projected 2012 playing time according to the depth charts. Pitchers are located here. Hitters can be found here.

UPDATE: The ten-years forecast shows seasons two through ten of a player expected to play all ten seasons. For players whose forecast falls below the attrition rate, no forecast is displayed.

Colin Wyers is an author of Baseball Prospectus. 
Click here to see Colin's other articles. You can contact Colin by clicking here

45 comments have been left for this article. (Click to hide comments)

BP Comment Quick Links

brooklyn55

10 year forecasts? They seem to be well hidden. A hint?

Mar 08, 2012 00:58 AM
rating: 0
 
BP staff member Joe Hamrahi
BP staff

They are at the bottom of each player's card for subscribers. See the lower section of Albert Pujols card: http://www.baseballprospectus.com/card/card.php?id=PUJOLS19800116A

Mar 08, 2012 04:58 AM
 
Hoff

Nope.

Mar 08, 2012 05:17 AM
rating: 0
 
schmub

not seeing them either.

Mar 08, 2012 05:20 AM
rating: 0
 
BP staff member Rob McQuown
BP staff

This was due to a mistake in setting permissions on my part - only Beta Test customers were able to see them until 9AM ET this morning. My apologies - they should be fine now.

Mar 08, 2012 06:15 AM
 
Joe Hornback

I have a premium subscription and don't see them either....

Mar 08, 2012 05:20 AM
rating: 0
 
TucsonTumbleweed

Okay its not that I didnt have my morning caffeine!

Mar 08, 2012 05:46 AM
rating: 0
 
slingblade73

Nothing, where are the comparable players? I do not see them either.

Mar 08, 2012 05:56 AM
rating: 0
 
BP staff member Dave Pease
BP staff
(2)

hi, we're still finishing the comps stuff... everything else should be up there now. thanks.

Mar 08, 2012 10:34 AM
 
slingblade73

Thank you.

Mar 08, 2012 19:44 PM
rating: 0
 
Eddie

I see them on Pujol's card. But they only go to age 40.

Mar 08, 2012 06:06 AM
rating: 0
 
Mooser

you dont stick around long once your below replacement

Mar 08, 2012 06:10 AM
rating: 0
 
BP staff member Rob McQuown
BP staff

We have the data to project further out, but this comment captures the reason some players don't have all the years in the future projections - they are cut off at -1 WARP, representing players who are likely done. We are discussing whether to expand this a bit, as it's sort of fun to see *how* bad certain players might be.

Mar 08, 2012 07:33 AM
 
Mooser

I do see them now. I think the WARP projections may not include fielding or positional adjustment. A number of better fielding players TAvs are increasing but WARP is less than current projections (see Brett Lawrie)

Mar 08, 2012 06:16 AM
rating: 0
 
jtanker33

Minor Player Card error: If you look at the "2012 Forecast" section for pitchers, the hits allowed column is showing HRs allowed. Brandon Morrow and Josh Beckett are the two I looked at.

Mar 08, 2012 06:35 AM
rating: 0
 
BP staff member Rob McQuown
BP staff

Fixed, thank you.

Mar 08, 2012 06:54 AM
 
Grizpin

Happy to finally see the new cards. I missed them this morning since I gave up looking for them lol!

Mar 08, 2012 07:01 AM
rating: 0
 
gerrybraun

Can someone give a quick explanation of the value to be derived from Ten Year Forecasts? They show long term trends, obviously, but is there any insight to be gleaned from the granular data?

Mar 08, 2012 07:35 AM
rating: 1
 
pakdawgie

Really like the 10 year forecasts with respect to keeper/dynasty leagues. One question that I have: is there are a way to extract a certain specific statistic projection (say 2015 VORP) for all players non-manually?

Mar 08, 2012 07:41 AM
rating: 1
 
swarmee

I had a similar request. Someone like Brandon Nimmo with a projected MLB start year of 2017 seemed like someone who would have some sort of MLB projection in 2015. Do they not show up since they're under -1 WARP?

But I would love to see some type of Excel spreadsheet with all non-MLB players 2015 projections or a mechanized way to grab them off the website.

Mar 08, 2012 08:44 AM
rating: 0
 
BP staff member Colin Wyers
BP staff

We'll see what we can do here - I'm warning you in advance it'd be a really, really big spreadsheet, though.

Mar 08, 2012 09:19 AM
 
Grizpin

Looking at a few of the 10 year forecasts for some younger players and they seem to be in line with this years projections. No real sign of improvement or regression. The previous upside projections seemed to show more of a projected value with one number. Maybe I'm missing something?

Mar 08, 2012 08:07 AM
rating: 0
 
Grizpin

Looking at it a little closer there is a bit of a projected gain/loss but nothing that shows a marked improvement or decline in comparison to the current year projections.

Mar 08, 2012 08:14 AM
rating: 0
 
doog7642

Thanks for the work. UPSIDE scores are coming soon?

Mar 08, 2012 08:08 AM
rating: 0
 
BP staff member Colin Wyers
BP staff

UPSIDE is coming soon, yes.

Mar 08, 2012 08:18 AM
 
Grizpin

Good news!

Mar 08, 2012 08:23 AM
rating: 0
 
aroland17

Thanks for the great work as always! Found one slight player error through first glance when looking at my fantasy dynasty team. Stephen Strasburg ten year projections have him making 46-47 starts per year.

Mar 08, 2012 08:31 AM
rating: 0
 
BP staff member Colin Wyers
BP staff

This is changed on the cards now; Strasburg was being projected at 5 IP per start, which accurately reflects his usage up until now but doesn't reflect the IP totals in his 10-year forecast (which are based on how many IP a pitcher of his talent level typically gets at peak performance). That's fixed now.

Mar 08, 2012 10:01 AM
 
MichavdB

Thanks! I just checked out Vlad Guerrero though and noticed that on his bio his 2012 projections how no Batting avg and no tav (both are included in his forecast further down though).

Mar 08, 2012 08:49 AM
rating: 0
 
BP staff member Rob McQuown
BP staff

Fixed.

Mar 08, 2012 10:34 AM
 
tbwhite
(361)

I find the 10 year forecasts a little disappointing because they don't really give you any idea about the upside or downside that a player might have. If it would be possible to display the p25 and p75 performance for each year I think that would lend some valuable context. For example, given two players who were projected for .270 TAv for the next several years it would be great to know if one had a p75 forecast of .290 and the other just .280. Not sure if that's possible, but thought I would throw it out there.

Mar 08, 2012 09:06 AM
rating: 3
 
BP staff member Dave Pease
BP staff
(2)

Thanks for the comment. If I remember correctly we've never provided that before. We can look into adding that somewhere.

Mar 08, 2012 09:53 AM
 
BP staff member Rob McQuown
BP staff

We are making improvements to the UPSIDE formula for 2012, which extends out 10 years. This should help address part of this need, we hope. Look for it soon.

Mar 08, 2012 14:05 PM
 
Stephenwdavis

I believe that there is a serious bug with the 10-year projections for some low minors pitchers. It's almost like there's a multiplier off by a decimal place or something. Here are some examples to review:

Zachary Rosscup, James Pugliese, Yao-Lin Wang, Robert Whitenack, Ben Wells, Hayden Simpson, Willengton Cruz, Luis Liria, Austin Reed, Hector Mayora, Sheldon McDonald, Jose Rosario, Starling Peralta, Ryan Hartman, Santo Rodriguez, Yilver Sanchez, Pete Levitt, Dustin Fitzgerald, Daniel Berlind, Colin Richardson, Jin-Young Kim, Joseph Zeller, Su-Min Jung, Cameron Greathouse, Marcus Hatley, Juan Serrano and Larry Suarez.

Mar 08, 2012 13:42 PM
rating: 1
 
BP staff member Colin Wyers
BP staff

Hrm - some of the cards appear to be caching older runs, rather than the ones in the database. We're working on this now, should have it addressed soon.

Mar 08, 2012 14:15 PM
 
BP staff member Rob McQuown
BP staff

These projections have been removed, they shouldn't have been getting displayed, as they didn't project to have WARP>-1.

Mar 08, 2012 14:24 PM
 
oakiegu007

What happened to the scrubs and stars chart from years past? Is that no longer available?

Mar 08, 2012 15:05 PM
rating: 1
 
BP staff member Dave Pease
BP staff
(2)

hi there. We switched graphing solutions and have not recaptured a lot of the old PECOTA charts. We are planning on releasing charting upgrades throughout the year. Thanks.

Mar 08, 2012 15:37 PM
 
oakiegu007

Dave,

Does this mean there is no way of seeing those numbers in a spreadsheet? Or is that information simply not accessible any longer?

Thanks

Mar 08, 2012 21:58 PM
rating: 0
 
BP staff member Dave Pease
BP staff
(2)

Well, the stars and scrubs chart was a visual representation of information that we do currently have, but we don't have it all rolled up in a chart. Does that answer the question?

Mar 10, 2012 21:18 PM
 
oakiegu007

Yes it does. Though the end result makes for a sad reader :)

I'm not sure how you got that info so I'm not sure how to figure that out from what is on the pecota card now. Bummer!

Mar 11, 2012 22:13 PM
rating: 1
 
Shaun P.
(676)

Hmmm . . . as a Yankees fan, I'm curious to see what the 10 year forecasts project for some career numbers for the left side of the infield, as they are both high on some all-time lists, and of course neither will actually play another 10 years, so the forecasts should have some value!

With the 30 HR forecast this year, plus the 10 year forecast, PECOTA currently projects A-Rod to hit 770 HR (1st all-time), with 2334 RBI (1st all-time), and 2276 runs scored (2nd all-time).

Jeter's forecast calls for him to finish with 3680 hits (4th all-time) and 2034 runs scored (8th all-time).

They'd be the only pair of teammates to ever retire with over 2000 runs scored each.

It will be interesting to see how close those end up to being right, and what the projections look like at this time next year!

Mar 09, 2012 14:31 PM
rating: 1
 
Ben F.

I'm guessing something is wrong with how saves are being projected for certain pitchers.

I looked at Heath Bell's 10 year forecast and noticed that the number of saves projected is very high. His 65 for 2013 would set the single season record except that Mariano Rivera is projected for 71. In 2018, when his ERA is projected to be over 5, his WARP below 0, Rivera still projects for 41 saves.

Mar 11, 2012 08:42 AM
rating: 1
 
BP staff member Dave Pease
BP staff
(2)

Thanks, we're looking at this.

Mar 12, 2012 12:15 PM
 
Ogremace

it would be great of there were a projected totals line along with the ten years, for the ten years alone and for career.

Mar 11, 2012 16:22 PM
rating: 1
 
You must be a Premium subscriber to post a comment.
Not a subscriber? Sign up today!
<< Previous Article
Premium Article Future Shock: Yankees ... (03/08)
<< Previous Column
Reintroducing PECOTA: ... (02/08)
No Next Column
Next Article >>
Fantasy Article Everyone's Perfect: Th... (03/09)

RECENTLY AT BASEBALL PROSPECTUS
Premium Article The Call-Up: Austin Hedges
Premium Article Raising Aces: Postmortem: Chris Sale vs. the...
Premium Article What You Need to Know: Jake!
Premium Article Monday Morning Ten Pack: May 4, 2015
Premium Article Rubbing Mud: The Best Rivalry In Baseball (R...
Premium Article The Call-Up: Eddie Rosario
Rubbing Mud: The Worst Rule In Replay

MORE FROM MARCH 8, 2012
Premium Article Prospects Will Break Your Heart: Spring Trai...
The Lineup Card: 10 Choices for 2012 Decline...
Premium Article On the Beat: Yumania
Premium Article Painting the Black: The Show Review
Fantasy Article Resident Fantasy Genius: Fantasy Tier Rankin...
Fantasy Article Preseason Value Picks: Starting Pitchers for...
The BP First Take: Thursday, March 8

MORE BY COLIN WYERS
2012-04-19 - Between The Numbers: A Funny Thing Happened ...
2012-04-04 - Manufactured Runs: Tragedy of the Commons
2012-03-23 - Extra Innings Excerpt
2012-03-08 - Reintroducing PECOTA: House of Cards
2012-02-08 - Reintroducing PECOTA: The Weighting is the H...
2012-01-10 - BP Unfiltered: Watching Jack Play
2012-01-09 - BP Unfiltered: Suspicious Minds
More...

MORE REINTRODUCING PECOTA
2012-03-08 - Reintroducing PECOTA: House of Cards
2012-02-08 - Reintroducing PECOTA: The Weighting is the H...
2011-02-07 - Reintroducing PECOTA: They're Here!
2010-10-01 - Reintroducing PECOTA: The Seven Percent Solu...
More...