keyboard_arrow_uptop

Welcome to our hastily-implemented beta tester program.  If you can read this, you've been vocal about quality in the past, and we want to know what you think, so I've added you to our beta tester user group, and you'll have access to data early.  (Please pardon my presumption if you aren't interested; you can either ignore this whole thing altogether, or shoot me an email at dpease@baseballprospectus.com and we'll remove the designation on your account.)  Thank you to dianagram and other users for suggesting this program.

The first order of business is the PECOTA cards.  As I said last week, we're generating the pitcher cards and we should have them out in the next couple of days.  In the meantime, we've got the third and hopefully final revision of the hitter cards ready for you to look at.

Please click here to access the beta testers only version of the cards.

Notes on this version:

  • We haven't set up a beta-tester-only header, so any links to the PECOTA cards in the header, and the search box in the header, will still take you back to the public PECOTA beta cards.  Sorry–we'll flesh out the program to handle this more gracefully as we get it established.  In the meantime, use your browser's "Back" button, or just head straight to the beta tester index to visit other players.
     
  • We've fixed the OBP bug for the 2010 percentiles projections.
     
  • No Kyle Gibon and Dustin Ackley yet.  We might be able to get them–I'm not sure.
     
  • Only 20 comps per for these, but all 100 will be there once we switch these over to their final resting place.
     
  • The stats in the "biographical" box are the playing time projection adjusted stats at 50th percentile, if a player has playing time projected.  Any differences in rates should be the result of roundoffs.  For example, consider Albert Pujols.  At his 50th percentile, Pujols is projected to go 164 for 511, for a .321 batting average. His playing time projection (used in the depth charts and PFM) changes that from 608 PA to 679, which results in (679/608*511) = 571 at bats and (679/608*164) = 183 hits, but 183/571=.32049, rounding down to .320.  If a player does not have any playing time projected, the stats in the box are his PECOTA 50th percentile projection.
     
  • We changed the color of the totals and weighted means lines to be easier to read.

The beta testers only version of the hitter cards is available at http://www.baseballprospectus.com/ddjp.

Please leave any comments you have right here, or feel free to contact us with them.  Thanks again for your help.

You need to be logged in to comment. Login or Subscribe
tbwhite
3/18
First, thanks for this opportunity. Second, I think it's a step in the right direction. Third, I think things are looking better. But...... Looking at Josh Hamilton, under his 10 year projection I see a TAv of .296 for 2010, but when I look at his 2010 projection by percentiles, that looks like it would be a level of performance around his 85th percentile. So, something doesn't quite make sense.
ccmonter
3/18
Yes, it appears that only players with last names up to R are included right now. S through Z don't show up under any circumstances.
clayd
3/19
We've got a player named Rivero whose raw pecota information was screwed up, enough to break the script used to process all the raw files...no one after him was processed. I'll figure out why his pecota was broken...or make sure he gets ignored if I can't.
dpease
3/19
They're all in there now--sorry about that.
jivas21
3/18
That's because his weighted mean projection is for a .296 TAv this year. See my comments below ... the differences between the medians and the weighted means are unusually large.
clayd
3/19
The TAv and EqR don't match up...and its the TAv that is wrong. Should be .279 for that EqR.
rbross
3/18
thanks for including me on this. I'll try to take a closer look at things later in the day but my first observation is that if a player has a last name toward the end of the alphabet they don't seem to appear on the index (whether searched by position or team).
ccmonter
3/18
Yes, it appears that only players with last names up to R are included right now. S through Z don't show up under any circumstances.
dianagramr
3/18
Thanks for implementing the suggestion. I'll be busy tonight, but hope to take a good look at this tomorrow.
Junts1
3/18
Its both spring break and my birthday today, but I will try and take a look at these this weekend and early next week. I gave a look at Matt Kemp just for starters: 1: I love the layout, it is much cleaner and easier to read. 2: I like the added tAv data for comparables 3: The links to nearly everything ever said about the player are wonderful With that said, I am moderately surprised by PECOTA's read of Matt Kemp: That age 24 was his career year by a fair margin. That might be because so many of his 20 comparables are downward arrows, though. Nonetheless, it is awesome to be able to see at a glance how those comparables performed in the past, and I know I will really appreciate that feature. It makes the context of comparable players so much more useful.
jrmayne
3/18
First off, I salute the effort. Things that look wrong: 1. Some of the Weighted Means issues I've discussed earlier look fixed (Druby, for instance.) But the TAv weighted means still look way wrong (see, for instance, Nolan Reimold.) Adding the percentiles and dividing by nine should come close; it doesn't and in fact no matched pair (90+10/2, 80+20/2) comes close. (I see tbwhite points this out also.) 2. The ski-slope playing time problem in the 10-year PECOTA's still exists. Jesus Montero's not going to be a half-time player in six years hitting 300/357/519. Jason Heyward, same deal. 3. ccmonster's right. 4. Something's wrong with the EqA translations. For 2009, St. Louis reads as a pitcher's park for Holliday (Eq's: 380/437/658 on raw data of 353/426/604) but a neutral park for Glaus (172/250/241 for both). This may be yet another rounding issue due to low PA's by Glaus in St.L, but if so, it's a good reason to not round; there's especially no reason to round to generate EqA's. Nate did not round for anything; the display of homers was decorative, but the actual Excel numbers had plenty of entries like 11.975 homers. Going to a rounding system has generated errors. If you are measuring old P with the rounding, you're going to get the projected slash stats wrong. Good luck. Off to work. --JRM
clayd
3/19
What is happening in point 2 is trying to find a way to represent the chance so being out of the league. Nate's version in prior years made a binary in/out decision, based on 50% attrition (he says off the top of his head, without double checking)...but the guy is either in or out, across the 75, 50, and 25 percentiles, either all in or all out. I am quite certain I do want to represent that a player is out of baseball faster under his 20% scenario than his 80% scenario, but it does introduce the problem of how those out-of-baseball years play into the weighted means. Without introducing a fake line (100 Ab of typical pitcher hitting, say)...which I strongly do NOT want to do...the best solution I've found is to cut into the PA. I don't particularly like it, but I haven't come up with better. The full Heyward projection for 2016 has TAvs between .324 (550 PA, comping to Juan Gonzalez, Miguel Cabrera and Adam Dunn at the 90% side) to .232 (113 PA, comping Chris Snelling, Jeff Reed, and Darnell Coles) at 10%. The weighted mean will tilt the rate stats strongly towards the high end, but the PA still get held down.
Junts1
3/19
I think the answer here something pecota does in other areas: those seasons should just be dropped from the weighted averages for out of baseball results. That lead to wierd bounces where people's weighted means rose as they aged sometimes (because they had to be out-doing their present line of performance in order to stay in baseball, so their weighted means for those years were essentially their 75th or higher percentile scores, since everything else was rendering a drop result .. Barry Bonds and Randy Johnson and some other very old players used to get these kinds of weighted mean charts). It isn't immediately obvious, but anyone making serious use of 10 year weighted means is informed enough to understand it, I think.
BurrRutledge
3/20
Clay et. al, thank you for including me in the beta-test program. I will probably learn more from participating in this venture than you will, so much appreciated. Without reading all the other comments below, I'd like to chime in on a potential anomaly I'm seeing in the 10-year projections of both Heyward and Montero, both age 20. I think we can all agree that these are guys who we would like Pecota to nail. And I am just not confident that is entirely the case. Looking at their 10-year projection SuperVORP and TAv charts, both of them peak at age 23 - in every one of the projection percentiles. Age 24 and 25 are declines - across the board in Heyward's case, and with some Age 25 stability in the upper projections for Montero. After Age 25, and into their so-called peak years, we see a slight spread in their projections. The upper percentile projections give them some respectability, but the red-50% line and other percentiles resign them to medicority or worse by age 27. Mike Stanton, another 20 year old, is seeing a similar profile. His upper two projections show .290 TAvs well through his age 27 peak, but his 50%TAv stays under .250 for from age 25 onward. I suppose it's possible that these three players all share the characteristics of an early peak, with only a slim chance at being star players from age 24 on. But it just seems odd to me that each player has such a decline in their 50% projection from Age 23 to Age 27 seasons: Heyward: .289 TAv to .265 Montero: .283 TAv to .265 Stanton: .252 TAv to .229 These seem to be 'better' projections than the wide spread projections we had seen in the first round of beta cards. But something still strikes me as being odd, that these guys are all going to peak at age 23 across all their projections, and have their 50th percentile trend down afterwards. Is this happening because of the "out of baseball" projections holding down his 50% projection? Just doesn't seem right to me. What happens if you toss those out of baseball percentages out? Does the 50% line more closely resemble a career path with peaks in the age 26 through 29 seasons? Again, thanks for including me as a beta-tester, and I'm sure I am going to learn more from this than you will. But hopefully you get something useful from my inclusion.
jivas21
3/18
Based upon a cursory glance, it seems to me as if the difference between the median and weighted mean forecasts are *much* larger than they've been in prior years. (Of course, if this is a fundamental change, it's not necessarily "wrong" simply because it's different). I looked at a few players (Carlos Gonzalez, Dexter Fowler, Cameron Maybin) and was extremely surprised at the differences, which I don't recall typically having been so large in past years. I then looked up a few players from last year - 2009 cards are still available if you search for a player and choose to see their PECOTA card - and sure enough, the differences just don't *seem* nearly as large in the past (I checked Ian Stewart and Matt Kemp for 2009). Obviously not a ton of empirical evidence there, just an observation. As I have time, I'll keep trolling to see if anything else stands out.
jivas21
3/18
Some examples of players and the difference between their median and weighted mean slugging percentages - I've picked young power hitters for whom you'd expect large differences between median and weighted mean outcomes: Player ('09 diff, '10 diff) Jay Bruce (.026, .007) Carlos Gonzalez (.031, .009) Matt Kemp (.018, .004) Just a few examples.
jivas21
3/18
If you guys do look into this, I'd be fascinated to know what you find out. The range of forecasts appears to be quite disparate this year compared to prior years, and intuitively it seems more of a true representation of player performance. However, I do recall Nate mentioning that he had tested his percentile ranges against actual outcomes and determining that they were, in fact, representative of reality.
Junts1
3/18
I don't understand your posts, Jivas; your initial post claims that '10 has wider variances, but your numbers suggest it has, in fact, much narrower variances, and your concern makes more sense in that context (there should be more variance between 50th and weighted outcomes for players with particularly high breakout or collapse rates).
jivas21
3/19
Mike: you are correct that my posts contained an error in thought - this is the cost of trying to observe, conclude, and write in a few minutes during lunch. Generally, the median and weighted mean should not vary based on the variances at all, *assuming* that the distributions are identically distributed. However, in this case, even if the rate statistic distributions were identically distributed, the playing time distributions are NOT - higher percentile projections are accompanied with more playing time, so therefore a projection with greater variance should (in my opinion, based on the above) lead to a greater difference between median and weighted mean. (So I think we disagree on this point.) More interestingly: are this year's projections more *skewed* than prior year projections? Because - aside from the difference being caused by the playing time as noted above - a difference in skewness would be a far more interesting explanation....
Junts1
3/18
So I've been looking again at the new formatting and I had an idea, though it leans more to an added feature than a problem: The EqA displays for comparables are helpful but of limited use: since the up/down is based on whether that player was over or under what pecota wouldve predicted for him, the EqA stat without context doesn't tell us if that red arrow means someone had a .277 eqa vs a predicted .280, or a predicted .308. It would be useful to do one of the following: 1: list the predicted EqA and let us click the name to see the actual result. Or 2: list both predicted and actual EqA, with one color-coded so we can easily identify it. This would let us gauge the magnitude of a comparable's impact on the prediction. I think this would improve our ability to understand comparable impacts.
rawagman
3/18
I think it would be great to know if the comps played in the majors that season - especially for prospects.
clayd
3/19
One thing I tried earlier was, in place of the up or down arrows, to list the numeric change in EqR between the comparables baseline and actual season. So Kemp's list would read Roberto Kelly +11, Ellis Burks +4, Snider +2, Davis -2, Dawson +17, Mondesi +12, Durham -8, Gutierrez -8, Smith -1, Crisp +15, rather than U---UUDD-U (Up, Down, - neutral). Would that help?
jrmayne
3/19
I would love that. --JRM
rbross
3/19
the upside numbers still seem high on the whole. With the old system, if you had anything above 100, you were considered a noteworthy prospect. Now it seems like nearly every prospect has not just 100, but 200 or more. If you're using a new system, that's fine. But I'd like to know how it translates. Also, to reiterate what some others say, I love the new format. It's a great idea to link to every mention of the player's name and the graphics look good. One very minor point that only very slightly bugs me: why not have a link on each PECOTA card to the BP home page? As it is now (and always has been), you have to go back to the PECOTA home page before you go back to the BP home page.
BCulhane
3/19
I agree with Bob's comment on linking PECOTA cards to the BP home page. It is a minor annoyance, but one I've noticed at least a dozen times.
clayd
3/19
I agree as well.
dpease
3/19
You can always click the BP logo in the upper left to get back to the front page in the new layout. We've got a breadcrumb beneath the button bar in the header, and there's a smaller logo link in the footer. Will those fill your needs here?
Junts1
3/19
I do think that would be an improvement. If you retain the color coding on the numbers the functionality of glancing at the list to see if the overall trends are up or down as well. As far as I can remember, the comparables adjust the projections last (yes?), which would mean that there aren't really comparable effects on breakput and collapse rates. Some player types, especially younger, are probably predisposed to exceeding their projections gigantically if they meet x criteria (one imagines those toolsy guys who finally get it with plate discipline) so I could see being able to note that comparables were only beating the projection on breakout leaps as valuable added information, for instance. Typing this on a blackberry makes for some runon sentences, apologies. Its possible if comparables are adjusting breakout rates etc that this is extraneous data but from my understanding of pecota, the comparable effects are not that advanced. In that case, I would shoot for aesthetic effect. However, just the +/- info would satisfy a lot of my curiousity and I am not even a fantasy player. I just like trying to get better perspective on the development and evolution of my team and their competition.
Junts1
3/19
Whoops, blackberry failed to make this a reply to Clay.
jivas21
3/19
Chase Utley's weighted mean OBP is approximately equal to his 10th percentile forecast. This must be an error; there is no reasonable degree of skewness that would cause this to result.
jivas21
3/19
Josh Willingham's weighted mean OBP is between his 20th and 30th percentile forecasts, even though his projection appears to be skewed heavily rightward on OBP. So there's an error here for some players.
ccmonter
3/19
Utley's weighted means look ok now. Was anything done to fix that, or was this a 'wrong' report?
clayd
3/19
There was an OBP error in an earlier batch run, and I think Jivas was inadvertently reading from those older files - I think the search function was still directing you to those old ones. That error is fixed.
jivas21
3/20
Hmm...I still see the same thing. I see a median OBP of .395 for Utley, with a weighted mean of .378, which is near his 10th percentile projection of .377. I do see the note about changes being made tonight, so this might be another fluke.
jrmayne
3/20
I also see the same thing; Utley's weighted mean isn't right. I see there are some Beta changes coming tonight; that's good. On an aside, can I suggest that things like prior OBP problems and weighted mean problems be acknowledged in a public post at some point? Some note that there were systemic problems with the 2010 PECOTA's seems like it would help regain trust, not just from the pitchfork-wielding mob, but also from those who saw what was illuminated by the torches. The step forward in opening the drawbridge to the mob in this exercise is a positive. Good luck on tonight's update. --JRM
jivas21
3/20
The 10-year forecast says that it is park-neutral, but the 2010 line appears to be taken directly from the players' 2010 actual weighted mean line (that is, inclusive of ballpark effects). I've reviewed this for: Rafael Furcal, Chin-Lung Hu, Starlin Castro, and Jeff Baker.
dpease
3/20
This is correct--that's an artifact, we changed it a while back, and we'll change the note to match. Sorry about that.
jivas21
3/20
Let me join the chorus - too late, but still - of those who think the changes that were made to the pages are outstanding. When the pages are fully functional, they'll be phenomenal. One minor quibble - and this may just be a function of the beta pages being temporarily housed at the moment - is that on last year's pages, the header of the internet window was the player's name, whereas on the beta cards it's "Baseball Prospectus - Your Source For All Things Baseball". It's much more convenient to have the player name in the header, where it's visible both on the top of each internet window, as well as on the taskbar. I've found myself on numerous occasions while perusing the current beta cards asking myself..."whose card am I looking at again?" And answering that question involves scrolling all the way to the top of the page, or inferring from the player comments at the bottom of the page. This issue doesn't take place with the 2009 cards. Again, this may be a temporary issue, but I wanted to point it out.
dpease
3/20
Name will definitely be in the final card page title.
jivas21
3/20
Pardon the numerous posts - I don't mean to monopolize this forum - but from the time the very first PECOTA spreadsheet was released this year I wondered: why no MLVr on the spreadsheets this year, or on the cards for that matter? I used to like using MLVr to sort position players by offensive value for Scoresheet purposes.
Junts1
3/20
I want to second Jivas here; I have always found mLVr to be a useful statistic and one of the ones my eyes are always drawn to in reading a PECOTA projection, since it's a single-number way to evaluate rate value. I would be interested in knowing why the stat has fallen to disuse, and if there will ever be an attempt to make it more relevant if its felt to be inadequate or uninformative.
jivas21
3/20
The projections appear to be *far* more skewed upwards than last year. Some examples - these are the differences in SLG between a player's 90th percentile forecast and median forecast, and then the difference between the median forecast and 10th percentile forecast, for each year (all math done quickly and in my head): Player (2009) (2010) Marlon Byrd (.088, .088) (.072, ,017) Aramis Ramirez (.074, .070) (.073, .052) Adam LaRoche (.077, .085) (.088, 056) Kelly Johnson (.075, .064) (.095, .035) So, I think this is why there is a larger difference between median and weighted mean forecasts in 2010 - the distributions are skewed upward in 2010 versus 2009. I'm heading out for the evening. I apologize for the volume of posts!
jrmayne
3/20
Jivas and all: I just looked at 15 2009 cards and 15 2010 cards; I think the effect you cite is real, and substantial. I think there's something wrong there at the low-end percentiles. Great catch, worthwhile post, and I'd love to know the reasons this is happening. A strong piece of evidence for this Beta Blog and wisdom of at least part of a crowd (or mob, as the case may be.) I might be wrong; this might not be as pervasive as it looks like. But it looks pretty damn pervasive; I found a handful of players with similar differences this year, no one with a long 10% tail, and lots with big 90% performances. I'm guessing this stems from the same issues as the 10-year projections - out of baseball isn't considered a performance decline, and that's one of the causes of being out of baseball. Are we handing out Great Catch Awards? One to Jivas, assuming he's right, seems in order. --JRM
Junts1
3/20
This is indeed a really nice catch, and it doesn't seem to only afflict players who don't seem to have enough potential collapse rate. However, the differences are larger for players who should have significant collapse rates (like Marlon Byrd). Is it possible that collapse rate is not properly being factored into hitter percentiles, but breakout is (or breakout is being over-emphasized and collapse is functioning properly), creating an upwards bias? Just from browsing some cards myself, it does look to me like these gaps are larger with players who have disproportionate collapse to breakout rates, for example: Casey Blake (breakout 5%, 50th to 90th vector .099 slg, collapse 27%, 50th to 10th vector only .048 slg). James Loney (breakout 18%, 50th to 90th vector .075 slug, collapse 10%, 50th to 10th vector only .034 slg). I picked those two players because they have identical median slugging projections (.424/.425) and identical park effects (one could surmise that parks like GABP would jack breakout seasons up even further than they would help mitigate collapse seasons, so I didn't want to pick players with different park effects). I wanted to quote the BETA for both players but that statistic does not seem present on the cards anymore; I really did miss this measure of prediction reliability and would like to see it re-implemented in some fashion.
jivas21
3/20
I'd like to second the re-introduction of the Beta onto the cards (or at least the spreadsheets). It represented useful information. Also, prior year cards had a Similarity Index figure to help identify the extent to which the comparable players matched with the target player (e.g. Ichiro's 17 indicating that his comps were less meaningful than average). Is there a reason why this has been omitted this year? Thanks....
Junts1
3/20
I agree, those indices were extremely useful in understanding how PECOTA worked, and if not a huge help to using it for any constructive purpose, were extremely useful for helping troubleshoot it.
BCulhane
3/21
I also miss the similarity index. It is nice to know if the comps are really close, or just the closest years in the system. My limited understanding of PECOTA is that the system makes predictions based on performance of previous players. It follows that knowing how good a fit a current player has to previous data is more valuable than knowing who to whom the player is being compared.
Junts1
3/21
PECOTA makes its projections based on the player's statistics and profile, which is based on position, statistical history, body type and a lot of other factors in a general way. It then determines the most similar players at position and age and, independently, generates a projection for them based on that data. It then compares the projection to what that player -did- do in the next season. If they under-performed, PECOTA slightly reduces its projection. if they over-performed, it benefits the projection across the board. These effects are slight and their weighting is dependent on the strength of the similarity score, so guys like Ichiro who always had very low similarity scores got less from their comparables than did players with more traditional profiles. PECOTA is not solely based on specific comparables and most of its body-type and statistical analysis that establishes the bulk of the projection are based on much more meaty statistical analysis of what things are significant factors in player performance. The comparables help tune PECOTA's accuracy, but they are a fine-tuning mechanism, not the bulk of the projection system. Being able to see the similarity scores is helpful because for players like Ichiro you are basically informed that the comparables are specious at best (Ichiro's best comps have traditionally had similarity scores that other players might see with their 150-200th best comps). The impact of those comparables on Ichiro's projections is very small, and is part of why PECOTA has a harder time with players like him, Randy Johnson, or Barry Bonds in the past. In those situations, it's forced to go with only its baseline weighting of factors, for players who have always succeeded in the face of overwhelming trends against them. The comparable system helps control that within the normal bounds where there are comparables. For the really extraordinary outliers, PECOTA struggles a little more because it has so much less context to work with.
dianagramr
3/20
Whoa .... I just update my cheat sheet based on the 3/19 PFM. Some of the 5x5 categories have changed a bit ... As an example: Mariano Rivera (old,new) 5/40/58/3.19/1.14,5/40/64/2.56/1.03 I don't have a PECOTA card to reference for him, but SOMETHING is different.
BurrRutledge
3/20
Yep, definitely some changes to the categories and rankings. Noticing a lot of players moving a few notches up or down. Haven't seen any massive changes yet, but looking. With the settings of my league entered, McLouth still rates just above Kemp and Werth in CF, but behind Sizemore. Mariano jumped ahead of Papelbon in the rankings of the relief pitchers, but Papelbon is still #2 (not that I draft top closers, but I try to keep an eye on what the competition might be looking for).
brokeslowly
3/21
A few things I noticed - It would be nice to have historical VORP data available as it was in previous PECOTA cards. In sampling a number of the hitters, the weighted mean numbers seem to be consistently higher than the 50th percentile data. I can understand this if a player is expected to be a starter, but this shouldn't be the case in a reserve player, unless I'm not understanding the weighted mean concept correctly. For example, Alberto Callaspo's 50th percentile VORP is 14.8, his playing time adjusted VORP under the bio data is 10.3 and his weighted mean VORP is 18.5. If the 50th percentile data is more accurate for hitters than the weighted means, why are the ten year forecasts based on the weighted means?