CSS Button No Image Css3Menu.com

Baseball Prospectus home
  
  


rssOur Latest Blog Entries
03-03Yankees-Pirates, Phils-FSU, 3/3 by Joh...
03-02Braves-Mets, 3/2 by John Perrotto
03-01Clay Davenport Now at BP Full-Time by ...

February 13, 2010, 12:44 PM ET
PFM/Depth Chart Update

by Clay Davenport

Sorry it took so long, but this turned out to be a major update in a lot of ways - particularly on the pitchers.

If there was one type of comment that repeated frequently in the threads, it was that the pitchers looked way too pessimistic in their projections - far more so than the hitters. My first instinct was to simply accept it as PECOTA being PECOTA - the normal tendency of the group is down. There are a lot more different ways, and far less effort involved, in making a major league player worse than there are to make him better.

On further study, it may still be that, and it is possible that I’m chasing an idea the wrong way, but it is the best I’ve come up with for now. There is really only one significant change that went into this projection set from the last one - the pitchers are shown based on their 75% score, not their 50% score. I’m still trying to figure out the exact whys, but when I reran the 2008 and 2009 projections  using the new PECOTA program, the best results for major league pitchers came from using a projection around the 75th percentile; using the 50th percentile resulted in large biases towards worse performance, exactly as noted in the player comments for the previous run. This effect was not found in the hitters; the best results for them did come from using the 50th percentile.

I believe that this is the result of a selection effect - that pitchers who do better than their true expectation are the ones who actually pitch in the majors; any slippage quickly results in reduced playing time or demotion to the minors (except for teams with no reserves, like last year’s Brewer rotation). While I would expect that to be true for hitters as well, I think the combination of lower margins (pitchers don’t separate from each other as widely as hitters do) and higher injury rates makes them more susceptible to the bias, and that’s what PECOTA is picking up. My worry is that while running the program in this configuration will make for better results in the major league forecasts, it will create undue optimism for a lot of borderline and minor league players - that by focusing on accuracy for a select subset (major league pitchers), I’ll degrade the accuracy for the group as a whole (all Organized Baseball pitchers). The problem does seem to have appeared with the 2009 PECOTAs; the main change between 2008 and 2009 was the inclusion of a lot more minor league players in the database, about 15 years worth.
But it does make a big difference in the performance of the model for major leaguers. Here are some results from my error testing. In each case, I have pro-rated the PECOTA projection to the pitcher’s actual innings pitched, and showing the root-mean-square error of the given component. So if I projected a 4.50 ERA, and the pitcher really had a 4.00 ERA in 180 innings, my error is 10 runs. The selection group was about 300 pitchers, chosen because they had forecasts from PECOTA and a number of other projection systems (like ZIPS and CHONE). The “new pecota” was run with all 2009 data removed from the system - it used no information that wasn’t available in February 2009.

  Hits ER SO BB HR
2009 Pecota 17.45 14.95 16.30 10.37 4.66 (from 2/14/09 weighted means sheet)
2009 CHONE 15.61 13.19 16.69 10.55 4.59
2009 ZiPS 16.54 14.26 17.61 10.79 5.20
2009 Marcel 16.01 13.53 17.07 10.32 4.75
2009 new PECOTA 14.51 12.83 16.35 9.92 4.61

Running at 75% produces a dramatic improvement  in hits and earned runs, a moderate improvement in walks, negligible improvement in HR, and a negligible worsening in strikeouts - and very narrowly missing a clean sweep of all categories.
Running just against the February 2008 version of the PECOTAs for the 2008 season, we get

Hits ER SO BB HR
Old PECOTA 12.51 11.35 11.26 8.86 3.52
New PECOTA 11.41 10.47 11.18 8.29 3.56

This was based on 600 pitchers, who averaged a lower inning total, which is why the overall scale of the numbers comes down, but we can see that the “new PECOTA”, running at 75%, follows the same pattern as in 2009 - big improvements in H and ER, moderate improvement in BB, negligible differences in SO and HR.

It is natural to ask, then, how the hitters compare to the old PECOTA. For 2009, about 400 players, and scaling to plate appearances, the errors by each category are

H DB TP HR BB SO SB CS R RBI
Old 10.91 5.26 1.92 5.20 9.59 12.08 4.71 1.86 9.55 10.99
New 9.99 5.21 1.86 5.05 10.05 12.57 4.63 1.94 8.75 10.80

With the hitters, we have an unfortunate step back in BB and SO, but improvements ranging from small to large in every other category.  I’m still looking to see if I can improve those”non-event” scores, but I’ll happily take the gains.
Now, as to the depth charts themselves, I think I’ve gotten all of the free agents purged from them. A lot of the odd minor leaguers showing up on the depth charts are also gone - most of those were listed because they were the next best option the teams appeared to have in their own system, at the time I did the depth chart; as teams have signed more players, the need to reach so far down in the minors has diminished, but I’m still concerned about a team like the Twins, with no one behind Span that I’d trust in CF for more than a couple of games. Please do keep in mind that these charts are trying to look at the entire season, not just the Opening Day roster, and they are working on the assumption that injuries will happen. When they do, someone will get called up - someone who might have no shot at the major leagues when everyone is healthy. Some of the playing time projections that seem low are because PECOTA expects a player to suck, and I have a hard time believing a team will stick with him all season, regardless of how they declared him a starter early on. That will become even more true as spring training gets underway, and someone “wins” a job on the basis of a .387 spring over someone with a longer, better history of performance.
Lineup order is generally not that big a deal for me - they are notoriously changeable, and it is primarily to  enhance or decrease the PA of players expected to be largely high or low in the order. When I do see a manager make specific statements about how he wants, I will work them in.

Unfortunately, the changes that came in to PECOTA here has set back the production of the individual player cards. I do think the hitter card is down to one bug that I can fix today, and that the pitcher card format issues can be done by Monday if I don’t have to shovel any more snow (I live just south of the snowiest major city in the USso far this year. Those are entirely on me; I keep changing things to try and get them right.

61 comments have been left for this post.

BP Comment Quick Links

gluckschmerz

Thank you. Thank you. Thank you. Your honesty and hard work are appreciated.

Good luck with the snow removal!

I eagerly await those player cards!

Feb 13, 2010 09:50 AM
rating: 3
 
rjwhite01
(994)

Thanks, Clay.

By the way, love that the depth charts now show a three-way tie at 80-82 atop the AL Central. A sub-.500 team is bound to make the playoffs one of these years, under the present format.

Feb 13, 2010 10:35 AM
rating: -1
 
sfastatsprof

Wow. Let me see if I can put into words the duality that I sense here. First, to accent the postive, I think the incredible attempt at making PECOTA better should be applauded . Clearly, Clay has been pulling some long hours to address many of the readers concerns.

One one side of the coin, I see this note as encouraging. BP indentifies a bias, looks into what would have made '08 and '09 better... Corrects it by adjusting pitchers to 75th percentile and voila...a hopeful set of 2010 pitching projections.

On the flip side, I think this note solidifies the opinion that BP's methodology is on somewhat thin ice at present. The key phrase I take away from Clay's note is "I’m still trying to figure out the exact whys...".

These changes seem to be the result of hueristics, rather than a solid statistical theory at this point. The problem with hueristic results that appear to create short term fixes is exactly that: the fixes may be short term. Unless the "whys" are discovered and explained during 2010, chances are there will be more bumps in the road ahead for PECOTA.

To me, while I very much appreciate the hard work and attemps at correcting things, I believe this note indicates something - with painful honesty - that many have come to realize during 2009: PECOTA is coming back to the pack in terms of projection systems. Sure, hindsight RMSE calculations indicate what we "should have done", but the future is what is most important.

In the past, I have faithfully defended PECOTA as the "main" component to my pre-season fantasy preparations and season preparations. In short order, that may again be true. For now, I will take a wait and see approach. Looking to see if both results and methodology become a little more concrete in 2010. I will continue to be a dedicated BP'er...upping my subsciption without a blink-of-the-eye. But in terms of projections for 2010.. I believe I will increase the percentage weight I give to ZIPS, CHONE and CAIRO and cautiously back down from a PECOTA overweight. I hope in the future, my doubts are subsided.

In conclusion, despite the advances of THT, BTF and FanGraphs, I personally , still believe the best overall place to be for coverage in this vein is right here.

Feb 13, 2010 10:59 AM
rating: 35
 
alskor

So... looks like someone's favorite team took a hit in the latest PECOTA standings.

Feb 13, 2010 11:35 AM
rating: -2
 
sfastatsprof

alskor...actually, I am a die-hard Padres fan so my club has been at the bottom of the NL West projected standings all along...by PECOTA and by most other systems! That gave me a chuckle, though.

Feb 13, 2010 13:15 PM
rating: 0
 
alskor

I was kidding...

I do agree *in part* with some of your criticisms, but Im torn. I like getting my PECOTA as early as possible, so I dont mind the beta - but I agree the wild swings we've seen so far don't exactly inspire me with confidence in the system/process.

Still, I think we would find every projection system except perhaps the simplistic MARCEL has these same sort of bumps in the process. The difference is we're just privy to them. People here seem turned off by seeing so much of the guts of the process - it is sort of like viewing an operation. For Clay and company, I would imagine the individual results that many BP readers are clinging to (like the processed standings) are far less interesting than playing with the process.

I dont mind seeing the guts, personally... I think its more the first release of PECOTA this year was just extraordinarily raw. Im not sure this really any different (systematically) than it has been in previous years. The projections constantly get tweaked throughout February-April. Not sure why so much attention is being paid and with so much dissatisfaction this year.

Feb 14, 2010 16:38 PM
rating: 8
 
sfastatsprof

You make some outstanding points. In fact, I really wish I had included exactly what you've written in my original post, but it was getting long. However, I think Clay essentially admitted the process is partially broken and hence back-engineered to create a fix for now. Until things get back on more solid ground, I'll temporarily view PECOTA as one of the crowd, rather than the leader. Personally, the 2009 downer weights a little in my mind. It is a little disconcerning that we are left to infer that the pitching projections in the book are now considered suboptimal from BP's own point of view. Your points about us viewing the bumps out in the open is entirely correct and certainly bears keeping in mind...good points.

Feb 14, 2010 17:44 PM
rating: 3
 
Michael
(736)

Maybe I'm recalling it wrong, but I don't believe the first PECOTA release was labeled "beta." If that's right, then the criticisms that there were bugs was fair.

Furthermore, the criticism that there still seem to be bugs is fair too.

Feb 14, 2010 17:48 PM
rating: 2
 
sfastatsprof

Michael, I think clearly something is mathematically wrong. Maybe I am looking at this from too much of my typical academic standpoint, but the PECOTA "model" is mathematically off at this point in my mind.

To adjust projections to the 75th percentile is a clear admission that the process is mathematically producing biased pitching estimates. You know, my overall guess is that BP "figures it out" and learns a tremendous amount from this years early struggles. When they do, I fully expect their projections to rise back to the cream of the crop and play a bigger role in my season preparations. But, as I originally said, I think I am in "observer" rather than "user" mode for 2010.

I should be completely fair and say I will use the batting projections as I normally do...Clay is clear that the issue is with pitching.

Feb 14, 2010 17:56 PM
rating: 6
 
alskor

I think its more like they changed some of the underlying principles - some of the basic inputs, and theyre finding out step by step how those interact with each advanced step of the process. It certainly hasnt been smooth so far.

Feb 14, 2010 18:01 PM
rating: 1
 
newsense

The problem with using the 75th percentile is that it's arbitrary. There's a sound theoretical reason for using the 50th pwercentile or weighted mean. If you show that the 75th percentile performs better than either of them, then you know there is a bias in the system. But you have no reason to believe that the 75th percentile is a better correction for the bias than, say, the 65th percventile or the 80th percentile unless you've tested all those possibilities

Feb 15, 2010 07:15 AM
rating: 11
 
colintj

there isn't much if anything you can't get Sean Smith to tell you about CHONE. it's free and among the most popular projection sets, it's the only one doing better than Marcel lately. ZiPS is somewhat more proprietary, but there's nothing to suggest that it isn't in the same neighborhood as PECOTA.

given how unwieldy PECOTA seems to be next to CHONE or ZiPS, which come out rapidly and more or less complete, it might be time to do some serious house cleaning. in the very least, BP needs it's own Marcel-like benchmark to show its readers.

i don't doubt they can work it out with Clay (and the new hires hopefully), but i think the transition (and, according to Nate, his bad code) has been a little less than smooth.

Feb 17, 2010 01:49 AM
rating: 1
 
gluckschmerz

A little playful sarcasm and this comment gets a -12. Sheesh

Feb 13, 2010 19:12 PM
rating: -1
 
IvanGrushenko

I appreciate the transparency in the projections over the past month.

Feb 13, 2010 12:26 PM
rating: 4
 
jerenkrantz

Thanks for the updates and insight into the improvements!

For the Rangers depth chart, according to T.R. Sullivan of MLB.com, Ron Washington has made a few statements about the batting order which differ greatly from the current depth-chart projections:

---
Washington already has the top part of his lineup made out. Guerrero will bat cleanup, behind leadoff hitter Julio Borbon, third baseman Michael Young and Hamilton. Ian Kinsler will likely hit fifth.
---

http://texas.rangers.mlb.com/news/article.jsp?ymd=20100111&content_id=7898670&vkey=news_tex&fext=.jsp&c_id=tex

(I think most Rangers fans are expecting to see it be Davis, Cruz, Salty, Andrus at the bottom of the order; but I don't recall any specific statements from Wash or JD on that.)

Feb 13, 2010 12:36 PM
rating: 1
 
jtrichey

Does this mean that the pitcher projections in the book are overly pessimistic or does this not affect the book? If it doesn't affect the book, then why is there a difference?

Feb 13, 2010 13:03 PM
rating: 9
 
dianagram

Thanks for the update Clay.

Once again, could I ask (please please please) to include some form on unique ID (be it BPID or Howe or Scoresheet) on ALL fantasy/projection player data. It would make it much easier to merge the various data components.

I mean, they appear in the weighted means spreadsheet, so how hard could it be to include it in the "biographical data" of the PFM?

Thanks!

Feb 13, 2010 13:09 PM
rating: 6
 
Rob Miller
(162)

What she said!

Please include SOME sort of ID field in all of the downloadable data.

Feb 14, 2010 00:49 AM
rating: 0
 
Marc Normandin

This is something we're looking at introducing if possible. I've alerted the proper folks about this issue, because I know you've been mentioning it!

Feb 16, 2010 04:21 AM
rating: 0
 
dianagram

Thanks!

Feb 16, 2010 07:54 AM
rating: 0
 
Hokieball

It would be extra awesome if this could show up in some of the real statistical reports as well. It's a PITA to compare a players projection with his actual performance when he's named, for example, "Manual Corpas" in one data set but "Manny Corpas" in another and there's no unifying ID link.

Feb 16, 2010 10:47 AM
rating: 4
 
bisanders
(329)

Thanks, Clay. Why the radical change in how the Stuff scores are calculated?

Feb 13, 2010 13:53 PM
rating: -1
 
jrmayne

It looks to me like its a miscalc, using a raw stat instead of a rate stat.

I wonder if the stuff scores are in the book, and were available to the authors? This seems like something someone would catch virtually instantly.

On the main subject, the conversion of PECOTA off a spreadsheet appears to have resulted in some errors, and the loss of Nate appears to have made it hard to fix those errors.

It's unclear if there are more invisible errors generated. Like sfastatsprof, I'm going to give far more credence to CHONE and Zips relative to PECOTA than in the past.

--JRM

Feb 14, 2010 09:04 AM
rating: 1
 
Jim Humdingding

If the projected ERA on the playing time adjusted tab of the new Weighted Means spreadsheets is giving the 75% projection now rather than the 50% projection, and that was "the one significant change" from the 2/1 spreadsheet, why did some pitchers see such an increase in projected ERA from the mj_pitchers tab on the 2/1 spreadsheet? Examples:

Pitcher.....2/1 MJ tab....2/13 ml tab

A.J. Burnett 4.19 to 4.33

Matt Cain 3.67 to 3.78

Doug Davis 4.42 to 4.62

Jon Garland 4.50 to 4.62

Harang 3.87 to 4.41

Kazmir 4.16 to 4.66

Maine 4.21 to 4.56

Meche 4.45 to 4.63

Nolasco 3.82 to 3.83

How do these guys get worse ERA projections when you give their 75% projection rather than their 50%, and that was the major change? Comparing spreadsheets with a buddy over the phone, it seemed like elite pitchers almost uniformly had their projected ERA drop, but lower quality pitchers often rec'd increased ERA projections.

Feb 13, 2010 14:14 PM
rating: 13
 
davidtoby

Add to that list:

Felix Hernandez
Previous forecast: 3.25 ERA, 1.17 WHIP, 15 wins
Updated forecast: 3.29 ERA, 1.20 WHIP, 14 wins

James Shields
Previous forecast: 3.40 ERA, 1.12 WHIP, 15 Wins
Updated forecast: 3.89 ERA, 1.19 WHIP, 13 Wins

Both had their K's boosted (Felix from 175 to 188, Shields from 152 to 163)

Feb 15, 2010 16:49 PM
rating: 4
 
R.A.Wagman

Still no projection for Kyle Gibson. Oversight, or does BP no longer run projections for all signed 1st rounders?

Feb 13, 2010 16:17 PM
rating: -1
 
redspid

Pecota is way, way, way overrated if you're playing fantasy. It always seems to love Chris Young(AZ), Corey Hart, Soriano, any A's, Dbacks or Indians. Just my experience from the past 5 years or so.

Feb 13, 2010 21:28 PM
rating: 0
 
brokeslowly

75% projection now = 50%, somewhat empirically. This shakes my confidence in this system to the core. I also think that this makes the PECOTA projections in the book on the shelves quite questionable. I think that there is too much procrastination at BP with the working out of these projections - why is this crisis being discovered in mid-February rather than well before?

I don't mean to make this sound like a slap in the face to Clay or other current BP staff, but the thought immediately jumps to my mind - has Nate been consulted about this problem?

Feb 14, 2010 07:58 AM
rating: 19
 
newsense

There also needs to be some acknowledgement about how poorly PECOTA performed last year. Was Nate too distracted by politics?

Feb 14, 2010 09:03 AM
rating: 3
 
Juris

Could be. But as I comment below, some of PECOTA's greatest misses in the past seem to have come from mistakes in the translation of minor league or international records, and in at least one year from the park factor that was used for a particular team (Montreal or Toronto -- can't recall). PECOTA's accuracy depends not just on the formula used (which hasn't changed a whole lot since 2003) but also on the adjustments to the input data for current and historical players (the latter being critical because PECOTA relies on choosing "comparables" in order to make projections).

Feb 14, 2010 11:49 AM
rating: 2
 
alskor

"Was Nate too distracted by politics?"

Hope you were joking... can't tell.

PECOTA is the most complex system out there. Its entirely natural and appropriate that it takes a long time to work out the kinks.

Feb 14, 2010 16:44 PM
rating: -2
 
colintj

if there's no value added, and recent comparisons suggest there isn't, then the complexity is just getting in the way. CAIRO, Marcel, CHONE and ZiPS were out well in advance of PECOTA. there's no reason they can't have a stripped down version up and running just as rapidly.

Feb 17, 2010 20:07 PM
rating: 1
 
sfastatsprof

You make a valid point, but to be technically correct, ZIPS really isn't completely out yet. Dan will post the final team (Angels) in the next 24 hours, but then the entire "Builds" as he calls them will follow. ZIPS tends to lag behind in terms of actually getting ALL MLB projections released. That annoys me a bit about ZIPS to be honest.

The "value added" point is particularly well taken, though. The simple fact that all these other systems are generally more transparent and free is a thorn in PECOTA's side. This may be a "bump in the road", but if PECOTA can't beat Marcel the Monkey over time, then I'd have to ask: "What's the raison d'etre"?

Feb 17, 2010 20:52 PM
rating: 1
 
ferret

Unless Pecota has been dramatically reengineered for this season, these issues could have been determined by a test drive in December. Appropriate reviews should have been performed before release. The original roll out last month was a disappointment and had too many obvious inconsistencies that were later "corrected". My faith has not been restored.

Feb 17, 2010 11:52 AM
rating: 3
 
Juris

Thanks, Clay. IIRC, some years ago when Nate compared PECOTA projections with those of other systems, he found that it was especially on the pitcher projections that PECOTA outperformed the other systems. For this "historical" reason, I am skeptical about the idea that PECOTA may be systematically underestimating pither performance -- at least among the relatively more active pitchers (in terms of IP) that all the major projection systems tracked and thus served as the set of comparison cases.

But your detective work, and the possibility of two types of selection bias: (a) unsuccessful pithers get benched or demoted more quickly; and (b) inclusion of more neophytes makes their projectsions a larger share of the overall PECOTA pitchers, but at the same time also makes PECOTA as a whole more sensitive to the DT's and MLE's.

It's my impression that over the years PECOTA's largest "misses" at individual and team level has come not from the internal mechanism of the PECOTA system but from two external soureces: the MLE's sometimes being too optimistic or pessimistic (perhaps for player records from some minor leagues or international play); and park factors being wierd for some reason.

Feb 14, 2010 10:45 AM
rating: 1
 
Matthew Avery

There's a difference between fantasy-relevant pitchers and pitchers in general. Making an adjustment to make PFM more useful to fantasy owners is NOT the same as saying that PECOTA as a tool for projecting performance amongst ALL players (or pitchers) has been incorrect to some degree.

Noticing PECOTA's shortcomings for a subset of users (IE, fantasy managers) and adjusting the product designed for them accordingly was definitely the right move to make, and I appreciate the long update explaining what the changes are and why they were made.

Feb 14, 2010 11:34 AM
rating: 8
 
Luke in MN

I appreciate you're trying to fix things and keeping us up to date. As a long-time BP fan, my confidence in the system as is is pretty shaky right now, but keep me updated on advances and you can win me back.

As a Twins fan I was quick to note that they are predicted to have an 802 team OPS but score only 778 runs--way too few for that high an OPS (off by about 80 runs). I noticed that other teams seem to be off too, so maybe it's a system-wide thing. (E.g., The Yankees scoring only 821 runs with an 825 OPS would be shocking.)

Feb 14, 2010 12:10 PM
rating: 5
 
RedsManRick

I'm seeing the same thing, Luke. The runs scored totals seem way too low. It definitely looks like the change to the pitcher projections did something funny system wide to the relationship between offensive performance and the run projections.

Feb 14, 2010 15:51 PM
rating: 1
 
Michael
(736)

This explanation -- that we should now use the 75th percentile projections for pitchers -- doesn't make sense to me. Let us know we all the bugs are worked out of the PFM so we can start treating the data as reliable. This does NOT seem like a permanent solution yet.

Feb 14, 2010 16:38 PM
rating: 6
 
Nathan J. Miller

Interesting. So from this unflitered post, it sounds like the issue may stem from PECOTA now having a larger pool of minor league comps -- a greater number of "similar" players who didn't make it? That seems like it could plausibly result in the more pessimism beginning in 2009... suggesting also a missing variable around selecting the comps -- the system is forming the wrong "actuarial group" for consistent MLB pitchers, perhaps. I'll give the 75% forecast a try for fantasy purposes, but agree with all the others that this is definitely an issue that merits continued investigation.

Feb 15, 2010 08:14 AM
rating: 3
 
Juris

Add to what you say the possibility that the translation of minor league stats into major league equivalents (MLE's) is an important step in the process. If you decide to downweight those MLE's for some reason, you drag down the expected performance of young ML players in particular -- since the projected 2010 (and beyond) performance on those on the ML roster is based on how their "comps" performed (or are estimated to have performed in the ML if they had been in the ML).

Feb 15, 2010 08:35 AM
rating: 1
 
Juris

To elaborate just a bit. PECOTA was ridiculed last year for its wildly positive projection for Matt Wieters. That projection, however, relied completely on the translation of minor league performance stats -- which includes assessing the difficulty of competition in the minor league in question, among other things (not just how the player's stats looked by themseves.) Maybe Clay decided to apply a more conservative adjustment for minor league performance this year, so that the MLE that gets fed into PECOTA would come up as less promising than it would have last year.

This is all just speculation on my part, of course, but it's important to acknowledge the sensitivity of PECOTA projections to the quality of the input data (both the players on the ML rosters who are being projected and the database of all player-seasons used to determine the most similar players or comparables that are essential for making PECOTA projections).

I imagine that when Nate handed off PECOTA last Spring, it took a while to get all the parts working in the new software/hardware environment. I imagine that to test the new routines they first "replicated" last year's projections to see if they got the same projections that Nate got last year. If they did this, then it's unlikely that the inards of PECOTA are producing the problem that's been reported. Then again, based on other analyses we're seeing (as reported by Tango and others), last year's PECOTA's seemed fluky compared with previous years. So maybe simply replicating 2009 projections was itself a bit problematic. Until Clay explains this further we just don't know.

Feb 15, 2010 08:54 AM
rating: 4
 
dianagram

Does BP headquarters have a "NateSignal" they can flash in the sky? :-(

Feb 15, 2010 09:35 AM
rating: 6
 
BindleStiff

There are several pitchers who are projected to allow 0 home runs. And in 60 to 100 or more innings...

This is giving them some remarkably low ERAs. This includes 33yo minor-league journeyman Winston Abreu, who projects for 0HR in 64IP and a MLB-leading 2.14ERA. Winston is a pretty extreme flyball pitcher with a career MLB HR/9 over 2.00 and he allowed 2HR in 6 major league innings last year.

Several other pitchers project for 0 home runs allowed, and the large majority all appear to have names that start with "A"... Adams, Aardsma, Arredono, Affeldt, etc.

Guys like Alfredo Aceves and Alberto Alburquerque do predictably well given the doubly A nature of their names... Aceves gets 0HR in 131IP and a 2.79ERA while the heretofore unremarkable Alburquerque gets 0HR in 105IP and a 3.03ERA.


Feb 15, 2010 09:51 AM
rating: 8
 
Juris

Thank you. Also note this is showing up in the weighted means spreadsheet itself, so it's not just a PFM problem.

Feb 15, 2010 09:57 AM
rating: 0
 
Juris

If anybody has the book already, can they please look up the forecasts for these pitchers so we can see if the problem is baked into the book?

Feb 15, 2010 10:00 AM
rating: 0
 
BP staff member Steven Goldman
BP staff

The problem is not in the book. The pitchers are projected to allow home runs, even Ryoto Igarashi... And Alfredo Aceves.

Feb 15, 2010 10:58 AM
 
Juris

Thank you! That's a relief.

Feb 15, 2010 11:04 AM
rating: 0
 
donaldekelly

Yikes

I do note that CHONE predictions had some adjustments in the last few days as well

Feb 15, 2010 10:09 AM
rating: 2
 
McLovins

Not to belittle the more pressing issues here, but is there any chance that we are going to get Upside and Beta numbers soon in the spreadsheet? As an aside, I'd love the following features added to PFM in the future: 1.) hybrid leagues 2.) allow display only columns in PFM.

Feb 15, 2010 13:48 PM
rating: 3
 
Juris

Shmooville: I think the folks at BP have told you that their primary focus is on getting the core number right, and then generating the PECOTA cards, and with that you'll get Upside and Beta since they are standard features.

Feb 15, 2010 14:53 PM
rating: 1
 
Ira

I've been asking about this for years, and I'm wondering if it will show up in the player cards. I was hoping to see the translations for the projections for minor leaguers which matched them with the level that they are expected to be at. While this has little to no value for fantasy owners, I assume that the projection of minor leaguers isn't for that. us prospect hounds are more interested in what, for example, Marcus Lemon will hit at either AAA or AA than what he might hit in Arlington. I know this is a bit rougher task, but it would be cool to include, next to each player who didn't play in the big leagues last year's major league stats, his stats at the highest level he was at last year, and at the level above it.

So, for example, Martin Perez was skipped from low-A to AA, but at 19, its unlikely he will be in the majors next year. It would be interesting to see his projection for AAA and AA next to his projected 5.27 ERA for the majors.

That's even more interesting for a lower level guy like Wilmer Font or Robbie Ross.

Feb 15, 2010 16:32 PM
rating: 3
 
John Carter

Just to let BP know, I consider this a huge screw-up. I have spent hours and hours feeding in PECOTA ERA projections into my player analysis for the upcoming season thinking I was using their mean projection. That's what I wanted - not a conservative estimate, but one that does include a fair portion of their upside. Just to be consistent, I will have to finish my analysis with the less desirable figures. Thanks, at least, for letting us know so that we can deal with with this in whatever ways we need to.

Feb 16, 2010 15:11 PM
rating: 5
 
Schere

"I believe that this is the result of a selection effect - that pitchers who do better than their true expectation are the ones who actually pitch in the majors; any slippage quickly results in reduced playing time or demotion to the minors (except for teams with no reserves, like last year’s Brewer rotation). "

This brought a couple of things to mind.

1) It should be pretty easy to test, if I understand the thesis: pitchers with high expectations (aces) would presumably be allowed to continute to pitch even if they were merely mortal and thus would not suffer from this selection effect as much as pitchers with lower expectations.

2) The other way that pitcher innings get limited is by injury - if some bad pitcher performance in the comps is just pitcher injury (surely some is), then the selection bias would come from this, too. I would guess (though it really is a WAG) that this would be a sort of fat tail in a probability distribution in a pitcher's comps - most of the good pitchers stay at a similar level of performance, some fail modestly and some fall apart due to injury. I guess this would imply that the weighted mean is less useful, since the distribution is less normal (if the WAG is correct.)

Feb 16, 2010 18:01 PM
rating: 1
 
forthepunx77

Any update on the progress of the player cards? Eagerly anticipating them.

Feb 18, 2010 09:09 AM
rating: 3
 
schlaack

Why do the rate stats for players differ in the BP 2010 book versus the spreadsheet and PFM on the site?

I feel like in years past, while the counting stats were subject to change based on playing time in the PFM, the rate stats were always constant. Am I remembering that wrong?

For example,

Alex Rodriguez

In the book: .282/.388/.532
In the PFM: .288/.403/.578

This is a pretty big difference.

Feb 19, 2010 05:49 AM
rating: 1
 
Mike

Will there be a post explaining today's update to the PFM?

Feb 21, 2010 18:19 PM
rating: 3
 
Mike

I want to give BP the benefit of the doubt here, but there are some serious customer service issues going on. The credibility and worth of 2010 PECOTA (and by extension the PFM and Annual) have been seriously called in question by BP users and prominent baseball writers (such as Neyer and Cameron). As such, the paying customers have been posting some pretty serious questions about the value and quality of the content for which they're paying and the BP staff have not made a sincere effort to address these concerns in a clear and meaningful way. While I appreciated the initial transparency regarding the PECOTA and PFM bugs, that transparency appears to have raised further questions that demand further transparency.

If there is a major quality issue with what has been put out hitherto my hope is it will be admitted to and addressed. However, if BP stands by what has been put out I wish they would make a greater effort to alleviate the concerns of their customers and detractors.

Feb 22, 2010 19:03 PM
rating: 1
 
donaldekelly

Yes, the Yankees are sinking

And what about the criticism that hitting is undervalued? That you can't have the OPS and Runs Scored stats that are on the chart? They are incongruous.

Dave Cameron said about the 2/13 projections:

"the projected team OPS and team RS totals are basically impossible. RS too low for given OPS."

and

"Last year, league OPS was .751. DCs have 26 teams beating that mark, yet league RS going down from 4.61 R/G to 4.52 R/G."

If this is not true, I would like to know. BP is the only site I pay money for, so I have a lot of respect for it. I just want to know how dependable the new calculations are.

Feb 21, 2010 18:55 PM
rating: 2
 
ccweinmann

Has there been any update that anyone is aware of on PECOTA? There are still all sorts of anomalies, and the PECOTA cards still aren't out. The silence from BP on the subject makes me nervous.

Feb 22, 2010 12:51 PM
rating: 5
 
jdouge
(968)

Given that, in the last two years, PECOTA has been off its past form at one-year forecasts, its strength for me, in deep keeper leagues, has become the long-term projections: Upside, long-term attrition rates, stars/scrubs percentages. I have found those invaluable and unavailable anywhere else.

But as this year's rollout has alternated between delay and debacle, I've had to go without these numbers for both of my long-term, deep keeper drafts. If I played roto or Scoresheet, I can't imagine having much confidence in the one-year projections at this point, either.

I understand and appreciate the work that has been done to update PECOTA's code and incorporate improvements. But with major discrepancies in every update, adjustments being made on the fly this late in the preseason, and having, basically, no numbers I feel I can trust (and none of the most important numbers, for me, at all) into late February has made PECOTA --which has (now unfortunately) been my cornerstone -- useless to me in my draft preps this year.

Still love BP, been a daily reader since before the subscription days, but my frustration here is boiling over.

Feb 22, 2010 17:11 PM
rating: 3
 
You must be logged in to post a comment. Not a subscriber? Sign up today!

Baseball Prospectus Home  |  Terms of Service  |  Privacy Policy  |  Customer Service  |  Newsletter  |  Masthead  |  Contact Us

Baseball Prospectus Unfiltered is powered by WordPress.
Copyright © 1996-2013 Prospectus Entertainment Ventures, LLC.