keyboard_arrow_uptop
Baseball Prospectus is looking for a Public Data Services Director. Read the description here.

Sometimes, when you realize you've stepped in something malodorous, there's nothing to do but pause and deal with the situation at foot. Keep walking, and the substance will remain on your shoe, where it will begin to attract unwelcome attention. We made a similar misstep, in a proverbial sense, with some of our PECOTA projections this season. Recently, we discovered that National League pitchers were projected to be substantially worse than PECOTA’s logic intended due to a bug that slipped past our testing.

As unpleasant as removing the offending substance from your footwear can be, there’s always reason to be glad when it’s gone. In our case, that reason is the increased accuracy of our projections. While Clayton Kershaw still isn't projected to lead the league in WHIP—his past control problems still count against him, as Colin proved they should, so PECOTA isn't quite as wowed by his recent greatness as you or I might be—he is now projected to have the fifth-best ERA among qualifiers. You’ll find that virtually all other NL pitchers have seen their stats improve as well.

We’ve made one other big change that wasn't really a “bug fix” but will impact anyone who looks at the Depth Charts. For years, people have wondered why the team AVG/OBP/SLG projections weren't really in line with what teams ended up hitting, especially in the National League. The answer has always been that we've excluded pitcher hitting from these projected rate stats, while incorporating it into our team runs scored and allowed estimates. Well, we've changed our minds and added pitcher hitting to the team totals. So, sharing some of the pain I endured while writing about Astros players for BP2012, I'll inflict their projected season batting line upon everyone:

AVG/OBP/SLG (TAv): .248/.304/.371 (.243)

Considering Houston hit just .258/.311/.374 in 2011 and shipped out some major contributors (Hunter Pence and Michael Bourn, mainly—there weren’t that many major contributors to begin with), that projection seems entirely reasonable, much to the dismay of Astros fans.

Oh, and that number in parentheses? We added TAv to the slash lines while we were at it. (Before we added the pitchers in, the TAv values didn't make sense, as almost all of them were above the .260 mark.)

And for more fun with Depth Charts, there's a new "Visual Depth Chart" page, inspired by Ben Lindbergh's article on position weaknesses around the league.

We also made some other improvements in this PECOTA run that will be reflected soon on the player cards, and for now in the downloadable spreadsheet (click on “manage profile”), Depth Charts, Player Forecast Manager, Team Tracker, and the new Scoresheet Draft Aid. Among them are:

  • A downgrade to many player FRAA values, which were being calibrated against historical values and coming in a bit too high.
  • A fix for a minor bug in stolen base projections. I noticed this while writing about Nyjer Morgan for Value Picks a while back—it turns out PECOTA was over-counting steals in seasons where a player played for multiple teams, as Morgan did in 2009, and the smart readers picked up on that before I could quickly fix the problem and sweep it under the rug. (Just kidding—we'll continue to keep you posted when we change things, as we’re doing with this post.)
  • A bug fix in pitcher WARP calculations that was originally fixed during the 2011 season but wasn't propagated to the earlier PECOTA runs. It’s in there now, so some pitchers will see a change in their WARP.

Writing this sort of “confessional” isn't much fun, but we at Baseball Prospectus do think that being open about such things is only fair to you, our readers. We hope that you’ll enjoy the improvements in the stats now that the fixes are in place. Meanwhile, we're paying even closer attention to where we step.

UPDATE TO THE UPDATE: Two more quick changes to tell you about. First, we've switched from using 2011 positional adjustments to using positional adjustments generated from the projected data in the depth charts themselves. Secondly, we've changed the weights on stolen bases and caught stealings to better reflect the run environment currently. –CW

Thank you for reading

This is a free article. If you enjoyed it, consider subscribing to Baseball Prospectus. Subscriptions support ongoing public baseball research and analysis in an increasingly proprietary environment.

Subscribe now
You need to be logged in to comment. Login or Subscribe
brooklyn55
3/01
Rob ... upfrontness (gotta be a Colbert word) appreciated. The update though has dropped the Japanese pitchers, Darvish, Chen, Wada ...
mcquown
3/01
Yeah, they haven't shown up in spreadsheets yet. In the interest of time, we haven't fully projected these players yet, but instead, we've used portions of PECOTA to provide interim forecasts so that standings and fantasy values could be computed reasonably. Expect slight changes to them at some point in the future.

You'll find the current projections for those guys in Depth Charts and in PFM.
jessehoffins
3/01
Wait, are the DC's for the NL pitchers accurate now?
mcquown
3/01
Yes, they should be fine. E.g. Kershaw has a 2.95 projected ERA.
jessehoffins
3/01
So when was this change implemented and why haven't all the updated dates for nl pitchers changed?
mcquown
3/01
Update Dates on the Depth Charts are changed whenever a Depth Chart playing time adjustment is made, not when projections are altered, as every change in PT for any player impacts all the projections, as the league baseline is reset, and that has a ripple effect. Obviously, it's usually a lot less impactful than this change was, however.

The date for the spreadsheet is on http://www.baseballprospectus.com/fantasy/

fieldofdreams
3/01
So should we ignore what is in BP 2012?
BarryR
3/01
This is a very good question and I'd like to see the answer.
alexknapik
3/01
You don't have to ignore them; just know that online you can find "more accurate" or "truer" PECOTA projections.
dpease
3/01
The effect of this bug was to slightly deflate NL pitcher performance, and the projections in the book were affected by the bug.

Softy Electric is also correct, and in general projections will improve over time and the latest projections from the site will always be better than the projections in BP2012.
modofacid
3/01
Rob,

I can't select players drafted during the draft to be on my team (the check box on the right side doesn't show up). Is it a Chrome issue? Or are other people having the same issue?
mcquown
3/01
Is this a PFM question? We test PFM using Chrome on both PC and Mac. What happens when you select them?
mcquown
3/01
Sorry, I added that last question and clicked Submit and then realized that you'd said you can't even see the boxes... Feel free to use Contact Us form to send questions about what you're trying to do and what's happening, please. I'm confused, but we'll make time to help.

If you mean the Scoresheet Draft Aid, reloading the page should remove any players who have been taken, as it polls the Scoresheet data directly.
o2bnited
3/01
It sounds like perhaps you don't have the user centric inflation option turned up. It defaults to "0 (None)" and you'll need to change it to 1 or greater to have the checkbox show up to indicate which players are on your team. Also, don't change it back to zero later, or it will forget which players you told it were on your team. If that doesn't work, please contact Customer Service and we can give you more individual support. Thanks!
jrmayne
3/01
Fixing things is good. As I've noted ad nauseum, I think that sticking to your guns on some of PECOTA's perceived flaws has been mistaken; finding and fixing flaws is important. There should be a mental bonus, rather than a mental penalty, for fixing things.

On this topic, I think "Colin proved they should," in regards to using older data weighted higher than other projections should read, "Colin asserted they should." If there's proof in the linked article, I didn't see it.
jfcross
3/01
Little bugs like this are inevitable in a program as complex as PECOTA so no worries and good job catching it.
Mooser
3/01
I agree. I have been eagerly awaiting this.
markpadden
3/01
ETA for 10-year projections?
kcheaden
3/02
I'd love to see a "Visual Depth Chart" for defense only.
mcquown
3/03
Ask and ye shall receive.
kcheaden
3/05
Awesome. Thank you.
acardonick
3/02
Has anyone pointed out that this change seems to have reduced P strikeouts across the board? For literally every one of the pitchers I checked, SO/IP went down, in some cases by a fair amount.
joechris96
3/02
We're going to check into this. Thanks
LynchMob
3/02
Was there an article posted which reviewed the 2011 PECOTA projections?

For example, what % of players exceeded their "2011 PECOTA Weighted Mean Projections"? (question: should it have been 50%?)

Another type of question I'm interested in ... what % of players which PECOTA said had a 5% chance of "Breakout" actually had a Breakout?

Thanks for PECOTA!
dpease
3/02
LynchMob, we do have some PECOTA results analysis that we're planning on running next week. Thanks!
mcquown
3/02
Not to split hairs too much, but it should be 50% in 50th Percentile, which is somewhat different than the Weighted Means lines, as, well, to put it simply, there's more room to go up than down.

Or, put another way, the difference between Weighted Mean and 50th Percentile is essentially the difference between statistical measures "mean" and "median".
davinhbrown
3/02
appriciate the upfront feedback with us, your customers.

Lots of other places would either leave it go with the mistakes, not inform us of particulars, or have the guts to answer the tough questions not only on this subject but all the articles.

Thanks.
winder
3/02
Well whats up with the AL pitchers? Pecota has knocked off about 10 points of VORP from guys like CC,Lester and CJ Wilson.I would like for that to be explained to me.Is there something I`m missing? With these projections fluctuating,it makes me wonder at what point I can start trusting them.I'm not being cranky but I just don`t understand.
BrewersTT
3/02
With respect, if your definition of trusting them is that they should consistently be less than 10 VORP off, you are maybe putting more faith in them than any sort of projection can really provide.
dpease
3/02
hi there. This is a result of the third point on the list that Rob mentioned. The pitcher WARP calculation had a bug in this year's version of PECOTA.
Kongos
3/02
Ha! In the time it took me to draft a comment complaining that the math in the Visual Depth Chart was screwy, you fixed it. Way to go!
derekdeg
3/03
Can someone tell me why PECOTA seems to dig Jose Altuve. Especially in NL 4x4 leagues. Whats with that?
mcquown
3/03
The projection for Altuve is .269/.300/.373, which hardly seems bullish to me, considering the damage he inflicted on the minors last year. For 4x4, he's fast, which always inflates a player's value.

I assume you're looking at PFM values, and this is where PFM can really help a fantasy owner get an edge. Second base is a very weak position (especially in the NL) this year. And while Altuve's low R and RBI totals hurt him, his batting average isn't terrible, his HR projection isn't quite as bad as you'd think based on his stature, his average is fairly neutral, and that speed really helps. All-in-all, PFM's scarcity-based valuations really bring to light how much a guy like that will help. And he's likely to cost only a fraction of his projected value in auctions.

kdierman
3/12
This (upgrade) is great ...better than adding Cliff Lee at the trading deadline.
BBlackwell
3/14
This might be the wrong place for this comment, but my PFM is occasionally acting a little... well... funny. After I update after the occasional pick it groups ALL pitchers, relief and starter, into one group estimated at $0.

My positional needs are not filled, even at P, and even if they were I would have bench room where I would imagine Livan Hernandez would not be as good a choice as Max Scherzer, despite what my PFM is telling me.

Does anyone know what's up with this? It was fine for a while - for example, it was telling me Aramis Ramirez, Michael Bourn, and Chris Carpenter were the top 3 available players for my needs. Then, another team selected Dustin Ackley. That caused Carpenter to lose $100+ in value and drop to $0.

Is this a common issue?
mcquown
3/14
Thanks for using PFM!

The most common cause of something like this is running up against the budgetary constraints you configured in the league. If you modify the the budget settings and still see issues such as this, feel free to send your CID (it's part of the URL for the PFM report you're looking at, so you can just paste in the entire URL) to me via the "Contact US" page, we can look into what's going on with your particular league.
cef1970
3/19
Rob (and all) --

Maybe I'm just missing something simple but here something I've noticed over the past couple of fantasy seasons, and I don't know if you can speak to Baseball HQ's methods, but:

For the past few seasons, running my fantasy leagues through both dollar valuators, BP's CDM and your PFM, I've noticed that Baseball HQ $ values seem higher for a large number of players, compared to Baseball Prospectus'. It may just be perception; I haven't added up the dollar figures for the entire pool.

I understand that the projection systems differ, but shouldn't, on the whole, everything even out?

In particular, the BP values seem to greatly undervalue starting pitchers compared to HQ, with BP valuing saves and steals more.

Any insight? A difference in playing time estimates? A difference in just how much certain stats are valued in the standings?

Is it that BP squishes guys to the middle more than HQ?

Thanks,
Foster
cef1970
3/20
Looking more into the comparisons, it may be just that I'd been using the defaults: BP "forces position" while HQ does not. That option in HQ lowers their values more in line with BPs.

Again, I understand we're talking about two totally separate systems; I just couldn't figure out why BPs seems so nearly universally lower than HQs, given the same player pool and $ to spend.
mcquown
3/20
Not ignoring these well-thought-out observations, sorry for slow reply.

The short answer is that PFM constantly revisits the player pool for available values in every category, and so there's a very accurate assessment of exactly how much each player is worth, in terms of lost opportunity and benefit to the team. Offhand, I don't know why this would end up with a flatter curve. I do know that the "player pool" for some other projection systems are not made in such a way as to come out to league totals, given league limits on playing time, whereas the BP Depth Charts account for 100% PT and so the PFM "pool" is always going to be exactly MLB. We are discussing this internally, and may end up with a more in-depth reply at some point -- this would be a good topic for the Fantasy team to cover in a full-length article at some point, in fact.