CSS Button No Image Css3Menu.com

Baseball Prospectus home
  
  
Click here to log in Click here for forgotten password Click here to subscribe

<< Previous Article
Fantasy Article Team Health Reports: L... (03/18)
<< Previous Column
Fantasy Article Fantasy Beat: Hot Spot... (03/18)
Next Column >>
Fantasy Article Fantasy Beat: The Prod... (03/19)
Next Article >>
Premium Article Under The Knife: Frick... (03/19)

March 19, 2010

Fantasy Beat

Weekly PECOTA Update

by Dave Pease

Here's a quick roundup of PECOTA news for the week:

  • Clay has posted the 3/19 depth chart/PFM update.  The 3/16 update was accompanied by a tour of the first half of the major leagues through spring training; if you missed it, please have a read.  We got enough positive feedback that we wanted to make sure the second half was seen by as many people as possible, and since the blogs system is still a work in progress its going to run as a full article.  It'll be published soon.
     
  • The PECOTA beta hitter cards have had another build which is undergoing review by our newly formed beta test squad.  We're hoping to release them in their final form this weekend.
     
  • We are hoping to have a build of the PECOTA pitcher cards released to the beta testers this weekend as well--we're finishing them up right now.  Fun fact: we're going to end up with full cards on nearly 6000 players this year.

Thanks for reading, and have a great weekend.

Dave Pease is an author of Baseball Prospectus. 
Click here to see Dave's other articles. You can contact Dave by clicking here

31 comments have been left for this article. (Click to hide comments)

BP Comment Quick Links

RSilver76

Sorry, I really don't want to be an ass. You guys are running a business. I pay for services from your business. Normally I think yout business does a wonderful job providing services, that's why I continue to pay for your services.

But really - REALLY - it is MARCH 20th. Almost all of us now have drafts this weekend. That's what I'm paying for - I can't speak for anyone else.

You're STARTING to create pitcher cards? Really? On March 20th? Really? And you think this is acceptable?

Just credit back my credit card and we will call ourselves even for 2010...otherwise....what the hell are you guys doing?

This has nothing to do with the QUALITY of your product.....

You owe me my subscriction fee back because the TIMINING is not acceptable.

Mar 19, 2010 21:35 PM
rating: 16
 
BP staff member Dave Pease
BP staff
(2)

Understood--the timing is certainly not what we planned, and I am sorry for the delay.

If the one-year projections are not enough for your draft prep, or if you have any other questions or concerns, please send me an email at dpease@baseballprospectus.com anytime.

Mar 19, 2010 21:48 PM
 
sbussmann

I've seen this notion posted more than once that the currently available one year projections are sufficient for those playing in non-keeper leagues. As someone who does not play in keeper leagues, the player cards are still very important to my fantasy valuation.

For example, let's say I don't believe Nate Mclouth's weighted mean PECOTA projection. In this case, I might be inclined to use his 30th percentile projection for my own valuation. Without the player cards, there is no way for me to do this. The player cards are important even for those of us that are not looking for multi-year forecasts.

Mar 20, 2010 15:09 PM
rating: 5
 
BP staff member Dave Pease
BP staff
(2)

Understood--the cards certainly present a much wider selection of information than the one-year weighted means (or 50th percentile) projections.

Let me put it this way: if most of the value of the fantasy or premium subscription for our subscribers who use our stats for draft prep is rolled up in the PECOTA cards, we surely need to work on an additional product offering, because I can't imagine that being efficient.

Mar 20, 2010 19:50 PM
 
Bodhizefa

I still haven't seen any actual reasoning or explanation for some of the ludicrous projections that were put out in the late February update. Chris Davis had a 90th percentile of 52 homeruns. Alex Rodriguez had a 90th percentile of a .700 SLG%. These and many many more questionable numbers were changed with the recent updates without any mention of cause. These errors (yes, I'm calling them errors because they were so out of whack with what PECOTA used to spit out) were not a playing time issue. They weren't a 10-year issue. They were severe problems in the one-year calculations for a bevy of players.

In other words, PECOTA was worthless weeks ago, and for us -- the subscribers -- to have any sense
of it having been reconciled, it would be nice if you guys would actually fess up to what in the world you did to screw these numbers up so badly and what you did to fix them. Sweeping it under the proverbial rug would not be a good first step, I would think.

I think it's telling that in 2003, when I was first introduced to the site, Baseball Prospectus told me everything I needed to know about statistics and value and the great world of sabermetrics in baseball. Now? I only come here for prospect reports phoned in by scouts. I think the message has been lost somewhere, gentlemen.

Mar 19, 2010 23:54 PM
rating: 20
 
Chomsky
(103)

+1

Mar 20, 2010 05:38 AM
rating: -3
 
Ira

While your point is valid, your examples aren't. Chris Davis has some serious power. 50+ home run power. he hit 31 last year in an obviously down year, and 40 the year before (split between AA, AAA, and the majors, plus 36 in 2007. And he's just now turning 24.

As far as A-Rod slugging .700, well, stranger things have happened. People forget that Alex Rodriguez is truly a once in a generation talent. Mickey Mantle, Hank Aaron, Barry Bonds, Babe Ruth, Honus Wagner level talent.

Mar 23, 2010 07:13 AM
rating: 0
 
Clonod

So. . . what is the new hitter card build intended to fix?

We're all paying for this stuff, Dave. I understand that BP is going through growing pains trying to make the system better and more efficient. And I'm confident that once this is over, we're going to end up with a better PECOTA. I get that.

But in the interim, some of us are entering fantasy drafts with hundreds of dollars in entry fees. And we pay BP for a quality product. I appreciate the updates, but, at minimum, an acknowledgment of what needs to be fixed would be appreciated. We've all figured out on our own that it has to do with outlier projections and multiyear projections, but an acknowledgment from BP as to what on Earth is going on would help out your paying customers.

One of the problems with this process has been the perceived lack of transparency. You come onto the blog and tell us you're working on it, but you don't say what the problems are. You owe it to paying customers to give us a little more detail.

Thanks,

Chris

Mar 20, 2010 00:48 AM
rating: 9
 
Sharky

Can't wait for the team-by-team article. When Clay published the last one, he said the remaining teams would be ready the next day! Definitely eager to see that. It was one of the best pieces I've seen on BP in a while. Any specific ETA?

Mar 20, 2010 05:50 AM
rating: 1
 
BP staff member Dave Pease
BP staff
(2)

coming Sunday morning... thanks

Mar 20, 2010 22:35 PM
 
bigdaddyleroy

I agree that transparency is the issue here. You guys clearly are still tinkering with PECOTA. Many of the projections on the weighted means sheet are still moving. Between 3/16 and today, virtually every pitcher's rate stats have changed, and many of them have changed dramatically. To pick just one example, Scott Baker's innings pitched haven't changed, but his projected ERA has moved from 3.92 to 4.11. So, you're making other adjustments to the projections, and it would be nice to know what they are.

Mar 20, 2010 06:03 AM
rating: 0
 
Richie

One thing going on. Things are backed up because they badly underestimated how difficult and time-consuming it would be to switch over from Excel to whatever platform they've chosen. They've said this already.

Second thing. The reason the PECOTA cards have gone to the Beta testers is that, so far as BP knows they are now ready to go. First they'll see whatever (hopefully minor, data misentry type) things might be wrong with them.

Mar 20, 2010 07:58 AM
rating: 5
 
Mike Juntunen

I think you guys are mistaking 'tinkering' with 'fixing'. It's clear that some aspects of PECOTA were not working properly at its first port out of Nate's coding, or even its 2nd or 3rd. Most of those seem to be fixed now, and the book projections have always been in line with what's going up on the website.

If you're not a keeper-league player or someone who's drafting minor leaguers, there's nearly nothing you're going to get from the new hitter or pitcher cards that you don't already have access to.

I think the transition issue has had less to do with platform and more to do with the intricacy of the system that used to be maintained by one person who designed it (Nate Silver) and now not only has to be ported, but had to be learned in the process by the people doing the porting.

It certainly sucks to feel like PECOTA is less than the most reliable rock you have available to lean on that it's been in the past, but I do think the histrionics are excessive: if you have access to the book projections, only a significant minority of fantasy players do not have access to the full data they need to draft this weekend.

Those of you who are pushing should keep in mind that the situation is as it is: an early release is just more likely to include bugs or problems or simple miss-links or accessing of old databases instead of the newer versions. Would you take a release today in exchange for a potential major database error, or a monday release without one?

Mar 20, 2010 10:24 AM
rating: 4
 
Bodhizefa

You are wrong, plain and simple. There were major issues with an assortment of player projections for the 2010 season up until very very recently (and I'm still not entirely sure they've been corrected). I mentioned some of them above in my first post, and it warrants bringing up again for those who have simply overlooked or not noticed the errors -- someone like you, for example.

Also, most fantasy publications I know had their very rock solid and very usable projections out in December (some were even ready in November!) That BP, in mid-to-late March, is still fixing (not tinkering, FIXING) their 2010 projections to approach even moderately believable levels of accuracy is beyond incomprehensible. Fantasy drafts are over. They're done. There is no point in owning BP's fantasy membership at all in 2010 any longer. It's a sunk cost.

I have loved Baseball Prospectus since the very first day I ever visited the site in 2003. But the quality control in the past two years, and most especially this year, in terms of their fantasy product has been absurd. If I had paid for a fantasy subscription, I would be absolutely livid right now. The fantasy product's time for usefulness has come and gone. Period.

As far as I'm concerned, BP has had well over a year to iron out the kinks in PECOTA's transition for Nate Silver's Excel files to the new program format. It's a year that they've apparently spent twiddling their thumbs instead of paying attention to glaring, outrageous errors in their system's output. And the damage control has been almost as outrageous, too.

Mar 20, 2010 11:59 AM
rating: 6
 
Mike Juntunen

I think its clear they don't really disagree with you. However, it is worth noting that at no point in this process has there been a line of complaint regarding the BP 2010 book projections. BP has never had PECOTA's out in Decembers for a wide variety of reasons, not limited to updating PECOTA, updating or creating new metrics, adding new features, and the re-evaluation and adjustment of metrics that contribute to PECOTA (like park factors and defensive metrics).

The cost of PECOTA's performance and greater information flow has always been that it takes longer to produce, and while it is clearly a problem this year - no doubt about it.

Equally, while the issues this year are clearly a loss, its quite apparent to me (admittedly, not a serious fantasy player) that the fantasy content is recieving a huge influx of attention, staff and time.

If you feel this time has been spent on thumb-twiddling, it is my belief you have no personal first-hand experience with excel-based programming and converting it into a real programming language. PEOCTA took Nate years to create, and it essentially had to be re-written from scratch into another language and every calculation (which, as was mentioned, used to take over 72 hours to process) verified to be working identically to its excel counterpart.

It was clearly a mistake that BP expected that they could do this over the winter with part-time work. That they addressed that too-late doesn't mean they didn't make the mistake - but it does mean that the mistake was already addressed over a month ago through Clay's full time employment to handle this specific task.

My point was more that there really is nothing to gain by complaining and pointing fingers over a series of problems that were already acknowledged and have already recieved attention. Unless you believe that the responses will prove to be inadequate (and it's really too early to tell if they're going to work for the rest of the year or not), all you're really doing is telling them to be ashamed of themselves for making mistakes in December and January and trying to correct them in February and March. Clearly, they are, but you're not really contributing anything constructive by doing so, since they've already taken steps to address pretty much every one of those issues.

Mar 20, 2010 12:25 PM
rating: 3
 
gluckschmerz

I agree Mike, especially when you write "you're not really contributing anything constructive..., since they've already taken steps to address pretty much every one of those issues." That is where I hope BP understands what is happening.

People need to vent.

The comments here and elsewhere are serving as a forum for venting frustration and disappointment. BP sets a pretty high standard for itself and has clearly fallen short of expectations. I have worked many years in customer service and sometimes an organization just screws up, drops the ball, chokes, ignores a problems, lets things spin out of control, gets overwhelmed etc. And when this happens you try to make things right with your customers, but you sometimes just have to take your medicine and let them vent their frustration.

Then hopefully you have another chance to earn back their trust and confidence. If so, you both move forward together. If not, you both move forward separately.

Mar 20, 2010 12:46 PM
rating: 4
 
Bodhizefa

How is pointing out mistakes that BP themselves have not admitted to considered complaining, Mike? Do you think I'm voicing my concerns because I think Year 10 of the 10-year forecast looks a little off? Of course not! I'm pointing out that there have been serious and sweeping errors in 2010 projections, which you (much like BP) continue to ignore in your comments.

Mike, it's clear that you have no interest in fantasy content and are not affected by the lack thereof at all. But for the thousands of subscribers that BP has that do look to BP for fantasy projections and content, the lack of any semblance of a timeline or reliability in the services offered has presented a very real issue that has not been voiced loudly enough in my estimation. That you are content with an ever-drawn-out delivery of the site's product doesn't make it okay. Your lack of fantasy interest has nothing to do with the thousands of people who paid for subscriptions to receive a product that has been virtually worthless since it's unveiling.

I'm not trying to shame BP for the errors they've made. I'm trying to make the point that their efforts are now essentially useless for a fairly large number of people who paid for their services. That you think BP has successfully addressed these issues with their opaque transparency and their expanding timeline is in many ways more preposterous than some of the numbers PECOTA has been spitting out for the past couple months. The only step that could possibly address the problems at this late in the fantasy game is either a refund or an upgrade.

Mar 20, 2010 13:16 PM
rating: 1
 
BP staff member Dave Pease
BP staff
(2)

Nobody's content with the way the PECOTA rollout went this year, ourselves definitely included. I've apologized for it personally, and will continue to do so. We were in a hard place, and we made some estimates of difficulty and execution that turned out to be optimistic, and if we had the ability to go back and achieve a more normal rollout, we certainly would do that.

We're not done talking about the PECOTA process and the lessons we've learned from it yet, but our top priority is getting the PECOTA package completely released.

Mar 20, 2010 13:40 PM
 
Mike Juntunen

I don't have a ton of interest in PECOTA as fantasy content, but I do have a lot of interest in it for the reason it was created, and certainly have had one of the main reasons I had a BP subscription deprived to me thusfar.

PECOTA has never strictly been a fantasy tool and in fact wasn't initially introduced as one. It's a forecasting tool and one that I, as a fan, spend a ton of time with in order to better inform my baseball watching experience.

While my use is a bit less deadline-sensitive than yours, I've certainly lost out on a big part of why I pay for a subscription.

Nonetheless, as someone who's coded many an excel calculator, I empathize with what they're going through right now because I appreciate how difficult it is to pick up someone else's excel sheet and modify it without breaking it.

Mar 20, 2010 16:19 PM
rating: 0
 
dianagram

Let's step back for a moment:

Do many of us love and support BP, and rely upon its PFM and PECOTA for our drafting? Yes
Do we sometimes look at a projection and think, "that doesn't look right" (either good or bad), and discount it from our draft sheet? Yes
Do we also seek out other fantasy draft projections at the same time? Yes

Is $40 for a year of all services a good deal, or $20 for fantasy features only a good deal? Yes

Has BP botched the normally smooth rollout of the PECOTA? Yes
Is this the first major glitch most of us have experienced in our dealings with BP? Yes
Do customers have a right to be disappointed/angry about this? Yes

BUT .... let's realize that there has been a lot of personnel turnover at BP during the last year, and they're trying to automate an Excel process featuring tons of data and inter-related algorithms.

Just my opinion here, but if you are playing fantasy sports for "thousands" of potential dollars, perhaps you've lost sight of the "fun" that is supposed to be involved.

Fantasy sports are supposed to ADD to one's enjoyment of the game. Spewing vitriol at a supplier of information used in fantasy gaming may be cathartic for the venter, but after a certain point, it only turns into static.

Like some of my favorite players, I'm willing to give BP an "off-year", and I'm willing to pony up another $40 to give them another chance.

Comeback player of the year, 2011 perhaps?

Mar 20, 2010 16:13 PM
rating: 20
 
cbelford

Well said, dianagram.

Mar 20, 2010 17:13 PM
rating: 2
 
vertumnus
Other readers have rated this comment below the viewing threshold. Click here to view anyway.

Sorry, but with all your comments you seem like a shill.

Mar 20, 2010 20:50 PM
rating: -5
 
dianagram

Well, it COULD be construed that way. However, I've made enough constructive comments/criticism over the years here, and have had enough e-mail/in-person conversations with BP staff, to give me some perspective on this current turn of events.

Look back over my recent comments:
1) I lobbied for, and got, a customer-based "beta-testing" group for PECOTA.
2) I've lobbied for, and have been promised, a unique ID system for much if not all fantasy content, which will enable easier merging of disparate data such as THRs, PECOTA, PFM, and the Weighted Means Spreadsheet.
3) I've lobbied for a downloadable form of the Annual. (Though various members of BP staff agree with me on this matter, its out of BP's hands. Its a publisher's decision, and they haven't been able to convince a publisher to go that route yet.)

I simply believe in the mission and performance of BP more than any other independent baseball research site.

I'm not on BP's payroll. I'm not kissing their asses. They appreciate my input.

What do you bring to the improvement of this site?

Mar 20, 2010 21:03 PM
rating: 4
 
dianagram

Let me apologize for the last line of that comment. It was uncalled for. The rest of my comment stands as written.

Mar 20, 2010 21:33 PM
rating: 3
 
vertumnus

Well, in retrospect, calling you a shill wasn't too cool either.

"Like some of my favorite players, I'm willing to give BP an "off-year", and I'm willing to pony up another $40 to give them another chance."

Felt like you were directly stumping for subscriptions, which seems a little strong to me. Why frame it in that context when defending BP? I guess it's the form that a lot of these complaints take...

I totally agree that BP delivers great content overall. For fantasy this year, though, they've been subpar, and they should face the music for that.

Mar 20, 2010 22:15 PM
rating: 2
 
Mike Juntunen

I think you might find that people who have read BP for the longest time are most keen to defend the problems with PECOTA, primarily due to having witnessed its evolution first-hand and having a bit greater understanding of its complexity.

This leads to a tendency to defend the difficulties inherent in it because we have a little more perspective on how it had evolved and how difficult it must have been to transition it from Nate's private excel sheets to a real program run by multiple people.

Mar 20, 2010 22:43 PM
rating: 1
 
leites

My biggest complaint with the 2010 hitter cards -- the lists of comparable players -- now appears to have at least been partially fixed.

Everth Cabrera no longer appears to be a Hall of Fame caliber player (although his #6 comp is still Hanley Ramirez, which is kinda of fun to think about).

Matt Weiters' comps, sadly, are still a bunch of guys who never really had career as MLB starters. Would love to know why PECOTA "thinks" this.

When the release is done, could you guys offer some commentary on the more controversial projections/comps?

Mar 20, 2010 17:20 PM
rating: 2
 
Mike Juntunen

I think I could answer the Wieters question simply because it's come up before.

Wieters is an uber-prospect who had great minors numbers but who's MLB abs didn't nearly live up to the potential. There are, frankly, a lot of guys like that who never panned out, and its not entirely within PECOTA's capabilities to weed out this uber-prospect from all the others who looked like it and fell flat the second they hit real major league pitching. Another season will probably differentiate Wieters from those guys, but his Baltimore numbers were not really those of the kind of player he's suspected of being and are, frankly, quite in line with a lot of people who we'll all hope he isn't.

How many big-bodied, great-bat catching prospects wash out and turn out to be significantly inferior hitters to what people talk about? Piazza types are the exception by a long shot, and for every guy who looks like he could be one in the minors and hits the majors and is great, there are literally dozens of guys who would be lucky to get Dioner Navarro's career to date. Remember when he was an uber-touted catching prospect? Not quite Wieters level hype, but he was expected to be all-star caliber, remember?

Mar 20, 2010 22:47 PM
rating: 2
 
HarleyBK3

As a subscriber who gets well over $40 a year in value even without PECOTA, I'm not about to bail.

Regarding PECOTA, I understand it will always be subject to improvement, and fully support that it continue to be improved. In my opinion, though, it's simply timing. That said, here is my expectation / proposal:

- Continue to work on improvements during the year, but with a clear "stop tinkering/improving" date that is WELL before season's end (e.g. Stop tinkering in October). And for gosh sakes, take full advantage of those volunteer beta test guys. A group that I would be more than please to be part of.

- Once the 2010 MLB season is over, simply add the 2010 data to PECOTA, knock out the FINAL version of the cards, and know that BP customers everywhere will rejoice.

- Repeat the cycle annually.

Although the above is clearly simplified, I think it makes the point.

I understand that PECOTA will never be "good enough" as long as there are brilliant minds with ideas that will improve upon it. But a very good product that is timely is much more valuable than an "even better" product that doesn't reach the shelf in time.



Mar 21, 2010 10:30 AM
rating: 2
 
SnakeDoctor18
Other readers have rated this comment below the viewing threshold. Click here to view anyway.

Anybody care to weigh in...my friend's and I were having a discussion, if you have the 5th pick in a 6x6 rotisserie league with the extra offensive category as OPS, who do you pick assuming 1-4 is pujols, hanley, arod, braun. I argued for Longo , others said Prince, but the consensus was Utley which I think is an overrated pick? Thoughts?

Mar 21, 2010 19:00 PM
rating: -5
 
Mike Juntunen

I would suggest comparing each to your own judgement of who the 2nd or 3rd best option for that position is. The one with the biggest differential is probably the best pick

My guess is that the poorness of second basemen makes utley the best. Its far easier to get a player like wright at 3b than it is to get performance of that quality at second base.

Mar 21, 2010 19:08 PM
rating: 0
 
You must be a Premium subscriber to post a comment.
Not a subscriber? Sign up today!
<< Previous Article
Fantasy Article Team Health Reports: L... (03/18)
<< Previous Column
Fantasy Article Fantasy Beat: Hot Spot... (03/18)
Next Column >>
Fantasy Article Fantasy Beat: The Prod... (03/19)
Next Article >>
Premium Article Under The Knife: Frick... (03/19)

RECENTLY AT BASEBALL PROSPECTUS
Premium Article What You Need to Know: Bryant vs. Harper
West Coast By Us: Whatever, I'm Wearing Jort...
BP Wrigleyville
Premium Article Minor League Update: Games of Tuesday, May 2...
Premium Article The Prospectus Hit List: Wednesday, May 27
Premium Article Some Projection Left: Top 100 Draft Prospect...
Premium Article Rubbing Mud: Rubbing Not-Really-Mud

MORE FROM MARCH 19, 2010
Prospectus Q&A: Terry Francona
Premium Article Transaction Action: American League
Premium Article Transaction Action: National League
Premium Article Under The Knife: Frickin' Laser Beams, Part ...
Premium Article Player Profile: Nomar Garciaparra

MORE BY DAVE PEASE
2010-03-30 - BP Unfiltered: Site Slowness
2010-03-23 - BP Unfiltered: PECOTA Cards: The Final Front...
2010-03-19 - Fantasy Beat: Weekly PECOTA Update
2010-03-12 - BP Unfiltered: PECOTA Update
2010-02-09 - Premium Article BP Unfiltered: The Untitled Joe Sheehan Book
More...

MORE FANTASY BEAT
2010-03-22 - Fantasy Article Fantasy Beat: Hot Spots: First Base, Third B...
2010-03-19 - Fantasy Article Fantasy Beat: Hot Spots: Starting Pitchers
2010-03-19 - Fantasy Article Fantasy Beat: The Prodigal Oriole
2010-03-19 - Fantasy Beat: Weekly PECOTA Update
2010-03-18 - Fantasy Article Fantasy Beat: Hot Spots: Relief Pitchers
2010-03-18 - Fantasy Article Fantasy Beat: Mailbag
2010-03-17 - Fantasy Beat: Depth Chart/PFM Update
More...

INCOMING ARTICLE LINKS
2010-03-20 - BP Unfiltered: 3/19 Pecota Update: Weighted ...