Notice: Trying to get property 'display_name' of non-object in /var/www/html/wp-content/plugins/wordpress-seo/src/generators/schema/article.php on line 52
keyboard_arrow_uptop


You… can be a millionaire… and never pay taxes! You can be a millionaire… and never pay taxes! You say… ‘Steve… how can I be a millionaire… and never pay taxes?’ First… get a million dollars.”

–Steve Martin, Saturday Night Live, 1978

I can’t tell you how to be a millionaire without paying taxes, but I can tell you how to beat PECOTA without a computer model. First, get the PECOTA projections. Here, I will explain how you can beat PECOTA once you do.

Many of the people in your fantasy league have heard of PECOTA and use its projections while drafting. Certainly many of the people who win their leagues do. So if almost everyone has a mini version of Nate Silver with them on draft day, how can you get an edge?

The key is to stay one step ahead by knowing what PECOTA‘s strengths and weaknesses are. But let’s make one thing clear-PECOTA is very smart. It knows a lot of things that the naked eye does not. It goes through a hundred years worth of baseball players, finds the guys that are most similar to the player it wants to project, and generates a projection. You and I don’t have that kind of memory. However, we can incorporate some information that PECOTA can’t, and that give us an advantage over our competition.


WHAT PECOTA DOESN’T KNOW

A lot of my personal research has been on batted ball statistics, such as groundball, flyball, and line drive rates, as well as BABIP on each of these three types of batted balls. I have found that this information can be incredibly useful in projecting player performance, but here’s something you probably don’t know about PECOTA-as strong as it is, it does not use batted ball statistics. That’s not an error, but a sacrifice that had to be made. There are no records of batted ball statistics for the 1950’s or consistently measured statistics for the 1980’s, nor are there records of BABIP on groundballs, flyballs, and line drives. That type of information is only available from 2003 and on. To use that information, PECOTA would need to sacrifice choosing from hundreds of thousands of pre-2003 player-seasons to find comparables and would fail miserably.

However, you do have that information at your disposal, and you can use it to your advantage. If you find the more recent players in PECOTA‘s list of comparables for a particular player, you can check some of their basic statistics and compare them with the player in question. If you do this, you can improve on PECOTA, resulting in better player projections, and a better chance at winning your fantasy league.

In my recent research, I developed a quick and dirty method to project BABIP and projected 277 players for 2009. Of those with sufficient plate appearances, my quick and dirty method has a .45 correlation with BABIP so far this year, and PECOTA has a .42 correlation with BABIP this year. Though this is clearly too small a sample size to judge conclusively, it’s worth noting that I have done as well on that component using a model that includes no comparables, doesn’t adjust for age, doesn’t adjust for position, doesn’t adjust for handedness, and only uses data starting in 2003.

This isn’t to say you should use my model in place of PECOTA for BABIP. Rather, what you should do is use a hybrid of the two by looking at batted ball statistics to see where PECOTA is being tricked and can be adjusted.


EXAMPLES

  1. Geovany Soto and BABIP on line drives, groundballs, and flyballs:

    PECOTA‘s projected BABIP for Soto: .334
    My projected BABIP for Soto: .306
    Current 2009 BABIP for Soto: .270

    On the heels of Soto’s .337 BABIP in 2008, PECOTA projected him to repeat that number this year. However, what PECOTA doesn’t know is that last year Soto’s BABIP on line drives was .805, which is .087 points above the 2008 league-wide average. You might expect something like that out of a monster power hitter, but not Soto. In fact, line drive BABIP has far lower year-to-year correlation than groundball BABIP and flyball BABIP (.11 for LD-BABIP, .32 for GBBABIP, and .32 for FBBABIP), so we should expect Soto’s inflated BABIP on line drives to regress to the mean. If Soto had a league average LD-BABIP, he would have had a BABIP of .313. Comparing him to his most recent top comparable, Chris Shelton, shows that Shelton has a career .334 BABIP but due to a high line drive rate of 23.1% (Soto’s is 20.5%). Shelton does not have the high BABIP on line drives that Soto had last year (Shelton’s was .746 for the comparable year, and Soto’s was .805). PECOTA doesn’t know Soto and Shelton differ in this way, which explains why it thought Soto’s BABIP luck would stick when it was much more likely to disappear this year.

  2. Jeff Francoeur and infield fly rate:

    PECOTA‘s projected BABIP for Francoeur: .307
    My projected BABIP for Francoeur: .286
    Current 2009 BABIP for Francoeur: .271

    Franceour’s most recent comparables (the ones for whom there are infield fly data) are Paul Konerko and Torii Hunter. They have 11.9% and 13.9% career infield fly rates, respectively. Francoeur’s is 15.4%, which is far higher than the MLB average of about 10% and both Konerko’s and Hunter’s. PECOTA is right to compare him to players with above average infield fly rates, but even those players’ rates are not as high as Francoeur’s. It is extremely difficult to have a .307 BABIP when you hit so many infield flies, since they are so easy to catch. This means that you can scale back his projected average and his runs and RBIs as well.

  3. Luis Castillo and groundball rate

    PECOTA‘s projected BABIP for Castillo: .300
    My projected BABIP for Castillo: .324
    Current 2009 BABIP for Castillo: .322

    Castillo is an excellent example of the importance of groundball rate. For his career, Castillo’s groundball rate is 64%. That’s probably about 22% more grounders and 22% fewer flyballs than the average player. Statistically, groundballs are hits about 10% more than flyballs. This translates to roughly 2.2% more hits per balls in play, or .022 of BABIP. Castillo’s top two comparables are Mark McLemore and Tom Herr, who had 47% and 49% career groundball rates respectively but not 64% like Castillo. A higher groundball rate than his comparables means Castillo should outperform his PECOTA BABIP projection this year.

  4. Michael Young and line drive rate

    PECOTA‘s projected BABIP for Young: .323
    My projected BABIP for Young: .342
    Current 2009 BABIP for Young: .366

    Michael Young has a career line drive rate of 25.1% (MLB average is 20%). Looking at his comparables, we see that they do not have similar line drive rates. Jeff Cirillo‘s is 19.3%, Edgar Renteria‘s is 22.7%, and Mark Grudzielanek‘s is 23.6%. Having 5% more line drives than the average hitter, holding everything else constant, is going to lead to a .030 higher BABIP. Line drive rate has less persistence than most people think. It only has a correlation of about .17. Groundball/flyball ratio has a correlation of .77. However, if certain players are line drive rate standouts every year, then unless their comparables are too, PECOTA will underestimate them as they have with Young.


TAKEAWAYS

PECOTA projections are some of the best tools fantasy baseball players can use. But you can combine PECOTA and a little bit of extra knowledge to beat competitors that rely on PECOTA alone. The key is to know how the system is created and how its deficiencies can be exploited.

For hitters, there are a few key things to look for that can help you identify overestimated and underestimated projections.

  • Check baseball-reference.com to see whether a young player’s BABIP was high due to high BABIP on line drives or high BABIP on groundballs or flyballs. If a player’s BABIP on line drives was significantly different than .720, then he may be due for a regression to the mean, and PECOTA may not know this.

  • PECOTA doesn’t know a player’s infield fly rate. Go to fangraphs.com, and check it out. Average infield fly rate is about 10%. If his recent comparables have very different infield fly rates than him, that’s a sign his projection might be off.

  • PECOTA doesn’t know a player’s groundball rate. Check out if there is a major difference between his groundball rate and his comparables using fangraphs.com (or calculate it from baseball-reference.com).

  • PECOTA doesn’t know a player’s line drive rate. If a player consistently puts up high line drive rates and his comparables don’t, then PECOTA will probably underestimate him.

The Steve Martin quote at the beginning says you can be a millionaire and never pay taxes, but the catch is that you first need a million dollars. PECOTA is your million dollars in this case, and you already have it. But, PECOTA has taxes of its own-shortcomings due to limited information. With the tips here, you won’t need to pay them.

Thank you for reading

This is a free article. If you enjoyed it, consider subscribing to Baseball Prospectus. Subscriptions support ongoing public baseball research and analysis in an increasingly proprietary environment.

Subscribe now
You need to be logged in to comment. Login or Subscribe
wcarroll
5/31
I'll admit I didn't like the first half of this aside from the Steve Martin quote. It seemed like he was just attacking PECOTA and I always flinch a bit when that happens. Instead, he tries to fill in the gaps. I'd love to know what Nate Silver thought of this, because I know there's a ton of factors already in the PECOTA stew. He makes a convincing case with well-selected and explained examples. Very solid piece of work again from Matt.
ckahrl
5/31
I found this nothing short of inspired as a piece of work, in no small part because it was informative and expansive instead of reductionist and formulaic. Encouraging a fan, reader, or fantasy GM to think beyond the virtues of a single tool is everything any of us in the industry should be all about. Bravo, Matt, really very remarkable.
kgoldstein
5/31
I'm not sure I felt like he was attacking PECOTA, but I'm with you Will, in that I liked what he did with it -- there's ALWAYS more information that can be valuable, and this is a very good example.
fsumatthunter
5/31
Easy thumbs up for me. Excellent writing, useful information, and pushing readers in new exciting directions. I don't think he was attacking PECOTA at all.
BurrRutledge
6/01
How to win at BP Idol: a) Catchy intro. b) Great explanation and insights for the assigned topic. c) Sweet conclusion. Rinse, repeat.
tkniker
5/31
Once again, one of my favorite articles of the week.

Matt is becoming the first article I click on each week.
jrmayne
5/31
As someone whose failed entry in the contest was improving on PECOTA with available data, I gotta like this article.

The use of the data available is a huge boon to fantasy players. Systems which don't use this data can be improved by using it. I thought this was a great piece, and I don't understand Will's visceral reaction to improving PECOTA.
markpadden
5/31
This article is great. The author needs to be hired immediately. PECOTA devleopment is being neglected to the point where it is nearing on a slippery slope to obsolesence -- regardless of what the BP brass may believe. It desperately needs someone to look at ways to improve accuracy, preferably from an outsider.

Other factors I feel PECOTA is ignoring/undervaluing at its own peril:

1) Using pitch type/velocity/movement data to find player comps. more accurately.

2) Grading past performance as reliever vs. as starter on different scales (and doing the work to find out exactly what that adjustment should be).

3) Using strength of schedule. No credible team performance prediction system would ever consider making the assumption that all opponents are league-average. Why would PECOTA? Sure, it's more difficult; but it cannot really be that hard to look at the quality of opponents face for each season.

4) Dealing with platoon advantage more fairly. This ties in with #3, as players who are used abnormally more against benefically-sided opponents (LHPs facing mostly LHBs, or LHBs facing only RHPs) will get inflated rate stats. This needs to be accounted for in the schedule difficulty calc., so that a situational lefty who posts a 3.00 eqERA will be penalized to reflect what he would have posted vs. a normal mix of batters.

Apologies if any of these topics have been fully addressed already in the latest PECOTA. But to my knowledge, they have not; and I am not getting the feeling that anyone at BP is committed to improving the algorithm as aggressively as it needs to be done.
jrmayne
5/31
evo34:

Mine was on the pitch data. You can massively improve projections with it. I have, and I feel like I'm just scratching the surface.

--JRM
tricky1
5/31
Criticism #4 was addressed in PECOTA two years ago, and I'm pretty sure that #s 2 and 3 were addressed prior to that. Criticism #1 is interesting, but there are likely issues of having a sufficient body of data for the pre-PitchFX era. While we sometimes lag in updating some of our metrics (most often where the original researchers have gone on to bigger and better things, like Woolner and Click), it's pretty unfair to accuse BP of not being "aggressive enough" in improving PECOTA, given that there have been improvements made to the system just about every off-season since it was first introduced.

Still, I agree with you that Matt did a great job--for my money, the best article of the round.
markpadden
6/02
The fact that you are "pretty sure" something has been addressed in the past more or less proves my point. No one appears to be taking the lead with driving PECOTA to become better as aggressively/swiftly as possible. We get daily updates on injuries and transactions [sorry, not everyone is absorbed with such topics], but an article on PECOTA research (or related topics) appears maybe once or twice a year.

If you find them, please point me to where ideas #2 and #3 were addressed -- as well as #4. And more generally, please be more transparent with what exactly PECOTA uses as factors and how it uses them. The secret's been out of the bag for a while now. It's time to actively solicit ideas and tweaks from your readership (after first explaining to them what you are currently doing).

tricky1
6/02
The only thing my "pretty sure" proves is that I didn't have my Baseball Prospectus annuals on hand when I replied to your earlier comment. Starter/Reliever adjustment is addressed specifically in Baseball Prospectus 2006, in the front of the book chapter "rearranging PECOTA." Nate's research on these adjustments can be found in various articles on this site--just search for the name Papelbon in the articles that he's authored, and you'll get some of them. Strength of Schedule has been a factor since 2004, as referenced in that year's season preview. The use of a platoon adjustment in PECOTA, which takes into account changes in role and the expected percentage of lefties a team is projected to face, is discussed in the Statistical Introduction to Baseball Prospectus 2008.

The fact that two of your concerns were addressed three and five years ago, respectively, should serve as evidence that Nate has been driving to make PECOTA better pretty darn aggressively. Your other complaints are somewhat ironic, given that you're commenting on a BP Idol thread--a contest where we've "actively solicit[ed] ideas" from our readers--and to an article that shows that readers who have done their homework have great tweaks to offer.
markpadden
6/02
Tell me something: who is currently managing PECOTA development at BP? I did not read the book this year [one less tree family shattered], but I am assuming from Nate Silver's absence on the site that it is no longer him, and from your cursory knowledge of the inner workings of the system that it is not you.

I do not have the 2006 book in storage, so I cannot check the specific adjustments to PECOTA you are referring to -- again, would be nice to have a detailed explanation avail. to subscribers in which the improvements/revisions made over the years were documented.

When I say "strength of schedule," I mean the quality of opponent and environment (park) each player actually experienced on a per-AB basis -- not a generic "the player played in this division, so we adjust him by this amount." E.g., if a Padre miraculously managed to play all 30 of his season PAs at Coors Field, your system would not account for this (to my knowledge). Furthermore, the link you quote refers to incorporating SoS in team win projections, which is not the topic of this discussion. [BTW, Silver's article *manually* applied SoS factors to the team projections in this article].

I.e., you have the data to know exactly whom each player was facing and where it was for every plate appearance. Why not use this to normalize the difficulty (for both pitcher and batter) of each PA? I recognize it is not a trivial procedure; but certainly something you should at the very least be in the process of developing presently.

As for handedness adjustments, the "Platoon Splits" section you refer to from BP 2008 is quite vague. It says that PECOTA now tries to estimate the handedness mix of the opponents a player is "likely" to face for the upcoming season, and adjusts the raw projections accordingly. It does not say that it evaluates on an AB-by-AB basis what handedness mix the player has faced for the past three years and how that is used to adjust his historical eq-Stats, if at all. Again, it's a matter of looking at what the specific difficulty of each PA (including handedness) and averaging all PA to come up with an adjustment, rather than making division-wide or league-wide assumptions as to the difficulty a player has faced.

I am certain you will (try to) correct me if I wrong. I am not a PECOTA historian, but rather someone who wants to see the most accurate baseball projection system possible. That is my sole motive. I am not sure what your motive is, and you appear more interested in playing defense than in looking for new ideas. And BP Idol clearly is *not* a solicitation of new ideas to improve PECOTA. It is a solicitation for articles and future employees. There is a big difference.

Oleoay
6/02
FYI, BP seems to do revision and/or enhancements at least somewhat often. This year's annual indicated that there were some fixes made to VORP. The year before that, EqBRR (whatever the baserunning stat abbreviation is) was introduced. The problem is that since not many people outside BP know how PECOTA works, we don't know when they update it either.

Also note that in this competition, the judges have looked favorably on what Matt did in regards to PECOTA and have praised other finalists for using non-BP metrics and concepts (assuming they presented them in a good fashion).
markpadden
6/03
Definitely there have been improvements every year, but they tend to be rather incremental compared to the ambitiousness of the original concept of PECOTA, and, frankly compared to the increasing ambitiousness of other baseball analysts in the past few years. It's disheartening to see people like Jacques act so defensive and then shoot down new ideas without even doing enough homework to determine if they have in fact already been implemented as described. [Note: based on the links he sent, none of what I suggest is already incorporated in player cards]. This attitude of "why are criticizing anything?; we already addressed those issues long ago...at least I think we did" simply cannot help PECOTA grow and improve.
tricky1
6/04
Mark, I know a few people, like Cory Schwartz, who think highly of your research, so when you initially posted a list of four improvements you thought you could make to the system, I thought you might actually want to know that three of your very general suggestions indeed had been previously implemented by Nate. I didn't do that to "shoot down" your ideas or defend the status quo, I did it since you seemed genuinely unaware of the improvements that had been made to the system, particularly ones that were announced in the BP Annuals.

Sadly, your offer of "Apologies if any of these topics have been fully addressed" seems to have been disingenuous, as your increasingly belligerent replies have shown. I am heartened, however, that you finally admitted that there have been improvements to PECOTA every year, even if you're irked that those improvements weren't done exactly to your taste. The only thing I objected to was the implication that the good people in charge of PECOTA were not diligent in their work. As Matt showed above, it's possible to suggest improvements to a system without denigrating the efforts of those who have and continue to work hard to improve it.
Oleoay
6/04
By definition, an original concept will be more ambitious then an improvement. Also, by definition, an improvement is incremental.

But here's the thing. I've attended one SABR convention in my life, I mainly browse here, CNNSI and ESPN. I don't really know what other analysts are out there and what kind of work they are doing. But I read the Foreword of the BP Annual and the essays in back and see the changes or attempts at changes being made. If me, with my lack of a background, read about changes to BP's metrics in their publication essays/articles about what changes they have tried to make, then how can you criticize them when you didn't do your "homework"? Now, we can debate their success or failure at modeling and properly implementing those changes, but you seem to act like BP is the medieval Catholic Church resistant to input, change and inclined to dogmatic thinking. I'll say this, though... I've never had a complaint about the responsiveness of BP's authors (though Christina tends to get real busy and can take awhile sometimes, but she's an editor so she probably does a lot of responding), even to my silliest questions. I'm sure if you did your homework and presented a well-founded innovation or change that PECOTA needed, they'd listen.

Hey, I'd love it if they published the minutae of how PECOTA works, but it's their proprietary information and I'll have to live with it just like I live with the idea that I drink Coca-Cola without any idea of cocaine's still in their syrup recipe or not. And if I don't like the taste of Pepsi, it doesn't matter what's in the formula anyway. Moral of the story is if you don't like PECOTA, you can suggest changes or you can just not use it.
JayhawkBill
5/31
Excellent work.

I enjoy reading articles from those who know what stats do tell and don't tell. I liked the article. More importantly, I now know that Matt is smart.

Thumbs up, with credibility in future rounds.
hhbliss
5/31
Great analysis, but the writing (including basic things like grammar and punctuation) could use some work. In that sense, you would have been a perfect fit at BP three or four years ago, when the annuals were littered with typos. I hope you keep working at it, because I am looking forward to reading your work for a long time.
BParlette
5/31
In my opinion, this was best article in the entire BP Idol series so far. Original, thoughtful, well-written, and entertaining.
PaulieNeu
5/31
Excellent again. Also love the format and the easy breakout with takeaway points, definitely think that will be helpful when I don't have a chance to sit and really analyze the article in depth while at the office.
maxpalmer
5/31
I thought this piece was great. The concrete takeaways at the end were exactly what I was looking for after the excellent discussion of PECOTA and the player examples. I especially liked this piece because the subject matter and tips are not season or player dependent - they will be just as useful next year as they are now.
daiheide
5/31
Giving credence to statistical measures developed by other organizations - like FIP and wOBA - is not something BP should shy away from. Likewise, a careful look at the strengths and weaknesses of PECOTA is something BP should embrace. I can't understand how that could count as a negative. Great piece.
SkyKing162
5/31
Hear hear.

Recognizing FIP is important, but since it's not the best thing out there, BPro authors shouldn't feel a need to play to it, other than the fact that's it's pretty damn popular. tRA, from statcorner, should probably get more play everywhere, and BPro's own Quick ERA deserves more mention.

wOBA is a bit more accurate than EqA, although we can quibble about more accurate vs. don't sweat it.
wcarroll
6/01
Where can I find this tRA, Sky?

I'll admit I don't keep up with stats, but a couple of the initial entries talked about FIP, so I went to Fangraphs and tried to get a grasp on it. As with many stats, I was left with a feeling of "Umm, so tell me smart guy, what's the result?" since that part's what I care about.
SkyKing162
6/01
tRA is available at statcorner.com

A great no-numbers explanation is available here (among other places, I don't think it appears at their site): http://www.beyondtheboxscore.com/2008/11/9/657217/tra-explained-sans-numbers

The explanation WITH numbers appears here: http://statcorner.com/tRAabout.html (also a glossary at the site)

What do you mean, what's the result? Like, what does FIP tell you and where would you use it? Remember DIPS? FIP is DIPS but with a really quick and easy formula.
llewdor
6/01
tRA is probably the best one-number pitching stat available today. It is to pitching the revolution that EqA was to hitting. It's incredible stuff.
wcarroll
6/01
I didn't count it as a negative. I said it started out feeling like it was going to, but then he took it someplace very useful. I think this is a brilliant article and if Matt ends up winning, I know where he can be put to use this winter.
bsolow
6/01
Considering that Matt's coming from the Penn econ program, he's probably got some serious math chops also and that would be a really useful place for him.

Maybe it's just because I'm also studying economics, but (despite the two game theory pieces not really containing anything ground-breaking for me) Matt seems to be at a level of creativity in his analysis that's far ahead of the others. Maybe when I finish reading the articles this week I'll disagree, but I've been largely disappointed by the other 4 articles I've read so far this week. This, though, was good.
jtrichey
5/31
Excellent and an easy thumbs up. BP, hire this guy right now to be your fantasy analyst. This article has it all. A set up, an explanation for why PECOTA can be tweaked, great examples, and a highly satisfying conclusion. All while being easy to read and yet allowing the reader to go exploring further. Loved it.
mattseward
5/31
I don't often read fantasy columns by many writers despite the fact I love fantasy because essentially I find i'm often being told the same things. This article while perhaps a little bit too stat heavy in the middle makes some excellent points and taught me something new so like the others a big thumbs up.

gwguest
5/31
The concept of article was the best in my opinion. It stood out to me because of the amount I could take away in the end. It also has a very BP feel to it. It was especially interesting to me because I let PECOTA factor very heavily in my fantasy baseball team decisions.

This was one of the last ones I read and one of the only that has something useful to say. I would read - and be willing to pay for - more writing like this.

roughcarrigan
5/31
Terrific piece. As someone who doesn't play fantasy baseball at all this was an article that, if I ever did, seems like it could be extremely useful.
Brecken
5/31
Bset insight I've seen. Must have killed him to share it.
metty5
5/31
The perfect do it yourself article. The information was great, and the analysis was better.

What I loved wasn't that your droned on and on, but you took very specific examples and detailed the analysis. Each detail had a purpose and in the end of the piece, each reader understands why batted ball data is important, and how to make these determinations for themselves.

Perfect.
braden23
5/31
My style has been to combine PECOTA, Bill James and Ron Shandler's projections. If all three models are saying breakout, I feel I am on to something. PECOTA is an awesome tool, and Matt gave a great breakdown on how to utilize it. Tremendous article.
Oleoay
5/31
I still wonder if anyone besides Nate actually knows PECOTA, so I started reading this wondering how Matt knew what PECOTA was and wasn't factoring in. Yet I found it interesting and kept reading it. Once it got to the list of players, I wondered if the players mentioned had been cherry-picked a bit and if the writing would tie it all in to fantasy or not. I think the Francouer analysis was the best of the four and did its job in tying the article into fantasy applications. And then...

The takeaway section was golden! Regardless of whether I agree with Matt's assumptions and whether I thought he was cherry-picking players that happened to meet his projection system or not, he provided a great concise summary of how PECOTA tends to operate and how players tend to profile in PECOTA, then gives a short-hand list of indicators to look for that can affect the valuation of your players in a fantasy draft.

Matt, you started out good and keep getting better and better... thumbs up!
rjblakel
5/31
I also felt the players selected may have been cherry-picked to serve the article's thesis, but I concur with everyone else.....best article so far this week.
swartzm
6/01
Yeah, I definitely cherry-picked the players that proved my point well, but the list of player projections on the website I linked to has my BABIP projections which have a .45 correlation with actual 2009 BABIP so far (as of Thursday when I sent in the article), and PECOTA is at .42. So obviously I picked players that explained my point well, left out players who I projected intelligently but had some bad luck, as well as left out some players that PECOTA just was bound to do a much better job than me with-- players for whom aging is playing a large role specifically. I also left out a few examples of strategies that I just couldn't find a player that matched well with, too.

What's funny is that I was looking for an example of a player who's groundball rate was higher than his comparables, and settled on Luis Castillo despite the fact that his mid-week BABIP was closer to PECOTA's projection than to mine. It just was a good example. But then he had a good week and regressed towards what I suspect is his true BABIP skill. It's funny because I'm a huge Phillies fan, and naturally root hard against the Mets all the time, and I had this weird conflict of interest rooting for a player to help my individual goals but simultaneously against him to help my hometown team's goals...I guess that really DOES make this fantasy baseball week at BP Idol!
Oleoay
6/01
Any time an example is used, it's cherry-picking... but what mattered most to me was the exemplary "Takeaway section". It allowed me to take your rules of thumb and apply them to players that I cherry-pick out and compare. And, just as well, it tied into fantasy baseball and the valuation of players. Without such a clearly delineated conclusion, your examples would have less support. It'll be neat to see some of those other strategies you mention in the future if you also provide us with the applications of those strategies as well as you did in this article.

And yeah, you know you are a fantasy baseball player when you try to figure out how your real life team can win while also allowing your fantasy league team to rack up stats. :)
fsumatthunter
6/01
I absolutely agree with this.

Although I will say that I live by the idea that "my Marlins are more important that my fantasy Marlins" if you know what I'm saying.

If not, I'd ALWAYS rather my real team succeed.
rbross
5/31
This is excellent. One of the best articles so far. It's too bad Nate Silver doesn't write regularly for BP anymore, or else we could really witness a great conversation here. Nonetheless, I like what Matt has to say. Thumbs up.
kcwilson
5/31
My first thumbs up of the tournament. Finally a non-regurgiatory piece.
DLegler21
5/31
I gave Matt an on the fence thumbs up last week - no such fence-sitting needed with this outstanding piece. Minor quibble with the line drive rate persistence (you say it doesn't correlate well year to year then use Young as an example based on historical line drive rate) but overall, the best I've read this round.
swartzm
6/01
Correct-- line drive rate has very poor persistence. There are not many players who can be trusted to be consistently above average at generating line drives. To find one, you need years of consistency at doing so. Michael Young is an example of this since he's been doing it a long time. Finding a 26 year old second or third year player with a high line drive rate the year before is not a trustworthy pick, but Michael Young is.
blcartwright
6/01
also remember Michael Young has the advantage of the hightest line drive park factor in the majors
swartzm
6/01
Interesting. I thought I heard that once, but then I looked on fangraphs.com and the Rangers' line drive rate seemed to be around 19-21% in every year but 2007, and I figured maybe I'd heard wrong or fangraphs normalizes line drive rate? Do you know if that's true?

Is it scorer's error, or is it that the stadium has small foul ground and depresses flyballs as a result?
blcartwright
6/02
I don't know if FanGraphs normalizes their LD%. I did an article there in January about line drives, and Baseball Analysts later did an article on Young that linked back to mine. There was some discussion on 'The Book' blog as well.

Responding to the BA article, I listed Rangers batters since beginning in 2003, and I think only one had more LDs on the road than at home. I still lean towards scorers bias, but it's probably a combination. I'm doing research on pitching at altitude, I can see if LD rate has any correlation.

When I compared parks I used LD% = LD/(LD+OFF). Smaller or larger foul territory should affect line drives and outfield flies equally, so that the ratio won't change.
Oleoay
6/02
How is a line drive defined? Is a Texas Leaguer (a one hopper past the infield) considered a liner?
joelefkowitz
6/01
Best Article of the week. easy thumbs up.
caprio84
6/01
Great article...
momansf
6/01
By far the best. If you don't win this competition then I don't know what other subscribers are thinking. You crush every week.
DrDave
6/01
The title caused me to read this one first. I'll be stunned if anything else beats it this week. This is EXACTLY the kind of thing a BP article should be doing:

1. Explain the state of the art
2. Explain how newly-available data extends the state of the art
3. Explain what you give up when you need to be able to look at a big chunk of baseball history on a uniform basis
4. Give people something they can *use*, now, to understand better or school their co-workers (or both).

I'd give this one 7 thumbs up if they'd let me.
greensox
6/01
I liked the article.
I wouldn't see a problem, however, if he had critiqued PECOTA more. He seemed to go out of his way to assure us that its weaknesses were not really the fault of the analysis - just data limitations to the record keeping 50 years ago.
I enjoy this site a lot, but a weakness is that the conclusions of tools such as PECOTA are seen as facts instead of theories or opinions. And any divergencies of outcome from projections are too easily called "luck".
Nothing wrong with introspection, challenging conventional notions, and looking to improve.
This article takes what is a useful tool and tries to improve it.
Well done.
myshkin
6/01
Matt, how does your method differ from those surveyed at ?
myshkin
6/01
Hm, that didn't work. The link I intended to include above is http://www.hardballtimes.com/main/fantasy/article/whats-the-best-babip-estimator/.
swartzm
6/01
The original article by Bendix and Dutton was used to estimate BABIP in the same year, to beat the old LD%+.120 rule for same year BABIP. It wasn't meant to be predictive. It had some very good insight, but a few mistakes.

I published this:

http://www.thegoodphight.com/2009/1/16/726379/babip-projection-and-new-s

a little while later that I had been working on a while on
how to do predictive BABIP by batted ball type-- I had a formula for BABIP on line drives, groundballs, and flyballs individually. I introduced infield fly rate as the largest correlate for flyball BABIP, and homerun rate as the largest correlate for line drive BABIP, and infield hit rate as the largest correlate for groundball BABIP.

A couple weeks later, this article came out to do a predcitve model using infield fly rate, homeruns per flyball, and an improved term to proxy for infield hit rate using handedness and groundball rate. The model was quite predictive, using only one year of data. However, it was a regression developed using coefficients directly aimed to match their dataset, so it was bound to be a little more predictive than Marcel, which it was.

I wrote a few subsequent articles

http://www.thegoodphight.com/2009/2/2/743228/improving-babip-estimation

and

http://statspeak.net/2009/02/babip-projection-batted-ball-types-and-interaction-terms.html

putting all the components together now that I felt I had a better grasp from my previous article, and then the article that I linked to in this article. By breaking it down using correlates of individual batted ball types' BABIP or their BABIP directly, I get a more predictive result with higher R^2.

Their article was definitely a strong one and one I cited in several articles on BABIP since it got a few things out there about BABIP. One statistic I didn't have access to when doing this article was what they call "spray" which measures how well a hitter spreads the ball around the field (as opposed to pulls the ball more often). That's a very useful statistic and one that would have improved the model for players without several recent years of data. It wasn't as useful if you had at least three years of major league experience with 300 PA or more, when I manually recorded it on a smaller sample size, and suspect it's more useful for second and maybe third year players.
Oleoay
6/01
I don't understand the part where you say "wasn't as useful if you had at least three years of major league experience with 300 PA or more, when I manually recorded it on a smaller sample size, and suspect it's more useful for second and maybe third year players." Don't you mean it was less useful because of the sample size issues?
Oleoay
6/01
Er, I mean... are you saying it was less useful for three year plus players than those who hadn't completed their second year? Seems like the third year would give more data...
swartzm
6/01
Actually, I'm pretty sure the sample size was fine-- I just mentioned that because I didn't have that data available for the study linked in the article, so I can't rule out its usefulness. Sorry that was unclear. I should have explained in more detail, because it's not a trivial point at all. I wrote an article discussing this topic of peripheral statistics a few months ago:

http://statspeak.net/2009/03/skills-repeatability-and-peripherals.html

In that article, I explained that there is certain information that can be obtained from more peripheral data which can be useful at predicting more typical statistics. A leading example in that article is strikeout rate and "contact rate" which is the percentage of pitches that a hitter does not miss when he swings. If you have only the previous years' strikeout rate, you can do an okay job of predicting a player's strikeout rate the following year. However, if you also know his contact rate (and hence how much he misses when he swings), you can predict it a little more accurately. If you know he also swings and misses more than other people with the same strikeout rate the year before, you know is he more likely to strikeout more the subsequent year. However, let's say you also know his strikeout rate from the year before that. That's actually more useful, and contact rate from the previous year is no longer very useful at all to predict strikeout rate. Simply, two years of direct strikeout rate data is better than one year of strikeout rate data combined with contact rate data from the same year. These scouting type peripheral statistics (like contact rate) are useful in filling in the gaps when you would like to have more useful information.

Consider the following as an easy example. You're considering signing Mark Teixeira or trading for Matt Weiters. Which move is the sabermetrician more useful with, and which move is the scout more useful with (at least relatively)? With Teixeira, he has years of data and he's young. To know how well he'll age, what his true ability level is, and how he'll play in your stadium, you value the sabermetrician. To know how Weiters will adjust to major league pitching, what his vulnerabilities are as a hitter, etc., you value the scout. Obviously it's better to have both, but relatively scouts are relatively more valuable with less statistical data available, and sabermetricians are relatively more valuable with more statistical data.

This leads into my discussion of spray rate. Knowing how well a hitter spreads the ball around the field is very useful for second year players. Ryan Howard has a ridiculous BABIP his first year and a half in the majors, but he pulls most of his balls in play (though surprisingly, not most of his homeruns). Knowing that he does not spray the ball around the field well would have been useful to predict his fall in BABIP during his second and third full seasons. However, knowing that information now is less valuable. In three and a half years of data, the effect of his tendency to pull the ball is reflected in his lower BABIP, specifically his lower BABIP on groundballs.

This effect seemed to hold true in analysis I ran when I had smaller samples than in this article. Using "spray" was useful in predicting BABIP for second year players, because there was already a lot of noise in their first year BABIP and a lot of adjustments that defenses would make as there was more information on the guy. Using "spray" was not useful in predicting BABIP for players with at least three previous years of BABIP because the effect of how frequently they pulled the ball was fully contained in the three years of BABIP data.
Oleoay
6/01
Ok, I follow the Teixeira example and I understand the part about Ryan Howard... the confusion comes from a section like this "However, let's say you also know his strikeout rate from the year before that. That's actually more useful, and contact rate from the previous year is no longer very useful at all to predict strikeout rate." The sentence suggests that information from two years ago is more predictive than information from last year... or am I reading it wrong? Are you saying last year's strikeout data + last year's contact rate is not as useful as two years of strikeout data?
swartzm
6/01
Sorry about that. Okay, let's say you want to predict strikeout rate in year "T" (call this K(T)), and you know only K(T-1) (i.e., strikeout rate of the previous year). That is useful but only so much. You can improve that if you also know contact rate in year "T-1" (call this C(T-1)). Using C(T-1) and K(T-1) to predict K(T) is better than just K(T-1) to predict K(T).

However, if you know K(T-2), that's more useful than C(T-1). In other words, if you try to predict K(T) using K(T-1) and K(T-2), you'll get more useful information than using K(T-1) and C(T-1). In fact, if you have K(T-2), K(T-1), and C(T-1), then you don't really even need C(T-1) at all.
Oleoay
6/01
Ok I understand better now what you were trying to say, thanks for the explanation. I do find it a bit interesting that the contact rate from last year C(T-1) doesn't help add some bit of precision even when there is knowledge of K(T-2).

I would guess to rephrase this is a bit that if C(T-1) has some of the aspects as BABIP and BABIP can fluctuate to an extent, then K/AB is a better predicter.

Fair statement?
swartzm
6/01
I was actually talking about predicting strikeout rate using contact rate in that statement. Contact rate does actually remain useful in predicting BABIP only because BABIP is noisier than K/AB. Did I understand you correctly or am I missing something?
Oleoay
6/01
You kind of did, I was actually taking things in a different direction. I was just wondering if the same concepts you used for batters also applied for pitchers, if strikeout rate is basically k/9 or not. I also find it interesting what aspects of the batter/pitcher relationship differ, such as BABIP which suggest that the batter mostly controls that.

*grumbles* I need some caffeine.
Oleoay
6/01
Yeah that was a muddled comment.

My thought was if strikeout rate is an indicator of future strikeout rate for batters, how well does strikeout rate correlate as a future indicator for pitchers? Also, if given only one year of a rookie pitcher's data (or, a rookie reliever), can something like BABIP, H/9 or some other form of "pitcher's contact rate" be constructive in projecting strikeout rates?
swartzm
6/01
Pitcher's contact rate shows a similar trend but it isn't really coming up all that significant even with one year. The point about using peripherals like contact rate to supplement one previous year of data but not bothering to use them to supplement two previous years of data seems to more or less hold, though.
Olinkapo
6/01
Little to add here, but had to say: Excellent.
psugator01
6/01
this article is so good that i feel compelled to delve into these intricate stats that i've been trying to avoid for so long. this takes sabermetrics even further. best article this round. just glad this guy isn't in my fantasy league.
timoseppa
6/01
I found much to like and dislike in this article. My biggest qualm: "It goes through a hundred years worth of baseball players" - That is not true and is a basic fact that should have been checked. From the BP Glossary: "PECOTA compares each player against a database of roughly 20,000 major league batter seasons since World War II." That would be 64 years. As good as the article is, you can't make mistakes like this. Thumbs down.
timoseppa
6/01
63, that is.
EJSeidman
6/01
So your number was off as well? Kind of ironic.
eligieryna
6/02
Thumbs down.
dpowell
6/01
Great point. I found this mistake crippling as well. Matt, your article isn't relevant to my 1913 fantasy league. Please rectify that.

(Great article, Matt.)



GraigNettles
6/01
To steal from Mel Brooks: "May the Swartz be with you" young Matt :-) Maybe because I'm a big believer in BABIP as a performance indicator, I really dug this baby once it got into the examples. And any piece that shows Frenchy as the hacking fraud that he is gets extra kudos from moi. Thumbs up.
llewdor
6/01
This was a surprise. I've already come to expect a certain level from performance from each of the BP Idol contestants, and I'll admit my expections for Matt so far have been pretty low.

But this was great. This is probably the class of the round. If Matt can write articles like this consistently, he needs a regular job somewhere.
crperry13
6/01
Great article. I may bookmark this one for my own personal Fantasy Dominance.

Also, A+ for having the cahones to scrutinize PECOTA. Dangerous thin ice, but you ended up leaving us all with a warm fuzzy feeling after all. Well played.
hotstatrat
6/01
Matt has gone back to being a little overly self-promoting for my taste, but this was easily one of the best articles this week.

What I am left wondering is: how significant are these nuances? We all have time constraints, we want to be able to decide if it is worth the added considerably effort of looking up BABIP on groundballs and line-drives, etc. to achieve this higher degree of accuracy. Is it worth checking for a consistently high line drive rate, but not bothering with the rest of the stuff? Would it be worth it to take lessons in VLOOKUP for this?

How reliable are reported line drive rates in the first place? That is a very subjective statistic, is it not?

I couldn't find BABIP on line-drives, etc, in Baseball-Reference. Could we get a more specific link and/or description of where to look, please?

Re: comments:
Where do you get spray data?

So, in predicting strikeout rates two years of strikeout rates are more significant than the more recent year's strikeout rate plus contact rate. I assume a higher strikeout rate in year-2 means that year-1 may have been fluky, so his strikeout rate should be in-between instead of presenting a trend downward. Correct? I am interested in finding when a trend is real or not.
Oleoay
6/01
He means (assuming I assimilated it right) that past strikeout rate is the most accurate metric for predicting future strikeout rate, then contact rate is the next accurate metric.

Thus, try to use past strikeout rate as much as possible.

But if there's only one year of data because the player is a rookie, there might be sample size issues, so then use strikeout rate in conjunction with contact rate to project future strikeout rates.
swartzm
6/01
Strikeout rate stuff: in general, if you have strikeout rate in year-1 and strikeout rate in year-2, you would expect that strikeout rate in year-3 would probably be between them, but you need to regress to the league average and then adjust for aging. PECOTA and other projection systems are very good at determining aging, and I have not worked on this. In general, I avoid assuming downward trends in a situation such as you described.

To find BABIP by batted ball type: go to a player's statistics on baseball-reference.com. Then you will see a tab for "splits [+]" above his statistics, and if you run your mouse over this, it will let you select the splits in a player's career or a specific year of your choosing. One of the last few splits (I think it's the one before splits by opponent and splits by stadium) is splits by "hit trajectory."

The reported line drives are measured with error, but so are the line drives used in my regression so it's mostly okay...(A little regression background now, but those who are not interested can skip over this: when any independent variables are measured with error, the regression process will bias the coefficients towards zero and away from the accurate values. This is called "attenuation bias" or occasionally "regression dilution." In a situation like this, it's a shame, but there is not much I know how to do to correct for it. There are some methods that are used, but often I've been told it's best to just understand that the effects are understated.)

I can't really tell you how valuable it is to use this stuff. If you have a lot of value on your performance in fantasy baseball, I'm sure this will help. I think how intensely you work on this is a matter of preference. You can certainly sort players by line drive rate, groundball rate, and infield fly rate at fangraphs.com, and then check a few extreme outliers to compare them to their PECOTA comparables. Alternatively, if you're the type of person who picks out which players to target in advance in a draft, it's probably smart to check over the dozen or two dozen guys that you specifically want on your team. It's really up to you.
hotstatrat
6/01
Yes, thank you, Matt. Thanks for your articles and taking the time to respond to us. I found those splits, now.

Of course, it is up to the individual how much time he spends on this. Sorting on those stats and looking for outliers as you suggest sounds worthwhile, but then as you point out one year of line drive BABIP isn't all that significant. I was trying to get a notion of much added improvement these checks would bring to PECOTA.

Ideally, Nate will consider them and test how and whether he should incorporate your ideas.
rubemode
6/01
Others have beat this to death, but count me in the best article of the week club for Matt.
jdavlin
6/02
Easily best of the week. I could care less about fantasy sports in general, so some of this week's entries have been slow going for me. But Matt's put together a wonderful piece of analysis that is convincing and highly readable, regardless of its application to fantasy.
krissbeth
6/02
Interesting that you can essentially write two articles for the price of one with the use of a simple link. I like the article, although I'm not sure how fair that 2 for 1 deal is. In weeks where the competition is closer, it might weigh against you with me.
SkyKing162
6/02
Here's another good example for why winning this contest and being a good addition to the BPro writing staff aren't the same goal. We, the writers, and BPro have a decision to make -- is it better to win and get the guaranteed contract, or come up a bit short, but show the world you have what it takes to succeed and negotiate your own contract (the Daughtry and Aiken route).
Oleoay
6/02
I disagree. It is perfectly possible (and judging from the finalists, quite likely) that whoever wins this contest will also be a good addition to the BPro writing staff. Also, whoever wins the contest can put that on their resume and will find a wider venue for their articles, perhaps in time even leading to a job with a major league team. It can happen for the other finalists too, but it will be harder.

And by definiton, to succeed in the BPro Idol contest, you have to be popular with this audience. If you don't win, you weren't as popular with this audience (for whatever reason), and to a prospective employer, might not be seen as popular with certain similar audiences or niches... thus your ability to negotiate your own contract as well gets diminished somewhat.

Also, the American Idol comparison does not work as well either since the winner of American Idol signs an exclusive contract on content and those terms are not in this competition from what I've seen.
hotstatrat
6/02
Why? I see that as a positive. It certainly isn't negative as we have the option of not reading the linked article.
krissbeth
6/02
Because it's a way around a word count requirement for the article that others abided by far more stringently than this contestant does with an entirely separate article for methodology.
swartzm
6/02
I understand your criticism, but reading the other article is not necessary to grasp this one or follow these rules. The other article was linked for those who want a more detailed method and to establish the credibility of the method. I thought it was foolish for me to throw out tons of research I had slaved over, but I was careful to leave the article self-contained for those who wanted quick tips.
Oleoay
6/02
You're kidding right? Every finalist is citing source material outside of their own article, whether it is a quote from a MLB employee, referencing a statistic like wOBA or BABIP, or a concept like replacement value or park factors. PECOTA, VORP, WARP, etc are all examples of material that are being cited by an article and citing such methods/ideas/definitions is a way to use validated tools to support or refute the thesis of a paper and its evidence.

Using outside material is a strength.
hotstatrat
6/03
Geez, krissbeth, even if it was a "way around the rules" as you see it (and I don't think it is fair to see it that way), what is wrong with that? You are penalizing creativity. It is still within the rules. And, because it reads fine without diving into the link, it is well within the spirit of the rules.
mafrth77
6/02
One thing that I don't get- If it misses high on Francoer's BABIP due to neglecting to account for pop-ups, how come it doesen't have the same effect on Hunter or Konerko?
swartzm
6/02
Well, it's a matter of whether it compares people to comparables with similar infield fly rates are not. Hunter and Konerko probably profile similarly to guys who pop up more than average, so their comparables probably included those type of players. Players similar to Francoeur in terms of their recorded statistics and body type do not generally pop up as much as he does, simply because it's rare that anyone does.
dguretz
6/02
Simply really good. Do not take this the wrong way but I had to break and go back and re-read this without any other articles skewing my opinion of good/bad/indifferent. My first impression was right, this is a fantastic example of tweaking something that is great to fit your needs and give you an edge. Awesome use of research, effective examples, and a decent metaphor to open/close. Easily the best of this week.
frqtflyr
6/02
A pleasure to read a solidly argued piece that takes an iconic method of projection and elaborates upon it. Everything is coherent, Matt refrains from becoming overly geeky (or condescending to the SABR-challenged), the examples illustrate the thesis perfectly, and the structure works (advancing the premise and even returning full circle to the lede). Bravo.