keyboard_arrow_uptop

Sometimes in these pieces, I delve into a long explanation of what I’m going to write and what I’m going to set out to prove. That’s not going to happen today. I’ve got a lot of tables to produce and a tight deadline so I’m just going to dive right into it. Today’s goal is to take a look at the PFM, take a look at expert prices, and determine whether or not I should be using the PFM more as a tool to devise my bid limits or if perhaps I should chuck my bid limits entirely. How’s that for a attention grabbing lead?

Baseball Prospectus fantasy writers get asked about PECOTA a lot. Some subscribers don’t understand why they should bother with my bid limits when the PFM serves the same purpose as my bid limits but with a more mathematical bent. Others find my bid limits interesting but think that the PFM should be featured more as we approach Draft Day.

I talked about the differences between my bid limits and the PFM last year. That linked article was an explanation of why I do things the way I do. It was not an examination of whether or not what I do is worth doing. This piece you are reading will attempt to answer this question.

As I noted in last year’s PFM piece, one thing the PFM does very well is measure how the player pool is going to do on a macro level. What I did not realize is how exceptionally well the PFM does this.

Table 1: 2014 American League Hitter Earnings and Salaries in Groups of 10

Group

$

CBS

LABR

Tout

AVG

PFM

1-10

$258

341

341

337

340

270

11-20

$215

253

250

251

251

218

21-30

$162

216

224

223

221

197

31-40

$182

212

191

194

199

186

41-50

$179

199

188

193

193

175

51-60

$146

144

132

134

137

165

61-70

$162

142

147

163

151

153

71-80

$117

126

133

135

131

140

81-90

$160

123

128

132

128

126

91-100

$138

125

122

114

120

115

101-110

$84

63

61

83

69

105

111-120

$60

58

59

59

59

92

121-130

$59

41

38

55

45

75

131-140

$45

14

27

30

24

51

141-150

$46

18

24

9

17

36

151-160

$11

7

8

4

6

22

161-170

$59

13

13

19

15

11

171-180

$26

5

7

5

6

9

You can quibble with individual hitter projections if you like (and I certainly will as we go along), but on a grand scale the PFM has a much better idea of how these players are going to perform in the future than the experts do. The PFM is $3 off in group 11-20, $4 off in group 31-40, and $4 off in group 41-50. This is consistency is pretty amazing.

The story isn’t any different in the National League.

Table 2: 2014 National League Hitter Earnings and Salaries in Groups of 10

Group

$

CBS

LABR

Tout

AVG

PFM

1-10

$215

325

305

318

316

282

11-20

$229

268

254

261

261

228

21-30

$209

244

227

241

237

210

31-40

$169

218

207

216

214

192

41-50

$142

182

174

187

181

173

51-60

$135

137

152

142

144

162

61-70

$142

131

128

141

133

150

71-80

$152

143

138

137

139

144

81-90

$117

118

129

125

124

128

91-100

$159

93

114

97

101

116

101-110

$57

69

94

83

82

96

111-120

$93

45

52

55

51

73

121-130

$63

26

45

32

34

57

131-140

$94

15

30

25

23

50

141-150

$41

16

34

14

21

36

151-160

$46

13

23

24

20

25

161-170

$26

8

16

15

13

12

171-180

$35

8

10

10

9

6

The earning distribution is different—particularly in the first three groups—but once again, the PFM has reality surrounded with its recommended bids. Granted, it is pretty far off on the top group of hitters, but even the PFM has nothing on the experts.

The PFM’s aversion to the most expensive hitters is starker when you look at it on a case-by-case basis.

Table 3: Top 20 Salaries, 2014 AL and NL Hitters with PFM

Player

$

Sal

+/-

CBS

LABR

Tout

MG

PK

PFM

2013

Mike Trout

$38

46

-8

43

45

49

43

44

38

$41

Miguel Cabrera

$32

43

-11

43

42

45

42

43

33

$42

Jacoby Ellsbury

$31

33

-2

34

35

29

32

31

30

$34

Prince Fielder

$3

33

-30

33

33

32

30

28

24

$23

Robinson Cano

$28

32

-4

34

33

30

33

34

24

$31

Edwin Encarnacion

$25

32

-7

30

34

33

31

29

23

$27

Adam Jones

$29

32

-4

34

33

30

31

29

24

$32

Chris Davis

$12

31

-19

32

31

30

32

30

23

$36

Jason Kipnis

$15

31

-16

26

33

33

31

28

20

$29

Adrian Beltre

$27

30

-3

31

29

29

28

30

24

$29

Andrew McCutchen

$34

38

-4

40

35

38

37

39

28

$38

Paul Goldschmidt

$24

37

-13

41

33

38

36

35

28

$41

Carlos Gonzalez

$9

35

-27

36

37

33

34

37

27

$31

Joey Votto

$6

35

-29

34

32

38

32

33

28

$30

Bryce Harper

$11

33

-22

34

32

33

32

33

19

$22

Ryan Braun

$22

32

-11

30

33

34

33

37

35

$12

Carlos Gomez

$34

32

2

32

32

31

32

30

28

$36

Hanley Ramirez

$21

32

-10

34

31

30

32

32

23

$26

Freddie Freeman

$24

30

-6

32

27

32

27

30

20

$31

Troy Tulowitzki

$23

30

-7

32

28

29

31

31

27

$26

Average

$22

34

-12

34

33

34

33

33

26

$31

For a description of what each column in Table 3 (and all subsequent tables) means, please read the descriptions from my retrospective player valuation series.

In Table 3, PECOTA and the PFM absolutely destroy the expert market. CBS, LABR, Tout Wars, Peter Kreutzer, and I are in lockstep while the PFM almost completely nails what these players are going to do as a group. It comes closer to predicting player earnings in 14 out of 20 cases, while the expert market only beats the PFM four times: on Jones, McCutchen, Braun, and Gomez. There are two ties.

The only top 10 AL- or NL-only hitter who turned a profit last year was Gomez. If you had followed the PFMs advice, you only would have walked away with Braun in the expert leagues. That’s still a loss for the experts as far as I’m concerned.

But (for me at least) this is all old news. The PFM is much better at predicting what the best hitters are going to do because it isn’t betting on any outliers. Nearly every player has some degree of failure built into his projection, which is reflected in the PFM. The experts, put simply, aren’t hedging their bets.

Should they be?

For the PFM to be definitively better than the experts, it has to tell me either when to buy or who to buy. I already know that it isn’t giving me any particular advantage in telling me when to buy, based on the evidence in Tables 1 and 2. In the 10 player groups where the PFM spends more money, the players in the AL earn $757 on an $848 investment. This is canceled out in the NL, where the PFM is more aggressive in groupings that earn $1,125 on $1,049 of recommended PFM bids. On the whole, the PFM’s passivity at the top of the player pool isn’t leading to a windfall of big profits on the players the PFM does like.

So this leaves us with the “who to buy” part of the equation. We kind of know based on Table 3 who the players are that the PFM doesn’t like, but are these the only types of guys that the PFM tells us to avoid or are there others?

Table 4: 20 Least Favorable PFM Projections vs. Expert Bids, 2014

Player

$

Sal

+/-

CBS

LABR

Tout

MG

PFM

PFM +/-

Jose Abreu

$34

24

10

21

24

26

25

12

-12

Jason Kipnis

$15

31

-16

26

33

33

31

20

-11

Miguel Cabrera

$32

43

-11

43

42

45

42

33

-10

Edwin Encarnacion

$25

32

-7

30

34

33

31

23

-9

Evan Longoria

$21

29

-8

29

30

28

27

20

-9

Prince Fielder

$3

33

-29

33

33

32

30

24

-9

Adam Jones

$29

32

-3

34

33

30

31

24

-8

Robinson Cano

$28

32

-4

34

33

30

33

24

-8

Alex Rios

$18

27

-9

27

29

25

27

19

-8

Brandon Moss

$17

19

-2

18

20

19

18

11

-8

Bryce Harper

$11

33

-22

34

32

33

32

19

-14

Allen Craig

$6

24

-18

24

23

25

23

12

-12

Freddie Freeman

$24

30

-6

32

27

32

27

20

-10

Andrew McCutchen

$34

38

-4

40

35

38

37

28

-10

Paul Goldschmidt

$24

37

-13

41

33

38

36

28

-9

Hanley Ramirez

$21

32

-10

34

31

30

32

23

-9

Carlos Gonzalez

$9

35

-26

36

36

33

34

27

-8

Ian Desmond

$27

27

0

28

26

27

28

20

-7

Pedro Alvarez

$13

22

-9

23

20

22

21

15

-7

Joey Votto

$6

35

-29

34

32

38

32

28

-7

Average

$8

4

5

3

3

5

6

12

8

There are a few others. But 12 of the 20 hitters on Table 4 are repeaters from Table 3.

For all of the knocks that PECOTA and the PFM have received over the years, Table 4 should be used in Baseball Prospectus’s publicity materials. Allen Craig at $12, Pedro Alvarez at $15, and Alex Rios at $19? These are the kind of predictions that win you leagues if you take them to heart and avoid these players. Abreu was a big miss, but with the exception of him, Moss, and Desmond, Table 4 is a big win for projection systems and staying conservative.

However, as I noted above we know that the market is too aggressive on the most expensive players. Does the PFM’s dour outlook on the best players translate to success on the players where the market is too soft?

Table 5: 20 Most Favorable PFM Projections vs. Expert Bids, 2014

Player

$

Sal

+/-

CBS

LABR

Tout

MG

PFM

PFM +/-

Dee Gordon

$34

4

30

4

2

7

9

17

13

Jon Jay

$15

5

10

2

9

3

4

16

11

Rajai Davis

$26

10

16

3

13

15

11

20

10

Craig Gentry

$11

2

9

2

1

2

4

11

9

David DeJesus

$5

2

3

1

2

2

8

11

9

Aaron Hicks

$3

1

2

1

2

8

10

9

Marc Krauss

$2

0

1

1

1

9

9

John Mayberry

$3

0

2

1

1

9

9

Omar Infante

$13

9

4

9

10

9

12

17

8

Robbie Grossman

$9

5

4

2

3

9

7

13

8

Raul Ibanez

$1

2

-1

2

2

2

5

10

8

Pedro Florimon

-$1

2

-3

1

1

5

2

10

8

Juan Uribe

$15

5

10

4

4

6

8

12

7

Jose Tabata

$4

5

-1

3

7

5

5

12

7

Darin Ruf

$2

4

-2

4

5

2

3

11

7

Tyler Flowers

$10

2

8

1

2

2

2

9

7

Travis D’Arnaud

$9

9

1

9

8

9

11

15

6

Junior Lake

$5

9

-4

10

9

8

9

15

6

Jordan Schafer

$3

2

1

2

2

2

4

8

6

Tony Campana

$0

1

6

6

Average

8

4

5

3

3

5

6

12

8

The easy takeaway is that the PFM kicks ass and takes names because it is way ahead of the field on Gordon, Jay, Davis, and Uribe. If you bought Gordon, Jay, and Uribe in your NL-only, it probably didn’t matter what the rest of your roster looked like; you cleaned up even if you merely purchased the rest of your offense at par prices.

The problem is that on the whole this group of hitters stinks. Eleven of the 20 hitters on this list earned $5 or less, and only two of the 20 recommended prices in Table 5 are under $8. If the PFM was rightfully conservative with the top-tier hitters, here it looks like an inebriated businessman throwing money around at two in the morning on the Las Vegas strip. The optimistic viewpoint is that the PFM pointed you toward Gordon, Jay, Davis, and Uribe. The negative viewpoint is that if you take out the $90 these four players earned, the remaining 16 players earned a combined $79 and the PFM recommended paying $176 for them.

The PFM’s recommendation on a high level (don’t spend money on any of the top players) is a sound one. But the PFM has to spend its money somewhere, and isn’t all that much better than the market is at figuring out who the best players are.

Table 6: PFM and Expert Market: Cost and Earnings on hitters with +/- $3 differential

Group

# of Hitters

$

CBS

LABR

Tout

AVG

PFM

P

M

T

AL PFM Favorable

51

$431

254

266

309

276

538

26

22

3

NL PFM Favorable

52

$489

326

357

372

352

590

23

29

0

AL Experts Favorable

47

$797

1,027

1,035

1,017

1,026

761

26

19

2

NL Experts Favorable

41

$667

868

864

887

873

647

17

23

1

Totals

191

$2384

2,475

2,522

2,585

2,527

2,536

92

93

6

Table 6 lists the players in both the AL and NL where the PFM had a $3-or-greater differential than the expert market in either direction. The P/M/T columns show where the PFM came closer to the actual player earnings (P), where the market came closer (M), or where there were ties (T). I thought about creating a scorecard for every player but decided against it; a $0-2 difference on a player projection is marginal and it seemed arbitrary to score the PFM or the market on players where the valuation was this close.

In Table 6 we see another aspect to love about the PFM: it takes a stand on its preferences. There were a total of 336 hitters purchased in AL and NL-only leagues combined; the PFM has a $3-or-higher swing on 57 percent of those hitters. Unlike the expert leagues—which hardly have any variance whatsoever—the PFM has guts.

This doesn’t necessarily translate into better projections. The advantage that the PFM gains on the most expensive hitters (AL and NL Experts Favorable) is more than canceled out by the hitters where the PFM takes a stand (AL and NL PFM Favorable). Overall, the amount of money that the PFM recommends spending nearly matches what the expert leagues recommend spending. On a hitter-by-hitter basis, the market narrowly edges out the PFM, 93-92. Put simply, when you get past the most expensive hitters, the PFM’s precision isn’t any better than the market’s guesses.

This isn’t a problem with the top hitters, but for the hitters in the middle and at the bottom the PFM is giving all of its gains away. In a perfect world, the PFM would tell you to stop overspending on the top hitters and point you to the right hitters much more frequently than they do not. But it doesn’t. In fact, when you take the hitters with $30-plus salaries out, the market comes closer to predicting earnings than the PFM does.

So what does this all mean? Should we throw the PFM out because of this?

Of course not. The advice to stay away from the top hitters in auction formats is sound, particularly as we continue to see offense lag behind pitching. The odds of a hitter cracking $40 is very low, and many of the $30-plus salaried hitters are going to lose money. We should stop pushing $30-plus on the elite hitters. The separation isn’t there anymore. Mike Trout and Miguel Cabrera are sinking along with the rest of the pool.

But where the PFM is failing is at the bottom. Part time players don’t have enough of a ceiling to justify $8-10 prices but the PFM continues to insist paying in this range for players like Mayberry. The failure rate on these guys is too high to pay par. If Mayberry disappears, you have budgeted $9 for nothing…and players like Mayberry often do disappear.

The auction market recognizes this and doesn’t pay these guys. But the lesson we should take out of this is not to distribute the money to the top as the expert leagues always do but instead to move it to the middle. A healthy redistribution of your dollars solves both for the problem of the market (overpaying the guys at the top) and the PFMs problem (paying good money for bench players).

How does the PFM do with the pitchers versus the market? I will examine this in Part 2 of this series next week.

You need to be logged in to comment. Login or Subscribe
eliyahu
1/30
This is great -- I really learned something from this piece. Exactly what I've been waiting for you guys to do for a while.
Michael
1/30
Thanks for being willing to both compliment and criticize as appropriate the PFM's performance.
adrock
1/30
I concur. A very solid analysis of PFM's strengths and weaknesses. I've used PFM as a general guide (who to value, who to avoid) for multiple years, but the specific $$ values haven't been close enough to my league's reality to use it for bid limits. Closers in particular have been way out of whack.
dbiester
1/30
Question -- is the PFM overspending at the bottom a result of inflation from not buying at the top? Did you use fpm values assuming a 67% budget allocation to hitting? The whole hitting budget has to be spent by the PFM, so if the top players are getting valued below typical market by the machine, doesn't that automatically mean that the $2 players are going to get a bump? And for the purposes of players in a league, unless you are bidding against someone also using PFM strictly, telling you to spend more at the bottom is useful advice that will likely net you the player without hitting the PFM max. For example, you would not have had to spend the whole $17 on Omar Infante, just more than the market, and likely that would net you a profit.
MikeGianella
1/30
I plugged in a 180/80 split (which is the "standard" setting). This is a 69% budget for hitting. And I did check and can confirm that the PFM does indeed add up (it is spending the $2160 on the top 168 hitters).
dbiester
1/30
Right, so it HAS to spend the money saved on not buying the top hitters. of course in an actual auction you stop bidding once you get the player, so PFM would not bet quite so hammered at the bottom. Basically PFM sets you up to have a late-draft value grab strategy for hitters, which is great if your league is deep enough that the top talent is getting spread out. Personally I have used the PFM values for years to distinguish players within tiers and to make more informed distinctions among the top tier players, but less so as a strict bidding guide.
frampton
1/30
This phenomenon seems to be a necessary function of making a list that added up. As you say, PFM (and anyone making a list) has to put the money it shorts the top players onto the salaries of the middle guys. The $5 to $10 tier on any bid list has guys who, in the auction, go for anywhere from $17 to $1, depending on a range of auction dynamics (the order that players are called, other teams' (ha) poor money management, etc.). It's difficult if not impossible to account for that in a bid list; it looks like most experts are more likely to give that to the best players, and maybe PFM and Mike are right to say it would be better to give it to the tier of hitters just below the stars. In a sense I guess it's self-correcting -- if Goldschmidt's salary starts moving closer to Adrian Gonzalez's, I'll be happy to get Goldschmidt. Lesson one, which we all know: you need to be a little flexible in an auction, fight for value at every roster spot, while spreading risk to the extent possible.
ravenight
1/31
This is exactly the flaw, though - the PFM is being forced to put a $ valuation that adds up to spending all money on the 168 best guys, but when you compare it at the end of the season, a huge number of those guys earn 0 (because they were outside the top 168). I mentioned this last year, but basically PECOTA is making projections and then treating them like end-of-year values, and it shouldn't be. To make a simpler example, let's assume a points league (though I think this still applies, in a more complicated way, to the composite or stat-level valuations in a 5x5). So I have my projection system and it says that on average across all outcomes, player A is going to score 550 points. So, now we look at all the projections and we see that the #169 hitter is projected at 510 points. We go through and add up the difference between the top 168 hitters' projections and 510, and get a grand total of 10800 points above replacement, or 5 points per $ (to keep it simple). So we value player A at (550-510) / 5 = $8. Our system came to its overall projection, though, by projecting a bunch of different possible outcomes, say scores ranging from 450-650, and giving each a likelihood (or a likelihood of being in it's neighborhood). So if we divide up the range from 450-650, perhaps it believes that every 10 point bucket along that range has an equal chance (5%) - it wouldn't be uniform like that in reality, but I don't think the real distribution changes the argument. That means that there's 30% chance our hitter will produce $0 or less (the buckets 450-460, 460-470, etc up to 510), then for each bucket above 510, we are adding $2 of production with a 5% chance, so we have $2 times the sum of all numbers from 1 to 14 (the 14 buckets up to 650) times 5% = $210 * 5% = $10.20. So the player earns an average of $10.20 in our post-season analysis. Now you might say "well, if we let his value go negative we account for this," but that isn't true, because the value of a dollar is set based on this production. If we assumed that literally everyone else in the league exactly nailed their average production (which is, on average, a good assumption), then we are saying that 30% of the time player A is replaced in our list by player #169 (who earns 510 points), and player #170 becomes the new replacement level (let's say he earns 510 also) leaving us with 40 fewer points above replacement, so decreasing the points-per dollar to 4.98 or so, basically distributing player A's $8 across the rest of the top 169 in proportion to their previous price. When he hits that 650 high-end and earns $28, he is pulling that extra $20 out of all the other players. So his effect on prices is only proportional to his performance when his performance lands him in the top 168 - the rest of the time it doesn't matter if he earns 508 points or 8 points. So this is how the PFM should do the calculations - it should calculate the averages, then for each player it should take a set of projections, and calculate the $ value of that projection against the weighted averages for the rest of the league (but with a min of $0) and then multiply the projected $ values by the weights for the projections to come up with a weighted average $ value. This will give positive value to a whole bunch of guys beyond the top $168, which is accurate (those guys have some chance of being worth paying for). I won't be possible for your league to spend its money that way (unless you have really deep benches), but what it will do is give you an accurate sense of how much someone is projected to earn. It will still miss on some players, but there is no reason for it to miss so wildly on aggregate groups when the stat projections themselves are accurate.
pigbird
2/04
I don't know the answer, but I think these are excellent points. I wonder if some of the problem is solved by allocating more $$ to pitching. As a related but also entirely separate point, I question the wisdom of the 67% allocation. I'm thinking that perhaps 60-40 makes more sense. Thoughts?
ErikBFlom
1/31
I would like to note that if you use PFM dynamically with inflation and you avoid the high priced hitters, the problems with the rest of the players get even worse. Yes, this is what you would expect if you think about it. I suppose this means that PECOTA needs to have a second function to deal with, say, the bottom half of the pool separate from the top half of the pool for a given league. I think it is better to push the excess money into the average to average-plus players than into lower ranked players. Otherwise, you cannot bleed excess cash fast enough.
brianjenner
1/31
Good article. Agree that the PFM is accurately valuing players, but it isn't taking into account the fungibility of the lower-tiered players. It's not that the experts are wrong, per se. The typical curve I see looks like this: http://i.imgur.com/WBxC5pB.gif I've been thinking of ways to deal with this, and how to shift those dollars to the better players. I may force the values of the bottom n players down to $1, (whatever number of players typically are bought for $1). Then re-calculate and re-allocate the remaining dollars. The problem here is that there may also be an inordinate amount of $2 players, but I think at this point the curves should be much closer, and it has the benefit of being somewhat logical rather than arbitrary.
dpmcnabb
1/31
I am surprised that values assigned by PFM in mixed leagues are so much higher for top-tier players. For example, in an AL only league PFM values Trout at $36. But for both leagues, Trout's value jumps to $51. Have the experts typically incorporated such a large difference for mixed leagues?
MikeGianella
1/31
Generally speaking, the experts incorporate some of this difference into the best players's price but don't have the guts to go all the way. I know that Scott White of CBS Sports did it in CBS's 12-team mixed, but in Tout Wars mixed, owners generally stop in the mid-40s or so.
Ecrazy
2/01
I usually drop 1 or 2 teams from my auction PFM, and let that shrink the value of the bottom players. (Essentially the same game the mixed values is playing). It shrinks the player pool, but as mike showed, trying to value and predict players #140-160 is basically useless anyway. Realistically, #130 - #200 are all essentially worthless anyway. There's no way to predict which 20 of the last 80 players will be taken, so don't try. Cut them out of the auction entirely. It will flex money evenly throughout the rest of the auction.
brianjenner
2/01
"I usually drop 1 or 2 teams from my auction PFM, and let that shrink the value of the bottom players." This is an excellent idea. I think for 15 team league, creating values for a 13 team league would be a good thing to try, while removing $46 from the total budget, in order to not inflate the values too high. Although when converting values dollar values, it may be a good idea to create the values based on 15 teams so that the replacement player calculations aren't off too much.