July 13, 2005
Lies, Damned Lies
Book Review, Scout's Honornot before Bart says, just as Homer is about to clip the wire:
Dad, I beg you to reconsider. Tractor pulls...Atlanta Braves baseball!It was not so long ago that the Atlanta Braves were a punchline. They lost an average of 96 games a season from 1985 to 1990, and were dead last in attendance during the latter three years of that period. The most distinct childhood association I have with the club is putting any and all Braves at the very bottom of my baseball card carrying case, where the cards were most likely to be warped, damaged, or simply forgotten about.
I don't need to tell you what came next. Whether it was the Reverse Curse of Bart Simpson or something else, the Braves have been the most successful franchise in baseball ever since. For my money, in fact, the Braves' performance during the past 15 seasons has been the second-most remarkable sustained run of success in baseball history, behind only the two-pronged Yankee dynasty of 1920-1964. I'm a big fan of everything that the Braves have done, and of the way that they do business.
That's why I was so disappointed with Bill Shanks' Scout's Honor. It would have been enlightening to read about how the Braves scrutinize and solve baseball problems. Instead, we are left with a series of anti-Moneyball platitudes, most of which have very little to do with the way that the Braves actually do business. Here is a typical snippet:
Moneyball was just as insulting to me as it was to so many scouts around the game. As it was explained in the book, the A's, and the 'moneyballers,' apparently care more about on-base percentage than the makeup of a player I knew the Braves' story, and the story of scouting vs. bean-counting, had to be told Baseball operated long before computer wiz kids got involved. It's about instincts, wisdom, and knowledge, not just a Microsoft spread sheet. (p. 8)Mind you, these are Shanks' words. The Braves' personnel themselves, in Shanks' interviews with them, are much more moderate, and much less venomous, in their statements. Their consensus, in fact, is that statistics are a tool, one of any number of tools useful to a baseball club which is exactly what a lot of us computer wiz kids might say. Only Shanks pits the two philosophies against each other in this fashion.
Indeed, there isn't much room to debate that the Braves have in fact benefited tremendously from the scouting and development sides of their organization. The following table provides the 30 highest Braves in VORP over the period 1991-2005, along with how those players were acquired (the VORP figures are for these players' tenures with the Braves only).
Top Atlanta Braves in VORP, 1991-2005 Player How Acquired VORP Greg Maddux Free Agent (1993) 780.0 Chipper Jones Draft (1st Round, 1990) 696.6 Tom Glavine Draft (2nd Round, 1984) 643.9 John Smoltz Trade (veterans, 1987) 542.2 Andruw Jones Latin America (1993) 372.9 Javy Lopez Latin America (1987) 314.6 Jeff Blauser Draft (1st Round, 1984) 222.9 Kevin Millwood Draft (11th Round, 1993) 202.8 Ryan Klesko Draft (5th Round, 1989) 198.5 Rafael Furcal Latin America (1996) 188.0 David Justice Draft (4th Round, 1985) 183.7 Fred McGriff Trade (prospects, 1993) 179.5 Steve Avery Draft (1st Round, 1988) 167.1 Marcus Giles Draft (53rd Round, 1996) 153.3 Gary Sheffield Trade (prospects, 2002) 140.0 Terry Pendleton Free Agent (1991) 121.8 Ron Gant Draft (4th Round, 1983) 110.0 Andres Galarraga Free Agent (1998) 103.4 Mike Remlinger Trade (veterans, 1999) 103.2 Denny Neagle Trade (prospects, 1996) 101.0 Kent Mercker Draft (1st round, 1986) 88.8 Greg McMichael Minor League FA (1991) 86.2 J.D. Drew Trade (prospects, 2004) 78.7 Mike Hampton Trade (veterans, 2003) 78.5 Kerry Ligtenberg Minor League FA (1996) 75.0 John Burkett Free Agent (2000) 74.6 Russ Ortiz Trade (veterans, 2003) 66.5 Julio Franco Minor League FA (2001) 65.1 Marquis Grissom Trade (prospects, 1995) 61.9 Brian Jordan Free Agent (1998) 61.6About 43% of the value represented in this table was acquired directly through the June Amateur Draft. Another 14% as acquired through the Braves' Latin American scouting pipelines (something which, sadly, is not discussed whatsoever in the book). Three minor league free agents--Greg McMichael, Kerry Ligtenberg, and a resurrected Julio Franco--add 4% here. Add 8% more for players acquired by trading prospects.
We've also put John Smoltz, who represents nearly 9% of the Braves' VORP all on his own, in the trade column. At the time the Braves acquired him from the Tigers for Doyle Alexander, he was struggling in obscurity in the New York Penn League, where he would post a 5.68 ERA, and a 86-81 strikeout-to-walk ratio in 130 innings. It takes at least some kind of a scouting judgment to see through those statistics--Smoltz, a 22nd-round draft pick, did not have a prestigious pedigree. If we add Smoltz to the scouting side of the ledger, then we're up to 78% of the Braves' talent being essentially home-grown, or acquired for home-grown talent.
There isn't much debate that scouting and development have been essential to the Braves' success. That's the what part of the story. The salient question, however, is not the what but the how. How did the Braves manage to see something in 20-year-old John Smoltz, or a 42-year-old Julio Franco, when nobody else did? How did they manage to take Marcus Giles, a 53rd round draft pick, and turn him into an All-Star? How does Leo Mazzone work his magic?
Those questions aren't really answered by Shanks. His best attempt is a 15-part platform that he outlines in Chapter 23 of his book, which he describes as the Braves' philosophy. I'm going to examine each of those 15 points here, with two important questions in mind:
Shanks #1. Makeup. Let me tell you what I think makeup is. I think makeup represents those personality characteristics that make a ballplayer more likely to achieve toward the upper end of what his physical and intellectual capabilities might otherwise make him capable of. B.J. Upton's PECOTA comparables chart, for example, includes players as good as Robin Yount and Alan Trammell, and as obscure as Dick Kokos and Shin-Soo Choo. I think makeup accounts for a fair bit of what tends to make Yount turn out like Yount, and Kokos turn out like Kokos. If I had to pick out a single synonym for makeup, it would be work ethic. I think Albert Belle, who worked his butt off and never missed a game, had tremendous makeup, even though he was a jerk of a human being. I think Sean Burroughs, who is either too stubborn or too indifferent to refine his swing to hit for more power, has piss-poor makeup, even though he's the son of a major leaguer. I think makeup is pretty important, especially for a player who has yet to really define what he can do.
The Braves, at least in Shanks' telling, define makeup somewhat differently. It seems to be, on the one hand, a sort of mental toughness--a Texas high school left-hander throwing 150 pitches in a swirling rainstorm to advance his team to the regional finals, or Chipper Jones (this is an actual story from the book) punching out a rival who is mocking his team's pitcher--and on the other, a sort of Southern-grown wholesomeness. As Director of Player Personnel Dayton Moore says:
"Many players don't make it to the major leagues because of who they associate with or what they do off the field, or the way they respond to negative things that happen to them on the field You've got to be able to apply moral principals in your life to be successful, and you've got to be able to have that balance on and off the field. The same character traits that make a baseball player successful are the same character traits that make a schoolteacher or a President of a company successful." (p. 351)I don't think Moore's definition of makeup is terribly incongruous with mine; in fact I think the two are pretty close once you peel away Moore's slight air of righteousness.
As for Shanks himself, his definition of makeup appears to be somewhat more fluid. It wouldn't be exaggerating too much to say that he thinks of makeup as those things that tend to make a Braves prospect turn out like a Brave, but a Devil Rays prospect turn out like a Devil Ray. Indeed, Shanks invokes the Devil Rays specifically in discussing the differences between Chuck LaMar's scouting philosophy and the Braves'. Quoting Braves' area scout Brian Kohlscheen:
"[Lamar] didn't spend as much time on the makeup, but on the ballplayer part [ ] Paul [Synder, former Braves' Scouting Director] was much more interested in high-ceiling, athletic baseball players with good makeup. You could cheat a little more on the makeup the higher they were, but really not a lot. Chuck was more 'we want high-ceiling and athletic players and I think we can teach them how to play the game is a little bit'." (p. 341)The implication is that the Braves draft high-ceiling, athletic baseball players, while the Devil Rays draft high-ceiling, athletic football players. Is it any wonder that the Braves emerge with the best of it?
The question left unanswered, of course, is how the Braves identify good makeup. Shanks provides some anecdotal examples, like the Chipper Jones story, as well as some evidence that Braves' scouts tend to spend a lot of time getting to know the draft picks and their families, which is probably of some value. It does seem clear enough that an organization that drafts a significant number of players sight unseen is going to run into some problems. But you could very probably author a story about the Pirates' or the Orioles' drafting philosophy and emerge with exactly the same sorts of anecdotes.
Kohlscheen's words, indeed, are revealing: "You could cheat a little more on the makeup the higher they were." You take the choir boy with the .310 batting average before the drunk with the .320 batting average. But you take both of them before the choir boy with the .240 batting average.
Shanks #2. "Don't discount stats, but "
This is exactly how Shanks phrases it. We've already indicated our agreement. Statistics are just a tool. So why the need to pit them against scouts in Ali versus Frazier fashion?
I will use this space to make one additional comment: Some of what is discussed in Moneyball, and much of what we discuss here at Baseball Prospectus, is not scouts versus stats at all. Rather, it's stats versus stats. It's not that you should draft the drunk with the .320 batting average before the choir boy with the .310 batting average. It's that you draft the choir boy with the .310 batting average and 80 walks a season before the choir boy with the .320 batting average and 30 walks a season. The Braves, indeed, have had a reasonably good idea of the value of on-base percentage. In the years between 1991 and 2004, they've finished in the upper half of the National League in walks drawn 10 of 14 times.
#3. Trust Your Scouts
Shanks' thrust here is not that scouts' opinions should be allowed to override all else, but more specifically, that those higher up the organizational food chain, such as the scouting director, should give credence to the opinions of area scouts. This seems plenty sensible: Why invest in area scouts if you aren't going to listen to them? This may in fact be something that the Braves do better than the rest of the pack.
Still, this is one of a number of items that falls into the category of good business practices, rather than into the scouts versus stats divide. Shanks has convinced me that the Braves have good business practices, but this isn't at the core of his anti-Moneyball argument.
#4. College versus High School
The question of whether college players or high school players make for better draft picks in the abstract is still at least somewhat open in analytical circles, though surely most sabermetrically inclined teams have tended to favor collegians. Shanks is careful to note that the Braves won't hesitate to draft a college player when the situation warrants. Nevertheless, it's clear in which direction the organization leans. Since John Schuerholz's first draft in 2000, the Braves have had 58 picks in the first three rounds of the June draft. Of those picks:
Shanks contends that the Braves might have a particular competitive advantage in developing prep players:
The Braves prefer to raise their own players, to get the players before they've developed bad habits. If they get the players when they're young, right out of high school, they're able to imprint their philosophies so the players know what to expect, what is expected of them, and how to play "The Braves' Way" (p. 352-353)A lot of what teams are gaining when they draft college players is information--you know exactly how a guy has developed from the ages of 18 through 21. I wouldn't phrase it exactly like Shanks does, but it's true that if the Braves do an exceptional job of developing players in that age range, they might well find it relatively more worthwhile to draft high schoolers. In fact, they might do such a good job of development that it's worth sacrificing the informational advantage that you get from collegians.
Nevertheless, this isn't a stats versus scouts issue. Rather, it's a matter of scouting and developing effectively. Moreover, it's a matter of recognizing and arbitraging that advantage in the draft. If the A's do a good job of turning collegians with high OBPs into championship-caliber ballplayers, it is fitting that they draft more of them. Similarly, if the Braves do a good job of turning high school players into championship-caliber players, the Braves should draft more of them.
5. Synergy between scouting and player development.
Exactly what I'm talking about. Organizations have different strengths in maximizing the values of different types of ballplayers; you ought to acquire players who fit those strengths. You sign Jaret Wright because you think Leo Mazzone can make a pitcher out of him, and so forth.
This might well be the linchpin of the 'Braves' Way.' It could also well have been the linchpin of 'Scout's Honor.' Unfortunately, apart from an anecdote here and there about Glenn Hubbard working with Marcus Giles on his defense, Shanks invests very little time in explaining just how the Braves develop players as well as they do. Mazzone, for example, is scarcely mentioned. Perhaps the Braves were reluctant to give away any trade secrets. That is understandable, but it doesn't make for a good read.
No, not plate discipline. The Braves have had a longer time horizon than other organizations; this is another reason that they can afford to draft high schoolers, waiting several years for them to become major-league ready.
Having a long time horizon is almost by definition going to be to a team's long-run benefit. Indeed, if there's one theme that underlies virtually all ineffective baseball organizations--think especially of the Mets of recent vintage--it is the desire to win immediately, superseding longer-term objectives. Even a smart general manager can make some dumb decisions if he fears for his job security.
Once again, however, there isn't any intrinsic disconnect between an analytically-oriented approach, and a longer time horizon. Moneyball itself might be somewhat responsible for this misunderstanding; Billy Beane's analytic approach is described as having emerged from the need for a sort of quick fix for a team in dire financial straights. But Beane's approach can be just as effective on a club with more resources, or the luxury of time--witness the Red Sox.
7. The strength of the draft dictates picks.
This is really a corollary of the previous point: think long-term, and draft for value, not for need. I don't have any further disagreements.
8. Protecting picks.
The Braves have done a very good job of making sure that the high-school players they draft actually sign with the club, either immediately or via a draft-and-follow, which is what Shanks is referring to here. Of the 50 high schoolers they've selected in the first three rounds of the draft in the Schuerholz Era, only four have defected to college. Shanks provides extensive examples of how the Braves manage to facilitate this. This is one place where having scouts getting to know their draft picks can pay some dividends; it probably also helps to be drafting a lot of kids from the South, many of whom will have grown up Braves fans.
This is one of the more compelling elements of the book, and certainly one of the Braves' competitive advantages. However, it falls squarely into the good business practices category, and is not an advantage of a scouting-oriented approach per se.
9. Pitching, pitching, pitching
The Braves haven't actually been especially predisposed toward pitchers in the amateur draft. As referred to above, of the 58 players selected in the first three rounds under John Schuerholz, a bare majority (30) were hitters. That's somewhat more pitching than a 'pure' sabermetric club might be inclined to select, but it isn't out of line with draft averages.
What the Braves have done a very good job of is developing those pitchers that they do acquire. Ironically, one of the ways that they manage to accomplish this is with the careful use of pitch counts. As Dayton Moore explains:
"We treat these pitchers with tremendous care We never try to win ballgames at the expense of developing our players. If a guy has a no-hitter in the sixth inning and he's reached his pitch count, he's coming out" (p. 356)The Braves take this stuff seriously; it's rare that you hear about a Braves prospect having been overworked. Elsewhere in the book, in fact, it is suggested that one of the reasons the Braves prefer high-school pitchers is that they can prevent pitchers from being overworked in college programs. Pitch counts, of course, are only one part of the puzzle, but Shanks leaves the rest of the developmental equation as an exercise for the reader.
10. Develop talent for trades
'Replacement value,' in Shanks' lexicon, refers to having a stable of major-league caliber performers available in the minor leagues in the event that a regular goes down, something that the Braves have benefited from tremendously this year. As with his previous point, developing talent for trades, this is a happy side-effect of having good organizational depth, which in turn is a result of scouting and developing players effectively. It is not really a point of philosophy unto itself.
12. Scouting your own players
It certainly sounds good, but the Braves don't have an especially good track record in this department. It's true that the Braves have managed to dump a lot of young talent on other clubs for good value, just before those players began to implode. A partial list would have to include: Rob Bell, Bruce Chen, Mike Kelly, George Lombard, Melvin Nieves, Andy Pratt, Tony Tarasco, and Ron Wright. On the other hand, they've also given up prematurely on Jermaine Dye, Jason Marquis, Odalis Perez Jason Schmidt, and Merkin Valdez. Schuerholz scores more impressively when acquiring young talent than when giving it away.
13. Follow the plan
It should go without saying that, regardless of the philosophy that an organization adopts, it ought to apply it consistently, both over the course of time and across different layers of the organization. This is also part and parcel of having a longer time horizon. The Braves do this very well, but it falls into the good business practices category.
14. Making an impact on the players
These later two principles boil down to treating your players well (#14), and treating your scouts and executives well (#15). I'm sure that the Braves do a good job of this, and Shanks' book is compelling in describing the sense of loyalty that the organization instills. But these things, too, are part of good business practice, and not emblematic of having a scouting-oriented organization.
I don't doubt that Shanks has done an adequate job of elucidating the Braves' organizational philosophy. Nor do I doubt that any baseball team that applies these sorts of principles, and applies them effectively, is likely to see far more success than failure. It's disappointing, however, that Shanks spends so much time talking about philosophy, and so little time talking about the execution of those philosophies. What does distinguish the Braves from the Devil Rays? We never really find out.
The bottom line is that winning organizations execute well. If you're scouting and developing players, do it right. If you're doing analysis, do it right. Adopt and apply championship-caliber business practices. Treat your people well, and make sure that everybody is on the same page. Shanks contends that the Braves' success is a repudiation of the analytically-oriented approach to the game. I can't see how he connects the dots that way.