Happy Holidays from Baseball Prospectus! We'll have new content on December 26th.
November 5, 2010
The BSAT Answer Key
When last we met in this space, I shared with you the Baseball Skipper Aptitude Test (BSAT), a semi-tongue-in-cheek multiple-choice exam designed to help identify quality managerial candidates based on their approach to such things as lineups, bullpen usage, and in-game strategy. My purpose was primarily to entertain, but a number of readers have asked that we divulge the “correct” answers, or at least the answers to which most Baseball Prospectus authors would subscribe. To that end, earlier this week I took a quick poll to discover how our staff members would answer these questions, and you can find the results below. I’m not surprised to report that some of the questions provoked a wide array of responses, and given the pulsing intellect and contrarian nature of our authors, a fair number chose to occasionally go outside the menu with their answers. .
First, a few caveats. When I was putting the test together, in addition to searching for questions that could be structured to support a Zucker-Abrahams-Zucker groaner or two, I tried to create answer sets not so much to see if a manager would select the optimal answer, but to see if they would avoid the worst answer. Keep this in mind, since sometimes this resulted in several answers being easy to defend as the best. Also, most of the strategy-based questions don’t give anywhere near enough detailed information about team- and game-state to be sure of the best answer, so when I say that more than one answer can be correct, or when a responder states that not enough information is available to choose, that’s certain to be true. And lastly, by no means do I think that any manager who aces this test would be good at the job—on the contrary, my personal suspicion is that the “leader of men” aspects of managing a baseball team are more important over a long season than the tactical side, and no quiz can assess a candidate’s strength in that area.
Without further ado, here’s how we answered:
1. A baseball is traveling in a straight line south at 91 mph. Before making impact with a wooden cylinder and rebounding 450 feet due north through a five-mph easterly wind. The man who threw the baseball, a pitcher on your team, has already recorded 14 outs in a must-win game, and your team’s lead has just been trimmed from 6-1 to 6-4. How would you best describe the vector of your travel immediately after this event?
Answers: A: 2 B: 1 C: 0 D: 0 E: 6
My answer was “E.” My point here, of course, is that a manager shouldn’t be wedded to the win statistic in a must-win game, though at least one respondent noted that the answer may very well be different if this isn’t a must-win game—as galling as it sometimes may be that players, fans, and the media can get hung up on pitcher wins, I agree that a good manager shouldn’t summarily ignore the psychological effect pulling a pitcher at this point might have.
2. Please rank the importance of each of these criteria when selecting a “closer”?
Answers: 2 CDABE, 1 CBADE, 1 CABDE, 1 CADBE, 3 C
My answer is CBADE. One respondent said there was no simple answer (which, of course, is true). Of course getting outs is the most important criteria for a closer; after that, there was some disagreement on the relative importance of being a “proven closer” not having failed as a closer, or just seeming to be a person wired to perform well in high-stress situations. I bought into the idea that failing as a closer was as good a proxy as I could find for intestinal fortitude, but a plurality seems to view previous ninth-inning failures as sample-size anomalies rather than a permanent case of the shakes.
3. In which of these situations is it acceptable to bet on a baseball game?
Answers: Never: 9
No comment needed.
4. It’s May 15 and your organization’s top prospect is a slick-fielding third baseman who’s raking at Triple-A. Your starting third baseman has just injured his wrist and will miss at least six weeks, while your backup third baseman is a career .283/.305/.355 hitter with an average glove. Your team wasn’t expected to contend, but currently sits only two games out of first place. At a post-game press conference, you’re asked if the organization should call up the Triple-A third baseman. How do you respond?
Answers: A: 1 B: 1 C: 0 D: 7
My answer was “D.” Several respondents essentially answered that they were thinking “C” but would answer “D” in the press conference. I’d love to hear an actual manager give answer “C” just to find out how fans and the media would react. Would they applaud the honesty? And if I were the GM, how would I feel about it? I confess that I’m not completely sure.
The respondent who said “A” pre-supposed that the kid really did have something to work on, and the “B” answer wanted the organization’s best players on the field—both defensible in their own way.
5. Intentional Walks are to Winning Baseball as Water is to:
Answers: A: 0 B: 2 C: 4 D: 2 Land: 1
My answer was “C,” in line with the plurality. I think there are appropriate times for the intentional pass, e.g., bottom of the ninth, tie score, runner on second, two outs, great hitter at the plate, terrible hitter on deck, no pinch-hitters available. Thus I don’t think the IBB and winning baseball are as combustible a mixture as sodium and water. But as anyone watching Joltless Joe during the ALCS can attest, continually relying on the free pass can be like picking at a scab—keep doing it, and you’re certain to leave a scar.
6. Below are the batting lines for several of your players. Which of them would you be most likely to bat leadoff?
Answers: A: 0 B: 3 C: 0 D: 6
My answer was “B,” and many respondents wisely added that their choice depends quite considerably on the rest of the lineup. The majority plumped for the highest OBP (and likely highest TAv) of the four options at the top of the order, and that may well be the best answer. Personally, though, I’d bat that guy second assuming the other guy is a better baserunner and might score on a few more two-out hits. Some might ask why we should even worry about this, since batting order doesn’t account for much over the course of a season. My response would be that (a) it’s just as easy to get this right as to get it wrong; (b) in a single game it might make all the difference in the world who gets an extra at-bat when; and (c) few things annoy me as much as speedy out-makers in the leadoff spot, and if I’m the GM I’m not going to hire a manager that does that. Period.
7. Please circle each situation in which you would never let your closer pitch:
Answers: A: 0 B: 0 C: 0 D: 0 E: 9
I guess this can be taken as a loud indictment of current closer usage patterns and support for using your best pitcher in the highest-leverage situations, and I don’t expect to get too much argument about this ‘round these parts.
8. Please fill in the blanks with the selection below that you feel best completes this sentence: Pitch counts are ___________and should be used ______________.
Answers: A: 2 B: 0 C: 7 D: 0
I answered “C,” as did the majority. I suspect the Murray Chasses of the world would find it surprising that a bunch of propeller-heads don’t worship at the altar of pitch-count orthodoxy (if there is such a thing), but you, Gentle Reader, know better than that. The consensus here seems to be that pitch counts are useful information both to help avoid injury and ensure pitchers depart before losing too much effectiveness, but can be a pretty blunt instrument and aren’t the one-stop-shop some purport them to be.
9. Which of these movies best exemplify your leadership style, and why?
Answers: A: 1 B: 0 C: 0 D: 3 E: 2 N/A: 2 Godfather Part II: 1
My answer was “D,” while two authors left this one blank. No clear consensus here, of course, but the most common response from our sabermetric panel involved Henry Fonda’s quiet but insistent appeals to open-mindedness and logic over the knee-jerk judgments and ingrained prejudices of his fellow jurors. Wonder why that is. Oh, and if your name is Fredo and your brother writes for us, here’s a word of advice: don’t go fishing.
10. After a series of meetings in a small conference room at our spring training facility in Florida, a staff member approaches you to say that several meeting attendees had been complaining about your bench coach’s unpleasant body odor. What do you do?
Answers: A: 1 B: 7 C: 1
My answer was “B.” Obviously there’s no deep insight to be gained here—I just put this in because I’ve literally been asked this question on nearly every job examination I’ve ever taken.
11. At what rate must basestealers generally be successful to make their attempted steals beneficial?
Answers: A: 0 B: 0 C: 3.5 D: 2 C/D: 3 E: 0.5
My answer was “D”—the “standard” answer is 75 percent, so I chose high since I hate it when my team makes outs on the bases. Our astute authors know this, of course, so the largest number actually noted that the answer should be somewhere between “C” and “D.” Several respondents noted that this is only true overall, not in a given situation, and I heartily concur.
12. What do you think of the Expected Runs Matrix?
Answers: A: 0 B: 1 C: 3.5 D: 4.5
My answer was “D,” as was the majority. Many of the comments I received indicated that knowing the Expected Runs Matrix is important but not a point-and-click guide to baseball strategy, and when reading the answer again I think both “C” and “D” are saying the same thing, with “C” merely placing a physical manifestation of the numbers at the manager’s fingertips. Note also that I suspect the author who answered “B” either mistyped or was playing with my head.
13. Circle each situation in which you feel it might be appropriate to authorize bunting, depending on inning and score:
Answers: A: 9 B: 2 C: 1 D: 1
My answer was “A,” which was almost unanimous. One author also said “B, sparingly,” and another said a situation could occur where any of them are worthwhile, but rarely. If there’s any BP groupthink to be found here, it’s in our common loathing of the inappropriate sacrifice bunt. Junior circuit managers, repeat 10 times after me: trading an out for a base is like trading a diamond for a potato. Sure, there may come a time when you need to do it, but not as a matter of course.
14. How important is on-base percentage for sluggers who aren’t very fast? Circle all that you feel apply:
Answers: A: 7 B: 2 C: 1 D: 0 E: 0
My answer was “A,” as was the majority. Choosing “B” over “A” likely was an acknowledgement that speed isn’t insignificant, an easily defensible choice when choosing between indefinite wiggle words such as very and somewhat. One author who chose “A” also chose “a little C” for similar reasons. For those who have asked me why I ended answers “C” and “D” in that way, you can find your answer here.
15. If you had Neftali Feliz on your team, can you envision a situation where you might let him pitch in the eighth inning to help stop a rally and protect a lead in a playoff game?
Answers: A: 9 B: 0
No comment needed.
16. There are two outs and a runner on second, and your team is down a run in the bottom of the ninth. The opponents have their right-handed closer, Lester Leviathan, on the mound, and you are about to pinch-hit for your own pitcher. On the bench you have Player X, a right-handed batter whose season and career numbers are approximately .280/.350/.440; and Player Y, a left-handed batter whose season and career numbers are approximately .260/.310/.380. Player X is 1-for-8 with a double and two strikeouts versus Leviathan; Player Y is 5-for-8 with two doubles and a strikeout versus Leviathan. Who would you use to pinch-hit?
Answers: A: 1 B: 0 C: 4 D: 3 E: 1
My answer was “D.” This question, not surprisingly, generated the most answer-‘splainin’ of the entire quiz, with many responders saying there needs to be more information to give a solid answer. The crux of the decision, I think, comes down to sample sizes: at what point can we convince ourselves that a player is so good/bad against a certain type of pitcher that we’d let an overall inferior batter hit ahead of an overall better one? I’ve written before that I believe platoon differences and batter/pitcher matchup data are sometimes given shorter shrift then they deserve by the stathead community, so it may surprise some that I’d automatically choose the better overall hitter here instead of considering those other factors. My reasoning is that the guy with the .690 career OPS is such a lousy hitter that I can’t imagine those other factors outweighing his overarching suckitude. Others may (and did) disagree. If I were actually giving this test to a prospective manager, I’d be less interested in hearing what choice the manager made than the thought process he used.
17. After a tough loss, what word best describes how you would likely respond to an insinuating question about a tough managerial decision late in the game?
Answers: A: 2 B: 1 C: 1 D: 1 E: 3 C or D: 1
My answer was “A.” Surprisingly, not a single respondent went off-menu and volunteered something like “magnanimous” or “patient.” I guess you can’t accuse us of not being self-aware.
18. Which of the following images best represents how you feel when you think of the word “rookie”?
A. B. C. D.
Answers: A: 1 B: 0 C: 4 D: 3 C and A: 1
My answer was “C.” Other than noting that many of us are “Simpsons” fans, the lesson here is that our authors are quite supportive of playing rookies. It’s fair to ask, though, if I’d have a different opinion if I actually had to put my financial future partially in the hands of some kid with an anarchy symbol tattooed on his backside.
19. How would you rate the relative importance of clubhouse chemistry and player talent in assembling a winning ballclub?
Answers: A: 2 B: 6 C: 0 D: 0 E: 0 A/B: 1
My answer was “B.” The majority of our sample feels that chemistry has some significance, but talent is baseball’s trump card. No argument from me.
20. What name do you plan to call an umpire when you feel the need to be kicked out of a game to motivate your team?
Answers: A: 2 B: 1 C: 0 D: 4 E: 2