Baseball Prospectus Needs Your Help! Check out our call for contributors!

A data analyst is hired to analyze a problem a baseball team is having. He studies and studies the problem from as many angles as possible but still can’t solve the issue. Finally, he comes upon some data that offers strong, tangible evidence of how to fix the problem and comes back to the team with his findings. The manager institutes the analyst’s recommendations and the team’s on-field performance improves. But then the general manager finds out about the change and orders the manager to go back to the old way of doing things. The analyst’s recommendations fly in the face of conventional wisdom, so despite the proof offered, the team rejects the analyst’s findings. Worse yet, not only is the analyst’s advice rejected in this specific instance, but he is discredited in the industry for presenting such a controversial theory as fact. He never gets a job in baseball again, and dies alone, penniless and forgotten.

This story sounds terrible, and I don’t mean that from the perspective of the unfortunate analyst who is now living on the mean and gritty streets of Hypotheticalville. It comes across as apocryphal nonsense that couldn’t have possibly happened in the real world and was devised by a numbers-oriented writer to pedantically prove a point.

The story, though, is based on real-life events. The only thing untrue about it is that it isn’t a baseball story.

In 1846, Ignaz Semmelweis was an assistant (resident) at the First Obstetrical Clinic in the Vienna General Hospital. At the time of his appointment, the mortality rates for women admitted to the First Clinic due to puerperal fever was 1-in-10. Vienna General’s Second Obstetrical Clinic had a 1-in-25 morality rate. The First Clinic’s reputation was so bad that some women refused to be taken to the clinic until after their babies were born.

It took the accidental death of a professor to lead to the solution to this horrible problem. This professor was accidentally poked with a student’s scalpel and died of symptoms similar to the puerperal fever that was killing women at the First Clinic. Semmelweis had already been studying this problem, but now he was able to conclude that there was a connection between contact with cadavers and the death rates in the maternity ward. Indeed, physicians in the First Clinic were going straight from handling cadavers in the morgue to delivering babies in the maternity ward. In the Second Clinic—where only midwives were delivering babies—there was no prior contact with cadavers.

Semmelweis immediately instituted a hand-disinfecting policy and within a year the death rate in the clinic plummeted to near zero. If only the story had ended on this happy note. Unfortunately, that wasn’t the case. Despite the success of the disinfection procedure, the medical establishment rejected Semmelweis’ findings. While his theories seem ho hum in the W21st century, germ theory hadn’t been established yet and few were willing to give his conclusions any merit. Within a few years, Semmelweis had been dismissed from the hospital, his procedures were abandoned, and the clinic’s mortality rate shot right back up to where it had been before Semmelweis had instituted his procedures. After years of trying in vain to convince the medical community that his theories were valid, Semmelweis’ mental health deteriorated. He was institutionalized in 1865 and died days later from a wound caused due to a beating administered by the institution’s guards.

So what does any of this have to do with baseball?

The parallel between the authority in medicine that stood up against Semmelweis and baseball’s so-called old guard that has been resistant to accept sabermetrics is that both parties stood up in the face of compelling evidence to the contrary. In Semmelweis’ case, he did a poor job of compiling data to prove his point before he instituted his changes. Nevertheless, it was a true tragedy that the authority figures of his day weren’t at least interested in the positive results of his conclusions and weren’t willing to allow his disinfection procedures to continue while further research was conducted.

I came across this story years ago in a college class I have long forgotten, but didn’t realize until I started researching this article that there is a name for this phenomenon: the Semmelweis Reflex. The phenomenon is defined as behavior that thoughtlessly rejects new knowledge because it contradicts established belief systems.

Baseball is always cited as a game that is slow to change because it is steeped in tradition, making it unlike other walks of life. The logical progression of this explanation is that baseball is unique in its resistance to accept change. In reality, this phenomenon is common in nearly every walk of life, even in areas where it would seem that science and logic should always take a front seat.

Near the end of Semmelweis’ life, Louis Pasteur performed formal experiments that discovered the origins of puerperal fever. These experiments are seen today as the origin of germ theory that we all accept today as commonplace. Eventually, disinfection became the custom in medicine and Semmelweis’ impact was posthumously recognized.

Baseball is in the “post-germ-theory” era. Many of the experiments have been conducted and the theories are being enacted—both on the field and in front offices throughout the professional ranks. The idea of an entrenched establishment that refuses to acknowledge sabermetrics is more of a mythology now than the idea that sabermetrics is roundly rejected by the lords of the sport.

To be certain, there are stubborn holdouts that continue to cling to the idea that the intangibles outweigh the math. But the ranks of the disbelievers have diminished, and continue to dwindle as time passes. The debates within the game have already changed, and mostly for the better. Whether it is a broadcaster discussing BABIP or swinging strike percentages, or a writer considering WAR in casting his MVP or Cy Young Award vote, we are already inhabiting a brave new world. Baseball’s Semmelweis Reflex remains, but it is weakened. The belief that there is no value to be gained in baseball through analytics is held by fewer and fewer with ties to the game every year. The old-school guard that insists that intangibles outweigh facts sound like they are telling us that the sun revolves around a flat Earth that is at the center of the universe. The “there is no value in analytics” crowd might never completely disappear, but there will come a time when they will sound as absurd as someone suggesting that poisonous vapors are making you sick.

Thank you for reading

This is a free article. If you enjoyed it, consider subscribing to Baseball Prospectus. Subscriptions support ongoing public baseball research and analysis in an increasingly proprietary environment.

Subscribe now
You need to be logged in to comment. Login or Subscribe
I am not familiar with the Semmelweis story, but I am guessing that the contexts are different between it and baseball. The baseball analysts have defeated their own purpose in many cases by acting like they are the smartest guys in the room, are the only ones who really understand baseball, have disdain for anyone who does not go all-in with their numbers and feel like the scouting community are Fred Flintstone to their Steve Jobs. Yes, baseball can be slow to change. Knowing and understanding this demands that those trying to institute change have both patience and humility in dealing with the old guard. Many sabermetricians have had neither and have made it more difficult for themselves, their theories and even for those who have gone about it in more understanding ways.
Though baseball has always been reliant on numbers, it is also the most complicated (in deliciously subtle ways) of all of our major sports. Every situation is rich with context that in some cases is not accounted for in a strict and antiseptic look at historical numbers.
I love the new analytics. They are revealing and they inform us as to some areas that need to be explored as we look at the game and as teams try to gain an advantage on everyone else. We can't forget, though, that every situation stands alone as an opportunity for one team or another to have success or failure. The enlightened fan or front office executive should realize that those opportunities are informed by numerous sources. There is the understanding of the numbers and then the understanding of the individual players involved. They each have strengths and weaknesses. The 2006 Tigers lost the World Series in part because their pitchers were poor fielders. Bunting on them might have been a good idea more than the sheer numbers might indicate, for example.
Sabermetricians have softened in many cases of late and many more people have thus been willing to listen to them more. Many have also come to understand that the sheer numbers are not the end of the discussion. The best sabermetricians, like the best front office and on-field personnel, now realize that there is value in all of the various pieces of information available and that there are no real absolutes in any given situation. Hopefully, this will continue and we can all move forward to incorporate multiple sources into the rich fabric of context that we can use to understand and enjoy the game.
Comment of they year? Great job frugalscott19.
Semmelweis was featured heavily in the book Superfreakonomics. According to Levitt and Dubner, his findings were rejected because the other physicians could not fathom that they carried germs (or whatever they were then thought of as being) on their hands. They considered themselves to be super people and above such uncleanliness.
I think the parallel is that the doctors in the Semmelweis story, and mainstream baseball thought, was that both were resistant to new ideas that challenged some of their established ideas. I think it's something of a straw man to say that the reason sabermetric ideas were rejected 30 years ago was because of the arrogance of Bill James et al. Sure, there's something of an attitude of "I can see this, what can't you?" that is somewhat off-putting, but I do think it was overused. Any acknowledgement of nuance in presentation of sabermetric ideas was also used as one way to dismiss the ideas (along with, "You never played, so what do you know?"). And, I don't know of anyone who claimed that scouts know nothing about the game.

Sure, over time the ideas have made their way into the mainstream, but I would like to think it was because open-minded establishment types came to see the merit of at least some of the new ideas. Not because sabermetricians "softened" their views; probably more likely that the establishment types softened theirs.

Apologies if I was misreading your comment.
*why* can't you? (not "what")
Excellent comment frugalscott. Sabermetricians can be snobby and condescending sometimes, and that can be very off putting to old schoolers. Nobody is going to have their views changed when somebody is talking down to them. I find that the animosity is much more prevalent among the anachronists, at least some of which is driven by a negative encounter with a sabermetrician. Make no mistake though, the majority reject analytics because of the power of the narrative. I would love to write something more in depth on this topic (Hey Ben!), but the bottome line is that everybody loves a good story. It makes us feel all warm and fuzzy inside. When facts screw that up, as they tend to do, people will respond with hostility. It hurts when your narrative is descredited, when something you believe has meaning is proven to have no meaning at all. The AL MVP debate of the past two seasons illustrated this so well. Go to any comment board on an article picking Trout over Cabrera, or that devalues the Triple crown, or even so much as mentions WAR, and you'll see exactly what I mean.
The best way to engage old schoolers is calmly and respectfully. That's the only way to keep people's minds open. I also recommend staying away from conversations on comment boards. Comment boards in general outside of BP or Fangraphs tend to be wastelands, but they are also a poor medium for debate and the people there tend to be rude and close minded.
Scouting will never die, and is just as important as it has ever been. Proper player evaluation comes from the combination of analytics and scouting. That's one of the reasons why I love Keith Law!
Sabermetricians will win the war (no pun intended) some day. It's going to be slow, and it's going to be painful sometimes, but the winds of change are in the air. Greinke and Hernandez won Cy Young awards, and Trout has brought our way of thinking more in the public eye. Someday, your average casual fan will look back and wonder how on earth Trout didn't win back-to-back MVP awards. One day, history will look back at all the writers who refused to change, who stubbornly held on to their narratives at the expense of facts, and judge them poorly.
A lecture on Semmelweis is one of my ealiest and fondest memories from medical school. We should always be looking for answers. More importantly, we should continually evaluate the questions that we are asking.
As a board of director president I actually sympathize with the challenges of instituting change in an organization. We are a non-profit arts group so have little wiggle room between success and failure, so pulling off a change in thinking and execution is actually a very complex task. Even if I in the leadership position believe in this change, it needs to be filtered down through every person in the organization for it to be performed properly. This takes proper training and education, which is a drain on resources. People who are threatened will become divisive and cause moral stress. Staff who already work hard and want to go home to their families will be asked to take on this additional load, and may resent that it came from “up top”. And the change will attract heightened scrutiny. Any mishap will be magnified, and the person associated with the idea will need to fight hard against the perception of potential failure. I find that it takes at least several years to truly enact even a small change, and in the meantime there will be acrimonious discussions about whether it is worth it. This is not to defend doctors who kill with their germs, just to put some perspective on the challenges a leadership may face in enacting great new ideas. It does have to be balanced with the reality of organizational ecology. Thanks everyone for this great discussion!