Eric J. Topol, MD: Hello. This is Eric Topol, editor-in-chief of Medscape. I'm really thrilled to have Michael Lewis, one of the most extraordinary storytellers of our era and of all time, with me on this podcast. His many books include Moneyball, Flash Boys, The Big Short, Liar's Poker, and The Blind Side. Today we especially want to get into his most recent book, The Undoing Project. And so, Michael, welcome.
Michael Lewis: Thanks for having me.
Dr Topol: It's really a delight. I think those in the medical community who have followed your work over the years will easily understand how The Undoing Project applies to the day-to-day world of caring for patients. One of the things that you have [focused] on for many years—in fact, dating back to Moneyball—is the idea of perception and cognitive biases. What got you started with that?
It Started With Moneyball
Michael Lewis: I stumbled into it. I was struck by the market inefficiency in baseball. In 2001 and 2002, when I was working on Moneyball, the Oakland Athletics organization was spending $30 million or so on its players, yet it ran circles around teams spending four times as much. And the question was, how could that happen if the market for baseball players is efficient? Why didn't the rich teams just buy all of the best players and win all of the games? When I dug into it, the answer was complicated. But part of the answer was, the experts in judging baseball talent—the baseball scouts—made a lot of mistakes. And there were systematic misjudgments that the Oakland A's exploited and found better ways to value the players, namely by using statistics as opposed to using the judgement of the experts.
Why were the experts making mistakes? Why was this essentially clinical misjudgment going on inside baseball? That was something I did not get to in Moneyball but I got to in my recent book. The people who started to dig into that question were a pair of Israeli psychologists, Danny Kahneman and Amos Tversky. My most recent book is about them and what they had to say about the mistakes people make when they are essentially using their intuition to render judgement.
Dr Topol: It's really a fascinating account of two very different people and the synergy they achieved. How did you go about getting into this? Danny was available to you but Amos [had passed away]. You had to work through the things that you could. How did you pull it all together?
Creating a New Field
Mr Lewis: It felt so different from anything I'd ever written. It took a lot of time. I worked on this on and off for over 7 years. In the beginning, I started with their papers and spent an awful lot of time with Danny Kahneman, who happened to live near me in Berkeley, California. But Amos Tversky had left behind file cabinets filled with his papers that remained 20 years after his death in the hallway outside his office at Stanford. It was stunning. These two guys created a field of inquiry called "Judgement and Decision-Making" that, among other things, spawned behavioral economics and fueled evidence-based medicine. It fueled the movement away from stock market picking towards indexing on Wall Street. They touched so many different spheres of human life.
Amos was regarded as a kind of genius in his lifetime. He died young and people clung to any piece of him they had. He was a touchstone for a lot of very interesting people, so it was not that hard to re-create him because he lived in people's minds. He had saved enough that I could kind of get at who he was and what the relationship was like between these two men that led them to create this extraordinary body of work. But it was messy; it was not systematic. Much of the story was in university towns around the world, so I traveled a lot. I talked to hundreds of people, trying to piece together a memory—not just of a collaboration but also of the two individuals outside of the collaboration.
Dr Topol: It's phenomenal. These are the two people who brought together the field and so many respects of behavioral science, behavioral economics, and a good understanding of human intuition going off track.
'Thinking,' Medicine, and Trailblazing Internist Donald Redelmeier
Dr Topol: As you well know, medical diagnosis is a serious problem. In the United States alone, there are over 12 million major misdiagnoses a year,[1] and more than half of Americans will have an important misdiagnosis in their lifetime. Nothing has really changed that.
In Danny's book, Thinking, Fast and Slow, he got into system 1 reflexive fast thinking and system 2 reflective slower analytical thinking. It has been documented that medical diagnosis today is all system 1 thinking, and that seems to be part of the problem. You really zoomed into that [in The Undoing Project] with Don Redelmeier at Sunnybrook Hospital, affiliated with the University of Toronto—a big part of chapter 7, "Going Viral." Can you tell us about that?
Mr Lewis: Part of the problem in writing this story was that Kahneman and Tversky had touched so many places where people were doing things; it was almost arbitrary where I went. For example, I could have gone to Wall Street as opposed to medicine. I went into medicine because Don Redelmeier was such an interesting character and because he had close relationships with both Amos Tversky and Danny Kahneman separately. Once Kahneman and Tversky started to catalog the kinds of mistakes the mind makes, they realized that medicine would be an interesting place to apply this and investigate the implications of what they had found. They both kind of used Redelmeier as their medical expert to help them design tests to see how doctors went wrong.
Redelmeier is a general internist at Sunnybrook in Toronto, but he functions as a kind of roving check on doctors' thinking inside the hospital, and people acknowledge him in this role. He is constantly preaching to his fellow doctors that they are always susceptible to the kinds of mistakes that people make when they are rendering intuitive judgements. He tries to get across to them that error is not shameful. We get ourselves in even more trouble because we think that making mistakes is a sign of ineptitude. Kahneman and Tversky taught us that error is human. We are hardwired to make certain kinds of errors, and this hardwiring will express itself in the medical environment in certain kinds of predictable ways. If you know this, you can either check yourself or have someone like Redelmeier sitting on your shoulder checking you.
[Redelmeier] wanders around the hospital, kind of questioning how people got to their diagnosis. That hospital is particularly interesting because it's situated next to a mammoth freeway where there are daily accidents. Ambulances rush in people in critical condition who often cannot communicate and must be diagnosed very quickly. It's exactly the kind of environment where people make mistakes. He is in a fertile place to be questioning human judgement, and it's a wonder that this role has not spread to other hospitals and is not formalized. Donald Redelmeier is famous for lots of things, but among other things, he has embarked on a creative research career almost on the side that is very much in the spirit of Kahneman and Tversky.
Dr Topol: He seems to be a disciple of carrying it into the world of medicine—almost like a conscience.
Mr Lewis: What he has done is broaden the notion of what a doctor should be talking about and thinking about. He was the first to make the connection between cell phones and higher risk for mortality behind the wheel of an automobile, some 20 years ago.[2]
Dr Topol: That was a vital thing that he uncovered—no question. He wrote a paper with Amos that you wrote about in the book.
Mr Lewis: [In this] really riveting study,[3] they used Stanford University doctors as lab rats. And Redelmeier showed what Kahneman and Tversky had illustrated in their work—that when people make judgements or decisions about things, often they aren't deciding between the things themselves; they're deciding between the way they're described. They went to these doctors and said, "You have a patient with terminal cancer who is going to be dead within 7 years. But experimental surgery can be performed on this patient and if it works, the patient's cured." Half of the doctors were told that there was a 90% chance that the patient would survive the surgery, and the other half were told that there was a 10% chance that the patient was going to die on the operating table. It's the same thing, just framed differently. Ninety-percent survival versus 10% death. Doctors who were presented with the 90% survival statistic were more than twice as likely to want to perform the surgery.
It's just chilling that you can even manipulate the decision of a serious doctor by presenting the information in a different way. This was just one medical expression of Kahneman and Tversky's ideas, but Redelmeier was behind the framing of the study.
Dr Topol: Another study[4] that really caught my eye in the book regarded the influence of the patient that you are directly caring for versus knowing the literature about patients like that, and how much of the bias and judgement is influenced by that person.
Mr Lewis: Both Amos and Danny would say that all doctors should have a basic course in statistics, among other things. I think it would be very useful for all doctors to have a basic understanding of regression to the mean, base rates and statistical concepts that will actually help them think about how they diagnose, combined with a sense of cognitive biases that we know about. For example, if we see three cases of appendicitis in a row walk into the emergency room, the fourth person who comes in manifesting some of those symptoms, but actually has something else, is more likely to be misdiagnosed with appendicitis. That kind of thing.
Dr Topol: The perceptual issues are abundant and pervasive. Have you personally, or your family, ever had any misdiagnosis that brought this to mind?
Mr Lewis: It's funny you ask, because my wife had a misdiagnosis. She did have appendicitis and was sent back home from the emergency room. Her appendix ruptured in a hotel room. This was just a couple of years ago.
Dr Topol: Oh my gosh.
Mr Lewis: So yes, I've seen that, but it was not on my mind when I was writing about this. The nature of the relationship between the doctor and the patient—the patient kind of regards the doctor as God, and if he got something wrong then they assume he got it wrong for a good reason. I'm very slow to imagine that my doctor is susceptible to cognitive error. Funnily enough, 3 years ago, when I told my doctor, Dean Nickles, a wonderful guy in Oakland, California, that I was working on this book, he said, "I teach all the young people who come in here to slow down, precisely because of this and because I've seen so much error." I'm not sure whether he knew Kahneman and Tversky's work, but their work resonated with him.
A Surprising Study in Radiology
Dr Topol: It's so true. Then the question is, can that switch to system 2 thinking, the "slow-down mode," be achieved? Another chapter in your book that really delved into this was on Paul Hoffman at the University of Oregon. Quoting the book, You could beat the doctor by replacing him with an equation created by people who knew nothing about medicine and had simply asked a few questions of doctors. Any thoughts about what you learned from Paul Hoffman?
Mr Lewis: It was not that psychologists were looking to meddle in the medical practice. In the beginning they were just trying to model what experts were doing when they made judgments. They were studying radiologists in this case. And they showed them x-rays and asked them to diagnose, essentially, gastric ulcers.[5] They asked the doctors the criteria by which they made these judgements and built an algorithm based on what the doctors told them. All they were trying to do was understand the doctors' thought process. They were innocent in the sense that they had no ambition to challenge the doctors' wisdom. They thought the doctors knew what they were doing. The doctors mostly did know what they were doing, except that once they had this model, this algorithm for diagnosing whether or not a gastric ulcer was cancerous, they were shocked to find that the model was better at predicting whether the tumor was malignant than the individual doctors were. And they were bewildered by this finding. This was back in the late '60s, early '70s, when Kahneman and Tversky were just getting going. And it happened in a place—Eugene, Oregon—where Kahneman and Tversky were. So they had this very crude model, built with the help of the doctors, that was outperforming the doctor; a model of man was outpredicting man. Kahneman and Tversky rolled in and they explained why. They explained the various things that were swaying doctors' judgements in real time, when they're presented with the x-rays, that might lead them to underperform the algorithm that the doctors themselves created.
Will Artificial Intelligence Help?
Dr Topol: With artificial intelligence, where do you think medicine is heading? Is that going to fix the problem or is it going to introduce new ones?
Mr Lewis: I'm not qualified to answer that question. But Amos had a very funny line when people asked him if his work had any bearing on artificial intelligence. He always said, "I'm much more interested in natural stupidity than I am in artificial intelligence." However, we are already at the point where there is a drift toward using statistical analysis and algorithms to diagnose patients, just like there is a drift toward using algorithms to evaluate baseball players and stock market strategies. If I had to guess, one day people will look back on the idea of a doctor in a room with a patient, rendering a diagnosis unaided by some statistical model, as archaic.
The other thing that enters into medicine and some other walks of human existence is new technology making it easier to render the diagnosis. With blood and fluid analysis, doctors are getting better information to make their diagnoses so that they will make fewer mistakes.
Dr Topol: That is a really good point. We now have deep phenotyping providing more in-depth information and the ability to use deep learning in analytics.
Moneyball and Medicine
Dr Topol: Here is a coincidence: A friend of mine here in La Jolla is Paul DePodesta.
Mr Lewis: Oh, really?
Dr Topol: You may remember him.
Mr Lewis: Paul DePodesta was played by Jonah Hill in the movie [Moneyball].
Dr Topol: Yes. He's with the Cleveland Browns now but has been with many different sports teams. I guess when you got to know him he was with the Oakland A's. But he does not look like Jonah Hill; he is really a fit guy. He has helped us because he is outside the forest and really into algorithms with his analytical mind. In medicine and biomedical research, we just do not think like that. So he has been a really interesting force for us. I just thought it was funny because if you put Jonah Hill and Paul together, you would say they looked diametrically opposite in body habitus in every way.
Mr Lewis: Yes, it was an odd choice of casting but it was unbelievable how good Jonah Hill was.
Paul introduces that element that he introduced into baseball—questions that you think are settled questions are not necessarily settled questions. An old-time baseball person before the Paul DePodesta era would say, "I know everything about baseball." Paul DePodesta would say, "No, there's an awful lot we don't know about baseball." He would say that the baseball field is a field of ignorance and that we should maintain a spirit of inquiry in the face of problems like how to value a baseball player.
It gets back to Don Redelmeier, who would say the same thing about medicine. The doctor is not infallible and does not know everything, and it's a shame that we need to thrust him into a position where he has to seem infallible in order to make everybody feel good. Because we are much better off if he's very aware of how highly fallible he is and how little we do know. That's how progress happens. You start by acknowledging what you do not know. I think a big thing Paul probably introduces to medicine is to ask really simple questions that everybody thinks they know the answers to when, in fact, the answers are more complicated.
Dr Topol: You nailed it; that is why he's so good for us. I guess you could say he's kind of the Don Redelmeier in our group.
I want to finish by saying that the contributions you've made by writing The Undoing Project, no less your prior books, really are impactful. It's great that our Medscape/WebMD community can tap into this in the hope of slowing things down. Diagnosis, as you can appreciate, is really the gateway to everything in medicine, and we have so much room to improve. It's great that you spent those 7 years working with Danny and Amos to understand how that whole field came together.
Thank you so much for this work and for the chance to discuss everything with you today.
Mr Lewis: Thanks for taking an interest.
Medscape © 2017 WebMD, LLC
Any views expressed above are the author's own and do not necessarily reflect the views of WebMD or Medscape.
Cite this: Moneyball Author Michael Lewis on Errors and Medical Misdiagnosis - Medscape - Nov 27, 2017.
Comments