How Doctors Think

Posted by Ben Simo

Michael Bolton recently brought Dr. Jerome Groopman’s newest book, “How Doctors Think”, to my attention. Michael suggested to the software-testing list that this book contains information that is relevant to software testing. In this book, Dr Groopman explores how doctors diagnose medical problems and what can be done to reduce bad judgment.

Michael Krasny recently interviewed Dr. Groopman on public radio about this book. You can listen to the interview here. Dr Groopman made a number of statements during this interview that I believe apply to us software testers’ search for and diagnosis of software bugs. I expect to find more in the book.

Here are some gems from the radio interview:

About computer-assisted diagnosis software.

It’s a two-edged sword. On one hand, it can be useful if you feed in the most important and relevant symptoms, but the problem is that you need to be able to illicit them effectively from the patient and so you return to language and you return to some of these thinking errors or these cognitive traps that we can all fall into. So if you put in for, example the first prominent symptom that you hear from someone into the software package, it may be that the fourth thing that the person complains about is the most relevant one. Recently you may have seen, for example, in reading mammograms, that computer assisted diagnosis actually generated more false positives. It caused the radiologist to think that shadows which he or she would normally think were benign were in fact cancer and didn’t add to the diagnosis of cancer –the true positives. So I think this technology is worth exploring, but I think that we have to be very careful about it because it some ways it can seem seductive. You cannot substitute a computer for a human being who listens and filters and judges and elicits from the patient the full story.

The same applies to test automation. It can be a useful tool if we don’t let it lead us astray. We human testers need to be actively involved in listening, filtering, judging, and eliciting the full story.

About time-constraints:

Its one of the biggest issues in the current delivery of medical care. You know, everyone is being told as physicians to see patients in 12 minutes. See patients in 11 minutes. So you’re sitting there with one eye on the clock and you can’t think well in haste. And there’s the natural inclination to anchor onto that very first impression as they stop there and just function as if you’re working on an assembly line in a factory. ... patients are being ill-served.
…
I think everyone in the medical system feels under siege. There’s not a lot of satisfaction but in a way this is penny-wise and pound-foolish because the cost of misdiagnosis is extraordinary. … It also costs in terms of dollars. Its much more
expensive to care for and treat an advanced problem than to make a diagnosis early on.

I think we have to force ourselves to resist and part of that can be done: sometimes we have to extend a visit. But we really are beholden to administrators and managed care plans, so one of the things that I’ve begun to do is if I can’t get to the bottom of a problem in my 15 minutes allotted visit,
then I say to the patient, you know, I haven’t figured it out. I need to think more. ... reschedule an appointment and spend more time.

Testers also make mistakes under time pressure. We also need to be aware that our first impressions may not be right. Testing is not a factory assembly line (even though this is a widely-held view). Testers need to be engaged throughout testing and not just mindlessly follow test scripts. And sometimes we need to lobby to spend more time than the schedule originally gives us.


About the art vs. science of diagnosis and improving the effectiveness of that diagnosis:

There’s a seductive nature of numbers, but statistics from clinical studies and so on are just averages and they may not reflect the individual in front of you. ... So real judgment involves customizing medicine: looking at scientific data but also seeing how it applies or doesn’t apply to the person sitting in front of you.
...
It’s not technology, but its language. Language is really the bedrock of medicine. Most of what we do with a doctor – or should be doing with a doctor – is talking… engaging in a dialog.
...
... all of us are wired to make these kinds of snap judgments to sort of go with our first impressions. to rely very heavily on intuition. … That often works but in medicine unfortunately too often it doesn’t work because we have a tendency to latch on to that first bit of information that we get from a patient and then run with it; as opposed to keeping our mind open and constantly doubting ourselves.

Testing is not an assembly line process. Testers need to keep their minds engaged throughout the testing process. We should not ignore our snap-judgements (blink testing), but we also need to look and think beyond those first impressions. We need to continually question both the software and our own judgement as we test.

We need to communicate more than numbers. We need to communicate stories. We need to translate bugs and metrics into language that matters to the business.