Getting “Testy” with Medical Testing!

Back in the 1700’s an English mathematician and Presbyterian minister named Thomas Bayes published two works in his life, one on math and the other on theology. Mr. Bayes never published the work he is most remembered for yet Bayes Theorem has been widely applied from mathematics to bioethics.

On the face of it Bayes Theorem isn’t that complicated. Essentially what the math symbols say is that predicting the future likelihood of any event is based on the present likelihood of that event and whatever evidence (test results) you have that might alter prediction of that future event.

Breast cancer offers a concrete example. To keep things simple, ignoring familiar cancer risk, the prevalence of breast cancer increases with age. A 75-year-old woman has a greater chance of breast cancer than does a 40-year-old woman all other things being equal. A clinician may do an examination in women these ages and find a lump yet the post-examination odds of breast cancer are going to be less in the 40-year-old woman because the pre-examination odds were lower to begin with.

Small numbers multiplied by typically small factors (the technical term is Likelihood ratios) render small results. If my odds for a disease are 1:1000 and a physical examination finding or test only contributes a multiplier (a.k.a. Likelihood ratio) of 2 then the odds of the disease remain low. Thought of differently, if my Likelihood ratio was 10 (and that’s an uncommonly good test) the change in probability of my subject having the outcome of interest increases 45%. That sounds like a lot but a 45% increase of a very small number is still a very small number.

Healthy people are well…healthy. The prevalence of many diseases just aren’t so common to make meager test characteristics add up to much diagnostic clarity. Herein lies the problem of screening healthy people. Among the sick some tests don’t improve upon what could be fairly well estimated based on cheaper studies. Bladder testing among low risk women is a good example. In this setting, basic office examinations have been shown to be as effective in predicting suitability for surgical treatment as expensive testing. Mr. Bayes was a genius.

Recently the American Board of Internal Medicine conveniently culled together the Five Things Physicians and Patients Should Question based on the evidence-based recommendations of specialty societies representing 500,000 physicians (check it out at Among the recommendations submitted by the American College of Obstetricians and Gynecologists was to not treat mild cervical dysplasia that is less than 2 years duration and not to screen for ovarian cancer in asymptomatic women at average risk. The reasoning behind these and the many other excellent society recommendations can often be traced back to Bayes Theorem. Either the prevalence of the disease is rare or the test/treatment performance is poor or both rendering Bayes equation to calculate it’s just not worth the effort. Indeed, beyond the cost of testing, false positive tests cause a lot of harm to a lot people every year.

Dancing gorillas say a lot about another problem with testing. In a recent study researchers snuck the image of a dancing gorilla on a radiograph that was later reviewed by radiologists. The radiologists did their job and reported what clinically they saw in the radiograph but when asked about the dancing gorilla they were stumped. Few saw the dancing gorilla. Is this really that surprising? Not really. There is collusion between our eyes and brain such that what we see is what our brain says is to be seen. Dan Gilbert’s book, Stumbling on Happiness nicely describes research supporting this eye-brain conspiracy. The problem of dancing primates is that a test might not always clarify one diagnosis or another but merely confirm the plan already hatched in the mind of the person ordering the test.

Such seems to be the case for a young woman I saw who had undergone three procedures to treat her bladder problem. She had undergone the usual testing to justify the procedures done. Surprisingly none of the tests confirmed the diagnosis pursued in doing the procedures. In other words, the tests didn’t matter. This scenario could be helped with a better understanding of Mr. Bayes theorem. For this young woman, is a troublesome bladder problem common such that three procedures would prove ineffective? Not so much. Herein was a red flag that the pre-test odds might make any test result suspect inspiring needed diagnostic and therapeutic caution. Understanding how common a disease is in your population can help you see the dancing gorilla.

I have surely abused some treasured statistical concept in my simplification of what can quickly become very complex. For example I have mixed odds and probabilities as if they are synonyms. They are related but importantly different. I also haven’t mentioned that tests results can be combined. Pre-test disease prevalence multiplied by test 1 and test 2 and test 3 may offer a remarkably good prediction for the presence of a given disease. Nonetheless Bayes theorem says a lot about the limits of evidence-based medicine even if it is at the heart of many of its recommendations. The “art” of medicine may be a clinician’s capacity to understand the prevalence of a given condition among her population. Good medicine will always rely on the merger of common sense and science and in a way this is what Bayes Theorem is saying.

Mr. Bayes’ ideas on conditional probability were not immediately embraced by his mathematics peers. Many clinicians would not embrace statistical methods in their daily practice. I am reminded of George Santayana who said, “Those who do not remember the past are condemned to repeat it.” In health care, for both patient and practitioner, Mr. Santayana’s sentiment echo Mr. Bayes theorem and could as much be reconfigured as, those who do no appreciate mathematics are doomed to spend lots of money.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s