Our challenge – the long tail


Category : Medical Rants

I listen to books while driving. Given the amount of time that I spend driving, I listen to many books. While I usually listen to fiction (especially mysteries), today I started listening to a non-fiction book – The Long Tail. Recently the Health Care Blog had an interesting entry about the long tail and medicine – Healthcare and The Long Tail – Searching for help when you’re on the wrong-end of the curve

In a recent THCB post, author Maggie Mahar writes that “Ambiguity haunts medical care”. She goes on to quote Dr. Atul Gawand “Uncertainty is the core predicament of medicine . . . the thing that makes being a patient so wrenching, being a doctor so difficult and being part of a society that pays the bills so wrenching.”

It’s important to note that for a great many cases, ambiguity is not really an issue. This is because the distribution of medical ailments follows a curve very similar to Chris Anderson’s The Long Tail, with a great many common blockbuster ailments stacked up high on the left-hand side of the curve. For those not familiar with the Long Tail, Anderson describes how Amazon, Netflix, and other online retailers sell lots of the usual blockbusters, but actually derive more total volume from 100s of thousands of niche products. In healthcare, it is the left side of this distribution curve which inspires (for better or worse) Wal-Mart, Target, and others to offer “Doc In A Box” services – Allergies, Bladder Infections, Bronchitis, Ear Infections, Pink Eye, Sinus Infections, and a full battery of vaccines – all served up for a fixed price while you wait.

On the right hand end of the curve though, the NIH Office of Rare Disease classifies over 6,000 conditions, each afflicting fewer than 200,000 Americans. Along this part of the curve, things do indeed get very ambiguous in a hurry “ both for patients and physicians. Specialization is a response to this range of ailments (nichefication in Anderson’s terms), and brings physicians repeated cases of a particular nature “ giving them the confidence that they can routinely diagnose and treat a high percentage of these patients. However, even within a particular specialty area, cases will naturally follow a distribution curve from typical to atypical. Unto themselves “ atypical cases are just that “ one of a kind aberrations that force physicians to go outside their typical comfort zone of diagnosis and treatment. For each individual physician, these atypical cases feel like the exception rather than the rule. What the Long Tail suggests though, is that taken in their entirety, these rare cases actually compromise a large percentage of all medical cases. In fact, over 25 million Americans suffer from a rare condition.

In academic medicine we try to make sense of the long tail. We physicians have a great challenge. When patients present with routine complaints, we have to quickly know whether the patient fits the left hand side of the curve, or the right (long tail) side. Our challenge is in knowing when the patient does not fit routine.

The best physicians understand this concept. They work hard to avoid “premature closure” in diagnosis. Retired Doc wrote about this phenomenon last year – Medical errors and role of premature closure

The authors state that defective knowledge as a cause of error was rare and more commonly reflected problems with synthesis of available information.This refers to a formulation of how humans solve problems namely by searching for an explanation that best fits and then the search stops. Physicians at all levels of training and experience do this.
Are there take-home messages here? With premature closure, the suggestion is to make a conscious effort to not completely stop after you reach a diagnosis but ask ” what alternatives should be considered?” This could be done initially and from time to time as the clinical case plays out. Some times things seem so obvious we do not do that but that practice seems like a good mental back up mechanism to minimize errors.

We cannot provide enough attention to the long tail of medicine if we do not give physicians enough time to think. Currently many physicians limit their thought processes because of time constraints.

Given that the long tail is very important (as The Health Care Blog describes), we must work to prevent long tail mistakes. Long tail mistakes are costly (both in terms of money and health). We probably should more systematically determine those costs. I am finding the book fascinating. Understanding the concepts are important if we want to understand quality health care.

Comments (12)

“What the Long Tail suggests though, is that taken in their entirety, these rare cases actually compromise a large percentage of all medical cases. In fact, over 25 million Americans suffer from a “rare” condition.”

The majority of he time it’s okay if your dr is arrogant, obnoxious, or short on time because they are usually right. They went to school to becomer the experts, they have a much better idea of what is wrong that the patient does. They don’t really need your input or insight given your relative level of ignorance.

Until you become a member of this “long tail” I guess. Then you begin to see how different drs respond to not having a clue. Some handle it gracefully and comfort you, some start to avoid eye contact and try and get rid of you faster, some are rude and abrupt and yell at you, some treat you like you have a mental problem. Most can’t admit they don’t have an answer.

When they most need to listen, they become least likely to. It is a blow to the ego not to know the correct answer. When they most need patient input, when they most need to step out of the diagnostic algortithm box and think creatively, they become most likely to end the appt quickly and send you off to another dr. You begin to take a ride on the Dr-go-round.

What is most disappointing, as a scientist, is the absolute lack of scientific thought most of my physicians exhibit. If the answer doesn’t pop out of an easy-bake box they don’t know what to do.

Numerous clues present themselves and leave a puzzle waiting for a solution yet they choose to disregard them and send me out the door hoping my symptoms don’t return. Rather than think a little and figure out the underlying problem, just throw meds at the symptoms.

Even if the patient is just a diagnostic puzzle you guys display a rather sad lack of interest in the more odd puzzles. In science the odd piece of data usually indicates that your theory is faulty and needs to be reevaluated.
In medicine your patient just needs to go away.

In medical school, very long ago, my phsical diagnosis instructor said, “If you don’t have a pretty good idea what’s going on with your patient after a good history and physical, you probably never will.” Not sure if it was ever entirely true; but, along with the aphorism “rare things are rare and common things are common,” I think it gives a sense of how things are. It’s indeed true and demanding of the best in us that when a patient falls outside a standard deviation or two, multiple factors combine to make it less and less likely that answers will be found.

Dear Tina,

I’m sorry you are having a bad time with doctors. It is probably illogical however for you to lump us all together. I doubt that you had an initial visit with a rare disease to every doctor in the country.

I have had a similar issue with patients who present to my private practice internal medicine clinic. Less than 1% of the time after testing and referrals I tell them i don’t know what they have and recommend they get a second opinion. They look at me like I am crazy. I suppose what they are thinking is either they don’t have anything or they have something rare and strange and don’t know where to go. Often I recommend the local university center. If you haven’t been to the nearest University center, I would recommend you go. The internal medicine department is probably the best place to start.

Like Tina, I’m a member of the Long Tail Club, patient division. And my experience getting an accurate diagnosis was similar to hers.

In my case, the initial misdiagnosis of a lumbar disc herniation by my then family practice doctor meant that I was sent to inappropriate specialists. Who then proceeded to order numerous treatments and tests. The former were ineffective, the latter inconclusive. The net result was that each specialist sent me back to the family doctor, saying either there was nothing wrong with me, or insinuating that I was a malingerer and/or a depressed whiner. The net result of this was that my doctor lost interest in helping me, and I gave up looking for help.

Now mind you, during this time I didn’t realize that I might have an uncommon condition. I simply trusted that the specialists that I saw (75% of which were at academic centers) would keep looking until a reason for my unusual set of symptoms was found. Instead, they became uninterested within 1-2 office visits.

I was finally diagnosed with a peripheral nerve disorder by a rehab specialist, one who took the time to listen to my history and symptoms. Oddly, he was not at an academic center, but instead in a private group practice. And he is a graduate of a Caribbean medical school, though he did work his way up to doing a fellowship at Johns Hopkins.

My experiences have been that academic physicians are no more astute than those in private practice. They just have better medical pedigrees.

What we patients need is a directory of physicians who have good clinical judgment, who know how to stay the course and who aren’t quitters. Would that such a directory existed.

After a few days of thought…
Walmart sells the most popular books at high volumes in a store as it has to maximize profit with respect to shipping and logistics.

Amazon can sell many “rare” books due to not having to deal with “stocking the shelfs”. You assume the consumer knows what they want.

Drs also have to work within logistics and profit on some level. Even specialists are still generalists in the sense that many of thier cases are still pretty routine. So the avarage doc rightly works in the high volume, routine case enviornment.

To simplify treatment of the “long tail” oddballs perhaps we could shift to an “amazon” style treatment model of physician-patient interactions via email and phone for the physicians that are the true experts for the rare disorder with six month or yearly visits in conjunction with a primary care doc. It becomes a matter of “knowledge management” for you guys. Diagnosis is still a problem but it would simplify treatment.

As for diagnosis you might try listening to whatever odd ideas the patients bring up. I know … it’s hard… but When all else fails they may have found the right idea on the all knowing google.

[…] is precisely the issue that the Long Tail or Pareto Principle predicts. Our challenge – the long tail Our training takes time because it takes time to see enough […]

[…] the minds of physicians when they put the pen to the paper.  In America, we strive to exclude  the long tail diagnosis.  Why? Because getting sued for 67 million dollars because you treated a torn aorta when all the […]

[…] The problem is in the problem representation.  If we stop our history taking at the words "sore throat", then we accept the term "Just a sore throat".  This thinking is seductive, because often this short cut works.  Most sore throats are unimportant.  Our guidelines and performance measures focus on the short head of sore throats and ignore the long tail. […]

[…] all miss diagnoses.  But well trained physicians have a better chance to know when the "long tail" of medicine involves this patient at this time. […]

[…] the years, I have written about the short head and long tail. You can use the search function to find several posts since […]

[…] problem here is a long tail problem.  As physicians, we strive to know when a patient’s presentation is not routine.  Our […]

[…] job as physicians is to recognize the long tail diagnoses.  Upper respiratory infections have well defined warning signs.  On the podcast I spend much time […]

Post a comment