Friday, February 28, 2014

Overediagnosis and Mitigated Overdiagnosis: Ongoing problems with Breast and Lung Cancer Screening

I got to thinking about cancer screening (again) in the last week after reading in BMJ about the Canadian National Breast Screening Study (CNBSS).  That article piqued my interest because I immediately recalled the brouhaha that ensued after the U.S. Preventative Services Task Force (USPSTF) recommended that women not get mammograms until  age 50 rather than age 40.  That uproar was similar to the outcry by urologists when the USPSTF recommended against screening for prostate cancer with PSA testing.  Meanwhile, changes in the cholesterol guidelines have incited intellectual swashbuckling among experts in that field.  Without getting into the details, observers of these events might generate the following hypotheses:
  1. People are comfortable with the status quo and uncomfortable with change
  2. People get emotionally connected to good causes and this makes the truth blurry, or invisible.  After you've participated in the Race for the Cure, it's hard to swallow the possibility that the linchpin of the Race might not be as useful as we thought; and is no longer recommended for a whole swath of women. 
  3. People are terrified of cancer
  4. Screening costs money.  Somebody pockets that money.  Urologists and radiologists and gastroenterologists LOVE screening programs.  So do Porche dealers.
The CNBSS tracked the outcomes of almost 90,000 women 25 years after they entered a study that compared annual breast examinations to annual examinations and mammography.  (It is important to note that the study did NOT compare mammography to nothing.)  The analysis of the prolonged follow-up shows the same thing as earlier analyses:  that there is no breast cancer specific mortality or all cause mortality benefit from screening.  The authors attribute these findings (partly) to the availability of adjuvant therapy for breast cancer during the period of study and follow-up - when the treatment for something improves, it matters less if you catch it early.

The other finding that this prolonged follow-up reveals is overdiagnosis, which some have said is a misnomer. "Diagnoses without consequences" or "inconsequential diagnoses" have been suggested as alternative ways of describing these cancers, but I disagree.  The diagnosed cancers would have had no consequences had they remained undiagnosed.  But once they are diagnosed, there are consequences.  In an interesting study entitled Cure Me Even if it Kills Me, investigators demonstrated that people wanted cancers cut out or treated even when presented with statistics suggesting that their outcome would be worse by doing so than if they left the cancer alone.  So indeed, overdiagnosis is an important side effect of cancer screening - because folks who are overdiagnosed will undergo unnecessary treatments and psychological harm from the diagnosis.  The overdiagnosis rate for mammography is on the order of 25% or more.

Overdiagnosis is said in the abstract to be an inconsequential diagnosis because a.) the cancer will regress or  not progress; or b.) the patient will die of something else before the cancer manifests symptomatically or clinically.  (Imagine a patient just completing the last round of chemotherapy for a cancer and then being killed in a motor vehicle accident on the way home from the final appointment.)  Overdiagnosis requires prolonged followup to discern in the data because enough time has to elapse for all of the "consequential" diagnoses show their consequences.  Assuming they all have at the end of prolonged follow-up, any excess in the number of cases of cancer in the screening arm represents overdiagnosed cases.  This is the excess from screening.  There is a similar number of such "clinically silent" cancers in the unscreened group, but they remain hidden.

It seems possible, then, that because of overdiagnosis and lack of efficacy compared to annual breast exams, mammography may be, on balance, harmful.

I then learned that the USPSTF has recommended CT scan screening for current and former heavy smokers.  This in turn piqued my interest in the National Lung Screening Trial that was published in NEJM in 2011, but which I didn't give much attention to at the time.  I should have, because there are several interesting findings in that trial which have captured the attention of astute editorialists and experts.

This was a large (53,454 patients) randomized controlled screening trial comparing CT scan with Chest X-Ray in "high risk" patients (30 or more pack years of smoking and current smoker or quit in the last 15 years) that used lung cancer specific mortality as the primary endpoint, but that also followed all cause mortality.  The period of follow up was a median of 6.5 years, which is not long enough to comment definitively on overdiagnosis, although it was probably there.  Lo and behold - both lung cancer specific mortality (that is, people who died as a result of lung cancer) and all cause mortality were reduced in the screening group.  (There is an ongoing debate about whether cancer-specific or all-cause mortality is the most appropriate endpoint for cancer screening trials:  here are the Pro and Con positions.  The debate hinges on whether you worry about ascertainment bias in the determination of causes of death - and possible other issues as alluded to below.)

This post on PulmCCM describes the high number needed to scan (NNS) to prevent one death from lung cancer (about 950 scans [it has been pointed out in the comments below that I have misconceptualized this - What I really mean to describe is the NSN - the Number of Scans Needed to prevent one death]), the high false positive rate of abnormalities detected with CT scanning (about 95%!) and the high percentage of people getting scanned who had a positive finding on one or more scans.  Here are the deaths that occurred in each category and the differences:

Both differences are statistically significant (although all cause mortality excluding lung cancer is not).  But pray tell, dear reader, how can CT screening for lung cancer save 87 deaths from lung cancer AND an additional 36 (123-87; 40% more) deaths from causes other than lung cancer (even though it lacks statistical significance)?  No explanation is offered for this curious finding in the article or any editorials I have read in relation to it.  But I have an intriguing possible explanation.

Because the randomized controlled design of modern cancer screening studies obviates concerns about lead time and length time biases, it has been taken for granted that it is an unbiased way of studying the screening question/problem.  But it is hardly unbiased.  People who are being screened by a given modality know their treatment/screening assignment and their behavior may change as a result of this knowledge.  Moreover, since 39% of people in the CT screening group had one positive test (which was highly likely to be a false positive, but which probably magnified cancer anxiety nonetheless), there was an extra impetus for this group to change their behavior as regards smoking (half of participants in each group were current smokers at baseline).  Compare this to a 16% rate of positive chest X-Rays in the control group.  Insomuch as getting a positive scan may serve as a "wake-up call" to change smoking or other behaviors, the differential effect of this lack of blinding in the NLST and other cancer screening trials cannot be dismissed.  And because smoking has so many hazards beyond lung cancer, it is certainly possible that changes in smoking status, if there were any, could have led to reduced all cause mortality as well, or at least contributed to it.  If data on smoking status at the end of the follow-up period in NLST are available, this hypothesis could be tested.

Overdiagnosis poses a particular problem in lung cancer screening that is quantitatively and qualitatively different than the one it does in breast cancer.  Firstly, the prognosis of all comers with lung cancer is much worse than that of all comers with breast cancer.  Secondly, the prognosis of smokers without cancer is worse than that of women without breast cancer.  Thirdly, the attributable mortality due to smoking is about equal parts cancer, heart disease, and emphysema.  Overdiagnosed patients with lung cancer have all sorts of risks of dying from other causes, especially if they continue to smoke.  Overdiagnosis has bigger implications if your risk of dying from other things is higher - this is why we shouldn't do screening colonoscopies in patients with advanced age or poor functional status.  And yet the USPSTF recommended CT screening for lung cancer up to age 80!

I remain confused about one aspect of all this screening business.  What if lung cancer screening does prevent lung cancer mortality, but it doesn't change the other kinds of mortality.  What if, in essence, we save a lot of people from lung cancer who then die from a heart attack 6 months later?  This seems to be an argument for focusing on all cause mortality, but I have not yet been able to find it articulated in this way.  If my thought experiment is correct, this is a bigger issue as we focus on those at higher risk of lung cancer, if their general health status is poorer and their risk of dying from other things likewise higher.  Granted, if you collect data long enough, all survival curves converge.  But perhaps "overdiagnosis" as a concept should be extended to those cases which, although cancer survival is prolonged, overall mortality from other causes which occurs only slightly after the expected time of death from treated or untreated lung cancer mostly negates the extended survival from diagnosed and treated lung cancer?  Let's call this "mitigated overdiagnosis."  This is something which merits further study, it it has not already been investigated.

In any case, whether the USPSTF should have recommended CT screening in the form they did is not at all clear to me.  It is recommended that we, as practitioners, discuss the implications of this study with our patients.  Here is how that conversation might go, as regards the big points:
  • Far and away, the best thing you can do to improve your health is to stop smoking, not go get a CT scan for screening purposes
  • For each CT scan we do for you, there is a 0.3% chance of finding a lung cancer that would have killed you if we hadn't found it.  If we do 3 scans, we reduce your chance of dying from lung cancer by about 1%
  • For each CT scan we do for you, there is about a 25% chance that we're going to find something abnormal that we're going to have to pursue further with more testing, but that is not cancer.
  • Whenever we find something abnormal on your CT scan, there is a 95% chance that it's nothing to worry about, a false positive.
  • Even if we do find and treat a lung cancer on the basis of screening, if you continue to smoke, you remain at risk of mortality from other causes such as heart disease, emphysema/COPD, and all the other maladies that continue to plague humanity
How many patients, hearing these statistics, would wish to proceed with screening?  If they are risk adverse enough that a 0.3% absolute reduction in lung cancer mortality is important to them, would it be better to stop smoking and call it a day?

If ignorance is bliss, then 'tis folly to be wise.  And ignorant of your smoking status, you are not.


  1. Well put Scott. I appreciate your ability to distill the relevant statistics and subsequently articulate them in a manner that non-MD's can grasp.

  2. We are implementing a lung cancer screening program in our hospital, but my concern is that our population is different to NSLT, we are in the Ohio Valley and I see lots of farmers (histoplamosis), factory workers, poor people on Medicaid. The participants in the NSLT were healthier and probably more educated. We can potentially, have more false positives, more overdiagnosis, worse mortality (all cause deaths). I wonder what could happen in places where whites are a minority. Latinos or asians with h/o TB, poor african americans, etc. I think than more than 90% of subjects in the NSLT where white.

  3. Dr. Aberegg,
    I have written a response to address the many misconceptions in your post about screening. I address both the Canadian National Breast Screening Trial and the National Lung Screening Trial. Please see my response in
    Guest Post: Does Cancer Screening Cause “Overdiagnosis”?

    1. You should make that link hot, if you can figure out to like my spammers all have.

      I appreciate your interest in these topics. There are several things to say about risk and its reduction, and points to clarify.

      Firstly, we should NOT ever talk about relative risk reduction. It inflates the perceived benefit. A person should be concerned with ABSOLUTE risk. The 20% RRR in the NLST exaggerates the MINISCULE absolute risk reduction which is more on the order of 0.5%.

      Next, I have to thank you for discovering a semantic error in my choice of words for the number needed to scan. What I meant to say is the number of scans needed (let's call it NSN). You had to screen 320 people, but you had to do 3 scans each. So, if we're interested in discussing radiation exposure (something elected to ignore), the NSN is a more valid metric than the NNT converted to NNS (which distinction recognizes that we're not "treating" but rather "scanning" or "screening".

      As regards age, I defer a full discussion to a pending post on Status Iatrogenicus I've been pondering for some time that will be entitled "Age Matters" or some variation thereof. I will simply say that age does matter, and if you're going to die of a heart attack at the end of hte year, you would not want to spend the beginning of the year getting testing and treatment for lung cancer which may extend you life only long enough to die of the heart attack. Since you're asymptomatic, it might be better to remain asymptomatic for the rest of the year, have no biopsies, no surgery, no radiation, no chemo, and then die suddenly, one month early. Trade-offs are important. In the end of the paragraph about age, you highlight that the USPSTF, like I, recognizes that age matters.

      Your patient discussion differs from mine. In yours, you are "selling" lung cancer screening. In mine, I'm saying "it's for sale" but giving actual statistics about the possible outcomes in different scenarios. I didn't state this, but the statistics I give are actually the conditional probabilities (given X, probability of Y). These are the actual statistics that are necessary for a rational decision maker to make a decision.

      I'm sorry that your husband fell victim to lung cancer. Lung cancer sucks. I deal with it all the time and I truly am sorry and sympathize with you.

      But my blog has a stated purpose to rationally search for the truth.

      At the end of the day, this MASSIVE screening experiment, enrolling 50,000 patients and following them for 6.5 years, costing untold millions of dollars, saved perhaps 120 lives. That, I'm sorry to point out, is a drop in the bucket, but the expense is not.

      If our goal is to save lives, not just save lives from a particular disease because it is has afflicted someone who is dear to us [aside: in a different life, for different people, in other realities, a different disease may have afflicted our loved one] 40,000 people die needlessly on American roadways each year. We could reduce the speed limit; raise the minimum or make a maximum driving age; enact universal helmet laws; etc, and save, at MUCH lower cost, FAR more lives than by scanning everybody for lung cancer.


  4. 160,000 people a year die of lung cancer in the US, 4x your comparison to 40,000 hwy deaths. As for absolute vs relative risk, I totally understand the difference (I am an engineer) Based on absolute risk nobody would ever get screened, adopt healthy behaviors or buy a lotto ticket; that's why we use relative risk. My goal is not to save maximum lives due to any cause. My goal is to advocate for fairness for lung cancer patients who have gotten the short end of the stick compared to other high burden (individually and to society) diseases due to the stigma associated with smoking. And before you lecture me on personal responsibility, 90% of all regular smokers start smoking before the age of 18 targeted by tobacco companies sucking them into a lifetime of addiction at an age when experimentation often trumps judgment. Smoking has been shown by epidemiological studies to be more addictive than heroin. So until we see 5 year survival rates over 50% for lung cancer, I will continue to advocate for lung cancer screening including research and modeling to better risk stratify the population to screen. I am back in school to get my MPH to make me more effective in my mission.
    I also fully appreciate the QOL discussion. My 81 year old mom is days from death due to tongue cancer. She has huge metastasized tumors on her neck. She elected no treatment due to her comorbid conditions. But it was her decision. Her Doc wanted to treat her. He did not think her comorbidities were sufficient to withhold treatment. Age by itself should not be the final say. Screening and treatment should be a joint decision with patient and the Doc.
    ps I'm not a spammer. No idea how to make link hot

    1. The Absolute/Relative distinction is totally pivotal. If one is to start at the position "we should screen" and then look for reasons that will convince people, relative risk is one way to do that, but it is not, in my opinion, an honest and transparent way.

      If instead we start from the position that we ought to decide for ourselves or to help others decide, based on a rational thought process, then absolute risk reduction is the critical metric to consider.

      It matters little if other preventative measures also have a small absolute reduction in risk, and that people won't do them if presented with absolute risk, because: 1.) autonomy gives them that right; 2.) autonomy is based on full knowledge and understanding. Unless we wish to paternalistically guide people to behaviors that WE think are good for them without real regard to THEIR values, absolute risks are what we must focus on.

      What we ought to do to resolve some of this is to look at "revealed preferences" and "willingness to pay" for certain risk reduction measures and then help people to compare them with other things that they're not doing, so that they can see any discrepancies if there are any.

      Moreover, if something is easy, or cheap, or without side effects, then small absolute risk reductions are still enough to motivate the behavior of the rational decision maker. The cost-benefit analysis is still favorable.

      But when things are not cheap or easy or benign, we have to look at trade-offs. Especially when society, not the individual is going to pay for them. Because society has to choose between competing alternatives for spending finite public resources.

      But you say you don't care about competing alternatives because you have a personal stake here, an emotional investment in lung cancer screening - so in some ways we realize we're speaking different languages. I the language of rational evidence appraisal, you the language of lobby for a cause.

      And it's fine that we disagree on those bases. I just want there to be no confusion about the facts.

    2. In this post:

      I show some of the discrepancies I'm referring to above, and the kind of analysis based on absolute risk I'm advocating.

      If you wish to use relative risk, you are building in a bias to the decision making process that favors treatments with SMALLER EFFECTS! Why?

      Because as the mortality somebody is facing increases (making treatment all the more important) any absolute efffect is proportionally smaller than the relative effect.

      Likewise, as the mortality burden decreases (meaning that treatment is less important, you might well get by without it), a small absolute risk reduction has a correspondingly LARGE relative risk reduction.

      The use of Relative Risk Reduction biases the evaluation and has no role in rational decision making about these kinds of choices.

  5. Alas, CMS has denied coverage for lung cancer screening - there's hope for reason after all....
    Medicare bucks USPSTF, denies coverage for lung cancer screening