Why Most Clinical Trials Fail: The Case of Eritoran and Immunomodulatory Therapies for Sepsis

The experimenter's view of the trees.

The ACCESS trial of eritoran in the March 20, 2013 issue of JAMA can serve as a springboard to consider why every biological and immunomodulatory therapy for sepsis has failed during the last 30 years.  Why, in spite of extensive efforts spanning several decades have we failed to find a therapy that favorably influences the course of sepsis?  More generally, why do most clinical trials, when free from bias, fail to show benefit of the therapies tested?

For a therapeutic agent to improve outcomes in a given disease, say sepsis, a fundamental and paramount precondition must be met:  the agent/therapy must interfere with part of the causal pathway to the outcome of interest.  Even if this precondition is met, the agent may not influence the outcome favorably for several reasons:

• Causal pathway redundancy:  redundancy in causal pathways may mitigate the agent's effects on the downstream outcome of interest - blocking one intermediary fails because another pathway remains active
• Causal factor redundancy:  the factor affected by the agent has both beneficial and untoward effects in different causal pathways - that is, the agent's toxic effects may outweigh/counteract its beneficial ones through different pathways
• Time dependency of the causal pathway:  the agent interferes with a factor in the causal pathway that is time dependent and thus the timing of administration is crucial for expression of the agent's effects
• Multiplicity of agent effects:  the agent has multiple effects on multiple pathways - e.g., HMG-CoA reductase inhibitors both lower LDL cholesterol and have anti-inflammatory effects.  In this case, the agent may influence the outcome favorably, but it's a trick of nature - it's doing so via a different mechanism than the one you think it is.

Herein I will argue that enormous biological complexity severely limits our understanding of the causal pathways of most diseases; that many agents studied for the treatment of disease do not meet the causal pathway precondition or are subject to the above limitations; that our hopeful quest to discover treatments for disease leads us to markedly overestimate the likelihood that any variable or pathway we discover and study is part of a causal pathway to outcomes that are of clinical interest.

Consider the simple causal pathway in Figure 1.

FIGURE 1

We know with reasonable certainty that S. pneumoniae can breech the body's defenses and cause clinical illness and sometimes death.  We also know with reasonable certainty that administration of antibiotics can often arrest this causal pathway and avert death, presumably by inhibiting multiplication of S. pneumoniae.  What we do not know are the identities of the intermediary steps represented by X, Y, and Z; how many intermediary steps there are; whether there are redundant pathways; the timing of the causal sequence; whether antibiotics have effects on other pathways; and the extent of associated epiphenomena.

Next consider Figure 2.  S. pneumoniae triggers a causal cascade that leads to death, but it also triggers several other sequences that do not lead to death, represented as A, B, C and J, K, L.  (These factors could be cytokines or any other immunological molecules.)  Suppose that investigators researching clinical illness with S. pneumoniae discover these nine molecules and further research shows that, compared to levels of these molecules in patients without disease, levels in patients with clinical infection are elevated and positively correlated with the probability of death.  Naturally we hope that the molecules we have discovered and can measure are part of the causal pathway to death from the illness, but we have no way of knowing based on these correlations.  In this simple example, only a third of the molecules discovered and correlated with death are causal intermediaries.  The other two-thirds are epiphenomena, and their manipulation by any means will not affect the causal pathway to the outcome of interest.

The omniscient's view of the forest (simplified).

FIGURE 2

Much time, effort, money, and resources are devoted to discovery and characterization of a given molecule in the study of disease.  When one is discovered, it is the natural hope of the investigator(s) that the molecule not only is critical for understanding the biology of disease, but also that it will lead to a treatment for the disease under study (or another disease).  But until the causal pathway is delineated, we have no way of knowing if the molecule we are studying is a causal factor or an epiphenomenon, even if its levels are strongly correlated with outcomes of interest.  It is only after the development of an agent that interferes with the molecule under study and its testing in a randomized controlled trial in humans that we can determine whether that molecule is likely to be part of the causal pathway in humans.   This problem represents a paradox:  investigators learn that a molecule is epiphenomenal only after they spend years characterizing it, developing a way to modify it, and testing it as a treatment for disease in Phase I, II, and III clinical trials.  During that process, the possibility that the molecule under study is simply a marker for disease is lost in the hope that they are pursuing a treatment for disease.  The possibility of finding only evidence of an epiphenomenal role is the epitome of cognitive dissonance.

An important but currently unanswerable question is, what is the ratio of [discovered] epiphenomena (markers for disease) to [discovered] causal intermediaries at any give point in history?  How many branches similar to A, B, C, and J, K, L in Figure 2 are present?  How long is the sequence represented by X, Y, Z?   If we knew these answers, we would have a better idea whether any molecule we discover that is correlated with outcomes of interest is a causal one.

The fact that so many (all?) immunomodulatory therapies for sepsis have failed suggests either that the ratio of epiphenomena to causal factors is high, that redundant mechanisms and causal pathways are common or highly time dependent, or a combination of these possibilities.  Regardless of which may be the case, the historical record of such therapies suggests that the probability of finding a variable that is not epiphenomenal and is not subject to redundant pathways, mechanisms, and/or insurmountable time dependence is very low indeed.  The same problems likely bedevil the investigation of variables and therapies in other diseases and disciplines.

This post, I'm afraid, will serve as little consolation to investigators of therapies for sepsis and other recalcitrant diseases.  But my purpose here is not to solace but to try to understand the evidence and its limitations, and to use this understanding to predict the results of future investigations and temper expectations with reality.  As far as immunomodulatory therapy for sepsis is concerned, the evidence speaks for itself: positive associations between known variables and outcomes of clinical interest are either epiphenomena or are intransigent because of redundancy of mechanisms, redundancy of causal pathways, or time dependency.  Finally, "proof" of causation is subject to the limitations of multiplicity of effects of "independent" (controlled) variables.  These problems with sepsis therapies provide a case study for therapies in other diseases.

(See also:  Why Sepsis Trials Fail by Roger Bone, JAMA, 1996; and an editorial by Opal and LaRosa in AJRCCM June 1, 2013.)