- As I have blogged in the case of intensive insulin therapy, Single center studies inflate treatment effects when compared to multicenter studies for reasons that are unclear, but which may be related to bias especially in unblinded studies. (The revelation that Rivers was an investor in one of the devices used in the trial raised additional concerns about bias.)
- Regression to the mean may lead to reduced effect sizes when trials are repeated, especially when the index trial has a very large effect size. In a similar vein, since large absolute mortality reductions are statistically unlikely in critical care medicine, Bayesian inference means that trials reporting large reductions are likely to represent type I statistical errors.
Thursday, March 20, 2014
Sepsis Bungles: The Lessons of Early Goal Directed Therapy
ProCESS trial of protocol guided therapy for early septic shock. This trial is in essence a multicenter version of the landmark 2001 trial of Early Goal Directed Therapy (EGDT) for severe sepsis by Rivers et al. That trial showed a stunning 16% absolute reduction in mortality in sepsis attributed to the use of a protocol based on physiological goals for hemodynamic management. That absolute reduction in mortality is perhaps the largest for any therapy in critical care medicine. If such a reduction were confirmed, it would make EGDT the single most important therapy in the field. If such reduction cannot be confirmed, there are several reasons why the Rivers results may have been misleading:
There were other concerns about the Rivers study and how it was later incorporated into practice, but I won't belabor them here. The ProCESS trial randomized about 1350 patients among three groups, one simulating the original Rivers protocol, one to a modified Rivers protocol, and one representing "standard care" that is, care directed by the treating physician without a protocol. The study had 80% power to demonstrate a mortality reduction of 6-7%. Before you read further, please wager, will the trial show any statistically significant differences in outcome that favor EGDT or protocolized care?
Indeed, as I wagered before reading the results (with my wife as a witness while we were vacationing in Panama), such differences were not found and there was essentially no signal favoring either of the protocols in this multicenter trial. While this could be because clinicians in the standard therapy arm were essentially doing EGDT because of its promotion as the standard of care for the decade leading up to the ProCESS trial (let's call this "therapeutic drift"), my conclusion is the same - we don't really need EGDT, a "bundle" of buzzwords, repeated lactate measurements, CVPs, Edwards Lifesciences catheters, ScVO2 measurements, dobutamine, transfusions, etc. We simply need to be attuned to the early identification and rapid treatment and resuscitation of sepsis. The editorialist says that this is a "dubious conclusion." I disagree vehemently. What is the point of doing this trial if we're going to ignore the negative result? Do we seek only to confirm? Would he have advised the ProCESS investigators to not conduct their trial because he will not recommend changing the current practice regardless of the result? That is dubious position. ("Never order a test if the result won't change your management.")
There are other lessons to be learnt by willing students of the epistemology and history of evidence in critical care in general and sepsis specifically. We have seen the rise and fall of drotrecogin-alfa and the promiscuous and corrupt influence of industry on professional societies and guideline development. We have seen the premature incorporation of evidence based on single center trials into practice guidelines, only to have the evidence later refuted at the multicenter level, the later evidence received with skepticism, and specious practices continued (insulin therapy). We have seen that few or no therapies in critical care reduce mortality by 5% let alone 10%, yet we continue to use mortality as the primary endpoint and power the studies for a mortality reduction of 10%. We have a recurring love affair with corticosteroids and an uncanny ability to read between the lines of negative studies to justify their use. And we enroll tens of thousands of patients into industry sponsored and controlled trials of immunomodulatory therapies for sepsis, knowing full well that industry will manipulate results, obfuscate uncertainties, and promote any approved molecule ferociously even though the prior probability of a Type I error is ridiculously high.
"Those who cannot remember the past are condemned to repeat it." We would perhaps be wise to incorporate the lessons of history into our interpretation and implementation of "evidence based medicine" lest we embark on other misadventures that we will realize were misguided only a decade or more in retrospect.