Ruling Out PE in the ED: Critical Analysis of the PROPER Trial

This post is going to be an in-depth "journal club" style analysis of the PROPER trial.

In this week's JAMA, Freund et al report the results of the PROPER randomized controlled trial of the PERC (pulmonary embolism rule -out criteria) rule for safely excluding pulmonary embolism (PE) in the emergency department (ED) among patients with a "low clinical gestalt" of having PE.  All things pulmonary and all things noninferiority being pet topics of mine, I had to delve deeper into this article because frankly the abstract confused me.

This was a cluster randomized noninferiority trial, but for most purposes, the cluster part can be ignored when looking at the data.  Each of 14 EDs in France was randomized such that during the "PERC period" PE was excluded in patients with a "low gestalt clinical probability" (not yet defined in the abstract) if all of the 8 items of the PERC rule were excluded.  In the "control period" usual procedures for exclusion of PE were followed.  The primary end point was occurrence of a [venous] thromboembolic event (VTE) during 3 months of follow-up.  The delta (pre-specified margin of noninferiority) for the endpoint was 1.5%.  This is a pleasingly low number.  In our meta-research study of 163 noninferiority trials including those in JAMA from 2010-1016, we found that the average delta for those using an absolute risk difference (n=137) was 8.7%, almost 6 times higher!  This is laudable, but was aided by a low estimated event rate in the control group which means that the sample size of ~1900 was feasible given what I assume were relatively low costs of the study.  Kudos to the authors too, for concretely justifying delta in the methods section.


In the follow-up period, the primary outcome of VTE occurred in 0.1% (one patient!) of the PERC group and 0.0% (zero patients!) in the control group.  This is where it can get confusing.  First, as mentioned in our article, JAMA does the sign/polarity of the 95% CIs the opposite of most other journals, where negative favors the new therapy.  Thus the reported 95% CI of -infinity to +0.8% means here that negative infinity extends in the direction of the control group and the data are consistent with up to an 0.8% increase in 3-month VTE with the PERC strategy.  It's unfortunate that JAMA does not routinely use a CONSORT inspired figure to display 95% confidence intervals, so I made one for you (2-sided):

The point estimate for the difference in proportions favors control (where zero VTEs were observed) compared to PERC (where 1 VTE was observed).  The (very narrow) 95% CI just barely crosses zero by 0.0009).  If it had not, we would have had a CONSORT scenario D where there is a statistically significant difference favoring the (old) control therapy, but the upper bound of the 95% CI does not cross (in this case the highly conservative) delta margin of noninferiority and the new therapy is considered noninferior.  In the PROPER trial, a determination of "noninferiority" is clearly warranted.  The authors conclude in the abstract that the PERC rule is "not inferior" and this confused me (and sent me to inspect the confidence interval) because "not inferior" is not a recognized term in the nomenclature of noninferiority trials.

Aside from the primary endpoint, the PERC group underwent 10% fewer CT scans (absolute number about 100 scans) and spend less time in the ED (perhaps because they got fewer time consuming CT scans).

Even the skeptic is obliged to acknowledge that the PERC rule is a safe way of avoiding CT scans in patients like those in this trial.  But like most RCTs, this trial is rich with other data that we can learn from.  

In the methods section, we learn that patients were eligible for inclusion if they had "new-onset shortness of breath or chest pain and a low clinical probability of PE, estimated by the treating physician's gestalt as an expectation below 15%."  And here's a very interesting finding: 

Among the control group who were tested in the usual fashion with D-dimer and CTPA (in those with a positive D-dimer), there was but a 2.7% incidence of diagnosed PE in the ED.

The 95% binomial confidence interval for 2.7% among 954 patients is 1.8% to 3.9%, meaning that the control cohort had at most about a 4% rate of pulmonary embolism.  This upper limit to the confidence interval doesn't even come close to the 15% expectation, meaning that clinicians were overestimating the prevalence of PE by a factor of about 4x.  In the methods section, the delta is justified by an expectation of a 1.5% rate of PE in the control group; adding a delta of 1.5% to that would give an upper limit of 3% and as the authors state:

"This 3% threshold corresponds to the failure rate observed after negative pulmonary angiography and is the threshold used in other studies [references 2,11,12]."

The point estimate of 2.7% in the control group is already less than the expected failure rate, and its associated 95% confidence interval of 3.9% marginally exceeds this.  What this means is that the rate of pulmonary embolism is so low in this cohort (78% of the control group had a Wells score less than 2) that clinician gestalt alone is sufficient to reduce the probability of PE below the acceptable threshold of 3%.  (We have not even considered whether we agree with this 3% acceptable threshold, or the issue of subsegmental PEs.)  And if the PERC rule saves patients CT scans, more power to it.

More than anything, the PROPER trial confirms that a proper use of clinical pre-test probabilities goes a long way towards excluding the diagnosis of PE, and that a lot of CT scans are needed to reduce the rate of undetected PE close to zero.