In this week's JAMA, Freund et al report the results of the PROPER randomized controlled trial of the PERC (pulmonary embolism rule -out criteria) rule for safely excluding pulmonary embolism (PE) in the emergency department (ED) among patients with a "low clinical gestalt" of having PE. All things pulmonary and all things noninferiority being pet topics of mine, I had to delve deeper into this article because frankly the abstract confused me.
This was a cluster randomized noninferiority trial, but for most purposes, the cluster part can be ignored when looking at the data. Each of 14 EDs in France was randomized such that during the "PERC period" PE was excluded in patients with a "low gestalt clinical probability" (not yet defined in the abstract) if all of the 8 items of the PERC rule were excluded. In the "control period" usual procedures for exclusion of PE were followed. The primary end point was occurrence of a [venous] thromboembolic event (VTE) during 3 months of follow-up. The delta (pre-specified margin of noninferiority) for the endpoint was 1.5%. This is a pleasingly low number. In our meta-research study of 163 noninferiority trials including those in JAMA from 2010-1016, we found that the average delta for those using an absolute risk difference (n=137) was 8.7%, almost 6 times higher! This is laudable, but was aided by a low estimated event rate in the control group which means that the sample size of ~1900 was feasible given what I assume were relatively low costs of the study. Kudos to the authors too, for concretely justifying delta in the methods section.
In the follow-up period, the primary outcome of VTE occurred in 0.1% (one patient!) of the PERC group and 0.0% (zero patients!) in the control group. This is where it can get confusing. First, as mentioned in our article, JAMA does the sign/polarity of the 95% CIs the opposite of most other journals, where negative favors the new therapy. Thus the reported 95% CI of -infinity to +0.8% means here that negative infinity extends in the direction of the control group and the data are consistent with up to an 0.8% increase in 3-month VTE with the PERC strategy. It's unfortunate that JAMA does not routinely use a CONSORT inspired figure to display 95% confidence intervals, so I made one for you (2-sided):
Among the control group who were tested in the usual fashion with D-dimer and CTPA (in those with a positive D-dimer), there was but a 2.7% incidence of diagnosed PE in the ED.The 95% binomial confidence interval for 2.7% among 954 patients is 1.8% to 3.9%, meaning that the control cohort had at most about a 4% rate of pulmonary embolism. This upper limit to the confidence interval doesn't even come close to the 15% expectation, meaning that clinicians were overestimating the prevalence of PE by a factor of about 4x. In the methods section, the delta is justified by an expectation of a 1.5% rate of PE in the control group; adding a delta of 1.5% to that would give an upper limit of 3% and as the authors state:
"This 3% threshold corresponds to the failure rate observed after negative pulmonary angiography and is the threshold used in other studies [references 2,11,12]."The point estimate of 2.7% in the control group is already less than the expected failure rate, and its associated 95% confidence interval of 3.9% marginally exceeds this. What this means is that the rate of pulmonary embolism is so low in this cohort (78% of the control group had a Wells score less than 2) that clinician gestalt alone is sufficient to reduce the probability of PE below the acceptable threshold of 3%. (We have not even considered whether we agree with this 3% acceptable threshold, or the issue of subsegmental PEs.) And if the PERC rule saves patients CT scans, more power to it.
More than anything, the PROPER trial confirms that a proper use of clinical pre-test probabilities goes a long way towards excluding the diagnosis of PE, and that a lot of CT scans are needed to reduce the rate of undetected PE close to zero.