A quick update before I proceed with the current post: The Institute of Medicine has met and they agree with me that sodium restriction is for the birds. (Click here for a New York Times summary article.) In other news, the oh-so-natural Omega-3 fatty acid panacea did not improve cardiovascular outcomes as reported in the NEJM on May 9th, 2013.

An article by the TOPPS investigators in the May 9th NEJM is very useful to remind us not to believe everything we read, to always check our premises, and that some data are so dependent on the perspective from which they're interpreted or the method or stipulations of analysis that they can be used to support just about any viewpoint.

An article by the TOPPS investigators in the May 9th NEJM is very useful to remind us not to believe everything we read, to always check our premises, and that some data are so dependent on the perspective from which they're interpreted or the method or stipulations of analysis that they can be used to support just about any viewpoint.

The authors sought to determine if a strategy of withholding
prophylactic platelet transfusions for platelet counts below 10,000 in patients
with hematologic malignancy was non-inferior to giving prophylactic platelet
transfusions. I like this idea, because
I like "less is more" and I think the body is basically
antifragile. But non-inferior how? And what do we mean by non-inferior in this
trial?

In this trial the outcome is bleeding of Grade 2 or higher
on the WHO grading system for bleeding events.
You can click here for the definition of Grade 2 bleeding inSupplementary Appendix 3. Scroll to page
13. What if only Grade 2 bleeding were
improved in a trial of prophylactic platelet transfusion? Is a reduction in bruising and vaginal
bleeding justification for prophylactic platelets? You be the judge, because it's a judgment call
for sure. (Grades 3 and 4 are more
unequivocal.) There's another problem: Do you consider bruises greater than 2 cm to
be on par with retinal hemorrhage with visual impairment? I don't, but the WHO does, and they're both
Grade 2 according to WHO. So we have uncovered
a potentially flawed premise: namely that the use of the WHO bleeding grading
system is a good idea for the design of these trials. This is the problem with a very many trials,
that their design is inherently flawed in a way that severely limits their interpretation and thus their utility.

If you're skeptical that prevention of a Grade 2 bleed
justifies a platelet transfusion, your next stop is Table 2 in the article on the
auspicious page 1777. Here we see that
there were 18 fewer Grade 2 bleeds in the prophylaxis group and 5 fewer Grades
3 and 4 bleeds in the secondary analyses section. (I'll return to the primary analysis in a
moment.) If you go back to the
Supplementary Appendix and scroll to Table S1 on page 17, you will see that
there were 504 transfusions in the no-prophylaxis group and 894 in the
prophylaxis group, for a difference of 390 transfusions. The NNT (Number Needed to Transfuse :-) is
therefore approximately 17 transfusions to prevent one bleeding event, and 18
out of 23 times the bleeding event is Grade 2.
(I'm taking some liberty here.
Technically the NNT of the trial as a whole is 1/0.084 or about 12. [Or, 1/0.07=14 if you use raw unadjusted
data.] But since patients received
multiple transfusions, I'm trying to hone in on the effect of one transfusion.)

There's more. Note
that the methods section, under statistical analyses, reveals that a

**90%**confidence interval (CI) was chosen for the analyses. A 90% CI is narrower than a 95% CI, making it more likely that you will have a statistically significant result. That's the same as saying that your p-value for significance is 0.10, and moving on without any further ado. Here, for reasons inscrutable, the editors (and the editorialist) let it pass without further mention. There's more still. The margin of non-inferiority (delta) was chosen to be 10% as is [sadly] the custom [well, we have 10 fingers!], and also was not justified, as is likewise [and sadly] customary. But lo! After the first interim analysis showed that the event rate was 48% rather than the hypothesized 20% (which means that the effect size would be smaller, requiring a larger sample size), instead of increasing the sample size, the authors elected to increase the delta margin to 15%! This is premeditated delta inflation in the first degree! Note also that the usual justification for using a 5% one-sided significance level (that is, a two sided p-value of 0.10) is that "we don't care if the observed difference goes in the direction opposite that which we hypothesized" is bankrupt here - the results of the trial did indeed go in the direction opposite that which was hypothesized. A final note is that the trial was analyzed, according to a priori plans, as intention to treat. That's great for a superiority trial, but it is against the CONSORT recommendation that non-inferiority trials be analyzed per-protocol, the more conservative method - nevermind whether post hoc analyses showed that both analyses produced similar results (refer,*yet again*, to the Supplementary Appendix).
When I try to reduplicate the analyses with STATA, I am
befuddled, perhaps because of the authors' adjustment "for diagnosis and
study treatment as minimization variables". I do not have the actual raw data so I cannot
perform the adjusted analyses. But STATA tells me that, based on the data
presented in Table 2 under primary end points, the difference between the groups is 7% and
the 90% CI is 0.3% to 13.7%. This is a
statistically significant result in favor of prophylaxis with an alpha of 0.10,
but of course the upper bound of the CI crosses the

**10% pre-specified margin of non-inferiority (delta) so CONSORT would consider it "inconclusive"***initial***. But the upper bound falls below the 15% delta that was modified after the interim analysis, which would make no-prophylaxis non-inferior with these assumptions. The raw unadjusted***for that delta***95%****CI**would be -1.0% to +14.96% so the result is**with a 15% delta and a 95% CI. Note also that, for superiority, prophylaxis is superior with a 90% CI (the CI does not include zero), but not with a 95% CI (the CI does include zero).***non-inferior*
And what the heck does all this mean? Perhaps a bullet point summary is in order to
sort it all out in digested form:

- The interpretation of the trial hinges critically on the premises of the study and the endpoint chosen: does prevention of WHO Grade 2 (or greater) bleeding make sense to you?
- If you accept the primary endpoint as valid and relevant to practice, does a margin of non-inferiority (delta) of 10% or 15% satisfy you?
- Are you OK with 90% confidence intervals and adjusted analyses and intention-to-treat analyses in a non-inferiority trial?
- Are you willing to declare superiority under less stringent criteria than you would have declared non-inferiority? (See this Letter to the Editor of JAMA regarding the recent CONSORT statement revision.)

In the end, this trial reminds me my conclusion for the Atleplase study for submassive pulmonary embolism: TPA now
saves you from TPA later. Do platelet
transfusions now just save you from platelet transfusions later? Perhaps a compromise is to sit tight with
thrombocytopenia in hematologic malignancy, and transfuse when bleeding occurs.
Those with a propensity to bleed will
declare themselves with sentinel WHO Grade 2 bleeding. Do I have data to support this approach? I don't. Do you have data to support another?

This comment has been removed by the author.

ReplyDelete