Using meta-analyses to analyze the medical literature

EBM teaching point for the evening concerned meta-analyses. A well
performed meta-analysis (MA) is thought by many (but not all) to be a the top
of the evidence pyramid, stronger even than a RCT. Poorly done MA evoke the
phrase “garbage in-garbage out” and are basically useless. What makes a good
MA? It’s important for it to ask a specific question, to include an exhaustive search
of the literature for all appropriate articles, and to rank and include only high-
quality, consistent (low heterogeneity) studies. Why do a MA? A well done analysis
of a large number of individual studies or of individual patient data can combine
many small under-powered studies and come up with a clinically and statistically
significant conclusion. It can contribute to the generalisability of study results,
and generate new research questions, especially with regards to subgroups in the
studies.


1) Colman I, et al: Parenteral dexamethasone for acute severe migraine headache:
meta-analysis of randomised controlled trials for preventing recurrence.BMJ
2008;336:1359-1361.

This meta-analysis was felt by folks in the room to be easy to read, relatively easy to
understand, and met the “gut-check” of meta-analyses: when you look at the Forest
plot, which is the visual representation of the comparison of the treatment effects
of the different papers, you get a sense that although almost all the studies were too
small to show a statistically significant treatment effect, they all trended in the same
direction (favoring steroids), and when examined together in a MA, did achieve
signficance, with a NNT of 9 to prevent the recurrence of migraine within 72
hours. Side effects between treatment/control groups were similar. This illustrates
the power of MA; most of the original studies produced false negative results (type
II error) because they were small, but together, they have power to demonstrate a
treatment difference. Doses used were between 10-24 mg of dexamethasone,
with doses >15 mg showing a non-sig. increased treatment effect. It was pointed
out that for both this and the third study, one of the authors of the MA was also an
author of one of the original papers. Does that bias the MA? Does he have an ax to
grind to prove his point, or is he simply acting in the spirit of scientific inquiry to
find the true answer? You be the judge.

2) Patrick S, et al: Supraclavicular Subclavian Vein Catherization: The Forgotten Central
Line. West J Emerg Med 2009;10:110-114.

This article is not a MA, but a review of some literature on the supraclavicular
approach to the subclavian central line. As such, although it includes information
about some different studies on this approach, it’s conclusions are much more “we
like it” rather than a definitive evaluation of the approach’s merits and potential
complications. That being said, everyone in the room (except Jaime and Drew
S.) seems to like this line a lot. It’s landmark based, avoids the chest during CPR,
and makes great intuitive anatomic sense. It’s non-compressible, so not for
coagulopathic patients, and to use US, you would need a small footprint (“pencil”)
probe. Complication rates seem no higher than other neck lines. There are
multiple variations, but the original and most popular seems to be the Yoffa
approach (just lateral to lateral belly of SCM, 1 cm posterior to clavicle, direct
at 45 degrees to sagittal and transverse planes, and 15 degrees below coronal
plane-great diagrams in Roberts/Hedges or online). So try it, you’ll like it!
Residents, an appeal to all of you to push your comfort zones and try all approaches
to these lines, both with and without US.


3) Annane D, et aL: Corticosteroids in the Treatment of Severe Sepsis and Septic Shock in
Adults: A Systematic Review. JAMA 2009;301(22):2362-2375.
Finally, the bruiser of the group. Unlike article #1, this MA is difficult to read,
complicated, and the conclusions are not intuitively obvious when looking at its
Forest plot. In the conclusion, the authors suggest that low-dose, prolonged
course steroids be given, although only to adults with vasopressor-dependent
septic shock (although the MA evaluated/drew their conclusions from studies
including patients with both severe sepsis and septic shock). It’s important to
remember that overall, this MA showed no sig. effect of steroids on 28 day mortality,
and only by carving out the prolonged course/low dose trials did they come up
with a significant reduction in 28 day mortality. This brings up the issue of sub-
group analysis, and whether there is ever a time when a subgroup analysis, even
derived a priori, showing a large treatment effect, and making intuitive sense, can
be used to support a conclusion in a study. Short answer: sub-group analyses
should only be used as hypothesis generators for future studies. Multiple sub-
group analyses open up the risk of a Type I error (false positive results-finding a
difference when one doesn’t actually exist). So, what do you do about steroids and
sepsis? As Chintan pointed out, in 2004 according to the Surviving Sepsis Campaign,
the answer to the question of “steroids in sepsis?” was Yes! , largely because of a
2002 study by Annane in JAMA. Then, Sprung et al published CORTICUS in NEJM,
which failed to demonstrate any mortality benefit with steroids. So in 2008 the
answer was No! Now, with this MA, the answer is ??? The informal consensus
from JC was support to giving steroids to patients with vasopressor dependent
septic shock who aren’t responding well to EGDT. That being said, there will
be significant practice variation, and this is yet another area of clinical uncertainty
in medicine. The philosophical point of the evening, brought home by E. Kulstad,
was that this uncertainty is common, and if embraced, can lead to shared decision
making, open doors for future study. There are currently 10 studies on low dose
steroids in sepsis in the pipeline.