Clinical research: The limits of clinical research.

It is worth understanding that the social process of publication in scientific journals, grounded as it is on the need to build careers and reputation through the accumulation of references and citations, is not without considerable limitations.

Even within highly cited research of clinical interventions and their outcomes subsequent contradiction and initially stronger effects are not unusual. Controversies are most common with highly cited nonrandomized studies, but even the most highly cited randomized trials may be challenged and refuted over time, especially small ones.

For example Ioannidis (2005)reported that of 49 highly cited original clinical research studies, 45 claimed that the intervention was effective. Of these, 7 (16%) were contradicted by subsequent studies, 7 others (16%) had reported effects that were stronger than those of subsequent studies, 20 (44%) were replicated, and 11 (24%) remained largely unchallenged. So the substance of the conclusion in a third of these paper had to be withdrawn

Five of 6 highly-cited nonrandomized studies had been contradicted or had found stronger effects vs 9 of 39 randomized controlled trials (P = .008). Among randomized trials, studies with contradicted or stronger effects were smaller (P = .009) than replicated or unchallenged studies although there was no statistically significant difference in their early or overall citation impact. Matched control studies did not have a significantly different share of refuted results than highly cited studies, but they included more studies with "negative" results.