For our part, the first hints of something being awfully wrong came from articles published in Nature concerning the “discovery” of a pain-specific nucleus in the human thalamus in 1994 which was later found to be non-existent (“a myth” [ 2]). The second clue came from the widely publicized media case of a minimally conscious patient who improved with deep brain stimulation in 2007, a feat that had already been reported by others years before (see Ref. [ 3]). Last but not least, no single paper published in Nature or Science over the past 30 years or so resulted in an effective treatment of neuropathic pain, one of our primary research focuses (unpublished observations). As has been highlighted, many Nobel Prize winners have had their work rejected by Nature and/or by Science (e.g., Fermi, Cherenkov, Krebs, Yukawa, and more) and the inventor of the polymerase chain reaction, Gary Mullis, was rejected by both [ 4], so this lack of progress should have not been totally unexpected.

We therefore set out to assess the therapeutic relevance of articles published in these two journals to clinical neurology. Neurology is one of the two specialties most represented among papers of biomedical relevance in both journals (Table 1, Figure 1, and Table 2). Our final aim was to ascertain if a drug (or procedure) stemming from such studies had reached the US Food and Drug Administration (FDA) or European Agency for the Evaluation of Medicinal Products (EMEA) approval and if the proposed therapeutic intervention had made it into standard clinical care, as assessed in contemporary, updated textbooks and databases. Diagnostic contributions were not considered, since arguably diagnosis without therapy is quite sterile to both the patient and the doctor. In addition, we did not consider papers dealing with the supposed elucidation of pharmacological mechanisms of action, unless the authors made therapeutically relevant observations. We assessed primary research only (Science: research articles, reports; Nature: articles, letters, and brief communications), but not reviews or progress articles. Only papers whose conclusions clearly highlighted the therapeutic relevance of the results and with clear-cut claims were included (see the Supplementary Information, Tables S1 to S4).

Given the supposedly high standards of refereeing applied by these two journals and their Impact Factor-supported cachet, we set stringent standards of assessment. A paper would pass muster only if:

(1)The authors clearly stated or implied that the molecule or procedure under study would substantially advance the treatment of a clinical condition;

(2)No previous or contemporary paper on the target study leading to similar conclusions was identified, that is, the conclusions were completely original (new) and thus fit for a potential patent application; and

(3)Without that paper, the treatment as per above would not have been possible.

We initially selected the year 2000 and assessed the predictions as of December 31, 2014. The process of drug development takes eight and a half years and includes: preclinical trials, involving animal and other laboratory tests (lasting one and a half years on average); clinical trials, involving tests on humans (five years); and FDA review (two years) [ 5]. Goodman and Gilman’s the Pharmacological Basis of Therapeutics (2006, p. 134) provides similar figures (global average 9.2 years, range 4−16 years). Consequently, a span of 14 years seemed adequate for the initial assessment. In fact, one might expect that the pharmaceutical and device industry would constantly monitor such research and then pursue the results in an expeditious fashion. Thirty-seven papers were culled from the two. Each article was assessed individually. Not a single piece of research published in these two prestigious journals led per se to a ground-breaking, clinically effective molecule or procedure, even though major innovations would have been expected—for example, for Alzheimer’s and Parkinson’s diseases. Many groups touted their findings as a real breakthrough, in general arising from “simple” interference with a single molecule or biochemical pathway. In fact, several papers appear to be a strange smorgasbord of animal, genetic, and biochemical data, and it is not even clear upon which data the authors based their overblown conclusions. Neurology has not benefited from Drosophila’s experiments, either.

It may be argued that the 14 year cut-off is still too short a time for a fair assessment of research [ 6]. We thus repeated the same kind of analysis for the year 1990. It is reasonable to suppose that if a study has not led to tangible advances in the clinic after more than two decades, most likely it was either irrelevant or later rejected by further research. Nineteen papers were culled from the two journals. Results are similar to the previous analysis. A few examples include:

(1)Apparently, Bergman et al.’s study (Science, Sep. 21) paved the way to subthalamic (STN) surgery for Parkinson’s disease. On closer scrutiny, however, the merit goes to the Oxford Movement Disorder Group. In a paper published in 1991 (Aziz et al., Movement Disord., 1991, 6: 288−292), the authors write, “lesioning of the STN in the parkinsonian primate is a logical extension of previous (1980s) work from this unit…the STN takes on a pivotal role in the mechanisms central to Parkinsonism. This has been confirmed in studies where…ibotenic acid lesion of the STN relieved limb hypokinesia,” that is, referring to the Bergman et al.’s study.

(2)The grafting of embryonic stem cells (Lindvall et al., Feb. 2) has not proven effective for Parkinson’s disease in randomized trials.

(3)NGF and congeners never made it into the clinic (Maisonpierre et al., Mar. 23).

It is worth remembering how the first papers leading to the DOPA treatment of Parkinson and cholinesterase inhibitors for Alzheimer were published respectively in a German journal (Arch. Psychiatr. Nervenkr. Z. Gesamte Neurol. Psychiatr., 1962, 203: 560−574) and in the Lancet (1977, Mar. 26: 668−671), not in Nature or Science.

Special consideration applies to multiple sclerosis (MS). The most widely accepted hypothesis for the cause of MS—an autoimmune reaction against myelin-related proteins—remains just that, “a hypothesis, one of several,” and “MS may be primarily a degenerative disorder rather than…primarily an autoimmune disease” [ 7, 8]. So it should come as no surprise that a 16 year assessment of the efficacy of interferon B1b, an immune-modulating drug, found no differences in outcome with placebo-treated patients using standard disability and magnetic resonance imaging (MRI) measures [ 9]. There is no reason to believe that newer immune-modulating drugs will fare any better when very long-term, independent follow-up data becomes available. All papers dealing with MS in both Nature and Science have proved o be therapeutically irrelevant and all turned on the immune hypothesis. Cannabinoids (Baker et al., Nature, 2000, Mar. 2) certainly do not represent a major advancement in the symptomatic treatment of MS; for example, the evidence finds cannabis of little benefit for central pain and with intrinsic toxicity [ 2].

We should stress that a similar analysis was carried out for all other medical disciplines, and the results hold up across the board, including infectivology, arguably the most successful of all medical specialties (unpublished results).

Nature and Science are not medical journals, but carry articles dealing with neurological research. We have now shown that all these neurologically oriented papers are therapeutically irrelevant, over a period of 20 years, in that they either lead nowhere or, when they do lead to practical results, similar research published elsewhere has contributed equally or to a greater extent. Paradoxically, this result is confirmed by the many “notes added in proof” that are scattered among the hundreds of reviewed papers, which cite simultaneous research published elsewhere that bears out their conclusions. Of course, it is possible that once in a while a true “gem” gets published, but in the face of our results the odds against it are high. It truly appears that scientific “journals often contain poor science…publishing studies that are scientifically weak (in that their conclusions are not supported by their methods and data) and irrelevant to practitioners (and so patients)” [ 10]. Ioannidis went so far as to suggest that most published research findings are false [ 11], adding that

…few advances in biomedical science materialize into human applications that affect health…the excuse that not enough time has passed is not really satisfactory…intellectual fascination in neuroscience for many decades has led to few new practical applications. It is unclear whether newly announced efforts in this extremely interesting discipline will fare any better…most Nobel prizes in medicine have been given recently for discoveries that offer brilliant mechanicistic insights, but have not yet moved (and may never substantially move) the dial of life expectancy. [ 12]

In the opening citation, Brunton adds, “We have the capacity and ingenuity to correct these shortcomings.” How?

It may be argued that, among many possible reasons for such humbling and disconcerting failure, including biologic processes simply being too complex for our current capabilities, two reasons stand out: “…peer review…(being) slow, expensive, ineffective, something of a lottery, prone to bias and abuse, and hopeless at spotting errors and fraud” [ 10] and animal models of human disease being irrelevant (e.g., Ref. [ 13]). As long as these two “pillars of failure” are not redressed dramatically, we can conjure no reason for hope. In the meantime, medically relevant research published in both Nature and Science should not make it into the media and delude patients (“Journals have an unhealthy relationship with the mass media…(they) might indeed be degenerating into a branch of show business” [ 10]). The “fall of modern medicine” [ 14] is both a reality and an embarrassment. Thus, we can certainly aver that “The future of safe and effective patient care” [ 1] does not lie in basic research and that, in economically dire times, the billions poured on such endeavors can be safely rerouted to more innovative, human based experimentation.

Sergio Canavero and Vincenzo Bonicalzi declare that they have no conflict of interest or financial conflicts to disclose.

http://engineering.org.cn/EN/10.15302/J-ENG-2015092 Tables S1 to S4