Impact of review method on the conclusions of clinical reviews: A systematic review on dietary interventions in depression as a case in point

Abstract

Background The recommendations of experts who write review articles are a critical determinant of the adaptation of new treatments by clinicians. Several types of reviews exist (narrative, systematic, meta-analytic), and some of these are more vulnerable to researcher bias than others. Recently, the interest in nutritional interventions in psychiatry has increased and many experts, who are often active researchers on this topic, have come to strong conclusions about the benefits of a healthy diet on depression. In a young and active field of study, we aimed to investigate whether the strength of an author’s conclusion is associated with the type of review article they wrote.

Methods Systematic searches were performed in PubMed, Web of Science, Cochrane Database of Systematic Reviews, and Google Scholar for narrative reviews and systematic reviews with and without meta-analyses on the effects of diet on depression (final search date: May 30th, 2020). Conclusions were extracted from the abstract and discussion section and rated as strong, moderate, or weak by independent raters who were blind to study type. A benchmark on legitimate conclusion strength was based on a GRADE assessment of the highest level of evidence. This systematic review was registered with PROSPERO, number CRD42020141372.

Findings 24 narrative reviews, 12 systematic reviews, and 14 meta-analyses were included. In the abstract, 33% of narrative reviews and 8% of systematic reviews came to strong conclusions, whereas no meta-analysis did. Narrative reviews were 8.94 (95% CI: 2.17, 36.84) times more likely to report stronger conclusions in the abstract than systematic reviews with and without meta-analyses. These findings were similar for conclusions in the discussion section. Narrative reviews used 45.6% fewer input studies and were more likely to be written by authors with potential conflicts of interest. A study limitation is the subjective nature of the conclusion classification system despite high inter-rater agreements and its confirmation outside of the review team.

Conclusions We have shown that narrative reviews come to stronger conclusions about the benefits of a healthy diet on depression despite inconclusive evidence. This finding empirically underscores the importance of a systematic method for summarizing the evidence of a field of study. Journal editors may want to reconsider publishing narrative reviews before meta-analytic reviews are available.