Researchers frequently overview the literature that’s generated by others of their subject. That is an integral a part of day-to-day analysis: discovering related analysis, studying and digesting the primary findings, summarising throughout papers, and making conclusions concerning the proof base as a complete.

Nonetheless, there’s a basic distinction between transient, narrative approaches to summarising a collection of research and trying to reliably, comprehensively summarise an proof base to help decision-making in coverage and apply.

So-called “evidence-informed decision-making” depends on rigorous systematic approaches to synthesising the proof. Systematic overview has change into the best commonplace of proof synthesis. It’s well established within the pipeline from analysis to apply in a number of fields together with health, the environment and social policy. Rigorous systematic opinions are very important for decision-making as a result of they assist to supply the strongest proof {that a} coverage is prone to work (or not). Additionally they assist to keep away from costly or harmful errors within the selection of insurance policies.

However systematic overview has not but completely changed conventional strategies of literature overview. These conventional opinions could also be prone to bias and so might find yourself offering incorrect conclusions. That is particularly worrying when opinions deal with key coverage and apply questions.

The excellent news is that the restrictions of conventional literature overview approaches might be improved comparatively simply with a number of key procedures. A few of these will not be prohibitively pricey when it comes to talent, time or sources. That’s notably necessary in African contexts, the place useful resource constraints are a day by day actuality, however mustn’t compromise the continent’s want for rigorous, systematic and clear proof to tell coverage.

In our recent paper in Nature Ecology and Evolution, we highlighted eight frequent issues with conventional literature overview strategies. We gave examples for every downside, drawing from the sphere of environmental administration and ecology. Lastly, we outlined sensible options.


These are the eight issues we recognized in our paper.

First, conventional literature opinions can lack relevance. It’s because restricted stakeholder engagement can result in a overview that’s of restricted sensible use to decision-makers.

Second, opinions that don’t publish their strategies in an a priori (which means that it’s printed earlier than the overview work begins) protocol might endure from mission creep. In our paper we give the instance of a 2019 review that originally said it was taking a look at all inhabitants developments amongst bugs. As an alternative, it ended up focusing solely on research that confirmed insect inhabitants declines. This might have been prevented by publishing and sticking to strategies outlined in a protocol.

Third, an absence of transparency and replicability within the overview strategies might imply that the overview cannot be replicated. Replicability is a central tenet of the scientific methodology.

Choice bias is one other frequent downside. Right here, the research which might be included in a literature overview will not be consultant of the proof base. An absence of comprehensiveness, stemming from an inappropriate search methodology, may imply that reviews end up with the wrong evidence for the query at hand.

Conventional opinions can also exclude grey literature. That is defined as any doc

produced on all ranges of presidency, lecturers, enterprise and business in print and digital codecs, however which isn’t managed by business publishers, i.e., the place publishing isn’t the first exercise of the manufacturing physique.

It contains organisational reports and unpublished theses or other studies. Conventional opinions can also fail to check for proof of publication bias; each these points may end up in incorrect or deceptive conclusions. One other frequent error is to deal with all proof as equally legitimate. The fact is that some analysis research are extra legitimate than others. This must be accounted for within the synthesis.

Inappropriate synthesis is one other frequent situation. This includes strategies like vote-counting, which refers to tallying research primarily based on their statistical significance. Lastly, an absence of consistency and error checking (as would occur when a reviewer works alone) can introduce errors and biases if a single reviewer makes decisions without consensus.

All of those frequent issues could be solved, although. Right here’s how.


Stakeholders could be recognized, mapped and contacted for suggestions and inclusion with out the necessity for intensive budgets. Greatest-practice pointers for this course of already exist.

Researchers can fastidiously design and publish an a priori protocol that outlines deliberate strategies for looking, screening, knowledge extraction, important appraisal and synthesis intimately. Organisations just like the Collaboration for Environmental Proof have existing protocols from which individuals can draw.

Researchers additionally should be specific and use high-quality steering and requirements for overview conduct and reporting. A number of such requirements already exist.

One other helpful method is to fastidiously design a search technique with an data specialist; to trial the search technique towards a benchmark checklist; and to make use of a number of bibliographic databases, languages and sources of gray literature. Researchers ought to then publish their search strategies in an a priori protocol for peer overview.

Researchers ought to think about fastidiously planning and trialling a important appraisal instrument earlier than beginning the method in full, studying from existing robust critical appraisal tools. Essential appraisal is the fastidiously deliberate evaluation of all potential dangers of bias and potential confounders in a analysis research. Researchers ought to choose their synthesis methodology fastidiously, primarily based on the information analysed. Vote-counting ought to by no means be used as an alternative of meta-analysis. Formal methods for narrative synthesis must be used to summarise and describe the proof base.

Lastly, not less than two reviewers ought to display a subset of the proof base to make sure consistency and shared understanding of the strategies earlier than continuing. Ideally, reviewers ought to conduct all selections individually after which consolidate.


Collaboration is essential to deal with the issues with conventional overview processes. Authors have to conduct extra rigorous opinions. Editors and peer reviewers have to gate-keep extra strictly. The neighborhood of methodologists wants to raised help the broader analysis neighborhood.

Working collectively, the tutorial and analysis neighborhood can construct and preserve a powerful system of rigorous, evidence-informed decision-making in conservation and environmental administration – and, finally, in different disciplines.

Source link


Write A Comment