Risk of bias (RoB) assessment is essential to the systematic review methodology. The new version of the Cochrane RoB tool for randomized trials (RoB 2) was published in 2019 to address limitations identified since the first version of the tool was published in 2008 and to increase the reliability of assessments. This study analyzed the frequency of usage of the RoB 2 and the adequacy of reporting the RoB 2 assessments in non-Cochrane reviews published in 2020. This meta-research study included non-Cochrane systematic reviews of interventions published in 2020. For the reviews that used the RoB 2 tool, we analyzed the reporting of the RoB 2 assessment. Among 3880 included reviews, the Cochrane RoB 1 tool was the most frequently used (N = 2228; 57.4%), followed by the Cochrane RoB 2 tool (N = 267; 6.9%). From 267 reviews that reported using the RoB 2 tool, 213 (79.8%) actually used it. In 26 (12.2%) reviews, erroneous statements were used to indicate the RoB 2 assessment. Only 20 (9.4%) reviews presented a complete RoB 2 assessment with a detailed table of answers to all signaling questions. The judgment of risk of bias by the RoB 2 tool was not justified by a comment in 158 (74.2%) reviews. Only in 33 (14.5%) of reviews the judgment in all domains was justified in the accompanying comment. In most reviews (81.7%), the RoB was inadequately assessed at the study level. In conclusion, the majority of non-Cochrane reviews published in 2020 still used the Cochrane RoB 1 tool. Many reviews used the RoB 2 tool inadequately. Further studies about the uptake and the use of the RoB 2 tool are needed.
Description
Frequency of use and adequacy of Cochrane risk of bias tool 2 in non-Cochrane systematic reviews published in 2020: Meta-research study - PubMed
%0 Journal Article
%1 babic2024frequency
%A Babić, Andrija
%A Barcot, Ognjen
%A Visković, Tomislav
%A Šarić, Frano
%A Kirkovski, Aleksandar
%A Barun, Ivana
%A Križanac, Zvonimir
%A Ananda, Roshan Arjun
%A Fuentes Barreiro, Yuli Viviana
%A Malih, Narges
%A Dimcea, Daiana Anne-Marie
%A Ordulj, Josipa
%A Weerasekara, Ishanka
%A Spezia, Matteo
%A Žuljević, Marija Franka
%A Šuto, Jelena
%A Tancredi, Luca
%A Pijuk, Anđela
%A Sammali, Susanna
%A Iascone, Veronica
%A von Groote, Thilo
%A Poklepović Peričić, Tina
%A Puljak, Livia
%C England
%D 2024
%J Research synthesis methods
%K grapp-caib
%P 10.1002/jrsm.1695--
%R 10.1002/jrsm.1695
%T Frequency of use and adequacy of Cochrane risk of bias tool 2 in non-Cochrane systematic reviews published in 2020: Meta-research study
%U https://pubmed.ncbi.nlm.nih.gov/38262609
%X Risk of bias (RoB) assessment is essential to the systematic review methodology. The new version of the Cochrane RoB tool for randomized trials (RoB 2) was published in 2019 to address limitations identified since the first version of the tool was published in 2008 and to increase the reliability of assessments. This study analyzed the frequency of usage of the RoB 2 and the adequacy of reporting the RoB 2 assessments in non-Cochrane reviews published in 2020. This meta-research study included non-Cochrane systematic reviews of interventions published in 2020. For the reviews that used the RoB 2 tool, we analyzed the reporting of the RoB 2 assessment. Among 3880 included reviews, the Cochrane RoB 1 tool was the most frequently used (N = 2228; 57.4%), followed by the Cochrane RoB 2 tool (N = 267; 6.9%). From 267 reviews that reported using the RoB 2 tool, 213 (79.8%) actually used it. In 26 (12.2%) reviews, erroneous statements were used to indicate the RoB 2 assessment. Only 20 (9.4%) reviews presented a complete RoB 2 assessment with a detailed table of answers to all signaling questions. The judgment of risk of bias by the RoB 2 tool was not justified by a comment in 158 (74.2%) reviews. Only in 33 (14.5%) of reviews the judgment in all domains was justified in the accompanying comment. In most reviews (81.7%), the RoB was inadequately assessed at the study level. In conclusion, the majority of non-Cochrane reviews published in 2020 still used the Cochrane RoB 1 tool. Many reviews used the RoB 2 tool inadequately. Further studies about the uptake and the use of the RoB 2 tool are needed.
@article{babic2024frequency,
abstract = {Risk of bias (RoB) assessment is essential to the systematic review methodology. The new version of the Cochrane RoB tool for randomized trials (RoB 2) was published in 2019 to address limitations identified since the first version of the tool was published in 2008 and to increase the reliability of assessments. This study analyzed the frequency of usage of the RoB 2 and the adequacy of reporting the RoB 2 assessments in non-Cochrane reviews published in 2020. This meta-research study included non-Cochrane systematic reviews of interventions published in 2020. For the reviews that used the RoB 2 tool, we analyzed the reporting of the RoB 2 assessment. Among 3880 included reviews, the Cochrane RoB 1 tool was the most frequently used (N = 2228; 57.4%), followed by the Cochrane RoB 2 tool (N = 267; 6.9%). From 267 reviews that reported using the RoB 2 tool, 213 (79.8%) actually used it. In 26 (12.2%) reviews, erroneous statements were used to indicate the RoB 2 assessment. Only 20 (9.4%) reviews presented a complete RoB 2 assessment with a detailed table of answers to all signaling questions. The judgment of risk of bias by the RoB 2 tool was not justified by a comment in 158 (74.2%) reviews. Only in 33 (14.5%) of reviews the judgment in all domains was justified in the accompanying comment. In most reviews (81.7%), the RoB was inadequately assessed at the study level. In conclusion, the majority of non-Cochrane reviews published in 2020 still used the Cochrane RoB 1 tool. Many reviews used the RoB 2 tool inadequately. Further studies about the uptake and the use of the RoB 2 tool are needed.},
added-at = {2024-02-25T21:22:10.000+0100},
address = {England},
author = {Babić, Andrija and Barcot, Ognjen and Visković, Tomislav and Šarić, Frano and Kirkovski, Aleksandar and Barun, Ivana and Križanac, Zvonimir and Ananda, Roshan Arjun and Fuentes Barreiro, Yuli Viviana and Malih, Narges and Dimcea, Daiana Anne-Marie and Ordulj, Josipa and Weerasekara, Ishanka and Spezia, Matteo and Žuljević, Marija Franka and Šuto, Jelena and Tancredi, Luca and Pijuk, Anđela and Sammali, Susanna and Iascone, Veronica and von Groote, Thilo and Poklepović Peričić, Tina and Puljak, Livia},
biburl = {https://www.bibsonomy.org/bibtex/21c4437d39546b6f5cc567a53da2bcc5b/nachoricci},
comment = {38262609[pmid]},
description = {Frequency of use and adequacy of Cochrane risk of bias tool 2 in non-Cochrane systematic reviews published in 2020: Meta-research study - PubMed},
doi = {10.1002/jrsm.1695},
interhash = {4fb9647ff998640a784890b625ddbb9d},
intrahash = {1c4437d39546b6f5cc567a53da2bcc5b},
issn = {17592887},
journal = {Research synthesis methods},
keywords = {grapp-caib},
month = jan,
pages = {10.1002/jrsm.1695--},
timestamp = {2024-02-25T21:22:10.000+0100},
title = {Frequency of use and adequacy of Cochrane risk of bias tool 2 in non-Cochrane systematic reviews published in 2020: Meta-research study},
url = {https://pubmed.ncbi.nlm.nih.gov/38262609},
year = 2024
}