• English
  • Español

B4O3 | Modifications to the Risk of Bias tool: a case study of 204 trials

Abstract text
Background: In 2008 The Cochrane Collaboration introduced the Risk of Bias (RoB) tool to assess internal validity of randomized controlled trials (RCTs). Modifications to the tool were released in 2011 after user testing and feedback.

Objectives: To describe the results of applying the modified RoB tool.

Methods: Two researchers independently applied the tool to 204 RCTs. Disagreements were resolved through consensus. We included several modifications to the original tool. First, we assessed blinding separately for participants, investigators, and outcome assessors. Second, we assessed potential influence of study sponsorship separately from “other sources of bias.” We assessed agreement across blinding domains using kappa. We assessed correlation between different blinding domains, and between funding and “other” domains using Kendall’s Tau.

Results: Risk of bias assessments varied among the three blinding domains. Of note was the higher frequency of low risk of bias for blinding of outcome assessors (51%) compared with blinding of participants (24%) and investigators (31%). The agreement for the blinding domains was fair (κ=0.32). The pair-wise correlation for the blinding domains was moderate (τ= 0.39, participant versus investigator; τ= 0.38, participant versus outcome assessor; τ=0.45, investigator versus outcome assessor). For other sources of bias, risk of bias was low in 118 (58%), high in 33 (16%), and unclear in 53 (26%). For sources of funding risk of bias was low in 77 (38%), high in 14 (7%), and unclear in 113 (55%). The correlation between “other sources of bias” and “source of funding” was weak (τ =0.04). Researchers applying the RoB tool found it easier to assess blinding as three separate domains rather than a single item.

Conclusions: This study provides evidence that risk of bias due to blinding varies depending on the targeted individuals. Risk of bias due to “other” sources is different from inappropriate influence of study sponsor.
Hartling L1, Bialy L1, Armijo-Olivo S1, Ha C1, Lacaze-Masmonteil T1, Vandermeer B1, Dryden DM1
1 Canada
Presenting author and contact person
Presenting author: 
Lisa Hartling
Contact person: 
Lisa Hartling (Contact this person)
Date and Location
Oral session B4O3
Friday 21 October 2011 - 12:05 - 12:25