• English
  • Español

B8O4 | Initial experience with a pilot Cochrane tool for assessing risk of bias for non-randomized studies applying a web-based survey of content experts to derive criteria for imbalance

Abstract text
Background: To date, few tools exist for assessing the risk of bias (ROB) from confounding variables in non-randomized studies (NRS).

Objectives: We tested a pilot Cochrane tool for ROB to assess its applicability and value in quantifying ROB in NRS.

Methods: We applied the Cochrane tool to a systematic review of NRS evaluating the effectiveness of lymph node dissection in patients undergoing radical cystectomy. Three independent review authors abstracted data on methodological quality using the Newcastle-Ottawa instrument and the Cochrane tool for assessing ROB.[1] The Cochrane tool assesses the precision, imbalance, and adjustment for each confounding variable, and overall ROB from confounding, blinding, incomplete outcome data and selective reporting, on 1 (low ROB) – 5 (high ROB) point scales. We pilot-tested the data abstraction form and developed detailed guidelines to optimize interobserver agreement.

Results: Five NRS met the inclusion criteria. Confounders were identified in advance by literature search and responses to a web-based survey of eleven bladder cancer experts who were also asked to categorize what difference in the proportion of a confounding variable represented “no”, a “small”, “moderate” or “large” imbalance. To evaluate imbalance, each confounder was evaluated dichotomously (e.g. organ-confined versus non-organ-confined). Threshold values were derived using the median responses; e.g. for non-organ-confined disease, these were set at ≤ 2%, 2.1-5%, 5.1 – 7%, > 7%, respectively, thereby providing explicit anchors for imbalance criterion scoring.

Conclusions: The novel ROB tool represents a valuable instrument for quantifying the ROB of NRS that deserves further formal investigation in a larger, more diverse sample of studies. The described survey method appears an effective and efficient method to define the necessary imbalance criterion thresholds.

1. Reeves BC, Shea B, Wells GA. Classifying non-randomised studies (NRS) and the assessing the risk of bias for a systematic review. Workshop, 18th Cochrane Colloquium, Keystone 2010.

Palmer DA1, MacLennan SJ2, Imamura M2, Reeves BC3, Clubb A1, Duboy AJ1, Neuberger MM1, Dahm P1
1 Department of Urology, College of Medicine, University of Florida, Gainesville, Florida, USA
2 Academic Urology Unit, University of Aberdeen, Aberdeen, UK
3 Clinical Trials and Evaluation Unit, University of Bristol, Bristol, UK
Presenting author and contact person
Presenting author: 
Philipp Dahm
Contact person: 
Philipp Dahm (Contact this person)
Date and Location
Oral session B8O4
Saturday 22 October 2011 - 12:25 - 12:45