Risk of bias: why measure it, and how?
Eye, 36(2), 346-348, 2022
Phillips et al.
DOI : 10.1038/s41433-021-01759-9
Article complet : lien
Abstract
What is risk of bias?
Clinicians and researchers regularly read and interpret randomized controlled trials (RCTs) to inform their practice… but how can they be sure that the RCT is accurate and reliable? Not all RCTs are the same, so it’s worth considering carefully whether the results of a randomized controlled trial merit changing the way you manage your future patients. The best way to assess the validity of a randomized controlled trial is to understand the possible risks of bias for that particular study. Bias occurs when something in the design or execution of a study has a systematic impact on the study results that deviates from the truth. When such a bias exists, a study may result in an overestimation or underestimation of the truth, compromising the validity of the study’s conclusions or results, even if all other facets of the study were appropriate [1,2,3]. Imagine, for example, that you provide navigation advice using a compass that doesn’t point precisely “North”, but tends to point “North-East”. Even if you provide comprehensive navigation instructions to a fellow traveler, the final result will not be accurate due to the bias caused by the compass’ inaccuracy. Similarly, an otherwise robust study with some form of bias may provide clinicians and patients with results that are not accurate, despite the completeness of the investigation. The same is true of a test that is supposed to measure this or that function or performance, but whose actual target is different or partial. With this in mind, it is important to understand the types of bias that can exist in randomized controlled trials, how to detect these potential biases and how to interpret the results of a study in the context of these possible biases.
What types of bias exist and how can they be assessed?
There are five main forms of bias that are important to consider in clinical trials: selection bias, performance bias, detection bias, attrition bias and reporting bias (Table 1) [1, 3]. The so-called Cochrane Risk of Bias Assessment Tool is the gold standard bias assessment tool for randomized clinical trials, as it assesses the risk of each of these forms of bias [3]. Below we provide a summary of each form of bias, and discuss how to minimize the risk of each when designing, conducting, analyzing and reporting trials.
Selection bias
High-quality randomized controlled trials randomize patients and, importantly, conceal this randomization. Why do they do this? To limit selection bias. Selection bias is described as a fundamental difference between patients included in the treatment groups of a study due to the way patients were allocated to the treatment groups [1, 4]. To assess selection bias, both random sequence generation and RCT allocation concealment methods need to be taken into account [3].
Sequence generation refers to the method by which patients were randomly assigned to treatment groups. A truly random sequence for treatment allocation means that the baseline characteristics of the two groups will be intrinsically balanced, but bias in this allocation can lead to systematic differences between the comparison groups [1, 3]. Certain risks of bias due to sequence generation may exist as a result of non-randomized or quasi-randomized allocation methods. These methods may allow clinicians to choose the treatment patients will receive in the study based on their expertise and previous experience (i.e. a non-random factor). In addition, allocation concealment refers to methods used to prevent anyone from predicting or inferring patient allocation [3]. Good allocation concealment can prevent anyone on the research team from determining or predicting which patient received which treatment in the trial. In summary, sequence generation refers to the way in which patients are allocated to comparison groups, and allocation concealment refers to the way in which this allocation is kept secret from all parties involved.
Performance biases
What about factors that may influence the performance of a patient or clinician in a randomized clinical trial? Performance bias may occur if there are differences between study groups due to systematic differences in performance outside the treatment received in the study [1]. Risks of performance bias can arise from methods of masking (or blinding) participants and staff [1]. If masking is correctly implemented, we can be sure that there has been no additional, undue influence on patient outcome outside the assigned intervention [3, 5]. Many critical outcomes involving subjective response, such as assessments of visual acuity or pain, could be biased if the patient or assessor were aware of the treatment assignment. When assessing performance bias, it is important to consider whether the absence of masking can reasonably be expected to have an impact on the outcomes being assessed [1].
Detection bias
Previous biases have focused on methods of randomization and masking of patients and clinicians, but what about biases in the way outcomes are measured? Detection bias can be described as the possibility of differences between comparison groups in the way outcomes are measured or assessed [1]. Detection bias also revolves around the concept of masking; however, it is the outcome assessor who needs to be masked in order to mitigate detection bias [3]. Masking of outcome assessors ensures that the methods used to measure an outcome do not differ between patients allocated to comparison groups, meaning that the outcome measure is consistent for all study participants [1, 3].
Attrition bias
Once patients have been included in a randomized clinical trial, it is always possible that they may withdraw from the study before completing their follow-up. Attrition bias can result from a systematic cause of patient drop-out in a study that disproportionately affects a certain subset of patients [1]. If a cause of drop-out is present – or more predominant – in comparison groups, the imbalance of drop-outs could have an impact on the results and conclusions drawn from the study [1, 3]. If a specific group of patients were more likely to withdraw from the study within one of the comparison groups, the imbalance would have obvious implications for the results [1, 6].
Declaration bias
The final form of bias that every clinician needs to consider when reading a randomized clinical trial is reporting bias. This type of bias can arise when there are concerns about the results reported in a study’s findings [1]. Selective reporting of results is the main concern in this form of bias, which refers to the reporting of some, but not all, measured outcomes in the results of a study [1, 3]. This bias generally manifests itself as a study reporting significant results while omitting results that are not significant [1, 7]. Although this phenomenon can be difficult to detect, it highlights the importance of a predefined study protocol that identifies all the outcomes that will be evaluated. As a reader, you should actively seek confirmation of this important step.
How to interpret risk-of-bias assessments?
The next time you read a randomized clinical trial, consider these risks of bias before changing your clinical practice. RCTs rated as having a high risk of bias should be interpreted with caution, as bias has a direct impact on the validity of the results [1]. Empirical studies have shown that studies with a high risk of bias may lead to an exaggeration of treatment effects within trials compared with studies with a low risk of bias [8, 9]. It is common to assess a study’s risk of bias based solely on the information provided in the study manuscript, but insufficient information is not synonymous with biased conduct [3]. This is an important distinction to make when assessing risk of bias, and requires careful consideration of the potential validity implications of study design decisions. You can refer to the Cochrane Handbook for Systematic Reviews of Interventions, “Chapter 8: Risk of Bias in Randomized Trials” for a comprehensive guide to assessing risk of bias for randomized controlled trials [1].
| Bias | Summary | Example |
|---|---|---|
| Selection bias | Bias due to the methods used to assign patients to study treatment groups. | A surgeon in a glaucoma laser versus topical medicine RCT can accurately guess the allocation of future patients. They may then preferentially wait to identify the “ideal” patient for each treatment arm, opposed to having them assigned at random. |
| Performance bias | Bias that occurs when patients or clinicians are aware of the assigned treatment, and perform differently as a result. | A patient learns that they received the placebo treatment in a study. When they are performing a visual acuity test they, consciously or subconsciously, do not perform their best due to knowing they received a null treatment. |
| Detection bias | Bias in the measurement of study outcomes when outcome assessors are aware of the assigned treatment. | A surgeon grading post operative inflammation in an ophthalmology RCT is not masked to the patient’s treatment, and this knowledge influences their assessments based on prior knowledge and experiences. |
| Attrition bias | Bias due to an influencing factor that causes non-random withdrawals from the study groups. | A study assessing visual acuity after retinal detachment has a large number of withdrawals that occurred primarily in patients of lower socioeconomic status. |
| Reporting bias | Bias in the outcomes reported by a study, mainly when non-significant findings are ignored. | A published RCT on cataract surgery stated that they would assess visual acuity, adverse events, and quality of life within their protocol; however, only visual acuity and adverse event outcomes are reported in the manuscript. |
TablePress table 2: “Types of bias Summary.”
Bibliographical references
Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions
of methodological quality associated with estimates of treatment effects in con-
trolled trials. JAMA. 1995;273:408–12. https://doi.org/10.1001/jama.273.5.408.
The Cochrane Collaboration. Cochrane Handbook for Systematic Reviews of
Interventions, Version 6.2. Chapter 8: Risk of Bias in Randomized Trials. Published
Tripepi G, Jager KJ, Dekker FW, Zoccali C. Selection bias and information bias in
clinical research. Nephron Clin Pr. 2010;115:c94–99. https://doi.org/10.1159/
000312871.
Higgins JPT, Altman DG, Gøtzsche PC, The Cochrane Collaboration’s tool for
assessing risk of bias in randomised trials. BMJ. 2011;343. https://doi.org/10.1136/
bmj.d5928.
Berger VW. Quantifying the magnitude of baseline covariate imbalances resulting
from selection bias in randomized clinical trials. Biom J Biom Z. 2005;47:119–27.
https://doi.org/10.1002/bimj.200410106. discussion 128–39.
Boutron I, Estellat C, Guittet L, Dechartres A, Sackett DL, Hróbjartsson A, et al.
Methods of blinding in reports of randomized controlled trials assessing phar-
macologic treatments: a systematic review. PLoS Med. 2006;3:e425. https://doi.
org/10.1371/journal.pmed.0030425.
Fergusson D, Aaron SD, Guyatt G, Hébert P. Post-randomisation exclusions: the
intention to treat principle and excluding patients from analysis. BMJ.
2002;325:652–4. https://doi.org/10.1136/bmj.325.7365.652.
Kirkham JJ, Dwan KM, Altman DG, Gamble C, Dodd S, Smyth R, et al. The impact of
outcome reporting bias in randomised controlled trials on a cohort of systematic
reviews. BMJ. 2010;340:c365. https://doi.org/10.1136/bmj.c365.
Wood L, Egger M, Gluud LL, Schulz KF, Jüni P, Altman DG, et al. Empirical evidence
of bias in treatment effect estimates in controlled trials with different interventions
and outcomes: meta-epidemiological study. BMJ. 2008;336:601–5. https://doi.org/
10.1136/bmj.39465.451748.AD.Editorial
348