Bias, Quality, and Limitations
Framework Position
This chapter deepens the Evidence Review stage by making one critical aspect explicit:
π No study is perfect.
All evidence carries:
- bias
- uncertainty
- limitations
The goal is not to eliminate them, but to understand them.
From appraisal to critical interpretation
In the previous chapter, we asked:
π How reliable is this study?
Now we ask:
π What could be misleading about this study?
This is where evaluation becomes truly analytical.
What is bias?
Bias is a systematic error that distorts results. (HernΓ‘n and Robins 2020)
It can:
- exaggerate effects
- hide true effects
- create false conclusions
Bias is not random noise.
It consistently pushes results in a direction.
Common types of bias
Selection bias
- non-representative participants
- inappropriate inclusion/exclusion
π affects generalizability
Measurement bias
- inaccurate measurements
- inconsistent methods
π affects outcome validity
Confounding
- external factors influence results
Example:
- age, comorbidities, baseline differences
π affects causal interpretation
Reporting bias
- selective reporting of outcomes
- omission of negative results
π affects completeness of evidence
Attrition bias
- loss of participants
- incomplete follow-up
π affects reliability of conclusions
What are limitations?
Limitations are constraints of the study.
They may include:
- small sample size
- short follow-up duration
- restricted populations
- limited endpoints
Limitations do not invalidate a study.
But they restrict what can be concluded.
Quality vs bias
These are related but different:
- Quality β how well the study was conducted
- Bias β how results may be systematically distorted
A study can be:
- high quality but still biased
- low quality with unclear bias
Why this matters
Without explicitly identifying bias and limitations:
- weak evidence may appear strong
- inappropriate conclusions may be drawn
- clinical claims may become indefensible
Structured approach
For each study, ask:
- What biases may be present?
- How large could their impact be?
- What are the key limitations?
- How do these affect interpretation?
From bias to confidence
Bias and limitations directly influence:
π how much weight a study should carry
Not all evidence is equal.
Key takeaway
Understanding bias and limitations is essential for:
- accurate interpretation
- balanced evaluation
- defensible clinical claims
What comes next
The next chapter focuses on risk-benefit evaluation, where evidence is translated into clinical value.