Appraise a study's risk of bias

Critical appraisal asks how much a study's result can be trusted, and the right questions depend on the design. Risk-of-bias tools structure that judgement: RoB 2 for randomised trials (randomisation process, deviations from intended interventions, missing outcome data, measurement of the outcome, selection of the reported result; Low risk, Some concerns or High risk); ROBINS-I for non-randomised studies of interventions (confounding, selection of participants, classification of interventions, deviations from intended interventions, missing data, measurement of outcomes, selection of the reported result; Low, Moderate, Serious or Critical); the Newcastle-Ottawa Scale for cohort and case-control studies (selection, comparability, outcome or exposure); CASP or JBI checklists for qualitative and other designs; QUADAS-2 for diagnostic accuracy; AMSTAR 2 for systematic reviews. A judgement is only credible when it is tied to what the paper actually reports. Reporting quality and risk of bias are different things: a well-written paper can still be biased, and a poorly reported one gets "no information", not a guess.

Appraise this studyOnly if [OUTCOME] is given: for the outcome "[OUTCOME]". Only if [DESIGN] is given: Stated design: [DESIGN]

1. Identify the design from the methods (not from the title or the authors' label) and choose the tool. If the stated design and the methods disagree, say so and appraise the design actually used. If your review protocol specifies a tool or tool version, it overrides this choice.
2. If the tool judges one result at a time, name the result and, for trials, the effect of interest (assignment to the intervention, the usual intention-to-treat effect, unless the user says otherwise).
3. Go through every domain or checklist item of the tool. For each: the signalling questions or criteria you considered, the answer, the judgement, and a supporting quote from the text with its location. Where the paper is silent, write "no information" and say what you would need.
4. Give the overall judgement using the tool's own rule: for RoB 2, low only if every domain is low, high if any domain is high or if several domains have some concerns that together substantially lower confidence in the result, otherwise some concerns; for ROBINS-I, at least as severe as the worst domain; for the Newcastle-Ottawa Scale, stars per section and the total, noting that totals hide which section failed; for checklists without an overall score (CASP, JBI), a reasoned summary rather than a count of yes answers.
5. Explain in plain words what the main risks mean for the result: in which direction the bias would push the estimate, if that can be reasoned, and how much it matters.
6. Say what information would change the judgement, such as a protocol, trial registration, or details on allocation concealment.

• Every judgement needs a quote or a precise pointer to the text. No quote, no judgement: use "no information" or "unclear".
• Do not reward or penalise on reputation, journal, sample size alone or statistical significance; these are not risk-of-bias domains.
• Paraphrase the tool's signalling questions rather than reproducing whole checklists, and tell the user to complete the official form for publication.
• Separate reporting gaps from evidence of bias.
• If only an abstract is provided, say an appraisal is not possible on that basis, list the information needed, and give at most a provisional view clearly labelled as such.
• Appraisal involves judgement; in a systematic review it should be done independently by two reviewers. Say so once.

Design and tool

Design as identified, tool chosen and why, result and effect being appraised.

Domain judgements

Table: domain or item | key questions considered | judgement | supporting quote (location).

Overall judgement

One line with the overall rating, then two to four sentences on what it means for the result.

What would change the judgement

Bullets.