Skip to Main Content
It looks like you're using Internet Explorer 11 or older. This website works best with modern browsers such as the latest versions of Chrome, Firefox, Safari, and Edge. If you continue with this browser, you may see unexpected results.

Evidence Based Practice: Critical Appraisal

What is a critical appraisal and why should you us it?

The critical appraisal of the quality of clinical research is one of the keys to informed decision-making in healthcare.  Critical appraisal is the process of carefully and systematically examining research evidence to judge its trustworthiness, its value and relevance in a particular context.

Critical appraisal skills promote understanding of:

  • which treatments or interventions may really work;
  • whether research has been conducted properly and has been reported reliably;
  • which papers are clinically relevant;
  • which services or treatments are potentially worth funding;
  • whether the benefits of an intervention are likely to outweigh the harms or costs;
  • what to believe when making decisions when there is conflicting research.

Critical appraisals are done using checklists, depending on the type of study being appraised. Common questions asked are:

  • What is the research question?
  • What is the study type (design)?
  • What selection considerations were applied?
  • What are the outcome factors and how are they measured?
  • What are the study factors and how are they measured?
  • What important potential confounders are considered?
  • What is the statistical method used in the study?
  • How were statistical results used and applied?
  • What conclusions did the authors reach about the research question?
  • Are ethical issues considered?

Information from Al-Jundi & Sakka, 2017; CASP, 2018; Mhaskar et al., 2009

Study Types

Different types of clinical questions are answered by different types of study design.

Randomised Controlled Trial (RCT)

Used to answer questions about effects. Participants are randomised into two (or more) different groups and each group receives a different intervention. At the end of the trial, the effects of the different interventions are measured. Blinding (patients and investigators should not know which group the patient belongs to) is used to minimise bias. 

Strengths Weaknesses
  • Best type of study to determine causality, high level of evidence
  • Expensive and time consuming
  • High level of blinding
  • High internal validity, but potentially only low external validity
  • Eliminates confounding bias (known and unknown)
  • Ethical constraints
  • Strict guidelines for study protocols
  • Confounding variables may still appear, regardless of randomisation

Non-Randomised Controlled Trial

This type of study does not apply randomisation or uses a method that does not meet randomisation standards, like eg. alternate assignment to groups, age-based groupings, etc. After the allocation of participants to groups, the Non-Randomised Controlled Trial resembles a Cohort Study.

Strengths Weaknesses
  • Inexpensive and can be applied to a sample of patients in a clinical practice
  • Methodology not as strict as for a RCT
  • Use of a concurrent control group
  • Selection bias can occur
  • More scientific rigour than analytical studies
  • Potential lack of generalisabilty
  • Uniform determination of outcomes for both participant groups
  • Limited method of randomisation may lead to issues with confounding factors

Cohort study

Participants or subjects (not patients) with specific characteristics are identified as a 'cohort' (cohort=group) and followed over a long time (years or decades). Differences between them, such as exposure of possible risk factor(s), are measured. Used to answer questions about aetiology or prognosis. Cohort studies are a form of longitudinal study design that flows from the exposure to outcome. Prognostic cohort studies start with a group of patients with a specific condition and follow them up over time to see how the condition develops.

Strengths Weaknesses
  • Multiple outcomes can be measured for any one exposure
  • Time-consuming and costly
  • Demonstrate direction of causality
  • Prone to bias due to lack of follow up
  • Multiple exposures can be considered
  • Study participation may alter participant behaviour
  • Measurement of incidence and prevalence
  • Prone to confounding

Case-control study

Looks at patients (cases) who already have a specific condition and match them with a control group who are very similar except they don't have the condition. Medical records and interviews are used to identify differences in exposure to risk factors in the two groups. Used to answer questions about aetiology, especially for rare conditions where a cohort study would not be feasible. 

Strengths Weaknesses
  • Cost effective
  • Prone to bias (esp. selection, recall and observer bias)
  • Does not require a long follow up period, due to being retrospective
  • Limited to examining one outcome
  • Good to examine multiple exposures
  • Choice of controls can be an issue for proper comparison
  • Efficient for diseases with long latency periods or rare diseases
  • Limited in estimating incident rates

Cross-sectional study/survey

A representative sample of a population is identified and examined or interviewed to establish whether or not a specific outcome is present. Used to answer questions about prevalence and diagnosis. For diagnostic studies, the sensitivity and specificity of a new diagnostic test is measured against a 'gold standard' or reference test. Cross-sectional studies can be descriptive or analytical.

Strengths Weaknesses
  • Quick and easy to conduct
  • Does not measure incidence, only prevalence
  • Multiple exposure and outcomes can be studied                                                      
  • Susceptible to bias
  • Can measure prevalence for all factors being considered
  • Not suitable for the study of rare diseases or diseases with a short duration
  • Useful for descriptive analyses and for generating hypotheses
  • Factors are measured together, resulting in difficulties to determine whether the exposure preceded or followed the outcome

Critical Appraisal Tools

"Critical appraisal is the process of carefully and systematically examining research to judge its trustworthiness, and its value and relevance in a particular context."  (Burls, 2009)

Choosing an appraisal tool

Critical appraisal tools are designed to be used when reading and evaluating published research.

For best results, match the tool against the type of study you want to appraise. Some of the common critical appraisal tools are included here.

Critical Appraisal Skills Programme (CASP) checklists

CASP provides a number of checklists covering RCTs, cohort studies, systematic reviews and more.

JBI Checklists

Joanna Briggs Institute Critical Appraisal Tools have been developed by the JBI and collaborators and approved by the JBI Scientific Committee, following extensive peer review. JBI offers a large number of appraisal checklists for both experimental and observational studies. Word and PDF versions of each are available.

STROBE Statement checklists

The STROBE checklists are designed for the reporting of observational (cohort, case-control, and cross-sectional) studies and can be applied to the critical appraisal of these types of study. Includes individual and mixed study checklists.

Critical appraisal

Charles Darwin University acknowledges the traditional custodians across the lands on which we live and work, and we pay our respects to Elders both past and present.
CRICOS Provider No: 00300K (NT/VIC) 03286A (NSW) RTO Provider No: 0373 Privacy StatementCopyright and DisclaimerLibrary Webmaster • ABN 54 093 513 649