Heterogeneity Checklists

Assessing Heterogeneity Within Studies

The “average” benefit observed in a clinical trial may not be completely representative of the treatment effect for a typical patient. This 5-item checklist is adapted from a framework developed by Kent et al.1 to assess differences in the risk of the outcome of interest in clinical trial populations. This checklist prioritizes the analysis and reporting of multivariate risk-based difference in treatment effect. A standardized and transparent approach to assessing heterogeneity of treatment effects could substantially improve the interpretation and use of clinical trial data. Click here to download this checklist.


Assessing Heterogeneity Across Studies

This 4-item tool for assessing heterogeneity across studies was adapted from an 11-item instrument that was developed and validated for assessing the methodological quality of systematic reviews, called the “assessment of multiple systematic reviews” or AMSTAR.2 To assess heterogeneity across studies, ask yourself these questions: 1) Were the studies presented graphically to visualize heterogeneity among the study results?; 2) Were the methods used to statistically combine the findings of the studies appropriate? 3) Was the likelihood of publication bias assessed?; and 4) Were the characteristics of the included studies provided? Click here to download this checklist.


  1. Kent DM, Rothwell PM, Ioannidis JP, et al. Assessing and reporting heterogeneity in treatment effects in clinical trials: a proposal. Trials. 2010;11:85. PMID: 20704705. Available at: http://www.trialsjournal.com/content/11/1/85. Accessed on: 07/30/2012. 
  2. Shea BJ, Hamel C, Wells GA, Bouter LM, Kristjansson E, Grimshaw J, Henry DA, Boers M. AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. J Clin Epidemiol. 2009;62:1013-20. PMID: 19230606.#