This document provides information about a workshop on evidence-based medicine (EBM) for residents. The workshop objectives are to present clinical EBM summaries to peers and critically reflect on applying clinical studies to practice. The document reviews the EBM process and provides worksheets and resources for critically appraising different study designs, including randomized controlled trials, cohort and case-control studies, and systematic reviews. Key points of the critical appraisal worksheets are summarized for each study design. Logistical details are provided for the next workshop.

1. Jewish General Hospital
LIFE SCIENCES LIBRARY Library
Workshop 4: Resident
Presentations
Chantal Cassis, MD
Robin Featherstone, MLIS
Francesca Frati, MLIS
Summer/Fall 2012 Roland Grad, MDCM, MSc, FCFP

2. Workshop 4 - Objectives
By the end of the workshop, you will be able to:
1. Present clinical EBM summaries to your peers
2. Critically reflect on the practical application of a
clinical study

3. Workshops
 July 25 - Introduction to EBM for Haematology
 Aug 8 - Hands-on Searching Workshops
 Aug 22 - Critical Appraisal
 Sept 5 - Resident Presentations
 Sept 19 - Review

4. EBM Process
Formulating
Workshop Evaluating the clinical
the Process Workshop
5 question
1
Your patient for whom
you are uncertain about
therapy, diagnosis, or Searching
Incorporating prognosis the Evidence
evidence into
Workshop
decision-making
2
Workshop
Workshop 3
4
Appraising
the Evidence

5. Worksheets & Resources
 Critical Appraisal Worksheets from JAMA Evidence:
http://jamaevidence.com/
 Critical Appraisal Worksheets from Dartmouth
(include calculations for odds ratio, relative risk,
absolute risk, etc.):
http://www.dartmouth.edu/~library/biomed/guides
/research/ebm-teach.html

6. Therapy (RCTs)1
1. Are the results valid?
A. Did intervention and control groups start with the same prognosis?
 Were patients randomized?
 Was group allocation concealed?
 Were patients in the study groups similar with respect to known prognostic
variables?
B. Was prognostic balance maintained as the study progressed?
 To what extent was the study blinded?
C. Were the groups prognostically balanced as the study progressed?
 Was follow-up complete?
 Were patients analyzed in the groups to which they were first allocated?
 Was the trial stopped early?
1. http://jamaevidence.com/criticalAppraisalWorksheet/27

7. Therapy (RCTs), cont.
2. What are the results?
A. How large was the treatment effect?
 What was the relative risk reduction?
 What was the absolute risk reduction?
B. How precise was the estimate of the treatment effect?
 What were the confidence intervals?

8. Therapy (RCTs), cont.
3. How can I apply the results to patient care?
A. Were the study patients similar to my population of interest?
 Does your population match the study inclusion criteria ?
 If not, are there compelling reasons why the results should not apply to your
population?
B. Were all clinically important outcomes considered?
 What were the primary and secondary endpoints studied?
 Were surrogate endpoints used?
C. Are the likely treatment benefits worth the potential harm
and costs?
 What is the number needed to treat (NNT) to prevent 1 adverse outcome or
produce 1 positive outcome?
 Is the retention of clinical endpoints worth the increase of cost and risk of harm?


9. Harm (Cohort Studies, Case-Control Studies)2
1. Are the results valid?
Cohort Studies: Aside from the exposure of interest, did the exposed and
control groups start and finish with the same risk for the outcome?
 Were patients similar for prognostic factors known to be associated with the
outcome (or was statistical adjustment done)?
 Were the circumstances and methods for detecting the outcome similar?
 Was the follow-up sufficiently complete?
Case-Control Studies: Did the cases and control group have the same
rise (chance) for being exposed in the past?
 Were cases and controls similar with respect to the indication or circumstances
that would lead to exposure?
 Were the circumstances and methods for determining exposure similar for cases
and controls?
2. http://jamaevidence.com/criticalAppraisalWorksheet/23

10. Harm (Cohort Studies, Case-Control Studies), cont.
2. What are the results?
A. How strong is the association between exposure and outcome?
 What is the risk or odds ratio?
 Is there a dose-response relationship between exposure and outcome?
B. How precise was the estimate of the risk?
 What is the confidence interval for the relative risk or odds ratio?

11. Harm (Cohort Studies, Case-Control Studies), cont.
3. How can I apply the results to patient care?
A. Were the study subjects similar to your patients or population?
 Is your patient so different from those included in the study that the results may
not apply?
B. Was the follow-up sufficiently long?
 Were study participants followed-up long enough for important harmful effects to
be detected?
C. Is the exposure similar to what might occur in your patient?
 Are there important differences in exposures (dose, duration, etc.) for your
patients?
D. What is the magnitude of the risk?
 What level of baseline risk for the harm is amplified by the exposure studied?
E. Are there any benefits known to be associated with the exposure?
 What is the balance between benefits and harms for patients like yours?

12. Summarizing the Evidence (Systematic Reviews)3
1. Are the results valid?
A. Did the review explicitly address a sensible clinical question?
 Is the underlying biology or sociology such that, across the range of interventions
and outcomes included, the effect should be similar?
 Did the review include explicit and appropriate eligibility criteria?
B. Was the search for relevant studies detailed and exhaustive?
 Were sources of evidence and search strategies specified in sufficient detail for
replication
 Was the likelihood and direction of publication bias considered?
C. Were the primary studies of high methodologic quality?
 Were clear methodological selection criteria specified?
 Were all included studies accessed by these criteria?
D. Were selection and assessments of studies reproducible?
 Was an explicit approach used to select and extract data from all included
studies?
 Was study selection and assessment validated by a blinded second observer?
3. http://jamaevidence.com/criticalAppraisalWorksheet/26

13. Summarizing the Evidence (Systematic
Reviews), cont.
2. What are the results?
A. Were the results similar from study to study?
 How similar were the point estimates?
 Do confidence intervals overlap between studies?
B. What are the overall results of the review?
 Were results weighted both quantitatively and qualitatively in summary
estimates?
C. How precise were the results?
 What is the confidence interval for the summary or cumulative effect size?

14. Summarizing the Evidence (Systematic Reviews),
cont.
3. How can I apply the results to patient care?
A. Were all patient-important outcomes considered?
 Did the review omit outcomes that could change decisions?
B. Are any postulated subgroup effects credible?
 Were subgroup differences postulated before data analysis?
 Were subgroup differences consistent across studies?
C. What is the overall quality of the evidence?
 Were prevailing study design, size, and conduct reflected in a summary of the
quality of evidence?
D. Are the benefits worth the costs and potential risks
 Does the cumulative effect size cross a test or therapeutic threshold?

15. Next Workshop
 10:30 am to 12:30 pm
 Rm 519 (Dean’s conference room), McIntyre Medical
Building, McGill
 Slides available: http://www.slideshare.net/featherr