This lecture was given to UQUMRC members on May 16th 2013.

1. Introduction to Critical
Appraisal
Omar S. Taibah

2. Who You Are?

3. Who You Are?
Doctors first.

4. Who You Are?
Doctors first.
Possible researchers.

5. What is critical appraisal?
• “Critical appraisal is the process of systematically examining research
evidence to assess its validity, results, and relevance before using it to
inform a decision”
(Hill and Spittlehouse, 2001, p.1).

6. What is expected of you?

7. What is expected of you?
• What type of study design?
• Is it reliable? Is it valid?
• What type of bias (if any)?
• How do you fix this bias?
• How would you analyze the data from the study?

8. Study Designs
• Observational vs Experimental.

9. Study Designs
• Observational studies sequence:
- Case Report: One clinical subject.
- Case Series: Group of clinical subjects.
- Cross Sectional: One point of time.
- Case Control: Disease vs Non disease.
- Cohort: Risk factor vs No risk factor.

10. Study Designs
• Experimental studies:
- Randomized controlled trials.
- Community trials.
- Cross over trials.

11. Reliability

12. Reliability
• Consistency!

13. Validity

14. Validity
• Accuracy!

15. Reliability vs Validity

16. Reliability vs Validity

17. Reliability vs Validity

18. Reliability vs Validity

19. Reliability vs Validity
No Reliability
No Validity

20. Reliability vs Validity
No Reliability
No Validity No Validity
Reliability

21. Reliability vs Validity
No Reliability
No Validity No Validity Validity
Reliability Reliability

22. Bias

23. Bias
• Deviation from truth.

24. Bias
• You MUST know type of bias AND best solution!

25. Sampling Bias (Selection bias):
• People in the study don’t reflect general population.
• Convenience sample:
MRC vs Students
• Berkson’s bias:
Hospital data for drugs
• Volunteer bias:
In all medical research

26. Sampling Bias (Selection bias):
• People in the study don’t reflect general population.
• Convenience sample:
MRC vs Students
• Berkson’s bias:
Hospital data for drugs
• Volunteer bias:
In all medical research
• SOLUTION: RANDOM SAMPLE, WEIGHT DATA

27. Measurement Bias:
• Clear:
What do you think of me?
• Hawthorne effect:
Here’s a story for you 
Put yourself in place of the research subject
The fact of observation by itself change what is observed
Measuring you changes you!

28. Measurement Bias:
• Clear:
What do you think of me?
• Hawthorne effect:
Here’s a story for you 
Put yourself in place of the research subject
The fact of observation by itself change what is observed
Measuring you changes you!
• SOLUTION: CONTROL GROUP

29. Expectancy Bias:
• Expectation affects outcome
• Works on level of interaction not evaluation
• Girls are bright, boys are dumb!

30. Expectancy Bias:
• Expectation affects outcome
• Works on level of interaction not evaluation
• Girls are bright, boys are dumb!
• SOLUTION: DOUBLE BLIND DESIGN

31. Recall bias:
• Subjects failing to give accurate answers about the past!

32. Recall bias:
• Subjects failing to give accurate answers about the past!
• SOLUTION: CONFIRMATION

33. Late-look bias:
• Severe diseases less likely to be found.
• Suicide studies.

34. Late-look bias:
• Severe diseases less likely to be found.
• Suicide studies.
• SOLUTION: STRATIFY BY DISEASE SEVERITY

35. Confounding bias:
• Every article is confounded!

36. Confounding bias:
• Every article is confounded!
• SOLUTION: MULTIPLE STUDIES OR META ANALYSIS

37. Design bias:
• Non comparable control group!

38. Design bias:
• Non comparable control group!
• SOLUTION: RANDOM ASSIGNMENT

39. Analyzing data from study

40. Incidence Data
Relative Risk (RR).
• How much more likely?
• Iexposed/Iunexposed
• What type of studies?
Attributable Risk (AR).
• How many more?
• Iexposed - Iunexposed

41. Example: Mortality rate after MI
Men: 10/1,000
Women: 5/1,000
• RR:
• How much more likely men to
die from MI compared to
women?
• How much less likely women to
die from MI compared to men?
Which one we use in rare diseases?
• AR:
• How many more men likely to
die from MI compared to
women?
• How many less women likely to
die from MI compared to men?
• Number needed to treat (Prevention)
• Number needed to harm (risk
factors)

42. Odd’s Ratio:
• Case Control studies only!
• AD/BC
Heart disease No heart disease
Smoking 60 120
No smoking 20 200

43. Odd’s Ratio:
• Case Control studies only!
• AD/BC= 5
• It gives us the strength of risk factor
Heart disease No heart disease
Smoking 60 A 120
No smoking 20 200 D

44. Inferential Statistics
Confidence Interval
• Are you right?
• Reality is not a number
• 95% CI: Z=1.96
• 99% CI: Z= 2.58
P-value
• Are you wrong?
• P < 0.05
• Reject null hypothesis.
• Alpha error.
• P > 0.05
• Do NOT reject null hypothesis.
• Beta error

45. Example on CI:
• Mean: 65
• S: 16%
• N: 16
• Calculate 95% CI

46. Example on CI:
• Mean: 65
• S: 16%
• N: 16
• Calculate 95% CI
• 65+ 1.96(16/4)= (57.1 72.8)

47. Interpreting Confidence Interval
• Change from a single number we have zero confidence in to a range
in which we have 95% confidence.
• Does NOT mean 95% of the subjects in that range.

48. Interpreting Confidence Interval

49. Interpreting Confidence Interval

50. Five Minutes Break

51. Applying critical appraisal
• Checklists are a quick and easy way to learn critical appraisal.
• They do not tell you about the quality of research.
• They enable you to more structured response to the subjective
question.
• Three steps:
- Internal validity: RAMBO
- Statistical validity.
- External validity

52. Let’s do it!

53. Questions?

54. Thank you
@LiverOmar
Omar.Taibah@gmail.com
Liveromar.wordpress.com