This document discusses evaluation of training and development programs. It begins by outlining Kirkpatrick's four levels of learning evaluation, including reaction, learning, behavior, and results. It then provides guidelines for conducting formative and summative evaluations using tools like questionnaires, surveys, tests, interviews, and observations. Common challenges in evaluating training like resistance are also addressed, with recommendations to emphasize feedback and continuous improvement. Overall, the document provides a comprehensive overview of evaluating the effectiveness of training programs using different models and metrics.

1. Evaluation of
Training &
Development
Presented By,
Janhavi Rege
– MMS 131
Ruchi Gupta
– eMBA 12112
Megha Thakkar
–
eMBA 12058
Supriya Semwal
–
PGDM 12119

2. Training Cycle

3. What is a training needs analysis?
 A training need is a shortage of skills or abilities, which
could be reduced or eliminated by means of training
and development.
 Training needs analysis identifies training needs at
employee, departmental or organizational level in order
to help the organization to perform effectively.

4. How can a need to identified?
• Complaints from staff, customers/clients
• Poor quality work
• Frequent errors
• Large staff turnover
• Deadlines not being met
• Conflict amongst staff
• New equipment systems
Indicators
of a
need

9. Telling only
Unidirectional
Blah blah blah…and
more blah blah…
Zzz
?
Teaching
Bidirectional
Can somebody
tell me what
addiction is?
Addiction is a brain
disease
Why?
Let’s use
some
examples

10. Conduct Training
•Training does not need to be limited to a four walled
room;
•It can be taken outside for demonstrative training,
such as Fire Extinguisher training where you would
use a live fire,
OR
•Waste Water Operator training where you could be
in an actual wastewater treatment plant.

11. What Is Evaluation
A program evaluation is the systematic collection of
information about the activities, characteristics, and
outcomes of programs to make judgments about the program,
improve program effectiveness, and/or inform decisions about
future programming.

12. Types of Evaluation
Formative Evaluation
Summative Evaluation
What is it?
•Formative evaluation occurs while
a training program is forming or
occurring.
Summative evaluation takes place
after the training program has
occurred.
When?
•At the time of incident
•End of the Course
•End of the day/weekly
•Mid-point in the course
Why?
•Analyze strengths & weakness
towards improving
•Feedback
•Document achievement
•Evidence of regular formative
evaluation

14. How to Conduct Formative
Evaluation
• Review the training materials with one or two trainees.
• Hold group discussions with the trainees to gain feedback.
• Use the materials in a situation similar to that of an actual
training program to see how the materials work.
• Assess the materials with managers and supervisors who oversee
trainees participating in training program.
• Observe trainee behavior.
• Give short tests to trainees.

15. How To Conduct Summative
Evaluation
• Ask trainees for their opinions about the training program after
it has been delivered.
• Test trainees to learn how well they grasped the information.
• Ask participants to demonstrate how they would use the
information learned in training.
• Conduct surveys or interviews with each participant to gain
better understanding of what they learned.
• Measure changes in production and quality of work that has
been accomplished after the training program.

16. Five Steps of Training Evaluation

17. • Learn from Experience for Future Improvement
• To decide who should participate in future programs
• To establish a database that can assist management in making
decisions

18. "The Four Levels of Learning Evaluation.“
Donald L. Kirkpatrick Model
• The model was defined in 1959 by Donald L. Kirkpatrick.
• Kirkpatrick redefined the evaluation model with his 1998 book
"Evaluating Training Programs.
•
The idea behind the model is for an organization to have
meaningful evaluation of learning in the organization.
• The degree of difficulty increases as you move through the
levels.

20. Level 1: Reaction
• Kirkpatrick refers to Level 1 as a measure of
customer satisfaction.
• Most of the forms that people fill out at the end of
a class or workshop are instruments for measuring
Level 1.

21. Guidelines that Kirkpatrick recommends to get
maximum benefit from reaction
1. Determine what you want to find out
2. Design a form that will quantify reactions
3. Encourage written comments and suggestions
4. Get a 100 percent immediate response
5. Get honest responses
6. Develop acceptable standards
7. Measure reactions against standards and take the appropriate action
8. Communicate reactions as appropriate.

22. Level 2: Learning
• Kirkpatrick defines learning as the extent to which
participants change attitudes, increase knowledge,
and/or increase skill as a result of attending a
program.
• So to measure learning we need to determine the
following:
 What knowledge was learned
 What skills were developed or improved
 What attitudes were changed

23. Guidelines for evaluating learning
1.Use a control group if it is practical
2. Evaluate knowledge, skills, and/or attitudes both before
and after the program. Use a paper and pencil test to
measure knowledge and attitudes and use a performance
test to measure skills.
3. Get a 100 percent response
4. Use the results of the evaluation to take appropriate
action.

24. Level 3: Behaviour
• Level three can be defined as the extent to which a change
in behaviour has occurred because someone attended a
training program.
• In order for change in behaviour to occur, four conditions
are necessary:




The person must have a desire to change
The person must know what to do and how to do it
The person must work in the right climate
The person must be rewarded for changing

25. Guidelines for evaluating behaviour
1.Use a control group if that is practical
2. Allow time for a change in behaviour to take place
3. Evaluate both before and after the program if that is practical
4. Survey and/or interview one or more of the following:
trainees, their immediate supervisors, their subordinates and
others who often observe their behaviour.
5. Get a 100 percent response
6. Repeat the evaluation at appropriate times
7. Consider cost versus benefits

26. Level 4: Results
• This involves measuring the final results that
occurred because a person attended a training
session.
• This can include increased production, improved
work quality, reduced turnover, etc.
• It is important to determine whether the conditions
set forth above in Level 3 have been met.

27. Guidelines for evaluating results
1.Use a control group if it is practical
2. Allow time for results to be achieved
3. Measure both before and after the program if it is
practical
4. Repeat the measurement at appropriate times
5. Consider cost versus benefit
6. Be satisfied with evidence if proof is not possible

29. Various evaluation tools can be selected depending on the purposes and
methods of evaluation.
•Questionnaires
•Surveys
•Tests
•Interviews
•Focus group discussions
•Observations
•Performance records

30. • Keep responses anonymous
• Distribute questionnaire forms in advance
• Pre-Test & Post-Test

31. Questions can fall into the following categories:
Questionnaire
• Keep responses anonymous
• Distribute questionnaire forms in advance
• Pre/post(Tests)

32. Questions & Scales
• 5-point and 7-point scales are the most used
options in questionnaires as they give
respondents the ability to discriminate
between choices.
• Note that coding the response options with
numbers can assist in data entry and
analysis.

34. How to deliver a questionnaire
• Written Questionnaires
• Web-based Questionnaires
• Phone Questionnaire

35. Advantages of questionnaires
• Economy - Expense and time involved in training interviewers
and sending them to interview are reduced by using
questionnaires.
• Uniformity of questions - Each respondent receives the same set
of questions phrased in exactly the same way. Questionnaires
may, therefore, yield data more comparable than information
obtained through an interview.
• Standardization - If the questions are highly structured and the
conditions under which they are answered are controlled, then
the questionnaire could become standardized.

36. Disadvantages
• Respondent’s motivation is difficult to
assess, affecting the validity of response.
• Unless a random sampling of returns is
obtained, those returned completed may
represent biased samples.
• An individual may hide his real attitude
and express socially acceptable opinions.

37. • Limitations of Surveys
• While there are many benefits of surveys, they also
have some limitations. For instance, a survey only
gathers information about the questions asked. In
contrast, during an interview, the interviewer can
explore important subjects in depth, as they are
uncovered. Generally speaking, surveys are
effective only when those surveyed have at least a
moderate degree of literacy. Although oral surveys
are possible, they are usually impractical.

38. Interview Advantages
• Allows the interviewer to clarify questions.
• Allows the informants to respond in any
manner they see fit.
• Allows the interviewers to observe verbal
and non-verbal behavior of the
respondents.
• Means of obtaining personal information,
attitudes, perceptions, and beliefs.

39. Interview Disadvantages
• Unstructured interviews often yield data
too difficult to summarize or evaluate.
• Training interviewers, sending them to
meet and interview their informants, and
evaluating their effectiveness all add to
the cost of the study.

40. Observation

41. Data Input
Recommendations
Data Analysis
Program
Description
Appendices
Using figures to
present data
Evaluation
report outline
Reporting

42. MEASURING TRAINING
EFFECTIVENESS & IMPACT
1.Prior to training
• The number of people that say they need it during the needs
assessment process
• The number of people that sign up for it
2.At the end of the training
•
•
•
•
•
The number of people that attend the session
The number of people that paid to attend the session
A measurable change in the knowledge or skill at the end of training
Ability to solve a “mock” problem at the end of training
Willingness to try to use the skill/knowledge at the end of the
training

43. 3.Delayed Impact
• Customer satisfaction at X weeks after the end of training
• Customer satisfaction at X weeks after the training when the customers know
the actual costs of the training.
• Retention of knowledge at X weeks after the end of training
• Ability to solve a mock problem at X weeks after end of the training
• Willingness to try the skill/knowledge at X weeks after the end of the
training

44. 4.On the Job Behaviour Change
•
Trained individuals that self –report that they changed their behaviour / used the
skill or knowledge on the job after the training ( Within X months )
•
Trained individuals who’s managers report that they changed their behaviour / used
the skill or knowledge on the job after the training ( Within X months )
•
Trained individual’s whose manager’s report that their job performance changed
( as a result of their changed behaviour / skill) either through improved
performance appraisal scores or specific notations about the training on the
performance appraisal form ( within X months )
•
Trained individuals that have observable /measurable ( improved sales , quality ,
speed etc.) improvement in the actual job performance as a result of their changed
behaviour/skill ( within X months )
•
The performance of employees that are managed by ( or are part of the same team
with) individuals that went through the training

45. RESISTANCE TO TRAINING
EVALUATION
• There is nothing to evaluate.
• No one really cares about Evaluating Training.
• Evaluation is a threat to my job.

46. DIFFERENT WAYS TO OVERCOME
THE RESISTANCE TO TRAINING &
DEVELOPMENT EVALUATION
• The employees /trainees should be made aware that every training
programme is designed with some goals or objectives in mind.
• And some type of knowledge, skill and attitude change is expected
from the participants.
• While long term results are important in making business decisions, the
day to day purpose of evaluation should be used as a feedback
mechanism to guide efforts to reward success

47. •
HDFC Life, one of India's leading private life
insurance companies, offers a range of
individual and group insurance solutions.
•
It has 25 retail and 9 group products in its
portfolio.
•
There are 10 optional rider benefits catering
to the savings, investment, protection and
retirement needs of customers.
•
HDFC life have 600 branches in India
touching customers in over 900 cities and
towns.

48. HDFC Vision & Values
Our Vision
'The most obvious choice for all'.
Our Values
Values that we observe while we work:
• Integrity
• Innovation
• Customer centric
• People Care "One for all and all for one"
• Team work
• Joy and Simplicity

50. Analyzing Training Needs
• Organizational
• Entire organization, single division or department
• At this level you prepare for future need
• Job/Task
• Single job category
• Indicated by low productivity
• Individual
• Individual employee
• Indicated by poor review or employee assistance
request

51. Think About it
• Would you internally train, use prepackaged
programs, or outsource for the following topics:
• Sexual Harassment (WHY?)
• Technical training on a manufacturing line (WHY?)

52. Step 2. Design
• Identify Target Audience
• Learning Styles: Visual, Auditory, Kinesthetic
• Identify and talk with Stakeholders
• Develop Training Objectives
• Develop course Content
• Consider Learning Curves
• Develop Evaluation Criteria

53. Think About It:
• Think of this class, Training and
Organizational Development as a prolonged
training program. Can you think of a
training objective for this class. Remember
be SPECIFIC!!!

54. Step 3. Development
• Decide on the Training Materials
• Leader guides, Manuals, Handouts
• Decide on Media Use
• Computer, Television, DVD player, etc.
• Instructional Methods
• Active (Facilitation, Case Studies, Simulations, Vestibule,
Socratic Seminar)
• Passive (Lecture, Presentation, Conference)
• Experiential (Demonstration, One-on-One, Performance)

55. THINK ABOUT IT
What type of training instructional method would you use in
these circumstances: Passive, Active, Experiential?
• You want to have a training program that helps employees
build stronger analytical and problem solving skills.
• You want to have a training program that will help
employees in the human resource department with the
newly installed HRIS software.
• You want to have a training program for hundreds of
employee on new organizational policies and procedures

56. Program Delivery Mechanisms
• Classroom
• Self-Study
• Programmed instruction
• E-learning
THINK ABOUT IT:

57. Step 4. Implementation
• Facility
• Depends on type of training, number of participants, and budget
• Onsite or offsite
• Theater-style, classroom style, banquet-style, chevron style, conference style, ushaped style seating
• Trainers
• Skills, knowledge, and/or abilities with training material
• Schedule
• Feasibility of shutting down operations, employee availability, trainer availability,
multiple sessions
• Arrange for delivery
• Training materials, needed media products
• If offsite
• Ensure everyone has a ride
• All expenses have been paid

58. THINK ABOUT IT
What would you do (onsite vs. offsite training)?
•
The organization for which you are trainer is currently not to doing to well
financially.
•
The only onsite training facility is directly across from the work plant, and it
is extremely noisy and distracting. Recently you accidentally overheard a
conversation between the vice president of HR and Budget management
about how they feel that you are a drain on HRs’ budget.
•
The training material that you are to present is extremely important and
needs to be clearly understood.
•
What would you do, would just deal with the noise of the plant or would
you argue for more money? If so, what points would you argue?

59. Final Step!!!!! Evaluation
• Goals/Outcomes of Training
Transfer of Training
How do you evaluate these training outcomes????
Productivity/Efficiency
Employee Knowledge
Safety/Quality

60. Common Models of Training Evaluation
•
Donald Kirkpatrick 4 levels of training evaluations in US Training and
Development Journal:
• Reaction Evaluation Method
– Survey (provides trainer feedback not organizational impact)
• Learning Evaluation Method
– Test (provides effectiveness of training info. but no feedback)
– Experimental design: Pre-test/post-test, Post-test/control group
• Behavior Evaluation Method
– Observations, interviews, or surveys 6 wks-6months later
– Provides info. if new skills were transferred to job
• Results Evaluation Method
– Impact on business
– Compares objective statements (turnover, sales, costs)

61. Training Evaluation Metrics
• Cost-Benefit Analysis
Typical Costs
-Trainer’s salary and time
-Trainee’s salary and time
-Material for training
-Expenses for trainer and
trainees
-Cost of facilities and
equipment
-Lost productivity
Typical Benefits
-Increase in production
-Trainer’s salary and time
-Reduction in errors and
-Trainee’s salary and time
accidents
-Material for training
-Reduction in turnover
-Expenses for trainer and
-Less supervision necessary
trainees
-Ability to use new capabilities
-Cost of facilities and
-Attitude change
equipment
-Lost productivity

62. Benchmarking
• Gather training data from your
organization (performance data, sales,
efficiency, etc)
• Gather training data on similar
organization (ASTD, or American
Productivity & Quality Center and the
Saratoga Institute)

63. THINK ABOUT IT
What type of Kirkpatrick’s evaluation methods
would you use?
(Reaction Eval. Method, Learning Eval. Method, Behavior Eval. Method, or
Results Eval. Method)
•
You work for organization whose upper management does not see the
benefit in having training programs other than the initial orientation. You
finally convinced them to give you resources to create an interview training
program for managers.
•
The HR department has become somewhat fearful that departmental hiring
managers will violate some employment laws. Therefore, you are instructed
to put together a training program for managers on basic human resources
employment law. Be sure that they walk away knowing the laws, otherwise
there might be some lawsuits!!
•
You have just completed a brand new training program aimed at decreasing
unnecessary work steps for a production line. You want to pilot test the
program, so you recruit a few employees from the line and a couple of their
managers to go through the training.

65. Evaluation in HDFC
• Smiley Sheets
• Feedback forms
• Pre-training assessment
• Post-training assessment

68. The Three DOMAINS

69. AFFECTIVE DOMIAN
EMOTIONAL LEARNING: FEELING –
Concerned with attitudes, appreciations, interests,
values and adjustments.

70. PSYCHOMOTOR DOMAIN
PHYSICAL LEARNING: DOING -
Emphasizes speed, accuracy, dexterity, and physical
skills.

71. COGNITIVE DOMAIN
RATIONAL LEARNING: THINKINGEmphasis upon knowledge, using the mind, and intellectual
abilities.

72. Level 1. Knowledge
When was this picture taken?
Where was this picture taken?
Question cues: List, define, tell,
label

73. Level 2. Comprehension
What is happening in this picture?
Why are these boys dressed like this?
Question cues: Describe, name, identify,
discuss

74. Level 3. Application
How would you describe the photograph to others?
What caption would you write for this photograph (say,
in a newspaper)?
Question cues: Modify, solve, change, explain

75. Level 4. Analysis
What do you know about their lives based on this photo?
Question cues: Analyze, separate, compare, contrast

76. Level 5. Synthesis
What might these boys say about their work in an interview
setting?
What might they say about their future?
Question cues: Create, construct, plan, role-play

77. Level 6. Evaluation
What is the significance of this photo for the time period
depicted?
Compare this photo with one of three boys from today of the
same age. How are their lives similar? How are they different?
Question cues: Give opinion, criticize, discriminate,
summarize

78. Challenges in HDFC
• Getting an honest feedback
• Problems arising in pre-test and post-test training
methods
• Test should be conducted in the right manner
• Trainees are resistant to evaluation