Training Evaluation and Kirkpatricks Level Model

Kirkpatricks's Four Level Training Evaluation Model

 

Donald Kirkpatrick's originally published his ideas of learning evaluation in 1959. His  book “Evaluating Training Programs” 1994  defined his theories more clearly so that his 4 levels theory’s of criteria or impact is now considered an industry standard across the HR and training communities.

More recently Don Kirkpatrick formed his own company, Kirkpatrick Partners, whose website provides information about their services and methods, etc

An additional level Return on Investment (ROI) (see Dr Jack Phillips) has been added as a optional 5th level to this training evaluation model.

The ROI of a learning initiative is a factor of its costs and the quantified business benefits.

ROI = Costs - Benefits/Costs x 100

Level

Measurement Focus

1. Reaction & Planned

Measures participant satisfaction with the learning opportunity and captures planned actions, if appropriate

2. Learning

Measures changes in knowledge, skills and attitudes (competencies)

3. Application

Measures changes in on the job behaviour or actions

4. Business Impact

Measures changes in business impact variables both tangible and intangible

5. Return on Investment

Compares project benefits to the costs

 

There is a causal relationship between the levels as a good reaction is more likely to lead to better learning, without learning their can be no application on the job, no application on the job means no business impact.
"Without doubt larger organisations would be conducting some training evaluation at

Levels 1 and 2 but how useful is this?

Are we guilty of doing what we have always done or doing what is easy as opposed to what is right?

 

More information on training evaluation here

Level

Measurement Focus

Ease

Frequency of Use

Value to Stakeholders

1. Reaction & Planned

Measures participant satisfaction with the learning opportunity and captures planned actions, if appropriate

Easy

Common

Low

2. Learning

Measures changes in knowledge, skills and attitudes (competencies)

Easy

Common

Low

3. Application

Measures changes in on the job behaviour or actions

Moderate

Uncommon

Moderate

4. Business Impact

Measures changes in business impact variables both tangible and intangible

Difficult

Rare

High

5. Return on Investment

Compares project benefits to the costs

Difficult

Rare

High

It is hard to justify a training evaluation approach that appears to ignore the potential needs of stakeholders.


More on developing an outcome focus using instructional design

Check out this introductory video on training evaluation and Kirkpatrick's levels

How Can You Use Kirkpatrick’s Model?

One straightforward approach to incorporating business objectives into training and its evaluation is presented by Dyer, who proposes that Kirkpatrick's levels of training evaluation be mirrored, to ask the questions shown in the diagram at the design stage of a training intervention:

The levels are used to understand needs and objectives in the design stage which are translated into a more robust programme with its evaluation already built in.

Having delivered the training, the training evaluation process is achieved by following Kirkpatrick's model in the normal fashion.

Level

Examples

Methods

Implications

1. Reaction

 

Did the trainees like and enjoy the training?

Did they consider the training relevant?

Was it a good use of their time?

Did they like the venue, the style, timing, domestics, etc?

Level of participation.

Ease and comfort of experience.

Level of effort required to make the most of the learning.

Typically 'happy sheets'.

Feedback forms based on subjective personal reaction to the training experience.

 

Important to know that people were not upset or disappointed

Can be done immediately the training ends.

Very easy to obtain reaction feedback

Feedback is not expensive to gather or to analyse for groups.

.

 

2. Learning

 

Did the trainees learn what was intended in the learning outcomes?

 

Typically assessments or tests before and after the training.

Can include trainer observation in role plays and activities

Can include development of action plans

 

Relatively simple to set up, but more investment and thought required than reaction evaluation.

Is easier for training with clear business outcomes like cost savings, service and efficiencies

Less easy for more complex learning such as attitudinal development,.

3. Behaviour

 

Did the trainees apply their learning when back on the job?

Was there noticeable and measurable change in the activity and performance of the trainees when back in their roles?

Was the change in behaviour and new level of knowledge sustained?

Is the trainee aware of their change in behaviour, knowledge, skill level?

 

Collect data from trainee and their manager and if necessary co-workers

Take the most conservative data to build credibility in evaluation

Review implementation of action plans

 

Behaviour change is measurement is more difficult to evaluate and interpret than reaction and learning data

Cooperation and skill of observers, typically line-managers, are important factors, and difficult to control.

Needs to be built in up front

.

 

4. Results 

 

Measures would typically be business or organisational key performance indicators, such as:

Output

Cost

Quality

Time

Service feedback

Many of these measures are probably already in place

The challenge is to identify which measures and how they relate to the trainee's input and influence and when to measure .

Important to agree expectations and measures up front with trainees

.

 

Assessing individual results can be achieved without too much difficulty

Survey techniques can be used where no other data is available

Assessing large numbers across the organisation is more costly and complicated.

If things improved how do you it was due to the training and not something else?

How to isolate the affects of the learning event from other business changes is also a challenge.



More on training needs analysis here

Back to learning evaluation