Measuring Coaching, Training, Consulting Intervention Efficacy and Needs

Practicing Empiricism in Learning Interventions

As learning professionals, we coach, train, consult and facilitate our client teams to measure their activities and create feedback loops so that they can adapt and iterate to learn. We call it empiricism or evidenced based management.

Phillips ROI Model

The Phillips ROI model is an extension to the Kirkpatrick model, the original training and educational evaluation model popular since the 1950s. From the four levels (reaction, learning, behavior, results) in the Kirkpatrick model, Jack Phillips extended it to five levels and also added awareness to intangible benefits and chain of impact to the learning intervention’s evaluation.

Level 1: Reaction, Satisfaction

How did the learners like the program? Seek feedback on intervention contents, format, duration, coach/trainer/consultant’s effectiveness etc. Would they be willing to recommend the program to others?

Level 2: Learning

Measure or assess to what extend the learners have learned the desired knowledge, skills, processes, techniques etc. from the training program, or for coaching programs, new knowledge and insights on their strengths and areas of development etc. What are their level of confidence for applying the new knowledge and skills to their jobs?

Level 3: Job Application, Implementation

Measure or assess behavioral changes observed on the job. To what extent are the learners effectively applying their new knowledge, skills and insights to their work? Any barriers or impediments that prevent the application of the learnings observed? Assess to feedback into the learning program for improved knowledge transfer.

Level 4: Business Impact

Identify areas of business improvement and impact created by the learners as a result of applying their learnings from the program. Collect both objective and subjective data as evidence; the former may include output increases (including productivity and sales), quality improvements, time and cost savings, and the latter may include customer satisfaction, employee satisfaction, customer retention etc. (Requires before and after data, and steps to isolate the learning program’s effect from other factors influencing the business impact.)

Level 5: ROI, Cost-Benefit Ratio

Convert the business impact data from the previous level into monetary value and compare it with the cost of the intervention.

Intangible Benefits

Many of the subjective business impact data are hard to convert into monetary values. It is important to raise the awareness of these intangible benefits as they may not be adequately represented in the ROI. For example, improved energy, motivation, commitment, trust and teamwork, reduced conflicts and stress, increased psychological, people more smiling — it would be hard to assign a dollar value to each of these factors, yet when assessed as a chain of impact (see next), they become apparent as major contributors to business impact.

Chain of Impact

While a client statement such as “Definitely the atmosphere has changed and we can really feel the difference!” (correlation) is much appreciated and a valid measure of success (level 1 “reaction”), a demonstration of success in the form of an “As a result of the program…” statement (causation) would be more tangible evidence for assessing the efficacy of the intervention. We can work together with the client in summarizing the “chain of impact” into a story or narrative following the level 1 to 5 impacts. For example:

Intervention Needs Scales Test

While we use the Phillips ROI model for inspecting and adapting our intervention quality and impact throughout the time we are engaged with a client organization, we need a separate tool to continuously assess whether or not the intervention we are providing to a client is producing value and is needed by the client. We consider it our fiduciary duty to inform clients when it is appropriate to pause, conclude or terminate an intervention.

Transparency, Courage and Commitment

Committing to transparency by continuously measuring ourselves requires courage. At Agile Organization Development, we consider this a hallmark of our professionalism.



Chief Coach, Agile Organization Development ( — we are a tribe of change, transformation, innovation experts

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Takeshi Yoshida

Chief Coach, Agile Organization Development ( — we are a tribe of change, transformation, innovation experts