Make it meaningful
The irony of the fact that data is almost immeasurable when it comes to proving the impact of your L&D isn’t something that escapes us here at Kineo. The secret to getting the most out of your data is being able to prove and measure what value has been added.
How do you measure impact?
If you’ve been in L&D long enough, you’ve probably been asked questions about the impact of learning solutions. Your internal voice may be saying, “Yeah, of course we’re making a difference,” but you probably also feel a sense of unease when you think about your ability to prove it. You are not alone.
Brandon Hall Group’s 2020 Learning measurement study found that less than 16% of organizations are very effectively able to identify and track a range of metrics including participation, satisfaction, knowledge transfer, behavior change and business impact across all their learning activities. In our experience, one of the main reasons for this shortfall is that most of the energy around measurement is spent discussing the best way to measure the impact of a specific course or program. While this makes sense in some cases, it also requires significant company commitment. On the other hand, we see that too little energy is being invested in a broader learning analytics approach.
A model for evaluation
Kirkpatrick’s evaluation model, first developed by Dr. Don Kirkpatrick in 1993, is still one of the most popular training models used today in evaluating training programs. It consists of four training evaluation levels, with each level building on the previous one. These include response, learning, behavior, and outcomes.
The levels described in Kirkpatrick’s assessment model establish clearly defined objectives to achieve productive metrics. This is one of the golden rules when it comes to getting the most effective results and understanding your bottom line. Know how the data you collect will help you: help you make design decisions, teach you about the training or the audience, or prove the impact?
“One in two L&D leaders are asked to prove that they add more value. That’s a pretty strong stat, with just 6% saying they were under less pressure to prove more value” – David Wilson, Fosway
7 steps to smarter data
Here are 7 approaches to data collection for smarter measurements.
1. Business Impact
Business impact is about trying to make a direct correlation between training and a business metric. A common example is linking a new sales training program to the success of your salespeople. It can be one of the trickiest data points to measure, but it can be done by thinking creatively about how learning ties into business outcomes, or by doing a control test, for example.
2. Behavioral Change
This approach starts with building a behavioral model as part of your needs analysis. The behavioral model identifies both positive behavior – what we want our audience to continue or increase – and negative behavior – behavior we want to reduce or eliminate. The measurement strategy revolves around different methods to quantify the frequency of this behavior before and after training.
3. Application:
Assessing an application is about creating different experiences in which the skills and knowledge covered in the training should be used. The most common approach is a scenario-based assessment that provides the opportunity to evaluate learners against hypothetical or anticipated work situations. This approach is especially useful for assessing students on areas where mistakes and/or failures at work have major consequences.
4. Knowledge retention
Knowledge assessments are ubiquitous in corporate learning events or courses to measure a learner’s ability to recall facts and terminology. Typically, these assessments appear at the end of a course or module as knowledge checks or quizzes, and as final assessments at the end of the course.
5. Trust
Reliability assessments are metacognitive (meaning thinking about your own thinking!), requiring students to indicate that they are aware of their thinking. It is the student’s self-assessment of their own confidence about a choice or decision, usually given in retrospect after the choice has been made. Students answer a question and then rate their confidence in their answer.
6. Engagement
Unlike the categories above, engagement data isn’t about the content being taught. Instead, it’s about measuring activity. The most common data in this category are course registrations or start, completions, and time spent.
7. Response
Response data is usually collected via ‘smile sheets’. This data reflects the learner’s view or response to learning. Questions can range from overall satisfaction (“did you like it”) to measuring helpfulness to the student’s work, to likelihood of improving performance.
kineo
Kineo helps the world’s leading companies improve their performance through learning and technology. We combine quality in learning with award-winning customer service and innovation. We are here to meet your learning and performance challenges and deliver results.
Originally published on kineo.com†