If there’s one question that L&D professionals wrestle with constantly, it’s the problem of establishing ROI from L&D programmes.
Unfortunately, there are plenty of reasons why clarifying ROI can be a challenge.
First is what we believe to be a structural problem with testing mechanisms. One of the reasons we set up Cognisco with its scenario-led, real-world testing regime is that traditional compliance testing is inadequate. We’ve all seen how classic compliance merely proves that an individual is capable of echoing back facts that they have just learned, on one particular day. Often these test no more than the ability to repeat words. And often, users take the same quizzes, year after year…
Nowhere else in business is an individual expected to achieve no more than they did last year and somehow remain competitive! These tests don’t help users to deal with the actual challenges they face every day, and they are not tied to commercial objectives. No wonder compliance encourages convergence around an average, rather than producing energised candidates who can improve business outcomes (and help less capable candidates to improve, too).
But in a recent conversation in a Cognisco podcast with Laurent Balagué, founder and CEO of ForMetris, a provider of L&D metrics, many other measurement issues became apparent. Here are just a few of our discoveries:
L&D is no longer one-size-fits-all
Once upon a time, technology was simple, and users had low expectations of it. Learning experiences were measurable precisely because they were limited. Today, we have a multitude of opportunities to learn, on multiple platforms (e.g. mobile) at multiple pricepoints and with, soon, remarkably variable user experiences (e.g. the arrival of augmented reality and virtual reality immersion).
Furthermore, different experiences appeal to different user groups, with younger audiences having a lower tolerance for duration and a higher tolerance for mixed media and technology. Not only are different learning experiences becoming uncomparable, it is clear that different approaches will have different results with different user groups. Translating different experiences into a commercially sound return is going to be a professional skill in itself.
The expectations of internal clients can disengage with those of L&D professionals
Balagué points out that business functions will call on L&D because they have a problem – a problem which they will express in terms of business outcomes (e.g. ‘We need better salespeople, to sell more of our new product”). L&D professionals, of course, deliver two types of change: changes in knowledge and changes in behaviour. Both are essential contributors to business outcomes, but they don’t offer a guarantee.
An effective L&D programme might give the sales team exactly the knowledge and behaviours they need to sell more product, but perhaps they’ll fail because the marketing is wrong, or the pricepoint is terrible, or the product isn’t competitive. L&D professionals must learn the art of engaging effectively with their internal clients, spotting problems before they are lumped at their own door, and clearly demonstrating benefit within their own remit.
As Balagué says, “You know what happens when a KPI goes up? Everybody raises their head and says, ‘It’s because of me’. And when the KPI goes down, they turn their head and look at their neighbour. So, you don’t want to be trapped in this kind of relationship with your business partners, focusing on business outcomes… There has to be a common ground, a common agreement between L&D professionals and operational executives.”
You should measure more than just test outcomes, because L&D isn’t just about tests
Perhaps most importantly, getting metrics from testing might be missing much of the point. Continuous Improvement Strategies (a huge part of Lean, Six Sigma and many other management strategies) is being mirrored in L&D by Continuous Learning Strategies. We learn best by repetition and improvement on the job. We learn best through practical experience. We learn best through cycles of “try, review, refine, repeat”. And this is slowly being applied in L&D contexts. Senior managers have always known this – it’s why they prioritise mentorship and coaching from peers. Annual testing, in particular, misses the iterative value of these engagements. It may be more productive to get qualitative feedback, particularly in 360° format, both at the start of an engagement and at the end.
The right metric depends on who cares about it
Finally, there’s no point measuring what a company doesn’t care about; and that’s a function of culture. Says Balagué, “I see more and more clients now, where engagement is a core KPI that is tracked by the Head of HR and the CEO. Others are still in a more top-down culture where engagement is not as important as doing what you’re told by your manager. But I think that second category is becoming a minority”. Businesses which value engagement tend to value learning for its own sake, and for the contribution it makes to aligned and productive employees. Either way, it’s essential to pick the right metrics for the business need – and again, the metrics from compliance testing should be the lowest baseline of your expectations, not your higher aspiration.
All of these are challenges which smart L&D professionals must navigate. That said, there is plenty of good news too. Companies with a culture of learning are becoming attractive to the best talent in their own right, and that should be a corporate objective in itself. As formal education ceases to be reliable for individuals across their entire careers, businesses are discovering that their L&D function is crucial in developing the talents they need in an increasingly competitive work environment. Similarly, employees see that the labour market is more fluid than ever, and that employees who offer training on-the-job are worth their weight in gold in career advancement.
And none of this is impossible, either. Connectedness, interoperability and the Big Data revolution mean that there are more metrics, qualitative and quantitative, available to us practically in real-time. There’s no excuse for L&D professionals to fly blind; they just need to engage more effectively with their colleagues, and better articulate what they’re aiming for.