About our guest
In this episode, we meet Amanda Green, Director of Expertise at Cognisco. Cognisco provide international clients in the private and public sectors with a range of competency-based employee assessments and consultancy services. Clients include BT, NHS and John Lewis Partnership, plus many more.
A graduate of Cranfield University, Amanda has a MSc in Applied Psychology and is a Graduate member of the British Psychological Society (MBPsS).
Overview
Do you know who your most competent people are? And perhaps more importantly, can you articulate to others how you know that these competent people invariably make the right decisions every time?
Do you think it might be possible to differentiate between employees who make the right decisions for the right reasons from those that just get things right by luck rather than judgment?
We are all aware that the rules and regulations in the business world are becoming more complex with greater penalties for organisations and managers that fall foul of them. Knowing who your most competent people are is no longer “nice to know”, it’s become an important factor in reducing risk in organisational decision making.
In this podcast, Amanda Green of Cognisco explains how bespoke business scenarios can be used to assess and judge not only an employee’s knowledge of a subject area on a particular day, but also their likelihood of making the right decision in context; and the confidence level they had when making that decision .
By the end of this podcast, you will have an understanding of how you find the answer to the question set out at the start: “Do you know who your most competent people are?”
Top quotes
- The Cognisco platform “is a multi-measure approach to identifying confidence, competence, and capability across the workforce. It provides organisations with a range of analytics that provide a better understanding of their workforce.
- We assess whether individuals are likely to do the right thing. So, by asking them about what they will do in a given situation, it identifies whether they’re likely to do that, or equally whether they’re likely to do the wrong thing. The assessment helps identify organisational risk associated with each scenario.
- If people are prepared to admit that they don’t know, they’re more likely to seek out help, rather than go gung-ho, decide, and do the wrong thing without checking it with anyone else.
- We categorise individuals based on how they’ve answered the question in terms of right or wrong, but also on how confident they were in their answers.
- The report hotspots show who is likely to do the wrong thing, who misunderstands a particular topic area. That’s where I think the combination of competence and confidence in our reporting really gives people an insight that they wouldn’t have had before.
The Big Takeaways
- Firstly, an organisation and Cognisco develop business scenarios that are designed to test decision making. This is the basis of all subsequent assessments and reporting.
- Then a number of multiple response questions are designed to examine the right, wrong or nuanced response to each scenario.
- Employees take a test where they are asked to address all of the response questions and select their level of confidence in making each choice
- The Cognisco team then provide a report that places employees onto a heat map of categories that reflect each employee’s decision-making competence. Managers are then able to use this heat map to identify their most and least competent decision makers.
- Finally, Cognisco can help organisations support the learning and development of those seen as risky or less capable decision makers.
Transcription
The People Insights podcast from Cognisco with Owen Ashby – oashby@cognisco.com
I’m here today with Amanda Green, and Amanda is the Director of Expertise at Cognisco. If you could start off Amanda by telling us a little bit about what you do for a living, and why you are the Director of Expertise, that would be awesome.
I’ve worked for Cognisco now for over 10-years, started as a solutions consultant, assessment solutions, progressing to Principle Consultant, but now I work with clients and subject matter experts internally, and externally, hence why I’m called a Director of Expertise. My background is occupational psychology, I work with clients and partners to establish what is it they’re trying to achieve organisationally, at individual team and organisational levels, and then, how can we help them to achieve that.
Obviously, the MINO platform provides a basis for looking at those elements, but the SJT is one part of the MINO platform, and an SJT helps us to identify whether people are likely to make the right decision in the contextually relevant scenario.
Okay, so what is the MINO platform?
The MINO platform, we call it a platform, and the way I use MINO is as a multi-measure approach to identifying confidence, competence, and capability really across the workforce. It provides to people analytics in different ways.
So, it’s really the Cognisco competence management assessment, management people, data, and insight platform, that we use called MINO.
Yes.
And you also talked about SJT, and SJT is what?
SJT is what we call a Situational Judgement Test, and typically we create scenarios with clients that are contextually relevant to assess decision-making, and judgement.
So, it’s an assessment model that we used, which is based on scenarios, that assesses… sorry, what were you saying?
Assesses whether individuals are likely to do the right thing in reality. So, by asking them about what they will do in a given situation, it identifies whether they’re likely to do that, or equally whether they’re likely to do the wrong thing, hence causing the organisation risk.
So, it’s about judgement, it’s about how people will apply knowledge in any given scenario. So, what’s the relevance of having scenarios specifically?
I think the relevance to generating scenarios especially when they’re customised and bespoke for organisations, that is what differentiates us in a lot of ways, because we design these from scratch with clients to make them contextually relevant, so when its deployed, people can decide. The scenario is based on, ‘Here’s a situation, think about that situation, and tell us what you would do in that situation, or tell us what’s the right thing to do, the wrong thing to do’. And because we have a multiple response approach to that, it gives us a number of different ways of assessing somebody’s knowledge and understanding of that topic area.
So, we write a scenario, or we present a scenario, and then we ask people to evaluate a number of different responses.
Yes.
And can they just pick any one, is there some right/some wrong, or is the case of maybe going through them all.
Yes, that’s probably a good opportunity for me to explain the difference between a multiple-choice question, and a multiple response question.
We’re all very used to multiple-choice, and if you’re anything like me, that’s how you got any O-levels or A-levels at all really!
Yes, multiple choice is very well known out there, you typically see them at the end of an assessment programme, or a piece of learning, you might get them at the end of a training session just to check the learning. It’s similar structure to ‘Who wants to he a Millionaire’, type of question, so you’re given a stem of a question, you’re given a number of responses, and you’re asked to pick the one that you think is right. For me, that’s very limiting in terms of it probably tests memory recall rather than anything else, it probably just assesses memory of what you’ve just learnt, you could probably get the right answer by guessing, or eliminating the ones that you know are definitely wrong. So, it’s not really assessing any actual behaviour, or actual understanding in an applied way.
Whereas multiple response however, the way that we write questions is that you have the scenario, and then you’re asked to respond to each and every response in its entirety, so you will then make a judgement as to whether you think that’s the right thing to do, or the wrong thing to do. We also give you the option to say, ‘I don’t know’, which is quite a useful methodology because in reality, if people are prepared to admit that they don’t know, they’re more likely to actually seek out help, ask for help rather than go gung-ho, decide, do the wrong thing without checking it with anyone else.
We often talk about the idea of the unconscious incompetent, so the people who think they know what they’re doing, when in fact they don’t and they’re very confident about it. So, somebody who says, ‘I don’t know’, clearly then is somebody who is consciously incompetent, is that right?
I think that could work against that, it’s somebody who understands that they may not know the answer, or they’re not sure about it, and the fact that they’re prepared to admit they don’t know would suggest that they might actually ask for help, rather than just go ahead and do the wrong thing.
Yes, that’s really good from a learning perspective I guess, in that they recognise they need learning or training, so they’re more likely to buy into the training, or the learning that’s presented to them afterwards. I always think about things as we do in Cognisco, from the business perspective, rather than simply the learning perspective. From an organisation’s perspective, I guess if we’ve got people who are aware that they need learning, aware that they don’t know the answer, that’s probably great from a risk perspective, would you say?
Yeah, I think that’s true to say. The way our questions work is, you have a scenario or a stem, individuals are asked to respond to that, to say what they think is the right answer, what they would or would not do. But equally we then have the confidence scale, we’re asking people, ‘How confident are you in your answer?’ which leads us onto quite nicely what you were saying there about the risk; because the way our reporting works is that we categorise individuals based on, how they’ve answered the question in terms of right or wrong, but also how confident they were in their answers.
For example, somebody who’s chosen to answer the questions, and largely got them wrong, but with a high level of confidence that they were right at the time, they would come out in what we call our misunderstanding with high confidence category, also known as high-risk. They’re at high risk of misunderstanding, high risk of doing the wrong thing in reality. On the other end of the scale you’ve got people who are consciously competent, so they are understanding the subject matter expertise, and they’re also highly confident in their answers.
Brilliant. So, we’re correlating two things, we’re correlating what people really know, and then we’re asking them to apply that knowledge to a given scenario, which I imagine can be quite challenging, and then we’re asking them how confident they are that it’s the right thing to do. So, we’re correlating really, confidence with competence, which I think is as unique as it gets really in the industry isn’t it? So, unlike just being able to test whether people can remember the facts or remember what the rules are in the rule book, we really want to know how people are going to apply that knowledge in any given scenario. So, I would imagine that’s quite useful, quite powerful in terms of where we point learning?
Yes, I think where we’re quite different to a lot of other SJTs out there… SJTs have been around since the 1950s, they were originally developed for recruiting armed forces, the army. That’s progressed but they’re still very much predominantly used for recruitment purposes. Where we’re different is, we use them purposely for development purposes, we use them to identify those who are likely to do the right or wrong thing in reality, identify the people-risk within an organisation. So, the heat map from the report provides what it says really, hotspots as to who is likely to do the wrong thing, who misunderstands this topic area. That’s where I think the combination of that competence and confidence in our reporting really gives a people analytic, that they wouldn’t have had before.
It’s a really interesting analytic, and it probably surfaces some information that people didn’t know about before, it’s asking as you’ve just explained, questions from a different perspective, so it’s about illicit different information. Then what do people do next, we’re not just leaving people with a problem that they didn’t know they have?
Quite often we can link into existing learning, so organisations will typically bring us in to assess what they’re doing, in terms of a development programme, a training programme, so it can be used as a training needs analysis in the first instance, or as a baseline measure, then obviously you can have an intervention in-between and that could be a face-to-face intervention that’s already been designed, or that we can inform. It can also be links to learning with an LMS platform, or PowerPoint presentations that link directly from individuals reports in the system, takes them to wherever they need to be to do that developmental remedial work, and then we give them the opportunity to reassess, and that’s served its purpose in terms of evaluating that they’ve understood the training, that they’ve learnt something, that they’ve improved their knowledge and understanding. It also helps to provide an evaluation level of ROI in terms of improving understanding, but also, likely behaviour.
Brilliant. So, we’re directing people to learning that is relevant to them, based on an assessment that evidences where they have a need, which means potentially all those good folks out there with lots of learning management systems, and great investments in those kinds of things can now start to sweat that asset somewhat, but also once people have been through that learning, we can reassess them, is that what you’re saying, to prove that we’ve closed that gap?
Yes, definitely. It serves two purposes, 1) serves the purpose for the trainer and the organisation to assess whether the learning has landed, whether that’s been effective, but also 2) from the individual’s perspective, they’ve managed to improve their learning and demonstrate that they understand this topic area.
That’s brilliant. So, situational judgement, multiple response analysis, where we are correlating both the knowledge, the competence if you like, also with the confidence to apply that. That’s a really interesting analytic insight into something that’s really special from Cognisco. I think it’s probably fair to say its our predominant analysis, it’s the thing that gets people really excited, and really engaged.
Have you got an example of a really good story where somebody has applied this, and it’s made a real difference? If you can mention company names, but if not then just give us a high-level view just to put it in context for people.
One off the top of my head that I can mention is in conjunction with a partner, the Chartered Management Institute’s worked with us to design management competency-based assessments for performance management, and employee engagement. This is based on national occupational standards, and it was what CMI deemed to be good practice, so this is helping to develop managers, typically at a level 5, but it works for other levels as well, and this is where we’ve created scenarios across different sectors in different spheres of life, to assess whether they were likely to make the right decision to achieve the right outcome, based on what good practice looks like, in line with national occupational standards.
So, we’re quite proud of that, we’re working with CMI on that, it’s been deployed to many different sectors, many different organisations, and its top and tailed some of the CMI learning programmes as well for accreditation, and it works very well. We’ve now developed a 360 to complement it, but that’s another session.
Brilliant, thank you Amanda. So, we’re going to have more conversations, we’re going to be taking about different assessment models, different analytics, how you can apply them, how they work in the workplace. I’m quite sure there’ll be people here who want to learn more about your experience more broadly, and the services that you offer in your capacity as Director of Expertise. What’s the best way of people getting hold of you Amanda, what’s your email address?
My email address is agreen@cognisco.com
Amanda Green, Director of Expertise, thank you very much indeed. Look forward Amanda to speaking to you next time.
Likewise, thank you very much.