Compertum Logo

The Value of Next Generation Assessment Technology: Measuring Knowledge Certainty

Posted by Michelle Sheils 19/01/2017


Why do employees forget information they have learned during training?

The connection between knowledge, confidence, retention and quality of knowledge was extensively studied by Dr James Bruno whilst working for NATO in the 1960s.

Dr Bruno discovered that the more confident a person felt about their knowledge of a particular subject, the more likely they were to implement that knowledge, and less likely to forget that information in the future.


For effective training outcomes, knowledge and confidence is required in unison.

To understand why employees fail to implement skills that they have learned during training, it is important to understand that mere knowledge is not enough to change behaviour. To empower an employee to implement training in the workplace, they also need to have a high degree of confidence of their knowledge to apply conviction to decisions that they make.

Dr Bruno demonstrated that traditional ‘one dimensional’ forms of assessment (where only the score is recorded) provide limited feedback of knowledge attained and no feedback as to the strength of overall confidence.

What is the best way to measure knowledge in the workplace?

Dr Bruno’s model requires simultaneous measurement of knowledge and confidence, and this is achieved by asking the participant to rate their own levels of confidence during an assessment. While useful in providing a self-reflection tool, asking the test taker to score their own confidence levels has a number of drawbacks:

  • Certain demographics feel reluctant to score their confidence levels as ‘high’ due to certain social norms

  • The capacity for some participants to score their own confidence levels may change based on group dynamics

  • Practice assessments are required, so that a group or organisation can familiarise themselves with the process and for scores to ‘settle’ into accurate measurements

Our online assessment platform provides the next generation in measuring knowledge and confidence simultaneously through an entirely automated process. During online assessment, knowledge and confidence levels of a participant’s responses are evaluated to establish distinct correlations and actionable insight.

  • Certain and knowledgeable
    The participant knows the correct answer and is certain in their knowledge. These employees are likely to take the knowledge learned in training and apply it successfully in the workplace, increasing positive outcomes. Confidence in knowledge also means that knowledge is more likely to be retained over a longer period of time.

  • Uncertain but knowledgeable
    The participant knows the correct answer and is uncertain in their knowledge meaning they are unlikely to act on it. There is also a greater chance their lack of confidence in their knowledge will mean that the information will be forgotten more quickly over a period of time.

  • Certain but misguided
    The participant has misguided certainty in their knowledge and therefore believes the wrong information. This is particularly problematic as these employees are more likely to act on their incorrect knowledge and pose risks to the business.

  • Uncertain and uninformed
    The participant doesn’t have the knowledge and is uncertain in their answers. Participants who have neither the knowledge nor confidence in their knowledge are unlikely to succeed in their current role without further training and development.

Improving outcomes in learning and development

Effective training and cost efficiencies can be achieved through increased insight into participants’ level of knowledge and certainty within that knowledge. Through the use of our assessment platform’s added insight, training departments are empowered to create effective tailored training plans as a result of more focussed training needs analysis.

It is important that employees have conviction and belief in their skills in order to retain information and be able to deploy their learning in practical, demonstrable ways to the benefit of the organisation.

Further reading:

Information Reference Testing (IRT) in Corporate and Technical Training Programs, by James Bruno; UCLA 1995. (paper-based)

Confidence based marking – towards deeper learning and better exams A.R. Gardner-Medwin chapter 12 Bryan C and Clegg K Innovate assessment in Higher Education (2006)

Testing in the workplace

Posted by Michelle Sheils 20/12/2016


​​How and why testing improves workplace learning and development

Attaching tests and quizzes to online training programmes serves a number of purposes; testing re-enforces important points from a training programme, and can help candidates with memory retention. Testing can grade workforce skills and identify areas of training that may need to be addressed. Testing has a number of crucial roles in workplace learning; used effectively and with the correct feedback, is a vital element of any workplace training programme.

There are a number of constructive ways testing can be used to enhance training:

Pre-training tests measure the knowledge at the outset of a course or programme of learning. The associated benefits include:

  • Identifies and measures the baseline skill of the workforce

  • Ensures the training course or online learning content is appropriate to the employee needs

  • Provides the employee with a preview of the learning terminology as well as an insight into the content of training

  • Sets the employee’s training expectations

Formative testing is the term used when the test results adjust the learning path or approach to training; the aim of the formative test is to provide feedback. The associated benefits include:

  • helps the employee recognise areas that they need to improve (or aim for)

  • Involves both the employee and the assessor reflecting on improving the methods of learning/training

  • Involves an element of self-reflection and self-assessment by the employee

  • Identifies the best learning resources or strategies to be used to achieve goals

Summative or Post training testing assesses the effectiveness of the course and whether learning objectives have been achieved. Post-test training is used for measuring learning outcomes.

Testing can and should be used in a variety of ways to enhance and improve the learning and development process. Without thorough testing the true outcomes of training are difficult to identify and difficult to plan for. Our platform’s robust reporting procedures are unique in the industry by providing a clear analysis of training success.

Our performance-response technology is an intuitive and user friendly platform for creating pre-training, formative and summative testing and uncovers key areas to improve training and enhance learning outcomes.

Our platform’s reporting software identifies:

  • The likelihood of candidates’ guessing test questions

  • Confusing test questions that require clarification

  • Poorly instructed training not representing skills required

  • Incorrect or inappropriate training effecting assessment results

  • Misleading test questions resulting in wrong answers

  • Whether the course training reflects the learning outcomes

Our platform enables a level of reporting that enables your organisation to gauge the true return of investment in all methods of testing and training by using our unique performance response technology.

Contact us to find out more about how our performance-response technology can help you realise the true impact of your workplace training.

​​Ten reasons to bring your workplace assessments online

Posted by Michelle Sheils 11/11/2016


The most effective way to determine if your organisation’s training programme is competent and successful, is to test and assess your employees.

Assessment enables you to track employee performance, identify training and performance gaps, qualify skills for new employees and provide robust evidence for regulatory compliance.

So why should you bring your workplace testing and assessments online?

  1. Online tests are extremely efficient and cost effective to deliver Online assessments can be devolved to the whole workforce at the click of a button. As assessments can be accessed on any mobile device, it is no longer necessary to use costly training classrooms and PCs to ensure that testing can reach the whole workforce.

  2. Geography is not a problem As assessments can be delivered at any time, in any place, the geographical location of employees is no longer a barrier to testing.

  3. Results and analytics are immediately available Online assessment software has built in reporting, which can supply immediate, rich, and detailed feedback; both to the candidate and to the assessor so they can determine the success of the test.

  4. Eliminates human error when marking Statistically human error is an inevitable part of marking paper based assessments, while automatic marking provided by online assessment tools is 100% error free, 100% of the time.

  5. Ease of creating unique exam papers As online exam software enables questions to be stored in a question bank, exam papers can be securely created, in minutes. Automating shuffling or randomising test questions reduces the likelihood that candidates can cheat and provides each person with an individual and appropriate test paper.

  6. Online assessments create a visually rich and interactive testing experience Image based testing, drag and drop activities, audio and video based testing means that multimedia based assessments make a more engaged experience for the user and can test how the employee would react in real life situations.

  7. Security Secure user names and passwords protect both the integrity of the question bank, and the security of test papers. Online exams can be deployed with a security wrapper to ensure access to any other programme is not allowed during the duration of the test.

  8. Collaborative authoring of tests Secure user permissions and passwords enable several people to access areas of a question bank, without enabling access to test papers. This means that subject matter experts can contribute quickly and easily to test papers, without compromising the security of the exam or assessment.

  9. Invigilator triggers Online assessment ensures that tests can be made available to groups of employees, or all employees concurrently, and tests can be made available or withdrawn during a specified time period.

  10. Online assessments identify problems with training processes and learning outcomes Not all workplace assessment tools are built the same. Resource Group’s Performance-Response Assessment software is an online assessment tool which enables the authoring, delivery and analysing of test papers in a unique way. Our platform allows your organisation to gain deeper analysis into the effectiveness of training, the effectiveness of the test and knowledge retention of employees with our unique data capture, analysis and reporting system.

Our platform is capable of providing all the benefits of other online workplace assessments whilst allowing you to access richer data and reports, adding a new dimension to your analysis.

Our platform is an intuitive tool for:

  • Building test papers

  • Delivering test papers on any pc or mobile device

  • Analysing test papers

  • Understanding how test results give an insight into return of investment in training

  • Misleading test questions resulting in wrong answers

Contact us to find out more about how our performance-response technology can help you realise the true impact of your workplace training.

How traditional methods of workforce assessment are failing

Posted by Michelle Sheils 04/09/2016


With the proliferation of learning management systems with built in online assessment features, online testing in the workplace is now accepted and widespread.

However, there are pitfalls with workplace testing (online or paper based) that until now, have proved difficult to account for:

  • Employees guessing answers

  • Employees misunderstanding questions

  • Identifying weak training processes

  • Identifying poor learning content

It is obvious that these factors cost time and money and invariable problems, but often are only recognised when avoidable and sometimes critical complications appear in the workplace.

How can we identify and counteract these obstacles to successful assessments?

Traditional testing processes measure the success of training via score based testing programmes, but statistically, for the reasons mentioned above, high scores in tests do not necessarily correlate to high levels of knowledge or competency skills in a job role.

Traditional workplace training models and online assessments look like this:

Training

Classroom training

Online training

On the job training

Testing

Online assessment

Scoring only

Outcomes

Pass or fail

Retrain

Resource Group's software is an online assessment tool with built in reporting to gain insight into barriers to learning:

Our platform's online assessment software looks like this:

Training

Classroom training

Online training

On the job training

Testing

Online assessment

Scoring + behaviour

Outcomes

Pass

Identify inadequate training

Identify inadequate content

Identify false positives/negatives

Retrain

Our performance-response assessment software enables companies to understand how their training may be improved to eliminate false positives and false negatives in their testing and assessment processes.

Our platform enables your organisation to test your workforce, but with real insight into the true levels of workplace knowledge.

Our platform is an intuitive tool for

  • Building test papers

  • Delivering test papers on any pc or mobile device

  • Analysing test papers

  • Understanding how test results give an insight into return of investment in training

Contact us to find out more about how our performance-response technology can help you realise the true impact of your workplace training.