Quality assurance is an ongoing process when it comes to online proctoring. There is always more we can do to improve the proctoring experience for our test takers and clients. At the start of 2024, while test taker and client feedback were positive, we could see that check-in and other stages of the online proctoring process sometimes caused friction points for test takers. Balancing test taker experience with secure test delivery, whether onsite or remote, is crucial with robust monitoring essential to safeguard against fraud. That’s why our recent focus has been on Bottom Quartile Management (BQM), putting our efforts into the performance metrics that fall within the bottom 25%.
As we examined the feedback data for opportunities to improve, we didn’t just want to shift the numbers. We wanted to uncover where the real issues were and bring about positive changes that made a difference to our test takers. We looked at opportunities to improve online proctoring for long-term clients, where recent technology has become available, or we might have missed something in the past. And we looked at ways we could improve the onboarding process for newer clients with our online proctoring service.
Read our online proctoring checklist for 2025.
Align quality and proctor training for online proctoring
A big area of focus was to better align our quality and training teams. We knew this would enable us to not only locate opportunities for improvement in online proctoring processes, but also quickly and effectively introduce those improvements into our daily interactions with test takers taking a remote online proctored exam. Various online proctoring methods can offer significant benefits – such as effectiveness, convenience, and efficiency – compared to traditional testing methods.
One tool we introduced was monthly Kaizens involving the quality and training teams. These monthly meetings are based on the Japanese philosophy of progressive business change. Key stakeholders are involved in identifying problems, analyzing existing processes, and generating solutions – leading to actionable plans and continuous improvement in the test taker experience.
These Kaizens were a step up in frequency from our previous quarterly meetings and offer a more regular opportunity to:
- Share any upcoming online proctoring platform or other product changes with the team.
- Remind the team of recent product or online proctoring process changes that have been introduced.
- Highlight and discuss friction points where improvements could be made.
Improve online proctoring processes for test security
Remote proctoring plays a crucial role in ensuring test security and integrity during online examinations, including identity verification and monitoring test delivery experiences. One example is the room scans that take place before a remote online proctored exam. Data from our surveys showed us these can be a friction point for test takers during check-in. Feedback and discussions with our check-in teams backed this up.
Every remote online testing environment is different, and this led to some ambiguity around test taker expectations and exactly what was required from check-in teams. Online proctoring needs to achieve a balance between test security and test taker privacy, as we are often in the test taker’s home. So, we introduced more detailed guidelines for check-in teams, to reduce ambiguity. In addition, we updated our test taker communications to clarify what they should expect from a room scan.
Another example is how we communicate with test takers. We reviewed and made improvements to the timing of communications and the email templates we use, so that online proctoring processes are clearer for test takers – before, during and after their test. This has led to a reduction in friction points and test taker queries.
Similarly, we looked at where testing programs have specific rules, how these are communicated to test takers, and how they are shared with our check-in teams and online proctors. We reviewed documentation for certain programs, including where there are exceptions to standard processes. This might be around the identification documents required during check-in, or materials allowed during a test. This is where oversights can happen, so clear communication with test takers and documentation for our own proctoring teams is essential.
Full circle of continuous improvement
Another area where we introduced improvements is around how we annotate test taker records related to behavior and environment during a remote online proctored exam. For example, has the online proctor seen any violations or do they have any observations? Different clients have different security levels with specific use cases, so recording the right level of detail is essential – particularly when further action is needed.
In all these cases, we moved through a full circle of continuous improvement. From the quality team identifying areas that needed improvement in the BQM, to introducing training or other enhancements, to feedback and ongoing dialogue that then drives product innovations in our online proctoring software, platform or services.
Overcome language barriers in online proctoring
As a result of this continuous improvement process, we have made enhancements to our online proctoring platform, specifically the templated scripts used by check-in and online proctoring teams. This came about as we noticed inconsistencies in communication with test takers, particularly during less frequent events or when the team had to go off script. In addition, a client asked us to conduct a piece of work around the language gap in the healthcare sector. This uncovered where there might be broader issues, so we extended the project to benefit all clients.
Further investigation showed that inconsistencies were largely due to the standard of English in our scripts – C2, the highest level of fluency. As we were hiring to B2 level, which is fluent at a more intermediate level, it’s understandable that some team members were struggling with C2 level scripts. As a result of this finding, we rewrote our scripts to B2 level, helping our team and test takers whose first language is not English. At the same time, we aligned our online proctoring platform to hiring practices and test taker needs.
We know that because we have onshore and offshore online proctoring teams that language can be a barrier, and this is a key area to reduce friction points. Verbal and written communications with test takers are a continued focus for us. For example, where industry specific terms don’t translate well or when a direct translation means something different in another language. We want to ensure that the words we use across all our systems don’t affect the testing experience.
Discover a rigorous process for inclusive and accurate test translations.
Increase familiarity with online proctoring
A test, exam or assessment is a way for our test takers to progress in their education or career. Testing is not their occupation. They may not have tested remotely before and might not be familiar with online proctoring. We need to remember this and look at our processes as if we are coming to them for the first time – just like they are. How do we break down what needs to happen in an easy-to-understand way, so they can become more familiar?
We can see that many BQM issues arise from this lack of familiarity and experience with online proctoring. A test taker’s room might be cluttered, or a door might be left open, for example. There’s a common misconception that taking an online proctored test is just like working from home, which is not the case. The testing environment needs to be configured in a way that supports test security.
It’s a multi-team effort to change this perception and improve our communications and processes to ensure test takers are ready for their test. This includes the product team, test taker experience team, quality and training teams, and technical support. All working in partnership with Client Success, who work in partnership with clients to regularly review their websites and documentation.
What’s next for online proctoring quality assurance
In 2024, all these improvements and enhancements contributed to a reduction in friction points of over half a million. Test taker call volumes dropped by 40% alongside a reduction in client service tickets of 30%. Client and test taker satisfaction scores increased and continue to rise.
We know there is still more to do. One priority for 2025 is security and introducing technology that supports our online proctoring teams. This includes advanced ID validation that uses AI backed by human decision making, as well as liveness checks that look for deepfakes. It’s exciting to see this evolution and how we can bring technology, people and processes together to improve the test taker experience.
Read our guide to quality assurance in testing.
A bit more about… online proctoring
Ensuring test integrity
Ensuring test integrity is crucial in online proctoring to maintain the credibility of the exam process. Several test security measures can be implemented to prevent test fraud:
- Proctor training: Proctors should be thoroughly trained to detect and prevent fraud, ensuring they are well-versed in maintaining the integrity of the exam process.
- ID verification: Test takers must verify their identity before taking the exam to prevent impersonation. This step is essential to ensure that the person taking the test is the registered test taker.
- AI-supported proctoring: This technology flags potential issues for further review by human proctors. Automated proctoring software monitors test takers and detects suspicious activities using AI algorithms.
- Live Online Proctoring: Live proctors supervise test takers in real-time, providing an added layer of security. They monitor webcam video and audio to detect any irregularities.
- Secure testing environment: The testing environment should be secure and free from distractions. This includes ensuring that the room is free from unauthorized materials and that the test taker is not interrupted during the exam.
By implementing these measures, online proctoring effectively safeguards the integrity of the exam process, providing a fair and secure testing experience for all test takers.
Ensuring integrity using live online proctoring
Live proctoring is a type of online proctoring that involves a human proctor supervising test takers in real-time. This method provides an added layer of security, making it ideal for high-stakes exams or certifications. During a live proctored exam, proctors monitor the test taker using the webcam and audio on their device to detect any suspicious behavior. This real-time supervision helps ensure the testing environment remains secure and that the test taker adheres to all exam rules.
Live online proctoring with a human proctor can be used with AI-supported automated proctoring software to create a comprehensive security solution. AI algorithms can efficiently monitor multiple test sessions simultaneously, supporting live proctors to focus on specific areas that require closer attention. This combination of human oversight and advanced technology provides a robust framework for maintaining test security and integrity.
With live online proctoring, organizations can ensure that their online assessments are conducted in a secure and controlled environment, thereby upholding the standards of their certification programs and protecting the value of their credentials.