This is Part II in our blog series on The Latest Test Delivery Trends. To read Part I, click here.
In the past, the assessment industry has often been seen as slow moving and averse to rapid change. This view is shifting. Our industry has been fast and effective in its response to the significant events of recent years. But what are the trends we see coming forward in the future of testing? What are the expected events we need to be ready for? And how can we be better prepared for the unexpected? Let’s pick up where we left off.
PSI Market research
We conducted a survey of over 350 individuals within the assessment industry. Respondents represent:
- Professional organizations, government agencies, awarding bodies and beyond.
- Multiple industries including construction, cosmetology, insurance, IT, adult education and more.
- Organizations with testing volumes ranging from under 100 to over 50,000
The results of our survey provide invaluable insights into the state of testing, now and into the future.
Test Delivery Services and Technologies
While testing in test centers remains the most popular option, the survey reflects our own testing data showing that multi-modal is increasingly popular. When it comes to online proctoring, audio and video monitoring is the most popular technology currently used. With real-time client monitoring, biometric authentication, and video analytics the most sought-after technologies for the future. The interest in other technologies such as AI-based marking, wearable technology, and virtual augmented reality was low across all sectors and sizes of organization.
What test delivery services and methods does your organization currently use?
- Test center: 34%
- Multi-modal: 32%
- Online proctoring: 18%
- On-demand testing: 16%
What technologies are you using for online proctoring?
- Audio and video monitoring: 49%
- Browser lockdown software: 43%
- Screen sharing: 23%
- Second camera: 8%
Which of the following technologies will help advance online proctoring?
- Real time client monitoring (Ability to join active sessions): 46%
- Biometric authentication: 36%
- Chatbots and virtual assistant: 31%
- Augmented reality for online proctoring: 13%
- Video analytics: 39%
- Real time data forensics: 32%
- Artificial Intelligence: 30%
Test volume differences
There were some significant differences in responses related to test volumes. Very small organizations (1-10 employees) had the strongest interest in online proctoring. And low-testing volume organizations (under 1,000) are focused on audio-video monitoring in online proctoring. These findings perhaps show that smaller testing organizations with lower volumes have taken longer to make the move to online proctoring, but they are now ready to switch.
High-volume testing organizations, who are likely to have been using online proctoring for longer, are more focused on the use of secure browser lockdown software as part as their online proctoring offer. This may reflect current concerns about the potential use of generative AI tools, such as ChatGPT, to cheat on tests and exams – which is prevented with the use of a secure browser.
While all organizations recognize biometric authentication as a technology that will enhance online proctoring capabilities, high-volume testing organizations express a greater interest in video analytics and real-time data forensics. In contrast, low-volume testing organizations find real-time client monitoring (ability to join active sessions) and video analytics more appealing as technologies for advancing their online proctoring capabilities.
Impact on authenticity, reliability, validity and experience
Despite interest in AI and a belief in its positive impact on test authenticity, validity and reliability, respondents don’t believe these technologies will alleviate test anxiety. A belief in the positive impact of these technologies is particularly prevalent in respondents from high-testing volume organizations.
For example, 62% of respondents from organizations with a testing volume over 50,000, either agreed or strongly agreed that the use of virtual reality and AI will positively affect the validity and reliability of test assessments. This compares to 38% of respondents from organizations with a testing volume under 1,000.
Human oversight remains a crucial concern among most respondents… to ensure fairness and reliability in testing, given the security and ethical considerations tied to the use of AI in testing.
Learn more about AI in Assessment here.
Diversity and inclusion in testing
Most testing organizations are taking multiple steps to ensure equal access to testing. Accessibility technology and physical accessibility measures both top the list of steps currently taken to ensure equal access.
The top way testing organizations are using to ensure the testing environment is inclusive, is through working with a vendor. There seems to be an awareness that testing organizations don’t have all the answers when it comes to inclusive testing and are looking to their testing partner for support in this important area. What’s more, ongoing monitoring and review of processes is also high on the agenda, recognizing that there is still work to be done and this is an ongoing process.
Read more about Diversity, Equity, and Inclusion in testing here.
What steps do you take to ensure test takers from diverse backgrounds have equal access to testing services?
- Accessibility technology (e.g. screen readers): 40%
- Physical accessibility: 37%
- Alternative formats: 31%
- Language translation services: 21%
How Can You Ensure the Testing Environment is Inclusive and Welcoming?
- Rely on or work with PSI or another vendor
- Provide equal/standardized treatment or accommodations
- Continue to monitor and review processes
- Provide information to test takers to create user-friendly processes
- Provide a warm greeting and/or a welcoming environment
Top 3 market research insights
- There is still a demand for in-person testing in secure test centers, despite the increase in popularity of online proctoring.
- Testing organizations are interested in new technologies, but still have concerns about the impact on test taker experience.
- Despite interest in AI, a human-first approach is still the gold standard, where human intervention is involved in any decision making
PSI Improvements and enhancements in test delivery
So, what are the latest test delivery advances that will help us respond to:
- Emerging trends in the testing data.
- Changing needs uncovered by our survey.
The evolution of online proctoring
- Security and integrity
- At the same time as wanting to explore new technologies, testing organizations are understandably wary of the potential impact on test security, reliability and validity.
- While it’s not new, with the increased potential of generative AI to cheat on tests, a secure lockdown browser is more important than ever. A lockdown browser prevents test takers from accessing tools like ChatGPT while they are in the secure testing environment.
- Equally, while video and audio monitoring has been used for many years, its importance is still recognized by testing organizations. For example, to check whether test takers are using additional devices or communicating with another person during their test.
- New technologies
- As well as being experts in tried and tested online proctoring technologies, the team at PSI continuously evaluate new and emerging security measures that may be beneficial to clients. Before implementing any new technologies, we conduct detailed reviews that assess increased test security against any negative impact on the test taker experience.
- For example, we only use AI to alert proctors to potential malpractice, such as noise in the testing environment or when the test taker looks away. The proctor then uses those alerts to inform which violations they flag – human oversight is always involved.
- More rigorous and accurate identity authentication is another tool in our test security kit, to ensure the person taking the test is who they say they are. We are improving the tools we use for this all the time and many PSI clients have adopted fingerprint biometrics as a method of accurate and reliable identification.
- Efficiencies and streamlined processes
- Developing the right test delivery strategy for your organization can be a balancing act between cost, security and test taker experience. When it comes to the test taker experience, we are facilitating a smoother check-in and testing experience with specialist roles in the process. For example, check-in specialists, proctor, customer support, and technical support specialists.
- Additional ways we are streamlining test delivery include integrations with HR systems and Learning Management Systems (LMS) such as Single Sign On (SSO) and Application Programming Interfaces (APIs) to share the latest data directly with client systems.
- Test taker communications is also taking a front seat, with a review of the language we use and the creation of short tutorial videos and animations to clarify processes and reduce test taker anxiety.
- To make sure you are adopting all available improvements and enhancements, it’s helpful to fully examine the test taker journey at every stage, from registering and scheduling to launching a test and receiving results.
Download our guide Trends in Test Delivery to learn more.
