In January 2024, PSI was acquired by ETS, creating a partnership that brings together a suite of products and services in measurement science – including industry-leading test security capabilities.
As reflected in the ETS Test Security Mission Statement, test security proactively mitigates the risk of malpractice to build trust and confidence in test score integrity. We achieve this through rigorous security measures that address real and perceived vulnerabilities in both remote and center-based testing.
ETS and PSI test security is evolving and improving to deliver this mission, leveraging innovation to mitigate malpractice. Led by Wally Dalrymple, Chief Security Officer at ETS leads the ETS Test security Officer, the first of its kind in the industry, below shares an overview of the methodology and procedures behind our approach to test security – and how this gives the testing organizations we work with a competitive advantage.
New test security controls
We’ve already initiated a lot of exciting improvements since I joined ETS as Chief Security Officer in 2022. And some of the most significant developments involve biometrics. For example, ETS deployed keystroke as our first biometric control post-Covid in 2022. In 2023, our biometric suite was further enhanced with the launch of vocal recognition and a facial recognition internal consistency tool.
Facial and voice recognition is particularly useful for verifying the identity of test takers who are retaking a test or exam, and we launched facial and voice retaker models earlier this year. The suite of security controls was also improved with the launch of biometric quality checks, essay similarity detection, and additional detection and prevention controls for both remote proctoring and test center delivery.
We are conducting further research in areas such as ChatGPT detector and room scan similarity, with more items on the way. It’s an exciting time to be working in test security and we are learning and sharing across the ETS and PSI teams to maximize our partnership and explore where we might go next.
Test security methodology
Of course, individual security controls and measures don’t exist in isolation. To be truly effective, they must be supported by a rigorous methodology and test security strategy that covers the whole assessment lifecycle.
The ETS global test security methodology does just that, starting with prevention, through to detection, response, and advanced fraud analytics. This is aligned with the PSI approach to test security and the importance of a strategy that takes a robust approach at every stage of the assessment lifecycle, from test development and how you work with Subject Matter Experts (SMEs), through to test delivery and post test data forensics.
Here are just some of the considerations for a robust test security methodology at four stages of the assessment lifecycle:
- Malpractice prevention
Online proctoring with video and audio monitoring, cyber security measures, privacy and data compliance, location assessments for test centers, PC health checks for remote testing. - Malpractice detection
Voice / facial / keystroke recognition, essay and speech similarity reviews (automated and / or AI based), identity checks, AI detection (ChatGPT), online and in-person proctoring. - Malpractice response
Real time monitoring and audits of test centers and online proctors, further investigations (review of recordings and identity checks), closing rogue test centers, internet and social media engagement, clear processes for cases and appeals. - Advanced fraud analytics
Statistical analysis of testing data (response times, response patterns, extreme changes or shifts), monitoring application use, flagging registration abnormalities, IP tracking, network and device use monitoring.
Multi-pronged and multi-layered approach
The ETS and PSI partnership brings together the largest and most experienced test security team in the testing industry. ETS is also the only assessment company that has three biometric checks (keystroke, voice and facial recognition) and we are working closely with the PSI team to explore where these technologies might be relevant to their testing services and beneficial for PSI clients.
ETS and PSI are strengthened by our shared position on key areas of test security:
- The importance of human intervention and review when AI technology is used.
- Security teams committed to threat hunting and threat assessment.
- Strict security procedures across our global network of test centers.
Test security procedures
As well as test center security, our shared position with PSI extends to many of the security processes applied in remote testing with online proctoring. This includes a thorough check-in process, 360-degree room scan, inspection of items such as glasses or watches, and procedures for restricting the use of additional devices such as cell phones during a test.
Again, we are working with PSI to explore areas where we can work together to enhance test security even further.
Leveraging testing experience
Learnings that we can share with PSI extend to how we jointly face the inherent challenges of test security. For example, ETS is official provider of the TOEFL test (Test of English as a Foreign Language), one of the biggest English language tests in the world, accepted by more than 11,000 universities and institutions in over 190 countries and territories.
With 2.3 million tests delivered every year, there is a lot that can be learned from TOEFL about emerging methods of test malpractice, such as the operations of cheating rings and how they communicate. We are also working with PSI to share lessons about the actions ETS takes in the event of test malpractice, such as turning information over to law enforcement, and processes for undercover operations and take down notices.
Shared focus for the future
Artificial Intelligence (AI) projects are an important area of focus for the future of test security, for both ETS and PSI. This includes AI enhanced authenticity detection, which evaluates responses for content similarity, ChatGPT created responses, and mechanical speech patterns. The ETS biometrics suite also holds potential for confirming the identity of test takers before, during and after the launch of a test to identify professional test takers and seat switching behaviors.
AI video indexing can also scan remote tests for a predefined set of banned or suspicious behaviors to address the use of prohibited devices or other persons in the testing environment. And AI investigative tools can mine large data sets for malpractice indicators to quickly identify large scale cheating rings or new suspicious methods. These tools have either already been implemented or are in development by ETS.
Working together, ETS and PSI take a threat driven approach to test security – securing test content to prevent pre-knowledge, protecting against impersonation or assistance by others, and securing the test environment. The focus for our partnership is to mitigate malpractice through the innovative use of technology, with informed use of AI supported by human intervention. This includes improving identification checks with biometric functionality, preventing the use of known software on remotely controlled computers, and hardware detection capabilities – all to the end of advancing our detection and response to test malpractice.