Data forensics is part of a broader test security focus for PSI in 2025. Our test security roadmap is ambitious. Integrating data forensics into the assessment lifecycle and moving towards real time data forensics is a crucial element of this.
Based on our extensive experience with successfully detecting fraudulent test taking, we know that data forensics works. The next step is to make it faster, speeding up the transition from detecting an issue to acting, and then moving into prevention. It’s a big step which we are taking in three phases.
A three-phased approach to real time data forensics
1. Automated next day flagging
We recently implemented next-day flagging for a specific use case, after detecting a group of test takers with the same IP address. This was hugely beneficial, as our security team didn’t have to wait for the full report to start investigating. The security team’s initial findings contributed to the full report, speeding up the process and adding depth to our early findings.
In the first half of 2025, we are planning to build on this early success with the launch of automated next day flagging using simple but powerful indices to complement our comprehensive suite of analyses. Our comprehensive suite of analyses ensures that test security includes a range of methods, from automate flagging to in-depth manual reviews.
Another element that will enhance our ability to quickly respond to data forensics flags is in how we manage and track investigations of flags. Working in partnership with ETS, we will be leveraging some powerful tools designed to see all the elements related to a security concern in one place and track them across the various stages of investigation and action. We are all collaborating effectively to build a case, compile the evidence, and move towards the necessary action right from the first stages of uncovering a potential security incident.
2. Near real time data forensics
Near real-time data forensics is a critical component of test security that involves analyzing data in near real-time to identify potential security issues. This approach enables test security teams to quickly identify and respond to security threats, reducing the risk of security breaches.
Near real-time data forensics involves using automated tools to analyze data from various sources, including test taker data, online proctoring data, and test center data. This approach helps security teams to identify potential security issues, such as proxy testing or stolen test content, and identifies security issues by analyzing data as it is collected. A test taker flag will be applied at the end of the test. There will then be an option to withhold a test result based on whether the flags raised exceed a preset criteria or threshold.
Near real-time data forensics is essential in today’s fast-paced digital landscape, where security threats can emerge at any moment.
3. Real time data forensics
The final phase is leveraging our data forensics capabilities to the point that we are supporting the proctor in real time. Used in this way, data forensics is another tool that helps our proctors focus their attention in the right place. It’s an added layer to the other test security measures in our toolkit, such as deepfake detection and advanced identity verification, that are part of our offering.
It’s exciting to be working with such a talented team, bringing the best people together with the right technology to make this happen.
Advances in data forensics algorithms to address security issues
Real time data forensics is just one of the advances we have been working on. My colleague Greg Hurtz, who laid the foundation for many of PSI’s internal algorithms, recently published a paper with Regi Mucino considering similarity analysis and test time to detect potential malpractice.
We’ve also developed cluster analysis, that goes beyond conducting a similarity of the two test takers in a group that have the most in common. The earlier approach meant we weren’t using a lot of potentially valuable information, as an individual test taker might match with multiple other test takers, not just one. Cluster analysis not only detects an issue but also detects the scale of the issue. Operational information is then applied to find whether it’s an issue associated with a test center, school or IP address, for instance.
Adding and updating indices
Other updates have been made to align our data forensics programs with changes in the broader testing world. For example, as more testing programs move from fixed forms to Linear On The Fly Testing (LOFT), we needed to expand our suite of indices to keep up. We have added indices to ensure our analysis stays sensitive to the things we’re looking for.
Read our blog on how to ensure the ethical interpretation of data forensics.
Web crawling 2.0
A powerful add on to our data forensics service is web crawling. It has been an extremely successful offering, and now we are taking it up a notch with two goals:
1. Increase the reach of detection
This includes getting as far as we can into internal and closed groups as possible. We are also incorporating image detection, with image to image and image to text detection. Dark web monitoring is another addition that was previously expensive but is now becoming more accessible.
2. Improve AI detection with automated tools
We are improving our ability to detect whether someone is using a generative AI tool, such as ChatGPT, to generate practice test content. Then we can make sure that practice test content isn’t matching anything in the live item bank that would compromise a live test. Improving AI detection is another way security testing helps facilitate discussions and understanding of test results.
Read our blog on identity-centric security to combat evolving threats.
Investing in people and technology
Our data forensics goals are big, and we are investing heavily to achieve them. This includes the technological capabilities needed to keep up with the evolving threats – and opportunities – that AI and machine learning present. As well as shoring up our team’s capabilities to take these developments forward. Our investments ensure that test security progresses at the same pace as the evolving risks, enhancing our ability to detect and respond to security threats.