Data forensics is part of a broader test security focus for PSI in 2025. Our test security roadmap is ambitious – and moving towards real time data forensics is a crucial element of this.
Based on our extensive experience with successfully detecting fraudulent test taking, we know that data forensics works. The next step is to make it faster, speeding up the transition from detecting an issue to acting, and then moving into prevention. It’s a big step which we are taking in three phases.
A three-phased approach to real time data forensics
1. Automated next day flagging
We recently implemented next-day flagging for a specific use case, after detecting a group of test takers with the same IP address. This was hugely beneficial, as our security team didn’t have to wait for the full report to start investigating. The security team’s initial findings contributed to the full report, speeding up the process and adding depth to our early findings.
In the first half of 2025, we are planning to build on this early success with the launch of automated next day flagging using simple but powerful indices to complement our comprehensive suite of analyses.
Another element that will enhance our ability to quickly respond to data forensics flags is in how we manage and track investigations of flags. Working in partnership with ETS, we will be leveraging some powerful tools designed to see all the elements related to a security concern in one place and track them across the various stages of investigation and action. We are all collaborating effectively to build a case, compile the evidence, and move towards the necessary action right from the first stages of uncovering a potential security incident.
2. Near real time data forensics
The next step is gathering information in ‘near’ real time – without directly interacting with a test taker or a proctor. A candidate flag will be applied at the end of the test. There will then be an option to withhold a test result based on whether the flags raised exceed a preset criteria or threshold.
3. Real time data forensics
The final phase is leveraging our data forensics capabilities to the point that we are supporting the proctor in real time. Used in this way, data forensics is another tool that helps our proctors focus their attention in the right place. It’s an added layer to the other test security measures in our toolkit, such as deepfake detection and advanced identity verification, that are part of our offering.
It’s exciting to be working with such a talented team, bringing the best people together with the right technology to make this happen.
Advances in data forensics algorithms
Real time data forensics is just one of the advances we have been working on. My colleague Greg Hurtz, who laid the foundation for many of PSI’s internal algorithms, recently published a paper with Regi Mucino considering similarity analysis and test time to detect potential malpractice.
We’ve also developed cluster analysis, that goes beyond conducting a similarity of the two test takers in a group that have the most in common. The earlier approach meant we weren’t using a lot of potentially valuable information, as an individual test taker might match with multiple other test takers, not just one. Cluster analysis not only detects an issue but also detects the scale of the issue. Operational information is then applied to find whether it’s an issue associated with a test center, school or IP address, for instance.
Adding and updating indices
Other updates have been made to align our data forensics programs with changes in the broader testing world. For example, as more testing programs move from fixed forms to Linear On The Fly Testing (LOFT), we needed to expand our suite of indices to keep up. We have added indices to ensure our analysis stays sensitive to the things we’re looking for.
Read our blog on how to ensure the ethical interpretation of data forensics.
Web crawling 2.0
A powerful add on to our data forensics service is web crawling. It has been an extremely successful offering, and now we are taking it up a notch with two goals:
1. Increase the reach of detection
This includes getting as far as we can into internal and closed groups as possible. We are also incorporating image detection, with image to image and image to text detection. Dark web monitoring is another addition that was previously expensive but is now becoming more accessible.
2. Improve AI detection
We are improving our ability to detect whether someone is using a generative AI tool, such as ChatGPT, to generate practice test content. Then we can make sure that practice test content isn’t matching anything in the live item bank that would compromise a live test.
Read our blog on identity-centric security to combat evolving threats.
Investing in people and technology
Our data forensics goals are big, and we are investing heavily to achieve them. This includes the technological capabilities needed to keep up with the evolving threats – and opportunities – that AI and machine learning present. As well as shoring up our team’s capabilities to take these developments forward.