We all remember the nerves that would kick in as we sat down to take an important test, whether it be in high school, undergrad, or graduate school. Did we study the right material? Would there be enough time to get through the entire test?
Taking a test can elicit enough stress when taken in a traditional in-person setting. Now imagine taking away that familiarity and moving that experience online. In addition to that, you now essentially have an algorithm acting as the middle man between you and a passing score or your instructor questioning your integrity. It can be unnerving to say the least. New questions come into play, adding to that initial stress. Do I know what steps I need to get through just to start my test? Am I allowed to use the restroom while I’m logged into the testing platform? Is the connection that I’m accessing going to be secure?
The compounding anxiety that students can feel in online proctoring is just one of the reasons why we firmly believe that a human should always be involved. But before making that move to implement online proctoring as part of your curriculum plan, there are multiple layers of information that you need to investigate and prepare for in advance. During a recent session at the Bbworld 2021 conference, I discussed some questions that you should be asking long before rolling out online proctoring at your institution and why they’re worth your consideration.
How is the test monitored?
This is likely the most critical question to consider while you vet online proctoring providers. How a vendor answers this question will have a direct impact on your students. And while many companies have opted to offer an online proctoring service that is fully automated and uses Artificial Intelligence (AI), here at PSI we subscribe to a different philosophy.
We feel that while AI has a place in many industries – including higher ed – it falls shorts on making interpretations on human behaviors. It cannot differentiate between a suspicious vs. accidental or typical behavior and is ripe with false positives. Its inability to understand and report on behaviors and the intentions of those behaviors can create potential bias within the testing process. And while human proctoring is not perfect, it is far more reliable than automation. This is why PSI takes such a strong stance against a fully automated proctoring platform. When evaluating potential test proctoring vendors, you must have a clear understanding of what the tools you’re exploring can do and what they can’t. Do your research. Check out reviews as well as peer and student user experiences from other institutions or commentary online.
If enlisting human proctors, how are they trained and monitored?
If your vendor is firm on their belief of human involvement when it comes to online proctoring like PSI, you’ll want to ensure that those humans are adequately prepared to administer these tests. We take quality assurance measures with all of our proctors beginning with a four-week onboarding program. This training culminates with a certification exam that each proctor must pass prior to overseeing any testing.
Additionally, PSI employs an ongoing auditing process where each day, 15% of all exams administered through online proctoring are then re-proctored by auditing specialists. The data collected from these audits is used to confirm that a proctor is meeting quality standards and metrics. If they were to fall short for any reason, they are retrained until they perform to the standard that PSI prides itself upon.
What questions do students typically ask?
It would be a drastic oversight if during this review you did not anticipate the questions that students and other stakeholders will have during the transition to online proctoring as well as ongoing questions. Work with your vendor to better understand how the technology works. Build out an FAQ document that addresses everything from logistical day-of questions to long-term implications. To help get you started, here are some common questions that we hear from students and faculty:
- What does the product do?
- What data does it collect?
- Is the application or browser plugin safe and secure?
- How is that data used?
- Where is the data stored?
- Is my data shared with or sold to other vendors?
- What happens if the power goes out while I’m in the middle of a test?
- What happens if my bandwidth drops?
- What happens if an unexpected person enters the room while I’m testing?
- What scenarios are flagged by a proctor?
- What happens if someone potentially flags me?
- How is a flag reported to my school?
Is online proctoring even necessary for my assessments ?
It has become increasingly apparent that several institutions and individual instructors overindulged in the “all you can eat” pricing models that some vendors offer for automated proctoring. Since there’s no extra cost for proctoring every quiz, low-stakes assessment, and higher-stakes exams, they applied online proctoring to everything. Is that the right strategy? Should you force feed proctoring onto your students for all low-stakes (or even no-stakes) quizzes or should you reserve the use and reliance of online proctoring for the few exams that are most valuable? Are there alternative ways to assess learning outcomes? Can you determine if the students are learning and understanding the material even when using quizzes and formative assessments without the added stress some students may feel about online proctoring?
While this is certainly not an all-inclusive list of questions, it’s a starting point that you can use to ensure that your technology aligns with your needs and doesn’t prompt any major concerns from the student body. Communication is critically important and you’ll want to work to answer questions before they even come up. And don’t forget to define what the misconduct policies are for your institution. You don’t want to heighten any test anxiety or give a “big brother” feel, but you’ll want to be sure that you have a consistent disciplinary process to reference if and when any potential misconduct occurs.
* * * * *
The purpose of any exam security measure is to maintain fairness, level the playing field, and protect the integrity of an academic qualification. There are multiple approaches to achieving these goals, so be sure to do your homework. Ask the tough questions upfront and be proactive about communicating to your students. Online proctoring can be an incredibly useful tool when implemented properly and for the right reasons. And if you take the advice of PSI, you’ll keep the human component involved, too.