Table of Contents
Background
The Associated Board of the Royal Schools of Music (ABRSM) is a global music education charity. The ABRSM seeks to inspire learners to experience the benefits of musical progression through the pathways and certifications they offer. This includes a suite of Music Theory exams that measure and validate a learner’s music literacy skills.
A popular exam is Grade 5 Music Theory, which recognizes and rewards an understanding of the language of music while allowing learners to progress on their musical journey to Practical and Performance Grades 6, 7 and 8.
Challenge
ABRSM Music Theory exams are delivered globally in multiple languages. Prior to 2020, exam delivery was paper based, delivered at designated venues, during limited testing windows a few times a year. The infrequency of test windows, particularly internationally, limited when learners could advance to higher-level grades. Some were required to travel long distances to a venue, this increased learner anxiety, inconvenience and expense. For many learners this was their first experience of a formal exam setting.
The administrative load on ABRSM was high with:
- Hard copy exams delivered to venues and stored securely until exam day
- Completed exams returned securely to ABRSM offices
- Exams fielded to a team of Music Theory markers
- Markers trained, assessed, and moderated every year
- ABRSM on hand to provide venue support irrespective of time difference
While ABRSM exams are open to all ages, most learners are 8-14 years old. Exams must be designed carefully to ensure an appropriate reading level, and lack of exam etiquette and technique must be taken into consideration. Multiple stakeholders are involved, as a teacher, parent or guardian is required to book and pay for an exam.
ABRSM’s move to computer-based testing with online proctoring was in planning from 2014, with a planned launch date of June 2020 and full rollout in 2022. When the pandemic hit in early 2020, it was clear the move needed to be expedited.
Solution
Starting with Grade 5, ABRSM wanted to offer their learners, stability during a time when every other aspect of their education had been turned upside down. ABRSM partnered with PSI because they operated globally, could offer the test platform and proctoring solution and were keen to support their vision to quickly move Music Theory online. PSI’s solution addressed ABRSM’s needs by helping them move from:
- Paper based to computer-based testing. With a change of exam constructs and adapted item types.
- In-person to Record & Review online proctoring. With related challenges of access to appropriate devices and stable internet connection, and privacy/safeguarding.
- Window to on-demand testing.
The pilot for the Grade 5 Theory exam was launched on 26th August 2020 as an event-based format to start the online assessment transition. Learners were given a set window during which they accessed the system and sat the exam, with retake opportunities provided on a given day at a given time. Online practice materials were created to support learners’ move to this computer-based format.
The remaining grades launched in November 2020. Event-based testing transitioned to continuous, or ‘on-demand’ testing a year later as the scale and frequency of the event-based format was challenging for ABRSM to support globally. PSI’s psychometricians advised ABRSM to ensure matched demand and maintained the integrity of the on demand format. The full on-demand offering launched in August 2021 in three languages, increasing to six from 1st September, 2021.
"The increase in my child’s confidence levels was evident from the start of the very first screen."
Marketing
Changes were underway for Music Theory before the pandemic hit and pressed fast forward on the digital roll-out. The marketing team understood that customers needed to be brought on a change journey and that stakeholders needed to feel engaged with why the changes were taking place. Changes were being made to the syllabus and the exam questions, as well as to the format of the exam, moving from paper to online. It was important to build a positive perception of change; this wasn’t just a response to COVID, above all the ABRSM were improving their Music Theory assessments and they would remain a quality qualification from a leading music awarding organization.
The initial campaign focused on why theory is so important in the first place and the team created a film that demonstrated that very effectively. These fundamental messages were followed up by highlighting the value of the changes being made and the benefits to learners: removing ambiguity in questions, using more universally understood language, modernising, ensuring accessibility whilst maintaining standards.
As the campaign evolved marketing focused on the ‘anytime, anywhere’ nature of Music Theory going online and on-demand with an additional focus on the ease of taking these new online exams. This remains the core focus of marketing messaging at present, building on the foundations laid by the ‘why Theory’ campaign. The move to an ‘anytime, anywhere’ availability approach also enabled ABRSM to address the digital poverty challenge presented by event-based testing.
Communicating around safeguarding and young learners
The additional role of responsible adult was implemented to alleviate the anxieties of some parents about their child being recorded and observed by a third party. The following were provided in simple accessible language:
- Separate pre-exam checklists, including technical checks, for the different roles involved – the exam booker, responsible adult and student learner.
- Video support materials to guide users through processes, such as the room scan.
- Frequently Asked Questions with ongoing updates in response to queries.
- A rigorous proctoring process that explains the purpose of the recording, what proctors look for, how long recordings remain valid, and what happens afterwards.
- Details of the customer service teams and how teams interact across ABRSM and PSI.
Adapting to a younger audience
PSI worked to adapt UK Child Safeguarding measures into the ABRSM-specific rules and regulations for the PSI proctor team. For example, children are unlikely to sit still for the duration of an exam and are more likely to look around. These behaviors would be flagged by a proctor under standard testing conditions, and the adapted guidelines for proctors reflected this.
Due to the demographic of the learners heightened social media use and awareness was anticipated and a social media playbook was created in advance that included proactive outbound social media messaging, plus a plan for inbound social media responsiveness and more traditional media questions.
Neurodiverse Learners
Many learners with special education needs have successfully completed the online and on-demand Music Theory exams. ABRSM worked hard to ensure they responded to the needs of these learners. For example, by understanding that a self-chosen start time relieved significant levels of anxiety, and the provision of a personalized clock would help limit the stress and time-pressures learners often feel in the exam room.
- The provision of optional colour backgrounds that can be changed at any point during the exam have been extremely well received. Learners do not need to use their own colour tinted cellophane pages placed over the screen, or specially tinted glasses.
- The layout of one item per page is easy to navigate and the use of simple language, especially the handling of UK/US terminology, has proven to settle anxious learners quickly. The structure of the exam meant that neurodiverse learners were able to give their best earlier in the test. One parent commented that, “the increase in my child’s confidence levels was evident from the start of the very first screen”.
- Being able to take an exam at a time that is best for the learner, in their own environment and being allowed to hum or sing and talk through any reasoning out loud is the opposite of being required to sit in silence in the exam room.
"The fact that you considered the needs for learners like me with ADHD and dyslexia in an online exam has given me the confidence that I can do exams and that I can succeed."
Results
Over 13,700 event-based exams were delivered in 2020.
Just under 68,000 exams were delivered in 2021– over 11,000 in August alone.
Future Plans
The ABRSM had been planning a move to online testing long before the onset of the COVID-19 pandemic, realising the efficiencies and opportunities that an online testing model would create. The previous paper-based testing model was no longer fit for purpose and expensive and inefficient to deliver. The acceleration of this initiative to move to online test delivery, specifically the use of record & review online proctoring, has allowed the ABRSM to innovate their music theory assessment ahead of time. Learners can now flexibly choose when and where they wish to take their tests. Future plans include continual and collaborative service improvement for online theory testing, with ABRSM and PSI working together towards short, medium and long-term milestones. The higher-level strategy is to leverage this move to extend into more markets and to further increase the digitization of core ABRSM services for the future.
"This is an ongoing partnership with PSI which will evolve and improve as we each learn more about our respective markets and skill sets. We are looking to move more of our exams to an anytime, anywhere model. This allows us to continue to put the learner, irrespective of their age, at the center of our plans."
