Search
Success Stories / SOA Innovates to Test Real-World Settings

SOA Enhances Item Development with Innovative New Solutions

The Society of Actuaries (SOA) needed to measure predictive analytics competency, so it partnered with Prometric to develop innovative items that measure knowledge in real-world settings.

Innovative Item Development Revolutionizing predictive analytics and actuary testing

Background

The Society of Actuaries (SOA) is the world’s largest actuarial professional organization with more than 30,000 members. Through education and research, SOA advances actuaries as leaders in measuring and managing risk to improve financial outcomes for individuals, organizations, and the public. In 2017, SOA sought to enhance their education efforts during all stages of their credential lifecycle (pre-qualification through professional development) by adding more content on predictive analytics. While they were able to quickly move forward on the professional development side, the addition of predictive analytics to its pre-qualification education was part of a larger restructuring project for its associate (ASA) curriculum.

Situation

Recognizing an increasing need to assess candidates in a manner that reflects how they perform their day-to-day functions and knowing the topic didn’t lend itself to traditional exam methods, SOA needed a way to incorporate software tools that are used by actuaries when performing predictive analytics, including Word, Excel, and R/RStudio. Further, the exam needed to be proctored and completed in a secure area to best measure candidates’ knowledge of the material and approaches to predictive modeling.

However, developing this innovative item type would require overcoming several obstacles, including licensing issues for Word and Excel products, testing center availability to avoid burdening candidates, and proper security protocols to guarantee the exam’s efficacy.

Strategy

Knowing they needed a testing development partner with a software licensing solution, an established global testing center network, proven security procedures, and the willingness to take the next step in exam development, SOA put out an RFP specifically for this initiative. Leveraging our 10+ year relationship and our ability to fulfill all of SOA’s requirements outlined in the competitive bid process, Prometric was chosen by SOA to develop an innovative item that would best test and capture an actuary’s ability to perform predictive analytics.

Together, Prometric and SOA began collaborating around the development of the predictive analytics item type, with Prometric developing a solution that integrated all three of the desired tools into the assessment. The innovative item, administered twice a year, requires:

  • Candidates to complete 10-12 tasks across 5+ hours via constructed responses.
  • Test takers to demonstrate their ability to work through each task by providing supporting files and documents via Word and R/RStudio, with Excel available if needed.
  • Human grading conducted by SOA.

Prometric also worked closely with SOA to develop an online demo to help candidates understand how to navigate the exam ahead of time.

From there, Prometric and SOA moved forward with a secure install of the software throughout the global test center network. A pilot was performed with subject matter experts (SMEs) at several locations around the world, with SOA and Prometric staff on hand to ensure a successful launch.

"The development and launch of this item type is a huge step forward for the actuarial candidates of the Society of Actuaries. It allows candidates to demonstrate proficiency in critical topic areas by using tools that are common in the workplace. The item type development required innovative efforts from a joint SOA-Prometric development team. SOA is enthusiastically looking forward to the adaptation of this concept to other components of our credentialing pathway."

Managing Director of Education Society of Actuaries

Outcome

With Prometric’s technology and willingness to combine various aspects of testing and technology support and SOA’s flexibility in reworking the item type and quickly resolving issues that arose, after a year of development and the SME pilot launch, the first real test was administered to candidates in December 2018.

The new item type has allowed for higher-level data analysis, while also serving as a differentiator to the competition. Not only is SOA the first in the actuarial industry to provide this type of real-world testing, they also have exclusive rights to the test’s bespoke design.

In 2019, SOA and Prometric decided to renew the ongoing partnership for an additional seven years; during which time we will continue working together to build on the success of the innovative item type, and find ways to differentiate and strengthen SOA and their exams.

Download Success Story

Advance your assessment program with Prometric solutions

Partner with Us