Search
Success Stories / CompTIA Harnesses Finetune AI

CompTIA harnesses Finetune AI to scale rapidly without sacrificing quality

CompTIA speeds up assessment development by creating high-quality test items with Finetune AI, scaling its global certification program to meet workforce demands.

CompTIA Shutterstock 2463474777

Background

CompTIA is the world’s leading provider of vendor-neutral IT training and certifications. Partnering with thousands of institutions, CompTIA equips learners with career-ready skills through best-in-class learning tools, respected credentials, and comprehensive career support.

To meet growing global demand—particularly for their A+ exam—CompTIA partnered with Prometric, leveraging Finetune AI for a custom AI solution. This partnership enabled a faster content creation process while maintaining quality, psychometric rigor, and expert oversight—advancing CompTIA’s mission to deliver efficient, career-ready credentials.

The Challenge

As global demand soared, CompTIA needed to accelerate content creation across 16 core programs. High-volume certifications like A+ posed a particular challenge. Each item had to meet rigorous psychometric standards for fairness, reliability, and legal defensibility.

  • Maintaining quality is too slow
  • General AI limits IP ownership
  • SMEs need to be more in control

The traditional development model couldn’t keep up. It was thorough but slow—too slow for CompTIA’s evolving needs. The team needed an approach that preserved SME involvement, editorial rigor, and full IP ownership, while enabling faster output.

CompTIA Shutterstock 2325628103

“Specialized models are much less prone to hallucinations because it’s been trained to weed that out, give us just the good stuff.”

Wendy Gavin

Senior Product Manager, Finetune AI

Strategic Partnership

CompTIA launched a multi-year collaboration with Finetune AI to harness artificial intelligence for item development. Together, they created a Finetune Generate® domain-specific AI model.

Unlike general-purpose tools like ChatGPT or Copilot, Finetune AI is the only custom, patented AI model in the US designed for and by assessment professionals, minimizing hallucinations and aligning with strict editorial standards. The A+ certification served as the pilot due to its structured content and high item volume.

The project brought together Finetune AI experts, CompTIA content development teams, and SMEs to train the model on proprietary blueprints, style guides, and objectives. The AI supported—but never replaced—SME authorship, empowering faster drafting and iteration.

Innovating with Finetune Generate

CompTIA embedded Finetune Generate® into a hybrid workflow designed for scale. The model was trained on CompTIA’s proprietary materials, ensuring outputs aligned with tone, structure, and learning objectives. SMEs often made few—if any—revisions to the AI-generated drafts.

CompTIA retained full ownership of AI-assisted content, maintaining control and security. The tool performed exceptionally well on Bloom’s Level 2 and 3 objectives—frequent in entry-level certifications like A+.

Key Benefits:

  • Custom Model Training
  • Efficient Drafting
  • Secure IP
  • Ideal for Lower Cognitive Levels

“We use Generate in a way that results in clear advantages in QC efficiency.”

- Danielle Castelli, Director, Exam Development, CompTIA

A psychometric review of A+ exam versions 1101 and 1102 validated the approach. SMEs produced 5.7 items per hour—up from 3.3—with most needing minimal edits. Psychometric benchmarks (difficulty, discrimination, reliability) stayed within expected ranges, and items successfully moved from pretest to scored status after field testing.

“Working with [Prometric], we’ve been able to augment our exam development program to make the process more efficient in a way that both our test developers and SMEs have benefited from.”

Danielle Castelli

Director, Exam Development, CompTIA

Outcome

Today, CompTIA continues to use Finetune Generate® for certifications with structured, lower-level cognitive demands—where speed and scalability are key. This AI-enhanced process lets SMEs concentrate on validation rather than first drafts.

  • Increased Output
  • Editorial Efficiency
  • Statistical Equivalence
  • Content Viability

Looking ahead, CompTIA is exploring additional use cases where focused, responsible AI can deliver similar value across its certification portfolio—always with an emphasis on quality and control.

Download Success Story