Standards Set: The PAEA End of Curriculum Exam Is Ready to Launch

The End of Curriculum™ exam is a 300-question, objective, standardized evaluation of a student’s medical knowledge. Set to launch on January 6, 2020, the End of Curriculum exam serves as one component of a student’s readiness for graduation. This evaluation, which utilizes criterion-based standards, is a new and unique part of PAEA’s suite of exams and is the result of countless hours of work by PA educators, psychometricians, and the PAEA Board of Directors and staff.  

Throughout its creation, refinement, and testing, numerous professionals lent their expertise to the process in order to develop an objective, standardized evaluation of students’ medical knowledge as one component of their readiness for graduation.  

End of Curriculum Development Process 

The development process included 63 PA educators who built, wrote, and reviewed the foundational materials and content of the exam. These dedicated volunteers represented 53 programs from every region of the country and were able to bring a variety of different perspectives to the process. PA students, in the last four months of their program, representing 38 programs from every region of the U.S., helped us collect baseline statistics. In total, 80 programs have participated in one or more ways to launch the first version.  

After years of hard work, we’re excited to see a new and valuable tool become available for PA programs. Programs can anticipate assessing students’ medical knowledge with an exam that is peer-reviewed, pilot-tested, statistically validated for accuracy and consistency, and standardized by a panel of PA education experts.  

Setting the Standards 

One of the final pieces of the puzzle was to standard-set the exam. The standard-setting study was conducted on November 1–4 in Chicago, convening a workgroup to produce recommendations for the PAEA Board of Directors.

The Board met in early December to discuss the workgroup’s recommendations, among other organization issues. PAEA Chief Assessment Officer Olivia Zeigler and NCCPA psychometrician J.B. Weir presented the results, asking the Board to:

• Affirm the validity of the process  
• Affirm that the process is defensible  
• Approve the recommended standards 

After an in-depth discussion, the Board approved the recommendations made by the workgroup and was satisfied by the process and its recommendations. The result will be a useful addition for programs to assess their students. 

The scores for the End of Curriculum exam are equated and reported on a scale, which provides a consistent way to measure performance on multiple exam forms administered at the same time or over different years. The scale score range for the End of Curriculum exam will be 1200 to 1800. The scores will be divided into three performance categories: Limited, Satisfactory, and Advanced Medical Knowledge. The two performance standards that will separate these three categories are 1400 and 1525.   

Standard-setting provides a valuable tool for faculty when considering a student’s readiness for graduation; however, programs must individually determine whether these categorical scores accurately represent pass scores within their own programs. PAEA recommends that programs monitor student and cohort performance on the End of Curriculum exam and adjust their local scoring process as needed. Additionally, PAEA recommends that programs have a thoughtful and meaningful reason as to why this tool is being used to evaluate their students. Programs must be able to document their decision-making process and rationale and ensure that:  

• Assessments, learning objectives, and instructional strategies are closely aligned 
• The content is appropriately mapped to the program’s stated goals, outcomes, and objectives 
• Additional evaluation tools are used to meet the other C3.04 (ARC-PA Accreditation Standards, 4th edition) or B4.03 (5th edition) requirements, as well as any specific individual program medical knowledge requirements, for students to qualify for a degree from the program 
•  Appropriate weighting is applied to each assessment as determined by the program

It is important for each program to focus on how they have made their decision or the rationale for adopting the End of Curriculum exam, their process for integrating this exam at the local level, and their process for interpreting the standard scores.  

To learn more about the End of Curriculum exam and how to bring it to your program, visit the PAEA Assessment website.