End of Rotation Faculty Guide

The PAEA End of Rotation exams are a set of objective, standardized evaluations intended to serve as one measure of the medical knowledge students gain during specific supervised clinical practice experiences (SCPE). PAEA End of Rotation exams are intended to be delivered on or near the last day of the student’s SCPE and used in conjunction with other evaluation modalities when assigning a student grade.

The End of Rotation exams may be scheduled with In-Person Proctored, Remote Proctored, or Unproctored delivery. The use of PAEA’s Secure Browser is required for In-Person Proctored and Unproctored exam delivery.

For each End of Rotation exam, PAEA publishes multiple equivalent exam forms. Multiple forms per exam helps to ensure exam security and to prevent over-exposure of exam content. These forms are equated in terms of content, difficulty, and reliability.

Students are randomly assigned one of these forms on exam registration. Except for Family Medicine, all End of Rotation exams have two equivalent forms. Family medicine has three equivalent forms.

The End of Rotation exams are multiple-choice exams which use a “one best answer” format. Items are comprised of a stem and answer options. The stem typically consists of a clinical scenario or vignette and a lead-in question. The answer options consist of four or five answer choices: the correct or best answer (known as the key) and three or four incorrect or inferior alternatives (known as distractors). The items are of a varying degree of complexity, with the vignette format allowing for the measurement of higher order thinking skills and providing a better approximation of real-life practice.

The End of Rotation exams are two hours long and scheduled for one day. Students are allotted 60 minutes per section, with one optional 10-minute break between sections, as well as an optional 10-minute tutorial before the start of the exam.

Accommodations are granted by the institution under the Americans with Disabilities Act (ADA). PAEA allows time extensions of time-and-a-half or double time for students requiring time-based accommodations. Exams using time-and-a-half (1.5x) accommodations receive 90 minutes per section. Exams using double time (2.0x) accommodations receive 120 minutes per section. Both of these accommodations receive the standard 10-minute break between sections.

Students may take all available forms of an End of Rotation exam once without restriction. This means two attempts for all exams except for Family Medicine. Family Medicine allows three attempts. After attempting all allotted forms the student must wait 21 days after the last administration to retake the available exam forms in random order.

The content of End of Rotation exams mirrors that of the medical knowledge expected to be gained in a particular SCPE.

Each PAEA End of Rotation exam blueprint has a two-dimensional structure that encapsulates both content and task areas. The Surgery End of Rotation exam further includes perioperative settings. Each exam aligns to both the blueprint and topic list specifications. Note that items included on the exam are considered a sample and may not reflect all content topics identified in the topic list

To support curriculum mapping and program evaluation, this guide includes a table identifying organ systems represented within each End of Rotation exam. The purpose of this table is to help programs visualize where organ systems are assessed across the End of Rotation exam portfolio.

Core Tasks and Objectives are assessed by all of the PAEA examinations (PACKRAT^TM, End of Rotation^TM, and End of Curriculum). The core tasks and objectives on the End of Rotation exams are:

• History and Physical
• Diagnostic Studies
• Diagnosis
• Clinical Intervention
• Clinical Therapeutics
• Health Maintenance
• Scientific Concepts
• Professional Practice

Items written for these exams expressly consider the broad diversity of patients that PAs will be called upon to treat, focusing on patient care setting (outpatient, inpatient, emergency department, perioperative, urgent care, and long-term care) and lifespan (Infant, Child, Adolescent, Adult, and Geriatric patients). Additional consideration was given to diversity including, but not limited to, race, ethnicity, nationality, gender identity, sexual orientation, socioeconomic status, cultural identity, religion, and functional diversity.

Scale scores are scores that have been mathematically transformed from one set of numbers (i.e., the raw score) to another set of numbers (i.e., the scale score), in order to make them more easily comparable between different forms of the same exam. Involved in scaling is a process known as equating, which ensures that no student is disadvantaged or advantaged by taking a more difficult or less difficult exam form. The primary benefit to scale scores is that they allow all scores on all versions and forms of the End of Rotation exams to be comparable across years as they all use the same scale metric. Additionally, this process corrects for variations in test form difficulty, ensuring that equivalent levels of performance result in the same scaled score regardless of the form administered.

The scale for the End of Rotation exams is 300-500.

After completion of an exam, programs and students are provided with feedback in the form of a score report. Every student receives a scale score and a table of their score compared to the national average. Feedback is provided by both content and task areas. For comparative purposes, this information is also paired with national comparison data in the program’s composite and cohort performance reports.

In addition to reporting a student’s total score, subscores are also reported across both content areas (e.g., Cardiovascular, Dermatologic, Endocrine) and task areas (e.g., History and Physical, Diagnosis, Clinical Intervention). For each content and task area, the following are presented: blueprint percentages, the student’s scale score for that area, and the national average for that area.

In addition to subscore feedback, keyword feedback is included on student score reports. Students are provided with a list of keywords of the exam items they answered incorrectly. Keywords are given in the format of “Content Area: Task, Keyword” (e.g., Pulmonology: Clinical Intervention, Asthma). This is the most granular and direct feedback provided to students and faculty and can help faculty/students identify areas where they may want to focus additional study efforts.

The Student Report shows how the student performed on the exam compared to national averages. Subscores help the student identify strengths and weaknesses in content areas and task areas relative to national average performance. The report can be downloaded as a PDF file for the student and faculty to review together.

The Cohort Performance Report shows program faculty how a student cohort compares to national averages in both overall and subscores. The comparative all-program data is based on the exam performance of students nationwide (i.e., the reference population) over a given time period and is updated regularly. The report can be printed or saved as a PDF.

The Composite Report organizes student scores by cohort to facilitate gradebook entry or reviewing multiple students’ performance simultaneously. The report shows each student’s overall score and subscores, the date they took the exam, and whether the results have been released to the student. The subscores can be organized by Content Area and Task Area, and the report can be exported as a comma-delimited file (e.g., .csv).