Exam Development Summit — Successful Meeting of the Minds
In this landmark year, 63 Exam Development Board (EDB) members gathered last month and reviewed a total of 1,472 items for upcoming exams.
The primary purpose of this annual meeting is to conduct the second peer review of exam questions before forms are constructed, reviewed, and published. Each EDB reviews all of that year’s questions, checking for appropriate structure and changes in medical standards in the months since they were written, making edits, and signing off on the final language. This is a major undertaking as each group meticulously reviews between 250 and 400 questions with the guidance of their chairs and in collaboration with editorial consultants.
The work from this meeting will lead to the publication of:
- Version 24 of PACKRAT®
- Version 7 of the End of Rotation™ exams
- Version 1 of the End of Curriculum™ exam (coming in January 2020)
We also conduct key validation at the Summit facilitated by our psychometric partners. Select volunteers review performance statistics for items administered in the current exams to determine whether they are performing as expected and can be kept, have minor issues and can be tweaked to meet expectations, or have serious interpretation problems and should be deleted.
This is also the primary venue for member development. New members of the EDBs receive their initial training, facilitated by Assessment Editor in Chief Kim Cavanagh. For the training this year, new members had a special visitor – PAEA President Jon Bowser.
As part of our ongoing development efforts for members, keynote speaker Sara Fletcher, PhD, PAEA’s interim CEO, addressed the challenges of aligning assessments to program objectives.
The Summit is also a key opportunity for EDB members — who work together in smaller groups throughout the year — to network as assessment community members. The EDBs represent 53 programs from every region of the country. Members come together not only to “geek out” about assessment, but also to socialize and share what’s going on at their programs.
Each year, we organize a social activity to provide a release for EDB members who work so hard over the four-day meeting. This year, we wanted to do something different by engaging in community service. We partnered with Operation Gratitude, a nonprofit organization that gives back to individuals who serve our country, to compile care kits.
The EDB members, partners, and staff compiled 800 kits. Many found this activity particularly meaningful given the history of the PA profession and its connection to returning service members.
You can learn more about the exam development process, including the additional steps taking place for the End of Curriculum exam over the coming year, on the process page of the assessment website.
We also captured a “day in the life” at the Exam Development Summit in PAEAonline Instagram stories. Check the highlights above the feed to see how our Friday went.