Crafting a clinical scenario (covered in our first writing tip) is the first step to writing a strong exam item; the second is equally important and sometimes difficult — writing the lead-in question. Rather than spend an entire article on what amounts to a single sentence, this piece focuses on one of the most challenging aspects: making sure the question matches the correct task area for your exam blueprint.
PAEA exams are developed based upon specific blueprints that match content areas and task areas. We view these as the learning objectives of our exams — task-based competencies that students should be able to demonstrate at the relevant stage of their education, whether it is following a clinical experience for the End of Rotation™ exams or at the midpoint and/or the end of their education for PACKRAT™. The same holds for any locally developed exams and course learning objectives. The point of creating blueprints for exams and tying them to learning objectives is to make sure that exams assess what you mean for them to assess.
Some content and task area combinations are relatively straightforward (i.e. Cardiology content area, Diagnosis task area). However, multidimensional blueprints can introduce complications, particularly when addressing tasks in certain content or exam areas. For example, health maintenance questions in emergency medicine, surgery, or psychiatry can be a challenge, as they are more procedural specialty areas. Here are some ways to approach writing these items.
Go back to the task area descriptions
One of the best ways to craft strong items is to start with the task area descriptions. Understanding the task areas and the differences between them can make writing items easier. These descriptions serve as objectives, which is a helpful reminder of the need to align exam content with curriculum. For instance, health maintenance tasks include: determining appropriate counseling (including patient education) related to preventable disease, communicable disease, immunizations, and healthy lifestyles. Health maintenance items also assess the ability to recognize the impact of stress on health, and the psychological manifestations of illness or injury. This area also includes routine screening, such as those for the recognition of abuse and neglect.
Some of the hardest to separate are clinical intervention and clinical therapeutics. These areas both involve management of diagnoses; however, clinical therapeutics questions are typically limited to pharmacotherapeutics, while clinical intervention items are all other types of management such as procedures or physical therapy. When creating distractors, one might include both pharmacologic and non-pharmacologic options. This is fine, provided there is more than one answer option from each category and the correct answer corresponds to the designated task area.
Review sample lead-ins
It is important that the lead-in matches the task area. The clinical intervention/clinical therapeutics differentiation is a good example of this. If an item is a clinical therapeutics question and the lead-in is “which of the following is the most appropriate medication,” then having non-pharmacologic answer options would cause the test taker to eliminate those options since the options don’t answer the question. Changing the lead-in to “which of the following is the most appropriate management” allows both pharmacologic and non-pharmacologic options to be plausible. Conversely, committing to a more specific lead-in can help remind you of the type of question you are expected to write. Reviewing the lead-in options can be a good trigger for developing an item for a specific task area.
Think about the disease process from multiple perspectives
Looking at a disease from a different viewpoint can make writing for certain task areas a little easier. For example, when writing a health maintenance question for a surgery exam about cholecystitis, the question does not only have to focus on what would have prevented the acute cholecystitis. The question might focus on patient care following a cholecystectomy, or the risk factors for which a patient needs to be screened after surgery. Psychiatry is another difficult area for health maintenance questions. But if you think about the care and monitoring provided beyond the diagnosis, you’ll be on the right track.
In other words
In order to write an exam that assesses what you mean for it to assess, you should be careful that each question matches a task area in your learning objectives or blueprint. Once you know what you’re going to write about, make sure the lead-in matches the answer options. Being specific about what you expect out of your questions will lead to more straightforward exam questions that follow logically and have only one right answer.
Emily Yunker, PAEA’s assessment manager, contributed to this article.