Writing Tip #1: Writing Higher-Order Questions

Now that we are 12 weeks away from our item submission deadline, I wanted to provide a reminder or two. During the form review process, across all exams, we noted an increasing number of new pre-test items — items written this past year — that were basic recall items. Some of these were one-sentence questions. We felt this was a good time to remind everyone about the importance of writing more challenging, higher-order questions that include case vignettes.

I know that writing vignettes can be hard, and it certainly takes time, but these types of items result in higher quality exams that better assess students’ capacity for problem-solving and critical thinking. Writing more difficult items does not mean writing questions about obscure topics or adding unnecessary information that tricks examinees; it means writing questions that require the student to interpret, analyze, and/or synthesize information to arrive at an answer. This is what they will do in clinical practice, so we would like to assess this ability in the best way we can on our objective multiple-choice exams.

Our form and style guidelines outline the appropriate information to include when developing a clinical vignette. I have included some information connecting Bloom’s Taxonomy to our types of questions and some examples below which show the difference between questions of a lower order (basic recall) and higher-order questions. I used these examples when doing some training with the new End of Curriculum Exam Development Board and I thought everyone might find this helpful. Additionally, please refer to the PAEA Item Writing module which has more information and examples. This should be a refresher from your new item writer training when you joined the EDB.

BASIC RECALL

Remember:

Which of the following is the first-line treatment of choice for streptococcal pharyngitis in a patient with no known medication allergies?

1. Penicillin VK
2. Amoxicillin, clavulanic acid (Augmentin)
3. Doxycycline (Doryx)
4. Cephalexin (Keflex)

Remember: Recall what bacterial pharyngitis is

This is a one-step thought process where the student needs to only recall information they may have memorized.

HIGHER ORDER

Understand/Apply

An 8-year-old boy presents to the office with acute onset of throat pain for 24 hours. The patient admits to difficulty swallowing, decreased appetite, and a mild nonproductive cough. His mom states he has had a fever at home. Physical examination reveals the nares to be patent without edema. The external auditory canal is patent without erythema; the tympanic membrane is dull with no bulging or erythema. The pharynx is erythematous with tonsillar enlargement and exudates. There is bilateral anterior cervical lymphadenopathy. Lungs are clear to auscultation bilaterally. Which of the following is the most likely diagnosis?

1. Bacterial pharyngitis
2. Influenza
3. Acute rhinosinusitis
4. Retropharyngeal abscess

Understand/Apply: Understand the patient scenario to connect to preexisting knowledge — make the diagnosis

This is a two-step thought process where the student must use information provided to arrive at a diagnosis. This requires connection to previously learned material as the student must interpret the history and physical exam findings provided.

Analyze/Evaluate

An 8-year-old boy presents to the office with acute onset of throat pain for 24 hours. The patient admits to difficulty swallowing, decreased appetite, and a mild nonproductive cough. His mom states he has had a fever at home. Physical examination reveals the nares to be patent without edema. The external auditory canal is patent without erythema; the tympanic membrane is dull with no bulging or erythema. The pharynx is erythematous with tonsillar enlargement and exudates. There is bilateral anterior cervical lymphadenopathy. Lungs are clear to auscultation bilaterally. Which of the following is the most likely potential complication if this condition goes untreated?

1. Glomerulonephritis
2. Splenic rupture
3. Periorbital cellulitis
4. Mastoiditis

Analyze/Evaluate: Break down information to relate to possible problems/issues; connect ideas in the scenario — most likely complications

This is a multi-step thought process where the student must first arrive at the diagnosis (which is not provided in the vignette) and then determine the most likely complication.

SUMMARY

Higher-order questions ensure we have an exam that is assessing more than what students can look up in a book. By focusing on items with complex scenarios, we are ensuring that the exam assesses clinical reasoning and the ability to apply knowledge as experienced during clinical rotations.