Assessment

10 Tips for Writing Multiple Choice Exam Questions

By Zachary BrittOctober 20, 2015

multiple choice large scantron and pencil

Credit: Shutterstock

Ready to take the plunge and write your own test questions? Some advice to get you started.

Tests don’t create themselves — faculty spend hours writing exams each year. But if you still haven’t taken the plunge into writing multiple choice exam questions, starting from scratch can be a scary concept. I come to the office with heart palpitations just thinking about it (BP of 150/90, HR 110bpm).

Well-written test items do a number of things: they address important content, are well structured, have a stem with a clear question, and contain a set of good answer options. Putting together a question that has these elements isn’t that difficult. Stick to the following 10 tips and you’ll be well on your way to writing effective exam questions.

1. Remember the elements

A one-best-answer test question always has three major components:

* Stem

A five-year-old boy is brought to the office by his father who reports a two-day history of low grade fever and coryza. The child awoke this morning with bright red cheeks. Physical examination reveals edematous, confluent plaques over the malar region of the face, and reticular rash over the child’s extensor surfaces.


* Lead-in question

Which of the following is the most likely diagnosis?


* List of answer options

A) Rubeola
B) Parvovirus B19
C) Respiratory syncytial virus
D) Epstein-Barr virus


Remember these three elements, and you’ll be golden.

2. The stem has all the answers

The stem is the first part of the item, includes relevant information needed to answer the question, and presents the item as a problem to be solved or a question asked of the respondent. In medical education, the stem typically consists of a clinical case presentation. For example:

A 49-year-old man is examined for a painful, swollen, and erythematous right great toe. Symptoms began 24 hours ago and have progressively worsened.


Regardless of what type of question you’re writing — whether it’s a medical knowledge question or a diagnosis question — the stem should always be written such that the examinee can correctly respond to the question posed without seeing the answer options. Now to that same example, lets add a lead-in question:

A 49-year-old man is examined for a painful, swollen, and erythematous right great toe. Symptoms began 24 hours ago and have progressively worsened. Which of the following will definitively establish the suspected diagnosis?

 –
In this particular case, with the information provided in the stem, a test-taker should be able to deduce that looking at the serum uric acid levels will definitively establish the suspected diagnosis (gout).

Since the stem is the part of the test item that stimulates higher order thinking, a good question stem contains all of the information that a student would need. The test-takers challenge is to process the information and apply what they know to arrive at the correct answer. This is difficult, if not impossible, if any of the necessary information is missing.

3. Be vogue, not vague

One of the worst things you can do when writing your test question is to be ambiguous. This vagueness can manifest in some of the following ways: using “may” and “may happen,” using “always” or “never,” posing negative questions, or writing a question in the format of “all of the options are correct, EXCEPT.” Instead, be specific and direct to create test questions that will give you a clear picture of your student’s knowledge and thinking skills.

4. When in doubt, go for a vignette

Clinical vignettes are a fabulous way to structure a question. Vignettes use realistic scenarios that touch on potential past or future experiences in practice. For example:

A 58-year-old man who is a known alcoholic comes to the office with acute onset of epigastric pain. The pain is worse when lying down and improves when sitting up and leaning forward. He has associated nausea and sweating but denies any vomiting. On examination, he is tachycardic and febrile. The abdominal exam reveals epigastric tenderness. Lab studies show leukocytosis. A plain film of the abdomen demonstrates a colon cutoff sign. Which of the following is the most likely diagnosis?


Just as with all stems, vignettes require enough information to be answered, but you also want to avoid adding extraneous information. Things to include in your vignette are: age, gender, care setting, duration, symptoms, physical exam findings, and diagnostic study results if indicated. Do not include information, such as race or ethnicity, unless it is relevant to the scenario and necessary to answer the question correctly.

5. Lead-in questions

Each stem should end with a lead-in question that the test-taker has to answer. Rather than saying:

The most appropriate diagnostic test is…


Try phrasing it as a question that asks:

Which of the following is the most appropriate diagnostic test?


The lead-in question of a stem is important because the test item needs to pose a clear question. Without this, the test-taker is unable to put forth an answer without looking at the answer options (see #2), which diminishes the evaluative nature of your question.

6. One, and only one, answer

Once you have your stem and lead-in question written, it’s time to focus on the list of answer options. First things first: make sure there is only one correct option! Choose your correct answer up front and make sure that, as you continue to build your list of options, they aren’t potential correct answers themselves. A good rule of thumb is to have one correct option and three or four incorrect options. Don’t throw in clearly wrong answers just to have more options. It’s better to have fewer good distractors then several poor distractors.

7. Answer options should be strategic

All of your options should come from a single category. It doesn’t make much sense to have one option be a diagnosis and the rest be clinical interventions, or one option a non-invasive therapy and the rest invasive procedures. For example, which one of these four options is an obvious outlier?

A) Deep brain stimulation
B) Ibuprofen (Motrin)
C) Prochlorperazine (Compazine)
D) Sumatriptan (Imitrex)


Depending on the stem, a savvy test-taker is going to rule out option A as implausible. To create high-quality test questions, it is important to create plausible distractors, or incorrect answer options, that force students to apply what they know in order to identify the correct option. Distractors that are obvious make the question a poor tool to assess a learner’s knowledge. For that reason, consider creating answer options that are similar in length and complexity, and that are grammatically parallel. Alphabetizing the options can also be a good tool to show that the order of answer options does not indicate the correct answer. Finally, use mutually exclusive options.

8. Avoid common mistakes

To write questions based on best practices, avoid making the correct answer too long, using implausible answer options, or using an “all of the above” option. While they may be commonly found in exam questions, they typically degrade the quality of a test item and hinder your ability to truly assess your learner.

9. Check your work

After you’ve written your stems and options, physically cover up all of your answer options and ask yourself, “Is the question still answerable?”

Also, take this time to ensure that your options are ordered on a single dimension and that the distractors are homogenous and plausible. For example, in the first set of options below, option D is unlike the other options — it’s a specific drug instead of a drug class, and it’s an over-the-counter drug instead of a prescription — all of which make it implausible. Note in the second set how much more homogenous the four options are as a whole:

A) Monamine oxidase inhibitor
B) Selective norepinephrine reuptake inhibitor
C) Selective serotonin reuptake inhibitor
D) St. John’s wort
– – – –
A) Monamine oxidase inhibitor
B) Selective norepinephrine reuptake inhibitor
C) Selective serotonin reuptake inhibitor
D) Tricyclic antidepressants

10. Peer review

Finally, send your questions to a colleague. Peer review is always the best way to create high quality exam questions. The more people you have review your work and provide feedback, the better the item becomes and the more effective it is as an assessment tool.

Follow these 10 simple tips, and you’re sure to develop a multiple choice test question worthy of a national exam.

 

The above information has been adapted from PAEA’s New Item Writer Workshop, which is attended by all new PAEA content experts on the Exam Development Boards, as well as from the National Board of Medical Examiner’s manual “Constructing Written Test Questions for the Basic and Clinical Sciences.”

I’d like to acknowledge Assessment Council Chair Kim Cavanagh, DHSc, PA-C, and Assistant Chief of Academic Affairs Olivia Ziegler, MS, PA, for their contributions.

Zachary Britt
Zachary Britt

Zach is manager of educational development for the Physician Assistant Education Association. He guides and oversees the work of the Academic Affairs team, helping to develop and manage the Association’s curricular and assessment resources.