10 Tips for Writing Multiple Choice Exam Questions

Tests don’t create themselves — faculty spend hours writing exams each year. But if you still haven’t taken the plunge into writing multiple choice exam questions, starting from scratch can be a scary concept. I come to the office with heart palpitations just thinking about it (BP of 150/90, HR 110bpm).

Well-written test items do a number of things: they address important content, are well structured, have a stem with a clear question, and contain a set of good answer options. Putting together a question that has these elements isn’t that difficult. Stick to the following 10 tips and you’ll be well on your way to writing effective exam questions.

1. Remember the elements

A one-best-answer test question always has three major components:

* Stem

* Lead-in question

* List of answer options

–
Remember these three elements, and you’ll be golden.

2. The stem has all the answers

The stem is the first part of the item, includes relevant information needed to answer the question, and presents the item as a problem to be solved or a question asked of the respondent. In medical education, the stem typically consists of a clinical case presentation. For example:

–
Regardless of what type of question you’re writing — whether it’s a medical knowledge question or a diagnosis question — the stem should always be written such that the examinee can correctly respond to the question posed without seeing the answer options. Now to that same example, lets add a lead-in question:

 –
In this particular case, with the information provided in the stem, a test-taker should be able to deduce that looking at the serum uric acid levels will definitively establish the suspected diagnosis (gout).

Since the stem is the part of the test item that stimulates higher order thinking, a good question stem contains all of the information that a student would need. The test-takers challenge is to process the information and apply what they know to arrive at the correct answer. This is difficult, if not impossible, if any of the necessary information is missing.

3. Be vogue, not vague

One of the worst things you can do when writing your test question is to be ambiguous. This vagueness can manifest in some of the following ways: using “may” and “may happen,” using “always” or “never,” posing negative questions, or writing a question in the format of “all of the options are correct, EXCEPT.” Instead, be specific and direct to create test questions that will give you a clear picture of your student’s knowledge and thinking skills.

4. When in doubt, go for a vignette

Clinical vignettes are a fabulous way to structure a question. Vignettes use realistic scenarios that touch on potential past or future experiences in practice. For example:

–
Just as with all stems, vignettes require enough information to be answered, but you also want to avoid adding extraneous information. Things to include in your vignette are: age, gender, care setting, duration, symptoms, physical exam findings, and diagnostic study results if indicated. Do not include information, such as race or ethnicity, unless it is relevant to the scenario and necessary to answer the question correctly.

5. Lead-in questions

Each stem should end with a lead-in question that the test-taker has to answer. Rather than saying:

–
Try phrasing it as a question that asks:

–
The lead-in question of a stem is important because the test item needs to pose a clear question. Without this, the test-taker is unable to put forth an answer without looking at the answer options (see #2), which diminishes the evaluative nature of your question.

6. One, and only one, answer

Once you have your stem and lead-in question written, it’s time to focus on the list of answer options. First things first: make sure there is only one correct option! Choose your correct answer up front and make sure that, as you continue to build your list of options, they aren’t potential correct answers themselves. A good rule of thumb is to have one correct option and three or four incorrect options. Don’t throw in clearly wrong answers just to have more options. It’s better to have fewer good distractors then several poor distractors.

7. Answer options should be strategic

All of your options should come from a single category. It doesn’t make much sense to have one option be a diagnosis and the rest be clinical interventions, or one option a non-invasive therapy and the rest invasive procedures. For example, which one of these four options is an obvious outlier?

–
Depending on the stem, a savvy test-taker is going to rule out option A as implausible. To create high-quality test questions, it is important to create plausible distractors, or incorrect answer options, that force students to apply what they know in order to identify the correct option. Distractors that are obvious make the question a poor tool to assess a learner’s knowledge. For that reason, consider creating answer options that are similar in length and complexity, and that are grammatically parallel. Alphabetizing the options can also be a good tool to show that the order of answer options does not indicate the correct answer. Finally, use mutually exclusive options.

8. Avoid common mistakes

To write questions based on best practices, avoid making the correct answer too long, using implausible answer options, or using an “all of the above” option. While they may be commonly found in exam questions, they typically degrade the quality of a test item and hinder your ability to truly assess your learner.

9. Check your work

After you’ve written your stems and options, physically cover up all of your answer options and ask yourself, “Is the question still answerable?”

Also, take this time to ensure that your options are ordered on a single dimension and that the distractors are homogenous and plausible. For example, in the first set of options below, option D is unlike the other options — it’s a specific drug instead of a drug class, and it’s an over-the-counter drug instead of a prescription — all of which make it implausible. Note in the second set how much more homogenous the four options are as a whole:

10. Peer review

Finally, send your questions to a colleague. Peer review is always the best way to create high quality exam questions. The more people you have review your work and provide feedback, the better the item becomes and the more effective it is as an assessment tool.

Follow these 10 simple tips, and you’re sure to develop a multiple choice test question worthy of a national exam.

 

The above information has been adapted from PAEA’s New Item Writer Workshop, which is attended by all new PAEA content experts on the Exam Development Boards, as well as from the National Board of Medical Examiner’s manual “Constructing Written Test Questions for the Basic and Clinical Sciences.”

I’d like to acknowledge Assessment Council Chair Kim Cavanagh, DHSc, PA-C, and Assistant Chief of Academic Affairs Olivia Ziegler, MS, PA, for their contributions.