EDI in Action: PAEA’s Cultural Humility in Item Writing Initiative
This month, the EDI team met with PAEA’s Casey Johnson, PhD, senior director of assessment services, and Ashley Tucker, project manager of assessment services, to discuss their “Cultural Humility in Item Writing” initiative.
In 2018, PAEA developed an anti-bias training that was given every year to volunteer members who were writing test items. Johnson understood the importance of ensuring test items were inclusive and reflective of the patients that students will be called upon to treat. The way race, ethnicity, culture, and identity are used in the construction of test items may introduce unconscious bias along with other test-related bias such as item-selection bias or reinforcing hidden curricula that perpetuate bias in medical practice.
Johnson and Tucker decided in 2023 to take this initiative a step further and create two task forces of experts. The first group was charged with reviewing all existing materials while the second group focused on helping current item writers. Through a call to PAEA’s membership, they recruited 22 volunteers who have been hard at work ensuring that bias checks and cultural humility are at the forefront of the test item writing process. The team tasked with reviewing existing materials has reviewed more than 2,000 items thus far, while the standing group has two more years left in their initial terms.
Throughout this process, they have developed a list of language best practices as it relates to age, gender, race, socioeconomic class, sexuality, disability, and more. For example, inclusive language as it relates to gender include using parent or primary care giver opposed to mother or father and spouse or partner instead of husband/wife.
Mentioning race is only necessary if it is directly relevant to the clinical scenario and if necessary to answer the question correctly. Even when it is deemed relevant, Johnson and Tucker suggest test item writers check to ensure that race/ethnicity is not being used as a proxy for genetic traits, socioeconomic status, or education.
Vague terms such as “at risk” aren’t preferred. For example, instead of writing that “at-risk youth are susceptible to self-harm,” test item writers can explain that “youth residing in foster care have higher rates of self-harm compared to youth in the general population.” Words such as patient with an amputation opposed to amputee and patient with a BMI of X opposed to obese patient, ensure the person is the center of the interaction rather than defining the person as their ailment.
It is also key to be mindful of intersections within questions, such as including race and incarcerations status and having the correct answer be anti-social personality disorder, or mental health and infectious disease questions, or diagnostic biases with race.
If you are interested in incorporating cultural humility into your writing, Johnson recommends a second or even third pair of eyes. It is easy for biased items to slip through the cracks. Step back and look at the exam holistically. Consecutive items referencing a specific group (age, gender, race, etc.) could unknowingly introduce or perpetuate a stereotype.
