What Your Program Should Know About AI and Admissions

In addition to other challenges in the admissions process, PA programs are now concerned about the ease of access to generative artificial intelligence (AI) programs that can write an applicant’s personal statement. 

For those unfamiliar with this new technology in the context of text creation, generative AI programs collect data, often from the internet, which is used to train large language models to create original works in response to user prompts. Generative AI platforms predict the text desired by the user in response to the user’s prompts. The program “learns” from the training data as well as each user prompt to improve its generative outputs to be better at predicting the text desired by users.  Generative AI platforms are most accessible when incorporated into chatbots, such as Open AI’s ChatGPT or Google’s Bard.

The concern is that PA applicants can use generative AI chatbots to create a personal statement to submit to a PA program.  PA applicants can easily enter prompts into chatbots that instruct the chatbot to produce a personal statement that is tailored to the personal qualities, academic aptitude and healthcare provider experiences of candidates emphasized by a particular program.  Applicants can add additional direction to chatbot prompts, whether true or not, to make up or embellish their qualities or experiences.

While generative AI adds a new dimension to concerns about the authenticity of personal statements, personal statements that may have been plagiarized or written by someone who isn’t the applicant is not a new problem for PA programs. Personal statements have long been evaluated carefully to ensure they are the original work of each applicant. 

Donna F. Murray, DMSc, MS, PA-C, senior director of admissions/CASPA at PAEA, said the personal statement is important because it gives program faculty an opportunity to see the applicant for who that person is and what sets them apart. 

“It’s the first clue we’re going to get into that person’s integrity,” Murray said. 

And Marc Weinstein, legal counsel to PAEA, added, this is why it’s so crucial that the personal essay be authentic. 

CASPA has a long-standing policy that required personal statements to be the work of the applicant and prohibited applicants from submitting “ghost-written” statements that were by a friend, purchased or copied from another source. 

The Association recently updated the policy to prohibit any statement written by generative AI. There is also language in the agreement that requires that letters of recommendation or evaluation are the original work of the person who is submitting them and can’t be created by generative AI. 

Weinstein said programs should create their own policies in advance of receiving any questionable applications to ensure transparency, fairness, and consistency in how they are treated by programs if personal statements are discovered to be inauthentic.  

Unfortunately, current tools available for detecting material written by a generative AI have fairly high false positive and false negative rates. Weinstein said using current technology to detect essays and personal statements written by generative AI platforms would be risky for programs given demonstrated reliability issues.  Weinstein add that the developers of the AI detectors themselves also warn in their terms of service against using them as the sole basis to make important decisions about students. 

PAEA will not investigate an applicant if the only evidence the applicant did not write his or her personal essay comes from AI detection software. Weinstein said the Association’s position is that the current AI detection tools are simply not reliable enough yet. 

One workaround is asking an applicant to write an essay during the in-person interview.  

“If you’ve got someone who’s right there at the interview, give them a prompt. That’s a good way to ensure that they are the author of (the response to) the prompt,” Murray said. 

However, Murray noted that by the time someone is present for an interview, that applicant has moved through a very critical portion of the selection phase of the process. Murray acknowledged that left a gap in choosing people who were worthy to be interviewed – something that had been accomplished in the past, in part, through the personal essay. 

Other solutions to ensure authentic character and personal background information from applicants include requiring short response interview-type prompts that applicants must answer in a remote-online proctored platform, through a lockdown browser.  Programs may also consider requiring applicants to submit short-form digital video statements (two minutes or less) in response to specific program prompts, which could be accomplished through recordings on mobile phones.

Generative AI technology is developing but PAEA staff members are continuing to have conversations as a team, with senior leadership, and with legal counsel, to gather information that will inform future changes to CASPA as needed. 

Responding to new technology and new methods that could be used to subvert the process of c selecting applicants with the best fit for a particular program has always been a challenge for admissions staff.  

Learn More  

In response to member requests, PAEA recently hosted a webinar to explain how generative AI platforms might be used and how PAEA and PA programs can respond. The webinar included a live demonstration of how a generative AI can create a personal essay based on external prompts.

The webinar, available to be viewed in the Digital Learning Hub, was hosted by Murray and Weinstein. The event was informational and not intended to provide legal advice for programs but allowed members to learn more about generative AI, like ChatGPT, and consider how they may want to respond. 

Watch the webinar.