Integrating AI Large Language Models into PubMed Searching for a Medical Student Grand Rounds Course

Abstract

Background: Librarians teach PubMed searching via lecture and graded exercise in a Medical Student Grand Rounds (MSGR) course at Texas A&M University School of Medicine. In this semester-long course, students work with mentors to discover and present about translational basic sciences research on a clinical topic at Grand Rounds Day. ChatGPT and other Large Language Models (LLMs) are intriguing options to streamline the search process but require discernment for proper use. The teaching team set out to incorporate guided exposure to LLM opportunities and challenges into the existing search training and exercise process. Description: Students in MSGR must use PubMed to search the biomedical literature to inform a semester-end presentation about translational research with potential to impact clinical care. Librarians have taught PubMed search skills in the course for several years, including the use of MeSH terms and subheadings, filters, and keywords. These skills are reinforced with a graded exercise taking students through the steps to gradually refine a search. Recognizing that students would likely experiment with ChatGPT to save time on their searches, librarians worked with course directors to revise the lecture and exercise to emphasize the process of searching, to highlight where artificial intelligence (AI) is already used in PubMed, and to suggest both uses and caveats of current LLMs to generate usable PubMed searches. The manual searches in the exercise were condensed to accommodate the additional generative AI search instructions. Students were then asked to use the LLM of their choice to generate search terms for the same topic, critique the LLM response, and share their experience with peers in a course discussion board. Grading and feedback centered on good-faith efforts to perform the search process and to evaluate thoughtfully the LLM tool. Conclusion: The recent emergence of publicly available tools for generative AI based on LLM presents new challenges for instruction, particularly when students are required to submit “original” work in assignments. The PubMed searching instruction and exercise have traditionally garnered strong positive feedback in prior student course evaluations. This paper will present student feedback on the inclusion of generative AI tools in the exercise and will describe differences between students’ work with and without the use of generative AI for their PubMed assignment. Librarians and course leadership will review the Spring 2024 course evaluations as well as the next developments in generative AI to continually improve the relevance of the course content. We expect the approach to evolve regularly.