Clarifying Critical Appraisal: How AI Can Help Librarians and Students Read Research
Submitted by: Molly Montgomery
Critical appraisal of research is a core skill for many health sciences librarians and for those we serve. Despite its importance, developing the knowledge and skills to read research effectively can be challenging. Most librarians have not taken formal coursework in critical appraisal and have developed their skills in this area somewhat piecemeal through workshops and webinars. Critical appraisal education in health sciences professions also lacks uniformity. Some programs offer semester-long courses, while others may offer only a session or two on EBM or reading research.
As someone who pursued librarianship in part because few to no math or statistics courses were required, I frequently struggle to understand the finer points of the methods and results sections of papers. In the past, I would use Google, the EBM books I own, YouTube, or ask a colleague. These all worked to an extent, but I would generally only have a surface understanding, or I would end up with more questions than I started with!
What I was getting were answers, but what I really needed was a thinking partner. That’s where generative AI tools come in. I primarily use Claude, though I also use NotebookLM when I want to limit the AI’s responses to what I’ve uploaded rather than what it retrieves from other sources. My workflow starts with skimming the article to form my own first impressions. Then I upload the PDF and have a conversation with the AI as I read. I ask for explanations of statistical tests, help interpreting results, and clear breakdowns of confusing figures and diagrams. When I want more structure, I’ll include a link to a critical appraisal checklist like CASP and ask the AI to work through it with me. The key is that it feels like a dialogue, not me just looking up facts.
This approach has also shaped how I teach critical appraisal. With students, I focus on using AI to demystify statistical concepts and build confidence in reading primary literature. For residents and faculty, the emphasis shifts toward using AI to efficiently evaluate study quality when time is limited. In each case, I frame the AI as a thinking partner, not an oracle. I encourage learners to start with their own reading, bring specific questions to the AI, and then critically evaluate what it gives back. The goal is to deepen engagement with the paper, not to outsource it.
Of course, any thinking partner has limitations, and AI is no exception. It can miss nuance, hallucinate details, or oversimplify methodological weaknesses. I have enough background to recognize when something doesn’t look right, but I worry that those newer to reading research may not. This is why I emphasize with learners that the skill isn’t just in using AI, it’s in knowing when to push back on it. Specific prompts tend to produce better results than broad ones. Asking “Can you explain what sensitivity analysis is and how it was used in this paper?” will get you much further than “What are the strengths and limitations of this study?” You can also ask the AI for references to verify the information yourself.
Reading research with AI as a guide has been a game-changer for me. I can ask endless questions without feeling embarrassed that I’m not getting it or that I’m wasting my colleagues’ time. I am engaging much more deeply with the content and walking away with a greater understanding. I am careful not to blindly trust the information AI gives me and not to offload my critical thinking. When used in this way, generative AI can help fill gaps we may all have when reading research. As librarians, we are well-positioned to lead in this space, not just by adopting these tools ourselves but also by helping others use them thoughtfully.
Molly Montgomery, MLS, MS (she/her)
Assistant Director of Medical Education and Access Services
Ruth Lilly Medical Library
molmont@iu.edu