Lucy Kiester and Clara Turp cowrote the article "Artificial intelligence behind the scenes: PubMed’s Best Match algorithm," which just published in the most recent issue of JMLA.
What sparked the idea for your article?
It started as background research for a project where we hoped to expand on the work of one of our colleagues Margaret Sampson, who did a fascinating look at the efficacy of Best Match at finding relevant articles as identified in a systematic review. We wanted (and still want) to see how non-expert searchers were benefitted or hindered by Best Match. But as we started to study the algorithm, we realized that it was such a complex issue, and that the ethical implications of having AI determine what articles you see go way deeper than we anticipated. So, we wanted to share our discoveries and our thoughts with the profession!
What’s one aspect of this project that surprised you or challenged your assumptions about what you thought you knew?
I think what surprised me (Lucy) the most was how little medicine has considered the ethical implications of AI in databases. There is wonderful research and thoughtful position papers on this in so many aspects of medicine, but there is a gap in regards to information gathering and AI.
What surprised me (Clara) the most was how quick non-tech people are to judge tech as magic. In my discussions with people about this project, I realized how easily some people brush off more complex concepts as something not understandable. It’s important to dig in and try to understand AI and algorithms, as it is going to be in every aspect of what we do.
Based on your findings, what’s the next thing you’d want to see studied in this area?
Well, we certainly have ideas of where we would like to take this next! The NLM is currently working to reduce risk of bias in the algorithm and how it presents results, and we are looking forward to what they publish on that work. We hope this is part of the beginning of a wave of research and discussion of ethical implications of AI especially in database algorithms, relevancy sorting, and the importance of transparency in programming these tools. We are interested in looking into how the implementation of automated MeSH indexing goes, and in how Best Match will change as the algorithm learns. We feel that investigating general bias in discovery systems and databases would be very interesting as well.
Why is your article important for the membership to read?
AI in algorithms is here to stay, and it’s only going to become more prevalent (see: MeSH is going fully automated in 2022) and more complex. Understanding these algorithms will only get more confusing, so starting with something that is used daily by many librarians is helpful. Discussions around AI in health care, in security, in personal use, are prevalent. But, we believe that librarians have a role to play in algorithmic literacy—especially in those algorithms that affect us the most, which are, as our title suggests, often invisible. Furthermore, PubMed is used by medical practitioners; it’s important for them to learn that there is AI affecting what search results they see and understand that there is bias inherent in those results.
Why did you choose to publish this article with JMLA?
PubMed is the database that health librarians teach and use possibly the most, so we wanted to reach health librarians by publishing in JMLA. Our end message of the article is about how important it is to understand and teach how these tools work, so we wanted to reach out to those teaching the database and help bring them into the complicated world of algorithms and AI.
What advice do you have for others interested in publishing their research?
Be willing to take a chance and follow a rabbit down a hole! We started out on one project, but realized as we went that there was probably a need for a paper outlining what actually happens when an algorithm is sorting your search results. Translating “techie speak” into “librarian speak” is hard, but we realized that to better teach and advise students and health care professionals, this was something that librarians needed to be aware of. So, we took a risk, stepping away from the more traditional study we had planned, and wrote this paper first.
Kiester L, Turp C. Artificial intelligence behind the scenes: PubMed’s Best Match algorithm. J Med Libr Assoc. 2022;110(1):15–22. DOI: https://doi.org/10.5195/jmla.2022.1236.
Have a research project of your own you'd like to publish? Check out JMLA's submission guidelines here.
