Upanzi Seminar: Mary-Anne Hartley, Yale University

February 28, 2024

5:00 p.m. CAT

Virtual

Abstract

Large language models (LLMs) have the potential to democratize access to medical knowledge. Unfortunately, the enormous potential of these models is either locked behind commercial/research licenses, in violation of privacy regulations, limited in scale, or not generalizable to underserved populations and resource-limited settings. To address this issue, we developed Meditron-70B, currently the world’s best-performing fully open-source chatbot for medicine, trained on carefully curated clinical practice guidelines from diverse settings. However, the performance of these chatbots is commonly measured on medical exam questions, which does not adequately evaluate real-world clinical utility and safety. In this talk, I will introduce Meditron and show how we are crowdsourcing incentivized expert evaluations that is putting Meditron to the test. I introduce the MOOVE (Massive Online Open Validation and Evaluation) platform that allows doctors to validate the real-world performance of Meditron in terms of helpfulness, harmlessness, bias, trust, and safety. In return for this rigorous validation, participants can get their own chatbot, adapted to their preferences and specialty.

Bio

Mary-Anne "Annie" Hartley is an Assistant Professor in Biomedical Informatics and Data Science at Yale University. Her research is focused on developing and validating novel data-driven tools designed to improve healthcare in low-resource settings, with a special interest in Africa.

She completed her undergraduate degrees at the Universities of Pretoria and Cape Town before moving to Switzerland, where she completed a MD at the University of Lausanne, with an MPH at the London School of Hygiene and Tropical Medicine. In 2019, she started the research group, "Intelligent Global Health" in the School of Computer Science at the Swiss Institute of Technology (EPFL) and continues this work in LiGHT (Laboratories for intelligent Global Health Technologies). Through these groups, she maintains a strong presence and partnership between EPFL and Yale through student exchange, research collaboration, and a visiting professorship. The groups collaborate with international NGOs and clinical partners to create and validate needs-based digital global health technology using novel approaches in data science and. informatics.

Upcoming Events