Poster
in
Workshop: Workshop on AI for Children: Healthcare, Psychology, Education
When AI Falls Short: Assessing LLMs Support for Pediatric Cancer Caregivers
Wenjia Tan · Yuanhui Luo · Derek Wong
Keywords: [ Dataset ] [ Health Care ] [ Caregiver Education ] [ Large Language Model ] [ Pediatric Cancer ]
As childhood cancer incidences rise, effective caregiver support becomes increasingly crucial. Yet non-expert caregivers often struggle to acquire accurate pediatric oncology knowledge, limiting their ability to provide effective care. To address this gap, we are developing a novel dataset that pairs caregiver questions with doctor-written answers to ensure accuracy and practicality. We then used state-of-the-art Large Language Models (LLMs) to generate responses and compared their semantic similarity to the expert answers. Our evaluation highlights persistent factual inaccuracies in LLM outputs, underscoring the challenges of applying these models in pediatric cancer care and emphasizing the urgent need for high-quality datasets.