Dylan Chen Rose Hills
Cohort-Based LLM Specialization for Expert-Level Domain Reasoning
Large language models (LLMs) like ChatGPT are remarkably capable at general tasks but struggle to reason reliably in specialized fields, producing plausible-sounding answers that experts can quickly flag as shallow or wrong. This research develops a new training pipeline that transforms general-purpose AI models into genuine domain experts by mimicking how humans develop professional expertise: first building foundational knowledge, then structuring how concepts relate, and finally practicing reasoning through complex cases. The approach trains a team of five architecturally diverse LLMs through curated curriculum, then has them collaboratively debate each case until they reach consensus, repeatedly cross-checking to catch errors. Neuropsychological evaluations serves as the initial proof of concept, a high-stakes task where clinicians currently spend hours on analysis while patients wait weeks for results. If the pipeline succeeds there, the project will test whether the same approach transfers to other complex fields such as financial analysis or cybersecurity, with the goal of designing a reusable framework that unlocks expert-level AI reasoning across any specialized domain.
Message To Sponsor
Thank you so much [donor] for supporting my summer research project on making LLMs more trustworthy in high‑stakes settings like neuropsychological assessment. By building AI systems that can accurately analyze clinical cases, I hope to advance AI systems toward expert‑level judgment to help health providers serve a greater population. Your generosity makes it possible for me to explore this new training pipeline with clinicians and patients in mind, and I am truly grateful for this opportunity.