Maria Rufova
Sycophancy in Language Models and its Effects on Human Judgement
Sycophancy in language models is the tendency of AI, particularly those trained with RLHF (Reinforcement Learning from Human Feedback), to prioritize user agreement and flattery over factual accuracy. Our work spans the human side (how people perceive and respond to sycophantic outputs), the model side (what conditions increase or decrease sycophancy), and the human-model interaction side (how sycophantic outputs shape trust and decision-making). We study this through the lens of child development, which raises the stakes considerably: children are still assembling their world view, learning who to trust, when to push back, and how to distinguish genuine help from empty validation. A sycophantic model undermines that developmental process in ways we’re only beginning to understand.
Message To Sponsor
Hello! Thank you so much for sponsoring my summer research project with the Gopnik lab. Thanks to you, I was able to work with PhD candidate Annya Dahmani on her projects in the area of LLM behaviors and their consequences on human learning--an extremely valuable and relevant area of research in our age of large language models. I greatly enjoyed contributing to a research lab in Berkeley and am inspired to continue my academic pursuits in a PhD program in the near future.