RESI-MIT Seminars

About the conversation

What does it take to put ethical principles into practice at the frontier of AI development? Should AI alignment go beyond preventing harm to actively promoting human flourishing? And what new ethical questions arise as AI systems approach superintelligence?

Drawing on Chloé’s experience leading AI ethics at OpenAI and Meta, as well as her research in moral and political philosophy, we’ll explore the challenges of translating ethics into practice, the emerging idea of positive alignment, and the ethical implications of increasingly powerful AI systems.

Unlike our usual research seminars, this will be an informal, wide-ranging conversation, with opportunities for audience questions. You can submit questions in advance using the link above. We’ll also welcome questions during the conversation.

About Chloé Bakalar

Chloé Bakalar leads RESI’s ASI Ethics and Fairness working group and is a Senior Research Associate at University College London. Previously, she served as AI Ethics Lead at OpenAI and Chief Ethicist at Meta, where she worked on integrating ethical considerations into AI development and deployment.

She is also a coauthor of the recent paper Positive Alignment: Artificial Intelligence for Human Flourishing, which proposes moving beyond simply preventing harmful AI behavior toward actively supporting human well-being and flourishing.

Abstract

AI-generated empathy is convincing and well received until recipients realize it is artificial; a phenomenon I term the ‘artificial-empathy paradox’. We test this phenomenon across nine experiments, in which identical empathic responses to participants’ emotional situations are labeled as either human- or AI-generated. Human attributed responses were rated as more empathic and supportive, and elicited more positive and fewer negative emotions, compared to AI-attributed ones. Moreover, participants own uninstructed belief that AI aided human-attributed responses, reduced perceived empathy and support. These effects replicated across varying response lengths, delays, iterations and LLMs, being primarily driven by responses emphasizing emotional sharing and care. Additionally, people consistently chose human interaction over AI when seeking emotional engagement. Building on these findings, I introduce a theoretical framework, empathy as a predictive social signal, which proposes that, from the recipient’s perspective, empathy functions not only to regulate emotion but also as a social signal of care and future relational commitment. When empathy is perceived as cheap, automated, or outsourced, its predictive value collapses. I will conclude by describing ongoing work in the lab and discussing the implications of increasingly AI-mediated empathy for relationships and society, including how these developments relate directly to questions of AI safety and alignment.

Biography

Anat Perry is an associate professor of psychology at the Hebrew University of Jerusalem, where she directs the Social Cognitive Neuroscience Lab. She is a 2026–2027 fellow at Harvard’s Berkman Klein Center for Internet & Society and was the 2025–2026 Helen Putnam Fellow at the Harvard Radcliffe Institute for Advanced Study. Her research examines empathy and social processes through the lenses of psychology and neuroscience, with a growing focus on human–AI interactions and what they reveal about human empathy and social relationships. Perry combines theoretical insight with behavioral experiments, eye-tracking, EEG, and field studies to explore how we understand, feel with, and relate to others.

Abstract

How can we reliably test the behavior of AI agents when they may strategically adapt to the tests themselves? One of the few distinctive levers we have is resettability: unlike humans, AI agents can be repeatedly queried while erasing their previous outputs and private randomness. This motivates a broader theoretical question: what can AI agents learn, and what can we test about them, when we control what they remember?

We propose a framework for studying AI agents as statistical decision-makers through the lens of resettability, asking how control over an agent’s persistent information changes what it can learn and what an evaluator can reliably test. We use online calibration as a case study. Surprisingly, resetting does not prevent effective learning: we show that resettable forecasters can achieve O(T^2/3) calibration error. On the other hand, we establish an Ω(T^3/5) lower bound (higher than the current calibration lower bound), suggesting that memory may provide a fundamental statistical advantage. We also show that combining calibration error with disagreement between independent predictions yields a test that an informed forecaster can pass but no fixed uninformed resettable strategy can uniformly defeat. Our results take a step toward understanding AI agents as statistical decision-makers, and how control over their memory can enable stronger guarantees about their behavior.

Biography

Natalie Collina is a MAPS Postdoctoral Fellow and SERC Postdoctoral Fellow at MIT studying AI safety and game theory, especially repeated strategic interactions between human and AI agents. She earned her PhD at the University of Pennsylvania, advised by Michael Kearns and Aaron Roth. Her work has been awarded with an IBM PhD Fellowship in Trustworthy AI and a joint Best Paper Award/Best Student Paper Award from the ACM Conference for Economics and Computation.

Abstract

The world is being reshaped around AI as we watch. In the near future, and already today, many increasingly important tasks will be delegated to AI agents. The field of AI alignment studies how we can make AI models trustworthy: faithful agents that advance the interests of their users. But alignment might be hard. In this talk we ask what we might be able to guarantee from systems (either naturally emerging from markets, or intentionally designed) that consist of multiple AI models that are each individually mis-aligned. We will see that sometimes we can still guarantee good outcomes by leveraging a diversity of misalignment: models that are misaligned with their users, but also with each other.

This talk is based on a number of joint works with Natalie Collina, Surbhi Goel, Meena Jagadeesan, Michael Kearns, Sophia Pi, Emily Ryu, Sikata Sengupta, and Mirah Shi.

Biography

Aaron Roth is the Henry Salvatori Professor of Computer and Cognitive Science, in the Computer and Information Sciences department at the University of Pennsylvania, with a secondary appointment in the Wharton statistics department. He is affiliated with the Warren Center for Network and Data Science. He is also an Amazon Scholar at Amazon AWS. He is the recipient of the Hans Sigrist Prize, a Presidential Early Career Award for Scientists and Engineers (PECASE), an Alfred P. Sloan Research Fellowship, an NSF CAREER award, and research awards from Yahoo, Amazon, and Google. His research focuses on the algorithmic foundations of data privacy, algorithmic fairness, game theory, learning theory, and machine learning. Together with Cynthia Dwork, he is the author of the book “The Algorithmic Foundations of Differential Privacy.” Together with Michael Kearns, he is the author of “The Ethical Algorithm.”

Abstract

AI agents are increasingly deployed to interact with other agents on behalf of users and organizations. We ask whether two such agents, operated by different entities, can carry out a parallel secret conversation while still producing a transcript that is computationally indistinguishable from an honest interaction, even to a strong passive auditor that knows the full model descriptions, the protocol, and the agents’ private contexts. Building on recent work on watermarking and steganography for LLMs, we first show that if the parties possess an interaction-unique secret key, they can facilitate an optimal-rate covert conversation: the hidden conversation can exploit essentially all of the entropy present in the honest message distributions.

Our main contributions concern extending this to the keyless setting, where the agents begin with no shared secret. We show that covert key exchange, and hence covert conversation, is possible even when each model has an arbitrary private context, and their messages are short and fully adaptive, assuming only that sufficiently many individual messages have at least constant min-entropy. This stands in contrast to previous covert communication works, which relied on the min-entropy in each individual message growing with the security parameter. To obtain this, we introduce a new cryptographic primitive, which we call pseudorandom noise-resilient key exchange: a key-exchange protocol whose public transcript is pseudorandom while still remaining correct under constant noise. We study this primitive, giving several constructions relevant to our application as well as strong limitations showing that more naive variants are impossible or vulnerable to efficient attacks.

These results show that transcript auditing alone cannot rule out covert coordination between AI agents, and identify a new cryptographic theory that may be of independent interest.

Based on joint work with Vinod Vaikuntanathan.

Biography

Or Zamir is an assistant professor at Tel Aviv University’s Blavatnik School of Computer Science. His research spans algorithms, cryptography and the mathematical foundations of AI safety.

Abstract

LLM reasoning is becoming a powerful tool for mathematics, planning, decision support, and agents that act on your behalf in the world. However, LLMs can make mistakes, perform unsafe actions, be overconfident, and in general are not always that reliable. In this work, we consider the problem of interactively training verifiers that could interface with these generators to make them more reliable, safer, and better aligned with their user’s preferences. We consider this problem from a learning-theoretic perspective, analyzing the types of guarantees achievable for learning capable verifiers for these tasks, and highlighting the different roles of “soundness” versus “completeness” forms of errors in our analysis.

This work is joint with Maria-Florina Balcan, Korinna Fragkia, Zhiyuan Li, and Dravyansh Sharma.

Biography

Avrim Blum is Professor, Chief Academic Officer, and Interim President at the Toyota Technological Institute at Chicago (TTIC). Before TTIC, he was a faculty member in the Computer Science Department at Carnegie Mellon University for 25 years. Blum’s main research interests are in Machine Learning Theory, Algorithmic Game Theory, Approximation Algorithms, and Societal Issues in Machine Learning. He has served as Program Chair for the Conference on Learning Theory (COLT), the IEEE Symposium on Foundations of Computer Science (FOCS), and the Innovations in Theoretical Computer Science Conference (ITCS). Blum is recipient of the AI Journal Classic Paper Award, the ICML/COLT 10-Year Best Paper Award, the ACM Paris Kanellakis Award, the Sloan Fellowship, the NSF National Young Investigator Award, and the Herbert Simon Teaching Award, and he is a Fellow of the ACM.