MIT Symposium Examines How AI Systems Fail When Their Logic Doesn't Match Ours
MIT's Schwarzman College of Computing hosted a full-day research symposium on April 30 to examine artificial intelligence's role in society and its ethical implications. The event featured research presentations, panels on AI alignment and education, and a keynote address by Cornell computer scientist Jon Kleinberg.
The central tension running through the symposium: as AI becomes embedded in education, hiring, medicine, and law, who decides what values the technology should reflect-and how do you translate human judgment into machine logic?
The alignment problem: whose values count
Panelists tackled a foundational question in AI development. Iason Gabriel, a philosopher at Google DeepMind, used the example of a judge. "You want a judge to have good character, but to still interpret the rules," he said. "A reasonable person, though not necessarily the best person who ever lived. When it comes to AI, it's not appropriate to model it as perfect."
The problem deepens when you consider power. Bailey Flanigan, an assistant professor of political science at MIT, said the most urgent question is "resolving fundamental questions on who is entitled to govern different types of AI systems in the first place."
Bernado Zacka, also an associate professor of political science, warned against moving too fast. "One of the most urgent problems is understanding the wisdom contained in the systems we are replacing, and why they function the way they do," he said. Institutions develop rules and procedures over decades. Swapping them for AI without understanding how they actually work risks losing knowledge embedded in those systems.
Despite deployment pressure, panelists expressed optimism about AI alignment's trajectory. They emphasized that human judgment remains central to building these systems responsibly.
In education, the risk is outsourcing thinking
As students use AI tools, schools face a practical dilemma: how do you use these tools without replacing the cognitive struggle that produces learning?
Samuel Madden, a computer science professor at MIT, described the problem directly. When students hit a difficult concept, their instinct is now to ask AI for the answer. "They don't see this as excelling in this process, and they haven't actually acquired the skill you're assessing," he said.
Eric Klopfer, director of MIT's teacher education program, agreed. Critical thinking is disappearing from the work itself. His suggestion: examine entire curricula and remove content that's no longer essential. "Some core content has to go. We keep adding, instead of parsing or pruning," he said.
The solution isn't a blanket policy. Pat Pataranutaporn, head of the Cyborg Psychology research group at MIT Media Lab, said AI tools should be designed differently depending on their purpose. "AI is not just one thing. It can and should be designed differently to promote things like creativity and critical thinking," he said.
Justin Reich, director of MIT's Teaching Systems Lab, noted that simply telling students AI is problematic doesn't stop them from using it. But inviting them into decisions about how and why AI gets used in their classes could help them make intentional choices.
The chess engine problem: when AI's logic becomes invisible
Kleinberg's keynote examined a specific failure mode: systems that work brilliantly on their own but confuse humans when they need to hand off control.
Modern chess engines play at superhuman levels. But when paired with human partners, their strategies become incomprehensible. A human chess player can't predict what the engine will do next because the engine thinks in patterns of probability and constraint, not the embodied knowledge humans develop through years of play.
"The danger of human-algorithm teams is that when the human takes over, the algorithm knows what it wants to do next, but the human doesn't," Kleinberg said.
This gap matters when humans and machines work together in fields like medicine, law, or aviation. The system's model of the world-built on pattern recognition and simulation-differs fundamentally from human reasoning, which is rooted in experience and intuition. The question Kleinberg posed: if the outcome is correct, does the mismatch in reasoning matter? In some contexts, the answer is no. In others, it's everything.
Learn more about Generative AI and LLM Courses or explore AI for Education to understand how these systems are being developed and deployed.
Your membership also unlocks:
