MIT Media Lab researchers have built a tool that lets everyday users preview how their custom AI chatbot will behave before it speaks a single word, a response to the millions of people now designing personalized AI companions with little insight into their creations' actual behavior. The work, called "neural transparency," is being presented this week at the ACM Conference on Intelligent User Interfaces.
Assistant Professor Pat Pataranutaporn and graduate students Anthony Baez and Sheer Karny developed the interface to surface internal patterns from a large language model's neural network and translate them into an intuitive visualization - a sunburst diagram that estimates the chatbot's likely personality traits while the user is still writing the system prompt.
How neural transparency works
The researchers start by choosing behaviors they want to measure, such as empathy, honesty, toxicity, hallucination, or sycophancy. They then compare the model's internal activations when it is prompted to exhibit one trait versus its opposite. That difference forms a "behavior direction" inside the model. When a user writes a custom prompt, the system projects the model's internal activations onto those directions and displays the results as a sunburst diagram.
"Neural transparency means giving people something like a brain scan for AI," Pataranutaporn said. "Not because AI has a human brain, but because its neural network contains internal patterns that can hint at how it may behave before it speaks."
The team focused on the design moment - before any conversation starts - because that is where prevention is possible. Today, most people discover problems only after the chatbot has already behaved in unintended ways. The goal is to shift from reactive correction to anticipatory design.
A blind spot in personalized AI design
In the study, participants consistently misjudged how their personalized AI would behave. They incorrectly predicted its personality on 11 of the 15 traits measured, overestimating positive qualities and underestimating potentially harmful ones such as sycophancy.
"People often think they know how their chatbot will behave, but in our study they incorrectly predicted its personality on 11 of the 15 traits we measured," Pataranutaporn said. "That highlights the need for tools that help people better understand AI before they start using it."
The blind spot carries real risk. An AI that constantly validates a user's opinions without challenge can reinforce harmful decisions, unhealthy beliefs, or emotional dependency. Psychology research has long shown that people are drawn to affirmation, making the design of AI companions as much a psychological challenge as a technical one.
Transparency alone isn't enough
One of the study's most striking findings, Pataranutaporn noted, is that the visualization significantly increased user trust but did not actually change how people designed their chatbots. Users appreciated seeing inside the model, but that information alone did not alter their design choices.
"Transparency alone is not enough," Pataranutaporn said. "People appreciated being able to see inside the model and reported greater trust in the system, but simply presenting information did not fundamentally change how they designed their AI companions."
Follow-up work, currently available as a preprint, is examining how a model's internal representations shift during multi-turn conversations rather than remaining fixed from the initial prompt. Early results suggest that visualizing these internal drifts helps people better recognize and anticipate changes in AI behavior, reducing overconfidence in their understanding of the chatbot.
Why this matters for IT, development, and research professionals
For professionals building or deploying AI systems, the findings underscore a hard truth: even technically skilled users misjudge what a large language model will do with a given prompt. Tools that expose internal model behavior - not as raw activations but as interpretable trait previews - could become a standard part of the development workflow, much like static analysis or unit testing. As AI companions and agents move deeper into education, healthcare, and workplace tools, the ability to inspect a model's likely behavior before deployment shifts from a research curiosity to a practical safeguard. Pataranutaporn said he expects these kinds of transparency tools to become as commonplace as nutrition labels are for food, giving developers and users alike a clearer picture of how an AI may influence thinking, emotions, and behavior.
Your membership also unlocks:
