Giving an AI system access to enterprise data is not the same as giving it the context needed to understand that data. A new research briefing from the MIT Center for Information Systems Research argues that a semantic layer is the missing component for many organizations deploying generative AI tools and agents, and that getting it right determines whether AI outputs are accurate or misleading.
The problem with raw data
When organizations pull information from applications, documents, data platforms, and outside sources, they often strip away the business context that gives it meaning. Different systems may label the same customers, products, or transactions in different ways. Organizations also apply different rules for quality, access, permitted uses, and regulatory compliance.
Without a consistent way to interpret those differences, AI systems can produce answers that are "technically plausible but also incomplete, misleading, or wrong," the researchers write in their briefing, "The Case for a Semantic Layer."
What a semantic layer does
A semantic layer sits between an organization's data and the people or machines using it, explaining what the data represents, how different pieces relate to one another, and which rules govern its use. It draws on data dictionaries, taxonomies, knowledge graphs, and ontologies to preserve the business context that disappears when information is removed from the application where it was created.
That context has become critical as organizations expand their use of Generative AI and LLM agents. Models need to know where data is stored, whether it's reliable, how it can be combined with other data, and which regulatory or privacy requirements apply. A semantic layer captures those details in a form that machines can act on.
The researchers cite Healthcare IQ as an example. The data and analytics company maintained hospital supply chain records and a catalog of nearly 6 million products from more than 25,000 manufacturers. The same medical product might appear under one description in one hospital system and an internal code in another, making it difficult to compare information. After building a semantic layer using custom data dictionaries, taxonomies, ontologies, and access-control databases, Healthcare IQ identified equivalent products, standardized descriptions, flagged data quality problems, and applied privacy controls. It also automated 80 percent of the work required to onboard new hospital customers.
Three steps to building a semantic layer
The researchers recommend three actions to ensure that investments in a semantic layer generate value.
Start with priority data assets. Leaders should identify the data needed for their highest-priority AI initiatives rather than attempting to describe and organize every piece of enterprise data at once.
Govern the semantic layer itself. The layer needs a clearly designated owner accountable for its quality and results, someone who works closely with data platform owners and participates in decisions about when data and definitions should be updated or retired.
Use AI to help create and maintain metadata. AI can support generating metadata, cleaning and classifying data, recommending access controls, and identifying connections among data assets. That capability will become increasingly important as organizations manage growing numbers of AI tools and volumes of unstructured content.
Why this matters for executives and strategy
In a 2024 survey of 349 executives, only 21 percent rated their organization's data curation practices as somewhat or very well developed. Organizations with more developed practices were more than three times as likely to report being effective at AI for Executives & Strategy initiatives that generated value, and twice as likely to say those initiatives provided a meaningful competitive advantage. As generative AI tools proliferate, competitive advantage will depend on how well organizations make their proprietary data accessible and understandable to both people and machines. Those that build an effective and direct layer can scale AI faster, at lower cost, and with greater confidence that their data is being used appropriately.
Your membership also unlocks:
