Large Language Models (LLMs) are transformative. From drafting reports to powering chatbots, they seem to understand and respond to our queries with uncanny fluency. But beneath this linguistic polish lies a critical flaw: LLMs don’t understand what they’re saying.

LLMs generate associations based on statistical patterns in their training data. That means they can, and often do, hallucinate — inventing facts, misclassifying concepts, and drawing inferences that sound plausible but lack grounding in truth. These errors are not incidental. They are a direct consequence of how LLMs are trained and how they generate responses.

While this article emphasizes LLMs as consumers of ontologies, there is growing interest in using them to assist with early-stage ontology development, such as identifying candidate terms, possible relationships, or draft category structures from large volumes of text. However, without expert review and clearly defined modeling rules, these suggestions can reinforce the same ambiguity that ontologies are meant to resolve.

No semantic grounding

LLMs lack semantic structure. In a sales context, they may not recognize that “revenue compression” refers to seasonal variation rather than a structural decline, or that “churn” might relate to contract cycles, not dissatisfaction. Without formal rules to constrain meaning, even familiar business terms can drift, mutate, or collapse into incoherence depending on the prompt.

These issues are often systemic. A low-frequency or ambiguous term introduced early in a session can disproportionately influence downstream responses. Researchers at Google DeepMind have documented this phenomenon in models like PaLM-2 and LLaMA. For enterprises in high-stakes domains such as healthcare, defense, or finance, this introduces serious risk.

These challenges — concept drift, ambiguity, and lack of structured constraints — underscore the need for semantic systems that impose clarity and coherence. This is where ontologies enter: not merely as documentation, but as formal mechanisms to define and enforce meaning at scale.

Semantics that scale

In enterprise data contexts, ontologies offer the structure that LLMs lack. An ontology is a formal, machine-readable model that defines the types of entities in a domain, such as people, processes, documents, or metrics and the relationships between them.

In a sales analytics setting, an ontology may define entities like sales region, revenue metric, and seasonal baseline. It ensures that metrics are consistently linked to regions and time periods, enabling valid comparisons, accurate anomaly detection, and reliable reporting across business units.

While taxonomies support structured relationships and play a key role in information architecture, ontologies go further by explicitly modeling a broader range of logic – such as roles, part-whole relations, cardinality constraints, and temporal dependencies. Ontologies are also machine-readable in ways that enable automated reasoning, inference, and validation, which taxonomies alone typically cannot support.

The absence of semantic alignment can be costly. During the Airbus A380’s development, engineering teams in Toulouse and Hamburg used incompatible CAD platforms: CATIA V4 and CATIA V5. Without a shared semantic framework, concepts like cable routing and spatial clearance were misinterpreted, leading to misaligned models. Over 500 kilometers of fiber optic wiring failed to align during final assembly, costing Airbus’s parent company, EADS, approximately $6 billion. This underscores that ontologies are infrastructure critical for managing complexity.

When operationalized, ontologies serve as the semantic foundation for trustworthy AI, data governance, and decision systems. They not only describe enterprise reality – they validate and protect it.

From documentation to operational execution

Making ontologies operational means moving beyond static representations like OWL definitions or UML diagrams and instead deploying ontologies as active components in real-time systems that reason, validate, and identify inconsistencies.

At the core of this approach are top-level ontologies (TLOs) such as the Basic Formal Ontology (BFO), which define universal categories like objects, processes, and roles to ensure semantic coherence across domains. Building on this foundation, mid-level ontologies (MLOs) like the Common Core Ontologies (CCO) represent common entities found in enterprises, information systems, and technical infrastructures.

These ontologies are enforced through executable constraints using SHACL, a language that ensures incoming data adheres to domain-specific logic. Additionally, SPARQL (SPARQL Protocol and RDF Query Language) queries provide analysts with powerful tools to interrogate knowledge graphs, enabling complex, logic-aware questions like “Which customer records are missing required consent documentation under current data privacy policies?” or “Which sales regions have performance anomalies that deviate from seasonal baselines?”

When integrated, these components form a complete ontology-driven execution pipeline – one that continuously validates inputs, infers appropriate roles and classifications, and detects gaps before flawed data propagates through downstream systems.

The most important standard you’re not using:

Shapes Constraint Language, or SHACL, is an industry standard used to check data quality by applying logic rules to RDF graphs. It works by identifying whether the data follows expected business rules. For example, a SHACL rule could state that every sales region must include both current and historical performance metrics, and that these metrics must be connected to a valid seasonal baseline. If any of these are missing, such as a benchmark or consistent time period, the system flags the issue before it affects reporting or decision-making.

Despite its value, SHACL remains underused. For CDOs, this highlights a broader issue. Ontologies that describe structure but do not enforce it can leave enterprises exposed to semantic drift. SHACL helps close this gap by ensuring the data is not only labeled correctly but also behaves according to defined expectations.

Ontologies vs. “Computational Epistemology”

There’s increasing buzz around “computational epistemology” – the idea that AI can model how people form beliefs and make decisions. But most implementations don’t model knowledge: They map correlations.

Consider a sales analytics scenario: an AI might infer that discounting in Q4 causes customer churn, simply because the two co-occur. Without ontological constraints, the system can’t determine whether this is a causal effect, a seasonal correlation, or a misattributed coincidence. It lacks the formal logic to test the validity or generalizability of its conclusion.

Enterprise governance demands more. Real epistemology and real data strategy require structured logic, validated relationships, and contextual awareness. Ontologies provide exactly that.

The future is hybrid: LLMs + Logic

LLMs are not going away. Nor should they. But their outputs must be tempered by constraints. The future of enterprise AI is hybrid: Using LLMs for language generation, and ontologies for grounding, validation, and structured reasoning.

What CDOs can do now

Organizations can begin integrating ontologies by selecting a foundational ontology such as BFO, which defines high-level categories like ‘process’ and ‘object’ that are consistent across domains. Domain-specific ontologies, tailored to the organization’s needs, can then be layered on top using standard mid-level frameworks. These ontologies should be implemented using SHACL constraints and integrated into execution pipelines through validation services or governance tools. This combination ensures logical consistency, real-time validation, and semantic integrity across systems without requiring business users to engage with the technical complexity directly. 

Ontologies should be embedded directly into execution pipelines to enable real-time validation, automated inference, and structural integrity. Suppose an LLM is asked, “What caused the drop in Q4 revenue?” Without ontology grounding, a system might hallucinate a response like “The drop was due to increased customer churn from higher prices,” even if prices remained stable.

With an embedded ontology and constraints in place, the system could cross-check known business logic, including seasonality definitions, churn metrics, and price histories, and respond: “Revenue decreased in alignment with defined seasonal patterns. Churn and pricing remained within expected baseline ranges.”

Most importantly, systems should abstract away complexity from users. Business users should be able to submit grounded instance data, while the ontology-driven system handles reasoning and validation transparently in the background.

Constraint is not a limitation; it’s the foundation

In high-stakes environments, distinguishing between a seasonal fluctuation and a systemic failure isn’t semantics. It’s the difference between coherence and collapse. When paired with formal logic and tools like SHACL, ontologies do more than model enterprise structure: They enable systems to protect it.

The integration of ontologies with LLMs is not just a technical enhancement – it’s a strategic necessity. By providing formal, machine-readable structures, ontologies ground LLMs in defined meanings, validated relationships, and logical constraints, preventing hallucinations that can derail high-stakes decisions. Tools like SHACL enforce these guardrails in real time, ensuring data and inferences align with enterprise reality.

In a world where semantic ambiguity can cost billions, as exemplified by the Airbus A380’s wiring failure, ontologies are no longer optional. They are the foundation for trustworthy AI, enabling enterprises to harness LLMs’ fluency while safeguarding coherence, precision, and operational integrity. 

Glossary

• BFO (Basic Formal Ontology): A foundational ontology that defines high-level categories like object, process, role, and quality. Used to ensure semantic consistency across domains.

• CCO (Common Core Ontologies): A suite of mid-level ontologies built on BFO, covering common enterprise concepts such as agents, organizations, documents, and events. Designed to promote interoperability in large-scale systems.

• MLO (Mid-Level Ontology): A reusable ontology that defines concepts common to many domains, more specific than a top-level ontology but not tied to a single application. MLOs act as a bridge between foundational models and operational needs.

• Ontology: A formal, machine-readable representation of the types of entities that exist in a domain and the objectively real relationships between them. Ontologies provide semantic structure that aligns data with reality, enabling consistent interpretation, logic-based validation, and reliable integration across systems.

• RDF (Resource Description Framework): A standard model for representing structured data as subject–predicate–object triples. RDF serves as the foundation for semantic technologies such as ontologies, SHACL constraints, and SPARQL queries.

• SHACL (Shapes Constraint Language): A W3C standard for validating data by applying logic-based rules to RDF graphs. SHACL ensures that data follows defined business rules and schema expectations.

• SPARQL (SPARQL Protocol and RDF Query Language): A query language for retrieving and analyzing structured information from RDF-based knowledge graphs. Allows users to ask complex, logic-aware questions across connected datasets.

• TLO (Top-Level Ontology): An ontology that defines universal categories like object, process, and role, applicable across all domains. Serves as the foundation for building consistent and interoperable ontological models.

*Disclaimer: The views and opinions expressed in this article are those of the author in their personal capacity and do not represent the views of CDO Magazine or its editorial team.

About the authors:

John H F Bittner II is a Senior Ontologist and strategic data leader, with over a decade of experience supporting the U.S. Intelligence Community and Department of Defense. He holds an MBA in Finance and is currently pursuing a Ph.D. in Applied Ontology from the University at Buffalo. His research centers on anomalies, AI systems, and computational logic. He is a member of the National Center for Ontological Research (NCOR) and the Institute of Electrical and Electronics Engineers (IEEE) Standards Association Ontology Working Group.

Timothy W. Coleman is a Senior Director and Senior Ontologist at BasisPath, Inc., with over a decade of experience directly supporting the U.S. Intelligence Community, Department of Defense, and law enforcement agencies. He holds an MBA in Finance, a Master’s in Public and International Affairs with a concentration in Security & Intelligence, and is currently pursuing a Ph.D. in Applied Ontologies from the University at Buffalo. His research focuses on decision theory and heuristic models for strategic artificial intelligence. He is the inventor on multiple U.S. patents spanning communications jamming, modular disaster response, and autonomous technologies.