The term “ontology” is now en vogue. Why? Agents, and the LLMs that power them, need relevant business context.
Before you buy into an ontology, ask the vendor: “When the ontology gets our business wrong, how do our people correct it—and what still depends on you?” The answer will tell you a great deal about whose model of the business you are buying into. It also points to the larger stakes: your ability to improve that model as the business changes, and to keep that investment useful if you change platforms.
It helps to distinguish an ontology from a knowledge graph, even though the two often overlap.
Ontology: What kinds of things exist, what relationships mean, and what rules apply. Equipment, components, teams, roles, processes. For example, an ontology can define an “Equipment” type and a “Contains” relationship to a “Component”. A knowledge graph can then assert, “Pump P-101 contains Seal S-7”: two specific entities and their relationship.
Imagine you tell an agent, “Compute same-store sales growth, but show me a side-by-side where we show all stores, then another where we exclude stores open less than 2 years; then break both sides of that up into domestic, international, and combined.”
That ask depends on details an agent cannot guess:
- what stores exist, when they opened
- where they are
- the internal definition of “same-store sales”
A policy might exclude stores until they have been open a year; the requested two-year group adds a further filter. Even then, do you compare calendar months as soon as a store qualifies, or include it only after a full prior fiscal year?
The ontology needs to connect those stores to the policy that governs the calculation. Finding related facts helps; applying the right rule is what makes the answer useful.
An agent can use relationships in the graph to retrieve relevant context before answering a question. And no wonder they are useful, when knowledge domains can easily get very complex.
Figure 1: Graph: combined public SysML models: TMT Thirty Meter Telescope (Open-MBEE/JPL, Apache-2.0), CubeSat System Reference Model (© 2022 INCOSE SSWG & OMG SDTF, CC BY 4.0), Architecting Spacecraft with SysML (Friedenthal & Oster, BSD-2-Clause), plus fictional MDC constellation content and public specs (IS-GPS-200, IS-GPS-705, GSFC-STD-7000B). Rendered in Kamiwaza Kaizen.
But extraction can also encode mistakes. Teams may use different names for the same entity, creating duplicates. Sources can disagree or become outdated. Even a clean source can be misread by an LLM. Making those mistakes look authoritative does not make them less dangerous.
Those risks grow when a model of the business becomes a system that acts on it. A wrong relationship can then trigger a wrong action, not just a wrong answer.
Risks of Ontology Products
First, be wary of inflexible models. In the name of making something “batteries included,” a vendor may make its own entities first-class citizens while limiting what yours can do. Customization is not enough if custom types lose search, validation, or action capabilities.
Second, look for a path from discovery to governance. A tool should let you bootstrap a graph from what an LLM sees, then validate it against explicit constraints. You should be able to escalate exceptions to people and preserve exploratory knowledge without treating it as approved business data.
The largest risk is tying business processes to a platform you cannot leave. Your ontology represents your business: its knowledge, processes, and relationships. That investment in curation and business logic should stay useful even if you change tools. Why should a vendor get to put that under lock and key?
Where Graph Embeddings Help
As knowledge graphs grow very large, it can be worth choosing a tool or vendor that supports graph embeddings. These are close cousins of the semantic embeddings that power many RAG applications. There are two useful benefits to consider.
First, a few hops through a densely connected graph can yield too many entities to process. By learning numerical representations of entities and relationships, embeddings offer another way to find relevant parts of the graph. For the right task, this can be more efficient than walking the graph.
Second, embedding models can rank candidate missing relationships. The same learning can reveal patterns in the attributes and relationships typical of an entity type, helping to build a more complete graph.
The Case for Progressive Improvement
To own your own destiny, avoid lock-in, and model your actual business instead of a vendor-dictated caricature of it, how do you begin without an army of forward-deployed engineers? This is where I think modern AI shines, if the tooling supports a path from discovery to verification.
First, use automated discovery and extraction. From business documents, policies, email, or Slack messages, you can construct an initial view of a domain. The critical part is that you cannot trust it as accurate until it is verified.
Verification is the second part. Your tools must give you a smooth path from extracted candidates to claims grounded in sources, and to knowledge validated by a responsible person or authoritative system. Preserve each claim’s source.
Freshness is ongoing work. The process for handling a trouble ticket, or the answer to “how many stores are there in New Jersey”, may not be the same today as it was three years ago. But updating today’s knowledge must not erase yesterday’s reality. When a fact was recorded and when it applied can differ; a useful ontology needs to support both current answers and an accurate historical view. Someone still must keep that knowledge current.
The scope can be massive. Start with a smaller corpus, map a useful domain, and build toward a comprehensive picture while creating value along the way.
Customer support is one useful slice: inbound calls, emails, a ticketing system, automated resolutions, a support team that can respond to customers, and a path to escalate true bugs to engineering. It connects knowledge to action and exposes the same risks and rewards in a manageable domain.
ServiceNow’s Common Service Data Model illustrates the constraint. Its CSDM 5 white paper warns, “Without this data in prescribed tables, you may not receive the full value of the ServiceNow AI Platform.” The warning itself creates a design and operational constraint: lean into the vendor’s model or accept a risk. What if that means cramming your round peg into their square hole?
Now suppose an ontology platform also supplies ingestion, transformations, actions, and business rules. You may be able to export entities, attributes, and relationships. But can you also port transformations and action pipelines that execute changes in your databases, call webhooks, and create other side effects? Tightly coupled integrations can leave the business itself locked in.
Why Flexibility Matters Now
Software is eating the world, and AI is eating software. As teams use AI both to build software and to power it, more business logic can become tied to these platforms. Each useful automation can also become another migration dependency. It’s dangerous to let a vendor stand between you and agility.
Conclusions
It’s an amazing time to be alive! Little wonder people are exploring ontologies to provide context to agents. Extending that context into processes and integrations can create real value. Come back to that opening question. Your people’s corrections should make the model a better reflection of your business. That accumulated understanding is too valuable to be stranded when you change platforms. It’s also a terrible time to be painted into a corner.