Interested in ontologies? Let me tell you what's what.
But shhh, don't tell Palantir!
Recently, you may have noticed a flood of posts from various people on LinkedIn singing the praises of Palantir’s Foundry Ontology. It seems a little coordinated, or perhaps there has just been an enthusiastic briefing to the analyst community.
Whatever, I am not buying it.
For sure, Palantir’s Ontology is a sophisticated way to map business operations and data relationships, but building and maintaining an ontology that accurately represents an entire organization’s data, processes, and models is extremely complex and demanding. Of course, this is one of the reasons that Palantir's business model is so intensive with consulting. Any ontology requires not only advanced technical expertise but also continuous updates as the business evolves. That's a lot of billable hours.
Expensive description becomes prescription
Ontologies, by their nature, require a structured approach to organizing data and processes, but they can be dangerously prescriptive - imposing a business model - rather than descriptive.
However, right now, with businesses undergoing seismic changes in working practices, supply chains and (thanks to AI and automation) fundamental shifts in their operational practices, an expensively created, authoritative ontology could be a greater burden than a benefit. Ontologies, while excellent at mapping and organizing the known, generally fail to adapt to evolving environments where entities and relationships are constantly in flux.
Innovation and ontological uncertainty
Why? Because innovation and adaptability require what David Lane has called ontological uncertainty.
Ontological uncertainty refers to a deep kind of uncertainty where the very nature of the entities we deal with is uncertain. It's not just that we don't know the outcome of a decision (which would be more like traditional uncertainty), but we don't even fully understand what the entities are that interact or how they might change.
For example, we might not fully understand what kinds of objects (like technologies, companies, or market participants) will exist in the future. We don't know what kinds of interactions or relationships these entities will have with one another. As a result, we can’t predict how these entities and relationships might evolve over time.
This kind of uncertainty is especially relevant in innovation because when you're creating something genuinely new, you can’t rely on the past to guide you. The existing categories, systems, or models might not apply, and the structure of the future you’re trying to build is unknown or still taking shape.
Innovation is not just doing the same things better; it demands that we do things differently. In other words, it demands that we break the very ontologies that Palantir would have us expensively build. With the changes coming to business, the very nature and structure of the entities, relationships, and systems that define a given environment or domain will change. Businesses and organizations cannot fully know or anticipate the kinds of entities (people, processes, technologies) and relationships (interactions, dependencies, dynamics) that will exist or be important in the future. You'd have thought a libertarian like Peter Thiel, founder of Palantir, would get this.
All the new problems are old
I have seen this before. In the 90s, I worked on a metadata repository and ontology for the credit agency of a large US car manufacturer. When it was complete, it represented their entire business in detail. Except ... within a few months, the business analysts had found a new analytic tool that they believed gave them a key competitive edge. (It was called Knosys, later ProClarity, later acquired and mostly destroyed by Microsoft, but that's a different sad story.)
This new, exciting tool created metadata that, at first, could not be enrolled in the repository. So, a project was begun to enable that. Meanwhile, using insights from this tool, the teams changed numerous practices and methods of tracking credit risk. By the time the ontology team had caught up, the business had moved on again.
And here was the real problem: an incomplete ontology is not just lagging behind; it's wrong. And if it's wrong, it's misleading and useless, or worse. And so the ontology, so carefully and expensively created, was abandoned within months.
Here, I must mention, if only for the sake of that authentic philosopher and grand artificer of analytics Stephen Swoyer, Derrida’s punning Hauntology. The pun works better in French. In Derrida's terms, Hauntology explores how the past, particularly ideas or events that have not fully manifested or have been repressed, continues to "haunt" the present and shape the future. It’s about the persistence of unresolved histories, ideas, or possibilities that linger, influencing the present even if they are not fully actualized or realized.
It’s just like your old business model, forcing people back into the office, even though that never really worked well in the first place. Leaders have a haunted nostalgia for the office environment—the "water cooler moments," face-to-face meetings, and the structure it supposedly provided. As Derrida might say, the return to the office—what is this return? A return to what? What specter lingers in this compulsion, this haunting desire to bring bodies back into space, as though the office itself were not already spectral, a place neither fully here nor fully gone? The "office," that site of supposed productivity, is constructed (de-constructed, even) in the image of its own impossibility.
Meanwhile, thank you, Stephen; let’s get back to business.
I have also done some work with a South African insurance company that had made an equally significant investment in ontologies and metadata. They faced the same problem as the auto credit team. Evolving practices invalidated the ontology, but an outdated ontology was worse than useless - it was wrong! Their answer - the rigid, prescriptive answer - was that no one could make changes to entities, relationships or schemas, without first pre-registering them with the repository to ensure they were captured. The ontology was running the business - and the tail was wagging the dog.
There’s another, wittier way to say this. Some time ago, I reviewed Paul Arden’s fine book, It’s Not How Good You Are, It’s How Good You Want to Be. He says …
It’s wrong to be right …
Being right is based upon knowledge and experience and is often provable …
Knowledge comes from the past, so it’s safe. It’s also out of date. It’s the opposite of originality.Experience is built from solutions to old situations and problems. The old situations are probably different from the present ones, so that old solutions will have to be bent to fit new problems (and possibly fit badly.) …
Palantir would have you spend hundreds of thousands of dollars building the right model of your business.
I think they are wrong.
Late to the game here, but I'm curious, Donald: You said,
"Knowledge comes from the past, so it’s safe. It’s also out of date. It’s the opposite of originality. Experience is built from solutions to old situations and problems. The old situations are probably different from the present ones, so that old solutions will have to be bent to fit new problems (and possibly fit badly.)"
This doesn't seem to be unique to ontologies. Isn't it also a limitation of LLMs?
This (excellent) discussion is remarkably similar to the one had for decades in Operating and Finance departments around the implementation of ERP systems like SAP. Do we want to design and enforce the most efficient semantic/ontological structure appropriate for a particular way of operating our firm, or do we want to create and maintain a fluid ontological/semantic exchange that allows our firm to adapt quickly its operating model? After working in this space for a few decades, I see two topics overlooked most often when answering the question.
First, the question is not a religion-philosophical one; it is a deeply strategic one. The IT folks should not be leading the meetings for a while, because the strategy folks have a lot of work to do first. If your firm operates in a mature, stable, even monopolistic environment, it will gravitate to the structured approach, despite the significant restrictions on operating flexibility that such an approach imposes on the firm. If your firm operates in a highly competitive, innovative, often-disrupted environment, it will gravitate to the fluid approach, despite the never-ending requirement for a huge amount of work on negotiated/curated/not-right-the-first-time adaptation.
Second, there is no free lunch. Think of the differences between structured and adaptive as lease-vs-buy, and you are getting close to the financial reality. There’s simply a huge economic cost to being able to understand one another and understand all of the complex, differentiated processes within a large organization. We can pay that price through various combinations of (a) constraints on our flexibility to adapt and (b) work managing our adaptations. Anyone who sells you the promise of cheap and easy semantic harmony for all conditions is … well, selling something.
A word of caution: the focus on cost too early can be a distraction. We need to ask ourselves “what is the value of understanding one another and all the process peculiarities of our business?” The answer is, as the kids say, VLN (very large number). In a way that’s good news: it means we’ve found a fulcrum point at which our work can create tremendous value for the firm. But, we must see the strategic questions clearly if we are to achieve that value.