Enterprises are facing a quiet but growing risk: the loss of institutional memory.

Workers 55 or older made up 24% of the U.S. workforce in 2022, up from 10% in 1994. When those employees walk out the door, they will take with them decades of institutional knowledge.

Today’s enterprises are incredibly interconnected, with vast webs of dependencies in both the digital and physical world. This complexity magnifies the risk of losing institutional memory.

Generative AI can help reduce that risk by making critical knowledge easier to capture, retrieve, and share across the organization.

The SME: Where institutional knowledge resides 

In many organizations, there are go-to people who understand the nuances, shortcuts and edge cases needed to command the digital and physical systems that run the enterprise. These subject matter experts (SMEs) are incredibly valuable. 

When SMEs leave, years of institutional memory and knowledge may be lost even with processes in place to transition that knowledge.

The effect can ripple across systems and teams, especially if the SMEs were the connective tissue across disparate parts of the business.

While there is no substitute for human insight, generative AI (GenAI) provides another tool in the knowledge capture and surfacing toolkit.

Why documentation may fail to preserve institutional knowledge

Documentation and the system that supports it are nothing new. They’re the foundation for any well-functioning enterprise.

But documentation doesn’t capture the human insights and edge cases that make an SME invaluable.

SMEs are experts because they’ve built insights over years. They know where to look when something breaks. They recognize patterns. They know the details and the reasons behind decisions.

It’s tough to document that wisdom, but responsible use of generative AI can help capture knowledge and make it retrievable at the right time and place.

How generative AI is changing knowledge retrieval and access

This is where generative AI becomes an accelerator – particularly large language models (LLMs) ‒ combined with retrieval-augmented generation (RAG) or knowledge surfacing via tooling actions (model context protocol or MCP).

Traditional documentation systems rely on users knowing the right keywords, the right sections to look in, and the right query to surface the knowledge most relevant in the moment.

LLMs change that dynamic because they are good at parsing natural language, gauging intent, and surfacing information from large corpuses of knowledge.

With RAG, when you type a prompt, the system performs a similarity search and hydrates the LLM’s context with the most relevant documents. The response is grounded in your enterprise knowledge, reducing hallucination and allowing you to verify the source.

Beyond RAG, LLM-based agents equipped with tooling via MCP or other protocols can call internal tooling sources to retrieve information – either to act on it directly or hydrate its context with this information.

Imagine a new employee working on a project. Instead of typing isolated keywords, they include the system they’re modifying, the context of the question, any constraints they’re facing and what they’re trying to accomplish.

The AI can retrieve relevant documents, parse those documents with knowledge of what the engineer is trying to accomplish, and surface that insight directly back to the employee.

The new employee can follow up to clarify where needed. With modern reasoning models, the interaction mirrors that of a mentor-mentee relationship with an SME.

This approach does not, and cannot, replace deep expertise and human interactions. But it can augment it and soften the institutional knowledge loss that follows an SME leaving.

From knowledge capture to proactive intelligence

Many organizations already have the building blocks to get started.

Most major cloud providers offer LLM plus RAG or LLM + MCP capabilities out of the box. You can start small, for instance, by loading FAQs and internal documentation into a system and piloting it for a specific use case.

A common example is internal support tickets: Instead of a Level 1 support team manually answering every request, a grounded AI chatbot can respond to common questions. The answers to these questions may already exist, and the chatbot makes them easy to find.

The impact can be measured by looking at whether response times are shorter, if there are fewer support tickets or fewer follow-ups, if the answers are more consistent (evaluations are important), and if general satisfaction of users has increased.

For enterprises already implementing AI, the above examples may seem trivial. But it gets more complex when you account for SME knowledge that isn’t easily captured in your enterprise FAQ. This requires a new process to understand questions an SME might answer that aren’t in your FAQ. This is the most difficult part.

I’ve seen this in practice with an SME who wanted to work with AI to capture their nearly 30 years of experience. They wanted their legacy imprinted in the organization, not just in the mind of a new employee. Through the above techniques, their knowledge was cemented and their colleagues thanked them. 

The risks of relying on AI for institutional knowledge

Like any tool, AI has limits. LLMs are not infallible and even with grounding, hallucinations are not solved. You still need verification layers, including human review, standardized evaluations, and the rigorous documentation practices.

That’s a broader workforce strategy question and indeed, a conversation we should continue to have. Institutional memory fades, systems evolve, and people move on. GenAI doesn’t magically solve that, but it can parse, contextualize and surface data in a way that softens the loss of institutional knowledge.

Get started

If you’re new to using generative AI to capture institutional memory, it’s important to understand exactly what you’re trying to solve.

I recommend starting with existing processes that are structured, repeatable, and backed by the latest data. Technical specifications, standard operating procedures, and system-level documentation are good candidates.

Solve processes where finding information is time consuming and frustrating for your employees. In parallel, begin thinking about what constitutes SME knowledge and how it might be captured. That’s when the real unlock happens.