The Hidden Risk in Enterprise AI: Bad Data

Better AI answers start with better sources.  

“The final will be an open-book exam.” Few phrases bring such immediate relief. A friend recently said the same after sitting for a professional certification: she knew the material, but having the book changed both the pressure of the exam and her confidence in the answers. 

In a closed-book exam, everything had to be in your head. You study, you walk in, and you answer from memory. If you studied the wrong chapter or half remember a date, there is nothing to fall back on. 

In an open-book exam, you can check the book. You still need to understand the subject, but you are no longer solely dependent on memory. When you’re unsure of a figure or a definition, you have the luxury of looking it up. 

AI presents the same choice, and I think it’s worth pausing more often to consider how we can give AI an open-book exam. Three situations are worth separating. 

The first is general knowledge. Ask AI how compound interest works, and a closed-book answer is usually fine. 

The second is anything only your organization knows: your benefits plan, last quarter’s numbers, the policy your team updated last week. Closed book does not work here at all. The model never saw those documents, so without them it will not know the answer (and it may even give a confident guess without telling you). 

The third is the murky middle, and it’s where I think an open-book model earns its keep. Ask for Apple’s earnings over the last three years, and AI will probably give you a decent answer. Probably. A figure may be a little off or out of date, and usually it’s hard to trace the answer (though general models are improving at this). Download the relevant earnings reports and attach them to your prompt. Now the answer can draw on documents you chose, making it easier to check the figures, trace the sources, and spot gaps in the analysis. 

Giving AI the book has a name: retrieval-augmented generation, or RAG. The system retrieves relevant passages from source documents and gives them to the model as context for its answer. The shorthand I use is grounding: anchoring an answer to chosen sources rather than relying on the model’s memory alone. Same student, now with a book on the desk. 

The Principle 

AI can answer from memory, and often does well. It does better when it answers from sources you chose on purpose and kept in good shape. 

There are four checks I recommend when considering knowledge sources for AI: 

1. Who can see it? Does the AI’s access match the access of the people using it? 

2. Is it right? Has someone confirmed it is accurate and complete enough for the job? 

3. Is it current, and can we tell? Can anyone tell the latest approved version from an old copy or a draft? 

4. Which source wins when two disagree? Who decides? 

None of this is new. “Garbage in, garbage out” as the saying goes. What AI changes is speed. It retrieves and summarizes faster than anyone can check by hand, so a bad source travels further, faster, before anyone notices. 

You 

Most of us already run the open-book version by attaching a document to a chat. A little discipline makes it more dependable. Try this prompt: 

Answer using only the attached documents. Do not use outside knowledge or guesses. For every important claim, name the source document and quote the supporting passage. If the documents do not contain the answer, say so. If two documents disagree, show me both passages, explain the conflict, and tell me what else you would need to resolve it. Do not assume the newer document is the correct one. 

Each line addresses a potential failure. “Only the attached documents” tells the AI to stay within the supplied evidence, though you should still verify that it follows the instruction. Quoting the passage gives you something to check instead of a smooth summary to trust. Surfacing disagreements helps prevent a clean answer from hiding a conflict. And remember: even a quoted passage should be checked against the original document. 

Check the file too. Is it the approved version? Do you know who wrote it and when? A file called Final_v3_revised next to Final_v3_REAL confuses people as much as machines. And use the Penelope Test: include one question whose answer you already know. If the AI misses it, read the rest closely. 

Your Team 

Teams can do something an individual can’t: agree on where the book lives. 

Picture a team that writes a weekly market analysis. The reports sit in inboxes, on desktops, and across a few shared drives. Someone asks AI how the team’s view has shifted over six months. It grabs an early draft, misses the newest report, or blends documents written for different purposes. The answer sounds fine, and nobody can say whether it reflects what the team thinks. 

Now picture one shared folder for approved analyses. Every published report goes in with a date and a title. Drafts stay elsewhere. One person owns the folder. When a new analysis revises an old one, the team says so. 

Because every file has a date, a title, and an owner, the system has better information to work with. It can be asked to identify the newest report, trace revisions, and flag disagreements—but the team should test whether it does those things reliably. Ask it how the team’s view has shifted, and have it name the reports it used. If it says the view changed in March, anyone can open the March report and check. 

The book doesn’t need to be a folder of documents, and the source does not need to be sophisticated. A well-maintained SharePoint folder or a simple SharePoint list of frequently asked customer questions and answers, for example, can be enough. Point an AI assistant at that list, and it has a small, clean book to read. Clear structure, named ownership, and a few consistent habits go a long way toward equipping a team to use AI with confidence.  

Your Systems 

For each AI use case, consider carefully the approved sources, the owner, the access rules, and how the system treats outdated or conflicting material. 

This is where retrieval-augmented generation, or RAG, becomes valuable. RAG gives the model relevant source material at the moment it answers. It is like asking a student, “What is the powerhouse of the cell?” and, before they respond, placing a passage in front of them that says, “The mitochondrion is the powerhouse of the cell.” The model is no longer relying on memory alone; it has the answer in context. In practice, that makes it possible to ground responses in approved information, link each claim back to its source, and show users where an answer came from. 

The stakes rise if you have a customer facing chatbot. Air Canada’s chatbot told a customer he could apply for a bereavement fare after his travel, while the airline’s own webpage said it did not allow refunds for travel that had already happened. A tribunal held the airline responsible for the information on its website, whether it came from a static page or a chatbot. Whatever the technical cause, the company had two answers in public and nobody had decided which one governed. Read the full story.  

The same risk appears inside an organization when a benefits chatbot pulls last year’s guide instead of the current plan document. The employee has no reason to doubt the answer. The leadership question is therefore not simply whether the organization has connected AI to its content, but whether the system can identify the right content, respect access controls, cite its sources, and acknowledge when the evidence is incomplete or conflicting. 

What This Means for Leaders 

Before approving an AI use case that depends on your organization’s information, ask: 

1. What sources does it use? Ask for the list.  

2. Who owns each source, and when was it last reviewed?  

3. What happens when two sources disagree? Does the system flag it, pick one, or say nothing? 

4. Does the AI’s access match its users’ access? Could it show someone material they shouldn’t see? 

Try this with your leadership team this week: pick one AI use case you run or plan to launch. List its sources, name the owner of each, and run the four checks. Every answer nobody in the room can give is a gap to close before you expand the use case. 

The Takeaway 

An open book gives AI something to work from. It doesn’t guarantee the answer is right. That still depends on whether the information is trustworthy, whether the system can find the right source, and whether it knows what to do when the evidence conflicts. 

The organizations that get the most from AI won’t simply give it access to more information. They’ll make sure it has access to the right information, with clear ownership and a way to tell what deserves to be trusted. 

Give AI an open-book exam when you can, and make sure it’s the right book. 

About the Author

Matthew Hisscock is a Senior Consultant at Thrivence specializing in AI strategy, digital transformation, and enterprise technology. Prior to Thrivence, he served as a Vice President and AI Product Manager at Goldman Sachs, where he led enterprise AI initiatives, product strategy, and large-scale transformation programs. Today, he helps organizations turn emerging AI capabilities into practical, scalable solutions that drive measurable business value.