AI Risks in Insurance: When Sensitive Files Enter Unapproved AI Tools

AI adoption inside insurance companies is no longer hypothetical. It is already happening in claims, underwriting, compliance, legal review, customer service, and other document-heavy functions. Employees are using AI to summarize files, search for facts, compare records, draft correspondence, and speed up work that once required hours of manual review.

Increasingly, some of that activity is happening outside the tools their employers have approved. That creates one of the most immediate AI risks in insurance: shadow AI.

Shadow AI occurs when employees use artificial intelligence outside their organization’s approved technology, security, procurement, and governance environment. In claims, that can mean uploading hundreds or thousands of pages of sensitive documents into a general-purpose AI tool and asking it to locate an answer, build a timeline, or produce a summary.

The appeal is obvious. Claims professionals regularly work with enormous files containing medical records, legal documents, financial information, policy language, correspondence, photographs, invoices, and other evidence. AI promises to make navigating all of that information dramatically faster.

The problem is that the employee may have little visibility into what happens after clicking upload. Where is the information processed? How long is it retained? Is the selected product and account configured for the organization’s privacy and security requirements? Did the system process the complete claim file? Can the user verify where the answer came from? Does the employee understand the system’s limitations well enough to know when not to trust its response?

At Nomad Data, we believe the most practical response is not to pretend employees will avoid AI. It is to provide a governed alternative that is designed for insurance work. Doc Chat for Insurance gives carriers, reinsurers, and TPAs a purpose-built environment for working with large, complex document sets while preserving human review and source verification.

The goal is not maximum AI usage. It is safer, more useful, and more accountable AI adoption.

The AI Risk Is Already Inside the Organization

Shadow AI is not an edge case created by a small group of technical employees. It grows whenever people discover a convenient tool that appears to solve a frustrating task faster than the approved process. Brad Schneider of Nomad Data asserts that:

“It's happening at almost every company. People are using AI to try to solve every task imaginable.”

This behavior is easy to understand. Consider an adjuster facing two 500-page documents. Somewhere inside those 1,000 pages are a handful of facts needed to move a claim forward. The traditional solution is manual review: search, read, take notes, cross-reference records, and reconstruct the relevant story.

Now there is another option. The adjuster can upload the documents to an AI tool and ask a question. That can be enormously useful if the system is designed to perform the task, if the account is approved for the data involved, and if the response can be verified.

The danger begins when employees assume all AI tools work in roughly the same way. They do not. A fluent answer can hide major differences in document ingestion, retrieval depth, retention settings, security controls, citations, and output consistency.

This distinction is also explored in Nomad Data’s analysis of general AI tools versus a document AI platform. A chat interface may look familiar across products, but the underlying workflow can be radically different. According to Brad Schneider, CEO at Nomad Data:

“People don't realize, for one, that consumer-grade AI tools don't actually read the entirety of large documents. Even when you ask them to entirely read a set of documents, they're not going to.”

If an employee believes the system reviewed the entire claim file, its answer carries an implied level of completeness that may not exist. The employee cannot easily see what was missed. In a claim file, the omitted detail may be the fact that changes the conclusion.

Why AI Risks in Insurance Are Different

Every industry faces questions about privacy, accuracy, and appropriate use. Insurance raises the stakes because the information is sensitive, the documents are unusually complex, and the resulting work can influence financial, medical, legal, and coverage outcomes.

The risks fall into two broad categories. The first concerns what happens to the information employees provide. The second concerns what happens when employees rely on the information the system returns. A responsible AI program needs to address both.

1. Sensitive Data Can Leave Approved Environments

Claim files can contain personally identifiable information, protected health information, financial information, medical histories, legal records, confidential correspondence, payment details, and internal claims notes. Insurers typically have detailed requirements governing how that information is accessed, processed, shared, stored, and retained.

An employee independently choosing an AI tool can create a mismatch between those requirements and the technology actually processing the information. The company may have a strong legal and security framework, but the individual workflow may sit completely outside it. Brad Schneider of Nomad Data adds:

“There's also HIPAA compliance. There's also data retention requirements. Every insurer has different requirements around these two things.”

The nuance matters. An AI provider may offer enterprise configurations that support particular privacy, security, or contractual requirements. That does not mean an employee using a free or consumer account has selected those protections. Product name alone is not enough. Insurers need to evaluate the specific service, configuration, contract, data flow, and intended use case. According to Brad Schneider, CEO of Nomad Data:

“Even if the tool can be HIPAA compliant, they might not have opted into HIPAA compliance. They may not have opted into zero data retention. And so you get a mismatch between the company's legal framework and what it's actually doing.”

This is why shadow AI is a governance problem, not simply a technology problem. Written policies cannot protect information if employees do not have an approved path that is clear, practical, and useful enough to follow.

2. Incomplete Context Can Create False Confidence

Large insurance files create a difficult technical challenge. A system may need to process hundreds or thousands of pages spread across PDFs, emails, images, forms, spreadsheets, and scanned records. A tool that works well on a short, clean document may behave very differently when confronted with a complete claim file.

The user often sees only the response, not the coverage of the review. If the system used only portions of two 500-page files but the employee believes it analyzed all 1,000 pages, an incomplete answer can be mistaken for a comprehensive one.

“When you only give it pieces of the story and not the entirety of the story, they're going to make assumptions about what exists there. That's where hallucination becomes a problem. That's where accuracy becomes a problem.” - Brad Schneider, CEO of Nomad Data

In claims, an omitted detail is not necessarily trivial. A date, diagnosis, prior condition, exclusion, statement, work restriction, payment record, endorsement, or piece of correspondence can change how the rest of the file should be interpreted. A confident tone does not reveal that the context was incomplete.

Relatedly, Nomad Data’s guide to insurance claims red flags explains why inconsistencies, missing documents, duplicate records, and policy details are so difficult to catch when evidence is fragmented across long files.

3. Errors Can Begin Before the AI Reasons

Insurance documents are not clean blocks of machine-readable text. Claim files can include scans, handwritten notes, tables, checkboxes, photographs, complex medical forms, inconsistent layouts, poor-quality images, and concatenated files assembled from multiple systems.

Before an AI system can reason over those documents, it must interpret them. A checkbox recognized as selected rather than unselected can change the meaning of a form. A table extracted in the wrong order can associate a value with the wrong field. A poor scan can obscure a date, diagnosis, amount, or policy term.

That means the quality of an AI answer depends on more than the model generating the final sentence. It depends on the document pipeline that came before it: ingestion, optical character recognition, visual interpretation, segmentation, retrieval, synthesis, and citation.

Purpose-built systems can devote more attention to that pipeline and tune processing to complex insurance document sets. This is a critical procurement point. Insurers should test the system on representative files, including the messy edge cases that create the most work and risk, rather than judging performance on a clean 10-page sample.

4. Unverifiable Answers Weaken Accountability

An answer without clear provenance asks the user to choose between blind trust and a complete manual reread. Neither option creates an efficient, defensible workflow.

For consequential insurance tasks, the user should be able to trace material statements to the source. Page-level citations can shorten review by directing the professional to the exact evidence supporting a date, condition, amount, clause, or conclusion. They also make errors easier to identify and correct.

Auditability should be designed into the workflow, not added as an afterthought. Teams need to know what information was used, what output was produced, who reviewed it, and how the final decision was made. The appropriate record will vary by organization and use case, but the principle is consistent: AI should make the work easier to inspect, not harder.

5. Inconsistent Use Can Produce Inconsistent Outcomes

Even an approved tool can create risk if every employee invents a different workflow. One adjuster may upload the full file, another may provide only a few documents, and a third may copy isolated text into a prompt. Their questions, verification habits, and escalation thresholds may also differ.

A governed program should define repeatable patterns for high-value tasks. Those patterns can specify the required source documents, approved prompts or presets, expected output format, review steps, and situations that require escalation. Standardization improves consistency while still leaving professional judgment with the claims team.

Blocking AI Does Not Solve Shadow AI

Given these AI risks in insurance, banning general-purpose AI can seem like the obvious answer. Prohibition may be appropriate for specific tools, account types, data classes, or workflows. But prohibition alone does not eliminate the underlying demand.

Employees reach for AI because they have real work to do. A claims professional does not want to spend hours manually searching hundreds of pages for information that software could surface in seconds. If the organization explains what employees cannot use but offers no effective alternative, the incentive to find a workaround remains.

There is another extreme that can be just as ineffective: giving employees broad access to AI and assuming innovation will happen organically. A license is not a use case, and access is not adoption.

“There's really almost too much possibility.” - Brad Schneider, CEO of Nomad Data

Without guidance, an employee may select the wrong tool for a task, get a poor result, and conclude that AI is not useful. Or the employee may get a plausible result, overestimate its reliability, and use it in work that deserves more scrutiny.

“Rather than creativity, what you get is frustration, the belief that AI cannot be used to solve a lot of their daily problems because these particular tools don't solve them.” - Brad Schneider, CEO of Nomad Data

Successful governance therefore needs more than an approved-tool list. Employees need to understand which systems are approved for which data, what each system does well, what it does poorly, when mistakes are likely, and what outputs require human verification.

The goal is not to make employees afraid of AI. It is to make them informed users who have a practical, approved way to complete the work.

A Practical Framework for Managing AI Risks in Insurance

A strong program connects policy, technology, workflow design, education, and oversight. The following framework gives insurance leaders a practical starting point. It is not a substitute for legal, privacy, security, or regulatory advice, and each organization should adapt it to its jurisdictions, products, data, and risk profile.

1. Inventory current AI use. Ask business units where employees are already using AI, including free tools, embedded assistants, browser extensions, meeting tools, and vendor features. The goal is to understand real behavior, not simply compare activity against a policy.

2. Classify data and workflows. Separate low-risk brainstorming from work involving claim files, medical information, policyholder data, legal materials, regulated decisions, or confidential company information. Controls should be proportional to the sensitivity and consequence of the task.

3. Define approved use cases. Give employees clear examples of what they may do, with which tools, using which account types and data. Concrete workflow guidance is more useful than a broad statement that employees should use AI responsibly.

4. Provide a useful approved alternative. The approved system must solve the task employees are trying to complete. If it cannot handle large files, messy scans, citations, or required output formats, users will continue to seek other options.

5. Test with representative documents. Evaluate performance on long, complex, low-quality, and multi-format files. Test both ordinary cases and edge cases. Confirm what the system processed, how it cites sources, and how errors are surfaced.

6. Build human review into the process. Identify which outputs require verification, who owns the final decision, and what evidence the reviewer should inspect. Human review should be a designed step with clear expectations, not a vague disclaimer.

7. Monitor, learn, and update. AI products and organizational use cases change quickly. Review usage, feedback, errors, policy exceptions, and emerging needs. Update controls and training as the program matures.

What Insurers Should Require In Document AI

If insurers want employees to use approved AI rather than consumer alternatives, the approved experience must be both governed and useful. At a minimum, buyers should evaluate the following requirements.

Enterprise-Grade Data Protections

The platform should support the organization’s requirements for sensitive insurance data. Review the actual contract and configuration, including how data is transmitted, stored, retained, accessed, isolated, and deleted. Confirm whether customer content is used to train shared models and what commitments apply to sub processors and support access.

Appropriate Identity, Access, and Governance Controls

Organizations should know who can use the platform, what information they can access, and how permissions align with existing security practices. Consider role-based access, single sign-on, user provisioning, logging, administrative oversight, and separation between business units or clients where relevant.

Capacity for Large, Complex Files

A tool that performs well on a short PDF may behave differently on hundreds or thousands of pages. Ask vendors to explain how the platform handles complete document sets, what limits apply, how it deals with mixed formats, and how users can know whether the full file was processed.

For a real-world view of document-heavy adoption, see how Continental General approached AI in insurance claims for long-term care eligibility reviews.

High-Quality Document Digitization

The system needs to interpret source material accurately before it reasons over it. Evaluation should include medical records, forms, tables, checkboxes, scans, handwriting, images, emails, and the inconsistent layouts common in insurance. Teams should understand how the vendor measures and improves extraction quality.

Grounded Answers With Precise Citations

A generated answer should not become a black box. Claims professionals should be able to open the underlying document at the relevant page or passage, verify the evidence, and correct the response when necessary. Citations reduce verification time and help build calibrated trust.

Repeatable, Workflow-Specific Outputs

Insurance teams need more than generic chat. They need summaries, timelines, comparisons, reports, letters, and structured outputs that follow the organization’s requirements. The platform should support repeatable workflows, approved templates, and clear quality checks.

Human Review for Consequential Decisions

AI should accelerate the path to professional judgment, not silently replace it. The insurer should define when a human must review the source, approve the output, or make the final decision. The higher the potential impact on a claimant, policyholder, payment, coverage position, or legal matter, the stronger the review process should be.

Citations Are How AI Earns Trust

A citation does more than show that an answer has a source. Done well, it creates a direct path from the generated statement back to the evidence. The reviewer can confirm the wording, understand the surrounding context, and correct any mistake without rereading the entire file.

“AI should always be checked, especially in the beginning in terms of clients first beginning their journey using it to solve problems. That's why citations are so important.” - Brad Schneider, CEO of Nomad Data

This changes the review process. Instead of asking an adjuster to trust an AI-generated summary, the system helps the adjuster verify it. Over time, repeated verification gives the team evidence about where the system performs well, where it needs improvement, and which tasks deserve additional controls.

“That builds a lot of confidence in the reviewer that the AI is, in fact, trustworthy. And if for some reason it makes a mistake, they can easily find exactly what the correct answer was.” - Brad Schneider, CEO of Nomad Data

Trust should not be demanded from employees or inferred from a polished interface. It should be earned through transparent, repeatable, verifiable performance.

A Safer Path to AI Adoption

At Nomad Data, Doc Chat is built around this philosophy. Insurance teams can work with complex document sets, ask questions in natural language, create tailored summaries and outputs, and navigate back to the underlying source material.

The broader opportunity is covered in Nomad Data’s guide to real-world generative AI use cases in insurance. The most valuable deployments start with a defined workflow, measurable friction, representative documents, and clear ownership of review.

Insurers also need to make the approved path attractive to employees. That starts with workflow. If moving a claim file from the organization’s document management system into its approved AI environment is cumbersome, employees have an incentive to choose something easier.

Then comes onboarding. Employees need practical instructions, walkthroughs, sample questions, and examples based on their work. Internal success stories can be particularly useful because they show how colleagues solved real problems while following the approved process.

“What can they do well? What can they not do well? When are they likely to make mistakes, and when are they likely to be accurate?” - Brad Schneider, CEO of Nomad Data

Those questions should be part of an insurer’s AI strategy. Training should help users match the tool to the task, recognize warning signs, verify important outputs, and escalate uncertainty. It should also make the boundaries around sensitive information unmistakably clear.

The Biggest AI Risk May Be Using the Wrong AI

The debate around AI risks in insurance is often framed as a choice between moving quickly and moving cautiously. The reality is more complicated. An insurer can move slowly at the corporate level while employees adopt AI rapidly on their own. It can also roll out AI aggressively while failing to educate users about what the technology can and cannot reliably do.

Both paths can produce the same outcome: consequential work being performed with technology that is not appropriate for the task.

“The biggest issue is making mistakes. If you're using the wrong type of AI system, if your users don't understand what it actually does, you're going to make poor decisions.” - Brad Schneider, CEO of Nomad Data

In insurance, poor decisions can affect claim outcomes, payments, customer trust, litigation exposure, regulatory scrutiny, and operational cost. That is why organizations need a middle path between uncontrolled experimentation and blanket prohibition.

The better path is governed adoption: purpose-built tools, appropriate data protections, clear guidance, verifiable outputs, repeatable workflows, human review, and employees who understand both the capabilities and limitations of the systems they use.

Shadow AI grows when employees have a problem that approved technology has not solved. Insurers have an opportunity to address both sides of that equation: protect sensitive claim information while giving employees a faster, more effective way to work with it.

The question is no longer whether insurance employees will use AI. It is whether insurers will give them the right AI to use.

See how Doc Chat for Insurance helps document-heavy insurance teams work with large files, produce source-backed answers, and build repeatable AI workflows within a purpose-built platform.