Practical guide to security and sensitive data in LLM applications

LLM applications do not need unrestricted access to everything. The point is to control what enters the model, what gets stored, who can access it, and what the AI can do.

Context

In the latest posts in the series, the conversation moved through important parts of an AI application: RAG, chunking, embeddings, retrieval, structured outputs, evals, harnesses, agents with tools, and observability.

After all of that, there is one question that cannot be left until the end: which data can this AI see, record, remember, retrieve, or use to act?

An LLM call is rarely only a question and an answer. It often includes a prompt, user data, document snippets, search results, history, metadata, files, logs, validation, fallback, and maybe tools connected to other systems.

The problem

When we talk about security, it is common to think first about login, passwords, route permissions, databases, encryption, or infrastructure. All of that still matters. But LLM applications add other paths through which data can move unnecessarily.

• A prompt can carry too much personal data.
• A log can store the full input and output.
• A RAG flow can retrieve a document the user should not see.
• An embedding index can keep references to internal content without a clear policy.
• An agent can have permission to call a tool that is too broad.
• A response can repeat sensitive information that was in the context.

The risk does not always come from a spectacular failure. It often comes from accumulated practical decisions: sending the whole document when a snippet was enough, storing full prompts for debugging, not separating retrieval permissions, or making a tool available for any intent.

Mental model

Before calling an LLM, I like to think through five questions.

11. What goes in?22. What comes out?33. What gets stored?44. Who can access it?55. What can the AI do?

These questions look basic, but they organize almost all of the reasoning. They cover prompts, context, files, visible responses, structured outputs, logs, memory, cache, traces, embeddings, end users, internal teams, support tools, and actions with side effects.

If an AI feature cannot answer these questions well, it is still too early to give it autonomy or sensitive data.

Minimize data before improving the prompt

The first security decision is simple: do not send the model what it does not need to see.

This applies to personal data, internal data, long documents, conversation history, tokens, credentials, metadata, and any information that entered the flow only out of convenience.

• A support summary may not need a tax ID, phone number, address, or full email.
• A document analysis may need three relevant snippets, not the whole file.
• A message classification may need the text and a category, not the user’s full history.
• A suggested reply may need summarized context, not all profile data.

Good context is context that is sufficient for the task. Too much context can increase cost, latency, confusion, leakage risk, and audit difficulty.

Sensitive data in prompts

A prompt is not just temporary text. In many systems, it can appear in logs, traces, observability tools, development environments, execution history, evaluations, or internal support screens.

• Is the data necessary for the response?
• Can it be masked?
• Can it be replaced by an internal identifier?
• Can it be summarized before reaching the model?
• Does it require explicit consent?
• Does it need to be removed from logs?

1Avoid:2"Summarize the support case for customer Bruno, tax ID 000.000.000-00, phone..."34Prefer:5"Summarize the support case. Remove personal data and focus on the problem, the action taken, and the next step."

It will not always be possible to remove everything. Some tasks genuinely depend on specific data. But that decision needs to be explicit, not accidental.

Safe logs

Logs are a double-edged sword in AI applications. Without logs, it is hard to investigate errors, cost, latency, fallback, validation, and model behavior. With too many logs, you can create a repository of prompts and responses full of information that should not be stored.

1{2  "feature": "document-summary",3  "model": "model-name",4  "promptVersion": "summary-v2",5  "status": "validated",6  "latencyMs": 1620,7  "inputTokens": 840,8  "outputTokens": 210,9  "validation": "passed",10  "usedFallback": false11}

This kind of record helps investigate the flow without storing the whole document. Depending on the case, it can make sense to store the internal execution id, prompt version, model, validation status, response time, token estimate, error type, and whether fallback was triggered.

A good log is not a log that stores everything. A good log lets you investigate without unnecessarily increasing risk.

RAG with sensitive data

RAG is often presented as a way to give external knowledge to the model. But when the knowledge base contains internal or sensitive information, the problem is not only retrieving the most relevant snippet. It is also retrieving only what that user is allowed to see.

Retrieval without access control can turn a private base into context available to the wrong person. This can happen silently: a document is indexed, semantic search finds a similar snippet, the snippet enters the context, and the model answers with something that looks useful.

• Store source and permission metadata with documents.
• Filter by user, organization, role, or scope before the final ranking.
• Avoid mixing public and private bases without clear labels.
• Record which document was used in the answer.
• Define a removal or reindexing policy when content changes.
• Review whether chunks, embeddings, and metadata contain sensitive information.

RAG does not remove access rules. It only changes where those rules need to appear.

Embeddings are also part of the system

Embeddings are sometimes treated as if they were a harmless version of the original text. In practice, they are still derived from real data and are part of the information architecture.

• Where did the indexed content come from?
• Who can query this index?
• Do the chunks contain personal or internal data?
• Is there a policy to delete or update data?
• Do metadata fields reveal more than they should?
• Do test environments use real data unnecessarily?

In a small project, this may start with a simple rule: do not index sensitive data that is not necessary for the experience.

Agents with tools need limits

A chatbot answers. An agent with tools can query, decide, fill in, change, send, create, delete, or publish. That difference changes the risk.

When AI has tools, the problem stops being only "it answered something wrong" and starts including "it did something wrong".

• Read tools are different from write tools.
• Reversible actions are different from destructive actions.
• Internal queries are different from external sends.
• A suggestion is different from execution.
• A regular user is different from an admin user.
• A test environment is different from production.

Useful autonomy is not full access. Useful autonomy is action within clear limits.

Prompt injection is not just theory

Prompt injection can feel distant until the application starts reading external content. If the AI reads pages, documents, comments, emails, tickets, or any input written by someone else, that content can try to influence the model’s behavior.

The basic safeguard is to separate roles: a system instruction is not a retrieved document; a retrieved document is not a command; a user message is not a security policy; a tool result is not authorization for another tool.

• Treat external content as data, not as a sovereign instruction.
• Limit available tools.
• Validate arguments before execution.
• Ask for human confirmation on sensitive actions.
• Block obvious exfiltration attempts.
• Record when an action was denied.

Output validation is also security

Output validation is not only for checking JSON. It can also prevent a response from being used when the model goes out of scope, exposes improper information, invents critical data, or attempts an action that should go through review.

• Does the response contain personal data that should not appear?
• Does the response cite a document without permission?
• Does the response recommend an irreversible action?
• Does the response exceed the allowed scope?
• Should the response ask for human review?
• Did the response fail to classify the confidence level?

A technically valid response can still be inappropriate. The right format does not guarantee that the content can move forward.

Human review is still part of the flow

Some automation looks mature in a demo and fragile in operation. It works well while inputs are comfortable, but starts failing when sensitive data, exceptions, ambiguity, permission conflicts, or real impact appear.

• Financial impact.
• Legal impact.
• External sending.
• Irreversible changes.
• Content publication.
• Use of personal data.
• Access decisions.
• Low confidence or conflicting sources.

A good AI feature does not need to choose between fully manual and fully automatic. It can automate preparation, summaries, triage, suggestions, and initial validation while keeping a person at the point where the decision truly matters.

Minimum checklist before publishing

• Which data enters the prompt?
• Can any sensitive data be removed, masked, or summarized?
• Is the full prompt stored anywhere?
• Is the full response stored anywhere?
• Do logs record enough metadata without storing unnecessary private content?
• Does RAG respect access permissions?
• Are retrieved documents traceable?
• Does the embedding index have retention or update rules?
• Which tools can the AI call?
• Does any tool require human confirmation?
• Does the output go through validation?
• Do critical cases require review?

This checklist does not automatically make an application secure. But it avoids a common mistake: treating AI as a magic layer that can see everything, remember everything, and do everything.

What I am not saying

This is not a legal guide. It does not replace LGPD, GDPR, compliance, internal policy, or specialized review. It is also not a complete cybersecurity guide.

The goal is more practical and limited: to help developers notice where sensitive data appears in LLM features and which questions need to be part of the technical design from the beginning.

Lessons learned

• Security in LLM apps also involves prompts, context, logs, RAG, tools, and responses.
• The first protection layer is minimization: do not send the model what it does not need to see.
• Logs help operations, but full prompt and response logs can become a new source of risk.
• RAG needs to respect access permissions from indexing to retrieval.
• Agents with tools require clearer limits when they can write, send, delete, or publish.
• Output validation is also part of security when a response can expose information or trigger a decision.
• Human review remains important for actions with real impact.

Limits and caveats

Each product has a different level of risk. An internal study feature, a local prototype, a public assistant, a system with personal data, and an operational automation do not require the same level of control.

There are also decisions that depend on legislation, privacy policy, vendor contracts, infrastructure architecture, and internal requirements. This article does not try to replace that work.

Conclusion

Useful AI does not need to be AI with unrestricted access.

A good LLM feature needs context, but it also needs limits. It needs enough data to solve the task, but not all available data. It needs logs to operate, but not a permanent archive of everything the user wrote.

In the end, security in AI apps starts with a simple decision: define what the AI can see, remember, record, and do.

Good AI in a product needs context, but it also needs boundaries.