Securing LangChain and LlamaIndex Applications in Production

Securing a LangChain or LlamaIndex application means containing three things the frameworks make easy by default: code-executing components reachable from user input, an enormous fast-moving dependency tree, and retrieval pipelines that feed untrusted documents straight into a privileged model. These frameworks are glue for the most dangerous integration in modern software — LLM plus tools plus your data — and their convenience functions do not distinguish between a demo notebook and production. The gap between those two contexts is where the CVEs live.

Know the CVE history before you trust the defaults

Both frameworks have accumulated a meaningful vulnerability record, mostly in exactly the places you would predict:

• LangChain: SSRF in the Web Research Retriever (CVE-2024-3095), path traversal, SQL injection via crafted prompts in SQL chain integrations (CVE-2023-36189 class), and the long tail of langchain-community and langchain-experimental issues. The infamous early ones — arbitrary code execution through PALChain and LLMMathChain prompt injection — are why code-executing chains got quarantined into langchain-experimental in the first place.
• LlamaIndex: command execution via unsafe eval in PandasQueryEngine (CVE-2023-39662), later reworked but with documented bypasses of the restricted evaluator, plus a 2024–2025 stream of issues in readers and integrations.

The pattern: core is reasonably audited, the integration packages are a bazaar. Treat langchain-community the way you treat random npm packages — because that is what its 600-plus integrations effectively are.

Kill the code-execution paths first

Inventory every component that can execute code or queries, then decide deliberately:

# Find the dangerous imports in your codebase
grep -rn "PythonREPLTool\|PALChain\|LLMMathChain\|PandasQueryEngine\|exec_\|allow_dangerous" src/

Rules that hold up in production:

• PythonREPLTool and anything from langchain-experimental do not ship to prod attached to user-facing agents. If the product genuinely needs code execution, run it in a real sandbox — gVisor, Firecracker, or a locked-down container with no network and a 5-second CPU cap — never in the app process.
• LangChain now requires allow_dangerous_code=True or similar flags on risky components. Grep for those flags in CI and fail the build if they appear outside an allowlisted module. A one-line Semgrep rule pays for itself immediately.
• SQL chains get a read-only database role with row limits, not your app's connection string. Prompt-injected DROP TABLE is not clever anymore, but it still works on default setups.

Contain the dependency sprawl

A fresh pip install langchain langchain-community llama-index pulls in a tree that routinely exceeds 200 packages once you add a vector store client, document parsers, and an eval library. Document parsers deserve special suspicion: unstructured, pypdf, image libraries — these process attacker-supplied files and have their own CVE histories.

Baseline discipline:

• Pin everything with a resolver (uv lock, pip-compile) and rebuild the lockfile on a schedule, not ad hoc. Framework minor versions break APIs often enough that unpinned prod deploys are self-inflicted incidents.
• Install only the integration extras you use: langchain-openai, langchain-chroma — not the community meta-package — and the specific llama-index-readers-* you need.
• Run continuous SCA against the lockfile; with this much surface, new CVEs land monthly, and staying under 30 days behind on framework patches is a defensible SLO. An SBOM per release lets you answer "are we exposed to the new pypdf CVE?" from a query instead of a rebuild — SBOM Studio or any CycloneDX tooling works, as long as it is generated in CI and not by hand.

Treat retrieval as an untrusted input channel

RAG is indirect prompt injection with extra steps. Any document your pipeline ingests — uploaded PDFs, scraped pages, Confluence exports — can carry instructions that your model will read with whatever privileges the agent holds.

Mitigations that are actually deployable:

• Separate privileges by pipeline stage. The summarize/answer model over retrieved content gets no tools. Tool-using agent steps consume the output of that model, schema-validated, never raw retrieved text.
• Sanitize at ingestion. Strip zero-width Unicode, flatten HTML comments, and drop embedded instructions patterns before chunks hit the vector store. Cheap, imperfect, still worth it.
• Scope retrieval by tenant and by user ACL at query time, not just at index time. Cross-tenant leakage through shared vector stores is the RAG bug class auditors now check for first — filter metadata on tenant_id in the retriever, and verify the filter is enforced server-side in your vector database, not just in application code.
• Log retrieved chunk IDs per response. When an answer goes wrong, "which documents produced this" is the first forensic question and most teams cannot answer it.

Runtime controls: callbacks are your audit log

Both frameworks expose hooks that make security observability nearly free — use them:

# LangChain: a callback handler that logs every tool invocation
class AuditHandler(BaseCallbackHandler):
    def on_tool_start(self, serialized, input_str, **kwargs):
        audit_log.info("tool=%s input_sha=%s", serialized["name"],
                       sha256(input_str.encode()).hexdigest())

Wire handlers (or LlamaIndex's instrumentation module) into: token spend per session (runaway agent loops show up here first), tool-call sequences, and retrieval sources. Add hard caps — max_iterations on agent executors, request timeouts, per-user rate limits — because an injected agent's first observable symptom is usually cost, not compromise. Platforms like Safeguard can then correlate the app-layer inventory (framework versions, integration packages, model IDs) with runtime findings, which is what turns "we patched LangChain" from a hope into a report.

Frequently asked questions

Is LangChain safe for production use in 2026?

Yes, with the same caveat as any fast-moving framework: core is solid, but you own the risk of the integration packages you pull in and the components you enable. Teams that pin versions, avoid langchain-experimental in prod, and gate dependencies do fine; teams that ship notebook code do not.

LangChain vs LlamaIndex — is one more secure?

Their risk profiles differ more than their quality. LlamaIndex's surface concentrates in readers and query engines (document parsing, eval-based engines); LangChain's in agents, tools, and the community integration sprawl. Pick by use case and apply the same hardening baseline either way.

How do I stop prompt injection through my RAG documents?

You reduce it rather than stop it: privilege-separate the model that reads retrieved text from anything holding tools or credentials, sanitize at ingestion, enforce ACL filters in the retriever, and monitor tool calls. Filtering alone will not survive a motivated attacker.

What belongs in the SBOM for a RAG application?

The full Python lockfile, the vector database client and server versions, document parser packages, embedding and generation model identifiers, and ideally the system prompt version. That inventory is what lets you respond to the next parser CVE or model deprecation in minutes.