Safeguard
Security Guides•

Is pandas read_hdf safe? HDF5 files and the pickle warning

pandas read_hdf can load pickled data from HDF5 files in the fixed format. What the pandas warning says, who is at risk, and how to exchange data safely.

Safeguard Security Team
3 min read

Not for files you do not trust. The pandas documentation warns that PyTables can serialize object dtype data with pickle in the fixed format, and that loading pickled data from untrusted sources can be unsafe. An HDF5 file from a stranger deserves the same suspicion as a pickle file.

What the warning says

The pandas page for read_hdf carries a warning box. In its words, pandas uses PyTables for reading and writing HDF5 files, which allows serializing object dtype data with pickle when using the fixed format, and loading pickled data received from untrusted sources can be unsafe. It links to the Python pickle documentation.

So the risk is not that HDF5 itself is a code format. The risk is that one path through the pandas and PyTables stack stores Python objects with pickle, and reading them back can run code.

Where analytics teams get caught

HDF5 files feel like data. They are binary tables with keys, often passed around as results of earlier jobs. People forget they are a Python serialization path when the columns hold arbitrary objects.

Typical exposure points:

  • A shared drive where several teams drop intermediate results.
  • A vendor or partner sending a store file for you to load.
  • A notebook that reads whatever file a user uploads.
  • A scheduled job that picks up the newest file in a folder.

Questions to ask about any HDF5 file

QuestionWhy it matters
Who created the file?Pickle risk depends on the producer, not the extension.
Does it contain object dtype columns?The pandas warning concerns object dtype data stored with pickle.
Was it written in the fixed format?The pandas warning names this format specifically.
Is the path local and controlled?The read_hdf page says only local file system paths are supported.

If you cannot answer the first question with a known internal team, treat the file as untrusted.

Safer ways to exchange tabular data

For data crossing a trust boundary, use a format that holds values, not Python objects. Parquet and CSV are common choices. Both have their own caveats, so keep parsers current and validate the schema you expect. For files you must read from outside, load them in a throwaway container with no credentials and no network, then export the clean result.

A short policy that works in practice:

  1. Internal, signed or access-controlled stores: allowed.
  2. Anything from outside the team: convert in a sandbox first.
  3. Never read an uploaded HDF5 file in a service that holds secrets.

Where Safeguard fits

Safeguard's dependency scanning and SBOM generation can show which of your projects depend on PyTables or pandas, so you know which services have this code path. A free scan is a quick way to get that inventory.

Quick check

Does read_hdf always run code?

No. The concern is pickled object data in the fixed format. The practical advice is to avoid loading such files from sources you do not trust.

Can I read remote HDF5 files with read_hdf?

The pandas page says only local file system paths are supported, not remote URLs or file-like objects.

Sources

Last reviewed: 19 October 2026.

Never miss an update

Weekly insights on software supply chain security, delivered to your inbox.

Self-healing security runs on Safeguard.

Your first fix PR is minutes away.

No sales call required, even your agent can complete the purchase over MCP.