Pin the exact versions of every Python package a data project uses, in a lock file committed to the repository, and install from it everywhere: notebooks, scheduled jobs and container images. Then scan the lock file. Unpinned installs make results unrepeatable and let a bad release reach your data jobs unannounced.
Why data projects drift
Notebooks encourage a pip install in a cell. Orchestrated jobs often install requirements at start, with loose version ranges. Each run may fetch something different from the last. That causes two problems.
First, repeatability. A result you cannot reproduce because a library changed behavior is a correctness issue. Second, security. A job that installs at run time with access to warehouse credentials is a powerful place to land a malicious or compromised release.
A workable pinning approach
- Declare direct dependencies in one file with sensible ranges.
- Generate a lock file that records exact versions of direct and transitive packages, ideally with hashes.
- Commit the lock file and install only from it, using a mode that fails if it does not match.
- Rebuild the image in CI, and ship the image rather than installing at job start.
- Update on a schedule through a pull request that shows what changed, then run tests.
Notebooks specifically
Do not install packages from inside notebook cells in shared or production environments. Build the environment ahead of time and attach notebooks to it. If a notebook needs a new package, add it through the same pull request flow. See the notebook that became production for what happens otherwise.
What to scan
Scan the lock file, not just the top level list, because most packages in a data stack arrive as transitive dependencies. Look for known vulnerabilities and licenses, and decide which findings to act on first. Reachability analysis helps here, since a vulnerable function your code never calls is lower priority than one it does.
Try it yourself
Open the dependency inventory demo and look at how a project's direct and transitive packages are listed, then note how much of the list you did not choose yourself.
Click through it at your own pace, or open it full screen.
Extra habits
- Use a private mirror or proxy where you can, so an upstream removal does not break jobs.
- Record the lock file hash in each run's lineage record.
- Review new packages before adding them: age, maintainers, and whether the name looks like a lookalike of a popular one.
How Safeguard helps
Safeguard provides SBOM and dependency scanning, reachability analysis, and policies and gates that can block pull requests failing your rules. Autonomous remediation with fix pull requests is available but off by default, with every category on Manual. Try the free scan at https://gold.safeguard.sh/scan.
Quick check
Why scan the lock file instead of the requirements list?
Most packages are transitive and only the lock file lists them all with exact versions.
Is installing in a notebook cell fine for production?
No. Build the environment ahead of time so every run uses the same reviewed set.
Last reviewed: 19 October 2026.
