Not necessarily. AWS documentation says that with the additional Python modules parameter alone, pip resolves your dependencies from PyPI on every job run, so a new upstream release can change what your job installs. Using a Python virtual environment, which resolves once and reuses the result, is the recommended route.
What the parameter does
AWS Glue uses the Python package installer to add modules to a job. The additional Python modules parameter accepts a comma separated list of modules, individual wheel files, and, on Glue 5.0 and above, a zip of bundled wheels or a requirements file. It can also change the version of a module that Glue already ships.
The convenience is real. The risk sits in a plain list such as a package name with no version. That job installs whatever is current at start.
What AWS recommends instead
The Glue docs now say that running a job inside a Python virtual environment is the recommended way to install modules, and that new jobs should use one while existing jobs should consider moving. Their stated reason is exactly the drift problem: a virtual environment resolves dependencies once and reuses the result, so later runs install the same versions.
Two ways of getting a virtual environment are described, a service generated one and one you supply, and they differ in when resolution happens and which parameters you set. A service generated environment inherits modules that Glue already provides. Check the current Glue page for the exact parameters, since they are version specific.
If you stay with a list
Then make it as repeatable as the format allows:
- Give every module an exact version.
- Prefer a requirements file with fully pinned entries over a loose list.
- Include transitive dependencies in the pinned set, since a pinned top level package can still pull a newer sub dependency.
- Add hashes where the format supports it.
- Keep that file in version control and change it through review.
A quick test for drift: run the same job twice, a week apart, and compare the installed package list from the job log. If the lists differ and your code did not change, you have drift.
Why it matters for data jobs
Glue jobs often hold broad data access and run unattended. A package that changes between runs is a way for different code to touch the same data with no review. Even without a malicious package, a new major release can change behaviour and quietly corrupt output.
Also record what actually installed. A job log line is a start, but a stored list per run is better, because it lets you answer questions later: which runs used the version that is now flagged?
See also pin Python dependencies in data projects and can pip require hashes protect an ML environment.
Try it yourself
Click through it at your own pace, or open it full screen.
Look at how a finding appears after a package was already deployed. That is the situation a drifting job puts you in.
Safeguard's SBOM and dependency scanning, with continuous monitoring, helps track that. A free scan is at gold.safeguard.sh/scan.
Quick check
Is a pinned top level module enough?
No. Its own dependencies can still resolve to newer releases. A virtual environment or a fully pinned requirements file closes that gap.
Do I need a different approach for every Glue version?
Parameters and supported formats vary by Glue version, such as zip of wheels and requirements files on 5.0 and above. Check the current AWS page for yours.
Sources
- Using Python libraries with AWS Glue, AWS documentation, fetched 19 October 2026.
Last reviewed: 19 October 2026.
