ETL secrets usually leak in five places: source code, config files, logs and error messages, notebook outputs, and shared orchestrator settings. Fix them by keeping credentials in a secret store, injecting them at run time, scoping each to one job, and rotating them on a schedule you actually keep.
Why pipelines are a rich target
A pipeline touches many systems: a source database, a SaaS API, object storage, a warehouse. Each needs a credential, and the jobs often run unattended for years. The credentials tend to be powerful because someone wanted the job to just work.
Leak point 1: source code and history
A connection string pasted into a script gets committed. Deleting it later does not remove it from history. Scan the repository including history, rotate anything found, and add a pre-commit or CI check so it does not come back.
Leak point 2: config files and images
A profile file with a password, baked into a container image, travels with the image. Keep config files free of values and read them from the environment or a secret manager when the job starts.
Leak point 3: logs and errors
Verbose logging prints connection details, and failed jobs print the whole configuration. Some tools scrub secrets for you if you follow a convention. dbt, for example, scrubs environment variables prefixed with DBT_ENV_SECRET_ from logs and error messages and obfuscates them in its UI. That protection depends on using the prefix, so name variables accordingly. Test it: trigger a failure on purpose and read the log.
Leak point 4: notebooks
Notebook outputs get saved into the file. A cell that prints a token stores the token. Databricks redacts values retrieved through its secrets utility when displayed, but only for literal values, so a transformed secret is not covered. Do not rely on redaction as the only line of defence. See also how the notebook becomes production.
Leak point 5: orchestrator settings
Airflow can read connections and variables from an external secrets backend. Without one, they sit in the metadata database, readable by anyone with enough access to it. Every shared orchestrator setting is a place secrets pile up.
A fix list in priority order
- Inventory every credential a pipeline uses. Start from the code and the orchestrator, not from memory.
- Move each into a secret store and inject at run time.
- Give every job its own credential with read only access where it only reads.
- Prefer short lived credentials from your cloud identity system over long lived keys where your tools support it.
- Rotate on a calendar and after any staff change.
- Add scanning for secrets in CI.
Detecting a leak you missed
Alert on use of a credential from an unexpected network or at an odd hour, and on spikes in rows read. A key that is never used from its expected place is cheap to detect and expensive to ignore.
Where Safeguard fits
Safeguard's policies, gates and guardrails can block pull requests that fail your rules, and dependency and repository scanning covers the code that holds pipelines. Run a free scan at https://gold.safeguard.sh/scan.
Quick check
Does deleting a committed password fix the leak?
No. It remains in history and in any clone. Rotate the credential, then remove it.
Is notebook redaction enough protection?
No. Databricks documents that redaction covers literal values only, so a transformed secret can still show up in output.
Sources
- dbt documentation: environment variables
- Databricks documentation: secrets
- Airflow documentation: secrets backends
Last reviewed: 19 October 2026.
