Airflow secrets backends: lookup order and the pitfalls it creates

Airflow looks for a connection or variable in a fixed order: the configured secrets backend first, then environment variables, then the metadata database. If the same key exists in more than one place, the first match wins, which can leave a stale or unintended credential in use without any error.

What a secrets backend is

A secrets backend lets Airflow fetch connections, variables and sensitive configuration options from an external store instead of keeping them only in the metadata database. You enable one by setting its fully qualified class name in the backend key of the secrets section of the configuration, with optional backend keyword arguments. Workers can use a separate backend configured in their own section.

The lookup order

Per the Airflow documentation, the alternative backend is checked first, then environment variables, then the metastore. A worker specific backend takes priority over the general one.

Three practical effects follow.

First, collisions. If a connection named warehouse_prod exists in your external store and also in the metadata database, the external one wins. A teammate who edits the database copy sees no effect and may not know why.

Second, writes. The documentation states that writes update the metastore while reads return the first match found. So saving a connection through the UI may change something that is never read.

Third, visibility. The Airflow UI displays only metastore stored values. Secrets in the external system will not show there, so you must inspect that system directly. People often conclude that no secrets exist because the UI is empty.

Hygiene steps

1. Pick one source of truth per secret type. Use the external backend for credentials and leave the metastore for non sensitive settings.
2. Search the metadata database and environment for duplicates of every key in the backend, and delete the extras.
3. Use a naming convention with prefixes per team or environment so keys cannot collide by accident.
4. Give the scheduler and workers read only access to the external store, scoped to the prefixes they need.
5. Log which source resolved a key during a test run, so you know the real path.

Worker considerations

If workers use a different backend from the scheduler, a connection may resolve differently at parse time and at run time. Test both. A DAG that works in a local run and fails in production often has this root cause.

Testing the order

Create a harmless test variable with different values in each of the three places and read it from a DAG. You should see the backend value. Remove the other two, confirm the job still works, and write down the result in your runbook. It takes ten minutes and prevents a class of confusing incidents.

Where Safeguard fits

Safeguard scans the dependencies of your DAG repositories and can apply policies and gates to pull requests that change them. A free scan is at https://gold.safeguard.sh/scan.

Quick check

Sources

• Airflow documentation: secrets backends

Last reviewed: 19 October 2026.