Never log credentials, connection strings, tokens, raw personal data, or a full dump of your environment variables to an experiment tracker. Trackers are built to keep everything and show it to many people, so anything logged tends to live long and spread widely. Log identifiers and hashes instead of secrets.
Why trackers are a leak path
An experiment tracker records parameters, metrics, tags, code versions, artifacts and sometimes command lines. Teams read it, dashboards render it, and exports copy it. Old runs are rarely cleaned up. A secret logged once can sit in a run from two years ago that nobody remembers.
The tool is not at fault. Logging is automatic, and people log whole config objects without reading them.
Fields that commonly leak
| Where it hides | What leaks |
|---|---|
| Logged parameters from a config object | Database URLs with passwords, API keys in nested settings |
| Command line captured with the run | Tokens passed as arguments |
| Environment snapshot | Cloud keys, tokens, internal hostnames |
| Artifacts | Notebooks with outputs, .env files, credentials files |
| Tags and notes | Pasted error messages with keys |
| Sample rows or predictions | Real customer records |
Habits that prevent it
- Allow list, do not dump. Log named parameters you chose, not an entire config or environment.
- Keep secrets out of arguments. Read secrets from a secret manager or environment inside the job, not from the command line.
- Mask on the way in. Add a small helper that drops keys containing words like token, secret, key and password before logging.
- Scan artifacts. Run a secret scan on any file before it is uploaded as an artifact.
- Sanitize samples. Log synthetic or masked examples, never raw customer rows.
- Review a run once. Open a recent run and read every field the way an outsider would.
Control who can read it
Even clean runs hold valuable information, such as which data and settings produce your best models. Use the tracker's access control. For example, MLflow can add authentication with read, use, edit and manage levels, described in how to turn on authentication for an MLflow tracking server. Give training jobs the lowest level that lets them write.
If a secret was logged
- Rotate the credential now. Deleting the run is not enough, since copies may exist in backups and exports.
- Delete or redact the run and artifact.
- Check access logs on the system the secret opened, covering the period since the run.
- Add the missing mask or scan so it cannot repeat.
- Tell the owner of the system, even if you see no misuse.
Make it a gate
A pull request check that flags logging of whole config objects, plus a secret scan on the repository, stops many of these before they run. Safeguard scanning and policy gates can support that kind of check, and you can try a free scan at gold.safeguard.sh/scan.
Quick check
Is it fine to log a hash of a secret?
Avoid it for low entropy secrets, since a short or guessable value can be recovered from its hash. Log a label such as the secret's name instead.
Does deleting the run remove the exposure?
Not fully. Rotate the credential regardless.
Last reviewed: 19 October 2026.
