Railway Environment Variables Breach (October 2026): Rotate Your Secrets

On 21 October 2026 Railway disclosed that an unauthorized third party may have accessed encrypted customer environment variables and obtained the decrypted contents of some of them. Railway says impact may be limited to a subset of users, but it asks every customer to roll all stored secrets, registry credentials included, and review service logs.

This post covers what Railway has said publicly, a step by step rotation runbook for database, cloud, registry and AI provider keys, and the longer term fix: fewer static keys and short lived credentials. Facts come from Railway's public pages or labelled news reporting, never private customer email.

What happened

Railway published "Security Disclosure: Wednesday October 21st" on its blog, written by Jacob Cooper. The post says Railway detected anomalous activity that morning involving systems that store customer environment variables. Its initial investigation indicates an unauthorized party may have accessed encrypted variables, and for some of them obtained the decrypted contents. Those variables can hold API keys and database or registry credentials.

The status page tracked the work under an incident titled "Railway Platform Outage". The times below are as shown on that page (UTC) when we read it on 22 October.

Railway's disclosure lists what it has done so far. It notified upstream providers so they could revoke keys proactively, and it paused all deployments while it investigates. The status page adds the bucket credential rotation, the API token reset and the OAuth session reset. Railway's bucket guide says the tokens it held for customers' GitHub accounts were also invalidated, and GitHub access tokens stored as registry credentials were removed from Railway (they still work on GitHub until you revoke them).

Railway says it will update both pages as the scope narrows and email affected customers.

What Railway has not said yet

As of 22 October, Railway's public pages do not give:

• the number of affected customers or projects;
• how the attacker reached the variable store, or how some values were decrypted;
• which upstream providers it notified;
• the window during which variables were readable.

News coverage from StartupHub.ai, Runtime Wire, LavX News and PANews notes the same gaps. Treat other figures as unconfirmed.

The link to attacks on AI inference providers

Railway's disclosure says it believes the incident is one of a series of recent attacks on platform companies, carried out "with the aim of compromising AI inference providers". That is Railway's assessment. It does not name the other platform companies or the inference providers, and we found no public reporting that does.

What the assessment means in practice is clear from Railway's own guides. Four of the thirteen cover AI services: OpenAI, Anthropic, OpenRouter and Grok. The OpenAI guide warns that a leaked key can drive spend up "without a matching change in your traffic". A stolen inference key is billable compute in someone else's name, and it can reach any data or fine tuned models that key can see. If your services call a model provider, put those keys at the top of your rotation list.

Who is affected

How it works

A platform as a service has to hand your secrets to your code. You type DATABASE_URL into a dashboard, the platform stores it, and at build and deploy time it injects the plaintext value into the container's environment. Encryption at rest protects the stored copy, but the platform must be able to decrypt it to run your app. Any system that can do that decryption is a high value target.

Railway has not said how decryption happened here, and we will not guess. The general point holds for any hosting platform: a variable store holds keys to many other systems, which is why Railway's advice reaches into AWS, GCP, GitHub, Stripe and AI providers.

Reference variables. A service can point at another service's value with syntax like ${{ Postgres.DATABASE_URL }}. When the source changes, every referencing service picks up the new value on its next deploy. Pasted copies do not. Railway's PostgreSQL guide says to find pasted connection strings and replace them with references.

Sealed variables. Railway's variables documentation says a sealed value is passed to builds and deployments but "is never visible in the UI nor can it be retrieved via the API". Sealed values are also left out of railway variables and railway run output. Railway has not said whether sealed variables were in or out of scope of this incident, so rotate them too.

Timing matters too: containers read variables at startup, as Railway's bucket guide notes, so a new value does nothing until the service is redeployed.

What to do now: a rotation runbook

Railway's guides (linked in Sources) cover each provider in detail. This runbook orders the steps for any PaaS user and adds commands.

1. Inventory every variable, without copying values

List variable names per service and environment. Do not paste values into a ticket or a spreadsheet.

railway login
railway link                      # pick the project
railway variables --service api --environment production --kv | cut -d= -f1
railway variables --service worker --environment production --kv | cut -d= -f1

Sealed variables will not appear in that output, so check the Variables tab for each service as well. Then add the names to a table with four columns: variable, issuing provider, owner, and where else the value lives (local .env files, CI, Terraform, a teammate's laptop). Shared variables in Project Settings count once but may feed many services.

2. Rank by blast radius

Rotate in this order unless you know better:

1. Cloud admin and IAM keys (AWS, GCP, Azure).
2. Database credentials, especially with public access enabled.
3. Payment and billing keys (Stripe).
4. AI inference keys (OpenAI, Anthropic, OpenRouter, Grok and others), given Railway's own assessment.
5. Source and registry tokens (GitHub, ghcr.io, Docker Hub).
6. Everything else: email, SMS, analytics, webhooks, signing secrets, session secrets.

3. Rotate at the source, then update Railway, then deploy

Changing the value in Railway does nothing to the old key. The old key stays valid at the provider until you revoke it there.

Databases on Railway. Railway's PostgreSQL guide says to use Database, Config, Connection, Regenerate rather than editing POSTGRES_PASSWORD by hand or running ALTER USER, which can leave the variable and the database out of sync. Regenerate restarts the database. Then redeploy every service that references it:

railway redeploy --service backend
railway connect Postgres

SELECT pid, usename, client_addr, application_name, backend_start, state
FROM pg_stat_activity
WHERE datname = current_database();

SELECT rolname FROM pg_roles;

Terminate sessions you do not recognise and look for roles you did not create. Railway has matching guides for MySQL, MongoDB and Redis.

AWS. Create a new key, swap it in, then deactivate the old one. Railway's AWS guide suggests deactivating before deleting so you can briefly reactivate it if a forgotten service breaks.

aws iam list-access-keys --user-name railway-app
aws iam create-access-key --user-name railway-app
# update AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY in Railway, deploy, verify
aws iam update-access-key --user-name railway-app \
  --access-key-id AKIAOLDKEYEXAMPLE --status Inactive

GCP service account keys.

gcloud iam service-accounts keys list --iam-account=app@PROJECT.iam.gserviceaccount.com
gcloud iam service-accounts keys create new-key.json --iam-account=app@PROJECT.iam.gserviceaccount.com
# update the Railway variable, deploy, verify, then:
gcloud iam service-accounts keys delete OLD_KEY_ID --iam-account=app@PROJECT.iam.gserviceaccount.com
rm new-key.json

AI provider keys. In each provider console, delete the old key and create a new one in the same project or workspace. Railway's OpenAI and Anthropic guides both note that deletion takes effect at once, and that you should also delete any Admin key that may have been stored. Update the variable in Railway, deploy, and confirm the logs show successful requests.

Registry tokens. For ghcr.io, revoke the personal access token on GitHub and issue a new one with only the read:packages scope, as Railway's guide recommends. Enter it under Settings, Source, Registry Credentials for each service and redeploy. Test a pull locally:

echo "$NEW_GHCR_TOKEN" | docker login ghcr.io -u YOUR_GITHUB_USER --password-stdin

Railway's own tokens. Railway has already invalidated all Public API tokens. Create new ones and update RAILWAY_TOKEN or similar in CI.

Storage buckets. Railway rotated these for you. Copy the new values from each bucket's Credentials tab, switch pasted keys to references, redeploy, and update anything off platform that used the old keys.

4. Revoke the old credential

For keys that may be exposed, the safe default is to revoke promptly. If revoking first would cause an outage you cannot accept, run create, update, deploy and revoke as one short change window, not over days. If you see signs of misuse, revoke immediately and accept the downtime.

5. Redeploy once builds return

Builds and redeploys came back in stages, Enterprise first. Until your tier is restored, new values sit staged. Railway's status page notes a workaround for Enterprise users while Redeploy was down (Deploy Latest Commit from the command palette). Keep a list of rotated keys whose consumers still need a redeploy, and tick them off.

6. Look for misuse at each provider

A CloudTrail search from the command line looks like this:

aws cloudtrail lookup-events \
  --lookup-attributes AttributeKey=AccessKeyId,AttributeValue=AKIAOLDKEYEXAMPLE \
  --start-time 2026-10-01T00:00:00Z --region us-east-1

Repeat per Region. Event history keeps management events for 90 days, and Railway's audit log retention depends on your plan, so export both now.

7. Write it down

Record each rotated credential, when, by whom, and what you checked; that is your incident evidence. If you found misuse, preserve logs before deleting anything an attacker created.

Moving to short lived credentials and workload identity

Rotation fixes this week. The structural fix is to store fewer long lived secrets anywhere.

Workload identity federation. AWS STS issues temporary credentials that expire after minutes to hours; AWS says that once they expire they can no longer be used. Google Cloud's Workload Identity Federation exchanges an external identity token for a short lived access token, and Google says this avoids the management burden and risk of service account keys. Both need the workload to present a token from an identity provider they trust. We could not find Railway documentation that describes workload identity tokens for running services, so check Railway's changelog before planning around it.

Use OIDC where you already can. GitHub Actions can request a per job OIDC token and exchange it with AWS, Azure, GCP or HashiCorp Vault, so CI pipelines need no stored cloud keys. Restrict the trust policy to a specific repository and environment through the sub claim. If your deploy pipeline holds cloud keys today, this is the easiest win.

Where a platform cannot federate, shrink each key.

• One key per service and per environment. Never share a production key with staging or preview environments.
• Least privilege. A service that reads one S3 bucket should not hold an admin key.
• Provider side limits. Where your AI or cloud provider offers project scoped keys, spending limits or budget alerts, use them.
• A broker for dynamic secrets. A secrets manager that issues database users with a short time to live means a leaked value expires on its own.
• Keep the most powerful keys off the platform entirely. Admin keys for clouds and AI providers rarely need to live in an app's environment.

What would have helped

Practices any team can use

• A secrets inventory per platform: name, provider, owner, scope and where copies live, so step 1 is a lookup.
• Reference variables and per environment scoping, so one rotation reaches every consumer and a staging leak cannot unlock production.
• OIDC in CI and federation at runtime where supported, as described above.
• Spend alerts on AI and cloud accounts, so abuse shows up as an alert, not an invoice.
• Audit logs exported off platform before an incident.
• A rotation script per provider, run quarterly, so the next notice is a rehearsed change.

Where Safeguard fits

Safeguard would not have prevented or detected this incident. It has no view into Railway's systems, it does not hold your Railway variables or provider keys, and nothing in it would have flagged this intrusion. Here is the narrower part it can play.

Knowing which services likely hold which keys. Safeguard builds SBOMs from repositories, container images and SBOM files (safeguard scan --dir ., --image <name>, --sbom <file>, with JSON results under ./output). The platform shows which projects contain a given component and version. Searching for the openai, @anthropic-ai/sdk, boto3, pg or stripe packages gives you a list of services that probably hold the matching keys, which is a quick cross check on the inventory in step 1. Our post on SBOM incident response explains the method.

Policies and gates. Platform policies can gate builds on what your software contains, which keeps a hurried mass redeploy from shipping a known vulnerable dependency. They do not inspect secrets.

Approved changes through the CLI. The Safeguard CLI's assistant can edit code and run commands on your machine, asking first in the default flow (/permissions offers Ask, Accept edits, Plan, Auto and Strict). You can ask it to list every environment variable name a repository reads, move hard coded connection strings to environment variables, or update an SDK call to read a renamed key, then run your tests, approving each step. The desktop and phone apps can send that work to a computer running the CLI and approve each command. In CI, safeguard exec runs the checks.

Zero-Day Discovery and Gold

These address a different problem. This incident is about credentials stored on a hosting platform, not a vulnerable package in your software.

• Zero-Day Discovery looks for previously unknown flaws before a CVE exists, in an agentic mode and a deterministic scanner mode, across seven package ecosystems. Findings are issued as SGZ advisories before they reach public CVE databases. It has nothing to say about how this intrusion happened, and Railway has not attributed it to a software vulnerability.
• Gold Open Source at gold.safeguard.sh is free and needs no login: searchable CVE intelligence with CVSS, EPSS, KEV status, affected and fixed versions and a recommended safe version, plus a free read API and a CI gate GitHub Action.
• Gold Registry at registry.safeguard.sh is paid: hardened drop in components and container images held to zero exploitable critical or high CVEs, built to SLSA L3+, signed with Sigstore, with evidence pages. An artifact loses its badge if it regresses.

The common habit is speed of rebuild. The teams that recover fastest from any supply chain event, whether a platform breach, a provider outage or a compromised package, know what they run and can redeploy from patched, signed components quickly. For the wider month, see October 2026 security incidents and what would have helped.

Common mistakes

• Editing the variable in Railway and calling it done. The old key is still valid at the provider until you revoke it there.
• Waiting for the email. Railway asks all users to act. The affected list is not final.
• Forgetting the redeploy. Containers read variables at startup. Railway notes health checks only run at deploy time, so they will not catch a service you forgot.
• Hand editing database passwords. Railway's guide warns that editing POSTGRES_PASSWORD or running ALTER USER can leave the variable and the database out of sync. Use Regenerate.
• Missing copies outside Railway. Local .env files, CI variables, Terraform state, BI tools and cron jobs on other hosts often hold the same values.
• Skipping sealed variables. They do not show in CLI output, so they drop off scripted inventories.
• Rotating only production. Preview and staging environments often hold the same provider keys.

FAQ

Was my Railway project affected?

Railway says its initial analysis suggests the impact may be limited to a subset of users, and it will email affected customers as it confirms them. It still asks every user to roll environment variables and registry credentials as a precaution. Act now either way.

What kind of data was exposed?

Railway says an unauthorized party may have accessed encrypted environment variables and obtained the decrypted contents of some. Those variables can include API keys and database or registry credentials. Railway has not said that application data in your databases was accessed.

Did Railway rotate anything for me?

Yes, some things. According to its status page and guides, Railway rotated all storage bucket credentials, invalidated all Railway API tokens, invalidated OAuth sessions and the GitHub tokens it held, and removed stored GitHub registry tokens. Everything else you store in variables is yours to rotate at the source.

Why does Railway mention AI inference providers?

Railway says it believes this was part of a series of attacks on platform companies aimed at compromising AI inference providers. It has not named those companies or providers. Treat AI provider keys as a priority and check usage and billing for unfamiliar models or spend.

Are sealed variables safe?

Railway's documentation says sealed values are hidden from the UI and API but are still passed to builds and deployments. Railway has not said whether they were affected here. Rotate them too.

How do I redeploy if builds are paused for my tier?

You cannot until Railway restores builds for your tier; they are returning in stages, Enterprise first. Rotate at the source and stage the new values now, then redeploy as soon as builds return. Check the status page for your tier.

Fix it without waiting for upstream

In incidents like this one, most of the risk sits in two gaps. The first is the time between a fix being published and that fix actually running in your production builds. The second, and worse, is the time before any fix exists at all, while a maintainer works out a patch and a release schedule. Waiting through both gaps is a habit, not a requirement. Safeguard gives teams three ways to close them.

1. Autonomous remediation: pull requests, not tickets

When a fixed version exists, Safeguard's Griffin models open a tested, policy gated pull request that moves the affected dependency to the safe version. You can run it on one repository or in bulk across every project that carries the vulnerable version. Remediation covers eight categories, including dependencies, licenses, security posture and findings. Reachability analysis ranks the vulnerable code paths your application actually calls first, so the first pull requests you merge are the ones that matter most. Autonomous remediation ships switched off, with every category set to Manual, so your team decides how much runs on its own and what still needs a human approval.

2. Gold Open Source: the safe version in seconds, free

gold.safeguard.sh is free and needs no login. Search any CVE or package to see affected and fixed versions, CVSS, EPSS, CISA KEV status and a recommended safe version, or use the "Am I affected?" checker. The same data is available through a free read API, README status badges and RSS threat feeds. The safeguard-sh/gold-scan-action GitHub Action can fail a build on a known exploited vulnerability, on any known vulnerability, or never, so a version you have just removed cannot quietly come back in next week's merge.

3. Gold Registry: start from components that are already fixed

registry.safeguard.sh provides hardened, drop in replacements for upstream packages and container images. Every Gold artifact is held continuously to zero exploitable critical or high CVEs, zero malware and a completed license check, with SLSA L3 or higher provenance, a Sigstore signature and a public evidence page you can hand to an auditor. An artifact loses its Gold badge if it regresses. Teams can point their own package directory at the registry so hardened equivalents resolve without changing how developers install anything.

When upstream has no fix yet, or will not ship one, Safeguard first works to get the fix accepted upstream. If the issue is severe or upstream will not move, Safeguard builds a fork, fixes the vulnerability there and publishes a hardened Gold build, so you are not stuck on a maintainer's release schedule. You can also request a hardened, compatibility tested build of a specific package. This fork and patch flow is in early access today, so ask which packages it covers before you plan around it.

One honest limit: when the vulnerable code lives in a vendor appliance, a hosted service or an operating system runtime you do not build yourself, the fix still has to come from that vendor or distribution. The steps above apply to the open source dependencies and container images you build and ship, which is where most teams carry the largest and least tracked share of their exposure.

See how Safeguard would have helped

None of these demos show the Railway incident. Each is a real Safeguard session, and the text says how the same flow applies to a rotation like this one.

The first demo is a CLI session that fixes a lodash CVE: the assistant reads the finding, asks before each command, edits package.json, runs the install and tests with your approval, and pushes nothing. For Railway, you would open the same session in a service's repository and ask it to list every environment variable the code reads and replace any hard coded connection string with process.env.DATABASE_URL, then run the tests. It asks before each edit and command.

Click through it at your own pace, or open it full screen.

The second demo is a desktop app session. The CLI session shows in the conversation list, you ask for work on your computer, pick the folder, then Approve or Deny each command and file edit, and it ends with an upgrade done and tests passing. During a rotation, a platform engineer can point the same flow at the infrastructure repository that holds CI variables and IAM policy, and approve each change to swap a stored cloud key for an OIDC role.

Click through it at your own pace, or open it full screen.

The third demo is a real Android session: work is sent to a computer running the CLI, and the command is approved on the phone with Allow once, Allow for turn or Deny. When builds come back for your tier late at night, the on call engineer can approve the redeploy and test commands from a phone, or deny one that looks wrong.

Click through it at your own pace, or open it full screen.

To try it on your own repositories, download Safeguard and follow the CLI documentation. For a broader playbook, see our incident response playbook for a compromised dependency.

Quick check

Sources

• Railway: Security Disclosure, Wednesday October 21st
• Railway status: Railway Platform Outage incident QH8L8B3E
• Railway status page
• Railway blog index
• Railway: Rotate PostgreSQL database credentials
• Railway: Check for anomalous AWS access
• Railway: Check for unexpected OpenAI usage
• Railway: Check for unexpected Anthropic usage
• Railway: Redeploy buckets and reconnect GitHub after the credential reset
• Railway docs: Variables
• AWS IAM: Temporary security credentials
• Google Cloud: Workload Identity Federation
• GitHub Docs: OpenID Connect in GitHub Actions
• Runtime Wire: Railway pauses deployments after possible exposure of environment variables
• StartupHub.ai: Railway tells all users to rotate secrets after breach
• LavX News: Railway discloses breach of encrypted environment variables
• PANews: Railway security incident report

Last reviewed: 22 October 2026. This is a developing story; check the linked advisories for the latest status.