Environment mismatch: production points to staging: pre-release and first 30 minutes checklists

A production service reads a staging endpoint, bucket or database after a config change.

The pre-release checklist maps each item to a test class from the incident page. The first-30-minutes checklist covers detection, containment, the rollback decision and customer communication.

8 pre-release items · 8 first-30-minutes items · verified 2026-10-21

Pre-release checklist

PR-01Before promotion, confirm that every release in the list returned by the progressive deployment API belongs to the production account. config
PR-02Compare the account ID of the script the production route will reference with the production account ID and block promotion on a mismatch. config
PR-03Search the rendered production config for staging hostnames, bucket names and database names and fail the deploy on any match. config
PR-04Confirm that every production monitoring metric filters on an environment label that excludes test environments. config
PR-05When a new test environment is provisioned, confirm its data reports under its own environment label before production dashboards read the shared metric. config
PR-06Configure the first canary step to alert when routed traffic reaches a destination other than the expected production script. canary
PR-07Configure the canary step to halt the rollout on any rise in HTTP 401 responses from the key-value store. canary
PR-08Rehearse the switch of the production route back to the last known good build, including the break-glass path, before the rollout starts. chaos

First 30 minutes

FM-01Compare the start of the HTTP 401 and 500 spike with the time the last release was promoted to production.
FM-02Read the account or environment of the script the production route references and compare it with the production account.
FM-03If the production route references a staging artifact, switch the route back to the last known good build, using break-glass access if normal tooling depends on the failing service.
FM-04Pause every remaining progressive rollout step until the release list is filtered to production releases only.
FM-05Before declaring an outage, confirm the failing requests come from production hosts and not from a test environment feeding the same metric.
FM-06Publish a status update that lists the dependent services returning errors and the time the route was switched back.
FM-07If the failures trace to a test environment, update the status page to state that customers saw no impact.
FM-08Close the incident only after the key-value store 401 rate returns to its pre-release baseline and the production metric reports production data only.

Download the checklist

Pre-release checklist (Markdown)1.8 KB · v1.0.0
First 30 minutes checklist (Markdown)1.4 KB · v1.0.0

Only in this checklist

Every test class in this checklist also appears in at least one sibling checklist in the group; the item texts differ: Before promotion, confirm that every release in the list returned by the progressive deployment API belongs to the production account.

Questions

What should be checked before a release to prevent Environment mismatch: production points to staging?

8 pre-release items, PR-01 to PR-08. PR-01: Before promotion, confirm that every release in the list returned by the progressive deployment API belongs to the production account.

Is there a rollback plan for Environment mismatch: production points to staging?

The first-30-minutes checklist has 8 items. FM-01: Compare the start of the HTTP 401 and 500 spike with the time the last release was promoted to production.

What should a team do in the first 30 minutes after Environment mismatch: production points to staging is detected?

The first-30-minutes checklist has 8 items. FM-01: Compare the start of the HTTP 401 and 500 spike with the time the last release was promoted to production.

Which configuration checks apply to Environment mismatch: production points to staging?

PR-01: Before promotion, confirm that every release in the list returned by the progressive deployment API belongs to the production account. PR-02: Compare the account ID of the script the production route will reference with the production account ID and block promotion on a mismatch. PR-03: Search the rendered production config for staging hostnames, bucket names and database names and fail the deploy on any match. PR-04: Confirm that every production monitoring metric filters on an environment label that excludes test environments. PR-05: When a new test environment is provisioned, confirm its data reports under its own environment label before production dashboards read the shared metric.

Is a Environment mismatch: production points to staging checklist still needed if deployment is automated?

Yes. Automation runs the steps; the checklist names what the tests must cover: configuration and environment tests, canary and progressive delivery checks and chaos and resilience tests.

Which pipeline stages does the Environment mismatch: production points to staging checklist cover?

The pre-release items map to the pre-deploy and post-deploy stages.