Blue-Green Deployment with Feature Flags: Cut Over Safely and Roll Back Fast
TL;DR
Blue-green deployment replaces an application version by running two production-capable environments in parallel. Blue serves the current release while green receives the candidate release and validation. When green is ready, the traffic route switches; blue remains available during a defined rollback window.
That infrastructure switch is fast, but it is coarse. It cannot expose a feature to selected users, measure a variation's product impact, or turn off one behavior inside an otherwise healthy release. Feature flags add that application-level control. The safest design uses both mechanisms deliberately: blue-green for environment replacement, Optimizely flags for feature release and measurement.
Blue-green deployment in one diagram
The pattern separates deployment from traffic cutover. The exact routing layer may be a load balancer, Kubernetes ingress, service mesh, reverse proxy, or DNS record, but the control boundary is the same.
flowchart LR
U[Production requests] --> R{Traffic router}
R -->|Active route| B[Blue: current application]
R -.->|Candidate route| G[Green: candidate application]
B --> D[(Compatible data stores)]
G --> D
B --> O[Optimizely decision layer]
G --> O
O --> OFF[Feature off / legacy path]
O --> ON[Feature on / new path]Blue serves production while green is prepared
Blue is the known production environment. Green is provisioned with the candidate application, production-equivalent capacity, configuration, networking, secrets, observability, and dependencies. Before customer traffic moves, green should pass:
Startup, readiness, and dependency health checks.
Smoke and integration tests.
Synthetic transactions through critical journeys.
Capacity and saturation checks.
Logging, tracing, metrics, and alert validation.
Security and network-policy checks.
AWS describes blue-green as shifting traffic between two identical environments running different application versions, with blue serving production while green is staged and tested. Its blue-green introduction describes near-zero-downtime releases rather than a guarantee of zero impact.
Cutover switches the traffic route
Cutover changes the infrastructure route from blue to green. That operation is distinct from an application flag decision:
The route selects an environment or service revision.
A feature flag selects behavior inside the chosen application.
Keep those controls separate in dashboards, permissions, runbooks, and audit logs. A flag cannot repair a candidate that fails to start, cannot reach its database, or exhausts memory before evaluating the flag.
Some platforms support an all-at-once cutover; others can shift a small traffic fraction as a validation step. Sending a small production cohort to green is canary analysis layered onto a blue-green topology. Blue-green itself does not imply gradual exposure.
Blue stays warm through the bake period
After the route switches, leave blue running and ready. The bake period is the interval when green serves production while both revisions remain available. Monitor technical and business guardrails, and route traffic back to blue if predefined thresholds are breached.
Amazon ECS defines bake time as the period after production traffic shifts while blue and green run simultaneously. Its blue-green deployment guide notes that this temporary overlap may double resource usage and that blue remains available for rollback until bake time expires.
Retire blue only after the rollback window closes, delayed outcomes have arrived, and the data contract remains compatible.
What blue-green solves—and what it does not
Blue-green reduces risk around application replacement. It does not make every stateful change reversible and should not be advertised as zero risk.
Strong fit: low-downtime application replacement
Use blue-green when a release benefits from:
Validating a complete candidate environment before cutover.
Replacing a whole service, fleet, or application version.
Keeping the prior application revision immediately available.
Isolating candidate failures from the active environment.
A predictable, automated cutover sequence.
The pattern is especially useful when in-place upgrades are slow or difficult to undo. It can minimize downtime to the route-switch interval, assuming the routing layer, application, data stores, and clients tolerate the transition.
Costs and operational complexity
During deployment, the team temporarily duplicates:
Compute and autoscaling capacity.
Load-balancer targets, ingress, and network policies.
Secrets and environment configuration.
Caches, queues, service discovery, and certificates.
Dashboards, alerts, logs, and traces.
Deployment automation and access controls.
Environment drift undermines the rehearsal. If green uses different secrets, feature-flag defaults, network paths, instance sizes, or observability, passing tests in green does not predict the actual cutover. Treat parity as a continuously checked property, not a checklist completed once.
The database is the hard boundary
Blue and green usually share state or process records written by both versions. A route rollback fails when blue cannot read data written by green or expects a column that a migration removed.
Use expand/contract migrations:
Expand: add new tables, columns, indexes, or APIs while preserving old ones.
Deploy code that can read and write compatible representations.
Backfill and verify data.
Cut traffic and complete the bake period.
Contract: remove old schema or behavior only after blue retirement and the rollback window.
Delay destructive migrations. Make message schemas forward- and backward-compatible. If green writes a new enum value, serialized session, or queue message that blue cannot parse, the infrastructure route may roll back while the application remains broken.
Blue-green vs canary, rolling, and feature-flag releases
These techniques control different boundaries and can be combined.
Strategy | Switching unit | Main advantage | Main tradeoff |
|---|---|---|---|
Blue-green | Whole environment or service revision | Validate candidate, then route back quickly | Duplicate capacity and state compatibility |
Canary | Small production cohort or traffic share | Limit initial blast radius | Mixed-version operation and gradual analysis |
Rolling | Instances within one fleet | Lower temporary capacity | Old and new coexist during replacement |
Feature flag | Behavior inside deployed code | Fine-grained targeting, kill switch, measurement | Cannot repair infrastructure or schema failures |
Blue-green switches environments
Blue-green proves that the candidate environment can run the application before it becomes the default route. The key decision is whether green is ready to replace blue.
Canary shifts a small production cohort first
Canary deployment exposes a small cohort, observes it, then ramps gradually. It reduces the initial blast radius but requires both versions to serve production concurrently and demands stable cohorting or traffic analysis.
The full workflow belongs in Canary Deployment: Progressive Delivery with Feature Flags. A team can canary green before a full blue-green cutover, but should name both mechanisms.
Rolling deployment replaces instances incrementally
A rolling deployment drains and replaces instances within one fleet. It uses less duplicate capacity than maintaining two complete environments, but old and new instances coexist. Requests, background jobs, and internal calls must tolerate mixed versions throughout the rollout.
Feature flags switch behavior inside deployed code
Flags separate code deployment from feature release. They can provide:
Internal allowlists and beta audiences.
Percentage exposure.
Remote kill switches.
A/B tests and product-impact measurement.
A default legacy or off path.
Flags do not roll back broken startup logic, infrastructure-as-code, network policy, incompatible dependencies, or destructive database migrations. Keep route and flag rollback available for different failure classes.
Layer feature flags onto both environments
The checkout-service v2 example uses blue-green to replace the runtime and an Optimizely flag named checkout_recommendations_v2 to release a recommendation module.
Deploy dormant code to green
Implement both branches and test the default path:
const user = optimizelyClient.createUserContext(accountId, {
region: checkout.region,
release_channel: checkout.releaseChannel,
})
const decision = user.decide('checkout_recommendations_v2')
if (decision.enabled && decision.variationKey === 'recommendations_on') {
return renderCheckoutWithRecommendations(decision.variables)
}
return renderLegacyCheckout()
The code contains a safe legacy path even when Optimizely is unavailable or the flag is off. Use a stable ID that matches the intended release unit; account-level behavior should not be randomized with an unstable request or session ID.
Optimizely Feature Experimentation scopes rulesets and results by environment. The official core concepts describe separate staging and production environments for internal validation and production delivery. Run and validate the flag in Development while keeping its Production ruleset paused.
Cut infrastructure over before releasing behavior
Switch production traffic to green while checkout_recommendations_v2 remains off. This isolates two questions:
Is the green runtime healthy under production load?
Does the recommendation feature improve the product without harming guardrails?
If both change simultaneously, an error-rate increase is harder to attribute. With the feature off, the team can validate runtime, dependencies, sessions, queues, and base checkout behavior before introducing recommendation traffic.
Release by audience or percentage
After green is healthy, choose the Optimizely rule type from the decision:
Need | Rule type |
|---|---|
Simple toggle or audience release without analytics | Targeted Delivery |
Progressive release of one known variation with metrics | Feature Rollout |
Compare control and treatment to determine a winner | A/B test |
Optimizely's Feature Rollouts documentation currently labels the rule type beta. It recommends 10% as a starting allocation and describes a one-variation progressive release with impressions, metrics, confidence intervals, and flag-evaluation insights. Targeted Deliveries do not provide the same analytics or consume impressions; use them when measurement is unnecessary.
A practical release sequence is:
Internal allowlist in Development.
Internal allowlist in Production.
Beta audience.
Feature Rollout at 10%.
Increase to 25%, 50%, and 100% when guardrails pass.
Conclude, simplify rules, and schedule flag cleanup.
Because Feature Rollouts are beta and require supported SDK versions and plan access, confirm eligibility before making the runbook depend on them. If unavailable, use an A/B test when comparative measurement is required or a Targeted Delivery for an unmeasured release.
Keep two rollback levers explicit
Use the smallest lever that addresses the failure:
Symptom | First rollback lever | Why |
|---|---|---|
Recommendation lowers checkout conversion | Pause flag or rollout | Green runtime remains healthy |
Recommendation API increases latency | Pause flag; then diagnose dependency | Removes the behavior without moving the fleet |
Green has startup, memory, network, or base-runtime failures | Route back to blue | Failure exists outside the flag branch |
Green writes data blue cannot read | Stop cutover and execute data recovery plan | Route rollback alone is unsafe |
Rehearse both controls. Name the person authorized to use each and define thresholds in advance.
Execute a production cutover
A safe cutover is a sequence with explicit gates, not one “deploy” button.
flowchart TD
A[Provision green] --> B[Verify parity and compatibility]
B --> C[Run smoke, synthetic, and capacity tests]
C --> D[Switch production route with feature off]
D --> E{Runtime guardrails pass?}
E -->|No| RB[Route rollback to blue]
E -->|Yes| F[Bake green]
F --> G[Enable feature for internal audience]
G --> H[Roll out 10%, 25%, 50%, 100%]
H --> I{Product guardrails pass?}
I -->|No| RF[Flag rollback inside green]
I -->|Yes| J[Close rollback window and retire blue]Preflight environment parity
Compare blue and green mechanically:
Runtime and dependency versions.
Configuration keys, secrets, certificates, and permissions.
Network policies, DNS, service discovery, and firewall rules.
Autoscaling limits and full-load capacity.
Cache namespaces and warm-up behavior.
Queue subscriptions, consumer groups, and dead-letter handling.
Scheduled jobs and singleton ownership.
Dashboards, logs, traces, alerts, and on-call routing.
Optimizely SDK keys, environment, flag defaults, and datafile refresh.
Fail the preflight when an unexplained difference remains.
Validate green with non-customer traffic
Use health checks and synthetic transactions through the real routing and dependency paths. Options include:
A test listener or internal hostname.
Authenticated internal headers stripped at the edge.
Synthetic checkout, sign-in, and refund journeys.
Read-only shadow requests where privacy and side effects are controlled.
Capacity tests against production-like load.
Do not let synthetic writes contaminate production analytics, inventory, payments, or experiment results.
Switch and observe
Monitor technical guardrails:
Error and timeout rates.
p50, p95, and p99 latency.
CPU, memory, connection pools, and saturation.
Queue depth and processing lag.
Cache hit rate and dependency errors.
Monitor business guardrails:
Checkout completion and payment success.
Sign-in and account-creation success.
Order value, refund rate, and support contacts.
Critical conversion-event volume.
Use the guardrail metrics guide to define action thresholds, not merely dashboard panels.
Bake, conclude, and clean up
Keep blue available for a predefined duration that covers traffic cycles, delayed failures, and data compatibility. When criteria pass:
Conclude the rollout or experiment.
Remove temporary allowlists and redundant rules.
Retire blue and duplicate infrastructure.
Complete contract migrations.
Archive short-lived flags after code cleanup.
Record the release outcome and rollback evidence.
Use Feature Flag Best Practices to prevent the rollback flag from becoming permanent conditional complexity.
Failure modes and recovery
The most damaging failures cross the boundary between environments and shared state.
Background jobs run twice
Blue and green may both start schedulers, queue consumers, or batch workers. Use leader election, separate consumer groups, explicit activation switches, or idempotent processing. Test ownership transfer before traffic cutover.
Sessions or caches do not survive cutover
In-memory sessions disappear when traffic moves. Externalize session state or verify compatible serialization and shared encryption keys. Namespace version-specific cache entries and prevent green from writing formats blue cannot read during rollback.
The schema cannot roll back
If green deletes a column, changes an enum, or rewrites stored data incompatibly, blue may start but fail on real requests. Expand first, keep dual-compatible code through bake time, and contract only after the rollback boundary closes.
Flag configuration differs across services
Multiple services evaluating the same feature need:
Consistent flag keys and variations.
The same environment and compatible datafile revision.
Stable IDs and identical attribute types.
Defined SDK initialization and refresh behavior.
A safe fallback when no decision is available.
Optimizely change history records flag, ruleset, audience, traffic, variation, and allowlist changes. Use that record during incident review, but also version the release plan and ownership in the team's operational system.
Blue-green deployment checklist
Green can carry full production load.
Configuration, secrets, networking, and observability match blue.
Database and message changes are backward- and forward-compatible.
Only one environment owns singleton schedulers and consumers.
Sessions, caches, and serialization survive both cutover directions.
Technical and business rollback thresholds are written.
Feature-off behavior works in blue and green.
Route rollback and flag rollback are separately rehearsed.
Optimizely environment, IDs, attributes, and fallbacks are verified.
Bake window, blue-retirement owner, and cleanup tasks are assigned.
Run the Optimizely Experiment QA Checklist before measured exposure, and use the Feature Flag Naming Generator to create consistent operational keys.
Frequently asked questions
What is blue-green deployment?
Blue-green deployment runs the current and candidate application versions in separate production-capable environments. Production traffic switches to the validated candidate while the previous environment remains available during a rollback window.
Is blue-green deployment zero downtime?
It can provide near-zero or zero application downtime when routing, dependencies, state, and clients tolerate cutover. It is not a universal guarantee: DNS behavior, long-lived connections, incompatible state, or routing errors can still create impact.
What is the difference between blue-green and canary deployment?
Blue-green primarily switches between complete environments. Canary deployment progressively exposes a small cohort or traffic share before ramping. A team can canary traffic to green, but the mechanisms and success criteria should remain explicit.
Do feature flags replace blue-green rollback?
No. A flag can disable behavior inside a healthy application. It cannot restore broken infrastructure, fix startup failure, reverse an incompatible schema, or repair a dependency needed before flag evaluation. Keep route-level rollback.
When should Optimizely Feature Rollouts be used?
Use a Feature Rollout when one known variation should be released progressively with impression and metric tracking. The capability is currently beta; confirm plan and SDK support. Use Targeted Delivery for unmeasured toggles and an A/B test when comparing variations to determine a winner.