Feature Flag System Design: Evaluation, Targeting and Propagation

A feature flag looks like an if statement, and the first version of every flag system is one: a config file, a map of names to booleans, a check in the code. The system-design question is everything that map does not answer. Where is the flag defined, and who can change it? How does a change reach two hundred servers and a million phones? How does the same user get the same answer on every request, at a cost of microseconds, even when the network is down? And what happens — to the user, the on-call engineer and the revenue line — when any one of those parts fails? This page is about that architecture: the components, the decisions inside each, and the failure modes. It is not a guide to using flags well; that is feature flag best practices, and the delivery patterns built on top of flags are covered in canary deployment and blue-green deployment.

The four components of a flag system

Every flag system that has survived contact with production has the same four parts, whether it was built in-house or bought.

flowchart LR
  CP[Control plane<br/>flag definitions, rules, environments, audit log] --> DIST[Distribution<br/>config snapshot on a CDN, polled or pushed]
  DIST --> EVAL[Evaluation engine<br/>SDK in the app: targeting + deterministic bucketing]
  EVAL --> APP[Application code<br/>if decision.enabled]
  EVAL --> TEL[Decision telemetry<br/>impression and conversion events]
  TEL --> RES[Results and monitoring]

1. The control plane is where a flag is defined: its key, its variations, the rules that decide who gets which, and the environments those rules apply in. It is a database with an API and a permissions model.

2. Distribution is how the current state of every flag reaches the running code. The design choice here — pull or push, how often, how big — sets the propagation delay and the blast radius of a bad change.

3. The evaluation engine turns a flag key, a user identifier and some attributes into a decision. It runs inside the application, in the request path, and it has to be fast, deterministic and safe when its inputs are missing.

4. Decision telemetry records what the engine decided, so that a rollout can be monitored and an experiment can be measured. Without it a flag system can ship a change but cannot tell you what it did.

Each of these has one design decision that dominates the rest, and the next four sections take them in turn.

The control plane: how a flag is defined

The data model is small. A flag has a key, unique within a project. It has one or more variations — at minimum on and off, often several named variants, each optionally carrying variables (a price, a copy string, a model name) so the code reads configuration rather than branching on a variation name. It has an ordered list of rules, each with an audience condition and a traffic allocation: "for users in Germany, serve variant B to 20%, otherwise fall through to the next rule". And it has all of that once per environment, so that staging and production can hold different rules for the same key with separate permissions on each.

The decision that dominates the control plane is what a change is. A flag system in which an edit takes effect the instant someone clicks Save has no audit trail, no review, and no way to answer "what was the state of this flag at 14:32 when checkout errors spiked?" The alternative — every change is a versioned, attributed revision of the environment's ruleset — costs little and is what lets a rollout be paused with confidence and a bad change be reverted to a known state rather than to someone's memory of it.

Two further constraints belong here rather than in code review. Flag keys must be unique and immutable, because the key is what the application code holds; renaming a live key is a production change. And the default served when a flag cannot be evaluated is a property of the flag definition, not of the calling code, or the two will drift.

Distribution: how a change reaches the running code

There are two ways for an evaluation engine to learn the current rules. It can ask on every decision — a network call to a flag service — or it can hold a local copy of the whole ruleset and refresh it periodically.

Remote evaluation is simpler to reason about and is how most first versions work. It is also a synchronous network dependency in the request path of every feature the flag guards, and its cost is paid on every request: latency, a failure mode, and a flag service that must be provisioned for the application's peak traffic rather than its own.

Local evaluation moves the whole ruleset into the process. The control plane publishes a snapshot — in Optimizely Feature Experimentation this is the datafile, a JSON document holding every flag, variation, rule and audience for one environment — to a CDN. Each SDK instance downloads it at startup, evaluates every decision against it in memory, and refreshes it on a schedule, by webhook, or both. A decision is then a function call, and a network outage after startup costs nothing: the SDK keeps using the last snapshot it fetched.

The decision that dominates distribution is the staleness budget: how long after a change in the control plane may a running instance still serve the old rules? Polling every five minutes bounds it at five minutes. A webhook that tells each instance to refetch cuts it to seconds, at the cost of a push channel to maintain. Streaming connections cut it further and cost more again. The right budget is a product question — a kill switch needs seconds, a copy change tolerates minutes — and the honest answer is that no local-evaluation system gives every instance the new rules at the same instant. Two servers will disagree for the length of the budget, and the application has to tolerate that.

The snapshot's size is the second constraint. Every flag ever defined in an environment is in the datafile, so a project that never retires flags ships a growing document to every client on every refresh. Retiring flags — finding the unused ones and deleting them — is not only hygiene; it is what keeps distribution cheap.

Evaluation: how a decision is made

Given a snapshot, a flag key, a user identifier and a set of attributes, the engine walks the flag's rules in order. For each rule it checks the audience condition against the attributes; the first rule whose audience matches is the one that decides. Within that rule the traffic allocation says what fraction of matching users receive each variation, and the engine has to pick one — consistently.

Consistency is the design decision that dominates evaluation. The same user must get the same variation on every request, on every server, in every SDK, without any server having to remember anything. The standard answer is deterministic bucketing: hash the user identifier together with the rule or experiment identifier, map the hash to a bucket in a fixed range, and read the variation off the allocation table for that bucket. No storage, no coordination, and the same input gives the same output everywhere. Optimizely's SDKs use MurmurHash3 over the user ID and experiment ID to produce a bucket between 0 and 9,999, and the datafile expresses every allocation in those units.

Two consequences follow from hashing on the pair rather than on the user alone. Different flags bucket the same user independently, so being in variation B of one experiment says nothing about their assignment in another — which is what keeps concurrent experiments from confounding each other. And the identifier chosen is load-bearing: a user who is bucketed on an anonymous cookie before login and on an account ID after it will change variations at the moment they log in. Systems that need continuity across that boundary pass an explicit bucketing ID separate from the user ID.

The engine also decides what to do with nothing. A missing user ID, an unknown flag key, an attribute the rule expects but the caller did not supply, a datafile that has not loaded yet — each must produce a defined answer, and that answer should be the flag's declared default, off, rather than an exception or a guess. In a well-designed system a decision is never an error.

Deterministic bucketing in twenty lines

The core of an evaluation engine is small enough to write out.

import hashlib

BUCKET_COUNT = 10_000

def bucket(user_id: str, experiment_id: str) -> int:
    """Map (user, experiment) to a stable bucket in [0, BUCKET_COUNT)."""
    key = f"{user_id}{experiment_id}".encode("utf-8")
    digest = hashlib.sha256(key).digest()
    return int.from_bytes(digest[:4], "big") % BUCKET_COUNT

def choose_variation(user_id: str, experiment_id: str, allocation: list[tuple[str, int]]) -> str | None:
    """allocation: ordered (variation_key, end_of_range) pairs in bucket units.

    [("control", 5000), ("treatment", 10000)] is a 50/50 split;
    [("control", 1000), ("treatment", 2000)] exposes 20% and holds 80% out.
    """
    b = bucket(user_id, experiment_id)
    for variation_key, end_of_range in allocation:
        if b < end_of_range:
            return variation_key
    return None  # bucketed out of the experiment: serve the flag's default

print(choose_variation("user-42", "checkout-redesign", [("control", 5000), ("treatment", 10000)]))

Production SDKs use a faster non-cryptographic hash and a user_id + experiment_id key, but the shape is this. Note what a ramp from 20% to 50% does under this scheme: the ranges grow, so every user who was in the experiment at 20% is still in it, in the same variation, at 50%. That is the property that makes a progressive rollout safe, and it holds only if the allocation table is edited by extending ranges rather than reshuffling them.

Consistency and propagation delay

Put distribution and evaluation together and the consistency model of the whole system follows. A flag decision is eventually consistent across instances, with the staleness budget as the bound, and strongly consistent for a given user within one instance's snapshot, because bucketing is deterministic. A rollback is therefore not instantaneous: after the control plane changes, instances converge over the budget, and during that window a user can be served the new rules by one server and the old by another.

Three practices keep this from becoming an incident. Stamp every snapshot with a revision number, log it with every decision, and put it in the request's trace, so that "which rules made this decision?" has an answer. Make the application tolerant of the window — a feature that cannot survive one server disagreeing with another for thirty seconds needs a different mechanism than a flag. And treat the kill switch as the flag whose budget matters most: if seconds count, use the push channel for it, and verify in staging that the push actually reaches every instance rather than assuming.

Failure modes and what they cost

The pattern in the last column is that almost every failure is prevented at design time and almost none can be fixed at incident time. A flag system's job is to make a change safe; a flag system that itself becomes the incident has inverted its purpose.

Build or buy: mapping the components to Optimizely Feature Experimentation

Each component above corresponds to something specific in Optimizely Feature Experimentation, which is useful both for teams adopting it and for teams building their own who want to know what "complete" looks like.

The control plane is the Optimizely application and its REST API: flags, variations, variables, rules, audiences and environments, with change history per environment. Distribution is the datafile on Optimizely's CDN, refreshed by the SDK's datafile manager on an interval and, optionally, by a webhook that tells your infrastructure a new revision exists. Evaluation is the SDK's decide call, which runs locally against the datafile using the bucketing scheme above; the SDK is available for server, browser and mobile runtimes, and the server-side A/B testing guide shows the server pattern in full. Telemetry is the event dispatcher, which batches decision and conversion events to Optimizely's event endpoint for the results page and for experiment analysis.

import { createInstance } from '@optimizely/optimizely-sdk'

const optimizely = createInstance({
  sdkKey: process.env.OPTIMIZELY_SDK_KEY,
  datafileOptions: { autoUpdate: true, updateInterval: 60_000 }, // staleness budget: one minute
})

await optimizely.onReady() // block once, at startup, so no decision runs against an empty snapshot

const user = optimizely.createUserContext(userId, { country: 'DE', plan: 'pro' })
const decision = user.decide('checkout-redesign')

if (decision.enabled) {
  renderNewCheckout(decision.variables.layout)
} else {
  renderCurrentCheckout()
}

Building the same four components in-house is a reasonable choice for a team with unusual constraints — an air-gapped environment, a bespoke identity model, a hard latency budget below what a general SDK offers. It is a poor choice when the reason is that the first version looked like a map of booleans. The map is the easy part; the audit trail, the staleness budget, the deterministic bucketing that survives a ramp, and the telemetry that makes a rollout observable are the system, and they are where the years go.

Frequently asked questions

What is the difference between a feature flag and a feature toggle?

None in practice; the terms are interchangeable. "Toggle" is older and tends to be used for on/off switches, "flag" for the general case including multi-variation and variable-carrying configuration.

Should flag evaluation be a service or a library?

A library, evaluating locally from a snapshot, for anything in a request path. A service is acceptable for evaluation that happens rarely or asynchronously, and for clients — such as some edge runtimes — that cannot hold a snapshot.

How do I keep the same user in the same variation across web and mobile?

Bucket on an identifier both platforms share, such as an account ID, and pass it explicitly as the bucketing ID. Anything device-specific — a cookie, an installation ID — will produce different buckets on each platform.

How many flags are too many?

There is no fixed number; the constraint is the snapshot size and the number of live conditionals a team can reason about. A flag that has been at 100% for a release cycle is done, and its removal is part of shipping the feature.