Integrate Amplitude with Optimizely Feature Experimentation

Optimizely Feature Experimentation uses SDK-based feature flags rather than a client-side snippet. Integrating with Amplitude sends experiment decision data into your analytics platform as user properties and events, enabling behavioral segmentation by experiment variation, funnel analysis comparing treatment groups, and cohort-based experiment targeting through Amplitude Audience Sync.

This guide covers JavaScript/Node.js and Python implementations using the SDK's Decision Notification Listener, explains how to structure the data for optimal Amplitude analysis, and walks through building segments and funnels filtered by experiment.

How the Integration Works

When the Optimizely Feature Experimentation SDK evaluates a feature flag using the decide() method, it fires a DECISION notification. You register a listener for this notification that captures the flag key, variation key, and experiment details, then sends the data to Amplitude as both a user property (via Identify) and an event (via logEvent). Amplitude then associates the experiment context with all subsequent events from that user.

sequenceDiagram
    participant App as Application
    participant SDK as Optimizely SDK
    participant Listener as DECISION Listener
    participant Amp as Amplitude SDK
    participant Dash as Amplitude Dashboard

    App->>SDK: createInstance(sdkKey)
    App->>SDK: addNotificationListener(DECISION, callback)
    App->>SDK: user.decide("flag_key")
    SDK->>Listener: DECISION notification fires
    Listener->>Amp: amplitude.identify() — set user property
    Listener->>Amp: amplitude.logEvent("Experiment Viewed")
    Amp->>Dash: User property + event visible in dashboard
    Dash->>Dash: Build segments, funnels, retention by variation

Decision Notification Data

The DECISION notification provides the following data through the callback arguments:

The decisionEventDispatched field indicates whether Optimizely dispatched a decision event to its own analytics backend. When false, Optimizely skipped sending the event (typically because the user was already counted). Your Amplitude integration should still send the data in this case, because Amplitude needs the experiment context associated with every relevant session.

User Property Format

The user property follows a consistent naming convention:

[Optimizely] flagKey = variationKey

For example, a user bucketed into the variation_a treatment of a checkout_redesign flag would have:

[Optimizely] checkout_redesign = variation_a

This format groups all Optimizely experiments under the [Optimizely] namespace, making them easy to find in Amplitude's user property list.

Prerequisites

Before starting the integration:

• Optimizely Feature Experimentation SDK installed for your platform (JavaScript SDK v6+, Node.js SDK, or Python SDK).

• Amplitude SDK installed and initialized for the same platform.

• A feature flag with an experiment rule configured in your Optimizely project.

• Consistent user ID between the Optimizely SDK and Amplitude. Both SDKs must use the same user identifier for the experiment context to be associated with the correct Amplitude user profile.

JavaScript / Node.js Implementation

The JavaScript implementation works in both browser and Node.js environments. The only difference is which Amplitude SDK you import.

Browser Setup

import { createInstance, enums } from '@optimizely/optimizely-sdk';
import * as amplitude from '@amplitude/analytics-browser';

// Initialize Amplitude
amplitude.init('YOUR_AMPLITUDE_API_KEY');

// Initialize Optimizely
const optimizely = createInstance({
  sdkKey: '<YOUR_SDK_KEY>',
});

// Wait for the SDK to be ready
optimizely.onReady().then(() => {
  // Register the DECISION notification listener
  optimizely.notificationCenter.addNotificationListener(
    enums.NOTIFICATION_TYPES.DECISION,
    ({ type, userId, attributes, decisionInfo }) => {
      // Only process flag decisions
      if (type !== 'flag') return;

      const { flagKey, enabled, variationKey, ruleKey } = decisionInfo;

      // Set user property: [Optimizely] flagKey = variationKey
      const propertyName = `[Optimizely] ${flagKey}`;
      const propertyValue = enabled ? variationKey : 'off';

      const identify = new amplitude.Identify().set(propertyName, propertyValue);
      amplitude.identify(identify);

      // Log the Experiment Viewed event
      amplitude.logEvent('Experiment Viewed', {
        flag_key: flagKey,
        variation_key: variationKey,
        rule_key: ruleKey,
        enabled: enabled,
        user_id: userId,
      });
    }
  );

  // Now make decisions — the listener fires for each one
  const user = optimizely.createUserContext(userId, userAttributes);
  const decision = user.decide('checkout_redesign');
  console.log('Variation:', decision.variationKey);
});

Node.js Setup

For server-side Node.js, use the Amplitude Node SDK:

import { createInstance, enums } from '@optimizely/optimizely-sdk';
import { init as amplitudeInit, identify, logEvent, Identify } from '@amplitude/analytics-node';

// Initialize Amplitude (Node SDK)
amplitudeInit('YOUR_AMPLITUDE_API_KEY');

// Initialize Optimizely
const optimizely = createInstance({
  sdkKey: '<YOUR_SDK_KEY>',
});

optimizely.onReady().then(() => {
  // Register the DECISION notification listener
  optimizely.notificationCenter.addNotificationListener(
    enums.NOTIFICATION_TYPES.DECISION,
    ({ type, userId, attributes, decisionInfo }) => {
      if (type !== 'flag') return;

      const { flagKey, enabled, variationKey, ruleKey } = decisionInfo;

      // Set user property
      const propertyName = `[Optimizely] ${flagKey}`;
      const propertyValue = enabled ? variationKey : 'off';

      const identifyObj = new Identify().set(propertyName, propertyValue);
      identify(identifyObj, { user_id: userId });

      // Log event
      logEvent('Experiment Viewed', {
        user_id: userId,
      }, {
        flag_key: flagKey,
        variation_key: variationKey,
        rule_key: ruleKey,
        enabled: enabled,
      });
    }
  );
});

// In your request handler
function handleRequest(userId, userAttributes) {
  const user = optimizely.createUserContext(userId, userAttributes);
  const decision = user.decide('checkout_redesign');
  // The DECISION listener fires and sends data to Amplitude
  return decision;
}

Listener Registration Timing

The notification listener must be registered before any decide() calls. If you register the listener after calling decide(), those decisions are lost because the notification has already fired.

// CORRECT: Register listener before decide()
optimizely.onReady().then(() => {
  optimizely.notificationCenter.addNotificationListener(
    enums.NOTIFICATION_TYPES.DECISION,
    callback
  );
  const decision = user.decide('flag_key'); // Listener fires
});

// WRONG: Registering after decide() misses the decision
optimizely.onReady().then(() => {
  const decision = user.decide('flag_key'); // Listener not registered yet
  optimizely.notificationCenter.addNotificationListener(
    enums.NOTIFICATION_TYPES.DECISION,
    callback // Too late — the decision above was already made
  );
});

Python Implementation

The Python SDK uses the same notification listener pattern. Use the Amplitude Analytics Python SDK to send data.

from optimizely import optimizely
from optimizely.helpers import enums
from amplitude import Amplitude, BaseEvent, Identify, EventOptions

# Initialize Amplitude
amplitude_client = Amplitude("YOUR_AMPLITUDE_API_KEY")

# Initialize Optimizely
optimizely_client = optimizely.Optimizely(sdk_key="YOUR_SDK_KEY")

# Define the DECISION notification callback
def on_decision(notification_type, args):
    decision_info = args.get("decisionInfo", {})
    decision_type = args.get("type", "")
    user_id = args.get("userId", "")

    # Only process flag decisions
    if decision_type != "flag":
        return

    flag_key = decision_info.get("flagKey", "")
    enabled = decision_info.get("enabled", False)
    variation_key = decision_info.get("variationKey", "")
    rule_key = decision_info.get("ruleKey", "")

    # Set user property: [Optimizely] flagKey = variationKey
    property_name = f"[Optimizely] {flag_key}"
    property_value = variation_key if enabled else "off"

    identify_obj = Identify()
    identify_obj.set(property_name, property_value)
    amplitude_client.identify(
        identify_obj,
        EventOptions(user_id=user_id)
    )

    # Log the Experiment Viewed event
    amplitude_client.track(
        BaseEvent(
            event_type="Experiment Viewed",
            user_id=user_id,
            event_properties={
                "flag_key": flag_key,
                "variation_key": variation_key,
                "rule_key": rule_key,
                "enabled": enabled,
            },
        )
    )

# Register the listener BEFORE any decide() calls
notification_id = optimizely_client.notification_center.add_notification_listener(
    enums.NotificationTypes.DECISION,
    on_decision,
)

# Make a decision — the listener fires
user = optimizely_client.create_user_context("user_123", {"plan": "premium"})
decision = user.decide("checkout_redesign")
print(f"Variation: {decision.variation_key}")

# Clean up when shutting down
amplitude_client.flush()
amplitude_client.shutdown()

Building Segments and Funnels in Amplitude

Creating a Segment by Experiment Variation

1. Go to Cohorts > Create Cohort in Amplitude.

2. Add a condition: user property [Optimizely] checkout_redesign equals variation_a.

3. Save the cohort (e.g., "Checkout Redesign - Variation A").

4. Repeat for the control group: user property [Optimizely] checkout_redesign equals control.

These cohorts update dynamically as new users are bucketed into the experiment.

Funnel Analysis by Variation

To compare funnel performance between variations:

1. Go to Analytics > Funnel Analysis.

2. Define your funnel steps (e.g., Feature Enabled > Add to Cart > Checkout > Purchase).

3. In the Segment by section, select the user property [Optimizely] checkout_redesign.

4. Amplitude displays conversion rates for each variation side by side.

Filtering Any Chart by Experiment

The user property [Optimizely] flagKey is available as a filter in every Amplitude chart type:

Retention Analysis

To measure whether a feature flag variation affects long-term retention:

1. Go to Analytics > Retention.

2. Set the starting event to "Experiment Viewed" filtered by flag_key = checkout_redesign.

3. Set the return event to your key engagement metric.

4. Segment by variation_key.

5. Compare Day 1, Day 7, and Day 30 retention across variations.

Amplitude Audience Sync

Amplitude Audience Sync works in the reverse direction: you create behavioral cohorts in Amplitude and sync them to Optimizely for experiment targeting. This enables running experiments on user segments identified through Amplitude behavioral analysis.

How Audience Sync Works

sequenceDiagram
    participant Amp as Amplitude
    participant Sync as Audience Sync
    participant Opti as Optimizely FX
    participant App as Application

    Amp->>Amp: Define behavioral cohort
    Amp->>Sync: Push cohort membership
    Sync->>Opti: Sync user attributes
    App->>Opti: createUserContext(userId, attributes)
    Opti->>Opti: Evaluate audience targeting
    Opti->>App: Return decision for cohort members

Setup Steps

1. In Amplitude, go to Data Destinations > Add Destination > Optimizely Feature Experimentation.

2. Enter your Optimizely SDK key and environment details.

3. Create or select a cohort in Amplitude (e.g., "Power Users" with 10+ sessions in 30 days).

4. Map the cohort to an Optimizely attribute name (e.g., amplitude_power_user).

5. Configure the sync schedule.

6. In Optimizely, create an audience that targets users where amplitude_power_user equals true.

7. Assign this audience to your experiment.

Use Cases for Audience Sync

Gotchas

Listener Registration Timing

The most common integration failure is registering the notification listener after decide() calls have already been made. The Optimizely SDK does not replay past decisions to newly registered listeners. Always register the listener immediately after onReady() resolves and before any decide() calls.

User ID Consistency

The Optimizely SDK and Amplitude SDK must use the same user identifier. If Optimizely uses user_abc123 but Amplitude identifies the same user as abc123, the experiment data lands on a different Amplitude profile than the user's other events.

For browser implementations, a common pattern is:

// Use the same user ID for both SDKs
const userId = getCurrentUserId(); // Your auth system

amplitude.setUserId(userId);
const user = optimizely.createUserContext(userId, attributes);

For anonymous users, use the same device identifier for both SDKs or let Amplitude's device ID serve as the Optimizely user ID.

Multiple Decide Calls

The DECISION notification fires for every decide() call, even if the user has already been bucketed. If your application calls decide() on every page render (common in React applications), the listener fires repeatedly. This is typically acceptable because:

• Amplitude deduplicates user properties (setting the same property to the same value is a no-op).

• Repeated "Experiment Viewed" events can be filtered in analysis.

If event volume is a concern, add a deduplication check in the listener:

const processedDecisions = new Set();

optimizely.notificationCenter.addNotificationListener(
  enums.NOTIFICATION_TYPES.DECISION,
  ({ type, userId, decisionInfo }) => {
    if (type !== 'flag') return;

    const key = `${userId}:${decisionInfo.flagKey}:${decisionInfo.variationKey}`;
    if (processedDecisions.has(key)) return;
    processedDecisions.add(key);

    // Send to Amplitude only once per unique decision
    // ... identify and logEvent calls
  }
);

Server-Side Flush Behavior

In Node.js and Python, Amplitude SDKs batch events before sending. If your process exits immediately after making decisions (e.g., a Lambda function), call flush() to ensure events are sent:

// Node.js — flush before process exit
await amplitude.flush();

# Python — flush before process exit
amplitude_client.flush()
amplitude_client.shutdown()

Troubleshooting

User Properties Not Appearing in Amplitude

If "Experiment Viewed" events appear but user properties do not:

• Identify API version: Ensure you are using the correct Identify class for your Amplitude SDK version. The API differs between Amplitude Browser SDK v1 and v2.

• User ID mismatch: If the identify() call uses a different user ID than the one associated with the event, the property lands on a different profile.

• Amplitude processing delay: User properties can take 1-2 minutes to appear in the dashboard. Check again after a short wait.

Events Not Appearing in Amplitude

If neither events nor properties appear:

• Listener not registered: Add a console.log inside your listener callback to verify it fires. If it does not fire, the listener was not registered before the decide() call.

• Amplitude not initialized: Verify the Amplitude SDK is initialized before the listener fires. In Node.js, ensure init() completes before processing requests.

• Network issues: Check for failed network requests to api2.amplitude.com (browser) or api2.amplitude.com/2/httpapi (server).

Decision Listener Fires But Data Is Wrong

If the listener fires but sends incorrect data:

• Check decisionInfo shape: Log the full decisionInfo object to verify it contains the expected fields. Missing fields usually mean the flag does not have an experiment rule (it may be a simple rollout).

• enabled: false handling: When a flag is disabled for a user, variationKey may be empty. The example code above sends "off" as the property value in this case.

• ruleKey is empty: If no experiment rule matched, ruleKey may be an empty string. This happens when the flag evaluation falls through to the default.

Data Discrepancies Between Platforms

Differences between Optimizely and Amplitude counts are expected:

• Counting method: Optimizely counts unique visitors per experiment, while Amplitude counts unique users per event. Identity resolution differences cause divergence.

• Server-side batching: The Amplitude Node/Python SDK batches events. Under high load, some events may be dropped if the batch queue fills up.

• Multiple decisions: If decide() is called multiple times for the same user and flag, Optimizely counts one unique visitor but your listener may send multiple events to Amplitude (unless you implemented deduplication).

Expect discrepancies of 5-15% between platforms. Investigate further if differences exceed 20%.