Two-Step Bucketing with DISABLE_DECISION_EVENT: Pre-bucket Users Without Tracking Impressions

by

Hello fellow Optimizely practitioners! Today we’re diving into a powerful but underutilized pattern in Feature Experimentation: two-step bucketing using DISABLE_DECISION_EVENT.

Understanding Two-Step Bucketing

Two-step bucketing is a strategic approach where you separate user assignment (bucketing) from impression tracking. Instead of the standard single decide() call that both assigns users to variations and immediately tracks impressions, this pattern involves:

  1. Step 1: Pre-bucket users with DISABLE_DECISION_EVENT – determines variation assignment without tracking
  2. Step 2: Track impressions when users actually see the feature – dispatches the decision event for analytics

What is DISABLE_DECISION_EVENT

The OptimizelyDecideOption.DISABLE_DECISION_EVENT parameter prevents the SDK from dispatching an impression event when serving a variation. As the official documentation states: “This disables decision tracking on the Optimizely Experiment Results page and the decision notification listener.” [Doc: Decide methods for the JavaScript SDK v6+]

When you use this option, the SDK still performs all bucketing logic—evaluating targeting conditions, applying traffic allocation, and determining the correct variation—but it doesn’t send the decision event to Optimizely’s analytics pipeline.

Bucketing vs Impression Tracking

It’s crucial to understand the distinction between these two operations:

  • Bucketing: The process of assigning users to experiment variations based on their ID, targeting rules, and traffic allocation
  • Impression Tracking: Recording that a user was exposed to a variation for analytics and statistical analysis

Optimizely normally logs an impression when “a user is exposed to a flag variation” through the sequence: “The Feature Experimentation SDK makes a decision for a user in an experiment through a Decide method. As a result, the SDK sends an SDK decision event asynchronously to the Optimizely Event API.” [Doc: What are impressions and decisions]

Two-step bucketing separates these operations, giving you precise control over when impressions are recorded.

Use Cases for Two-Step Bucketing

Mobile App Performance Optimization

Mobile applications often need to pre-load content and make UI decisions at app startup to ensure smooth user experiences. However, users might not immediately see all features that were pre-configured.

Consider a news app that pre-loads article layouts, recommendation algorithms, and UI themes at startup. Without two-step bucketing, every user opening the app would generate impressions for experiments they might not actually encounter during that session.

With two-step bucketing:

  1. App startup: Pre-bucket users for all experiments with DISABLE_DECISION_EVENT
  2. Feature visibility: Track impressions only when users navigate to screens where experiments are active

Cache Warming and CDN Strategies

Edge computing scenarios benefit significantly from two-step bucketing. As the documentation explains, this pattern is “particularly useful when you need to serve the correct cached version of your page or component based on the user’s variation. For example when using an edge worker or cloud function.” [Doc: Decide methods for the Go SDK]

Implementation workflow:

  1. Edge worker receives request and pre-buckets user for all active experiments
  2. Serves appropriate cached content variation based on bucketing results
  3. Client-side JavaScript tracks impressions when content becomes visible to users

Analytics Accuracy

Two-step bucketing ensures your experiment analytics reflect actual feature exposure, not just technical decisions. This is particularly valuable for:

  • Features that load conditionally based on user behavior
  • Experiments with complex visibility logic
  • A/B tests measuring engagement metrics that require actual user interaction

Implementation Approaches

Pattern 1: Global Default Configuration

Set DISABLE_DECISION_EVENT as a default option during SDK initialization. This approach works well when you want to apply two-step bucketing to most or all experiments.

import { createInstance, OptimizelyDecideOption } from '@optimizely/optimizely-sdk';

// Initialize SDK with DISABLE_DECISION_EVENT as default
const optimizelyClient = createInstance({
  sdkKey: 'YOUR_SDK_KEY',
  defaultDecideOptions: [OptimizelyDecideOption.DISABLE_DECISION_EVENT]
});

// Pre-bucket at app initialization
function preBucketAllExperiments(user) {
  const decisions = optimizelyClient.decideAll(user);
  // Store decisions in app state for later use
  return decisions;
}

// Track impressions when features are actually viewed
function trackImpression(user, flagKey) {
  // This will track the impression since we're not using DISABLE_DECISION_EVENT
  optimizelyClient.decide(user, flagKey);
}

Pattern 2: Selective Method-Level Control

Apply DISABLE_DECISION_EVENT only to specific decide calls where you need two-step bucketing. This gives you fine-grained control over which experiments use this pattern.

import { OptimizelyDecideOption } from '@optimizely/optimizely-sdk';

class ExperimentManager {
  constructor(optimizelyClient) {
    this.client = optimizelyClient;
    this.preBucketedDecisions = new Map();
  }

  // Pre-bucket specific experiments without tracking
  preBucketExperiment(user, flagKey) {
    const decision = this.client.decide(user, flagKey, [
      OptimizelyDecideOption.DISABLE_DECISION_EVENT
    ]);
    
    this.preBucketedDecisions.set(flagKey, decision);
    return decision;
  }

  // Track impression when feature is actually viewed
  trackExperimentImpression(user, flagKey) {
    // Verify we have a pre-bucketed decision
    if (!this.preBucketedDecisions.has(flagKey)) {
      console.warn(`No pre-bucketed decision found for ${flagKey}`);
    }
    
    // This call will track the impression
    return this.client.decide(user, flagKey);
  }

  // Get pre-bucketed variation without additional tracking
  getVariation(flagKey) {
    const decision = this.preBucketedDecisions.get(flagKey);
    return decision ? decision.variationKey : null;
  }
}

Pattern 3: State Management Integration

Integrate two-step bucketing with your application’s state management system for complex scenarios requiring persistence across user sessions.

// React example with state management
import { createContext, useContext, useReducer, useEffect } from 'react';
import { useOptimizely } from '@optimizely/react-sdk';
import { OptimizelyDecideOption } from '@optimizely/optimizely-sdk';

const ExperimentContext = createContext();

function experimentReducer(state, action) {
  switch (action.type) {
    case 'SET_PRE_BUCKETED_DECISIONS':
      return {
        ...state,
        preBucketedDecisions: action.payload,
        isPreBucketed: true
      };
    case 'TRACK_IMPRESSION':
      return {
        ...state,
        trackedImpressions: {
          ...state.trackedImpressions,
          [action.flagKey]: true
        }
      };
    default:
      return state;
  }
}

export function ExperimentProvider({ children }) {
  const { optimizely, isReady } = useOptimizely();
  const [state, dispatch] = useReducer(experimentReducer, {
    preBucketedDecisions: {},
    trackedImpressions: {},
    isPreBucketed: false
  });

  // Pre-bucket all experiments when SDK is ready
  useEffect(() => {
    if (isReady && optimizely && !state.isPreBucketed) {
      const user = { id: 'YOUR_USER_ID' };
      const decisions = optimizely.decideAll(user, [OptimizelyDecideOption.DISABLE_DECISION_EVENT]);
      
      dispatch({
        type: 'SET_PRE_BUCKETED_DECISIONS',
        payload: decisions
      });
    }
  }, [isReady, optimizely, state.isPreBucketed]);

  const trackImpression = (flagKey) => {
    if (!state.trackedImpressions[flagKey] && optimizely) {
      const user = { id: 'YOUR_USER_ID' };
      optimizely.decide(user, flagKey); // This tracks the impression
      
      dispatch({
        type: 'TRACK_IMPRESSION',
        flagKey
      });
    }
  };

  return (
    
      {children}
    
  );
}

Code Examples by SDK

JavaScript SDK v6+ Implementation

Here’s a production-ready implementation with comprehensive error handling:

import { OptimizelyDecideOption } from '@optimizely/optimizely-sdk';

class TwoStepBucketingManager {
  constructor(optimizelyClient) {
    this.client = optimizelyClient;
    this.preBucketedDecisions = new Map();
    this.impressionTracker = new Set();
  }

  async initialize(user) {
    try {
      if (!this.client) {
        throw new Error('Optimizely client not initialized');
      }

      // Pre-bucket all active flags
      const decisions = this.client.decideAll(user, [
        OptimizelyDecideOption.DISABLE_DECISION_EVENT
      ]);

      // Store decisions for later reference
      Object.entries(decisions).forEach(([flagKey, decision]) => {
        this.preBucketedDecisions.set(flagKey, {
          ...decision,
          timestamp: Date.now(),
          userId: user.id
        });
      });

      console.log(`Pre-bucketed ${Object.keys(decisions).length} flags`);
      return decisions;
    } catch (error) {
      console.error('Failed to pre-bucket decisions:', error);
      throw error;
    }
  }

  getVariation(flagKey) {
    const decision = this.preBucketedDecisions.get(flagKey);
    if (!decision) {
      console.warn(`No pre-bucketed decision found for flag: ${flagKey}`);
      return null;
    }
    return decision.variationKey;
  }

  trackImpression(user, flagKey) {
    // Prevent duplicate impression tracking
    const impressionKey = `${user.id}-${flagKey}`;
    if (this.impressionTracker.has(impressionKey)) {
      return;
    }

    try {
      // This call will track the impression
      const decision = this.client.decide(user, flagKey);
      this.impressionTracker.add(impressionKey);
      
      console.log(`Tracked impression for ${flagKey}: ${decision.variationKey}`);
      return decision;
    } catch (error) {
      console.error(`Failed to track impression for ${flagKey}:`, error);
    }
  }

  // Batch track impressions for multiple flags
  trackMultipleImpressions(user, flagKeys) {
    const results = {};
    flagKeys.forEach(flagKey => {
      results[flagKey] = this.trackImpression(user, flagKey);
    });
    return results;
  }

  // Clean up old pre-bucketed decisions (optional)
  cleanup(maxAgeMs = 3600000) { // 1 hour default
    const now = Date.now();
    for (const [flagKey, decision] of this.preBucketedDecisions) {
      if (now - decision.timestamp > maxAgeMs) {
        this.preBucketedDecisions.delete(flagKey);
      }
    }
  }
}

React SDK with Hooks

The React SDK provides hooks that work seamlessly with two-step bucketing:

import React, { useState, useEffect, useCallback } from 'react';
import { useOptimizely } from '@optimizely/react-sdk';
import { OptimizelyDecideOption } from '@optimizely/optimizely-sdk';

function FeatureComponent({ flagKey, onVisible }) {
  const { optimizely, isReady } = useOptimizely();
  const [variation, setVariation] = useState(null);
  const [impressionTracked, setImpressionTracked] = useState(false);

  // Pre-bucket on component mount
  useEffect(() => {
    if (isReady && optimizely) {
      const user = { id: 'YOUR_USER_ID' };
      
      // Get variation without tracking impression
      const decision = optimizely.decide(user, flagKey, [
        OptimizelyDecideOption.DISABLE_DECISION_EVENT
      ]);
      
      setVariation(decision.variationKey);
    }
  }, [isReady, optimizely, flagKey]);

  // Track impression when component becomes visible
  const trackImpression = useCallback(() => {
    if (!impressionTracked && optimizely) {
      const user = { id: 'YOUR_USER_ID' };
      optimizely.decide(user, flagKey); // Tracks impression
      setImpressionTracked(true);
      onVisible?.(flagKey, variation);
    }
  }, [optimizely, flagKey, variation, impressionTracked, onVisible]);

  // Example: Track impression on intersection (when visible)
  useEffect(() => {
    const observer = new IntersectionObserver(
      (entries) => {
        entries.forEach(entry => {
          if (entry.isIntersecting) {
            trackImpression();
          }
        });
      },
      { threshold: 0.5 }
    );

    const element = document.getElementById(`feature-${flagKey}`);
    if (element) {
      observer.observe(element);
    }

    return () => observer.disconnect();
  }, [trackImpression, flagKey]);

  if (!variation) {
    return <div>Loading...</div>;
  }

  return (
    <div id={`feature-${flagKey}`}>
      {variation === 'treatment' ? (
        <div>New Feature Implementation</div>
      ) : (
        <div>Control Implementation</div>
      )}
    </div>
  );
}

Mobile SDK Examples

Here’s how two-step bucketing works in mobile environments, using Java for Android:

// Java Android example (for reference)
import com.optimizely.ab.android.sdk.OptimizelyClient;
import com.optimizely.ab.android.sdk.OptimizelyUserContext;
import com.optimizely.ab.config.OptimizelyDecideOption;
import java.util.Arrays;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;

public class TwoStepBucketingManager {
    private OptimizelyClient optimizelyClient;
    private Map<String, OptimizelyDecision> preBucketedDecisions;
    private Set<String> trackedImpressions;

    public TwoStepBucketingManager(OptimizelyClient client) {
        this.optimizelyClient = client;
        this.preBucketedDecisions = new HashMap<>();
        this.trackedImpressions = new HashSet<>();
    }

    public void preBucketOnAppStart(OptimizelyUserContext user) {
        List<OptimizelyDecideOption> options = Arrays.asList(
            OptimizelyDecideOption.DISABLE_DECISION_EVENT
        );
        
        Map<String, OptimizelyDecision> decisions = 
            optimizelyClient.decideAll(user, options);
        
        preBucketedDecisions.putAll(decisions);
    }

    public void trackImpressionOnView(OptimizelyUserContext user, String flagKey) {
        String impressionKey = user.getUserId() + "-" + flagKey;
        
        if (!trackedImpressions.contains(impressionKey)) {
            optimizelyClient.decide(user, flagKey); // Tracks impression
            trackedImpressions.add(impressionKey);
        }
    }
}
// JavaScript equivalent for mobile web apps
import { OptimizelyDecideOption } from '@optimizely/optimizely-sdk';

class MobileTwoStepManager {
  constructor(optimizelyClient) {
    this.client = optimizelyClient;
    this.preBucketed = new Map();
    this.tracked = new Set();
  }

  // Call during app initialization
  async preBucketOnAppStart(user) {
    if ('serviceWorker' in navigator) {
      // Cache decisions in service worker for offline access
      const decisions = this.client.decideAll(user, [
        OptimizelyDecideOption.DISABLE_DECISION_EVENT
      ]);
      
      navigator.serviceWorker.ready.then(registration => {
        registration.active.postMessage({
          type: 'CACHE_DECISIONS',
          decisions,
          userId: user.id
        });
      });
      
      this.preBucketed.set(user.id, decisions);
    }
  }

  // Track when user navigates to feature screen
  trackOnScreenVisible(user, flagKey) {
    const key = `${user.id}-${flagKey}`;
    if (!this.tracked.has(key)) {
      this.client.decide(user, flagKey);
      this.tracked.add(key);
    }
  }
}

Pros and Cons Analysis

Performance Benefits

Resource Optimization:

  • Lower bandwidth usage on mobile devices
  • Reduced server load on Optimizely Event API
  • More efficient battery usage on mobile apps

Analytics Trade-offs

Decision Notification Impact:

When using DISABLE_DECISION_EVENT, your decision notification listeners won’t be triggered during the initial bucketing phase. This affects:

  • Real-time personalization systems that depend on decision listeners
  • Custom analytics integrations that hook into decision events
  • Third-party tools expecting immediate decision notifications

Results Page Tracking:

Impressions won’t appear in your Optimizely Experiment Results page until the second decide call occurs. This means:

  • Delayed visibility into experiment performance
  • Potential confusion if team members expect immediate impression tracking
  • Need for clear documentation of your two-step implementation

Implementation Complexity

Additional State Management:

  • Must store and manage pre-bucketed decisions in application state
  • Need to handle decision expiration and refresh logic
  • Requires coordination between pre-bucketing and impression tracking phases

Potential Race Conditions:

  • User context might change between pre-bucketing and impression tracking
  • Network failures during second decide call need graceful handling
  • Multiple components might attempt to track impressions simultaneously

Best Practices and Gotchas

Decision State Management

Proper Storage Strategies:

  1. In-Memory Storage: Use Map/Set objects for short-lived sessions
  2. Session Storage: Persist decisions across page reloads within same session
  3. Local Storage: Cache decisions across browser sessions (with expiration)
  4. Service Workers: Enable offline access to pre-bucketed decisions
class PersistentDecisionManager {
  constructor(optimizelyClient, storageType = 'session') {
    this.client = optimizelyClient;
    this.storage = storageType === 'local' ? localStorage : sessionStorage;
    this.STORAGE_KEY = 'optimizely_prebucketed_decisions';
  }

  saveDecisions(decisions, ttlMs = 3600000) { // 1 hour default
    const data = {
      decisions,
      timestamp: Date.now(),
      expiry: Date.now() + ttlMs
    };
    
    this.storage.setItem(this.STORAGE_KEY, JSON.stringify(data));
  }

  loadDecisions() {
    try {
      const stored = this.storage.getItem(this.STORAGE_KEY);
      if (!stored) return null;

      const data = JSON.parse(stored);
      if (Date.now() > data.expiry) {
        this.storage.removeItem(this.STORAGE_KEY);
        return null;
      }

      return data.decisions;
    } catch (error) {
      console.error('Failed to load stored decisions:', error);
      return null;
    }
  }

  clearExpiredDecisions() {
    this.loadDecisions(); // This will remove if expired
  }
}

Error Handling Strategies

Fallback Behavior:

  1. Network Failures: Cache last known good decisions locally
  2. SDK Errors: Implement graceful degradation to default experiences
  3. User Context Changes: Re-bucket and track new decisions appropriately
import { OptimizelyDecideOption } from '@optimizely/optimizely-sdk';

class RobustTwoStepManager {
  constructor(optimizelyClient) {
    this.client = optimizelyClient;
    this.fallbackDecisions = new Map();
  }

  async safePreBucket(user, retryCount = 3) {
    for (let attempt = 1; attempt <= retryCount; attempt++) {
      try {
        const decisions = this.client.decideAll(user, [
          OptimizelyDecideOption.DISABLE_DECISION_EVENT
        ]);
        
        // Cache successful decisions as fallback
        this.fallbackDecisions.set(user.id, decisions);
        return decisions;
      } catch (error) {
        console.warn(`Pre-bucket attempt ${attempt} failed:`, error);
        
        if (attempt === retryCount) {
          // Return cached fallback or defaults
          return this.fallbackDecisions.get(user.id) || {};
        }
        
        // Exponential backoff
        await new Promise(resolve => 
          setTimeout(resolve, Math.pow(2, attempt) * 1000)
        );
      }
    }
  }

  safeTrackImpression(user, flagKey) {
    try {
      return this.client.decide(user, flagKey);
    } catch (error) {
      console.error(`Failed to track impression for ${flagKey}:`, error);
      
      // Return fallback decision without tracking
      const fallback = this.fallbackDecisions.get(user.id);
      return fallback?.[flagKey] || { variationKey: null };
    }
  }
}

Testing and Validation

Development Testing Checklist:

  1. Verify pre-bucketing occurs without generating impressions
  2. Confirm impression tracking happens exactly once per user/flag combination
  3. Test behavior when user context changes between steps
  4. Validate fallback handling for network failures
  5. Check decision consistency between pre-bucket and impression calls
// Testing utilities
class TwoStepTestingUtils {
  constructor(optimizelyClient) {
    this.client = optimizelyClient;
    this.impressionLog = [];
    this.bucketingLog = [];
    
    // Mock impression tracking for testing
    this.originalDecide = this.client.decide;
    this.client.decide = this.mockDecide.bind(this);
  }

  mockDecide(user, flagKey, options = []) {
    const hasDisableEvent = options.includes(OptimizelyDecideOption.DISABLE_DECISION_EVENT);
    
    if (hasDisableEvent) {
      this.bucketingLog.push({ user: user.id, flagKey, timestamp: Date.now() });
    } else {
      this.impressionLog.push({ user: user.id, flagKey, timestamp: Date.now() });
    }
    
    return this.originalDecide.call(this.client, user, flagKey, options);
  }

  validateTwoStepFlow(userId, flagKey) {
    const bucketEvents = this.bucketingLog.filter(
      log => log.user === userId && log.flagKey === flagKey
    );
    const impressionEvents = this.impressionLog.filter(
      log => log.user === userId && log.flagKey === flagKey
    );

    return {
      preBucketCount: bucketEvents.length,
      impressionCount: impressionEvents.length,
      isValid: bucketEvents.length >= 1 && impressionEvents.length === 1
    };
  }

  reset() {
    this.impressionLog = [];
    this.bucketingLog = [];
  }
}

Two-step bucketing with DISABLE_DECISION_EVENT offers a powerful way to optimize performance while maintaining analytics accuracy. The pattern separates user assignment from impression tracking, enabling sophisticated caching strategies and ensuring that your experiment data reflects actual user exposure to features.

Key takeaways:

  • Use for mobile performance optimization and edge computing scenarios
  • Implement proper state management for pre-bucketed decisions
  • Plan for error handling and fallback strategies
  • Test thoroughly to ensure impression accuracy

Next steps: Have you implemented two-step bucketing in your applications? What challenges did you encounter with impression tracking accuracy or performance optimization? Share your experiences in the comments below—we’d love to hear about your real-world implementations and any additional patterns you’ve discovered!

Leave a Comment