Integrate Hotjar with Optimizely Feature Experimentation

Loading...·7 min read

Hotjar shows you what people do on a page: heatmaps of clicks and scroll depth, session recordings, and surveys asked at the moment something goes wrong. Optimizely Feature Experimentation decides which variation of a flag each user receives. Connecting the two lets you segment every one of those artefacts by variation — and, more importantly, stops the ones you already look at from lying to you.

That second point is worth stating plainly, because it is the strongest reason to do this integration and it is rarely the reason people give. A heatmap of a page under an A/B test is the average of two different pages. If half your traffic sees a rearranged checkout, the click map you open shows a blend: clicks on a button that only exists in one arm, spread across a layout that only exists in the other. It looks like a heatmap. It describes nothing that any user actually saw. Once the variation is a Hotjar user attribute, you can filter to one arm and get a heatmap that means something.

Feature Experimentation exposes no Custom Analytics Integration UI, so the bridge is a decision notification listener — a callback that fires on every decide() — calling Hotjar's Identify and Events APIs.

How the Integration Works

Hotjar runs in the browser and has no server-side ingest endpoint. The variation is attached with hj('identify', ...) as a user attribute, which is what recordings, heatmaps and survey targeting can all filter on, and with hj('event', ...), which is what a survey or feedback widget can trigger on.

flowchart LR
    A["user.decide('checkout_redesign')"] --> B[DECISION notification fires]
    B --> C{Where was the decision made?}
    C -->|Browser| D["hj('identify', userId, attrs)"]
    C -->|Server| E[Variation returned to the page]
    E --> D
    D --> F["hj('event', 'experiment_viewed')"]
    F --> G[Filtered recordings and heatmaps]
    F --> H[Surveys targeted at one arm]

Decision Notification Data

The listener's decision info object carries these fields for a flag-type decision. The browser and Node SDKs expose camelCase keys; the Python SDK exposes snake_case keys.

Field (camelCase / snake_case)

Type

Description

flagKey / flag_key

string

The flag key that was evaluated

enabled / enabled

boolean

Whether the flag is on for this user

variationKey / variation_key

string

The delivered variation key

ruleKey / rule_key

string

The experiment or delivery rule that matched

decisionEventDispatched / decision_event_dispatched

boolean

Whether Optimizely sent an impression for this decision

Attributes Versus Events

Hotjar offers two hooks, and this integration uses both because they are read by different parts of the product.

User attribute

Event

Called with

hj('identify', userId, attributes)

hj('event', 'name')

Scope

The user, across sessions

A point in time

Filters

Recording filters, heatmap filters, survey targeting

Survey and feedback triggers

Budget

A small, fixed number of attributes per user

Names should be few and stable

Best for

"Only show me the treatment arm"

"Ask this question when they enter the experiment"

The attribute budget is the constraint that shapes the design. Hotjar allows only a handful of custom attributes per user, so one attribute per flag will exhaust the allowance as soon as you run several experiments at once. Two conventions work: use one attribute per actively analysed flag and remove it when the experiment ends, or use a single experiments attribute holding a compact string such as checkout_redesign:treatment, which never runs out but must be filtered with a "contains" match rather than an exact one.

Prerequisites

  • Optimizely Feature Experimentation SDK — the JavaScript SDK v6 or later in the browser, and the Node.js or Python SDK if decisions are made server-side.

  • The Hotjar tracking code installed and running on the pages under test, so that window.hj exists before the listener fires.

  • A Hotjar plan that includes the Identify API. User attributes are not available on every tier; if hj('identify', ...) appears to do nothing and the recordings carry no attributes, check the plan before debugging the code.

  • A shared user identifier — the ID passed to hj('identify', ...) should be the one passed to createUserContext().

  • Consent handling, if you operate under GDPR or similar. A user attribute is personal data attached to a recording; forward it only for users whose consent covers Hotjar.

Browser Implementation

import { createInstance, enums } from '@optimizely/optimizely-sdk';

const optimizely = createInstance({ sdkKey: '<YOUR_SDK_KEY>' });

// Hotjar's attribute budget is small, so accumulate rather than send one call
// per flag with only that flag's value.
const experimentAttributes = {};

function identifyWithExperiments(userId) {
  if (typeof window.hj !== 'function') return; // tracking code blocked or not loaded
  window.hj('identify', userId, experimentAttributes);
}

optimizely.onReady().then(() => {
  optimizely.notificationCenter.addNotificationListener(
    enums.NOTIFICATION_TYPES.DECISION,
    ({ type, userId, decisionInfo }) => {
      if (type !== 'flag') return;

      const { flagKey, enabled, variationKey } = decisionInfo;
      const value = enabled ? variationKey : 'off';

      experimentAttributes[`flag_${flagKey}`] = value;
      identifyWithExperiments(userId);

      // Event names should be stable and few; the variation lives in the
      // attribute, not in a proliferation of event names.
      window.hj('event', 'experiment_viewed');
    },
  );

  const user = optimizely.createUserContext('user_123', { plan: 'pro' });
  const decision = user.decide('checkout_redesign');
  console.log('Variation:', decision.variationKey);
});

The typeof window.hj !== 'function' guard is not defensive padding. Hotjar's script is blocked by a meaningful share of browsers and content blockers, and an unguarded call throws inside the notification listener — which, depending on your SDK version and call site, can surface as a failure in code that was only trying to evaluate a flag. The flag decision must never depend on a third-party script loading.

Resist the temptation to encode the variation in the event name — experiment_viewed_treatment, experiment_viewed_control. Hotjar's event list is a flat namespace shared by your whole site, and one experiment per two names becomes unmanageable within a quarter. The attribute is what carries the variation.

Server-Side Decisions

Hotjar ingests only from the browser, so a decision made in Node or Python has to travel back to the page before it can be recorded. Return the variations with whatever bootstrap payload the client already fetches.

const express = require('express');
const optimizelySdk = require('@optimizely/optimizely-sdk');

const optimizely = optimizelySdk.createInstance({ sdkKey: process.env.OPTIMIZELY_SDK_KEY });
const app = express();

app.get('/api/bootstrap', async (req, res) => {
  await optimizely.onReady();

  const context = optimizely.createUserContext(req.user.id, { plan: req.user.plan });
  const decisions = context.decideForKeys(['checkout_redesign', 'new_nav']);

  const experiments = {};
  for (const decision of Object.values(decisions)) {
    // Only decisions Optimizely counted as impressions; a rollout is not an arm
    if (!decision.ruleKey) continue;
    experiments[decision.flagKey] = decision.enabled ? decision.variationKey : 'off';
  }

  res.json({ userId: req.user.id, experiments });
});

And on the page:

async function applyServerDecisions() {
  const { userId, experiments } = await fetch('/api/bootstrap').then((r) => r.json());

  const attributes = {};
  for (const [flagKey, variation] of Object.entries(experiments)) {
    attributes[`flag_${flagKey}`] = variation;
  }

  if (typeof window.hj === 'function') {
    window.hj('identify', userId, attributes);
    window.hj('event', 'experiment_viewed');
  }
}

Send the flag key and variation only. Hotjar attributes are visible next to a session recording, and the user attributes you bucket on — plan, country, account age — are more than a recording needs to carry.

Python Implementation

A Python backend cannot call Hotjar directly, so its listener records the decision for the response and for your own logs. The DECISION callback takes four positional arguments and exposes snake_case keys in decision_info; reading decision_info.get('flagKey') returns None without raising.

import logging
import os

from optimizely import optimizely
from optimizely.helpers import enums

logger = logging.getLogger('experiments')

# Request-scoped in a real service; a module dict here for illustration.
current_request = {}


def on_decision(decision_type, user_id, attributes, decision_info):
    if decision_type != 'flag':
        return

    flag_key = decision_info.get('flag_key')
    enabled = decision_info.get('enabled')
    variation_key = decision_info.get('variation_key')
    rule_key = decision_info.get('rule_key')
    dispatched = decision_info.get('decision_event_dispatched')

    if not dispatched:
        # A targeted rollout is not an experiment arm; sending it to Hotjar
        # would put users in a filter that has nothing to compare against.
        return

    value = variation_key if enabled else 'off'
    current_request.setdefault('experiments', {})[f'flag_{flag_key}'] = value

    logger.info(
        'decision user=%s flag=%s variation=%s rule=%s', user_id, flag_key, value, rule_key
    )


optimizely_client = optimizely.Optimizely(sdk_key=os.environ['OPTIMIZELY_SDK_KEY'])
optimizely_client.notification_center.add_notification_listener(
    enums.NotificationTypes.DECISION, on_decision
)

user = optimizely_client.create_user_context('user_123', {'plan': 'pro'})
decision = user.decide('checkout_redesign')
print('Variation:', decision.variation_key)
print('To send to the page:', current_request.get('experiments', {}))

Verifying the Integration

  1. Load a page as a test user with a flag that is on, and let the session run for a minute so a recording is captured.

  2. In Hotjar, open Recordings and find the session.

  3. Confirm the User Attributes panel lists flag_checkout_redesign with the expected variation.

  4. Confirm the experiment_viewed event appears on the recording timeline.

  5. Apply a recordings filter on the attribute and confirm your session is returned.

  6. Repeat with the flag off and confirm the attribute reads off rather than being missing.

Recordings and attributes appear with a delay, and a session is only captured if the page was sampled. Absence after thirty seconds is not evidence of a broken integration; absence in a recording that exists and shows no attributes is.

Analyzing Experiments in Hotjar

Filter heatmaps to one arm. This is the point of the integration. Open the heatmap for the page under test, apply the attribute filter for one variation, and read it. Then do the same for the other. Comparing two clean heatmaps tells you where attention moved; the blended heatmap you would otherwise have been reading tells you nothing.

Watch recordings from the losing arm. Filter recordings by the variation whose metric fell, and sort by the frustration signals Hotjar surfaces. A dozen of those usually explain a result that a results page can only report.

Trigger a survey at the moment of exposure. Target an on-site survey at the experiment_viewed event, filtered to one variation, and ask a single question about the thing you changed. Asking only the treatment arm gets you an answer that is about the change, not about your site in general.

Compare scroll depth. For a layout change, scroll maps per variation frequently reveal that a section that used to be seen no longer is — a cause that click data alone will not show you.

Troubleshooting

Symptom

Likely cause

Fix

No attributes on any recording

The plan does not include the Identify API

Check the plan before debugging code

hj is not a function in the console

The tracking code is blocked or has not loaded

Guard every call; never let a flag decision depend on it

Attribute missing when a flag is off

variationKey is null for a disabled flag

Send 'off' explicitly

Second flag overwrites the first

Only the attributes in the latest identify call are set

Accumulate them in one object

Attribute list is full

One attribute per flag, across many flags

Retire finished experiments, or use one combined attribute

Heatmap looks contradictory

It is aggregating both variations

Filter the heatmap by the variation attribute

Server-side decisions never appear

Hotjar has no server ingest

Return the variation to the page and identify there

Sessions in the filter exceed Optimizely's exposed users

Rollout decisions forwarded as experiment arms

Skip decisions where decisionEventDispatched is false

Related Reading