Integrate Crazy Egg with Optimizely Web Experimentation

Loading...·6 min read

TL;DR

An A/B test tells you which variation won. It does not tell you why, and on a losing variation that is the only question anybody actually wants answered. Crazy Egg's heatmaps, scrollmaps and recordings answer it — but only if you can look at them one variation at a time. A heatmap that averages both arms of a running experiment is a picture of nothing: it shows where people clicked on two different pages superimposed.

This integration is therefore narrower than most on this site, and more useful than its size suggests. It is one field in a custom analytics integration, and what it buys is the ability to open Crazy Egg, filter to variation B, and see that nobody scrolled far enough to reach the button you moved. It pairs naturally with reading the Optimizely Results page — the results tell you the size of the effect, the heatmap tells you the mechanism.

What Integrating Crazy Egg with Optimizely Actually Means

Crazy Egg records where visitors click, how far they scroll, and what they do in a session, and groups those recordings into snapshots. It has a small API for attaching your own labels to a visitor's session, called user variables, and those labels become filters in the Crazy Egg interface.

flowchart TD
    A[Visitor loads a page] --> B[Optimizely snippet decides the variation]
    B --> C[track_layer_decision fires with campaign and decision]
    C --> D[Wait until the Crazy Egg script is ready]
    D --> E[CE2.set writes experiment:variation as a user variable]
    E --> F[Crazy Egg records the session with that label]
    F --> G[Heatmaps and recordings filterable by variation]

Why This Integration Runs One Way

Every other integration on this site forwards events into Optimizely. This one does not, and the reason is worth stating plainly rather than leaving as an omission: Crazy Egg does not emit an event stream into the page for anything to subscribe to. It observes, it batches, and it sends its own data to its own servers. There is no crazyegg.on('click', …) to wrap.

So the only meaningful connection is outbound — Optimizely telling Crazy Egg which arm this session belongs to. If you also want click data as an Optimizely metric, that is a separate job: instrument the click yourself and push a custom event, exactly as the custom analytics integration guide describes.

Prerequisites

  • The Optimizely Web Experimentation snippet URL for your project.

  • The Crazy Egg script installed on the pages you intend to observe, with your account's script id.

  • A free user-variable slot. Crazy Egg gives you five, numbered one to five, and they are account-wide. Agree which one belongs to Optimizely before you write the code, because two teams writing to slot one will overwrite each other.

  • A snapshot already running on the page under test. User variables label sessions; they do not create snapshots.

Step 1: Load Both Scripts in the Right Order

Optimizely first, in the <head>, synchronously. Crazy Egg's script can load whenever it likes — the integration below waits for it.

<head>
  <script src="https://cdn.optimizely.com/js/YOUR_PROJECT_ID.js"></script>
  <script type="text/javascript" src="//script.crazyegg.com/pages/scripts/0000/0000.js" async></script>
</head>

Crazy Egg's tag is legitimately asynchronous: it observes, so arriving a moment late costs a moment of observation and nothing else. The Optimizely snippet is not, because a variation applied after paint is a variation the visitor saw the wrong version of first.

Step 2: Tag Crazy Egg Snapshots with the Optimizely Variation

Optimizely Web Experimentation calls a custom analytics integration once per decision, with campaign and decision in scope. The track_layer_decision field holds a top-level script, not a function body: a bare return in it raises 'return' outside of function and the integration never runs at all.

{
  "plugin_type": "analytics_integration",
  "name": "Crazy Egg User Variable",
  "options": {
    "track_layer_decision": "var experimentName = campaign.name;\nvar variationName = decision.variation_name;\nvar label = experimentName + ':' + variationName;\nwindow['optimizely'].get('utils').waitUntil(function () {\n  return window.CE2 && typeof window.CE2.set === 'function';\n}).then(function () {\n  window.CE2.set(4, label);\n});"
  }
}

Three details in nine lines, each of which is a bug if you get it wrong.

The state lookups happen beforewaitUntil. campaign and decision are in scope when the integration is invoked; the deferred callback runs later, and code that reaches for them inside it is depending on a closure that may already have moved on. Reading them first and closing over plain strings is what makes the pattern safe.

The wait exists because Crazy Egg's script is asynchronous and window.CE2 will frequently not be defined at decision time. Calling CE2.set optimistically works on a fast connection and fails on a slow one, which produces the worst kind of bug: one that only affects the visitors whose behaviour you most wanted to see.

And the label is a single composed string rather than two variables, because Crazy Egg's user variables are flat strings. Encoding both parts with a separator lets you filter to one experiment or to one arm of it, and keeps a second concurrent experiment from making the first one's label ambiguous.

Step 3: Read a Heatmap by Variation

In Crazy Egg, open the snapshot for the page under test and apply a user-variable filter on the slot you chose. You now have two heatmaps of the same URL, which is the point.

What to look for depends on the result you are explaining. If the variation lost, the useful comparison is the scrollmap: a change that pushes the primary call to action below where 50% of visitors stop scrolling explains a loss without any further investigation. If the variation won but by less than expected, compare click distribution — a new element that draws clicks away from the conversion path is a common and easily missed cause.

Recordings are worth the time only after the maps have narrowed the question. Watching twenty sessions at random is a poor use of an afternoon; watching five sessions of visitors who reached the button and did not click it is not.

One statistical caution. A heatmap is not evidence of an effect, only of a mechanism. It is generated from a sample Crazy Egg chose, not from your experiment's population, and clicks are not independent of one another. Use it to form the next hypothesis, and use the experiment to test it — the hypothesis template is a reasonable place to write that next one down.

Gotchas

The user variable is set per session, not per pageview. A visitor who is bucketed into an experiment on one page carries the label for the rest of the session, including onto pages that were never part of the experiment. That is usually what you want for recordings and misleading for heatmaps of other pages.

Five slots, account-wide. If another team is already using all five, this integration has nowhere to write. Auditing them is a five-minute conversation that saves an afternoon of silent failure.

Snapshots have their own sampling. Crazy Egg stops a snapshot after a configured number of visits. On a high-traffic page a snapshot can complete before your experiment has been running a day, in which case the filtered heatmap covers a window that does not match the experiment's.

Redirect experiments break the mapping. If the variation is a different URL, the two arms produce snapshots of two different pages, and the user variable adds nothing that the URL does not already tell you. That is fine — just do not expect a single filtered snapshot.

Concurrent experiments overwrite the label.CE2.set on the same slot replaces the previous value. With two experiments running on one page, the last decision wins, so treat the label as "the most recent experiment" unless you allocate a slot per experiment.

Verifying the Integration

Activate the experiment, load the page, and check in the console that window.CE2 becomes defined and that your slot holds the composed label — Crazy Egg exposes the current values on the CE2 object. Then throttle the connection to a slow profile and reload: this is the case the waitUntil exists for, and it is the only way to see it work.

Then wait for real traffic. Crazy Egg needs sessions before a filter has anything to show, and a filter that returns nothing an hour after activation usually means low traffic rather than a broken integration. Confirm by checking the same slot on a handful of recordings before you change any code. Finally, cross-check the two arms' filtered visit counts against Optimizely's own visitor counts: they will differ, because Crazy Egg samples and Optimizely does not, but a ratio near zero on one arm means the label is not being written for that arm at all.