Run A/B Tests on a Contentful Site with Optimizely Web Experimentation

Loading...·7 min read

TL;DR

Client-side A/B testing assumes something that a Contentful-backed site quietly breaks: that the content the variation edits is already in the HTML when the variation runs. On a server-rendered page it usually is. On a React or Next.js front end pulling entries from the Contentful Delivery API at runtime, the element your selector targets may not exist for another two hundred milliseconds, and a variation that runs first applies nothing at all.

That is the whole difficulty of this integration, and it is a timing problem rather than a Contentful problem. This guide covers where the Optimizely Web Experimentation snippet goes on a headless front end, how to make variations wait for content without reintroducing flicker, how to model variation copy in Contentful so editors own it, and when the honest answer is that the test belongs in a server-side implementation instead.

How Optimizely Web Experimentation Fits a Contentful Site

Contentful is a content API. It renders nothing and emits no browser events; it hands JSON to a front end, and that front end decides what the page looks like. Optimizely's snippet sits in that front end's document head, decides a variation before paint, and then has to cooperate with a render it does not control.

flowchart TD
    A[Request for a page] --> B[Framework renders shell]
    B --> C[Optimizely snippet decides the variation]
    C --> D{Is the content already in the HTML?}
    D -->|SSR or SSG| E[Variation applies immediately, no flicker]
    D -->|Client-side fetch| F[Target element does not exist yet]
    F --> G[Contentful Delivery API responds]
    G --> H[Framework renders entries]
    H --> I[Variation re-applies on mutation]

Why a Headless Front End Changes the Timing

Optimizely applies a variation's changes as soon as it can and then watches for the elements it could not find. That watcher is what saves a client-fetched page, but it also means the visitor sees the original content first and the variation after — which is flicker by another name, and it is worse on slow connections precisely where your conversion rate is already worst.

Static generation makes the problem disappear. If pages are built at deploy time with content baked in, the variation runs against complete HTML and behaves exactly as it would on a traditional CMS. If your front end uses incremental static regeneration, you are in the good case for most requests and the bad case for the first request after a revalidation.

Two Places a Variation Can Live

A variation can change the rendered DOM, or it can change the content the renderer is given. The first is what Web Experimentation does natively. The second requires the front end to ask Optimizely which variation is active before it fetches, and then request different entries — which is a genuinely different architecture and closer to feature flagging than to visual testing.

Most teams should start with the first and move to the second only when a test needs to change content that arrives asynchronously and cannot be made static.

Prerequisites

  • The Optimizely Web Experimentation snippet URL for your project.

  • Control of the front end's document head — app/layout.tsx in a Next.js App Router project, _document.tsx in the Pages Router, index.html in a Vite app.

  • A content model you can extend. Adding a variation field to an existing content type is a schema change, and on a live space it needs the usual care.

  • Custom events created in Optimizely for the conversions you intend to record, with API names noted. An event pushed under a name that does not exist in Optimizely is discarded without an error.

  • Agreement on what editors own. The most common failure of this integration is organisational: an editor updates the entry a running experiment depends on, and the test's control quietly changes underneath it.

Step 1: Put the Snippet in the Framework's Document Head

The snippet must be synchronous and must be the first script in the head. In a Next.js App Router project that means a raw <script> element in the root layout rather than next/script, because every next/script strategy either defers or hydrates late.

export default function RootLayout({ children }: { children: React.ReactNode }) {
  return (
    <html lang="en">
      <head>
        <script src="https://cdn.optimizely.com/js/YOUR_PROJECT_ID.js" />
      </head>
      <body>{children}</body>
    </html>
  )
}

Framework warnings about synchronous scripts in the head are correct in general and wrong here. The cost of the blocking request is the price of not showing the visitor the wrong content first.

Step 2: Make Variations Wait for the Content

For anything the client fetches, write the variation so that it is idempotent and re-runs when the content arrives. Optimizely's own utilities give you the wait; the important part is that applying the change twice does nothing the second time.

var utils = window['optimizely'].get('utils');

utils.waitForElement('[data-contentful-entry="hero"] h1').then(function (heading) {
  if (heading.getAttribute('data-variation-applied') === 'true') {
    return;
  }
  heading.textContent = 'Ship experiments, not opinions';
  heading.setAttribute('data-variation-applied', 'true');
});

Two things make this hold up. The selector targets a data-contentful-entry attribute rather than a styling class, so a design refactor does not break it. And the guard attribute means a React re-render that restores the original text and triggers the watcher again does not fight with the variation forever — a loop that is easy to create and unpleasant to debug.

Step 3: Model Variation Content in Contentful

Hard-coding variation copy in the Optimizely editor works and is the right choice for a one-week test. For anything longer, or anything an editor should be able to correct, put the alternative copy in Contentful and let the variation select it.

A minimal content model for this adds one field to the content type under test:

{
  "name": "Hero",
  "fields": [
    { "id": "headline", "name": "Headline", "type": "Symbol", "required": true },
    { "id": "experimentKey", "name": "Experiment key", "type": "Symbol", "required": false },
    { "id": "variationHeadlines", "name": "Variation headlines", "type": "Object", "required": false }
  ]
}

variationHeadlines holds a map from Optimizely variation name to copy. The variation code then reads the entry that the page already fetched instead of carrying the text itself, and an editor fixing a typo fixes it in both arms without touching the experiment.

This has one real cost worth stating: the control arm's content is now editable while the test runs. Agree with the content team that entries with a non-empty experimentKey are frozen until the experiment ends.

Step 4: Measure Conversions Without a Page Load

Headless front ends route on the client, so a conversion metric defined as "pageview of /thank-you" will not fire. Push a custom event at the point the conversion actually happens.

window['optimizely'] = window['optimizely'] || [];
window['optimizely'].push({
  type: 'event',
  eventName: 'lead_form_submitted',
  tags: {},
  properties: { 'Content Type': 'hero', Source: 'pricing-page' },
});

Optimizely reserves two tag names: revenue, which must be an integer number of cents, and value, a plain number. Everything descriptive goes in properties, and each property name has to exist in the Optimizely UI first — five predefined names plus ten custom slots, fifteen per event. A property sent under a name nobody created is dropped without an error, which is the same silence you get from a misspelled event name.

Gotchas

Preview mode. Contentful's preview API serves draft content, usually on the same front end. Preview sessions are real browser sessions and will bucket into experiments. Exclude the preview host or the preview query parameter from targeting, or an editor reviewing drafts becomes part of your sample.

Locales. Contentful returns different fields per locale, and a variation targeting text will not match a translated page. Target attributes rather than text, and scope experiments per locale where the copy differs.

Rich text is not HTML. Rich-text fields arrive as a document tree the front end renders. A variation that rewrites the rendered output of a rich-text field will be undone the next time that component re-renders. Guard it as in Step 2, or move the change into the entry.

Draft entries in the results. If a test starts before its variation entry is published, the variation arm renders the control content and the experiment measures nothing but noise. Publish first, then activate — and read the first day's results page with that in mind.

Verifying the Integration

Load a page with the network tab open and confirm the Optimizely script is the first request after the document. Then throttle the connection to a slow 3G profile and reload: this is where a client-fetched page shows the original content before the variation, and it is the only reliable way to see the flicker your visitors get.

Next, confirm idempotency. Trigger a client-side route change back to the same page and check that the variation is applied exactly once and the guard attribute is present. Finally, fire the conversion event by hand from the console and confirm it appears against the right variation, then leave the experiment in draft for a day and check that both arms are receiving traffic in the ratio you configured before you trust anything the results say.