Run A/B Tests on Vercel with Optimizely Web Experimentation

On most hosts, installing Optimizely Web Experimentation is a snippet in the head and a conversation about flicker. On Vercel it is a snippet in the head, a conversation about flicker, and a much more interesting conversation about the CDN — because Vercel's whole value proposition is serving a response it prepared earlier, and an experiment is a promise that two visitors get different responses.

This guide covers installing the Web Experimentation snippet on a Vercel-hosted site, preventing flicker when the framework hydrates over your variation, keeping the edge cache from serving one visitor's variation to everybody, and forwarding conversions as custom events. Size the test on the sample size calculator before you build the variation, and read how long to run an A/B test if the number surprises you.

How Web Experimentation Runs on a Vercel Site

Web Experimentation is a client-side product. The snippet loads synchronously in the head, decides the visitor's variation in the browser, and mutates the DOM before paint. Vercel's caching sits entirely upstream of that: the CDN decides which HTML the browser receives, and the snippet then decides what to do with it.

flowchart TD
    A[Request] --> B{Vercel edge cache}
    B -->|HIT| C[Cached HTML returned]
    B -->|MISS| D[Function renders HTML]
    D --> C
    C --> E[Snippet runs synchronously in head]
    E --> F[Variation applied before paint]
    F --> G[Framework hydrates]
    G --> H{Hydration overwrites the variation?}
    H -->|yes| I[Reapply on a mutation observer]
    H -->|no| J[Stable page]

Those two decision points — the cache and hydration — are what makes this different from a plain static host, and they are the two sections most worth your attention.

Prerequisites

The Optimizely Web Experimentation snippet URL for your project.
Head access in your framework. In Next.js App Router that is app/layout.tsx; in the Pages Router, pages/_document.tsx; in Astro, Nuxt or SvelteKit, the equivalent document template.
Custom events created in Optimizely, with API names noted, for every conversion you plan to count.
A view of your caching configuration: which routes are static, which are ISR, and what Cache-Control your functions send.

Step 1: Load the Snippet Synchronously in the Head

Put the snippet first in the head, before stylesheets and before any framework script, and load it synchronously.

// app/layout.tsx
export default function RootLayout({ children }) {
  return (
    <html lang="en">
      <head>
        <script src="https://cdn.optimizely.com/js/YOUR_PROJECT_ID.js" />
      </head>
      <body>{children}</body>
    </html>
  )
}

Two things that will be suggested to you and that you should refuse. next/script with strategy="afterInteractive", which is the framework's default advice and is wrong here: it loads the snippet after hydration, which guarantees a visible flash of the original page. And a tag manager, which adds its own async load in front of Optimizely's — the snippet must be the first blocking script or it cannot beat the paint.

The cost is real: a synchronous script in the head blocks rendering until it returns. That is the trade Web Experimentation makes, and the mitigation is a small snippet, not an async one. Keep unused experiments archived so the project's JavaScript stays small.

Step 2: Stop the Cache From Serving One Variation to Everyone

The snippet decides in the browser, so a cached HTML response is not itself a problem — every visitor runs the snippet and gets their own variation. The problem is the reverse case: server-side personalisation cached at the edge.

Check for these three, in order:

A route that renders differently per visitor and is cached. Anything reading a cookie, a geolocation header or a session, on a route with revalidate set or a Cache-Control: s-maxage header. That response is shared. Add cache: 'no-store', or move the varying part into a dynamic child inside <Suspense>.
Vary headers. If you vary a response by a cookie you set yourself, the CDN needs Vary: Cookie to key the cache correctly — and a high-cardinality cookie makes the cache useless. Prefer deciding in the browser, which is what the snippet already does.
ISR revalidation windows. A page regenerated every sixty seconds is still one page for everybody in that minute. Fine for Web Experimentation; fatal for anything you rendered per-variation server-side.

The honest summary: let Optimizely decide in the browser and let Vercel cache freely. Mixing a server-side variation into a cached route is where the results become fiction.

Step 3: Survive Hydration

A React, Vue or Svelte app renders HTML on the server, and then, in the browser, reconciles that HTML against its own component tree. Your variation changed the DOM between those two moments, so the framework can and will revert it.

Symptoms: the variation appears and vanishes within a few hundred milliseconds, or works on a hard refresh and disappears on client-side navigation.

Three fixes, in order of preference:

Change something the framework does not own. CSS injected by the variation, a class toggled on <body>, a style rule keyed to a variation attribute. Nothing to reconcile, nothing to revert.
Reapply after hydration. Wrap the DOM change in Optimizely's utils.waitForElement, or observe the container with a MutationObserver and reapply if the framework overwrites it.
Read the variation and let the app render it. For anything structural, expose the variation to the application and render the difference in the component. At that point you have a hybrid, which is fine — the decision is still Optimizely's.

Client-side navigation deserves a line of its own: on a single-page app, the next page does not reload the snippet. Enable the SPA/dynamic-website handling in your Optimizely project so re-activation happens on route change, or trigger it yourself from the router.

Step 4: Forward Conversions as Custom Events

<script>
  window['optimizely'] = window['optimizely'] || []

  function trackConversion(eventName, tags) {
    window['optimizely'].push({
      type: 'event',
      eventName: eventName,
      tags: tags,
    })
  }
</script>

Call it where the conversion happens — a form's success handler, a route change to /thank-you, an add-to-cart callback:

trackConversion('signup_completed', { plan: 'pro', revenue: 4900 })

revenue is an integer in cents. Every other tag is an event property and must exist in Optimizely first, or it is discarded at ingestion without an error. The event name must exist too — pushing an unknown name is a silent no-op, which is the single most common reason a correctly built experiment reports no conversions.

Step 5: Keep Preview Deployments Out of the Data

Every push to Vercel produces a preview URL running the same snippet as production. Left alone, your own team's preview traffic joins live experiments.

Two options. Restrict each experiment's page targeting to the production domain, which is exact and requires no code. Or gate the snippet on the deployment environment:

{process.env.NEXT_PUBLIC_VERCEL_ENV === 'production' && (
  <script src="https://cdn.optimizely.com/js/YOUR_PROJECT_ID.js" />
)}

The second is cleaner but costs you the ability to QA a variation on a preview URL, so most teams use page targeting and keep the snippet everywhere.

Verifying the Integration

View source on a production URL and confirm the snippet is present and not deferred. curl it rather than reading the browser's DOM, which shows you the post-hydration result.
Check the cache header.x-vercel-cache: HIT is fine for a client-side experiment and a red flag on any route you personalise server-side.
Force a variation with the optimizely_x query parameter and confirm it survives hydration for at least five seconds and one client-side navigation.
Watch the network tab for logx.optimizely.com after a conversion. No request means the event name does not exist in the project.

Common Failure Modes

next/script instead of a plain tag. Guaranteed flicker; the most common mistake on this stack.
A cached route that varies per visitor. Quiet, and it invalidates the results rather than breaking the page.
Variations reverted by hydration. Looks like a flaky experiment; it is a reconciliation.
No re-activation on client-side navigation. The experiment works on the landing page and nowhere else.
Preview traffic in production experiments. Small sites can move a result with a single afternoon of QA clicks.