A common question we get from new merchants is:
Can I run A/B tests on my headless storefront?
The short answer is, absolutely!
A/B testing is beyond the scope of Nacelle's product offering, but we hope this document helps inform the conversation.
A/B Testing & Progressive Roll-Out Best Practices
For further reading, please consider Optimizely’s documentation on the test design pattern.
SEO, Caveats & Other Considerations
When split testing between two separate URLs there are a couple of considerations to keep in mind:
301 (Permanently moved) redirects are considered more SEO friendly than other redirect methods.
rel=canonicaltagging on your control pages helps Google’s Search engine better understand the presence of URLs with duplicate/similar content.
To prevent event attribution problems during the test, it might be worthwhile to exclude paid search traffic from the test by excluding requestors who have a
utm_campaignparameter included in their request.
You should consult an SEO expert before starting this kind of experiment to ensure your specific scenario is well understood.
Keep in mind that new browser features like Safari’s Intelligent Tracking Prevention will place limits on how long a cookie can be present on a user’s browser. Using
Set-Cookiein the response header of the original request is the most durable method for extending the life of the cookie.
A/B Testing - React on Vercel
For those using React and Vercel, Vercel has a great article on how to build experiments at scale.
Client-side vs Server-side testing
The good people of medium.com wrote a worthwhile piece on client-side vs server-side testing. It is more philosophical than directly applicable, but it is helpful if you are debating between these two options.