In this tutorial, I’ll show you how to setup and run your first A/B test in Optimizely. We’ll start with a very simple experiment to test to see whether our call-to-action (CTA) is more effective for conversions as a button instead of a link on your page. For further ideas for A/B testing, see the links at the end of this post.
On the home page, you’ll have a list of all your previous experiments (if any). You can filter these by status or by typing in the name of the experiment. Ensure you give your experiment a relevant name, so it is easy and relevant to find later!
Give the experiment a name (make it something memorable and relevant to what you are testing) Enter the URL of the page where the experiment will sit.
Rename variables to describe what the test is doing so it makes sense when you come back later on. Click on the dropdown and select Rename Variation to do this.
For our test, we wanted see if the “Join us” call-to-action would be more effective for conversions as a button instead of a link within the bullet points as show below. This page has since been re-designed but is a nice reminder that A/B testing is key to constantly improving and iterating your pages based on customer feedback.
And here is the final button we implemented. You’ll notice we also changed the wording from “Join us” to “Join CAF Give As You Earn” as it was much more descriptive and assured the user what they were signing up for.
To place your button on the page, it is best practice to click on a link, button or wherever you’d like to add your new element, and click Select Container and then choose the div or container that is most relevant. In this case I had a plain div around my button.
Selecting the Division “div” makes an outline around the button like so:
Here it is again once selected:
We do this because selecting the whole div and editing the HTML is much easier for editing and positioning things on the page.
I can then edit the styles around the div to position the button easily. Don’t use the Move & Resize option as it’s unreliable - edit the HTML instead.
This is so important as it will allow us to identify our element so we can create goals against it. Make it “a unique class name” and this should be done before you set up a goal.
Right-click → Edit Element → Edit Classes
Select the button → Track Clicks → Create New Click Goal
Give the goal a name (make it relevant to what you are tracking!) and in this case we are tracking clicks on that button.
Next click on Advanced at the bottom of the page
In the selector box we should see our newly created class name.
Back in the original tab, in our example the two things you want to test against each other should have the same class, in our example, join-us-link.
You can switch to another page within the Optimizely editor. Go to Options –> Editor –> Settings and type in the URL to navigate to this page.
On our Join us page there is a download link which downloads a PDF of an application pack. We’d like to track how many people click this so let’s create a goal for this.
Give your element a class name - make that “a unique class name”
Click on the link → Edit Element → Edit Classes
Complete the following fields like this:
Click on Start experiment to put your test live!
Now the fun part starts, you need to make sure the 2 versions of our test are working correctly so open up a new window (preferably in another browser or Optimizely will log you out). You’ll need to clear your cookies and open up the page where your test sits. Repeat the process of clearing cookies and refreshing until you see both versions - it may take a number of attempts :)
In Internet Explorer, to clear your cookies go to Settings → Internet Options → General tab → under browsing history click Delete and make sure cookies are ticked.
You’ve started your first A/B test - congrats! Now keep an eye on the results over the next few days to make sure everything is running as expected.
Senior UX Designer @cafonline. Previously Senior Designer @newsfromfta. I blog about UX, design, and the latest tech.
Let's design something incredible together.