A/B testing and Google Analytics experiments

Published on August 9, 2016 by Edwin Guilbert

A/B testing has been a growing trend among web content editors and marketers in the last few years. It's not a secret that high performing webpages will lead to more page visits, lower bounce rates and more conversions, which usually means more revenue. However, we need to find out what we need to achieve with a specific page or content and try to measure it before performing any comparison tests.

We can measure how the web page is performing through metrics. Some examples are:

  • Call to Actions (clicking on a button, playing a video)
  • Product purchases
  • E-mail signups

From these metrics you could calculate conversion rates, which basically state how many visitors have turned into customers in a given time frame. Examples of conversions could be visitors buying products on an e-commerce website or readers clicking on advertisements in a blog website.

You may be wondering what A/B testing has to do with any of this? Well, the answer is simple: In A/B testing you could improve your conversions by comparing different versions of the same page. The idea is to change key elements that might improve your conversion rate. You usually have a original page (a so called control page) and a variation of it. The users will get any of these versions randomly and after a period of time you can compare which version did better according to a goal (or conversion).

But we're not here just to explaining how A/B testing works. We actually want to do some A/B testing in a real web page with a concrete CMS and analytics software. So we will use Magnolia CMS for managing web pages and Google Analytics Experiments for recording/analyzing statistics.

In Magnolia you can create variations of a page and edit them independently (this is only available on Enterprise editions.)

We are going to work with the demo website travels, specifically with the "about" page which contains a video. The goal here would be get this video played more times, so the metric we are going to use will be the play event on this element of the dom.

The original page looks like this:

And the variation will have a wider video, removing the left side links which might distract the visitor and prevent her from playing the video (our goal in this case.)

To create a variation of a page, you just have to select "Add page variant" in the pages app of Magnolia.

Open the variant created and update the video component:


After publishing the variant, you need to connect these pages to Google Analytics using a Javascript snippet for tracking page views. You can embed this snippet with Marketing Tags in Magnolia (don't forget to include it in all your pages):



After publishing the snippet (or marketing tag in Magnolia) you need to send an event for every time a user plays the video included in the pages we want to test, so we need to create another marketing tag for that:



$(".video-wrapper video").on('play',function(){


After publishing the snippet, you need to actually record the number of play video events as conversions or "goals" in Google Analytics:





After the goal is configured, you might want to test it in the real time tab of Google Analytics, so every time you play a video in your page, an event gets registered as a conversion:




At this point you finally have everything prepared to start your A/B testing on the page variants measuring video played metrics as a goal. You can automatically create experiments (as Google calls them) in Magnolia using the A/B testing module:


As shown in the screenshot below, you can define the name of the experiment, the page, and variants involved, the duration and the goal for the test. The goal is dynamically retrieved from the Google Analytics account configured:



The module will create a Javascript snippet as a marketing tag, which you need to publish, so Google can forward users to one variant or the other:



After publishing the snippet you can already start the experiment in Google Analytics through the A/B testing module:



And that's all folks, you can always check your statistics in Google Analytics behaviour tab:



I hope you enjoyed my first post and find this useful. I will write a second part of this post explaining how to do A/B testing manually without using the A/B testing and Personalisation modules  (both need the Enterprise version of Magnolia).



{{item.userId}}   {{item.timestamp | timestampToDate}}

About the author Edwin Guilbert

Edwin Guilbert is a solutions architect and trainer at Magnolia in Madrid, Spain. He joined Magnolia in 2014, having worked almost 10 years in consultancy as an architect and functional analyst. He regularly trains developers worlwide as he creates and conducts custom on-site workshops at client locations. He is also constantly involved with the current Spanish market needs as he carries out demos and POC developments for pre-sales.

See all posts on Edwin Guilbert