A/B Testing Mastery ft. CXL Institute <> Review
This is Part 7/12 of learnings that I gained from CXL Institute’s Growth Marketing Minidegree.
Continuing with A/B testing in this segment you will get to know about 6V research model and some other interesting concepts. Do have a read!
It’s all about finding good ideas to test. Its always the story where winning tests are the ones that are backed by research and data. So here we have a 6V research model to tick all the dots together for a real winner.
— VALUE:
Identifying the Company Values that are important. Something that gives boost to overall business impact.
— VIEW:
It’s all about analytics. Here we gather and study data found from web analytics and even web behavior data.
— VERSUS:
Competitor research is what we do in this part. Find out about your direct competition what they are doing, and market best practices.
— VOICE:
Here we listen to customer voice. It can be in any form, like reviews, feedback, things they say about you on social media or survey data. Maybe go ask this to customer service reps they know all about it.
— VERIFIED:
Check out the scientific research and other models available.
— VALIDATED:
Things that you want to experiment, is that all validated before like in other tests or experiments? if so what was the result?
So with all these points we try to collect more and more data, digging deep and making it more in line with what customer expects out of our brand and why they must choose us. Honestly I can connect the dots with what I previously grasped from the ResearchXL Model. Highlighted talk on research was to learn maximum from competitors and listening to their A/B tests, knowing what they are doing and if it works you can go on and do more educated research to make it better. Clearly copy pasting stuff blindly won’t help much unless you are very lucky.
Look for audience overlap between competitors, you and competitors, maybe go there and buy from them. Feel their services, note every positive thing that makes your journey as a customer seamless and do something with the things that you note and apply it on your site. Another important part is to track changes on their website and see how they are improving and what is the reason behind those upgrades or downgrades. Are they just doing it to be in trend or they are going with the flow like bringing cool animations and updating some colors randomly or is there any message they are trying to convey with aesthetic design. Looking too premium can sometime intimidate crowd.
There’s this tool called VisualPing that helps you keep track of your competitors and it will tell you the slightest of change made by them. You can also gain insights on what tools and technologies they are using by checking on BuiltWith. Furthermore, you can use some specialized google chrome plugins to see what tests they are running, they are available with almost every service.
You can even move to a traditional method of researching which is via scholarly articles, research papers and other such sources. Maybe even read some case studies in your niche to get an overview and idea on what can be done next. Some sources that you can look for are Google Scholar, Semantic Scholar and Deep Dive.
Next part is about formulating a hypothesis. It’s like if I change X then what all things can go right that are going wrong in current version. Three major things that you include while writing a hypothesis is The problem, the proposed solution and the predicted outcome. Your tests must clearly show how do you want success and the way to measure it.
It’s like, if I change X then more people will relate with the landing page copy and thus it will convert more people. So if you put a relevant image near your tag line then it will drive more people to signup for your email news letter.
Another important part that you note down is what group of users will the change that you are about to do will impact in a positive way. Sometimes the answer will be its better for everyone then it might not be an A/B test because that feature (X) is essential, it could be something like making your website responsive now that is a standard practice.
Now that you did an A/B test successfully and the results are in your hands, so when do you implement those changes? how do you know that these results that you hold are not rigged or are full of false positives?
There’s this Simpsons paradox that comes in to play which suggests that you must not divert or try to increase traffic on an on-going A/B test just so you know to maybe make it short or something like that.
Here, you can see a trend in data and it will clearly show that B is winning but in reality you don’t know that if it is actually winning because A didn’t have that spike like ever. So be considerate while running the tests and more importantly have patience. Which means to let them complete its cycle. Plus if you want to shorten it you can do so by designing A/B-test is such a way that it will lead to a bigger effect. Like instant high ticket purchases by making some unicorn level changes.
There’s also pre and post A/B testing, mostly to ensure that it is suitable (pre) done with small set of users, maybe internally and the post is with live set of audience generally also gives you an idea that code isn’t broken or to make sure errors does not exist.
In this course I learned some useful tricks to get started quickly but professionally. Most importantly it encourages you to try it with your own business. Even with ResearchXL we saw that A/B testing is super important. It also important to understand that using tools is great, it’s like the shiny object that lets you setup quickly, but in the longer run, only the research backed tests will be of some value.
Next I took on Landing Page Optimization. The reason being, I can test different landing pages with A/B test plus after I setup a page I will jump on to analytics section so I will have something to run experiments.
Signup for A/B testing mastery course at CXL