A/B Testing Analysis4:02 with Anya Mezak
An important part of A/B testing is the analysis that follows the study. In this video we will go over how to formulate that analysis.
VWO A/B Testing Calculator
Additional considerations when reviewing the study outcome
- Are you using multiple sources of data?
- Did your design solution not improve the outcome?
- Did you study the user segments?
FaceBook Feed - Ever notice that your FB feed looks different, but when you mention it to your friends, they don’t know what you’re talking about? That’s likely because you’re part of an experiment in FaceBook is running. Let’s imagine you see a new Live Feed design. What do you think is a possible metric that FaceBook is evaluating? It could be time on site, click-through-rate on their ads or a friend’s post, etc.
Shopping site checkout flow - Let’s say that Amazon wants to know if a slightly darker shade of orange on their Checkout button will increase conversions. Given their size, they might allot just 1% of their traffic to use the new color and after a few days, they’ll have a large enough sample to know if this change is worthwhile.
You need to sign up for Treehouse in order to download course files.Sign up