AB test is using data like user behavior to make a decision.
Which part of the future is better.
AB testing rather than useful for new experiment.
Overview
Example
Choose a metric
Review statistics
Design
Analyze
Examples of when to use A/B testing
・Movie recommendation site: new ranking algorithm
・Change backend-page load time, results users see etc.
・Test layout of initial page
probability
Repeated measurement of click-through-probability
visitors = 1000
unique clicks = 10
click-through-probability ≒10%
Binominal Distribution
@ p = 3/4
mean = P
std dev = √p(1-p)/N
P^ = 16/20 = 4/5
types of outcomes
independent events
identical distribution
