The method
A/B testing, or split testing, shows version A to one random group and version B to another at the same time, then compares a single outcome such as purchases or signups. Random assignment is what makes it a test: both groups face the same days, prices and news, so a difference in results can be credited to the difference in the versions. Change one thing at a time, or you will not know which change caused the result.
Decide the winning metric before you start. If you pick it afterwards, you can always find some number where your favourite version looks better.
How much data is enough
Results need enough conversions to be more than luck. A rough rule: aim for at least 100 conversions per version before trusting a small difference, and more if the gap is under ten percent. With 20 purchases each, a 15 versus 25 split can easily flip next week. Free significance calculators online will put a probability on the gap; use one rather than eyeballing.
Run the test for full weeks so weekday and weekend behaviour both show up, and do not stop the moment one version pulls ahead, because early leads often shrink.
A/B testing creator videos
Ad platforms offer built-in split tests that divide the audience cleanly. For creator content, the most useful single variables are the opening seconds, the creator, and the closing call to action. Testing a whole new video against another is still useful, but it tells you which video won, not why.
Most teams move from strict A/B tests to rolling creative testing, where many variants run and spend flows to the best. The split test remains the tool for settling one clear question.
How it's used
We split-tested the same video with two hooks; the problem-first hook won at 140 purchases against 96.
Let's not call it yet, we have 18 conversions per side and the gap is inside the noise.