Testing only one variable for each experiment is considered as a best practice when implementing an experience.
- Divide your audience into as many arms as possible.
- Start with multiple hypotheses.
- Assign two campaigns to each experiment arm.
- Test one variable for each experiment.
The correct answer is: Test one variable for each experiment.
Decide on a clear hypothesis before creating an experiment, and only test one variable to help ensure your experiment is clean and yields discernible results.
A common mistake is that more than one variable is adjusted (e.g., you’re looking to test creative, but each experiment arm has a slightly different budget). Doing so makes the experiment “unclean,” as you’re unable to discern which change caused the impact.
Read more here: https://skillshop.exceedlms.com/student/activity/69415-test-and-measure-creative-with-video-experiment/