In most cases the A and the B in an A/B test really should otherwise mostly identical, but one thing is different.
If almost everything is different, it’s hard to learn for next time what exactly what led to a change in which ever dependent variable your observating.
Yes, lots of comments that seem to not really understand what A/B testing should actually be. I expect the implementation of A/B testing of video content on YT is unlikely to be done well either though so maybe it's correct criticism for incorrect reasons.
Yeah I don't expect most youtubers to understand that, and much of the guidelines may seem to insist upon themselves without understanding the underlying motivation for the whole approach.
It does sound like youtube will be requiring experiments to pass some sort of similarity test before they can be run as an experiment.
I think it'll be interesting to see how it goes.
akst · · focus · HN ↗
If almost everything is different, it’s hard to learn for next time what exactly what led to a change in which ever dependent variable your observating.
sceptic123 · · focus · HN ↗
akst · · focus · HN ↗
It does sound like youtube will be requiring experiments to pass some sort of similarity test before they can be run as an experiment.
I think it'll be interesting to see how it goes.