I strongly believe A/B testing users without their knowledge and enthusiastic consent is unethical. If you don't have enough confidence in your changes to make them carte blanche, then don't make them at all. Users don't want their shit changing all the time.
> A/B testing users without their knowledge and enthusiastic consent is unethical.
Yep, and A/B testing as experienced by uninformed, unaware end-users is a dark pattern.
It undermines the perception of (and trust in) continuity which is necessary to make effective use of a tool. The best way I can describe it to the skeptical is: imagine the dials on your car's dashboard rearrange themselves occasionally overnight, and on some commutes to work you suddenly can't work the radio or the AC while moving at ≥35mph. Of course, since the widespread use of touchscreens, that example became very literal.
So the car manufacturer has figured out the "optimal" arrangement of dials and buttons on their dashboard for their preferred levels of user engagement. Great. How many of those users now associate their car's brand with inconsistency? "I can't trust the damn buttons to be in the same place the next time I drive."
I think this is a bad analogy (or a good analogy for bad A/B testing).
I would say it's more like: imagine if your dashboard controls and icons, had occasional tweaks in size and shape that made them slightly harder/easier to use, but over time ended up with controls you found more intuitive and easier to use.
I would never want this unless I needed to give clear and obvious consent to enable it in the first place, and have access to a 'reset to default and never change again' button. Ideally I'd probably also have 'good change' and 'bad change' buttons to point it in the right direction.
This is so far removed from actual A/B testing that happens in the real world that it's unhelpful to even pretend it could be.
OkayPhysicist · · focus · HN ↗
sfRattan · · focus · HN ↗
Yep, and A/B testing as experienced by uninformed, unaware end-users is a dark pattern.
It undermines the perception of (and trust in) continuity which is necessary to make effective use of a tool. The best way I can describe it to the skeptical is: imagine the dials on your car's dashboard rearrange themselves occasionally overnight, and on some commutes to work you suddenly can't work the radio or the AC while moving at ≥35mph. Of course, since the widespread use of touchscreens, that example became very literal.
So the car manufacturer has figured out the "optimal" arrangement of dials and buttons on their dashboard for their preferred levels of user engagement. Great. How many of those users now associate their car's brand with inconsistency? "I can't trust the damn buttons to be in the same place the next time I drive."
sceptic123 · · focus · HN ↗
I would say it's more like: imagine if your dashboard controls and icons, had occasional tweaks in size and shape that made them slightly harder/easier to use, but over time ended up with controls you found more intuitive and easier to use.
Telaneo · · focus · HN ↗
This is so far removed from actual A/B testing that happens in the real world that it's unhelpful to even pretend it could be.