Comment by DanielHB

I worked at an almost-medium-sized company and we did quite a lot of A/B testing. In most cases the data people would be like "no meaningful difference in user behaviour". Going by gut feeling and overall product metrics (like user churn) turns out to be pretty okay most of the time.

The one place that A/B testing seem to have a huge impact was on the acquisition flow and onboarding, but not in the actual product per se.