This actually isn’t a post on Netroots information, I read this fascinating article from Wired Magazine on how one of the core Google tools, the A/B test, is transforming business and now, other arenas as well. The reason I want to bring it up here is because I realize how quickly it’s become a common part of the conversation here at Netroots. Panelists are talking about A/B testing, nested A/B testing, and now multivariate A/B testing.
So, what is it? Google is built on a core of testing everything they do, if you have data to prove something is better, do it. The primary way they get data is by live testing different versions of something, an ad, a landing page, an email format, whatever. So they put version A and version B up on the web and then measure which one got a better response (time spent on the page, more satisfaction, a desired clickthru, etc.). That’s A/B testing. According to Wired mag so much of Google is a permanent ongoing mega multi level A/B test that we’re all their nonstop guinea pigs. Also according to Wired, when a Google staffer ported this tool to a presidential campaign it was credited with generating $75M in additional donations. Why? Because they’d been making decisions like so many of us do, by taking the highest paid person’s opinion (HiPPO). Once they substituted live A/B testing, data showed the HiPPO was often very incorrect. Nicely on the internet you can measure precisely how much response an ad gets, so you know x% click thru with version A, and y% with version B. That’s how they came up with the result that substituting A/B testing generated $75M in additional revenues. Seriously, read the Wired mag article, fascinating.
So, I wonder how this might impact health next? Are any of us already using A/B testing for our health promotion campaigns? What about in our for policy change engagement work? And what about NIH? Obviously clinical trials can’t be handled this way, but what about all the work to modify health behaviors? Or even data collection? Strikes me that A/B testing gives a lightning fast measure of efficacy — wonder when it’ll get to health and what it will change for us?
LGBT HealthLink, The Network for Health Equity 