The Surprising Lesson This Blue Cow Can Teach Us About Success at Ecommerce
This is an image of a Belgian Blue cow. It has anywhere from 40% to 100% more meat than a regular cow and 70% less fat. It is an engineered species created by selective breeding over several generations.
What can this cow teach us about increasing online sales? Well, 85% of all A/B tests designed to improve sales fail [1] because they do not produce a meaningful, or statistically significant lift. As a result, A/B testing tends to waste precious time, effort and money.
But there is a better way and it is inspired by selective breeding. Seeking to produce an optimal cow, a farmer first started with a skinny one and after several generations of selective breeding, he eventually got the Belgian Blue.
The Birth of Evolutionary Algorithms
In the 1950s, the computer scientist Alan Turing (famous for breaking Nazi codes during World War II) posed the following question: “Could computers solve problems using algorithms inspired by natural selection?" It turned out the answer was yes — and thus was born the field of evolutionary algorithms, also known as genetic algorithms.
One of the earlier profitable uses of these algorithms was in the pharmaceutical sector in the late 1990s to early 2000s. They were used to discover New Molecular Entities, or NMEs, that had certain desirable properties. These NMEs then become candidates for new drugs.
My first real job was as a research programmer in the pharmaceutical industry, and it was fascinating to see how computers 'found' — with the help of evolutionary algorithms — unlikely winners that human scientists alone never could have, by searching exhaustively through the vast ‘solution space.’ Some of these NMEs turned into blockbuster drugs, reaping huge profits for the firms involved.
At the time, using evolutionary algorithms was a prohibitively expensive endeavor that required massive computational power. With the ability for one blockbuster drug to bring in tens of billions of dollars in new revenue, using genetic algorithms made sense for BigPharma but not for most other industries. Today, in comparison, computing power is dirt cheap.
CRO — also known as LPO (Landing Page Optimization) — is usually done by running a series of A/B (or multivariate) tests, in which you split your visitor traffic between two or more pages and then track which page produces the most sales, leads or app downloads. By keeping the winner each time, you can increase your overall conversion rate, and as a result, improve your sales and profits.
Most A/B and multivariate tests done using traditional tools (Optimizely, VWO, Google Experiments) fail to produce a meaningful conversion lift that leads to more sales. If you run an A/B test between one page that reports a conversion rate of 1.00% and another that reports 1.06%, neither page is statistically better than the other and it’s time to start all over again with a different set of variations.
So How Do Evolutionary Algorithms Solve This Problem?
Evolutionary algorithms allow companies to run many more experiments simultaneously, selecting those that show weak positive signals and then “crossbreeding” them to produce stronger positive signals — much like the skinny cows that were bred to produce successively beefier cows, eventually creating the ‘optimal’ Belgian Blue.
Evolutionary algorithms for CRO can run tens of thousands of variations in a single experiment (versus at most a handful with traditional A/B or multivariate testing).
Using the example above, let’s say the baseline (‘control’) conversion rate is 1% and the evolutionary algorithm has 2,000 variations in the first generation. It might select pages that are showing weak positive signals of 1.0023%, 1.0000058%, 1.00000041% and “crossbreed” these to produce the second generation, with slightly stronger weak positive signals.
Repeat this over several generations and you eventually get variations with much stronger positive signals (1.34% or 1.45% say) and a high degree of statistical significance (above 95%), meaning the improvement could not be due to chance. If you run a $100 million revenue business, that would be $45 million in extra revenue. Crucially, your profit increases disproportionately because the incremental revenue didn’t require additional media/traffic spend.
While the initial improvements in the first generation are tiny, over hundreds of generations, the gains compound because they are multiplicative (rather than additive). Einstein famously quipped that compounding was the eighth wonder of the world.
This all happens automatically, dramatically increasing the speed at which you can run new experiments, while saving you time and effort.
In the example below, 381,000 different treatments were tested in about 8 weeks. You couldn’t accomplish the same thing using traditional A/B testing tools even in 100 lifetimes! “If you want to be twice as successful, run twice as many experiments” Jeff Bezos often advises, and credits much of Amazon’s success to this practice.
Putting It All Together: A Case Study
To give a real world example, here’s a case study and a simulation of how the evolutionary algorithm works. Below is a screenshot of the initial “control” page being optimized. It is a lead generation form in the medical education industry.
To run the evolutionary algorithm, first you create a “test matrix” and seed it with values for each of the variables being tested. For example, you populate the matrix with four different headlines, three different hero images and several other variables shown in the table below. We have not shown all the values used, but instead show just the final winning ones.
Test Matrix Used
Results
On the right is the winning design, found after testing 381,000 variations in 8 weeks. It produced a 45% conversion lift.
The Technology Is Only as Good as the Human Skill Using It
While AI-powered CRO tech is revolutionary for the optimization industry, it takes human skill to get it to perform optimally — seeding the test matrix with high probability values makes all the difference. As with many other exciting technologies, the GIGO (Garbage In, Garbage Out) principle applies. According to the book The Second Machine Age, a combination of a skilled human and an artificial intelligence system will always trump either working alone.
Jasper Kuria is the Managing Partner of the Conversion Wizards, a consultancy that specializes in CRO (Conversion Rate Optimization). He is also the co-host and producer of the popular podcast How We Made It In Ecommerce.
Footnotes
[1] Various lectures (recorded) during the A/B Testing Summit, Mixpanel and VWO
