I will kill the suspense, autoplay was more effective.
On one of my websites, Mouthguard Store, which focuses on a variety of dental products but mostly snoring and sleep apnea mouthpieces at the moment, I decided to A/B test autoplay on videos. Almost all the pages on the site have videos that I filmed and edited featuring my father, who is a dentist. We were having some debate about if it was better to autoplay or not and most people we asked seem to say they don't enjoy autoplay so at first I didn't have it.
Being skeptical and generally believing that it's better to test people's actions rather than trust what they say, I wanted to A/B test this situation. To do the testing I found an excellent class library for the project, which runs in C# with ASP.NET. It's called Fairly Certain and allows a simple call to a method to conditionally display the mark up server side. After that a call to another method scores the result, which in my case is a cash money sale. Being one of the rare people to have both a marketing and programming background, I thought this was a great idea. Server side conditional display is better than the Google JavaScript one because it gives more granular control and improves performance.
The results are stored in an XML file and they give a basic UI to you through an .aspx page all for free. The sample size for the test was 1668 unique visitors, 836 were autoplay and 832 non-autoplay. Surprisingly, all 8 of the sales were received were autoplay. That's approximately a 90% likelihood that autoplay is the best way to increase sales on the site. One note, I made sure to exclude testing with browsers where playing the video was an issue and also didn't test on mobile devices since they don't allow autoplay with my video provider.
Prior to testing I had researched the subject of autoplay in E-Commerce. That report from 2009 said autoplay was better at add to cart but worse at increasing sales, just slightly. I assumed this to pan out the same more or less for me but it was completely wrong. One theory I have about why autoplay worked out best in this case is that the product page layout I have resembles a YouTube page. The YouTube page layout invites certain expectations, because they do autoplay, so perhaps people just expect autoplay. Another reason to discredit the report that I had found from 2009 is that I never actually saw the product page they used so assuming it would be similar was highly presumptuous.
The lesson is to test traffic, neither existing reports or what people prefer is necessarily going to increase conversions. A/B testing is the only way to know for sure. Every site has a different audience and layout so testing is imperative. This test went over several weeks, sad how little traffic the site gets right now, so that should have gotten rid of any slant towards days that are more inclined to get buyers.
From here I am going to test if it's better to have related products or not under the product. My theory is that if people don't just jump to a pretty picture but rather stay focused on the product they are seeing, they will be more likely to buy. I'm happy to at least know that autoplay is the clear winner for the desktop audience and hope that my next test will be as definitive.
