A few months ago, we got together a cartoonist and a developer to create a crazy-ass parallax scrolling page. This particular piece of beauty is probably the easiest guide to A/B testing on the web. It is really good and we were mighty excited to share it all over the interwebs. As part of that, we first shared it on HackerNews.
Pretty sweet huh? The post was smartly shared on HN in the morning when most of the US was asleep and it stayed on HN’s front-page for almost the entire day. We got a lot of comments and our traffic saw an immediate (but obviously temporary) spike.
Even better, our active trials (the CTA at the end of the page was to sign up for a free trial of Visual Website Optimizer) shot up like there’s no tomorrow!
So all this felt really nice. We spread this in some other places and got interesting responses. I happily reported to the CEO that we’re reaping significant benefits and the campaign is working well.
The truth unfolds
This is what our active free trials looked like after the campaign (time frame: start of campaign to three months later).
At the end of the day, this was a push that bumped up our ‘free trial’ signups, but couldn’t sustain that number. Kind-of obvious. One can’t expect HN to always be following you. But what was surprising was that not one of those free trials converted to a paid account. That’s right, not one single convert from all the HN traffic.

The mistake
We ran after vanity metrics like page views and “sign up for free trial” (yeah, in this case that’s a vanity metric). The bigger mistake was that the we got carried away with our work and ran to show it off to those who would appreciate the technical aspects of it. In the process, we neglected the Small Business Owners for whom it was originally meant, and who would actually be influenced by the page. The readers of HN understand A/B testing very well and don’t need a simple story/analogy like Bob. But small business owners worldwide would have really connected with it.
Unfortunately, very few of them actually got to see it.
What we learnt from this
Here’s what we (and you!) can learn from this:
- HN is great to introduce new technical concepts (we created a cool parallax page to explain A/B testing and introduce Visual Website Optimizer)
- HN might be good if your product solves the problems of technical people
- Before creating any marketing campaign/message, think very carefully about the target audience and how you want to reach them
- At the end of the day it is all about being relevant
Here’s another nugget of insight that we gleaned from the conversation that happened on HN.
Found this useful?
If yes, I suggest you sign up for my newsletter. I’ll continue to send more such stuff focused around SaaS and startup marketing. Sign up here.
Did you convert any people outside the HN audience?
This is definintely an area that we’re focusing on, it’s so hard to drive signups that it gets to the point where you constantly need to scrutinise your onboarding process to get that user to the “A-Ha” moment as fast as possible 🙂
Hi Stuart,
We got free trial signups, but the conversion rate took a nosedive because of the excessive HN traffic. Now we’re working on our content strategy so that this page is presented as the “Absolute Beginner’s Guide to A/B testing” (because of obvious reasons).
Thanks for dropping by 🙂