Conversion rate optimisation when you haven't got the traffic to test

A client emailed me last year wanting to A/B test the colour of his enquiry button. He'd read something. Green versus orange, he wanted to know which one we thought would win, and could we set it up.

I asked him to send through his analytics before I answered. About three hundred and fifty sessions a month, most of them on mobile, roughly seven enquiries.

So I did the sum for him instead, and that email turned out to be more useful than any test would have been.

The arithmetic that ends the conversation

Say your site converts at 2% and you'd be happy to find a change that lifted it to 3%. That's a 50% improvement, which would be a genuinely large win, and it's the sort of gap a button colour will never produce anyway.

Put those numbers into Evan Miller's sample size calculator, which is the one most people in this field actually use, and it'll tell you that you need somewhere around four thousand visitors per version. Eight thousand people through the site before you're allowed to believe the answer.

At three hundred and fifty a month, that's about two years. And that's for a big effect. Chasing something realistic, a few percent either way, and the number climbs past anything worth calling a plan.

The version of this that actually happens is worse, because nobody waits two years. They run it for a fortnight, one variant is ahead, they call it, and they've learned nothing. Statistically the fortnight result is noise with a colour attached to it. I've watched people make real decisions off that noise and then wonder why the number didn't move.

None of which means testing is rubbish. Above a certain volume it's the best tool there is, and if you're running an online store with real ad spend behind it, test everything. It's just not a tool you own yet.

What people are actually buying when they hire a CRO consultant

Worth saying plainly, because "conversion rate optimisation" as a service has drifted a long way from what most Australian small businesses need.

The discipline was built for high-traffic ecommerce. Test infrastructure, statistical rigour, an iteration cycle. Sold down to a business with a few hundred visitors, what you get is that same process running on data too thin to support it, and a report full of confident recommendations that came from the consultant's judgement rather than from your visitors. Which would be fine, honestly, if it was described that way.

At your traffic level, the useful work isn't inference. It's observation. Nobody sells that as a package because it doesn't scale and it doesn't look sophisticated in a deck.

You probably can't tell me your conversion rate

Here's the bit that catches nearly everyone, and it has to be fixed before any of the rest matters.

Ask a small business owner what their conversion rate is and you'll usually get a shrug, or a number derived from the contact form and nothing else. Meanwhile the phone rings, people message the Facebook page, someone replies to a quote from three months ago, and a good share of the enquiries the website generated never touch the website's form at all.

Until enquiries are counted, and counted at the point they arrive rather than remembered at the end of the month, you're optimising something you can't see. That's most of what we set up on the analytics side, and it's unglamorous: form submissions recorded properly, calls from the site attributable, and one place where every enquiry lands so you can count them without relying on memory.

A month of honest counting will teach you more than a year of testing you were never going to finish.

Ring the last five people who enquired

This is the thing I'd do first, and almost nobody does it.

Call five people who contacted you recently, including one or two who didn't end up buying, and ask what nearly stopped them. Not "how did you find us". What nearly stopped them. Whether they had to go looking for something. Whether anything on the site made them hesitate.

You will get five specific answers, immediately, for free. I've had a client discover that half his enquiries thought he only worked in one suburb because that was the only place named anywhere on the site. Another found that people were ringing to ask a question the site answered, which sounds like a win until you realise it means the answer was buried where nobody found it.

Five phone calls is a sample size of five, and it wouldn't survive ten seconds of statistical scrutiny. It's also the richest data you will ever get about your own website, because these are the actual humans, and they'll tell you things no test could isolate.

Then watch some sessions. Most analytics tools will record real visits, and twenty minutes of watching thumbs move around your site on a phone will make certain problems obvious in a way no report ever does. People give up in the same place. You'll see it.

The leaks you don't need evidence to fix

Some things are broken regardless of what any test would say, and you can just go and fix them.

If the site is slow, that's costing you people before they've read a word, and it's not a preference question. We went through what that actually costs in why your website is slow. If your form asks for a project description and a budget before anyone's spoken to you, it's too long. If your prices are hidden entirely, some proportion of people are leaving to find someone who'll tell them. The full list of these lives on our conversion rate page and I won't repeat it here, because the checklist isn't the hard part.

The hard part is that fixing them feels unsatisfying without a number proving it worked, and you won't get that number. You have to make the change because it's obviously better, and accept that your evidence is a slow drift in enquiries over the following quarter rather than a chart with a winner on it.

If you only do one thing this month

Count your enquiries. All of them, from everywhere, for four weeks, in one place, even if it's a notebook next to the phone.

Then ring five of them. That's the whole program, and it's more conversion work than most businesses your size have ever done.

If the counting says you're getting the traffic but not the enquiries, that's a site problem and worth attacking properly. If it says almost nobody is landing on the site in the first place, then conversion isn't your bottleneck and you've just saved yourself a lot of money going the wrong direction, which is the order of operations we'd argue for anyway. We'll look at your site for free and tell you which of those two it is, and occasionally the answer is that neither one is the problem and the phone just isn't being answered.