Algeria AgencyConversion
A/B testing
Comparing two versions to settle a question on data rather than opinion — when your traffic supports it, and only then.
An A/B test compares two versions of the same page across comparable visitors and lets the numbers decide. It is the only way to replace an internal argument with an answer, and it is also the method easiest to misuse.
The condition of use is volume. Below a certain number of visitors and conversions, a test cannot distinguish a real effect from ordinary fluctuation. Many test reports are in fact presenting randomness, and they drive bad decisions with the confidence of numbers.
So we refuse to test when the traffic will not support it. That is not excessive caution: a badly concluded test is worse than no test, because it closes a question that was open and justifies a change that improves nothing.
When the volume is there, the method is strict: a hypothesis written beforehand, one difference tested, a duration set in advance, and a result accepted even when it contradicts what everybody expected. That last part is the difficult one.
What we usually find
- Your decisions are made on the preferences expressed in the last meeting.
- You "tested" two versions for three days and concluded something.
- You change several things at once and do not know which one worked.
- A supplier showed you a conversion gain that never appeared in your sales.
What changes
A question settled
An internal debate replaced by a measured answer, which ends the discussion for good.
Justified changes
What gets deployed is deployed because it produced an effect, not because somebody preferred it.
Cumulative learning
Every test, successful or not, teaches something about your visitors that informs later decisions.
What you get
Feasibility calculation
Before any test: how many visitors and how long are needed to conclude anything at all.
A written hypothesis
What we think, why, and what we will do depending on the result — written before launching.
One difference only
Two versions differing in a single element, otherwise the result attributes nothing.
Duration fixed in advance
Stopping a test when it turns favourable is the most common way of producing a false gain.
Measured on real conversion
Enquiries and sales rather than clicks, because a gain in clicks can hide a loss in quality.
Documented results
What was tested, what came out of it, and what we concluded — failures included.
How we work
Check feasibility
Current volume and rates, to see whether a test can conclude and in what time. Often the answer is no.
Write the hypothesis
What we are changing, why we think it acts, and what we will do in each case.
Build both versions
One difference, verified on mobile and inside social app browsers.
Let it run
To the planned duration, without checking every morning, because checking encourages concluding too early.
Conclude and deploy
The result accepted as it stands. An inconclusive test is written up as inconclusive, not rounded.
Is this the right fit for you?
This is for you if
- You have enough conversion volume to conclude within a few weeks.
- You accept that a result may contradict an internal preference.
- You have a precise question to settle, not a general wish to improve.
This is not for you if
- Your traffic is low. Qualitative work will produce more, and we will say so.
- You want to test ten things this month. One test at a time, or nothing is attributable.
- You will deploy your preferred version regardless. The test then has no purpose.
What we commit to
The calculation before the test
We tell you first whether a test can conclude for you. When the answer is no, we do not sell it.
The duration is not shortened
Stopping as soon as a result looks good is the main source of false gains. We hold to the agreed duration.
A null test is reported as null
Most tests show no difference. Saying so is what makes the other results believable.
Why few Algerian sites can test
Most sites we audit receive a conversion volume that will not conclude in a reasonable time. That is not a fault: it is the reality of a market where many businesses are starting. It simply means the right method is elsewhere.
What produces at that traffic level is qualitative work: watching five people use the page, fixing speed, cutting a form, showing a price. Those corrections have effects large enough to be visible without statistical testing.
Messaging complicates the arithmetic too. A share of conversions happens in a conversation, sometimes days later. A test counting only form submissions measures a fraction of the result, and can label a winning version as the loser.
Frequently asked questions
How many visitors do we need?
It depends on your current rate and the size of effect being sought. We run the calculation before proposing a test, and it often concludes there is not enough volume.
How long does a test run?
Long enough to cover several weekly cycles, because behaviour varies by day. Too short a duration measures a particular day rather than a trend.
Can we test several things at once?
Not without substantial volume. With two simultaneous differences, a result does not say which one acted, and you may deploy the one that was harming you.
What do we do with an inconclusive test?
Report it as such and move on. It is a frequent and useful result: it stops you deploying a change that adds nothing.
Can we test ads instead of pages?
Yes, and it is often more accessible: advertising platforms generate volume faster than a site page does. We sometimes start there.
Do testing tools slow the page down?
Some do, and by enough to distort the very result they are measuring. We check the speed impact before launching, particularly for a mobile audience.
How to start
Tell us how many enquiries you receive each month and on which page, and what question you would like settled.
We come back with the calculation: whether a test can conclude for you, in what time, and if not, what we would do instead.
What we have written on this subject
A/B testing: what your traffic lets you conclude
The question is not which version wins, but whether you have the volume to know. Nearly always the answer is no — and there is better work available.A year of changes, no test: the log that makes them readable
Without the volume to test, the one thing that turns modifications into decisions is a dated register, kept at the moment of the change.Landing page: the same sentence as the ad
The page’s first line has to repeat the ad’s promise word for word. It is the biggest leak and the cheapest one to repair.Analytics: five numbers, defined, every month
A useful report fits on one page and carries the same five numbers every month. The rest describes the agency’s activity, not yours.
Let us talk about your project
A free audit, no commitment: we look at your online presence and tell you what is holding it back.
- The calculation before the test
- The duration is not shortened
- A null test is reported as null