Testing velocity: why tests per month is the wrong question to ask a CRO agency

When a founder asks a CRO agency how many tests they will get each month, they are asking about testing velocity, but they usually mean something else: will this make our paid traffic more profitable, and how will we know? Velocity is a fair thing to want, but it is a poor way to choose who to work with.
Most CRO proposals open with output. You will see a line like "four to six tests and two to three landing pages a month", and a price underneath. It looks clear, and it is easy to compare against the next agency's quote. The trouble is that it tells you what will be made, not what will change in your business, and it quietly sets up the wrong conversation for the months that follow.
The term
What is testing velocity?
Testing velocity is the number of experiments a team launches or completes in a given period, usually counted per month. It is a useful health check for an in-house programme, because a team that rarely ships tests rarely learns anything. It becomes a problem when it is the thing you buy from an agency, because it measures activity rather than whether the tests were worth running.
The real question
What you are actually buying from a CRO agency
A founder rarely wants six A/B tests for their own sake. They want to know why acquisition costs keep rising, why fewer first orders turn into subscriptions than they should, or why more traffic is not turning into more profitable revenue. Tests are how you find and prove the answers, so they matter, but they are the method rather than the goal.
That is why we judge every test on revenue per visitor and, for subscription brands, on subscription take rate. Conversion rate on its own can rise while order value or subscriber share falls, and a programme counted by tests shipped will happily call that a win. If you want the detail on that metric, our guide to revenue per visitor walks through it, and if you are new to working with one, start with what a CRO agency does.

Why counting fails
Three problems with buying CRO by testing velocity
The first problem is that volume becomes the scoreboard. If the contract says six tests and the team ships four, the conversation that month is about the missing two, even if one of the four moved revenue per visitor in a way that pays for the whole year. The team starts optimising for things that are quick to launch rather than things that are worth knowing.
The second is that a test count hides the size of each test. Changing a button label and rebuilding a subscription offer are both one test, but one takes an afternoon and the other needs research, new copy, design, a careful build and QA across your subscription app. On Cadence we tested ten different offer structures before the subscription take rate settled at around 70%, and each of those was a substantial piece of work.
The third is traffic. Testing velocity is capped by how many visitors you have, because a store can only run as many tests as its traffic can read in a sensible time, and a promise of six a month either assumes traffic you may not have or pushes the team into calling results early.

A better way to judge it
What to ask a CRO agency for instead
You still need to know what you will get each month, so the answer is not to drop output altogether. It is to separate three things that usually get blurred into one line on a proposal.
A good proposal then gives you an illustrative month rather than a fixed quota. One month might be two offer tests and a new landing page, another might be one large subscription rebuild and the rollout of last month's winner. You should be able to see roughly how much gets built, and the agency should be able to explain why this month looks the way it does.

In practice
How we structure it at PM Digital
On the CRO & Scale Plan your pod is a CRO strategist, copywriter, designer, Shopify developer, QA specialist and project manager. We agree the monthly scope with you up front, based on your traffic and where the biggest opportunities are, and every test runs in AB Genius, our own Shopify testing app, or in Intelligems if your brand already uses it.
Because we are judged on results rather than volume, we also show the record honestly. With Bond Touch, 8 of 13 completed tests won, and the ones that lost still taught us something we used in the next round. If you want to see how a programme adds up over a year, the Cadence case study shows store revenue up 389% in 12 months on our CRO & Scale Plan, with the caveat that the figure is store-wide and includes Cadence's own acquisition, brand and retail growth.
If you are earlier on, or want to try us on one problem first, the CRO sprint is a different kind of project rather than a smaller plan, and we explain the difference in Shopify CRO services: a one-off sprint or an ongoing programme. For what agencies charge and what each fee usually buys, see Shopify CRO agency cost in the UK.
Questions
Speed only helps if each test is sound, so read how long to run an A/B test and how to brief a CRO test before you add more tests to the roadmap.
Frequently asked questions
What is a good testing velocity for a Shopify brand?
Should a CRO proposal include a number of tests?
What should a CRO programme be judged on?

How we work
Built for what happens after the click.
Most brands don't need more traffic. They need the traffic they already pay for to convert.
We work on what happens after the click, on the CRO and Scale Plan. Decisions are driven by the data in the CRO Brain, with PMD Cortex AI supporting the team, and a programme of tests rather than one lucky win.
The founder and the team
Paddy McLarnon, founder, and the team that does the work.
Paddy McLarnon founded PM Digital and has been a name in CRO for the last four years. He takes the first call with every brand himself, and stays close to the strategy. He has spoken at DTC Growth Summit in Miami and New York, and hosts the Funnel First podcast.
Meet the agency
Our team
The in-house team behind the work.
Your pod does the work, and the whole team backs it. Plus you, in a dedicated Slack channel with all of them.


