Hire a CRO agency or build it in-house: the honest comparison
Build in-house when you have the traffic to run several tests at once and the headcount to keep a programme moving for years. Hire an agency when you need velocity now, when the skill gaps are analysis and hypothesis quality rather than execution, or when a single hire would cost more than the whole engagement.
What the in-house version actually costs
A functioning in-house programme is not one person. It is someone who can read data properly, someone who can build variants without waiting on the development queue, and someone senior enough to decide what gets tested. Most brands hire the third role, give them the first two jobs, and then wonder why three tests shipped in six months.
The salary maths is only half of it. The other half is ramp. A new hire spends their first quarter learning your funnel, your customers and your platform before they ship anything meaningful, and if they leave in year two that knowledge leaves with them.
What an agency actually buys you
Pattern recognition across many funnels, mostly. An agency that has run the same test on twenty stores knows which version usually wins and, more usefully, which tests are not worth running at all.
That matters because the base rates are unforgiving. Optimizely’s analysis of more than 127,000 experiments puts the average win rate near 12%. ConversionTeam’s audit of 2,288 tests found 19.1% reached statistical significance. Most ideas fail. Hypothesis quality is the whole game, and hypothesis quality is what experience buys.
The comparison, without the sales pitch
| In-house | Agency | |
| Time to first test | 8 to 12 weeks including hiring | 2 weeks |
| Cost shape | Fixed salary, rises annually | Monthly retainer, cancellable |
| Depth of context | Very high after six months | Moderate, improves quickly |
| Breadth of pattern | One funnel | Many funnels |
| Risk if the person leaves | Programme stops | Continuity is the agency’s problem |
| Best at | Long-horizon platform work | Velocity and prioritisation |
Where each one genuinely loses
Agencies lose on context. Nobody outside your company understands your customer as well as your own team will after a year, and an agency that never asks about margin, returns or supply constraints will eventually recommend something operationally stupid.
In-house loses on honesty. Someone who designed the page is a poor choice to run the test that might kill it. Self-reported agency numbers should be treated with the same suspicion, incidentally. In the Econsultancy and RedEye optimisation survey summarised by Blend, the average reported winner rate was 35% for companies and 39% for agencies, which is far above what audited datasets show. Everybody grades themselves generously.
The hybrid that usually works
Most brands under fifty million in revenue land in the same place. The agency runs the programme, sets the roadmap and builds the tests. One internal person owns the relationship, protects the roadmap from internal opinion, and carries the context. That person does not need to be a CRO specialist. They need to be able to say no to the CEO’s homepage idea.
The failure mode of the hybrid is an internal owner with no authority. If they cannot decline a request from the founder, the roadmap becomes a suggestion box and the retainer gets spent building other people’s opinions.
What a real programme produces
Timeline matters more than talent here. Across the engagements we run, one supplements brand moved subscription take rate from 25% to between 55 and 60% over six months, roughly tripling the subscriber base. Six months, not six weeks, and the result came from a sequence of offer and post-purchase tests rather than any single redesign.
That shape is the argument for whichever option you can sustain for a year. A brilliant in-house hire who leaves in month nine produces less than a competent agency that runs for three years.
How to decide this week
Answer three questions honestly. Do you have enough traffic for concurrent tests, or will you be running one at a time? Can you protect a roadmap from internal politics for twelve months? And would the fully loaded cost of the hire you need exceed the retainer you would pay?
Two yeses and a no points in-house. Anything else points to an agency, or to the hybrid.
If you want to see how an external programme is structured before you decide, our conversion rate optimization services page sets out the audit, research and roadmap sequence.
