Switching CRO agencies: what to move and what to abandon
Take three things with you when you change agencies: the test archive, the research, and the tool accounts. Leave the roadmap, because it was ranked against a method the next firm will not use. Most switches lose six months of accumulated knowledge, and almost all of that loss is avoidable with a week of preparation before notice is served.
What the archive is actually worth
Every concluded test is a recorded fact about your customers. Not about the page, about the people. A threshold test that lost tells you something durable about how your buyers weigh shipping cost, and that remains true under a new agency, a new theme and a new platform.
The value compounds because the hit rate is low. Optimizely’s analysis of more than 127,000 experiments puts the average win rate near 12%, and ConversionTeam’s audit of 2,288 tests found 19.1% reached statistical significance per test. A year of testing buys you a small number of confirmed facts at considerable expense. Losing them means buying them again.
Ask for the archive in a format you can actually read: hypothesis, variant description, sample size, runtime, result, and what was concluded. A folder of screenshots is not an archive.
The accounts problem, and why it should never exist
Testing tool, analytics, tag manager, heatmap tool. If any of these sit under the agency’s login rather than yours, the switch becomes a negotiation rather than a handover.
This is entirely preventable and almost never prevented. Put every account in your company’s name at the start of any engagement, with the agency added as a user. It costs nothing on day one and it is the difference between keeping your history and starting over.
If you are reading this mid-engagement and the accounts are not yours, fix it this week rather than at notice. Requests made calmly during a working relationship get handled. Requests made during an exit get deprioritised.
What to leave behind
| Take | Leave |
|---|---|
| Test archive with hypotheses and results | The ranked roadmap |
| Research: recordings, survey coding, themes | Slide decks and monthly presentations |
| Tool and analytics accounts | Their internal prioritisation scores |
| Baseline numbers and cohort data | Half-built variants |
| The decision log | Anything you cannot interpret without them |
The roadmap is the surprising one. It was ranked using one firm’s estimate of effect size and confidence, and a new firm will rank differently because it weighs evidence differently. Handing it over intact usually means the new team either follows a ranking it does not believe or spends a month politely rebuilding it.
Give them the research instead and let them rank it themselves. You will get a better roadmap and a faster start.
The handover week
Book two hours with the outgoing team while the relationship is still cordial. Not an exit meeting, a knowledge transfer.
Cover four things: which hypotheses they never got to and why, what they believe about your customers that is not written down, which tests they would rerun with more traffic, and what they would warn the next team about. That last question produces the most value and it is the one nobody asks.
Record it. The person who attends will leave your company eventually too.
The gap to plan for
A new engagement takes about two weeks to the first test and around six to the first readable result, because the median test in DRIP’s database ran 42 days. So a switch costs roughly two months of momentum even when it goes well.
Plan around that rather than pretending it away. Use the gap for work that needs no test to justify: documented checkout fixes, speed, and the app audit. Baymard’s meta-analysis of fifty studies attributes 48% of cart abandonments to unexpected extra costs at checkout and 19% to mandatory account creation, and shipping those during the transition means the new team starts from a cleaner baseline.
The question to ask the incoming firm
Ask how they would use the previous archive. A firm that wants to read it before proposing anything is taking your history seriously. A firm that dismisses it and offers a fresh audit is selling a restart, and a restart is exactly what you are trying to avoid.
Ask too what they would do differently from the previous team and why. You are listening for a methodological difference rather than a criticism, because a firm that can only tell you the last one was bad has not diagnosed anything yet.
Our handover process, including what we ask for from a previous agency before proposing anything, sits under conversion rate optimization services.
