Red flags in a CRO pitch
Four things in a CRO pitch should stop the conversation: a guaranteed uplift percentage, a win rate quoted without its definition, a case study with no losses in it, and a roadmap presented before anyone has seen your analytics. Each is a specific claim you can test in the room, and each reveals something about how the engagement will actually run.
Red flag one: a guaranteed uplift
Nobody can promise a percentage honestly, because nobody controls the variables that produce it.
The base rate makes the promise arithmetically strange. Optimizely’s analysis of more than 127,000 experiments puts the average win rate near 12%. A firm guaranteeing a result is guaranteeing an outcome from a process where most attempts fail, which means either they intend to measure it generously or they intend to reach it by methods you would not choose.
Ask what happens if the guarantee is missed. If the answer is a fee refund, ask who calculates the uplift. It will be them.
Red flag two: a win rate with no definition
There is no industry standard, which makes an undefined figure meaningless and a quoted one revealing.
ConversionTeam published its numbers under every definition in circulation: 50.5% of its 2,288 audited tests produced a raw winner, 19.1% reached statistical significance per test, and 63.7% was the decisive rate once inconclusive tests were excluded. Same programme, three headline numbers.
Self-reporting inflates further. The Econsultancy and RedEye survey summarised by Blend put the average reported winner rate at 39% for agencies, well above audited datasets. Ask for the definition. A firm that has one will answer in a sentence.
Red flag three: no losses anywhere in the pitch
Most tests lose or land inconclusive. A deck containing only winners is a selection, and the selection is doing work the firm hopes you will not notice.
Ask directly for a test that failed and what they learned from it. A good answer is specific and slightly uncomfortable. A bad answer is a reframed win.
This matters beyond the pitch, because it predicts the reporting. A firm uncomfortable discussing losses in a sales meeting will be uncomfortable reporting them in month three, and you will end up paying for a programme while seeing a highlights reel.
Red flag four: a roadmap before the data
A detailed roadmap produced before analytics access is a template. It cannot be prioritised against your funnel because nobody has seen your funnel.
Prioritisation is the product. Ranking by traffic multiplied by expected effect multiplied by evidence quality is what separates a year of useful tests from a year of cheap ones, and it is impossible without your numbers.
| In the pitch | What it suggests |
|---|---|
| Guaranteed uplift % | Measurement will be generous |
| Undefined win rate | The most flattering definition was chosen |
| Only winners shown | Reporting will be a highlights reel |
| Roadmap before data | A template, not a diagnosis |
| No exclusions mentioned | Scope disputes in month two |
| Nobody asked about margin | They will recommend something unprofitable |
What a good pitch looks like instead
Mostly questions. Margin, return rate, supply constraints, who approves changes, what you have already tried and what failed.
That last one is the strongest single signal. It has no marketing value and considerable diagnostic value, which is precisely why the firms worth hiring ask it and the rest do not.
A good pitch also disqualifies. Somewhere in it is a description of the conditions under which this is the wrong purchase for you. A firm that has never turned down a client has capacity rather than a method.
One more check after the pitch
Ask for a client reference at your stage, not your aspiration. A firm’s best logo is usually its least representative account, and a brand three times your size had a different engagement with different resourcing.
Then ask that reference one question: what surprised you in month two. The answer is almost always more informative than anything in the deck.
The question to end on
Ask who will actually do the work, and whether they are in the room.
The gap between the people who pitch and the people who deliver is the oldest problem in agency work and the easiest to check. Ask to meet the person who will write your hypotheses, then ask how many other accounts they carry. Senior attention does not scale past a handful of brands.
Our scope, our reporting format and the cases where we are the wrong fit are published under conversion optimization services.
