Your Shopify theme is deciding how much you can test
Build time caps your test count, and build time is set by your theme. A heavily customised theme two major versions behind turns a two-day variant into a two-week one, which halves what you can learn in a year regardless of how much you spend on the programme.
The arithmetic nobody runs
Tests per year is runtime plus build time plus approval time, divided into fifty-two weeks.
Runtime is roughly fixed: the median test in DRIP’s database ran 42 days. Approval time is organisational. Build time is the variable you can actually change, and on a difficult theme it can exceed the test runtime itself.
A store that can build and QA a variant in two days runs meaningfully more tests than one taking two weeks, on identical traffic and identical budget. At average win rates near 12% across more than 127,000 experiments, more attempts is the only reliable route to more winners.
What makes a theme expensive to test on
Heavy customisation. Every bespoke section is code that has to be understood before it can be safely altered, and understood again by whoever inherits it.
Version lag. A theme two or more major versions behind will not accept modern sections cleanly and often breaks in ways that consume a day to diagnose.
App-injected markup. When six apps write into the same template, a variant has to be tested against all six behaving correctly.
No staging environment. Building directly on live means every change carries risk, which means every change carries extra QA.
Undocumented custom code. Written by a developer who has moved on, with no comments. The most common and the most expensive.
| Theme condition | Typical variant build | Realistic annual tests |
|---|---|---|
| Modern, lightly customised, staged | 1 to 2 days | 10 to 14 |
| Modern, heavily customised | 3 to 5 days | 8 to 10 |
| Two versions behind | 1 to 2 weeks | 5 to 7 |
| Legacy, undocumented, no staging | 2 weeks+ | 3 to 5 |
The gap between the top and bottom row is the difference between a programme that compounds and one producing a couple of results a year.
The app weight problem underneath
Apps add scripts, and scripts cost speed. Only around 42% of mobile sites pass all three Core Web Vitals, and Google and Deloitte’s analysis of over 30 million mobile sessions associated a 0.1 second improvement in mobile load time with an 8.4% rise in retail conversion and a 9.2% rise in average order value.
Apps also complicate every build. Auditing them improves speed and test velocity at the same time, which makes it one of the highest-return maintenance tasks on a mature store.
List every app, what it was installed to do, and whether anyone can point at evidence it worked. Most stores find three or four nobody can account for.
How to check your own position
Ask your developer three questions. When was the theme last updated to a current version. How much custom code exists outside standard sections. Is there a staging environment that mirrors live.
Then ask a fourth, which is the real test: how long would it take to build a variant of the product page with one section changed. If the answer is more than three days, your theme is the constraint on your programme and no agency can work around it.
What to do about it
Not necessarily a rebuild. Theme work is usually cheaper than it sounds relative to what it unlocks.
Prioritise a staging environment first, because it removes risk from every subsequent change. Then documentation of the custom code, which is unglamorous and permanently valuable. Then the version update, scheduled for a quiet month rather than before peak.
Do this before commissioning a testing programme rather than during one. A programme spending its first quarter fighting the theme is a programme billing you for development archaeology.
How theme condition changes the scope and the realistic test count is set out under Shopify conversion optimization services.
