In July I wrote about a hotel group where pricing, marketing and an easy year-on-year comparison all claimed the same good quarter. Add the claims up and they came to more than the actual gain. The question was how to tell which team was right.

That example was fictionalised, but I've sat through a version of it when I ran digital at Premier Inn. Every team had numbers and every set looked good. The easiest way for marketing to hit its targets was cheap traffic that barely converted, and nothing in our measurement would have flagged it, because traffic was up and traffic was the metric. Product, marketing and pricing each had data showing they'd driven the same revenue. Nobody was lying. Their tools measured correlation, and correlation is generous with credit. The loudest case won the budget, and when someone asked whether an investment had worked, the honest answer was that nobody knew.

The usual answer is to test. And a well-run A/B test does answer the causal question, for one lever, at one time.

The trouble is that each test knows nothing outside itself. It tells you what your change did, on that audience, in that quarter, with every other team's activity frozen exactly where it happened to be. What works at peak won't necessarily work off peak. What marketing's test measured shifts once pricing moves. So every team ends up holding a valid local answer, and the answers still don't fit together. There's nothing for them to fit into.

And there's a question tests can't reach at all. Looking back, we want to know how much of the result was down to us. Looking forward, we want to know what will happen if we act. If we cut prices again, does revenue rise, or have we trained customers to wait for discounts? A test run last quarter can't tell you what this quarter's change will do once every other team has moved too.

What's missing is the thing the tests plug into: a model of how your business works, built with the teams who run it and agreed by them. It maps how the moving parts drive the outcomes in your business, rather than a template of your industry. Each team's test and each team's claim feeds into the same structure. Every contribution comes with a range, the contributions have to add up to the result the business actually got, and you can see what teams achieve together as well as alone. It acts as the umpire, and the dashboards report into it.

That's also why twenty years of better tools haven't fixed this. You can't buy a model of your business off the shelf, because the structure is different in every business, and finding it means careful work with the people who run it. It's a discipline more than a product, and very few organisations have it yet.

So the question from July still stands: can you tell who's right, or are you backing the team with the best story? Testing gets you part of the way. The rest needs a structure that everyone's numbers have to agree with.