Imagine you’re in a monthly management meeting, reviewing the numbers for your business. You’re looking at sales numbers, and you see this:
| Last month | This month | % change |
|---|---|---|
| 231 | 190 | -17.7% |
Cripes! Sales down over 17%, a double-digit decline in a single month. That doesn’t look good. Why is that? You’d better go and have some strong words with the sales team, get them to explain their under-performance, and fire them up to do better next month.
The next month rolls around, and you return to the numbers in the management meeting. Having delivered your rousing speech and kicked some sales-team butt, how are things looking now? Have your motivational techniques had the desired effect?
| Last month | This month | % change |
|---|---|---|
| 190 | 229 | +20.5% |
Brilliant! A 17% decline has become an incredible 20% improvement. You’re a management genius, the sales team are great, pats on the back all around.
Except… how much of this variation is actually down to your actions, and how much is the result of random chance, of events outside your control? Might this change have happened anyway? The sales team are subject to all sorts of forces they’re not able to influence, and not everything about their results is determined by their performance – or, indeed, your management of them.
The problem here is that, with the information we’re given, we lack an understanding of the natural variation within the numbers. Only by understanding that can we know whether a decline of 17% is a reason to act – or just business as usual.
The simplest way of doing that is to zoom out, and look at things over a longer period of time. A process behaviour chart is simple to generate, and gives us all the information we need in order to understand what variation is natural and what variation isn’t. Here’s a process behaviour chart for 2.5 years of sales data:
Let’s look at two regions: the January–March 2023 range, that sparked our interest in the month-to-month reports, and then an earlier period:
The jumps in January, February and March of 2023 (1), while large in percentage terms, are actually well within the bounds of what’s expected from natural variation, as shown by the upper and lower limits. There’s no reason to be concerned by the decreases, or elated by the increases. Taking action here is likely to be the wrong thing to do.
But what else do we see from the chart? Actually, there was a problem here, but it happened over a year earlier. There was a period from August 2021, that lasted almost year, that was well outside the boundaries of what we’d expect from natural variation (2). There are nine consecutive values that are below the average (which is significant), and one that is completely outside the lower process limit, indicating it’s out of control. Something happened that caused a meaningful decline in sales.
But the difference from month to month is often actually very small, and crucially we see months where things look like they’re improving, too. If we’re looking at percentage changes month-to-month, we might never notice this problem, or think that things were improving again. And so it’s easy to imagine a business that didn’t realise this meaningful change had happened at all, and so missed an opportunity to fix it.
In this case, things seemed to recover, albeit after several slow months. But it’s easy to imagine more disastrous consequences. Only by understanding the variation within the numbers you’re measuring can you understand whether changes are worth paying attention to – and so avoid either acting unnecessarily, or missing something big.