The rhythm of the business

First: I’ve had issues over the last few weeks accessing my Substack account. I was worried it had been hacked, but all is good now. Expect a flurry of catch-up posts over the next week! Here’s the first …

Generative AI (I'm talking about you, Claude and ChatGPT) loves to start articles with phrases like In this fast-paced world of data ... But, two complaints I commonly hear are that business users are overwhelmed with data and at the same time they are under pressure to be even more fast-paced. They want help with the volume of decisions and analysis thrown at them but also want to make decisions closer to real-time.

My view is that the drive to consume more analytics and for more real-time decision-making is often a mistake because it blindly assumes that more and faster is better. It's not.

The cadence of our analytics and decision-making should be geared to the response time of the process. To put it another way …

By this, I mean the time period during which taking action could meaningfully affect the outcome of a process.

• In manufacturing, if there's a quality issue, your impact window might be minutes or hours while the defective product is still being made

• For a marketing campaign, your impact window could be days or weeks while you can still adjust messaging or targeting

• For strategic planning, your impact window might be months or quarters, while you can still change course

So, analytics should be carefully designed around this rhythm of the business, focusing on how to deliver the right data at the right time and in the right format for your purpose. If we do this well, we can both reduce the anxiety of missing out on important information and improve our timely decision-making.

There are numerous factors involved when professionals feel overwhelmed by the sheer volume of data that they are expected to act upon. I will focus here on the overload that stems from the current trend for real-time decision-making.

You've got rhythm

The rhythm of a business is its natural operating cadence, which determines, among other things, the rate and volume of data collection and the need for decisions and actions.

Different industries and functions within a business exhibit distinct rhythms, from rapid production lines to slower processes, such as workforce planning. Perhaps it's better to think of the rhythm of a line of business or the rhythm of a business function.

When we align analytics with these rhythms, we can greatly reduce the stress on decision-makers because they'll not be overwhelmed by irrelevant data or by information they cannot act on.

Let me take an extreme example, but one which, for many reasons, I am happy to share. My son works in the Scotch whisky industry as a distiller, and his work involves several processes that operate at very different cadences.

Nothing like where my son works!

You may be familiar with the aging process of whisky, where any spirit must be matured for at least three years. But high-end and rare whiskies (the kind my son is working with) may mature for 10 to 15 years, 30 years or even 50 for the rarest and finest. This aging process is governed by long cycles because changes in the product, such as its flavor profile and the quality of its maturation, are slow and gradual. Decisions may include when to sample barrels for quality checks, when to bottle and release specific batches and how to manage stock strategically to meet forecast demand years in advance. These are slow-moving decisions: the processes involved are gradual, and the actions to be taken in response to a decision are also relatively unhurried.

Not only is it impractical to provide "real-time" updates, but it would also be unhelpful because the system cannot respond at a speed that matches the changes.

However, the whisky business also includes processes that have a short-term operational rhythm. For example, warehousing and storage conditions, such as temperature and humidity, must be tightly controlled. Alarm systems monitor for environmental anomalies, and decisions in response to these alarms are immediate and tactical. These may involve adjusting environmental controls or urgent actions to prevent loss of stock or damage to infrastructure. The business needs a much faster rhythm of information and decision-making in these operations.

Tasks beyond the casks

Of course, the whisky industry is exceptional in many ways and, to be fair, exceptionally interesting to me. But there are other examples. In manufacturing, production lines are often high-speed, and they require real-time data to detect and resolve issues with equipment or quality defects as they occur.

However, management of the very same factory faces strategic decisions that operate on much longer timescales, such as planning their product roadmap. While the production line needs minute-by-minute monitoring (or even faster), decisions about which products to develop or deprecate unfold over quarters or years.

The analytics supporting these decisions draw on slower-moving signals: market trends, technology evolution, and changing customer preferences. Real-time data would be meaningless here - you need time to spot patterns, validate trends, and ensure you're responding to genuine shifts rather than noise.

A manufacturer trying to make strategic pivots based on daily or weekly data risks overreacting to short-term fluctuations. The impact window for strategic changes - the time between decision, action and results - might be 12-18 months, so the analytics supporting these decisions should match this longer rhythm.

A smooth finish

It's crucial to note that even when data arrives frequently - like daily sales figures or continuous customer feedback - this doesn't mean we need to make decisions at the same pace. For strategic planning, we might collect real-time market data but deliberately analyze it on a quarterly cadence, smoothing out daily noise to reveal meaningful trends. The key is not how often data arrives but how often we can meaningfully act on it. A streaming feed of customer sentiment is valuable, but strategic product decisions based on that data might still best be made during scheduled quarterly reviews when we can step back and see the fuller picture.

This need not be complex analytically. Simple moving averages can transform high-frequency data into a rhythm that matches your decision-making cadence. For example, while your website might collect customer feedback daily, a 90-day moving average could provide a clearer signal for strategic product decisions. The key is choosing an analysis window that aligns with your ability to respond meaningfully to changes in the trend.

A simple moving average smooths out the stress of daily fluctuations in data, if not the stress of falling sales!

Who could ask for anything more?

The problem of data overload - or that odd new noun "overwhelm" - often arises because businesses deliver more data than decision-makers can effectively use. In their rush to be data-driven, organizations can overload their teams with real-time feeds and instant alerts that don't match their actual ability to take meaningful action.

If you can align analytics with the rhythms of the business, you can reduce stress and improve decision-making. As we've seen, different processes demand different cadences of analysis. Even simple techniques like moving averages can help transform high-frequency data into insights that match your decision-making rhythm.

Balancing real-time and right-time analytics ensures that decision-makers receive the right data at the right time, empowering them to act confidently and clearly. The key is not how quickly data arrives but understanding when and how you can meaningfully respond. Simply put …