Not so fast: why premature enumeration spoils your analytics
When I work with enterprise clients, I often ask permission to sit in on some meetings as a silent observer to see how they use data and analytics during their work. There’s a typical pattern …
Every meeting begins with fumbling: projectors, screens, connectivity, dial-ins, audio and video. It’s not just you. I have been in meetings at Microsoft, Cisco and Google (who build meeting tech), and they still faff around for the first five minutes.
Then the meeting properly starts. In teams that aim, or claim, to be data-driven, I will generally see data up on screen quickly. It could be a slide with charts, a table, a live dashboard, or often a spreadsheet. And there’s always something to call out - a trend, a pattern, some signal in the data worth paying attention to. Perhaps augmented analytics have already highlighted this. Look - sales in the western region are falling while others are stable or increasing. The meeting digs into the data - slicing, dicing, drilling down and across. And then the debate starts.
Decision defense, not decision support
The ensuing discussion, however, is rarely data-driven. It’s not as if people pull up more detailed or additional analyses. Instead, the topics are nearly always about what is not captured in the data. Our marketing doesn’t resonate with the western region’s demographics; the weather is terrible; the weather is good; the regional economy is different; the sales team is new and inexperienced; or the sales team is too entrenched and unable to change. There’s a lot of defensive talk if the numbers are wrong. If the numbers are good, there’s a lot of praise and projection for the future. But it’s all anecdote and what we must now call alternative facts.
Data doesn’t enter the conversation. Indeed, the conversation rarely, if ever, returns to the numbers. Instead, it moves forward based on our emotional reaction to the numbers and the analysis.
How can business intelligence or analytics teams handle this? Unfortunately, the most common answer is the least effective. Data analysts try to pre-empt contextual conversations by doing more up-front analysis to be better prepared.
This exemplifies what the economist Tim Harford calls premature enumeration. We short-cut the process of context-gathering and dig into the data immediately, with more analysis. But premature enumeration leaves us unsatisfied and frustrated. Only then do we explore broader conditions and external factors - more as a form of decision defense than decision support.
Treating premature enumeration
Part of the problem here is that intelligent use of data is an iterative and collaborative process.
● Orient yourself in the data and its context.
● Glimpse something interesting.
● Bring in more context and more conversation.
● Gather more data.
● Rinse and repeat.
However, in our businesses, we treat the presentation of findings as a one-shot opportunity to make a case or defend a decision with data. We create the illusion of being data-driven without actually leveraging data effectively.
We can do better.
Sometimes, the best way to get to the right answer quickly is to start slowly.
For one thing, take it slowly. Even in a crisis - perhaps especially in a crisis - pressure will not help. Try not to over-analyze, but remember that the data is only meaningful with its full context. So, sometimes, distracting yourself a little can help to put off premature enumeration. Don’t rush into crunching numbers, but explore the context more fully, anticipate where the emotional conversation will go and appreciate that understanding metadata, business circumstances and economic climate are just as important, if not more so, than yet more statistics.
It’s also helpful to talk through issues with stakeholders before any meeting where the data will be discussed. I’ll write more about consensus building in my following Creative Differences newsletter. But, the more context you can bring to the data before discussion, the more time can be spent discussing the implications of your findings rather than trying to set that context.
This doesn't mean we should dally or delay decisions unnecessarily. But it does mean we should resist the pressure to prematurely enumerate and instead create space for collaborative sense-making. Sometimes, the best way to get to the right answer quickly is to start slowly.
Another fine post, Donald. Agree completely. I think the “data-driven decision-making” fad has been a huge distraction for a lot of people.
Data are the shadows on the wall of Plato’s cave — weightless loosely-correlated byproducts of the real drama unfolding outside the cave in the push/pull, cause/effect, give/take of customers and products and channels and employees and competitors and regulators and supply chains. Data are AT BEST the information scent of something real that’s interesting (ht Peter Pirolli, Information Foraging Theory).
And remember, data are not a lever: you cannot grab hold of an ROI statistic and move it upward, however much you might like to have that result. If you want higher ROI, you need to interact with real things in the real world, to adjust how they relate to each other through policy and process, such that… if you are right and/or lucky… the ROI statistic later begins moving in the desired direction.
All the drama and all the levers of change are outside the cave of data-shadows.
Another interesting post, thanks Donald. Perhaps the solution involves less speaking and more written reflection. In my experience, people tend to be more considered and deliberate when they express their ideas in writing rather than through conversation. It might be beneficial to share the dashboard before the meeting, allowing everyone to arrive with their thoughts already articulated in writing. Additionally, sharing these insights prior to the meeting could enhance discussions and productivity.