Everyone wants to dance with Big Data. We are in love with its velocity, volume, and variability. It is an exciting partner that hypnotizes us with the allure of all of its possibilities. The market lavishes money on it and everyone wants to say they know it.
The reality is a little more mundane. Often, it has 5 o’clock shadow and wears grandpa shoes.
Big Data is far more often about decision made in the many ‘normal’ moments and much less about eureka moments delivered with trumpet flourishes. The dimensions of decision making are nuanced and complex. To make very good real-time decisions, arguably the most valuable part of Big Data, requires a very good understanding of…ready for this?…business requirements and process.
We live in an age where the mantra is better and faster, but without understanding the implications, simply pursuing velocity or volume isn’t the answer. Like driving a car, the faster you go, the faster you must make decisions. But if speed kills, faster isn’t better. The best pattern for any decision is found at the intersection the right speed, the right amount of information, and the right context.
Not just fast
The best enterprises make good decisions, not just fast decisions. Getting there isn’t simple…there’s a point where the decision needs to be made on the right amount of information at just the right time. Made early, the decision lacks key data. Made late, the moment to decide may be over. Just like the speed decision, the ‘just enough information’ decision is a tuning exercise and not a one-size-fits-all.
Not just fully informed
Many organizations are still depending on traditional business intelligence (BI), which isn’t about having the right decision in the right moment. Instead, it is often about understanding something that has already happened and is over-the-shoulder. BI is only marginally helpful and is losing relevance in real-time decision making.
Decision making is also about the impact from getting it wrong or not making the decision at all. Clearly, making a large trade that has high risk and reward can be put off or cancelled, but decisions in an emergency room can’t. Making the wrong choice when free climbing a mountain has very negative implications.
Kinds of decisions
Gartner’s W. Roy Schulte describes three kinds of decisions in Approach Operational Decisions Differently From Tactical or Strategic Decisions:
- Strategic decisions are made in months to years. Should we acquire a company? Should we expand into South America?
- Tactical decisions are made in days to weeks. What customers should we pursue? What features will go into our next product?
- Operational decisions are made when the speed and amount of data align. How should we respond to a customer service request? Is a credit card being used fraudulently?
Operational decisions are the ones that are being driven by technology that goes into decision support systems that rely on process. A good decision support system connects to the many sources of information that go into a broad range of decision and also creates a context (time, place, level, business unit) that is relevant for making consistently wise choices. The best systems are about environmental and situational awareness and that’s being done more and more often through event processing.
As the human ability to decide is mimicked more and more by technology and less by humans. This is driven by cost, accuracy and the speed or complexity of information that goes into a decision. The human brain still soundly trounces the best automation, but that gap is closing.
Flexible and adaptable
Many decisions spawn processes or other decisions, and any system needs to be able to flow back into itself or kick off an external set of activities. Rigid systems don’t do this well and are impossible to maintain. The best systems being developed now have user interfaces that blend process, analytics and information so that you can change things on the fly. Anyone playing in what most call the real-time space need to have event processing as a core capability.
Done well, you have a system that helps you know how much data is just enough to make the fastest, best, most contextual decision possible.
This article was inspired by a presentation by Gartner Distinguished Analyst W. Roy Schulte at TIBCO’s TUCON 2012.