It is beginning to feel like the excitement about Big Data is peaking. Thanks to computer and communication technology we are now able to collect enormous amounts of raw information (data) about people and how they spend their time on line. This information is vitally important to understanding who our customers are and how they act. Now the challenge is to use that information to increase efficiency, improve products, and increase profit.
What do we do with all the data? Data has no intrinsic economic value. Data only has value if it can change decisions. I use Steve Haeckel’s information hierarchy to think about Big Data and its cousin, Analytics. (Haeckel was a researcher in IBM’s Research Center. See the hierarchy above.) This information hierarchy is discussed in Vince Barabba’s book titled, “Hearing the Voice of the Market: Competitive Advantage through Creative use of Market Information.” Vince uses it in the chapter on, “The Problem of Market Information and Decision Making.”
The information hierarchy starts at the bottom with data and ends at the top with wisdom. Big Data and Analytics are working their way up from the bottom of this hierarchy. We are getting somewhere around “intelligence” and “knowledge.”
IBM’s Watson is probably somewhere in the “intelligence” range. Watson can deal with huge volumes of information and make inferences. It’s output is a set of probabilities on possible answers to a question. When the probability of the right answer gets high enough Watson pushes the buzzer. The Jeopardy-playing monster is certainly very good at inference. Playing Jeopardy is a long way from strategic decision making.
The people in Big Data are beginning to move out of the data crunching space and into the space of operational decisions and Analytics. Big Data is helping organizations make decisions like, How much stuff do we need at each place during each time period? The “stuff” can be FedEx packages, iPhones, or advertizing dollars. You name it.
Operating decisions require some engineering. We have to frame the problems, develop alternative solutions, and analyze profitability. In doing so we learn and develop knowledge. This is where the real value is added to our organizations.
It is going to take even more engineering to move Big Data up the hierarchy and into the domain of strategic decision making. If Samsung is thinking about making an iWatch, for example, then they will want to gather information on who wears watches and what current production costs are in various parts of the world. Big Data can help them with this. Big Data can also show them what people are saying about the iWatch on Facebook. Big data cannot tell them what competitors are going to do, how fast technology will develop, future costs and prices, and which operating system to choose.
Data is about the past. Strategic decisions require information about the future. The future is uncertain. This is where the decision engineering is needed. Decision engineers know how to build structural models of markets based on knowledge of micro economics and technology. They know how to develop and use scenarios. They know how to interview experts.
Big Data will have to team up with decision engineers to impact the big strategic decisions: product and service planning, geographic expansion, and infrastructure upgrading. Executives making strategic decisions need people who can help them get from knowledge to wisdom. This requires a focus on decisions. It requires framing, alternative generation, analysis, and synthesis. All things decision engineers can do.
Decision makers need decision engineers who can use Big Data. They also need decision engineers who can help them synthesis the voices of customers, the voices of stakeholders, and the voices of experts to facilitate wise decisions. As decision engineers we should be following Big Data very closely. How can it help us with the really big decisions? What can Big Data tell us about the future of technology? Future prices? Future political developments? Sorting through these issues takes knowledge and wisdom.
Baker Street posts about twice a month on topics of interest to decision engineers. If you would like to receive these posts automatically by e-mail, just submit your address here.