In the insightful post Big Data: New Oil or Snake Oil? recently published on wired.com, Simon Moss distinguishes between two different types of Big Data:
- New data volumes capturing web-based digital retail, mobile activities, social media interactions and Internet search criteria. This data tends to be homogeneous and relatively simple in its structure.
- Ever growing enterprise data – heterogeneous, complex and highly distributed across a wide expanse of disparate technology and data platforms.
Moss points out that the successful approaches for harnessing Big Data of the first type fail when one tries to apply them to the enterprise data. He says:
“…a majority of business problems faced by the enterprise are not Big Data problems. They are distributed data problems with information, data, value and analysis hugely distributed across heterogeneous locations, technology and sources. Yet we insist on trying to solve this growing distribution problem with the same centralization models of the past. These models work great when data is already more or less stabilized in a common look and feel – something we can see with the new mega successes around social media and digital retailing. It is something we absolutely cannot see in banking, insurance, healthcare and a huge number of other business problem areas.”
The main obstacles to realizing value from this data lay in its complexity, heterogeneity and distribution. Yet, it arguably remains the most important and valuable enterprise asset. As data grows in complexity, whether it is internal or external to the enterprise, the promise of Big Data is not justifying the huge system and data integration projects required to realize its value. Doug Laney from Gartner summed it up recently, pointing out that through 2017 ninety percent of Big Data projects will not be leveragable because they will continue to be in siloes of technology or location.
Volumes of siloed ETL scripts and diverse integration layers that don’t expose or share common semantics are increasingly turning enterprise data integration into an intractable problem. These are doable in theory, but, in practice, they take too long and cost too much. Even worse, each new data integration effort creates a new silo that quickly becomes inadequate as the enterprise moves forward.
This challenge is the driving force of TopQuadrant’s mission. Leveraging semantic web technologies, our solutions (such as TopBraid Insight, TopBraid Enterprise Vocabulary Net, and TopBraid Enterprise Data Governance) focus on the new approaches for bringing together complex, diverse and distributed data faster. We do this less expensively and without losing any of data’s meaning. Instead of a single centralized model, TopBraid technology gives enterprises ability to distribute and modularize models developed for different business needs and perspectives yet connect them as needed.
The accelerated route to extracting value from enterprise data and powering business agility is here. It is a different approach. It requires some new skills, different thinking and pioneering. Then, again, innovation has always been key to solving difficult problems that can’t be solved by continuing to do more of the same.