Why Big Data Projects Failing?

According to a recent Gartner report, only 15% organizations were able to take big data projects into production in year 2016. Before we look into why big data projects are failing we need to understand why businesses need to move their existing data management systems to big data. The main reason is some sectors of industry have seen rapid data growth over last decade and that is going to pick up even more speed in years to come. Some organizations in following sectors have seen data growth of up to 25% a year; - telecommunication and utilities - governments - health - education - banking (financial services, insurance etc.) - manufacturing The key sources of data are system transactions, appliances and devices, social media, Internet of things (IoT) , user activity etc. In most cases, process automation or real time and interactive behaviour are the driving factors behind large volumes of complex data being generated at high velocity ( 3 V's of big data). Unfortunately, traditional data management systems are unable to deal with the 3 V's of big data. Hence, these organizations need to migrate data management systems to big data platform sooner than later. Why are organizations failing to migrate to big data successfully? Gartner rightly points out two main reasons; 1. lack of effective and visionary leadership; it cannot foresee how big data is the right solution and a way to survive competition in years to come. They think consolidating existing systems and meeting minimum performance threshold is enough. That might be so in short time but will be problematic in future. 2. less tangible ROI upfront and near future - the leadership cannot see the competitive advantage the organization may have by exploiting big data technologies in future. Gartner report says that only 11% leadership considers big data projects as or more important than other information technology initiatives and 46% think it is even less important. Since leadership cannot realized the important of big data technology and its competitively positive implication in future, therefore either organizations are lagging behind to kickstart big data projects or have started half-heartedly and with unclear vision; hence most of them have not been able complete and deploy into production successfully.