Why is everyone talking so much about Big Data? What is it about this term that is getting the industry all in a frenzy? Is Big Data and analytics one and the same? Well I too had all these questions some time back. I went out and read a whole lot of books and blogs on the subject and today can say that I have some understanding of Big Data which I will share with you.
One of the best definitions of Big Data I havefound isquite naturally on Wikepedia. They say that Big data is the term for a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications. As compared to traditional structured data which is typically stored in a relational database, big data is characterised by it’s volume, velocity, and variety. Lets understand more about these three Vs.
By the way, it was Gartner analyst Doug Laney who first introduced the concept of the 3Vs in his 2001 MetaGroup research publication, 3D data management: Controlling data volume, variety and velocity. It’s worthy to mention that of late few additional Vs are doing the rounds, variability – the increase in the range of values typical of a large data set — value, which addresses the need for valuation of enterprise data and volatility.
Big data at the outset first means huge, gargantuan volumes of data. This data is generated by people, machines and networks. It is very common to have Terabytes and Petabytes of the storage system for enterprises. As the database grows in volume, the applications and architecture built to support the data needs to keep pace. This huge size of data is what represents Big Data.
We are in the digital age and ‘recent data’ has a whole new meaning. Everything is real time and updates are reduced to fractions of seconds. This high velocity of data is another characteristic of Big Data. It refers to the pace at which data flows in from a variety of sources. The flow of data is continuous and in great amounts.
When data comes in from a variety of sources both traditional and untraditional, in both structured and unstructured form, we call it variety. These varied sources create challenges in terms of storage, mining and analyzing and is another representation of what can be called Big Data.
So why is the industry so excited about Big Data. Well in simple terms big data can create a significant competitive advantage for companies in every kind of industry. Big Data can also help to create new growth opportunities. There is even a whole new industry spurned out involving all those who manage this Big Data and aggregate and analyse it. And best of all, all of us as consumers also stand to gain from Big Data. It can and will even more potently in future impact our daily lives and make it better.
It now makes it easier to understand the difference between Big Data and analytics. Big Data is the raw data that we have access to. It is huge, comes from various sources, it can be unstructured and untraditional and its velocity can be real time. Now the tools and technologies that is used to analyze this Big Data is what is called as analytics. Simple isn’t it? Yes that’s Big Data for you in a nutshell.