Big data is not only a buzz word but it has also started playing a vital role in business strategy. Today, many companies such as Facebook, Microsoft, Google, Twitter are taking advantage of big data analytics and machine learning to predict customer’s behavior. If you look at the Top 8 Technology Trends For 2015, you will see big data is one of the latest trends of 2015 and this is just the beginning.
So what is big data?
Menish Gupta writes on big data:
Big Data is a set of technologies that allows users to store data and compute leveraging multiple machines as a single entity. Think of it as a poor man's super computer.
In early 2000, Google developed its own version, to analyze all the world's websites to power its search engine. Today, it's used behind all the major products powered by Google, such as Google Maps.
Later on, Google published a series of papers that were chosen by engineers at Yahoo, who eventually made the technology open source and hence here we are with the Big Data movement.
Wikipedia defines big data:
Big data is a broad term for data sets so large or complex that traditional data processing applications are inadequate. Challenges include analysis, capture, data curation, search, sharing, storage, transfer, visualization, querying and information privacy. The term often refers simply to the use of predictive analytics or certain other advanced methods to extract value from data, and seldom to a particular size of data set. Accuracy in big data may lead to more confident decision making, and better decisions can result in greater operational efficiency, cost reduction and reduced risk.
Big data can be described by the following characteristics:
Volume
The quantity of generated and stored data. The size of the data determines the value and potential insight- and whether it can actually be considered big data or not.
Variety
The type and nature of the data. This helps people who analyze it to effectively use the resulting insight.
Velocity
In this context, the speed at which the data is generated and processed to meet the demands and challenges that lie in the path of growth and development.
Variability
Inconsistency of the data set can hamper processes to handle and manage it.
Veracity
The quality of captured data can vary greatly, affecting accurate analysis
Berkeley has collected definition of big data from 40 leaders. One of them says,
Daniel Gillick
Senior Research Scientist, Google
Historically, most decisions — political, military, business, and personal — have been made by brains [that] have unpredictable logic and operate on subjective experiential evidence. “Big data” represents a cultural shift in which more and more decisions are made by algorithms with transparent logic, operating on documented immutable evidence. I think “big” refers more to the pervasive nature of this change than to any particular amount of data.