“Big Data” is a popular phrase these days as large companies have demonstrated the value of analyzing the huge data streams generated by their clients. Passing the data through a suitable analysis environment creates more business opportunities and fosters growth.
The definition of “Big Data” is flexible, depending on the topic of the article you are reading. Large businesses such as Google may need to analyze petabytes of data per day that is coming to their servers. A large international scientific experiment such as ATLAS in Switzerland may generate a petabyte of data in a second. A petabyte is 10^15 bytes of data (that’s 1 with 15 zeroes after it). At the time this article is being written, disks with a capacity of 4 terabytes are readily available and storing a petabyte will require 250 of those 4 TB disks. Simply storing all this data isn’t feasible and on-the-fly data analysis is extremely important.
Big Data can be defined as data that is too large to be handled by a standard relational database (Thomas H. Davenport), or data that can’t be stored and analyzed by a single standard desktop machine. If your business has data accumulating and you don’t do something to extract value from it, you are spending money to store that data, make backups of the data and to pay the person who is care-taking that data. This data is coming into your organization faster than you can make use of it and it isn’t helping you to make decisions.
If you are unable to process this data and gain some insights, this data is Big Enough Data. Big Enough to be costing you for its maintenance Big Enough to be holding some valuable insights Big Enough to be worth some attention. Big Enough Data doesn’t need a worldwide network for data analysis.
It doesn’t need millions of computing cores or petabytes of dedicated memory. Big Enough data might just need one person with a laptop computer to begin analyzing it for you. Is your data Big Enough? Make a start to gaining insights.