Big Data has truly come of age in 2013 when Oxford English Dictionary introduced the term “Big Data” for the first time in its dictionary. That of course begs the question ‘When was the term Big Data first used and Why?’. My curiosity led me to lot of research material but I relied mostly on Mr. Gil Press’s “A Very Short History of Big data” from Forbes, Mr. Steve Lohr’s “The Origins of ‘Big Data': An Etymological Detective Story“ from The New York Times, Mr. Mark van Rijmenam’s “A Short History Of Big Data” from DataFloq.com, and “The OED, Big Data, and Crowdsourcing” from WhatstheBigData.Com articles. Source links are at the bottom.

Before I started researching into this article, I asked myself if it’s worth the effort to know who coined Big Data and why. I answered it in the affirmative because history always provided me a proper framework to understand and analyze if a trend is for real or just a fad. With so much buzz around Big Data, I needed to figure out the origins of Big Data and the context in which it came about. So here we go.

I’ve reviewed the Big Data origins from two angles, one is from the first time use of the term ‘Big Data’ itself and the other from the first time use of ‘Big Data’ referring to its modern definition i.e. information explosion and large sets of data (as outlined in my first article ‘What is Big Data’). The New York Times article presents a fascinating ‘Whodunit’ detective story of how they discovered the person who coined the term ‘Big Data’.

My version of the ‘whodunit’ starts in 1944 (yup right in the middle of World War II) in a university library in Connecticut. Following infographic is a quick summary and the story follows right below the info graphic.

1944, Fremont Rider, Wesleyan University Librarian, “The scholar and The future of Research Library” (book)

Mr. Fremont Rider, based on his observation in 1944 that American University libraries are doubling every 16 years, speculated in 1944 that the Yale Library in 2040 will have

“approximately 200,000,000 volumes, which will occupy over 6,000 miles of shelves… [requiring] a cataloging staff of over six thousand persons.”

Obviously, Mr. Rider didn’t predict the digitization of libraries but he accurately predicted the information explosion.

From 1944 to 1980, there were many articles and presentations that observed the ‘information explosion’ and the need for storage capacity.

Oxford English Discovery folks discovered that Mr. Tilly was the first person to use the term Big Data in this sentence in his article,

“none of the big questions has actually yielded to the bludgeoning of the big-data people.”

Whatsthedata.com editors argued that Mr. Tilly’s use is not in the context of what we understand Big Data to be today and I agree with them.

Mr. Denning, in my opinion, introduced the notion of what is possible with Big Data in these following sentences.

"The imperative [for scientists] to save all the bits forces us into an impossible situation”

and he went on to say

“It is possible to build machines that can recognize or predict patterns in data without understanding the meaning of the patterns. Such machines may eventually be fast enough to deal with large data streams in real time.”

Mr. Denning truly nailed what is possible with Big Data way back in 1990. Impressive!

Now we fast forward to 1997-1998 when we begin to see the actual use of the term Big Data in its modern context.

Mr. Cox and Mr. Ellsworth’s article is the **first article in the ACM digital library** to use the term “big data” as follows.

“Visualization provides an interesting challenge for computer systems: data sets are generally quite large, taxing the capacities of main memory, local disk, and even remote disk. We call this the problem of big data. When data sets do not fit in main memory (in core), or when they do not fit even on local disk, the most common solution is to acquire more resources.”

Here’s where the plot thickens. The New York Times article credits **Mr. Mashey with the first time use of the term ‘Big Data’**. Even though Michael Cox and David Ellsworth seem to have used the term ‘Big data’ in print, Mr. Mashey supposedly used the term in his various speeches and that’s **why he is credited for coming up with Big Data.** In Mr. Mashey’s own words,

“I was using one label for a range of issues, and I wanted the simplest, shortest phrase to convey that the boundaries of computing keep advancing.”

As per the Times article, Mr. Diebold thought (may be tongue in cheek) that he was the first one to use Big data term until Times gave the credit to Mr. Mashey. Nevertheless, Mr. Diebold should get some credit for his prescience is stating that

“Recently, much good science, whether physical, biological, or social, has been forced to confront—and has often benefited from—the “Big Data” phenomenon. Big Data refers to the explosion in the quantity (and sometimes, quality) of available and potentially relevant data, largely the result of recent and unprecedented advancements in data recording and storage technology.”

Mr. Diebold for the first time linked Big Data explicitly to the way we understand the term today.

Ah, the famous Three ‘V’s. I referred to this in my first article here. I don’t have to say much beyond that.

2005 is the year in which Tim O'Reilly published his groundbreaking article 'What is Web 2.0?' that set off the Big Data race. Additionally, Mark from Datafloq.com states that Roger Mougalas from O’Reilly Media explicitly used the term ‘Big Data’ to refer to a large set of data that is almost impossible to manage and process using traditional business intelligence tools. This is definitely the current widely understood form of Big data definition.

2005 is also the year that Hadoop was created by Yahoo! which is built on top of Google’s MapReduce. As many of you know, nowadays the open-source Hadoop is used by a lot organizations to crunch through huge amounts of data. More on this in my subsequent articles.

So you can say, 2005 is the year that the Big data revolution has truly begun and rest, as they say, is history.

Hope this article was helpful. If you want to stay in touch, please **‘Connect’/'Follow'** on LinkedIn or follow on **twitter @rkdontha1. Thank you for reading.**

So what other Big Data questions you have but are afraid to ask? Go ahead and comment below OR send me a note. Look forward to hearing from you!!!

A Very Short History of Big Data by Gil Press of Forbes

© 2019 Data Science Central ® Powered by

Badges | Report an Issue | Privacy Policy | Terms of Service

**Most Popular Content on DSC**

To not miss this type of content in the future, subscribe to our newsletter.

**Technical**

- Free Books and Resources for DSC Members
- Learn Machine Learning Coding Basics in a weekend
- New Machine Learning Cheat Sheet | Old one
- Advanced Machine Learning with Basic Excel
- 12 Algorithms Every Data Scientist Should Know
- Hitchhiker's Guide to Data Science, Machine Learning, R, Python
- Visualizations: Comparing Tableau, SPSS, R, Excel, Matlab, JS, Pyth...
- How to Automatically Determine the Number of Clusters in your Data
- New Perspectives on Statistical Distributions and Deep Learning
- Fascinating New Results in the Theory of Randomness
- Long-range Correlations in Time Series: Modeling, Testing, Case Study
- Fast Combinatorial Feature Selection with New Definition of Predict...
- 10 types of regressions. Which one to use?
- 40 Techniques Used by Data Scientists
- 15 Deep Learning Tutorials
- R: a survival guide to data science with R

**Non Technical**

- Advanced Analytic Platforms - Incumbents Fall - Challengers Rise
- Difference between ML, Data Science, AI, Deep Learning, and Statistics
- How to Become a Data Scientist - On your own
- 16 analytic disciplines compared to data science
- Six categories of Data Scientists
- 21 data science systems used by Amazon to operate its business
- 24 Uses of Statistical Modeling
- 33 unusual problems that can be solved with data science
- 22 Differences Between Junior and Senior Data Scientists
- Why You Should be a Data Science Generalist - and How to Become One
- Becoming a Billionaire Data Scientist vs Struggling to Get a $100k Job
- Why do people with no experience want to become data scientists?

**Articles from top bloggers**

- Kirk Borne | Stephanie Glen | Vincent Granville
- Ajit Jaokar | Ronald van Loon | Bernard Marr
- Steve Miller | Bill Schmarzo | Bill Vorhies

**Other popular resources**

- Comprehensive Repository of Data Science and ML Resources
- Statistical Concepts Explained in Simple English
- Machine Learning Concepts Explained in One Picture
- 100 Data Science Interview Questions and Answers
- Cheat Sheets | Curated Articles | Search | Jobs | Courses
- Post a Blog | Forum Questions | Books | Salaries | News

**Archives**: 2008-2014 | 2015-2016 | 2017-2019 | Book 1 | Book 2 | More

**Most popular articles**

- Free Book and Resources for DSC Members
- New Perspectives on Statistical Distributions and Deep Learning
- Time series, Growth Modeling and Data Science Wizardy
- Statistical Concepts Explained in Simple English
- Machine Learning Concepts Explained in One Picture
- Comprehensive Repository of Data Science and ML Resources
- Advanced Machine Learning with Basic Excel
- Difference between ML, Data Science, AI, Deep Learning, and Statistics
- Selected Business Analytics, Data Science and ML articles
- How to Automatically Determine the Number of Clusters in your Data
- Fascinating New Results in the Theory of Randomness
- Hire a Data Scientist | Search DSC | Find a Job
- Post a Blog | Forum Questions

## You need to be a member of Data Science Central to add comments!

Join Data Science Central