Every year, human beings and our machines generate more data than we ever have before.  We’ve all heard the famous estimation from Eric Schmidt when he was CEO at Google in 2010: “There was 5 Exabytes of information created between the dawn of civilization through 2003,” he proclaimed, “but that much information is now created every 2 days, and the pace is increasing.”

There is, of course, no way to quantify data generation prior to the computer age.  Neither can an…

Hi!

I will show you how to create a simple application in R & Shiny to perform Twitter Sentiment Analysis in real-time. I use RStudio. First, I create a Shiny Project. Then, in the ui.R file, I put this code:

shinyUI(fluidPage(



titlePanel("Sentiment Analysis"),…

Social networks, in one form or another, have existed since people first began to interact. Indeed, put two or more people together and you have the foundation of a social network. It is therefore no surprise that, in today’s Internet-everywhere world, online social networks have become entirely ubiquitous.

Within this world of online social networks, a particularly fascinating phenomenon of the past decade has been the explosive growth of …

Tis the season for 2016 predictions, and Tamr has several to offer thanks to the forward-thinking minds of our co-founders, executives and advisors including:

Each of these articles were viewed between 15,000 and 100,000 times, and were shared countless times. They were posted on Data Science Central in 2015. Later this month, we will publish our most popular, very recent articles.

Enjoy the reading!

• Visualizations: Comparing Tableau, SPSS, R, Excel, Matlab, JS, Python, SAS …

Assume you were going to move to another city with which you weren't familiar . What elements would you consider while picking an area, road or address as your new home? Obvious things like housing prices, crime rates, and proximity to schools and public transportation would play into your decision. Nevertheless, some factors have influence especially on you.…

For quite some time I have been interested in determining whether material or mobility is more influential in determining the outcome of a game in chess, and I recently did a statistical analysis on a large database of expert-level games to try and answer that question.

The results are rather surprising, and I have written a brief technical summary of the methodology as well as the results. The R code used to perform the…

The mock border closing for the Civilian Data Army was just the beginning in 2019. The goal to catalog every American in America had been laughed at so much by media it has become cliche. The rebellion has all the makings of a massive movement. The media could not control it. Everyone wants answers. Real complete answers. The people are ready. Fueled by anger and fed up with the secrecy and disrespect of personal data they demand change.



The early years of data science in the 21st…

Certifications exist to keep HR happy and provide another stream of revenue for the companies giving the certification. They test you on a lot of material you will never use. Also, by the time you get the certification, they are out of date. Of course, you can also say the same thing about most technical education, particularly now. It is better to prove you actually can do the job (somehow) rather than pass the certification. I am an example. About 15 years ago, I was out of work, so I…

By Andy Palmer, Co-founder and CEO of Tamr

We all know by now that most “big data” challenges can be characterized into one of the “3 V’s”:

Volume:  You have too much data

Velocity: It is coming at you too fast

Variety: It is coming at you from too many places/silos

Over the past 5 years, my partner Mike Stonebraker and I have been on a mission to raise awareness of the…

Summary:  If you’re looking for a high growth, high demand area in data science where there is very high value and not many expert practitioners, look at time series forecasting, especially as it applies to Supply Chain Management demand forecasting.

 …

For ever it seems, we’ve been warning about the dangers of low quality data. Our warnings have been reinforced and echoed by some of the world’s biggest think tanks. However, despite this, some organisations still haven’t acted to improve the quality of their data.  And we’re wondering why?

Over the last 12 months, we’ve blogged about business automation , and about…

The global big data market is expected to develop rapidly by 2018, with major contribution from the North America regional market. In the past few years, there has been a growth in ‘big data’, which is generated in several sectors across the globe. Growth in the amount of data has led to the development of technology that can be used to analyze and manage big data. For…

IT Ops Challenge

Each layer of technology in the data centre is becoming progressively more complex to control and manage. The average server environment now has thousands of configuration parameters (e.g. Windows OS…

Many Machine Learning articles and papers describe the wonders of the Support Vector Machine (SVM) algorithm. Nevertheless, when using it on real data trying to obtain a high accuracy classification, I stumbled upon several issues.

I will try to describe the steps I took to make the algorithm work in practice.



This model was implemented…

Guest blog post by Tony Abraham, Nikhita Koul, Joe Morales. A data science exploration of rap lyrics and what it takes to make it onto the billboard charts. Originally posted here. The original version offers full-size, complete interactive Tableau graphics, as well as the hit prediction algorithm box, available as an API. …

In Discover Magazine this month is a really frightening article…

What will you need to know next year, in five years, in ten years, to be successful in your industry? What will your industry look like in the future? How has it evolved?

Think about how different your job or field was in the 1980s or 90s. Did your job even exist then? That’s only 20 or 30 years ago, and things have changed incredibly rapidly. In the early 90s,…

I have to admit, I love the swell of Internet of Things devices and technology we’re seeing right now.  

Smart alarm clocks that can wake you without waking your bedmate, smart showers that help you save water, a smart kettle or coffee pot you can start…