Summary:  Simpson’s Paradox.  A source of risk for real time analytics and for the citizen data scientist.

Most of us practicing the predictive arts know to look for sources of bias in our data.  There are seven that are common, the first six of which are:

1. Confirmation…

Guest blog post by Vijay Rajan.

Introduction

As catchy as the title of this post is (which I often do to get audience attention, approvals or argue), the real title should be "What moved the median?". This is a long read compared to my earlier post because I drill down really deep into…

NASA is using big data to make complex knowledge more readily available. Learn how graph visualization can help turn large corpus of documents into concrete insights.

A database of lessons learned

Even in a mature and knowledge-driven organization like NASA, finding an answer to a common business issue can be frustrating. Past surveys at NASA have shown that most people have trouble finding the…

Open data repositories are valuable for many reasons, including:

(1) they provide a source of insight and transparency into the domains and organizations that are represented by the data sets;

(2) they enable value creation across a variety of domains, using the data as the “fuel” for innovation, government transformation, new ideas, and new businesses;

(3) they offer a rich variety of data sets for data scientists to sharpen their data mining, knowledge discovery, and…

On September 2nd, 2015, President Peña of Mexico will give his 4th State of the Country address to 120 million Mexicans (who will actually watch is another matter entirely). Given these troublesome times in terms of economy, and our endemic problems as a country, like drug traficking, corruption and violence, it will be a message worth observing and analyzing.

To this end, a few independent news outlets have taken upon themselves to embark on a live 'fact-checking' event.…

Check out this infographic from XO Communications about 2016 being the year of the Zettabyte. 

XO…

Introduction

Previously, we discussed the role of Amazon Redshift's sort keys and compared how both compound and interleaved keys work in theory. Throughout that post we used some dummy data and a set of Postgres queries in order to explore the…

 Guest Blog by Jose Dianes at R-Data Science

The purpose of many data science projects is to end up with a model that can be used within an organisation to solve a particular problem. If this is our case, we need to determine the right representation of that model so it can be shared in the easiest, cheapest, and most effective way. Web data products are an ideal…

I’m a big fan of statistics. Other than being fun to play with and fun to illustrate, they serve a lot of important tasks for researchers. They can quickly identify which of 500 comparisons is statistically significant. They can offer data to show whether your brand users comprise 2 distinct groups of people or 7 distinct groups of people. They can offer data to show which price your consumers would refuse to pay.

But there are two ways to use statistics. The right way and the wrong…

A simple metaphor for projects in predictive analytics

The analytical scene has recently been dominated by the prediction that we would soon experience an important shortage of analytical talent. As a response, academic programs and massive open online courses (MOOCs) have sprung up like mushrooms after the rain, all with the purpose of developing skills for the analyst or its more modern counterpart, the data scientist. However, in the …

Sometimes I don’t trust Data Science, probably because my duty of care is more pronounced on account of working mostly in Legal Analytics. You see as an Analytics Practitioner in the Legal field my Data Science methodology cannot afford to yield wild guesses, these are people’s lives I’m dealing…

A lot has been said about the value of data viz, but the folks at R2D3 have truly taken this to a whole new level by using very sophisticated but also very intuitive data viz techniques to teach the basics of machine learning.  I was really blown away by the way the step-by-step visualizations on this page lead the reader through all the intuitive steps to arrive at a pretty clear understanding of machine learning, in this case focusing on decision trees.

If you are an experienced…

This is not about attacking a guy - a friend of mine - who, at first glance, seems extremely overpaid, like any top executive. Indeed, the question is about whether data scientists should be coders (spending 50% to 100% of their time writing code) or not.

I believe the answer is negative. There are…

As data continues to grow at unprecedented speeds, organizations must embrace a data-driven mind-set to stay competitive. With the influx of bigger data and new types of data, companies of all sizes are increasingly dependent on large sets of information to make better business decisions. 

Marketing departments can rely on data to discover up-sell and cross-sell opportunities and to improve customer relationships. According to …

The full version is always published Monday. Starred articles are new additions or updated content, posted between Thursday and Sunday. The picture of the week is from the contribution marked with a +, where you will find the details.

Announcement

• The right data can greatly accelerate your business, but how do you put your aspirations into action? How do you…

With an ever-growing number of businesses turning to Big Data and analytics to generate insights, there is a greater need than ever for people with the technical skills to apply analytics to real-world problems.

Computer programming is still at the core of the skillset needed to create algorithms that can crunch through whatever structured or unstructured data is thrown at them. Certain languages have proven themselves…

Data Scientist is the rock star job title of the moment, and why not? There is a huge demand and a very small pool of qualified candidates. And those who get hired are making big six-figure salaries — who wouldn’t want in on that?

But just because you aspire to be a data scientist doesn’t mean you already are qualified to be one.  Here are a few…

Companies are directing a lot of resources towards data mining and data analytics. Analysis of big data can help improve a business’s online marketing strategy, since the needs of the customer are better understood, and the…

Guest blog by R. Bhargav

What does “Big Data” mean?

The term “big data” is self-explanatory -a collection of extremely big data sets that normal computing techniques cannot process. The term not only refers to the data, but also to…

There are always two aspects to data quality improvement. Data cleansing is the one-off process of tackling the errors within the database, ensuring retrospective anomalies are automatically located and removed. Another term, data maintenance, describes ongoing correction and verification – the process of continual improvement and regular checks. 

Often, businesses…