Summary: Simpson’s Paradox. A source of risk for real time analytics and for the citizen data scientist.
Most of us practicing the predictive arts know to look for sources of bias in our data. There are seven that are common, the first six of which are:
Guest blog post by Vijay Rajan.
As catchy as the title of this post is (which I often do to get audience attention, approvals or argue), the real title should be "What moved the median?". This is a long read compared to my earlier post because I drill down really deep into…Continue
Added by Vincent Granville on August 31, 2015 at 7:30am — No Comments
NASA is using big data to make complex knowledge more readily available. Learn how graph visualization can help turn large corpus of documents into concrete insights.
Even in a mature and knowledge-driven organization like NASA, finding an answer to a common business issue can be frustrating. Past surveys at NASA have shown that most people have trouble finding the…Continue
Open data repositories are valuable for many reasons, including:
(1) they provide a source of insight and transparency into the domains and organizations that are represented by the data sets;
(2) they enable value creation across a variety of domains, using the data as the “fuel” for innovation, government transformation, new ideas, and new businesses;
(3) they offer a rich variety of data sets for data scientists to sharpen their data mining, knowledge discovery, and…Continue
Added by Kirk Borne on August 30, 2015 at 2:09pm — No Comments
On September 2nd, 2015, President Peña of Mexico will give his 4th State of the Country address to 120 million Mexicans (who will actually watch is another matter entirely). Given these troublesome times in terms of economy, and our endemic problems as a country, like drug traficking, corruption and violence, it will be a message worth observing and analyzing.
To this end, a few independent news outlets have taken upon themselves to embark on a live 'fact-checking' event.…Continue
Check out this infographic from XO Communications about 2016 being the year of the Zettabyte.
Added by Sheldon Smith on August 28, 2015 at 10:00am — No Comments
Previously, we discussed the role of Amazon Redshift's sort keys and compared how both compound and interleaved keys work in theory. Throughout that post we used some dummy data and a set of Postgres queries in order to explore the…Continue
Added by sasha blumenfeld on August 28, 2015 at 7:20am — No Comments
Guest Blog by Jose Dianes at R-Data Science
The purpose of many data science projects is to end up with a model that can be used within an organisation to solve a particular problem. If this is our case, we need to determine the right representation of that model so it can be shared in the easiest, cheapest, and most effective way. Web data products are an ideal…
I’m a big fan of statistics. Other than being fun to play with and fun to illustrate, they serve a lot of important tasks for researchers. They can quickly identify which of 500 comparisons is statistically significant. They can offer data to show whether your brand users comprise 2 distinct groups of people or 7 distinct groups of people. They can offer data to show which price your consumers would refuse to pay.
But there are two ways to use statistics. The right way and the wrong…Continue
The analytical scene has recently been dominated by the prediction that we would soon experience an important shortage of analytical talent. As a response, academic programs and massive open online courses (MOOCs) have sprung up like mushrooms after the rain, all with the purpose of developing skills for the analyst or its more modern counterpart, the data scientist. However, in the …Continue
Added by Geert Verstraeten on August 27, 2015 at 11:30pm — No Comments
Sometimes I don’t trust Data Science, probably because my duty of care is more pronounced on account of working mostly in Legal Analytics. You see as an Analytics Practitioner in the Legal field my Data Science methodology cannot afford to yield wild guesses, these are people’s lives I’m dealing…Continue
Added by Mkhuseli Mthukwane on August 27, 2015 at 7:35am — No Comments
A lot has been said about the value of data viz, but the folks at R2D3 have truly taken this to a whole new level by using very sophisticated but also very intuitive data viz techniques to teach the basics of machine learning. I was really blown away by the way the step-by-step visualizations on this page lead the reader through all the intuitive steps to arrive at a pretty clear understanding of machine learning, in this case focusing on decision trees.
If you are an experienced…Continue
This is not about attacking a guy - a friend of mine - who, at first glance, seems extremely overpaid, like any top executive. Indeed, the question is about whether data scientists should be coders (spending 50% to 100% of their time writing code) or not.
I believe the answer is negative. There are…Continue
As data continues to grow at unprecedented speeds, organizations must embrace a data-driven mind-set to stay competitive. With the influx of bigger data and new types of data, companies of all sizes are increasingly dependent on large sets of information to make better business decisions.
Marketing departments can rely on data to discover up-sell and cross-sell opportunities and to improve customer relationships. According to …Continue
The full version is always published Monday. Starred articles are new additions or updated content, posted between Thursday and Sunday. The picture of the week is from the contribution marked with a +, where you will find the details.
Added by Vincent Granville on August 26, 2015 at 9:30am — No Comments
With an ever-growing number of businesses turning to Big Data and analytics to generate insights, there is a greater need than ever for people with the technical skills to apply analytics to real-world problems.
Computer programming is still at the core of the skillset needed to create algorithms that can crunch through whatever structured or unstructured data is thrown at them. Certain languages have proven themselves…Continue
Data Scientist is the rock star job title of the moment, and why not? There is a huge demand and a very small pool of qualified candidates. And those who get hired are making big six-figure salaries — who wouldn’t want in on that?
But just because you aspire to be a data scientist doesn’t mean you already are qualified to be one. Here are a few…Continue
Companies are directing a lot of resources towards data mining and data analytics. Analysis of big data can help improve a business’s online marketing strategy, since the needs of the customer are better understood, and the…Continue
Added by Jack Dowson on August 25, 2015 at 9:44pm — No Comments
Guest blog by R. Bhargav
What does “Big Data” mean?
The term “big data” is self-explanatory -a collection of extremely big data sets that normal computing techniques cannot process. The term not only refers to the data, but also to…Continue
There are always two aspects to data quality improvement. Data cleansing is the one-off process of tackling the errors within the database, ensuring retrospective anomalies are automatically located and removed. Another term, data maintenance, describes ongoing correction and verification – the process of continual improvement and regular checks.