This applies to data science research as well as any other analytic discipline. For centuries, scientific research was performed in Academia, by university professors managing their own labs. Much of the research was carried out by young scientists who just completed their PhD. The selection process has always favored the same type of personality. The basic rule is "publish or perish" which produces the following drawbacks:

• Re-use of old material (rather than brand new material)…

Most statisticians are great professionals, working on various data-intensive projects, and they don't care about their job title. You can say the same about data scientists, and me in particular. However, there is a small cluster of statisticians - Andrew Gelman seems to be their leader and their only influencer - who have been challenging us, even publicly insulting us recently.…

To be more precise, this kind of attack would rely on business hacking, rather than computer hacking. Other attacks, some potentially as massive as to turn Google into the worst search engine, are described below.

The Sony attack

I believe that such an attack could be accomplished by an insider…

The full version is always published Monday. Starred articles or sections are new additions or updated content, posted between Thursday and Sunday. Articles marked with a + have interesting visualizations.

Featured

• Regression Analysis using R…

Every data scientist worth her salt will immediately notice that the biggest Earthquakes (magnitude above 9) took place in the last 60 years or so.

Northridge Earthquake

Most journalists, and even some…

Given the right data being correctly collected, and analyzed using sound predictive models, what can be predicted, and what can't be predicted no matter what?

I believe that I have an answer to this question. All systems and processes that rely on some energy source can be predicted, and the other way around. Note that energy…

The full version is always published Monday. Starred articles or sections are new additions or updated content, posted between Thursday and Sunday. Articles marked with a + have interesting visualizations.

Featured

• The data science project…

Guest blog past by Rohit Yadav, from BRIDGEi2i Analytics Solution

The Net (Part 1)

The plot goes something like this – Sandra Bullock plays a computer expert Angela Benett, her life changes when she is sent a program with a crazy glitch to ‘de-bug’. Soon she finds out some vital government information on the disk, things gets nutty as fruitcake, her life becomes a nightmare with her records getting erased and she is given a new identity of some chick with a…

The definition of 'best' depends on which school you follow. Data science and classic statistical science are at the opposite ends of the spectrum. So let's clarify what 'best solution' means in these two opposite contexts:

'Best', according to statistical science:

• It usually means the global maximum of a mathematical optimization problem
• The objective function involved is usually a maximum likelihood function, KS, c-statistics, or some function…

Guest blog post by Bernard Marr, first published here.

The field of Big Data requires more clarity and I am a big fan of simple explanations. This is why I have attempted to provide simple explanations for some of the most important technologies and terms you will come across if you’re looking at getting into big…

These predictions were published by the International Institute for Analytics (IIA). They produced a nice infographics, featured below, and re-tweeted many times by various bloggers, using the hash tag #2015Analytics. Other interesting predictions include…

The full version is always published Monday. Starred articles or sections are new additions or updated content, posted between Thursday and Sunday. Articles marked with a + have interesting visualizations.

Featured

• Discover, Access,…

The purpose of this article is to clarify a few misconceptions about data and statistical science.

I will start with a controversial statement: data science barely uses statistical science and techniques. The truth is actually more nuanced, as explained below.

1. Data science heavily uses new statistical…

Here I compare these 5 rules published in 1999, with the new 2014 version. Data has changed so much that the opposite rules are now followed. Yet many statisticians and big businesses still stick to the outdated rules.

These rules were initially published in the featured book (see picture) first published in 1999, when software (e.g. SPSS) could not adapt to…

The full version is always published Monday. Starred articles or sections are new additions or updated content, posted between Thursday and Sunday. 

Featured

• Top down or bottom up? 5 tips to make the most of your data…