Text analytics, sometimes alternately referred to as text data mining or text mining, refers to the process of deriving high-quality information from text.  High-quality information is typically derived through the devising of patterns and trends through means such as statistical pattern learning.  Text mining usually involves the process of structuring the input…

Weekly digest from Data Science Central and Analytic Bridge:

• Is Data Scientist the Sexiest Job of the Century? | Harvard Busines...
• The 8 worst predictive…

With the launch last week of Datascape I thought it would be worth putting an MD’s perspective on the product – how we got here, what the philosophy is that lies behind it, and where we hope to go with it. For a more formal view of the academic and commercial background see our Immersive Data Visualisation white…

Working and valorising Big Data is business-as-usual for companies that have built their business model on it. For companies that don’t compete on analytics, that is for whom analytics is not a core element of their strategy, it’s a huge challenge.

But Big Data is the talk of town nowadays. I think that a part of the growing management interest is due to two factors:

1. The relentness search for cost efficiency and growth in the current economic downturn;
2. A…

Hey good lookin’. Yep, I’m talking to you, or at least the data scientists reading this. (The rest of you are incredibly good looking, intelligent, and clearly have good taste, as well.)

TheHarvard Business Review has…

The goal of  Data Analytics (big and small) is to get actionable insights resulting in smarter decisions and better business outcomes. How you architect business technologies and design data analytics processes to get valuable, actionable insights varies.



It is critical to design and build a data warehouse / business intelligence…

In today's post, we dive into understanding Association Rules for Market Basket Analysis and discuss three numeric measures that should be considered before deciding to act on / make a business decision based on associations that have been observed in the data: (1) Support (2) Confidence and (3) Lift.



Association rules are typically written in the format:



Left hand side Implies Right hand…

There are various offerings out there if you want to use machine learning in your analysis nowadays. Nick WIlson spent his internship at BigML comparing three SaaS Machine Learning Services (BigML, Prior Knowledge and Google Prediction API), with WEKA as a  benchmark. He wrote a series of blog posts about his findings. In his final post he gives a summary of his work, with links to the different blog posts for details. He let me re-blog his summary here.

"Part 1 -…

The goal is to design and build a data warehouse / business intelligence (BI) architecture that provides a flexible, multi-faceted analytical ecosystem for each unique organization.

A traditional BI architecture has analytical processing first pass through a data warehouse.

In the new, modern BI architecture, data reaches users…

Lift and Gain Charts are a useful way of visualizing how good a predictive model is. In SPSS, a typical gain chart appears as follows:







In today's post, we will attempt to understand the logic behind generating a gain chart and then discuss how gain and lift charts are interpreted.





To do this,…

Data scientists are the new astronauts. Everyone wants to become one. And it is not difficult to understand the reason for this.

In this age of “Big data”, more and more businesses are relying on people who can make sense of the vast amounts of information generated around us – people who can use sophisticated tools and complex-sounding statistical techniques to derive insights from larger and larger mounds of data.

Businesses have started to understand the power of data. They…

This is about how to boost your analytic career and/or revenue by leveraging our professional network to the fullest extent.

We invite you to post blogs, or participate in forums (including answering questions asked by peers) on DataScienceCentral and…

This Saturday, I've noticed that Facebook now displays a few new boxes on everyone's profile page (not just me). The box that worries me most is the one that shows all the places where you've traveled and where you've lived, including your current location.

To compound the problem, the box in question clearly…

These are the articles that I enjoyed reading this week: 

• More Data Beats Better Algorithms — Or Does It? My answer: Better (not necessarily more) data beats better algorithms, also read …

Copyright © SAS Institute Inc., Cary, NC, USA. All Rights Reserved. Used with permission. 

Optimization answers the question: How do we do things better? What is the…

Do you agree with this? I don't, I think this Forbes article is using a provocative title to get you to read it. While assembler programmers in the seventies were eventually replaced by compilers and programming language interpreters, I believe that real statisticians and data scientists can't fully be replaced by machines or software. When they are,…