Subscribe to DSC Newsletter

Extreme data science is a way of doing data science to deliver high value very fast. It can best be compared to extreme  rock climbing, where individuals managed to climb challenging peaks in uncharted territory, in minimum amount of time with a small team and small budget, sometimes successfully using controversial climbing routes or techniques. For instance, Reinhold Messner climbing the Everest in winter, alone in three days with no oxygen, no sherpa.

Extreme data science borrows from lean six sigma, agile development and analytics, extreme programming, and in many cases advanced statistical analyses on big data performed without statistical models, yet providing superior ROI (part of it due to saving money that would otherwise have been spent on a large team of expensive statisticians).

Just like extreme rock climbing, it is not for the faint hearted: very few statisticians can design sound predictive techniques and confidence intervals without mathematical modeling and heavy artillery such as general linear models.

The only data scientists who can succeed with extreme data science are those with very good intuition, good judgement and vision. They combine deep analytics and advanced statistical knowledge with consistently correct gut feelings and the ability to quickly narrow down to the essence of any problem. They can offer a better solution to business problems, in one month of work, than a team of "regular", model-oriented data scientists working six months on the same project. Better also means, simpler, more scalable, more robust.

These data scientists tend to be polyvalent. Sometimes their success comes from extensive automation of semi-mondane tasks.

One of the problems today is that many so-called data scientists do extreme data science without knowing it. But they don't have the ability to successfully do it, and the results are miserable as they introduce biases and errors at every step when designing and implementing algorithms. Job interview questions don't test the ability to work as an extreme data scientist. The result? If you had 50 respected professional rock climbers, climbing Everest in winter with no sherpa, no oxygen, little money, 48 would fail (not reach the summit), though 25 would achieve a few milestones (5 almost succeeding), 5 would die, 10 would refuse to do it, 8 would fail miserably, and 2 would win. Expect the same with extreme data science. But those who win will do better than anyone equipped with dozens of sherpas, several million dollars and all the bell and whistles.

Note that since (by design) the extreme data scientists work in very small teams, she must be polyvalent. Just like Reinhold Messner in his extreme rock climbing expeditions, where he is a rock climber, a medical doctor, a cameraman, a meteorologist etc. all at once.

Have you ever seen extreme data science in action? Any success stories you want to share? 

Views: 1214

Comment

You need to be a member of Data Science Central to add comments!

Join Data Science Central

Follow Us

Videos

  • Add Videos
  • View All

Resources

© 2017   Data Science Central   Powered by

Badges  |  Report an Issue  |  Privacy Policy  |  Terms of Service