During an earlier post I wrote about how the use of ‘small data’ can still have a big impact on an organization’s ability to drive performance improvement. While so called ‘big data’ techniques can be incredibly powerful, not every data analytics challenge is a ‘big data’ analytics challenge.
That said, we’re still asked nearly every day “should we invest in big data [or insert name of generic big data technology]’?”
The first question you should be asking is “Have we effectively integrated data science into the management of our organization?”
You should invest first and foremost in establishing a data science program and, if your data science needs warrant it and your organization has access to the right expertise, you should consider investing in big data. If you don’t have a formalized data science program in place, then it’s imperative you start there first.
Big data technologies are a lot of things, but they’re certainly not plug-and-play. Even some of the nice end-user graphical packages available still require a lot of preparation on the back end to make sure those apps have nice data to chew on. If a big data vendor tells you their technology is so polished and slick that you don’t have to worry about what’s going on inside their ‘proprietary’ black box, you should probably be getting worried not excited.
Before diving into big data an organization needs to be [continue reading]...