Start with Good Science on Good Data, Then we'll Talk 'Big Data'

We are currently witnessing a land rush of investment in Big Data architectures promising companies that they can turn their data into gold using the latest in distributed computing and advanced analytical methods. Although there is indeed much potential in applying machine learning and statistical analysis to largedatasets, many companies are hardly sitting on the kind of data that will allow them to compete using hundreds of machines chugging through terabytes of data.

But that's okay. There is a massive benefit to just getting an organization to understand what data they do have and how they can deploy intelligent models on this data to disrupt their current approaches to doing business. This does not require the latest in parallel computing or bursting into new map-reduce paradigms just to derive insight. It does not require huge data warehouses or terabytes of unstructured data. What it does require is good science on good data. This is where organizations need to start; becoming ''data aware' and building an organizational culture that understands data as a real asset.

The majority of organizations have barely moved beyond static BI reports and are unaware of the actual potential their data holds. Going from being 'data unaware' to investing in a big data architecture in one leap sets a company up for a bad ROI in analytics. A solid investment must begin with understanding what data is actually available and identifying the low hanging 'data fruits' that can lead to real value for the company. This can be used to build lightweight solutions that are imperfect but hugely beneficial. It can provide real-world tools for decision support via recommended actions or highlighted opportunities in real-time. It can offload much of the routine repetitive decision-making to algorithms so that professionals can operate with a more strategic view of the organization and bring their creative talents to their company's challenges.

With time, this foundation can lead to some huge benefits as the organization starts understanding how to compete using data. As their data-awareness matures, the fruits of Big Data become a real possibility and the architectures for dealing with terabytes of data and highly advanced models now stand a chance to offer a real return on the company's investment in analytics.

When it comes to analytics we are in the inevitable bubble that comes from any new technology that appears highly disruptive. It makes it difficult for organizations to separate the real potential from the over-hyped claims of vendors looking to get every company running their latest Big Data stack. But the potential for Big Data is real and can make a true difference to your organization; it just takes time to build a real analytical foundation.

Start with doing good science on good data. Hire a great scientist to identify what data you do have and how this can be used immediately to start offering real-world solutions to your challenges. Get them to build simple but effective models that can automate routine decision-making and elevate your CXOs with a more strategic view of the organization. Get them to develop 'data-forward' strategies by identifying where you are data-poor so that you can start collecting rich data for future analytics efforts. With time, your company will move into a position where they can reap the real benefits of Big Data using the latest machine learning algorithms and distributed computing architectures. It's an investment worth the effort.

Views: 4446


You need to be a member of Data Science Central to add comments!

Join Data Science Central

Comment by Kenneth C Black on November 19, 2014 at 10:51am


You are a good writer with a clear message. I wrote a post a month ago that discusses using available historical data coupled with a few analytical techniques to help businesses Make More Money. Here is the link: Using historical data to help businesses make more money.



Comment by Kartick Sekar on November 12, 2014 at 4:17pm

Great Post! Without the science and the understanding of what you eventually want to do using the latest and greatest BI/OLAP/Big Data solutions, collecting vast amounts of data amounts to nothing except a huge TCO. To recognize great ROI and reduce TCO, the science and the goal must be understood correctly and just enough and exact data must be collected (Agile model of closed for modification but open to expansion). Data gathering "dust" is a huge waste of cost & resources. Data should work for the organization not the other way around. Great reading

Comment by Ian Thurman on November 6, 2014 at 9:35pm

Really great to hear someone setting out the need for 'good' data and analysis. In the UK there's a tendency amongst analysts and the media for any 'Big Data' and the analysis that flows from it to be seen as valid. In this regard I'm increasingly reminded of the Danny Kaye song lyric "The King is in the altogether".    

© 2021   TechTarget, Inc.   Powered by

Badges  |  Report an Issue  |  Privacy Policy  |  Terms of Service