From Data Analyst to Predictive Modelers to Data Scientists

Summary: Gartner says that predictive analytics is a mature technology yet only one company in eight is currently utilizing this ability to predict the future of sales, finance, production, and virtually every other area of the enterprise.  What’s holding them back?

In an earlier posting we argued that much of what is holding companies back from adopting predictive analytics is a lack of understanding about how to recognize opportunities and create value with these tools.  It’s equally as true that once the opportunity is realized there is also a knowledge gap in the nuts and bolts of the process and where and how to acquire the talent to produce and maintain these models.  Developing predictive analytic models is a different process from responding to requests for charts and graphs based on existing information.  It is much more intuitive and creative and requires the mastery of different tools.

The mastery of how to use predictive analytic tools is not common and not easy.  You are unlikely to find competence in predictive analytics anywhere in your management team unless you have specifically hired for it.  Once business leaders recognize the opportunity, they follow one of two paths, or a combination of the two.

Grow Your Own:  Some organizations have set out to hire fully trained and competent predictive analysts.  .  There is a large and growing shortage of people with these skills.  A recent survey of individuals actually performing these tasks estimated that it took between two and eight years to achieve competence.  The shortage of educated and experienced practitioners is just now beginning to be addressed by higher education.  This can be a productive approach but requires investment, sponsorship, and an environment in which these folks can learn to apply their skills to each company’s unique problems. 

On the side of the technology industry developing predictive analytic suites there has been a concerted effort to make them simpler to use.  To some extent this has been successful but not to the level these application developers would like you to believe.  These are not tools that your middle managers are going to be able to learn quickly.  Proficiency can only be gained by repeated use, preferably with the coaching and mentoring of highly skilled analysts.  There are still too many ways that these tools can produce the wrong answer.  Be very careful of going down this path.

Outside Consultants:  Much as we would like to avoid expensive outside consultants, this is one area where paying for competence and experience is a good idea.  There are some elements of this practice that really are rocket science.  Using a consulting data scientist as the core of your start-up analytics group where that person serves as instructor and mentor to your in-house dedicated staff is an approach that has proven successful at many companies and grows internal capacity while limiting cost.

Finally, the Complication of Big Data:  In a later post we will address this in more detail, but predictive analytics is now even more complex than it was before, made so by the three horsemen of Big Data:  volume, variety, and velocity.  You can see this in massive data bases in new non-traditional storage formats (volume: Hadoop, NoSQL, and the like), in the new capability to merge and match disparate types of data, some structured, some unstructured, some semi-structured (variety), and in the fast flowing data from sensors and web logs and the desire to take advantage of this in real time (velocity). 

From Data Analyst to Predictive Modeler to Data Scientist: Two decades ago the folks who prepared our reports, graphs, and visualizations were ‘data analysts’ who knew how to extract data from relational data warehouses and run it through reporting and visualization tools like Crystal Reports.  Ten years ago, predictive models were built by ‘predictive modelers’ who understood both the extraction and preparation of the data as well as the specialized predictive analytic tools like SAS and SPSS that allowed them to prepare predictive models.  In the last few years, Gartner now declares that we need ‘data scientist’ who have all the above skills but also understand the complexities of the new NoSQL databases like Hadoop and can marry data from many sources and types together to produce useful and profitable predictive models.  The requirement for broader and deeper skills is real and must factor into any business decision to build in-house capacity, as well as vetting potential consultants.

Business sponsors recognize the great value and quick ROI that is possible from predictive analytic programs.  More importantly, they recognize that their competitors also see this and are probably hard at work capitalizing on these competitive advantages right now.  Organizing for the successful implementation of Big Data and predictive analytics can follow several models.  There are challenges to be overcome but thinking through where you are in the development process is worth the effort.

Bill Vorhies
Editorial Director, DSC

Views: 5725

Tags: analyst, analytics, data, modeler, predictive, scientist


You need to be a member of Data Science Central to add comments!

Join Data Science Central

Comment by Chris on August 27, 2014 at 9:38am

Many thanks for your input shed light to predicte modeler. I am curious on the current day merging data,

how to organize a storage layer that will support different modalities of data, http://www.wired.com/2014/08/robobrain/

Invite you to check a system that enhances the understanding of large datasets through embodied navigation and natural gestures using the mixed reality space called eXperience Induction Machine (XIM) applied in human brain connectome http://www.frontiersin.org/video/BrainX3_Large_Scale_Neuronal_Data_...

Comment by Julie M Van Fleet on August 22, 2014 at 8:21am

Thanks! Those are very useful.

Comment by William Vorhies on August 22, 2014 at 7:54am


Thanks for the question.  I wasn't able to locate the original Gartner study in my library but did find several sources also quoting this 13% figure from the 2013 Gartner study.  Here are a few:



As for predictive analytics being a mature technology, you should be able to track that back on the Gartner hype curve to at least 2009.

Comment by Julie M Van Fleet on August 21, 2014 at 4:10pm

Can you point me to the source for the Gartner reference, e.g., a report, study?

© 2021   TechTarget, Inc.   Powered by

Badges  |  Report an Issue  |  Privacy Policy  |  Terms of Service