Subscribe to DSC Newsletter

As a data scientist, I have helped business analysts write Oracle queries in batch mode run 20 times faster via Perl/Python scripts: instead of waiting for hours for Brio or Toad to return results in a browser (with a crash if the number of rows being returned was above 50,000), it took only minutes on a Unix station: I also spent 30 minutes providing basic Unix training to analysts so they could run and fix my Perl scripts alone. It was a win-win-win: for the business analyst, for executive management, and for myself.

One thing that helped a lot is the fact that my scripts would accept (as input) a SQL query (text file - something business analysts could easily and quickly produce with Brio or Toad and then export to the Unix station) and produced as output the results of the query (tab-separated text file) and another file about the success/failure of the query (with explanations as why the query failed, if it did, to help fix bugs in the SQL code). This made the life of the business analyst very easy. I also taught them about 20 important Unix/FTP commands that could solve all their Unix needs. 

I have been deeply involved (as a data scientist) in helping design efficient dashboards as well as selecting which metrics or KPI's should be tracked in database systems, including data modeling. Currently many people believe this is the job of a business analyst or executive, but don't you think that data scientists should be deeply involved in data modeling, vendor selection and everything related to data harvesting and storing?

What do you think?

Anyway, feel free to download the magic code to boost BI analysts work by a factor 20.

Views: 8013

Reply to This

Replies to This Discussion

I completely agree.   There's the IT-side that does what the business tells them and the business-side that does not really know how to translate business into data.  The data scientist / data analyst is the middle-person that understands how to turn data into value and work/communicate on both sides.

I think their relation is like data base architect and network architect. While their work overlaps, they are specialist in their own area and probaly won't enjoy doing everything else the other one is doing.

A Business Analyst is more of a functional role.  Whereas a Data Scientist is more like “The-All-Seeing-Eye”. The Data Scientist will always follow/create/verify the logic that is necessary.  The Data Scientist will clarify and authenticate the requests from the Business Analyst.   Not all Data Scientist will function in this role because this is not their main focus.   The Data Scientist will be more heavily concentrated in the integrity of the structure, processing, speed of delivery. So, this is just my opinion, I think they will be more focused with the system side – machine learning, database- DBAs, data warehousing, and maybe a little help with the application developers.  However, a data scientist is not a application developer or a frontend BI designer.    Data Scientist will be the master mind of answering the question, "How do we put all this data together in a meaningful way so we can examine it”? They create the matrices.


It seems to me that a BA's job is not to get up close with the technology but rather, for example, to identify which operations need to be accelerated or what business problems need to be adressed. Data architects aid in the structuring of data. Isn't data science, at least where it is today, mostly aposteriori? After all, data science has existed for a long time, that is why SAS is so rich. I remember them from before they entered businesses ans were competing with SPSS.

Open to learning,

Sometimes business analysts who have worked for a long time within an industry, and especially within a single company, have a lot of business process knowledge that explains how certain data came to be coded into the db a certain way -- that can be of tremendous value to a data scientist who is less familiar with the origins of the datasets they use.  Enterprise data are rarely so clean and simple to untangle that any newbie can figure out all the linkages that you need to pull together before you have a dataset that you can work with.  Furthermore, the origins of the data may hold vital clues as to whether that persistent outlier you see is significant, or if it is just something that came about because someone entered a typo somewhere and it happened to propagate forward unchecked.  Those origins can sometimes offer valuable clues when you have to re-engineer a dataset to suit the purposes of your specific modeling task.  I personally would be lost without the business analysts who help us out on a daily basis.  So, I don't think a data scientist can adequately replace the functions of a business analyst anytime soon.  The BA has often (through long experience) done the hard work of translating the lingo and thinking of the DBA into something the rest of us can understand and use.  Assuming that Data Scientists don't change industries, or jobs, then *maybe* that DS would be able to replace a BA.  But, I think that's a bad assumption in general given that by nature a data scientist is hungry for new challenges and problems to solve.  If a company doesn't offer them that continuous challenge, they are more likely to take their skills somewhere else.  So, if my view is correct, it will be easier to hire and retain Business Analysts, than it is to hire and retain Data Scientists. 

Whether business people see the merit of ds in their companies is their choice. However, with the intense competition (esp. in this suboptimal economic climate) I don't think the mindset with walk. The future IMO belongs to the business people who are insightful enough to see value in hiring ds in their ranks, while everyone else will be weeded out by natural selection (just like the social media before linkedin and facebook gradually died when the latter appeared on the web).


I was surfing the web for my research on a data problem, happen to found this ...very few of the companies apply data science starting with a true definition of business problem and solving using data science techniques.. this looks  promising check




  • Add Videos
  • View All

© 2019   Data Science Central ®   Powered by

Badges  |  Report an Issue  |  Privacy Policy  |  Terms of Service