Why Are Data Science Leaders Running for the Exit?

Guest blog post by Edward Chenard, Contributor at DataScience.com.

I've had several conversations recently with people I know in the data science space that always start out about business and then drift to the state of data science as a whole. One theme constantly comes up in these conversations: There are a lot of people currently running data science teams at large organizations and the vast majority of them — I believe we are talking about 80-90% — want to leave their jobs. Why is that? Within smaller organizations, the number isn’t as great. So what is going on in larger organizations that is causing such a mass exodus? Having worked in and with many large organizations in a data science leadership capacity, I have a few theories. 

Academia Can't Do For-Profit

Many large companies have fallen into the trap of believing that you need a PhD to do data science. You don’t. Of the top five data scientists I have ever worked with, only one had a PhD and it wasn’t even in stats or data science — it was in biophysics. I call this the "academia trap." There are some smart people who know a lot about a very narrow field, but data science is a very broad discipline. When these PhDs are put in charge, they often discover they are out of their depth. They were never taught how to run a P&L, manage a team, or deal with people, competitive intel, market assessments, building a business case, etc. 

Add to this the world they came from: Many peer-reviewed papers in the academic world that are really good don’t see the light of day. Why? The reviewers may have a competing theory and don’t want their ideas to get superseded. It is shocking how often this happens in the data science space. I have always found the academic world to be more political than the corporate one. When your drive is profits and customer satisfaction, the academic mindset can be more of a liability than an asset. I have yet to see a data science program I would personally endorse. In my opinion, most are run by people who have never done the job of data science outside of a lab. That’s not what you want for your company.

If you’re not willing to give data science managers the training and support they need — or create an organizational structure that brings people with business backgrounds into data science strategy — what will happen is your budget will grow and your results will drop. I’ve seen leadership teams spend $10 million plus a year with very little return. I have never spent that kind of money, yet I have built billion-dollar data products. I’m also not a PhD. The difference is, I know business. PhDs do great work, but the type of work that they excel in isn’t what most companies are asking them to do in this new age of enterprise data science. Don’t make this mistake.

Wrong Expectations

Doing data science and managing data science are not the same, just like being an engineer and a product manager are not the same. There is a lot of overlap, but overlap does not equal sameness. Sometimes, I envy data scientists — much of their time is spent cleaning data sets, testing algorithms, and researching new methods. Compared to the job of someone who needs to run the practice, a data scientist has it pretty good.

"Doing data science and managing data science are not the same."

The leader of a data science practice needs to focus on data governance, MDM, compliance, legal issues around the use of algorithms, and documentation just in case someone sues for wrongful use. There are hiring issues and staffing problems to deal with, budget and funding to gain, P&L to run, business cases to build, market research to conduct, vendor meetings to hold, tech life-cycle management, the evangelizing of projects (both internal and external), and turning that work into data products that sell — all this while trying to ensure profit for the company. Big difference.

Most data scientists are just not ready to lead teams. This is why the failure rate of data science teams is so high right now. Often, companies put a strong technical person in charge when they really need a strong business person in charge. I call this person a data strategist. Right under that data strategist is the strong technical person; there needs to be a very solid and strong interpersonal relationship between the two. If they are competing, it will cause friction and less-than-desired results.

Bad Methods

Agile has taken the tech world by storm. It works fairly well for software development and, as a result, many companies enforce it on data science. But data science is not software development — it’s really a field of discovery, whereas software development is about assembly. I have worked with companies that demand agile and scrum for data science and then see half their team walk in less than a year. You can’t tell a team to solve a problem in two sprints. If they don’t have the data or tools, it won’t happen.

"You can’t tell a team to solve a problem in two sprints. If they don’t have the data or tools, it won’t happen."

Data science is a discipline that requires its own methods. In addition, most companies are still treating data products like they do physical products — but the economics are not the same. When I build a recommendation engine, my cost per unit is pretty much zero, unlike a physical product which certainly has a per unit cost. I can make a million product recommendations with that engine or just one and, other than the electricity, my cost per recommendation would be the same. The cost of a physical product would be a lot more and involve many different variables if I wanted to increase the number from one to million.

We have to understand that the economics of data products are different. A lot of large companies don’t even have this conversation, which can cause a lot of frustration for those in charge of data products. In essence, they have one hand tied behind their back.

Originally posted here.

DSC Resources

Views: 24353


You need to be a member of Data Science Central to add comments!

Join Data Science Central

Comment by Kenneth Tobin on April 20, 2018 at 4:54pm

Bingo!  You hit the nail squarely on the head.  I have observed this same situation in my career as I have implemented financial reporting systems to remediate audit issues.  Also, I have routinely built analytical procedures though-out the audit, working independently and with programmers.  Success in data science requires a team approach and skills needed include the technical data scientist (programming) and CPA (GAAP, IFRS, project management and cost accounting).  Typically neither team member is an expert all by himself because to master each takes a long time ... But as a team, they will be successful most of the time!

Comment by Robert Lake on April 20, 2018 at 4:42pm

There is a lot of bait and switch in job advertisements at the moment. I interviewed for a role that was supposedly for a Data Scientist leader but it was actually for a product manager. I have seen that with several companies this year.

Comment by Vincent Granville on April 20, 2018 at 4:37pm

Also, positions are sometimes advertised to attract PhD's, by minimizing the leadership role and emphasizing some kind of applied research. There is a mismatch between the job ad (and even the interview process) and the actual job. The result is hiring a candidate that is not a good fit. In my career, many times I was attracted to roles where creativity was supposedly very important, when actually, after a few months working for the company, creativity was discouraged, or the job was mostly about coding.

In my case, as a PhD data scientist (an outlier), I spend 80% of my time running a company, in a role similar to COO, and I love it. Most PhD's would hate doing this, and they tend to believe that it is just a mundane occupation (it is not!) The other 20% (the most visible part of the iceberg for people reading my articles) is similar to PhD work (except that over time, I have stopped using jargon and arcane theories). Unfortunately, I would not be a good fit either in the traditional corporate world, but for reasons not related to my PhD. I am wondering if companies are trying to find the elusive unicorn who can do everything. But if he/she can do everything, he/she will start her own company instead. 

I remember Google allowing employees to spend 1 day a week on projects of their own choosing, hoping some good stuff would come out of some of these projects. Are they still doing it? A PhD might fit better in such an environment.

Comment by Desmond Aubery on April 20, 2018 at 3:36pm

To put things in perspective:

A PhD shows that the person is able to conduct research into an academic topic & is able to document their findings & arguments in a Thesis - which they then defend against a group of fellow academics. 

Other than the particular field to study outlined in the Thesis, this says nothing at all about their ability to lead, command, direct, encourage, inspire, coach & the myriad of talents that a good manager requires.

Comment by Paul Bremner on April 20, 2018 at 10:42am

Very interesting observations.  Thanks for posting this.  I’d be curious to know where all these Data Science directors/managers are planning to go and what they expect to be doing.

Below are results of some Indeed searches I did looking for job notices with “Data Scientist” in the title and then various terms in the body of the requisition.  On the bright side, only 43% of the reqs were asking for a Ph.D (the actual number requiring a Ph.D might be lower since I didn’t look to see if it was an ironclad requirement as opposed to just a preference) so the Ph.D thing may be less of a barrier than commonly thought.

No surprise here, but on the not so bright side only a small fraction of reqs even bothered to mention the main entities responsible for driving strategy in companies (Marketing, Sales, Finance.)  This, despite the fact that these entities presumably are some of the main customers for Data Science analysis.  The actual job skills used by Marketing, Sales and Finance (market assessment, market/competitive intelligence, business cases, etc) are virtually never mentioned in Data Science reqs.  So it shouldn’t be a surprise that Data Scientists, hired on the basis of programming and statistical skills, encounter issues when it comes to managing Data Science teams where they must get involved with more “business-y” concepts as they work across an organization.

Requisitions for US Data Scientists (from Indeed, 4/20/18, no date limit)

Total Data Scientist reqs = 3,427

Data Scientist reqs including the following terms:

PhD =1,446 (42%)

Marketing = 781 (24%);   Sales = 369 (11%);   Finance = 347 (10%)

Business strategy = 92 (3%);   Market research = 42 (1%);   Business case = 23 (<1%);   Market intelligence = 9 (<1%);   Competitive intelligence = 4 (<1%);   Market assessment = 0 (<1%)


Comment by Rebecca Barber, PhD on April 20, 2018 at 10:27am

I want to 2nd this sentence: "Often, companies put a strong technical person in charge when they really need a strong business person in charge."  I keep seeing 'Director of Data Science' positions that require the person to also be the star technical performer.  They ask for someone who has a proven track record, strategic vision, and top-notch technical skills.  Then companies use the technical skills for screening.  Unfortunately, many strong technologists get fed up and bail back to individual contributor status once they discover that they will almost never USE those technical skills in a leadership role.  

Comment by Dorothy Hewitt-Sanchez on April 20, 2018 at 5:50am

I don’t want the play down people with PhDs.  I think they are very valuable. If I was working with one, I would feel very grateful.  Besides, they should and probably know more than I do.  However, they need the resources to be successful.   This is where communication and interpersonal skills play a big role.  Everybody on the team is valuable. If they know and understand that, they will be successful.  

Comment by Jevgenijs Pankovs on April 19, 2018 at 9:33pm

Excellent article!

Besides, I see a complete misunderstanding of the data science process. Managers know nothing about TDSPCRISP-DMKDDSEMMA.

Comment by Jeffrey Lapides on April 19, 2018 at 2:29pm

Great article! The comment from above:

But data science is not software development — it’s really a field of discovery, whereas software development is about assembly.

This concept, largely absent from the IT world, states much of the challenge.

Comment by Robert Lake on April 19, 2018 at 10:35am

The other issue is that some hiring leaders feel that you need to have a PhD to manage a PhD. A good leader can be a Data Scientist that has great business acumen and can make strategic decisions. Great article.

© 2021   TechTarget, Inc.   Powered by

Badges  |  Report an Issue  |  Privacy Policy  |  Terms of Service