Editorial Note: Ajit Jaokar’s contribution this week hits home a point that I think it very important. As with any job, it is the value that you bring as a problem solver and someone capable of looking at obstacles as challenges rather than the base skills that you have in a given discipline, that will ultimately determine whether you succeed or fail in your career. — Kurt Cagle
The media loves to glamorize the data science profession. Data science is supposed to be the hottest skill of the 21st century (variously attributed to HBR, Tom Davenport and DJ Patil).
But banner headlines are mental shortcuts and hide much detail.
Lets explore this idea further
Early in my career I wanted to be a DBA (Data base administrator).
This ambition was inspired by a DBA at our project who had a God like presence almost like a surgeon.
If your SQL query ran slowly, he would come over and suggest a tweak or two and it would then almost be fixed magically!
Fast forward a few years and today, a senior DBA in the UK makes 53K GBP on average as per glassdoor.
Far less than most data scientists make.
And overall, data science salaries are increasing and DBA salaries not correspondingly so
But the question is, will the data science job salaries also become like the DBA role i.e. technically strong but paid relatively less over time
There is a dystopian view presented by a book I read called average is over
The books main thesis is based on a concept called hyper-meritocracy which creates a few big winners and a large number of people who have unskilled jobs at the lower end of the skill set.
But the middle will be hollowed out (hence the average if over).
So, the operative word is average.
Indeed, there are very today highly paid DBA roles but far fewer in number.
And the rest of the roles have a downward pressure on salary lowering the overall average
And the biggest driver of this downward trend is technology itself
So, is the same likely to happen to the data science profession?
Or are we, somehow, magically immune to the downward pressure of technology?
I believe that the hottest job of the 21st century slogan hides some detail which is often overlooked
The Cloud, AutoM, Low code technology trends are already driving down skill required for basic data science roles
So, what should you do to remain valuable as a data scientist?
Here are some ideas depending on your aptitude and existing focus, one or more of these could be useful for you
- Solve hard problems
- Focus on fundamental research in AI (which will develop into products in the next 4/5 years)
- MLOps
- Inter disciplinary problems
- Areas not yet impacted largely by AI (ex Engineering)
- Work with very large datasets
- Work with techniques for small datasets (ex in bioinformatics etc)
- Develop defensible IPR
- Finally, Develop skills to be an AI product manager
I see an AI product manager as someone who understands business problems quantitatively and can work with others to develop a product that can demonstrate improvement in the existing state of the art. She can also demonstrate that the solution is superior.
The AI product manager will be a key role that will be hard to fulfil in my view
So, to conclude, everyone in the data science industry will soon need to think how they want to reposition themselves
The value of your skill is at a point in time and often tied to your current job
This will be eroded over time
But the value you provide beyond your skill based on the ideas above is potentially longer lasting
