Home » Uncategorized

A Self-Study List for Data Engineers and Aspiring Data Architects

This article was written by John Hammink. John is Chief Evangelist for Treasure Data. An 18-year veteran of the technology and startup scene, he enjoys travel to unusual places, as well as creating digital art and world music.

With the explosion of “Big Data” over the last few years, the need for people who know how to build and manage data-pipelines has grown.  Unfortunately, supply has not kept up with demand and there seems to be a shortage of engineers focused on the ingestion and management of data at scale.  Part of the problem is the lack of education focused on this growing field.

Currently, there seems to be no official curriculum or certification to become a Data Engineer or Data Architect (that we know of). While that’s bad news for companies that need qualified engineers, that’s great news for you! Why? Because it’s basic supply and demand and data engineers and data architects are cleaning up! In fact, we did a little research and found that the average salary for a data engineer is around $95.5k.  While the salaries for data architects average around $112k nationally, the main path to this strategic, coveted position (and salary) means cutting your teeth as a data engineer and working your way up or making a lateral job move. 

What does it take to start getting a piece of this action? Don’t worry: we did research on that, too!  We did a survey of a pool of  ‘Data Engineer’ and ‘Data Architect’ job ads on LinkedIn and GlassDoor and ranked the top 20 sought after job skills.  They are:


Percentage of times these skills showed up in data-related job descriptions

Keep in mind that most job ads (and recruiters) are behind the curve on invoking new technology, (where’s Fluentd?!?!  What about ELK Stack?) because recruiters and HR typically get their information second hand.  Plus, cutting-edge technology typically enjoys a period of minimal adoption while it is being tried out.

If you’re starting from the beginning, and set an end goal to become a data architect, the first step is to learn the skills of the data engineer.  While this article won’t – and can’t –  completely connect the dots for you, our aim is to get you thinking about this actively and starting to look for yourself in the right direction.

To read more and see the original article, click here

Top DSC Resources

Follow us on Twitter: @DataScienceCtrl | @AnalyticBridge