Home » Uncategorized

IMPACT – Part 2: Be a problem solver first, an engineer second.

This is part 2 of a 3 part series: “How to make your mark on the world as a talented, socially conscious data scientist.” 

You can find part 1 here: “Choose a domain which enables you to create scalable solutions to m…

We looked in the first article in this series at how a socially-conscious data scientist might choose a domain to make the greatest impact. This article focuses on how to maximize the impact you can make on your organization once you’ve chosen your path.

***

The world needs innovative leaders, critical thinkers, and pragmatic business people who know how to use data science methodologies to solve real problems.

This requires a thorough grasp of the business model, operational challenges, and markets in which your product or service operates. First and foremost, it requires a deep understanding of the customer you’re serving. All too often, the ambitious and talented data scientist forgets about the real world implications of their hyper-optimized ML model. (Hyper-optimized for what, exactly?)

Adversarial neural networks are not to be fetishized.

The wheat farmer does not lie awake at night thinking about how to make a better plow. He’s thinking about how to grow more wheat on the same amount of land. If a better plow will help him increase yields – and he knows how to make one – then he will develop that plow. But the plow is only the means to an end. In the same way, the compute, storage, modeling, visualization, and networking technologies at a data scientist’s disposal are tools in a toolbox – not an end unto themselves.

You don’t improve your product by tweaking hyper-parameters on your regression model to get a better MSE – at least, not necessarily. Improving your error metric is only useful insofar as your model is correctly aligned to deliver a practical solution to the real world problems at hand. Before you start comparing the performance of a random forest versus a support vector machine versus a neural network, be damn sure you are optimizing for the right problem.

The data scientists who will transform our world and solve humanity’s greatest challenges are the ones who can wrap their heads around complex challenges and apply their technical expertise toward solving them. Crucially they are they then able to communicate with their technical and non-technical colleagues, their customers, investors, and their partners – to coordinate end-to-end testing, delivery, and operationalization of products which provide useful solutions to real problems.

*

Focus on the physical implications of digital innovations.

There are an ocean of talented engineers in the world, and universities will only continue to churn out more developers. That’s a good thing – we need them. What we will likely continue to lack, however, are talented problem solvers.

As a former colleague of mine – a senior engineer himself – once astutely observed: “The problem is, people often think, ‘my job is to connect the Hadoop cluster to Kafka, to the production DBs, and so on’. Some of them never stop to consider who will actually be using the product, and how.”

We need engineers who are business people, leaders, and problem solvers. Be the person who connects the dots. Think big picture. Focus on the physical implications of digital innovations. Keep always in your thoughts the human beings whose lives you will improve – or fail to improve – through your work.

*

Be as well-versed in your company’s business model as your CEO

To be a great data scientist you need to have a firm understanding of your company’s business model. You need to know your customers, their pain points, and the way your company’s product provides solutions – and where it still falls short.

You should have a high level understanding of how your marketing team sources leads and what kind of conversations your sales team has with prospects. Who is your competition – and what is their unique value proposition? How does your company deliver and support its product, and what does its supply chain look like?

Regardless of your specific area of focus as a data scientist on a team within a larger organization, you need to know the answers to these questions. You will inevitably make a larger contribution to the organization if you first understand the organization and its customers, from end to end.

Talented data scientists have hugely valuable and in-demand skills. They’re skills that can make a profound difference in the world – or not; help solve real challenges that face millions of people – or not.

Perhaps you see yourself at the helm of your own company. If this is you, it is even more important that you assume the role of problem solver and pragmatist first – and engineer second.

***

In the first post in this series, we explored the kinds of global, systemic problems that data scientists should look to help solve, to make the most difference for the greatest amount of people.

In this post, we examined the importance of asking the right questions, thinking big picture, and understanding how to use your tools as a data scientist as a means to an end – rather than ends in themselves.

In our next post, we will finish the series with a discussion around why the best data scientists are world class communicators.