Home » Uncategorized

The Halley’s comet Data Scientist


Image source: Wikipedia


This post is about how not to be the Halley’s comet Data Scientist i.e. to keep yourselves motivated in your Data Science journey

The views expressed here are my own

Many professionals want to transition their career to AI.

But most will not

Often, we see a specific type of learner – who I call as the Halley’s comet Data Scientist

Like the Halley’s comet .. they shine brightly for a brief time ..

Then streak on into the darkness ..

Never to be seen again for a long time

But periodically after a long time .. they reappear .. bright and enthusiastic

Only to disappear off again ..

And so it goes ..

How to stay motivated as a Data Scientist?

Comets apart, we can see this phenomenon as a motivation issue i.e. instead of saying the learners are giving up on Data Science we could ask – “How could we stay motivated on the Data Science learning path”

Data Science is a hard and a complex subject ..

From my personal experience, here are things to know to stay motivated on your data science path

1) Your information diet – Read and talk to experts

One of the most motivating interviews I read is  Andrew Ng’s interview On Life, Creativity, And Failure  

In response to Can you talk about your information diet, how you approach learning? Andrew Ng says

I read a lot and I also spend time talking to people a fair amount. I think two of the most efficient ways to learn, to get information, are reading and talking to experts. So I spend quite a bit of time doing both of them. I think I have just shy of a thousand books on my Kindle. And I’ve probably read about two-thirds of them.

At Baidu, we have a reading group where we read about half a book a week. I’m actually part of two reading groups at Baidu, each of which reads about half a book a week. I think I’m the only one who’s in both of those groups [laughter]. And my favorite Saturday afternoon activity is sitting by myself at home reading.

1000 books and more papers … we all have a long way to go – but it shows you how much information you need to absorb

But the motivating idea is .. it’s all there for us .. often free and from many people on the web who are genuinely helpful in sharing knowledge

2) Imposter syndrome in Data Science and how to handle it

How to manage the imposter syndrome in data science is very much needed to stay motivated. HERE is a good link on this subject

The way that I’ve dealt with imposter syndrome is this: I’ve accepted that I will never be able to learn everything there is to know in data science — I will never know every algorithm, every technology, every cool package, or even every language — and that’s okay. The great thing about being in such a diverse field is that nobody will know all of these things (and that’s okay too!).

3 People claim to know more than they do

In contrast to the Imposter syndrome .. there is a contradictory phenomenon ..

People will claim to know more than they really do

If you do not realize this, it can make you feel demotivated

Early in my career, I attended a talk where the speaker was very knowledgeable

He decided to explain a complex subject in layers of depth…

He would explain to a point and stop .. and ask the audience .. should he go deeper ..

And what did the majority say?


Every time ..

Did the majority really get the talk?

I doubt it

But everyone wanted to seem knowledgeable!

No one wanted to appear to be clueless

But at one point .. everyone was ..

4 Passion is over rated

Andrew Ng also points out in the same interview that Passion is overrated and says ..

“I wish we as a society gave better career advice to young adults. I think that “follow your passion” is not good career advice. It’s actually one of the most terrible pieces of career advice we give people.

If you are passionate about driving your car, it doesn’t necessarily mean you should aspire to be a race car driver.

But often, you first become good at something, and then you become passionate about it. And I think most people can become good at almost anything.”

That means that there will be a lot of hard work before you are passionate about Data Science

5 Women in data science

If you are a woman in Data Science, like in many fields, you face extra challenges.

why you should encourage your daughters to become data scientists provides a good overview of significance of Data Science for women

6) Offline meetups

Offline meetups are great for learning complex areas. I started the Data Science for IoT meetup and now we have grown to more than 2000 members. We have also launched an AI lab spun out from the meetup. All these are great learning experiences but also a support group of sorts

7) I get by with a little help from my friends ..

I have a great network of friends who I can sound out to check/understand technical ideas. Dr Kirk Borne, Cheuk Ting Ho, Dan Howarth, Sebastian Raschka, Dr Brandon Rohrer and others. Everyone needs such friends!

8)  Learning to handle negative experiences

As a Data Scientist, you have to often learn to handle negative experiences to stay motivated on your path. For me, these have included: companies who did not have data but wanted to create a Data Science algorithm, companies who over promised to investors, companies who wanted to use specific tools just because they had done so before, companies who refused to consider a Cloud based model even when they had no resources in house etc.

9)  Persist in coding

Persist in coding .. again many ways to achieve this for example #100DaysOfMLCode

10) A vision of the promised land – quantitatively  ..

You could create a set of quantitative goals over a period. Say a number of applications in github over three years. You could also focus on a vertical by choosing datasets in a vertical. This is all achievable even with very limited resources.  


And there is one more way to motivate yourseleves ..

Its Fear ..

Like it or not AI and Data Science will impact many jobs -a study finds that up to nearly of jobs are vulnerable to automation. That’s enough to keep most people motivated.

Leave a Reply

Your email address will not be published. Required fields are marked *