Subscribe to DSC Newsletter

Why You Should be a Data Science Generalist - and How to Become One

The new advice today for data scientists is not to become a generalist. You can read recent articles on this topic, for instance here.  In this blog, I explain why I believe it should be the opposite. I wrote about this here not long ago, and provide additional arguments in this article, as to why it helps to be a generalist.  

Of course, it is difficult, and probably impossible to become a data science generalist just after graduating. It takes years to acquire all the skills, yet you don't need to master all of them. It might be easier for a physicist, engineer, or biostatistician currently learning data science, after years of corporate experience, than it is for a data scientist with no business experience. Possibly the easiest way to become one is to work for start-up's or small companies, taking on many hats as you will probably be the only data scientist in your company, and will have to change jobs more frequently than if you work for a big company. To the contrary, for a big company, you are expected to work in a very specialized area, though it does not hurt to be a generalist, as I will illustrate shortly. Being a specialized data scientist could put you on a very predictable path that limits your career growth and flexibility, especially if you want to create your company down the line. Let's start with explaining what a data science generalist is.

The data science generalist

The generalist has experience working in different roles and different environments, for instance, over a period of 15 years, having worked as a

  • Business analyst or BI professional, communicating insights to decision makers, mastering tools such as Tableau, SQL and Excel; or maybe being the decision maker herself
  • Statistician / data analyst with expertise in predictive modeling
  • Expert in algorithm design and optimization
  • Researcher in an academic-like setting, or experience in testing / prototyping new data science systems and proofs of concept (POC)
  • Builder / architect: designing APIs, dashboards, databases, and deploying/maintaining yourself some modest systems in production mode
  • Programmer (statistical or scientific programmer with exposure to high performance computing and parallel architectures - you might even have designed your own software)
  • Consultant, directly working with clients, or adviser
  • Manager or director role rather than individual contributor
  • Professional with roles in various industries (IT, media, Internet, finance, health care, smart cities) in both big and small companies, in various domains ranging from fraud detection, to optimizing sales or marketing, with proven, measurable accomplishments

In short, the generalist has been involved at one time or another, in all phases of the data science project lifecycle

The generalist might not command a higher salary, but has more flexibility career-wise. Even in a big company, when downsizing occurs, it is easier for the generalist to make a lateral move (get transferred to a different department), than it is for the "one-trick pony". 

Timing is important too. If you become a generalist at age 50 (as opposed to age 45) it might not help as getting hired becomes more difficult as you get past 45. Still, even if 50 or more, it opens up some possibilities, for instance starting your own business. And if you can prove that you have been consistently broadening your skills throughout your career cycle, as generalists do by definition, it will be easier to land a job, especially if your salary expectations are reasonable, and your health is not an issue for your future employer.  

An example

I consider myself to be a generalist, and here, I explained how I became one, and the benefits that it provides both in small and large companies, and especially as an entrepreneur. It is true that there are many areas that I do not master, but as a generalist, you can hire the right employees to do what you are not so good at, or what you enjoy less. 

I started my career in the academia, and I was expected to become a university professor. I failed an interview for such a position at Iowa State University around 1997, and realized soon after that there was too much competition for me to succeed in this endeavor.  Twenty years later, I think this is the best thing that happened to me: not only do I continue to do research and publish (independently) but I don't have the pressure to "publish or perish" and can write articles or books, even state-of-the-art, that are accessible to a much larger audience. I had to unlearn how to write esoteric scientific articles in the process.  But I recognize that did gain a strong research training during these PhD and post-doc years.

Later on, I worked for a number of Internet companies, between 1998 and 2002, in the volatile bubble years of the Internet. No job was safe, and in the process I had to change jobs multiple times. That was the beginning of my corporate years, and having worked with many smart people in different places, if anything, was the beginning to start having a large network of influential people, and getting noticed. While market volatility scares many people, if you can handle the risks (I was young back then, which makes it easier) it can boost your career and experience as a generalist. I played in the stock market back then, sold my home in the Bay Area at the best time, worked for Visa and Wells Fargo for a while, which further diversified my experience. This would not have been very successful without trusting my gut instincts (what to do, when to sell or buy or where - activities that involve analyzing data but also intuition.)

At Wells Fargo, my biggest contribution was about identifying a big IT issue that made all the Internet logs completely wrong, even though I was hired as a BI analyst. I am still remembered at that bank for fixing this multi-million dollar issue. I could not have done that had I been a pure data scientist working exclusively on the mission I was hired for in the fist place. My previous experience in failed Internet companies is what helped me, as well as asking a salary a little below average to increase my chances of being hired. 

At NBCi (at their peak they had 1,600 employees) I was the key guy who helped them NOT sign a multi-million deal with Netscape, based on past experience and analyzing the poor traffic coming from that source. Also, although hired as a BI analyst, I knew how to run scripts (Perl scripts at that time) to extract/summarize data from databases and deliver it to my boss automatically by email, in a way that quickly conveys all the important information he needed about weekly business metrics. Usually, BI people do not know how to code to that level. My boss complained one day that I did not work much (I had automated my job of course) but he still was happy. The time I saved doing the number crunching by automating it, I used it to start my own business. 

Later in 2010, I trained BI people to write their own queries in Perl. I coded the low level part, all they had to do is put their SQL queries in a text file at the right place (on the right server) then enter a simple one-line UNIX command to run their query. It saved them a tremendous of time (they used Toad before, a very, very slow process to extract big data out of Oracle databases) and everyone was happy. Note: you need to be a very good friend with your IT team to have them approve this process.  Again, despite me being perceived as someone not working a lot (that is, working smart rather than hard) everyone was happy (my boss -- the CEO --, BI analysts, and myself.) I also mentioned some Google and Bing API's available to download large list of keywords with pricing and volume information - a risky move for me as it could have killed my job consisting of building these lists myself manually, even though it requires smart web crawling and intelligent NLP to build these keyword lists  (it did not killed my job.) Something even the software engineers (in a search company!) were not aware of. They ended up using my Google/Bing accounts (and even my credit card to pay for the services!) to optimize this process, though after initially using my own Perl code to access the API's, my software engineer colleagues switched to Python.  

I took advantage of this free time to further develop my business, and that was pretty much my last experience working for a company. Along those years, I learned enough to run my business, I was ready to become full-time entrepreneur working from home, which is what I did. My experience with new business models was more extensive than that of MBA graduates from top schools, and I was no longer afraid by competition. (that said, a good MBA degree is great for all the fantastic lifetime connections you will make during your college years.)

Even in personal life, being a generalist helps: buy or sell a home (or investing in general) with better outcomes than average, blending both your data science experience with intuition to maximize results. When we launched DSC, I knew more about tax, legal issues, advertising, marketing, sales, IT (automating processes), editorial process - especially regarding managing the specific business in question - than many who have a degree in those fields. It helped a lot saving costs, not having to hire tons of specialized experts. Along the way, I also learned how to significantly increase my negotiation skills, to the point of being able to purchase homes that few would qualify for (and reduce the price without sounding aggressive), the most recent one resulting in a 7-digit loan, even though these other home buyers had a stronger position than me in the first place (them being a US citizen, not born in poverty, on a payroll, working for some big company, sometimes very well paid, with an official job title - I have none of this, other than provable, decent, stable / increasing, income, as proved by many years of tax returns.) 

These are some of the benefits of being a generalist.

To not miss this type of content in the future, subscribe to our newsletter. For related articles from the same author, click here or visit www.VincentGranville.com. Follow me on on LinkedIn, or visit my old web page here.

DSC Resources

Views: 10297

Comment

You need to be a member of Data Science Central to add comments!

Join Data Science Central

Comment by Sergey Buduchin on March 10, 2019 at 12:28pm

Five stars!

Comment by Vincent Granville on January 15, 2019 at 5:23pm

Hi Geoff,

I believe that getting too many high credentials (PhD's, patents, books, etc.) will hurt you just as much as having too few. At best, if everything goes well. you will end up being a multi-millionaire, but not a billionaire. 

My experience is that getting very original and street-smart about running a business, doing stuff that you won't learn in MBA classes, even without business plan (my case) can help tremendously. At the same time a good MBA will not teach you how to operate a modern business, but it will give you tons of highly valuable contacts (your classmates) and it is worth well the price you pay for that kind of education.

Best,

Vincent

Comment by Geoff Gordon on January 15, 2019 at 2:38pm

Hi Vincent, Thanks for sharing this post.  Could you speak to "how to become one" a bit, particularly the need for an advanced degree, and the PhD vs. MS options?

Thanks Geoff

Comment by Willie Zhang on December 28, 2018 at 2:14pm

Thanks Vincent! That was really helpful

Comment by Dinesh Mistry on December 27, 2018 at 12:13pm

Excellent post. Thanks for sharing.

The part about learning to write scripts caught my attention. After reading your post, I went back and counted all the scripts I wrote (in R) last few years related to various data quality inquires and remediation - there are 207+. I mostly kept quiet about the scripts since I work around programmers and non-programmers (including me). Non-programmers are supposed to figure out problems using Excel. I have 10+ years on you and still enjoy scripting and the power it brings to my work. It was nice to know it wasn't just me.

BTW - I can relate completely with the generalist role.

Dinesh

Comment by Tuan Vu on December 27, 2018 at 9:20am

Great article! Thanks for sharing your wisdom and experience in your 20 years journey to become a data science generalist.

Comment by Vincent Granville on December 27, 2018 at 6:40am

Hi Willie,

To answer your question, I was born in 1964. I really started entrepreneurship around 2006, when a business guy approached me regarding some new technology that I had developed (click fraud detection), and that I was well known for. We eventually became partners and raised VC funding to start a company. It did not last long and by 2008 I had discovered Ning (the platform used by DSC; all blog posts here are still hosted on Ning as of today) as a very inexpensive, do-it-yourself way to launch a social network. Back then, Ning was very popular with recruiters. I posted articles about analytics -- the name was Analyticbridge.com, and it was operated by Datashaping.com back then. It grew pretty fast. Around 2010 I was selling newsletter advertising and job ads automatically posted by recruiters. I was not making a lot of money, but it was not peanuts either; by 2012 about 35% of my income came from that activity, and 65% came from a full time job. In 2012, I was approached by two guys who wanted to acquire my little company, for a little sum of money. I did not sell, but we entered a partnership that still exists today, and we launched DataScienceCentral.com together. It started to grow much faster, both in terms of traffic, revenue, and people working with us. That is my story.

Even as early as 2002, I was running little apps that predicted stock market trends in real time, or to perform copyright infringement searches in real time, based on web crawling technology, offered to clients for a fee, paid by credit card on my e-store. I was making very little money out of it, but had a few happy and faithful customers. On occasions, I made more money working (consulting gig) with an hedge fund.

In my case, the most important skill was the ability to run a business with very little overhead costs and high profit margins, so that finances would never be a problem. I did not work from a garage contrarily to the popular myth, quite the opposite. But not having an office helped a lot - there was no more commute, and I saved a lot of time, to focus on my business. Also, automating (or outsourcing in India or Eastern Europe) anything I could automate helped a lot, as well as producing high quality content (initially myself, then from our members.) Finally, back-up plans (in case Ning went bankrupt for instance), were critical to reduce some stress and make the environment safer and more robust. So was growing faster than competition (initially I had no competition.)

Best,

Vincent

Comment by Willie Zhang on December 26, 2018 at 6:56pm

Excellent article.  Would love to learn more about your entrepreneurial experiences.  At what point did you realize you could start your own business and what specific skills were most important to you?  What sort of businesses did you start?  

Videos

  • Add Videos
  • View All

© 2019   Data Science Central ®   Powered by

Badges  |  Report an Issue  |  Privacy Policy  |  Terms of Service