How to compete against data scientists charging $30/hour

While companies complain about lack of analytic talent, professionals complain about lack of jobs. Everyone wants to work for Facebook, LinkedIn, Google, Intel, Apple, Twitter or some hot start-up. It creates fierce competition getting a job interview, let alone a job. But companies that do not belong to this circle see very few candidates applying for their data scientist open positions; in addition, they are only hiring what I call technical developers (defined by a narrow set of technical skills, usually R, Python, NoSQL, Hadoop, Map-Reduce, software engineering). They are not interested in real data scientists, so many data scientists that would apply would (erroneously) not be perceived as bringing value, and not interviewed.

The problem with consulting is of a different nature. Companies are looking for the cheapest consultant having the minimum set of qualifications to perform the task (the candidate will be asked to provide details about previous projects). Because the work is performed from home, consultants compete with people all over the world to land a gig. Analytic professionals in India, found on websites such as Elance, charge $30/hour. On Statistics.com, you can hire consultants in India for $59/hour.

When I wrote my article about my salary history, a few people mentioned that my consulting rates (from $45 to $100/hour) were absurdly low given my expertize. But compared with rates in India or Romania, it is actually not low. Those charging $150 to $250 per hour are having a difficult time finding new clients. And if all your great skills and expertise are not considered useful to a client, he won't pay for it, especially if this expertise is not used to generate greater revenue. Indeed, many PhD statisticians work as part-time adjunct professors with salaries even far lower, or write other PhD students theses for a fee - typically for $5,000 - as these are the only clients that they can get. So, in some sense, there is more talent than the job market can absorb, especially for PhD's.

So what are the solutions, for a consultant?

Here's a list of ideas:

Work on big data projects, a lot of automation is possible, and big data is cheaper to process. You need the correct expertize though, but many companies lack talent to properly identify the right data and extract value. I would charge by the project rather than by the hour, for such projects.
Work on stuff no one else know. Avoid stuff everyone know such as logistic regression, decision trees, neural networks, as you will be competiting against dozens of professionals who can do that too - some in India or Romania.
Automate your analysis and EDA (exploratory data analysis) as much as you can
Outsource mundane tasks (reporting, data gathering and cleaning, EDA) so you can lower your bill: use an intern or someone in India paid $20/hour to do 80% of the analysis, pay yourself $200/hour for the 20% high level stuff; the average turns out to be $56/hour for the client - a rate lower than what haircut professionals charge.
Make sure you can measure value added and convince clients about the amount of (quantifiable) added value that you provided. Many clients (you need to find them - I am one of them) would not mind paying you $100,000 if you helped them generate another $500,000 in revenue, even if it took you only 50 hours of work.
Reduce your costs: I work entirely from home (I don't need a car), I have no health insurance (by choice, not a good idea for many people), I bought a house well below what I could afford (and same with car), I use open source tools, and my education is mostly free (no expensive training, I read stuff online, buy books). I'm lucky to have no marketing costs; if you can (you need to build a strong social presence), your marketing costs - which typically eat 50% of your revenue and include attending conferences - could drop to zero. I can charge low rate because I have no expenses.

Some arguments to convince a client to work with a more expensive, US-based consultant

Your data is confidential: it is protected in my hands. Data processed outside of the country could destroy your IP, patents, can be re-sold, can cause privacy violations etc.
Easier to sue me or collect money in case of problems
Easier for you to deduct these consulting fees (for the IRS)
We are in the same time zone, avoiding unecessary delays every day
We can meet in person rather than via video-conferencing
We can work on sensitive data (security clearance)
We are an established business, not a one-man operation
Our English is easy to understand
We use professional, enterprise tools; we have a SAS license (we don't use pirated software or virus-laden computers)

Finally, if you really have great expertize spanning across multiple domains, the easiest solution might be to just stop consulting and make a living as a business or growth hacker:

Instead of helping businesses protect themselves against hackers, you become a hacker, knowing that you can outsmart all the consultants and experts working for these companies. I'm talking about legal business and growth hacking. You can create your company, for instance a website selling books listed on Amazon; you don't actually sell the books, you get a commission each time a website visitor goes to Amazon to purchase a book listed on your website. Traffic hacking (one of the hacking systems among many others used to optimize your business) could consist in generating a huge volume of high quality web traffic, through creation of hundreds of (fake) interesting profiles that automatically post interesting stuff via a well diversified set of mailing lists and social networks, without being detected (each profile posting no more than 3 links or pieces of content per day; a different IP address is used for each profile). Your business acumen, network security, traffic scoring, and fraud detection expertize allow you to defeat the algorithms designed to block you. Instead, these algorithms generate false positives because they rely heavily on spam reported by users; you can take advantage of this to get your competitors blocked. Note that this business model does not require any sales or talking to people and is typically run from home. Other advantages include higher revenue, no meetings, no boss, and better job security. If you are good at financial engineering to reduce taxes and other money issues (I call it financial hacking), you will even make more money.

Other resources

7 members like this

Comment

You need to be a member of Data Science Central to add comments!

Join Data Science Central

Comment by Roman Gavuliak on March 9, 2015 at 11:10pm: Vincent,

Thanks for the reply. I just wanted to provide the other perspective since I felt the article was too one-sided. You are right, that Data Science in Europe isn't as big as it is in the US and some countries are better off than others (UK) and I personally come from a small country where it is well possible that the number of people really doing data science is certainly below 100 and possibly even below 20. I was recently hiring for a Data Scientist position and it wasn't easy. That's why I think people who do real data science over here are really enthusiastic and invest a lot of their working and free time to develop their skills and that's why I think some of the points don't make them (us) justice. Some of yours points are still valid.

Best,

Comment by Vincent Granville on March 9, 2015 at 8:57am: Roman, I am an EU guy myself (from Belgium) and even though we are in US (I now live near Seattle), our tech guy is in Eastern Europe. What I suggested apply to consultants in US. My favorite one is "automate your projects, choose projects that you can easily automate". This way you can reduce your hourly rate by a factor 3, and yet make twice as much money as before, because you work 6 times faster.

What I remember about Belgium 20 years ago, is that there were almost no jobs for data scientists. Not even freelancing opportunities. Most of my colleagues ended up working as programmer, for a bank or for the government. That's also why I left Belgium. Maybe things have changed now. I see some folks working in clinical trials, and very rarely, working in marketing analytics.

Comment by Roman Gavuliak on March 8, 2015 at 11:28pm

Dear Vincent,

Please take into account that this site is read by professionals outside of the US as well. As a one of such people I have to disagree with quite a few points. For the record - I live in the EU, so you can assume the rule of law is decent. I also do not do much freelancing, because I have interesting work in my main job, but I have scanned the freelancing sites myself.

Yes, it may be demotivating for the US Data Scientist to be undercut by one from other countries, but the truth is that we are not responsible for born outside of the US and my living costs are lower. I would much rather charge rates comparable to the US Data Scientists because in many cases I feel I can do the job on par or better than a person from the US would, but I can't charge the same fees, because my country of origin isn't US. Now to counter some of your points:

Re-selling data? Have you ever mat a freelancing data scientist from outside of the US? Because this just seems as if you are trying to paint a black vs white picture where they are just evil. I do not think this is a fair argument in the debate.
Easier to account for in accounting - you can get an invoice from anywhere and I would suspect it is not all that different to when I order something from the US, but maybe I am wrong, do you have any experience with how difficult it is?
Time zone - the difference between NY and SF is 3 hours, the difference between me and NY is 6 hours (and I am usually up during the 8-6 working hours of that), so again, this advantage is relative on a case by case basis.
Meeting in person, I think this depends on the scope of a job, if you have a consultancy job, would you fly across the US in every case? Probably not, so the teleconferencing vs in person argument is not always applicable
Again, I highly doubt every freelancing data scientist in the US is a part of an established organisation rather than a one man individual and again there are established organisations and networks outside of the US. A very bad generalization.
Yes, a person outside of the US is prone to have a worse proficiency of English as opposed to a native speaker. I myself do not claim to have a perfect command of English, but you are reading my post which conveys hopefully coherent arguments. Besides most of the Data Science literature (papers, books) are in English, so all the technical terms are a non-issue.
You would be surprised, SAS exists and is used outside of the US. Besides R and Python are valid tools and they are for free, so I do not see a reason for a Data Scientist outside of the US to pirate software. Your mention of virus-laden computers is just silly - again antiviruses exists outside of the US and most people I know that freelance are the cream of the crop rather than your average guy.

I am not saying that hiring a person outside the US doesn't have downsides, but it is a bit like with creating a predictive model. Sometimes you could invest lot of resources to get just a small increase in accuracy (or any other relevant metric), but it is just not worth it. Same might go for this case - yes, you get advantages by hiring a US data scientist, but in some cases these advantages might not be important for you or might not be worth the significant increase in costs.

Best,

Roman

Comment by Lucky Balaraman on April 29, 2014 at 10:24pm: Dr Granville,

Thanks for laying bare your conclusions apropos the business end of data science.

My question is, assuming that you outsource everything up to and including EDA, what is the specific nature of the value you add?

Best,

Lucky

///////

Comment by Meena Mani on March 10, 2014 at 8:33am: To Mark, suganya and others,

I am new to this board but have also "held my peace" for a long time in regards to the many many irregularities that take place in academia. Statistics consultancy is widespread and commonplace. If you are in a statistics department, you will get unsolicited requests to help out with homework, projects and theses. What is less common (and frankly quite puzzling) is when professors write the thesis for a student. I only know of one person who does this. Said professor is exceptionally good, world class in fact. But he is in a third-rate university and possibly because of student quality writes all the papers and even theses of some of his students. One of his students (after winning best student paper awards and such in top conferences--the papers written/all work done by the professor) is now a professor at a mid-level university in the mid-west. I happen to have the draft of this thesis (with the clear distinct stamp of the professor --including the English mistakes he usually makes). Heh heh, lots of things happen in academia (and there is very little oversight). If you are interested in my unbelievable stories, you can write to me.

Comment by Vincent Granville on March 6, 2014 at 4:56pm: Hi William,

Your comment is very interesting and resonate with me. I am technically a PhD statistician - though if you look at the official diploma, it says PhD in mathematics - and I call myself Data Scientist anyway. Many people know me as founder, entrepreneur, CEO, CFO, or owner, rather than Data Scientist.

I also published in IEEE Transactions on Pattern Analysis and Machine Intelligence, Journal of Number Theory, Journal of Royal Statistical Society series B, and a bunch of computational statistics journals - I have a data science Wiley book to hit the market by March 31. Still trying to figure out how to change statistics curricula to make them more useful, maybe it is a lost cause. Anyway, I created my own data science program.

Vincent

Comment by william e winkler on March 6, 2014 at 2:24pm: I can give a context on Ph.D. work.

15 years ago I was in a position to hire several new Ph.D.s for positions working with me. I interviewed ten Statistics Ph.D.s (or soon to graduate ones). From each one, I obtained their Ph.D. dissertations and, if available, one or more of their papers. During my interview I would first question them about very elementary statistical ideas that related to their dissertation and then attempt to get them to answer a few questions about the more advanced ideas in their dissertation. Not a single individual was able to answer the questions in what I felt was a minimally satisfactory manner. With the subset that had done what appeared to be nontrivial programming for their dissertation, I proceeded to ask about how they went about their data analysis and programming. Again, none could answer my questions to my satisfaction.

I ended up hiring two applied math Ph.D.s who had been professors at top 200 schools who really impressed me with how they thought about problems and who had done exceptionally difficult programming. Additionally, I was able to hand these two non-trivial papers from journals such as the IEEE Transactions on Pattern Analysis and Machine Intelligence and similar journals that I could have never given the Ph.D. statisticians. The two were both able to learn and do highly nontrivial programming in operations research and in machine learning.

The takeaway was that the Ph.D. statisticians were quite likely second-tier students at the (good) universities whose dissertation advisors were able to hand-feed them a problem.

Comment by Daniel J. Kocis Jr., Ph.D. on March 6, 2014 at 10:15am: I have told many a client that DISRUPTION is GOOD only to have them run back to the status quo.

I think the reason for this is they have budgets and plans against which they are judged and any deviations (plus or minus) creates issues which need to explained and that puts them in the spot light (something cube-farm inhabitants hate). Group membership requires diffuse responsibility of action ( the royal WE at so and so company).

I enjoyed reading about your billing rates and cleaver ways to augment your net income, however I agree with Mark Stones issues on writing a thesis for others. This just creates a new Ph.D. without an original thought.

Comment by Michael Malak on March 6, 2014 at 6:11am: Vincent: you inspired my blog post today: The End of Data Science As We Know It

Comment by Vincent Granville on March 5, 2014 at 4:00pm: I posted the following comment on a LinkedIn group. It might shed some light as to why I advocate becoming a hacker, if you are a very talented and creative person:

I think creative talent is under-valued in US and elsewhere, indeed it is feared. From kindergarden to university to the corporate cubicle, everyone wants to transform you into a good, obeying soldier, have you comply with pre-established social and legal rules, and not bring disruption or question anything. If you manage to resist and fight back, you are probably a leader, and you can succeed in various ways, but consulting or working as an employee is not the ideal that comes to my mind for these rare individuals.