Subscribe to Dr. Granville's Weekly Digest

How to compete against data scientists charging $30/hour

While companies complain about lack of analytic talent, professionals complain about lack of jobs. Everyone wants to work for Facebook, LinkedIn, Google, Intel, Apple, Twitter or some hot start-up. It creates fierce competition getting a job interview, let alone a job. But companies that do not belong to this circle see very few candidates applying for their data scientist open positions; in addition, they are only hiring what I call technical developers (defined by a narrow set of technical skills, usually R, Python, NoSQL, Hadoop, Map-Reduce, software engineering). They are not interested in real data scientists, so many data scientists that would apply would (erroneously) not be perceived as bringing value, and not interviewed.

The problem with consulting is of a different nature. Companies are looking for the cheapest consultant having the minimum set of qualifications to perform the task (the candidate will be asked to provide details about previous projects). Because the work is performed from home, consultants compete with people all over the world to land a gig. Analytic professionals in India, found on websites such as Elance, charge $30/hour. On Statistics.com, you can hire consultants in India for $59/hour.

When I wrote my article about my salary history, a few people mentioned that my consulting rates (from $45 to $100/hour) were absurdly low given my expertize. But compared with rates in India or Romania, it is actually not low. Those charging $150 to $250 per hour are having a difficult time finding new clients. And if all your great skills and expertise are not considered useful to a client, he won't pay for it, especially if this expertise is not used to generate greater revenue. Indeed, many PhD statisticians work as part-time adjunct professors with salaries even far lower, or write other PhD students theses for a fee - typically for $5,000 - as these are the only clients that they can get. So, in some sense, there is more talent than the job market can absorb, especially for PhD's.

So what are the solutions, for a consultant?

Here's a list of ideas:

  1. Work on big data projects, a lot of automation is possible, and big data is cheaper to process. You need the correct expertize though, but many companies lack talent to properly identify the right data and extract value. I would charge by the project rather than by the hour, for such projects.
  2. Work on stuff no one else know. Avoid stuff everyone know such as logistic regression, decision trees, neural networks, as you will be competiting against dozens of professionals who can do that too - some in India or Romania.
  3. Automate your analysis and EDA (exploratory data analysis) as much as you can
  4. Outsource mundane tasks (reporting, data gathering and cleaning, EDA) so you can lower your bill: use an intern or someone in India paid $20/hour to do 80% of the analysis, pay yourself $200/hour for the 20% high level stuff; the average turns out to be $56/hour for the client - a rate lower than what haircut professionals charge.
  5. Make sure you can measure value added and convince clients about the amount of (quantifiable) added value that you provided. Many clients (you need to find them - I am one of them) would not mind paying you $100,000 if you helped them generate another $500,000 in revenue, even if it took you only 50 hours of work.
  6. Reduce your costs: I work entirely from home (I don't need a car), I have no health insurance (by choice, not a good idea for many people), I bought a house well below what I could afford (and same with car), I use open source tools, and my education is mostly free (no expensive training, I read stuff online, buy books). I'm lucky to have no marketing costs; if you can (you need to build a strong social presence), your marketing costs - which typically eat 50% of your revenue and include attending conferences - could drop to zero. I can charge low rate because I have no expenses.

Some arguments to convince a client to work with a more expensive, US-based consultant

  • Your data is confidential: it is protected in my hands. Data processed outside of the country could destroy your IP, patents, can be re-sold, can cause privacy violations etc.
  • Easier to sue me or collect money in case of problems
  • Easier for you to deduct these consulting fees (for the IRS)
  • We are in the same time zone, avoiding unecessary delays every day
  • We can meet in person rather than via video-conferencing
  • We can work on sensitive data (security clearance)
  • We are an established business, not a one-man operation
  • Our English is easy to understand
  • We use professional, enterprise tools; we have a SAS license (we don't use pirated software or virus-laden computers)

Finally, if you really have great expertize spanning across multiple domains, the easiest solution might be to just stop consulting and make a living as a business or growth hacker:

Instead of helping businesses protect themselves against hackers, you become a hacker, knowing that you can outsmart all the consultants and experts working for these companies. I'm talking about legal business and growth hacking. You can create your company, for instance a website selling books listed on Amazon; you don't actually sell the books, you get a commission each time a website visitor goes to Amazon to purchase a book listed on your website. Traffic hacking (one of the hacking systems among many others used to optimize your business) could consist in generating a huge volume of high quality web traffic, through creation of hundreds of (fake) interesting profiles that automatically post interesting stuff via a well diversified set of mailing lists and social networks, without being detected (each profile posting no more than 3 links or pieces of content per day; a different IP address is used for each profile). Your business acumen, network security, traffic scoring, and fraud detection expertize allow you to defeat the algorithms designed to block you. Instead, these algorithms generate false positives because they rely heavily on spam reported by users; you can take advantage of this to get your competitors blocked. Note that this business model does not require any sales or talking to people and is typically run from home. Other advantages include higher revenue, no meetings, no boss, and better job security. If you are good at financial engineering to reduce taxes and other money issues (I call it financial hacking), you will even make more money.

Other resources

Views: 8793

Comment

You need to be a member of Data Science Central to add comments!

Join Data Science Central

Comment by Lucky Balaraman on April 29, 2014 at 10:24pm

Dr Granville,

Thanks for laying bare your conclusions apropos the business end of data science.

My question is, assuming that you outsource everything up to and including EDA, what is the specific nature of the value you add?

Best,

Lucky

///////

Comment by Meena Mani on March 10, 2014 at 8:33am

To Mark, suganya and others,

I am new to this board but have also "held my peace" for a long time in regards to the many many irregularities that take place in academia.  Statistics consultancy is widespread and commonplace. If you are in a statistics department, you will get unsolicited requests to help out with homework, projects and theses. What is less common (and frankly quite puzzling) is when professors write the thesis for a student. I only know of one person who does this. Said professor is exceptionally good, world class in fact. But he is in a third-rate university and possibly because of student quality writes all the papers and even theses of some of his students. One of his students (after winning best student paper awards and such in top conferences--the papers written/all work done by the professor) is now a professor at a mid-level university in the mid-west. I happen to have the draft of this thesis (with the clear distinct stamp of the professor --including the English mistakes he usually makes). Heh heh, lots of things happen in academia (and there is very little oversight). If you are interested in my unbelievable stories, you can write to me.

Comment by Vincent Granville on March 6, 2014 at 4:56pm

Hi William,

Your comment is very interesting and resonate with me. I am technically a PhD statistician - though if you look at the official diploma, it says PhD in mathematics - and I call myself Data Scientist anyway. Many people know me as founder, entrepreneur, CEO, CFO, or owner, rather than Data Scientist.

I also published in IEEE Transactions on Pattern Analysis and Machine Intelligence, Journal of Number Theory, Journal of Royal Statistical Society series B, and a bunch of computational statistics journals - I have a data science Wiley book to hit the market by March 31. Still trying to figure out how to change statistics curricula to make them more useful, maybe it is a lost cause. Anyway, I created my own data science program. 

Vincent

Comment by william e winkler on March 6, 2014 at 2:24pm

I can give a context on Ph.D. work.

15 years ago I was in a position to hire several new Ph.D.s for positions working with me.  I interviewed ten Statistics Ph.D.s (or soon to graduate ones).  From each one, I obtained their Ph.D. dissertations and, if available, one or more of their papers.  During my interview I would first question them about very elementary statistical ideas that related to their dissertation and then attempt to get them to answer a few questions about the more advanced ideas in their dissertation.  Not a single individual was able to answer the questions in what I felt was a minimally satisfactory manner.  With the subset that had done what appeared to be nontrivial programming for their dissertation, I proceeded to ask about how they went about their data analysis and programming.  Again, none could answer my questions to my satisfaction.

I ended up hiring two applied math Ph.D.s who had been professors at top 200 schools who really impressed me with how they thought about problems and who had done exceptionally difficult programming.  Additionally, I was able to hand these two non-trivial papers from journals such as the IEEE Transactions on Pattern Analysis and Machine Intelligence and similar journals that I could have never given the Ph.D. statisticians.  The two were both able to learn and do highly nontrivial programming in operations research and in machine learning.

The takeaway was that the Ph.D. statisticians were quite likely second-tier students at the (good) universities whose dissertation advisors were able to hand-feed them a problem.

Comment by Daniel J. Kocis Jr., Ph.D. on March 6, 2014 at 10:15am

I have told many a client that DISRUPTION is GOOD only to have them run back to the status quo.

I think the reason for this is they have budgets and plans against which they are judged and any deviations (plus or minus) creates issues which need to explained and that puts them in the spot light (something cube-farm inhabitants hate). Group membership requires diffuse responsibility of action ( the royal WE at so and so company). 

I enjoyed reading about your billing rates and cleaver ways to augment your net income, however I agree with Mark Stones issues on writing a thesis for others. This just creates a new Ph.D. without an original thought.

Comment by Michael Malak on March 6, 2014 at 6:11am

Vincent: you inspired my blog post today: The End of Data Science As We Know It

Comment by Vincent Granville on March 5, 2014 at 4:00pm

I posted the following comment on a LinkedIn group. It might shed some light as to why I advocate becoming a hacker, if you are a very talented and creative person:

I think creative talent is under-valued in US and elsewhere, indeed it is feared. From kindergarden to university to the corporate cubicle, everyone wants to transform you into a good, obeying soldier, have you comply with pre-established social and legal rules, and not bring disruption or question anything. If you manage to resist and fight back, you are probably a leader, and you can succeed in various ways, but consulting or working as an employee is not the ideal that comes to my mind for these rare individuals.

Comment by Vincent Granville on March 4, 2014 at 5:53pm

Tony: Yes, a rate that is too low does not inspire trust. There's an optimal pricing for each country/consultant. I once offered to do consulting for free (just as a test), nobody was interested despite heavily promoting my offer. I guess potential clients thought "too good to be true", despite the fact that it was real. Also some PhD's are not good at selling themselves.

Demanding higher wages might not be the best strategy. Moving in a different direction is an interesting solution, possibly working as an employee or as an entrepreneur, I know bloggers (true experts in their field) paid $3,000/month to post 2 blogs a week and do community monitoring; it takes them 10 hours of work a week, so their actual salary (if they combine 4 jobs like that) would be $144,000 per year (with no benefits, but plenty of deductions). Not bad. Or you can sell data, like trading signals based on predictive modeling, accessible via an API. I did it for a while, and I will probably do it again (though it will be a different type of data, more similar to research data). You can write great papers (in PDF format) and sell them on Amazon/Kindle -  no need for middlemen - you are the publisher, and buyers download your PDF (it's paperless). It worked better than I thought to generate good pocket money in my case, and I'll do it again on a bigger scale. All these alternatives involve working exclusively from home, which is a plus. 

Comment by Tony DiLoreto on March 4, 2014 at 4:22pm

Dr Granville,

I can totally appreciate the competition argument, especially since clients sometimes (for better or worse) see consultants as interchangeable, regardless of where they are. However, there is a reason that Giorgio Armani charges thousands of dollars for his suits, even though there are people in China, India making suits for much less. There is an appearance to uphold; if Armani was to lower his prices down to market levels for suits (not Armani-quality suits), people would not value his products as much. Sometimes price is indicative of quality and experience, both which you have an embarrassment of riches. If I was to hire a data scientist, and I saw you and a few others for $100/hr, I would easily choose you. 

Don't you feel the same applies here, where those with experience and talent should demand those wages? Perhaps if PhDs and those with great experience are having trouble finding work, it is a function of their selling points and not their price?

Comment by suganya on March 4, 2014 at 1:56pm

This information is a real eye-opener and so shocking to hear . you have an awesome experience and expertise .

Follow Us

Videos

  • Add Videos
  • View All

© 2014   Data Science Central

Badges  |  Report an Issue  |  Terms of Service