How to compare and rank data science programs?

I started an attempt at program comparison in my article Zipfian Academy versus Data Science Apprenticeship, comparing two apples, rather than comparing apples (stuff like our program, known as DSA) with oranges (Coursera) and lemons (university data science curricula).

But how would you create a universal ranking or score, that would simply put everything (apples, lemons, oranges...) in one bucket and rank them according to satisfaction and results. Results could be ROI. But in the case of our DSA, since cost = 0, ROI is infinite. So a better metric is needed to measure lift or yield or return.

I propose using the following attributes, and have readers decide themselves about the weights to attach to each attribute. In short, I propose to develop an app to rank data science programs (including certifications), that provides customized rankings to each user. Maybe some JavaScript code would allow users to enter the weights for each attribute, to compute the final score.

The attributes I have in mind are:

user satisfaction
time to complete
cost
exposure to real world big data and tools
too many remnants from old statistics courses
online versus face-to-face training
On demand versus pre-scheduled
prerequisites (low or high)
help with job placement
lift (salary or satisfaction level, before and after completion)
delivers a recognized diploma

Do you have other attributes in mind? Or a better solution to create these rankings?

9 members like this

Comment

You need to be a member of Data Science Central to add comments!

Join Data Science Central

Comment by Vincent Granville on November 19, 2013 at 9:29am

At the request of many prospective participants, here's an update about our DSA (Data Science Apprenticeship):

Stage 1 (Available now): DIY (do-it-yourself) for self-learners: material is available for free thoughout DSC, including data sets and projects to work on.

Stage 2 (April 2014): Participants will purchase my Wiley book (DSC members get a discount) as well as our data science cheat sheet to get jump started.

Stage 3: Projects will be evaluated for a fee, and a certification delivered.

Comment by Vincent Granville on November 19, 2013 at 9:22am: This is my reply to a comment posted on LinkedIn:

Cheap does not mean easy, and easy does not mean fewer difficult topics included in the program, it could mean that they are presented in such a way that they are easy to understand. Regarding the cost, I disagree: unless you are already wealthy, cost is critical, especially since jobs are hard to find, and high quality low-cost programs are available. In some cases, cost is high due to terrible financial management and money not spent on training but on sports, buildings, administration, compliance and funding other students. You don't want to pay tons of money for such programs, when only a fraction is used for training, and training is provided (despite the cost) by cheaply paid adjuncts far less qualified than the well paid experts running and teaching in the cheap programs.

Method of delivery is important to executives who do not have time to spend in commuting or flying to attend classes, with schedules conflicting with their own business schedules. These people are more likely to be interested in online, on-demand delivery, which is also the least expensive delivery mechanism.

Comment by Rafael Alarcon Borghi on November 18, 2013 at 11:23am: Very good and straight to the point

Comment by Dr. Z on November 18, 2013 at 2:07am: Very insightful. Thank you for sharing.

Comment by diana kornbrot on November 18, 2013 at 12:28am: well the attributes certainly are a fruit salad presumably as intended.

As a psychologist, suggest approaching from intending students' perspective and dividing attributes according to A user needs and B quality

A includes content (pre-requisites, syllabus, data samples as exercise [real or not, size]), timing, software, cost, diploma etc. which determine whether students are to sign up and benefit their careers

B includes course satisfaction, drop put rate ad fn time, quality of in course feedback and assessment, salary increase (pre, immediate, + 6 months, +1 year?), job satisfaction (pre, immediate, + 6 months, +1 year?).

This enables potential students to compare quality of courses on their short list according to quality. For example, more advanced courses may get systematically higher or lower satisfaction than beginner courses, but any given student will have chosen their level and want to compare like with like. I have massive data that content has a major influence on satisfaction on UK higher ed courses

Any single index for B may be misleading and all need access to ratings on all attributes.

I wouldn't bother with ROI, as students can view A and B and make their own personal ROI estimate based on cost, & as others have mentioned, their time.

Finally, monitor the outcome. Whihc attributes predict overall satisfaction for each kind of course? The UK data also shows that this depends on content too

best

Diana

Comment by Vincent Granville on November 17, 2013 at 5:28pm: @Sudhir: Yes, programs from SAS, EMC etc. must be taken into account.

Comment by Sudhir Kale on November 17, 2013 at 2:31pm: Good attributes, Vincent, how about the program associated with companies such as SAS or emc?

Comment by L George, Ph.D. on November 17, 2013 at 1:34pm: Great inquiry!

User satisfaction sounds like an overall attribute or rating. Its components - and, as you mentioned, weight - would differ from person to person.

I would add:

- Quality of instruction/instructors

- A well-thought-out curriculum. (After completing a few great MOOCs it's unclear what to take next and how to avoid a lot of overlap - though hearing the same thing from different perspectives can be helpful.)

- Level of job placement support offered (interview prep, partnerships with top companies, etc.)

A couple general thoughts not linked to specific attributes:

I see subgroups developing within the field, though I don't think they're clearly differentiated yet. For example, I view 'data science' as working with the data to solve problems/answer questions, and 'data engineering' as developing platforms - storage, access, parallelism, extensions, etc. It's not that simple and there's a need for each group to understand other area(s), but focus and skillsets would differ. Training would optimally be tailored for different paths.

Due to the nature of the field, and with the availability of MOOCs and other viable free or inexpensive learning opportunities, students have a wide range of backgrounds. A clear understanding of student characteristics and prerequisites would be especially important for data science programs. Starting each program at basics - what is big data, a review of statistics, etc. - would be inefficient for many potential students.

So interesting to see the field develop!

Comment by Casson Stallings on November 16, 2013 at 2:30pm: Regarding ROI, investment of time needs to be counted. I would suggest some changes. Add 'delivers a recognized certificate'; some judgement is still necessary as to how respected they are. On demand and online seem like attributes that are not necessarily better or worse, but a matter of preference. I think it is important to include measures of individual attention available or for MOOCs the level of group support available. And 'too many remnants...' seems to have a built in judgement, granted a judgement you've made clear you have. Instead let people rank the breadth of techniques covered somehow. So I suppose we'll have to wait six months or so to learn what the lift is from the apprenticeship program. I'm looking forward to it.

Comment by Steven Miller on November 16, 2013 at 2:08pm: How to rank what's not been defined? First we need some consensus on what data science is, what a data scientist is, perhaps what sub-specialties exist, and so on.

Based on the conversations I've participated in of late we are far from achieving consensus.