Comment

Comment by Christopher Majka on January 18, 2017 at 8:56am

It makes sense.  We are still heavily involved in the ETL process, hence the use of SQL and other querying tools, such as SAS.

Comment by Javier Alonso on January 18, 2017 at 7:16am

And I agree with you in disagreeing with me. Every tool has an use, and SQL complies perfectly with DB management, data extraction, table joining, etc. mainly when in the ETL process.

Case is, my students (managers rather than data scientists) focus in the use of graphical tools like KNIME or Orange to get fast (simple, uncomplicated) insight to their data. They are more or less past the ETL phase of analysis.

Paraphrasing the SciFi writer Robert Heinlein: "Data scientist are like butterflies... Endless variety.".

Yours,   F.J.Alonso

Comment by Christopher Majka on January 18, 2017 at 6:51am

Javier - I respectfully disagree.  We use SQL daily in our analysis.  I suppose it depends on the data being analyzed and the industry you are in.  Regards

Comment by Javier Alonso on January 17, 2017 at 3:59pm

Second afterthought:

"There are ~20 people who did not score at all. They either faced some technical problem or did not like the test or did not know SQL."

Those we call 'outliers'. You should dispose of them in some way or another before facing analysis (ETL cycle).

"There is another population which looks to have normal distribution between scores 1 to 10. These people either started the competition late and hence could not get enough time or they are just beginners in SQL."

You can't say a part of a distribution is 'normal'. Or the whole distribution is normal or it isn't. this one seems skewed, I'd say.

Third population looks to have distribution between scores 10 and 41 and looks like a representative of people in industry. For this group, mean is 25.8 and standard deviation ~ 6.5 . So any one with a score of more than 32 is in top 16% of the population.

Why do you think it is representative? Because they are the most? Are your numbers (mean, sd.) referring this segment or the whole population?

Descriptive stats are Ok. for a first glance at data, but do not lead to conclusions...

Yours,   F.J.Alonso

Comment by Javier Alonso on January 17, 2017 at 3:23pm

Afterthought:

"If there is one language, every data science professional should know – it is SQL"

Why? I learned SQL back in 1992 as a DBAdmin and user. It's been quite some time I don't use SQL anymore in data analysis. Of course, it's still a good asset in your resumé (Curriculum Vitae) but I think it's much more interesting to have a sound knowledge of R, maybe Python, and tools like Tableau, Knime and the like. Not to speak about Machine Learning, Neural nets...

I've taught many courses in enterprise data analysis and never said a word about SQL. No one complained.

Yours,   F.J.Alonso

Comment by Javier Alonso on January 17, 2017 at 2:56pm

Sorry, SQL is not a programming language. It's a query language. It is not a programming language because it doesn't comply with what we call 'Turing completeness' (it's not 'Turing complete') so it's structure does not have a semantic clausure in its context.

Besides that, "practical aspects and challenges people encounter while using Excel" has nothing to do with SQL.

Yours.  F.J.Alonso