Here's one of the main differences between data engineering and data science: ETL (Extract / Load / Transform) is for data engineers, or sometimes data architects or DBA's.

DAD (Discover / Access / Distill) is for data scientists. Sometimes data engineers do DAD, sometimes data scientists do ETL, but it's rather rare, and when they do it, it's purely internal (the data engineer doing a bit of statistical analysis to optimize some database processes, the data scientist doing a bit of database management to manage a small, local, private database of summarized info (not used in production mode usually, though there are exceptions).

What DAD means:

• Discover: Find, identify the sources of good data, and the metrics. Sometimes requires the data to be created (work with data engineers, business analysts).

• Access: Access the data. Sometines via an API, a web crawler, an Internet download, a database access or sometimes in-memory within a database.

• Distill: Extract essence from data, the stuff that leads to decisions, increased ROI and actions (such as determining optimum bid prices in an automated bidding system). Involves exploring the data (creating a data dictionary, exploratory analysis), cleaning (removing impurities), refining (data summarization, sometimes multiple layers of summarization - or hierarchical summarization), and analyzing: statistical analyses (sometimes including stuff like experimental design which can take place even before the Access stage), both automated and manual.

The last step might or might nor require: statistical modeling (many predictors are now model-independent), presenting results to management (less important if the purpose is to design a machine-to-machine communication system, instead a proof-of-concept or prototype might be required first), or integrating results in some automated process. Documenting is always part of all these steps.

DSC Resources

• Career: Training | Books | Cheat Sheet | Apprenticeship | Certification | Salary Surveys | Jobs
• Knowledge: Research | Competitions | Webinars | Our Book | Members Only | Search DSC
• Buzz: Business News | Announcements | Events | RSS Feeds
• Misc: Top Links | Code Snippets | External Resources | Best Blogs | Subscribe | For Bloggers

Additional Reading

• 50 Articles about Hadoop and Related Topics
• 10 Modern Statistical Concepts Discovered by Data Scientists
• Top data science keywords on DSC
• 4 easy steps to becoming a data scientist
• 13 New Trends in Big Data and Data Science
• 22 tips for better data science
• Data Science Compared to 16 Analytic Disciplines
• How to detect spurious correlations, and how to find the real ones
• 17 short tutorials all data scientists should read (and practice)
• 10 types of data scientists
• 66 job interview questions for data scientists
• High versus low-level data science

Follow us on Twitter: @DataScienceCtrl | @AnalyticBridge