What is ETL?

Put simply, an ETL pipeline is a tool for getting data from one place to another, usually from a data source to a warehouse. A data source can be anything from a directory on your computer to a webpage that hosts files. The process is typically done in three stages: Extract, Transform, and Load. The first stage, extract, retrieves the raw data from the source. The raw data is then transformed to match a predefined format. Finally, the load stage…

A new 2016 survey entitled "Big Data Executive Survey 2016" concludes that data variety is the top data priority for most firms. Seasoned data science practitioners have long known that …

Variety, Velocity, Volume and Veracity are the four Vs for Big Data. Most of the technologies available have shown how to treat the Volume. However, due to the increasing number of streaming data sources, the Velocity problem is as relevant as never before. Moreover, Veracity and especially Variety problems have increased the difficulty of the challenge.…

While large data sets may provide significant value in certain cases, data diversity and integrating smart data points will provide more consistent actionable insights and high value intelligence leading to better decision-making.



For example, consider NFL football data. Focusing on large football game data sets is usually not helpful and often misleading creating…

Data Veracity, uncertain or imprecise data, is often overlooked yet may be as important as the 3 V's of Big Data: Volume, Velocity and…