There has been a growing trend among software engineers to write about their experiences building data infrastructures. Software engineers at Asana wrote about their experience last November; Spotify published a blog post on the same subject in May; Zulily shared their experience earlier this summer; and two days ago IFTTT chimed in with their experience as well.
When reading these posts, you get an incredible look into the thought process and various considerations at these top organizations. RJMetrics did a roundup of these posts to provide a high-level overview of the data infrastructure trends. You can read the full analysis on the RJMetrics blog, but here are the highlights.
Primary data sources being consolidated
Data tech being used
Visualization tools being used