Time: January 27, 2016 from 6pm to 8:15pm
Street: 500 Eldorado Boulevard
City/Town: Broomfield, CO
Website or Map: http://www.meetup.com/Data-Sc…
Event Type: free, and, open, to, all
Organized By: Michael Walker
Latest Activity: Dec 18, 2015
Wednesday, January 27, 2016 - 6:00 PM
Oracle - 500 Eldorado Boulevard, Broomfield, CO
6:00pm Pizza and networking
6:40pm Maximum Flow algorithm implementation on Spark GraphX, by Ryan Langewisch
7:50pm Demo of 4-node Raspberry Pi Spark cluster, by Andrew Weekley
Maximum Flow algorithm implementation on Spark GraphX - Abstract
GraphX is an API for graph computation built upon Apache Spark, a fast and generalized engine for large-scale data processing in the cloud. While the popularity of Spark and GraphX is growing, the relatively young technology has yet to explore the breadth of graph problems that exist in the field. In order to examine and gain insights into the capabilities of GraphX, this thesis approaches the framework with the intention of implementing a solution to the Maximum Flow Problem, a complex graph problem without a trivial distributed approach. Specifically, the implementation is to be based on the serial Push-Relabel algorithm. An original MapReduce-based approach to the problem is presented, as well as an implementation of the approach in GraphX. In addition to the implementation, experimentation and deployment to an Amazon EC2 cluster allowed observations on caching and checkpointing intervals to be made.
Ryan Langewisch - Bio
Ryan Langewisch graduated from Colorado School of Mines with his BS in Computer Science in December of 2014, and his MS in Computer Science in December of 2015. For his Master's work, Ryan gained exposure to algorithms in the context of distributed computing, ultimately leading to his thesis, "A Performance Study of an Implementation of the Push-Relabel Maximum Flow Algorithm in Apache Spark's GraphX." In 2016, Ryan will join ReadyTalk in Denver, CO as a software engineer.
Demo of 4-node Raspberry Pi Spark cluster - Abstract
Raspberry Pi computer board are inexpensive, low power, fairly capable devices that can be easily networked. There have been numerous examples of small multi-node systems that have been constructed. I have built a 4-node, 16 core system and installed Spark. This talk will give some of the details and lessons learned from constructing the system and describe a simple use case. The goal of this project was to gain experience building a distributed system and learn more about Spark.
Andrew Weekley - Bio
Andrew Weekley is a Data Analyst at the National Renewable Energy Lab working in the Strategic Energy Analysis Center. He works on solar insolation data sets, primarily on the creation of synthetic data that is used in solar integration studies. He also performs general time-series analysis, and inter-comparison of ground measured, modeled and forecasted solar insolation data sets.