All Videos Tagged Xiangrui (Data Science Central) - Data Science Central 2020-07-05T04:12:47Z https://www.datasciencecentral.com/video/video/listTagged?tag=Xiangrui&rss=yes&xn_auth=no DSC Webinar Series: State of the Art Deep Learning on Apache Spark™ tag:www.datasciencecentral.com,2018-10-31:6448529:Video:773134 2018-10-31T19:21:53.776Z Tim Matteson https://www.datasciencecentral.com/profile/2edcolrgc4o4b <a href="https://www.datasciencecentral.com/video/dsc-webinar-series-state-of-the-art-deep-learning-on-apache-spark"><br /> <img alt="Thumbnail" height="135" src="https://storage.ning.com/topology/rest/1.0/file/get/2781532536?profile=original&amp;width=240&amp;height=135" width="240"></img><br /> </a> <br></br>Big data and AI are joined at the hip: the best AI applications require massive amounts of constantly updated training data to build state-of-the-art models. Increasingly more Spark users want to integrate Spark with distributed machine learning frameworks built for state-of-the-art training.<br></br> <br></br> Here's the problem: big data… <a href="https://www.datasciencecentral.com/video/dsc-webinar-series-state-of-the-art-deep-learning-on-apache-spark"><br /> <img src="https://storage.ning.com/topology/rest/1.0/file/get/2781532536?profile=original&amp;width=240&amp;height=135" width="240" height="135" alt="Thumbnail" /><br /> </a><br />Big data and AI are joined at the hip: the best AI applications require massive amounts of constantly updated training data to build state-of-the-art models. Increasingly more Spark users want to integrate Spark with distributed machine learning frameworks built for state-of-the-art training.<br /> <br /> Here's the problem: big data frameworks like Spark and distributed deep learning frameworks don’t play well together due to the disparity between how big data jobs are executed and how deep learning jobs are executed.<br /> <br /> In this latest Data Science Central webinar, we'll share how Project Hydrogen, a Spark Project Improvement Proposal led by Databricks, is positioned as a potential solution to this dilemma.<br /> <br /> We will cover:<br /> <br /> Barrier execution mode for distributed DL training<br /> Fast data exchange between Spark and DL frameworks, and<br /> Accelerator-awareness scheduling<br /> Speaker:<br /> Xiangrui Meng, Software Engineer - Databricks<br /> <br /> Hosted by:<br /> Bill Vorhies, Editorial Director - Data Science Central