Machine learning applications require powerful and scalable computing systems that can sustain the high computation complexity of these applications. Companies that are working on the domain of machine learning have to allocate a significant amount of their budget for the OpEx of machine learning applications whether this is done on cloud or on-prem.

Typical machine learning application (especially when auto ML is used) can scale to as much as 30 nodes or higher (servers) to provide useful results, increasing significantly the TCO. For example, Auto ML automate many data analysis and modeling processes by trying to find the optimum configuration. In order to achieve this it has to perform several test cases using machine learning applications consuming hume amounts of resources it terms of time and cost.

However, cloud providers (like AWS, Alibaba and Nimbix) have started deploying new platforms that can be used to speedup and reduce the OpEx of these applications. Programmable hardware accelerators, like FPGAs, can provide much higher throughput compared to CPUs and GPUs and at the same time help reduce the OpEx of computational intensive applications.

In this use case we show an example on how a company can save over $200K by utilizing hardware accelerators in the cloud or on-prem.

**Use case: Training on logistic regression on 30 nodes**

One of the most widely used applications in the domain of machine learning is logistic regression. Logistic Regression is used for building predictive models for many complex pattern-matching and classification problems. It is used widely in such diverse areas as bioinformatics, finance and data analytics. In this case, we evaluate the total OpEx for training a large dataset on 30 nodes. We evaluate the performance and the cost on the large MNIST dataset (however any other dataset can be applied).

If a company is using a cluster of 30 nodes for the machine learning training, the total cost on a yearly basis is around $302,220 (assuming a cluster of r5d.4x on aws that costs $1.15/hour and assuming 24/7 operation.)

R5d.4x cost: $1.15*24*365*30 = $302,220

However, if the same company utilize the new f1 instances on AWS, it can speedup significantly the training of the models and at the same time to reduce significantly the OpEX. The accelerated ML suite from InAccel that is tailored made for the f1 instances on aws can achieve up to 15x speedup compared to r5d.4x for the training of logistic regression. That means that the company can either select to run their applications 15x faster using the same number of nodes or it can reduce the number of nodes to only 2 nodes. The figure below shows that total OpEx savings using the hardware accelerators (f1) on aws.

f1.4x cost: $3.3*24*365*2 = $57,816

The cost of the ML accelerators is $1.15/hour. So the total cost of the license of the accelerators would be $20k/year for 2 nodes.

To use the hardware accelerators, in the past you had to change your application in order to make sure that the processor will offload the most computational intensive algorithms in the hardware accelerators. However, using InAccel’s libraries, the user does not have to change a single line of code. ML applications, e.g. based on Apache Spark, can run seamlessly without changing their code at all and just by using InAccel. Even taking into account the license fees for the FPGA resource manager (1k/year/node) the company can save over $200k in total.

Examples on how to test and deploy your machine learning applications using accelerators instantly are available both on AWS and on Nimbix.

The most important advantage however, as that the most valuable resource of the company (the ML engineers and the data engineers) will be able to perform much faster their tasks and get much faster their results which means additional savings in terms of personnel cost.

Views: 563

Tags: accelerators, cloud, fpga, hardware, learning, logistic, machine, regression, speedup

© 2020 Data Science Central ® Powered by

Badges | Report an Issue | Privacy Policy | Terms of Service

**Most Popular Content on DSC**

To not miss this type of content in the future, subscribe to our newsletter.

- Book: Statistics -- New Foundations, Toolbox, and Machine Learning Recipes
- Book: Classification and Regression In a Weekend - With Python
- Book: Applied Stochastic Processes
- Long-range Correlations in Time Series: Modeling, Testing, Case Study
- How to Automatically Determine the Number of Clusters in your Data
- New Machine Learning Cheat Sheet | Old one
- Confidence Intervals Without Pain - With Resampling
- Advanced Machine Learning with Basic Excel
- New Perspectives on Statistical Distributions and Deep Learning
- Fascinating New Results in the Theory of Randomness
- Fast Combinatorial Feature Selection

**Other popular resources**

- Comprehensive Repository of Data Science and ML Resources
- Statistical Concepts Explained in Simple English
- Machine Learning Concepts Explained in One Picture
- 100 Data Science Interview Questions and Answers
- Cheat Sheets | Curated Articles | Search | Jobs | Courses
- Post a Blog | Forum Questions | Books | Salaries | News

**Archives:** 2008-2014 |
2015-2016 |
2017-2019 |
Book 1 |
Book 2 |
More

**Most popular articles**

- Free Book and Resources for DSC Members
- New Perspectives on Statistical Distributions and Deep Learning
- Time series, Growth Modeling and Data Science Wizardy
- Statistical Concepts Explained in Simple English
- Machine Learning Concepts Explained in One Picture
- Comprehensive Repository of Data Science and ML Resources
- Advanced Machine Learning with Basic Excel
- Difference between ML, Data Science, AI, Deep Learning, and Statistics
- Selected Business Analytics, Data Science and ML articles
- How to Automatically Determine the Number of Clusters in your Data
- Fascinating New Results in the Theory of Randomness
- Hire a Data Scientist | Search DSC | Find a Job
- Post a Blog | Forum Questions

## You need to be a member of Data Science Central to add comments!

Join Data Science Central