Home » Uncategorized

Intuitive Machine Learning : Gradient Descent Simplified

This article was written by Roopam Upadhyay. Roopam is a seasoned professional of advanced analytics with more than a decade of experience in statistical modeling, data science, predictive analytics, optimization, & business consulting.


How do machines learn? They learn the same way as humans. Humans learn from experience and so do machines. For machines, experience is in the form of data. Machines use powerful algorithms to make sense of the data. They identify underlining patterns within the data to learn things about the world. Like humans, machines use this learning to make decisions. For example, amazon.com uses machine learning to recommend products to you based on your surfing patterns on their website. Forrester Research estimates sales conversion through this recommendation engine as high as 60%. Companies across the globe are making huge profits through decision making by machine learning algorithms.

Many powerful machine learning algorithms use gradient descent optimization to identify patterns and learn from data. Gradient descent  powers machine learning algorithms such as linear regression, logistic regression, neural networks, and support vector machines. In this article, we will gain an intuitive understanding of gradient descent optimization. For this let’s take a dive towards these concepts with some help from…

– Gravity 

A few years ago my wife told me this incident about her friend who has a young daughter. One day my wife’s friend discovered that her young daughter had dropped and broken a lamp while playing. When she inquired from her daughter about how that happened, the little girl responded : “Grabili did it, mommy, it’s not [entirely] my fault, it’s Grabili”. The girl was referring to gravity and she was right that gravity was an equal accomplice in her mischief. Like the little girl, we all know through the works of Galileo and Newton that gravity is responsible for everything that falls.

Gravity has important lessons that will help us gain an intuitive and simplified understanding of gradient descent.

What you will find in the article: 

– Gravity & Gradient Descent Optimisation
– Gradient Descent – Finding Minimum Value of a Function

Method-1 : Differential Calculus

Method-2: Iterative Calculation

– Linear Regression – Loss Function & Gradient Descent

Method-1 : Solving Linear Regression by OLS

Method-2 : Solving Linear Regression by Gradient Descent

To check out all this information, click here

Top DSC Resources

Follow us on Twitter: @DataScienceCtrl | @AnalyticBridge

Leave a Reply

Your email address will not be published. Required fields are marked *