R-squared measures how well your data fits a regression line. More specifically, it's how much variation in the response variable your linear…

Introduction

In a world where we generate data at an extremely fast rate, the correct analysis of the data and providing useful and meaningful results at the right time can provide helpful solutions for many domains dealing with data products. We can apply this in Health Care and Finance to Media, Retail, Travel Services and etc. some solid examples include Netflix providing personalized recommendations at real-time, Amazon tracking your interaction with different products on its…

I pulled out a dusty copy of Thinking Stats by Allen Downey the other day. I highly recommend this terrific little read that teaches statistics with easily understood examples using Python. When I purchased the book eight years ago, the Python code proved invaluable as…

The energy industry is undergoing a rapid transformation in recent past owing to the enhanced role of renewables and enhanced data-driven models making the value chain smarter. In the context of the primary constituents of this sector comprising of coal, power, renewables, solar energy, oil, and gas, there is a huge role AI can play.

We illustrate some key use cases below:

1. Smart Grid

The biggest disruption in power in recent times is in the smart grid…

Introduction

Data science is a multidisciplinary blend of data inference, algorithm development, and technology in order to solve analytically complex problems. At the core is data. Troves of raw information, streaming in and stored in enterprise data warehouses. Much to learn by mining it. Advanced capabilities we can build with it. Data science is ultimately about using this data in creative ways to generate business value

The broader fields of understanding what data…

The future offers interesting and exciting times ahead for most businesses. With data being a big influencer in the enterprise of the future, it is a matter of time before we jump into the era of intelligent enterprises.

Intelligent enterprises are going to be…

What is the factory of the future? Is it a synonym to Industry 4.0, or is it a different concept in its own right? Industry 4.0 and the factory of the future might sound similar, but they are different in some ways. To begin with, the factory of the future is an elusive concept that isn’t as common as Industry 4.0.

The factory of the future is…

After testing this idea for the last few months, we have formally launched this concept

The idea of ‘Data Science Coding in a weekend’ originated from meetups we conducted in London

The idea is simple but effective

We choose a complex section of code and try to learn it in detail over…

The analytics market is booming, and so is the use of the keyword – Data Science. Professionals from different disciplines are using data in their day to day activities, and feel the need to master the start-of-the-art technology in order to get maximum insights from the data, and subsequently help the business to grow.

Moreover, there are professionals who want to keep them updated with this latest skills such as Machine Learning, Deep Learning, Data Science, and so either to elevate…

Here is a simple trick that can solve a lot of problems.

You can not trust a linear or logistic regression performed on data if the error term (residuals) are auto-correlated. There are different approaches to de-correlate the observations, but they usually involve introducing a new matrix to take care of the resulting bias. See for instance here.  …

This simple introduction to matrix theory offers a refreshing perspective on the subject. Using a basic concept that leads to a simple formula for the power of a matrix, we see how it can solve time series, Markov chains, linear regression, data reduction, principal components analysis (PCA) and other machine learning problems. These problems are usually solved with more advanced matrix calculus, including eigenvalues, diagonalization, generalized inverse matrices, and other types of matrix…

Summary:  Real Time Computer Vision (RTCV) that requires processing video DNNs at the edge is likely to be the next killer app that powers a renewed love affair with our mobile devices.  The problem is that current GPUs won’t cut it and we have to wait once again for the hardware to catch up.

 The entire…

A.I. based automated Anomaly detection system is gaining popularity nowadays due to the increase in data generated from various devices and the increase in ever evolving sophisticated threats from hackers etc. Anomaly detection systems can be applied across various business scenarios like monitoring financial transactions of a fintech company, highlighting fraudulent activities in a network, e-commerce price glitches among millions of products, and so on. Anomaly detection system can work…

All the data we need today is already available on the internet, which is great news for data scientists. The only barrier to using this data is the ability to access it. There are some platforms that even include APIs (such as Twitter) that support data collection from web pages, but it is not possible to crawl most web pages using this advantage..

This…

Azure Machine Learning concepts - an Introduction

Introduction

Last week, we launched a free book called Classification and Regression in a weekend. The idea of the ‘in a weekend’ series of books is to study one complex section of code in a weekend to master the concept. This week. we plan to launch a book called “An…

Monday newsletter published by Data Science Central. Previous editions can be found here. The contribution flagged with a + is our selection for the picture of the week. To subscribe, follow this…

The covariance matrix has many interesting properties, and it can be found in mixture models, component analysis, Kalman filters, and more. Developing an intuition for how the covariance matrix operates is useful in understanding its practical implications. This article will focus on a few important properties, associated proofs, and then some interesting practical applications, i.e., extracting transformed polygons from a Gaussian mixture's covariance matrix.

I have often found that…

As a senior datascience professional and analytics manager, I get countless requests for job search advice, resume feedback and heart-breaking stories from brilliant students who are unable to snag a job in this exciting field. There are tons of books on how to learn the skills to become a data scientist/ data analyst, but none to prepare folks for the frustrating job search.

I've repeated this advice to dozens of people, most of whom found their dream datascience job with…

Cross Validation explained in one simple picture. The method shown here is k-fold cross validation, where data is split into k folds (in this example, 5 folds). Blue balls represent training data; 1/k (i.e. 1/5) balls are held back for model testing.

Monte Carlo cross validation works the same way, except that the balls would be chosen with replacement. In other words, it would be possible for a ball to appear in more than one sample.…