Subscribe to DSC Newsletter

Machine Learning - Anomaly Detection: "Finding a Needle in a Haystack"

After exploring formulationclassificationbenchmarking, we explore another facet of Machine Learning: anomaly detection. This part is key in the IoT transformation, as it enables internet-connected AI devices to alert, adapt and respond accordingly. Once properly trained, an IoT could not only warn and prevent imminent failure, but also execute a response, adaptive to the anomaly detected.

In this process, we'll explore intrinsic hurdles that makes the anomaly detection process a non-trivial task of "finding a needle in haystack".

Opportunities abound to explore, and any univariate sequential dataset (time series) can be used to demonstrate applicability. For this illustrative purpose, a common public source is certainly weather data.

Medias have been covering extensively 'global warming' and climate changes at the planetary level. Recently, the COP21 saw many warming trends exposed and targets projected. In view of current observations in the northern hemisphere, we choose to analyze specific data trends to see if anomalies could be detected, and understand the complexity of this process. We also explore FFT filtering that can be applied to periodic signals, and compare original and filtered signal data, to demonstrate applicability regardless of the signal characteristics. We also check historical data to compare findings.

Click here to see full anomaly detection analysis (long article.)

Views: 4767

Tags: Analytics, Anomaly, Data, Detection, Learning, Machine, Python, science

Comment

You need to be a member of Data Science Central to add comments!

Join Data Science Central

Comment by Bonson Sebastian Mampilli on January 6, 2018 at 8:34pm

Hello,

The link to the article is not correct. Would like to read the article on anomaly detection.

Kindly provide the same.

Regards

Bonson

© 2019   Data Science Central ®   Powered by

Badges  |  Report an Issue  |  Privacy Policy  |  Terms of Service