The Challenge

The main challenge solved with this Data App is the construction of a benchmark of anomaly detection algorithms, implemented in different python libraries, in order to find a powerful anomaly detection solution for one of the annotated time series in the Numenta Anomaly Benchmark (NAB).  

Methodology

We cover the following steps to build the solution data app. 

Step 1: An exploratory data analysis (EDA)  

Step 2: Comparing seven anomaly detection algorithms.

Step 3: Modelapplication, visualization and comparisonofresults and anomalies. 

Additionally, the case itself comes with some other complex issues, such as anomalies and duplicated data points that do not seem to follow predictable patterns, and evaluate algorithms implemented in different libraries. 

Finally, we need to build visualizations that easily help understand the performance of anomaly detection. 

Metrics

For this project, we chose to use simpler metrics in order to allow for a broader range of algorithms to be effectively compared: missed alarm rate, false alarm rate, recall, and execution time. 

Algorithms 

We have compared seven anomaly detection algorithms in the benchmark:  

Hotelling’s T2  

One-class SV  

iForest algorithm  

Local outlier factor    

Changefinder algorithm  

Matrix profile   

Standard deviation filter  

These algorithms have been compared using the aforementioned metrics. The results obtained using the available data with each of the models provides us with all the informaiton needed to choose the best model.

Synthesized Resolution  

The starting point is a dataset that covers the past three months of temperature readings from an internal component of a large, expensive industrial machine. Our objective is to be able to predict the four abnormal periods indicated in blue below. By identifying these anomalies, we can take appropriate action before any damage or negative impact occurs on the industrial process. 

With a sufficient large number of false positives (an alarm is triggered every time the temperature drops below 65 or above 100 degrees Celsius), three of the four anomalies are preempted by the algorithm, and in only one of them it takes 4% of the duration of an anomaly perior (2 days) to trigger the alarm, indicating that the system could be safely used for predictive maintenance. 

Results

The following key results have been derived from the development of this use case:  

Detection of an error and duplicated data points.  

It takes 1 second to extract all the anomalies from more than 22k data points in the dataset.  

The “Sigma” method provided a missed alarm rate of 39% and a false alarm rate of 6%. 

How does Shapelets help solve this challenge? 

A) Shapelets provides an improved visualization, processing and storing data streamed at near real-time rates from sensors.   

B) An integrated benchmark that feeds from different libraries and frameworks.  

C) Enhanced development and customization within one platform. 

In this use case specifically, Shapelets helps in executing a comparison of anomaly detectionmodels from different libraries or frameworks and producing and integrated benchmark with the purpose of selecting the best solution.  

As a result, this data app allows us to present an accurate predictive maintenance plan to detect anomalies in the sensor data to prevent machinery failures.  

Consequently, this use case serves as a starting point to develop a solution that can be used to detect anomalies in an online setup. It presents a suitable anomaly detection method to obtain reasonable performance, given the little and uncontextualized data available, which are simply temperature values. 

Now it’s your turn to use this study to develop you own solution. What are you waiting for? 

If you have any questions or would like some business guidance, you may contact us here.