Run a self-training anomaly detection algorithm against a given dataset. The selected algorithm is Moving Median Decomposition as seen in this whitepaper for Anomaly Detection for Predictive Maintenance. This Library is for identification of items, events or observations which do not conform to an expected pattern or other items in a dataset.
This is an ipm package, which contains one or more reusable assets within the ipm Community. The 'package.json' in this repo is a ipm spec's package.json, here, which is a superset of npm's package.json spec, here.
Ready out-of-the-box
Requires API keys from Email and SMS Alerts via SendGrid and Twilio.
See AnomalyDetectionConstants Library.
AnomalyDetection - Portal for viewing and configuring live datafeeds, rules, and alerts
ExampleDetectAnomaly - Runs anomaly detection against an example dataset, returns an array of Anomalies with metadata.
AnomalyDetection - Core library for detecting anomalies in datasets
AnomalyDetectionConstants - Optional credentials for Alerting (SMS via Twilio, Email via Sendgrid)
jStat - Dependency on jstat-statistics-toolkit
TwilioSMS - Dependency on twilio-sms-library
SendGridEmail - Dependency on(https://github.com/rreinold/clearblade-sendgrid-email-integration)
Anomalies - Tracks all detected anomalies
AnomalyConfiguration - Stores user-defined rules and alerts for Anomaly Detection
- AnomalyDetection()
Detects Anomalies with Moving Median Decomposition See attached whitepaper for Anomaly Detection for Predictive Maintenance https://files.knime.com/sites/default/files/inline-images/Anomaly_Detection_Time_Series_final.pdf
Run a self-training anomaly detection algorithm against a given dataset
- Twilio(user, pass, sourceNumber)
Sends a text message using Twilio's REST API.
- DetectAnomaly() ⇒
AnomalyVisual Detects an anomaly in the last X MQTT Messages Uses 'AnomalyConfiguration' config to check for a key in the JSON of each message. ex. 'sensor_key' will be set to 'temperature', so we know to pull the temperature key/value pair from a mqtt message like this: {"temperature":40,"humidity":31}
- ExampleDetectAnomaly() ⇒
Array.<Anomaly> Example logic for detecting anomalies in a dataset of sensor values
- SaveAnomalyConfiguration()
Ingests updated anomaly configuration, and saves to AnomalyConfiguration Collection
- SendAlert()
This is triggered by an Anomaly being recorded, and inserted into the Anomalies Collection
- Fetches the anomaly row
- Checks for alerting configuration
- Sends email, text alert
- SimulateSensorFeed()
Simulates a sensor feed from a sensor network of temperature sensors Publishes NUM_MESSAGES messages to TOPIC
Creator: Robert Reinold Updated: 2017-10-02T00:00:00Z Version: v2.0 Tags: email, sendgrid, marketing, mail, api, rest, http
Usage:
- Create a free SendGrid Account.
- Log into your SendGrid account, and view the Settings > API Keys tab. Create an API Key with full access to "Mail Send" rights.
- Replace <SEND_GRID_API_KEY> with SendGrid API key
- Replace <ORIGIN_EMAIL_ADDRESS> with your desired email address. This will be the 'sender' of the email.
- Add 'SendGridEmail' as a dependency to your code services (Settings > Requires > Add)
Detects Anomalies with Moving Median Decomposition See attached whitepaper for Anomaly Detection for Predictive Maintenance https://files.knime.com/sites/default/files/inline-images/Anomaly_Detection_Time_Series_final.pdf
Run a self-training anomaly detection algorithm against a given dataset
Kind: global function
Parameter: number[] dataset array of numbers
Use precomputed anomaly detection calibration profile This can be used to speed up performance for real-time anomaly detection
Kind: inner method of AnomalyDetection
AnomalyDetection~detect() ⇒ Array.<Anomaly>
Run the algorithm against the provided dataset.
Note: This is the most computation-heavy method in this library
Kind: inner method of AnomalyDetection
Returns: Array.<Anomaly> - anomalies - the list of anomalies found in dataset
Parameter: number strictnessOverride (optional) Set the strictness of the anomaly detection
AnomalyDetection~getCalibration() ⇒ Calibration
Fetch the computed anomaly detection calibration parameters
Kind: inner method of AnomalyDetection
Returns: Calibration - calibration
Detects an anomaly in the last X MQTT Messages Uses 'AnomalyConfiguration' config to check for a key in the JSON of each message. ex. 'sensor_key' will be set to 'temperature', so we know to pull the temperature key/value pair from a mqtt message like this: {"temperature":40,"humidity":31}
Kind: global function
Returns: AnomalyVisual - data to visualize this anomaly detection
ExampleDetectAnomaly() ⇒ Array.<Anomaly>
Example logic for detecting anomalies in a dataset of sensor values
Kind: global function
Returns: Array.<Anomaly> - anomalies
Ingests updated anomaly configuration, and saves to AnomalyConfiguration Collection
Kind: global function
Parameter: AnomalyConfiguration anomalyConfiguration - map of settings to update
This is triggered by an Anomaly being recorded, and inserted into the Anomalies Collection
- Fetches the anomaly row
- Checks for alerting configuration
- Sends email, text alert
Simulates a sensor feed from a sensor network of temperature sensors Publishes NUM_MESSAGES messages to TOPIC
Kind: global typedef
Properties
| Name | Type | Description |
|---|---|---|
| min | number |
min value within threshold |
| max | number |
max value within threshold |
| strictness | number |
how strict the threshold is against the dataset |
| medians | number |
intermediary dataset representing the moving medians |
| pointsPerWindow | number |
calibration, number of points per processing window. may be increased for larger datasets |
| numWindows | number |
calibration metric derived from pointsPerWindow |
Kind: global typedef
Properties
| Name | Type | Description |
|---|---|---|
| index | number |
index in the provided dataset |
| value | number |
value of the data point that is detected as an anomaly |