mozilla

Available Sandbox Modules

Alert Module

API

Stores the last alert time in the global _LAST_ALERT so alert throttling will persist between restarts.

queue(ns, msg)

Queue an alert message to be sent.

Arguments
  • ns (int64) current time in nanoseconds since the UNIX epoch.
  • msg (string) alert payload.
Return
  • true if the message is queued, false if it would be throttled.
send(ns, msg)

Send an alert message.

Arguments
  • ns (int64) current time in nanoseconds since the UNIX epoch.
  • msg (string) alert payload.
Return
  • true if the message is sent, false if it is throttled.
send_queue(ns)

Sends all queued alert message as a single message.

Arguments
  • ns (int64) current time in nanoseconds since the UNIX epoch.
Return
  • true if the queued messages are sent, false if they are throttled.
set_throttle(ns_duration)

Sets the minimum duration between alert event outputs.

Arguments
  • ns_duration (int64) minimum duration in nanoseconds between alerts.
Return
  • none
throttled(ns)

Test to see if sending an alert at this time would be throttled.

Arguments
  • ns (int64) current time in nanoseconds since the UNIX epoch.
Return
  • true if a message would be throttled, false if it would be sent.

Note

Use a zero timestamp to override message throttling.

Annotation Module

API

add(name, ns, col, stext, text)

Create an annotation in the global _ANNOTATIONS table.

Arguments
  • name (string) circular buffer payload name.
  • ns (int64) current time in nanoseconds since the UNIX epoch.
  • col (uint) circular buffer column to annotate.
  • stext (string) short text to display on the graph.
  • text (string) long text to display in the rollover.
Return
  • none
create(ns, col, stext, text)

Helper function to create an annotation table but not add it to the global list of annotations.

Arguments
  • ns (int64) current time in nanoseconds since the UNIX epoch.
  • col (uint) circular buffer column to annotate.
  • stext (string) short text to display on the graph.
  • text (string) long text to display in the rollover.
Return
  • annotation table
concat(name, annotations)

Concatenates an array of annotation tables to the specified key in the global _ANNOTATIONS table.

Arguments
  • name (string) circular buffer payload name.
  • annotations (array) annotation tables.
Return
  • none

prune(name, ns)

Arguments
  • name (string) circular buffer payload name.
  • ns (int64) current time in nanoseconds since the UNIX epoch.
Return
  • The json encoded list of annotations.
remove(name)

Entirely remove the payload name from the global _ANNOTATIONS table.

Arguments
  • name (string) circular buffer payload name.
Return
  • none

set_prune(name, ns_duration)

Arguments
  • name (string) circular buffer payload name.
  • ns_duration (int64) time in nanoseconds the annotation should remain in the list.
Return
  • none

Anomaly Detection Module

API

parse_config(anomaly_config)

Parses the anomaly_config into a Lua table. If the configuration is invalid an error is thrown.

Arguments
  • anomaly_config (string or nil)

The configuration can specify any number of algorithm function calls (space delimited if desired, but they will also work back to back with no delimiter). This allows for analysis of multiple graphs, columns, and even specification of multiple algorithms per column.

Rate of change test

Only use this test on data with a normal (Gaussian http://en.wikipedia.org/wiki/Normal_distribution) distribution. It identifies rapid changes (spikes) in the data (increasing and decreasing) but ignores cyclic data that has a more gradual rise and fall. It is typically used for something like HTTP 200 status code analysis to detect a sudden increase/decrease in web traffic.

roc(“payload_name”, col, win, hwin, sd, loss_of_data, start_of_data)
  • payload_name (string)

    Quoted string containing the payload_name value used in the inject_payload function call. If the payload name contains a double quote it should be escaped as two double quotes in a row.

  • col (uint)

    The circular buffer column to perform the analysis on.

  • win (uint)

    The number of intervals in an analysis window.

  • hwin (uint)

    The number of intervals in the historical analysis window (0 uses the full history). Must be greater than or equal to ‘win’.

  • sd (double)

    The standard deviation threshold to trigger the anomaly.

  • loss_of_data (bool)

    Alert if data stops.

  • start_of_data (bool)

    Alert if data starts.

e.g. roc(“Output1”, 1, 15, 0, 2, true, false)

Mann-Whitney-Wilcoxon test http://en.wikipedia.org/wiki/Mann-Whitney

Parametric

Only use this test on data with a normal (Gaussian http://en.wikipedia.org/wiki/Normal_distribution) distribution. It identifies more gradual changes in the data (increasing, decreasing, or any). It is typically used with something like server memory analysis where the values are more stable and gradual changes are interesting (e.g., memory leak).

mww(“payload_name”, col, win, nwin, pvalue, trend)
  • payload_name (string)

    Quoted string containing the payload_name value used in the inject_payload function call. If the payload name contains a double quote it should be escaped as two double quotes in a row.

  • col (uint)

    The circular buffer column to perform the analysis on.

  • win (uint)

    The number of intervals in an analysis window (should be at least 20).

  • nwin (uint)

    The number of analysis windows to compare.

  • pvalue (double)

    The pvalue threshold to trigger the prediction. http://en.wikipedia.org/wiki/P_value

  • trend (string)

    (decreasing|increasing|any)

e.g. mww(“Output1”, 2, 60, 10, 0.0001, decreasing)

Non-parametric

This test can be used on data with a normal (Gaussian http://en.wikipedia.org/wiki/Normal_distribution) or non-normal (nonparametric http://en.wikipedia.org/wiki/Nonparametric_statistics) distribution. It identifies overlap/similarities between two data sets. It is typically used for something like detecting an increase in HTTP 500 status code errors.

mww_nonparametric(“payload_name”, col, win, nwin, pstat)
  • payload_name (string)

    Quoted string containing the payload_name value used in the inject_payload function call. If the payload name contains a double quote it should be escaped as two double quotes in a row.

  • col (uint)

    The circular buffer column to perform the analysis on.

  • win (uint)

    The number of intervals in an analysis window.

  • nwin (uint)

    The number of analysis windows to compare.

  • pstat (double)

    Value between 0 and 1. Anything above 0.5 is an increasing trend anything below 0.5 is a decreasing trend. http://en.wikipedia.org/wiki/Mann-Whitney#.CF.81_statistic

e.g. mww_nonparametric(“Output1”, 2, 15, 10, 0.55)

Return
Configuration table if parsing was successful or nil, if nil was passed in.
detect(ns, name, cbuf, anomaly_config)

Detects anomalies in the circular buffer data returning any error messages for alert generation and array of annotations for the graph.

Arguments
  • ns (int64) current time in nanoseconds since the UNIX epoch. It used to advance the circular buffer if necessary (i.e., if no data is being received). The anomaly detection is always performed on the newest data (ignoring the current interval since it is incomplete).
  • name (string) circular buffer payload name
  • cbuf (userdata) circular buffer
  • anomaly_config (table) returned from the parse() method
Return
  • string if an anomaly was detected, otherwise nil.
  • array of annotation tables

ElasticSearch Module

API

bulkapi_index_json(index, type_name, id, ns)

Returns a simple JSON ‘index’ structure satisfying the ElasticSearch BulkAPI

Arguments
  • index (string or nil)

    String to use as the _index key’s value in the generated JSON, or nil to omit the key. Supports field interpolation as described below.

  • type_name (string or nil)

    String to use as the _type key’s value in the generated JSON, or nil to omit the key. Supports field interpolation as described below.

  • id (string or nil)

    String to use as the _id key’ value in the generated JSON, or nil to omit the key. Supports field interpolation as described below.

  • ns (number or nil)

    Nanosecond timestamp to use for any strftime field interpolation into the above fields. Current system time will be used if nil.

Field interpolation

Data from the current message can be interpolated into any of the string arguments listed above. A %{} enclosed field name will be replaced by the field value from the current message. Supported default field names are “Type”, “Hostname”, “Pid”, “UUID”, “Logger”, “EnvVersion”, and “Severity”. Any other values will be checked against the defined dynamic message fields. If no field matches, then a C strftime (on non-Windows platforms) or C89 strftime (on Windows) time substitution will be attempted, using the nanosecond timestamp (if provided) or the system clock (if not).
Return
  • JSON string suitable for use as ElasticSearch BulkAPI index directive.