Heka Message Schema

New in version 0.5.

Plugin Name: SandboxFilter

File Name: lua_filters/heka_message_schema.lua

Generates documentation for each unique message in a data stream. The output is a hierarchy of Logger, Type, EnvVersion, and a list of associated message field attributes including their counts (number in the brackets). This plugin is meant for data discovery/exploration and should not be left running on a production system.

Config:

<none>

Example Heka Configuration

[SyncMessageSchema]
type = "SandboxFilter"
filename = "lua_filters/heka_message_schema.lua"
ticker_interval = 60
preserve_data = false
message_matcher = "Logger != 'SyncMessageSchema' && Logger =~ /^Sync/"

Example Output

Sync-1_5-Webserver [54600]

slf [54600]

-no version- [54600]

upstream_response_time (mismatch)

http_user_agent (string)

body_bytes_sent (number)

remote_addr (string)

request (string)

upstream_status (mismatch)

status (number)

request_time (number)

request_length (number)

Sync-1_5-SlowQuery [37]

mysql.slow-query [37]

-no version- [37]

Query_time (number)

Rows_examined (number)

Rows_sent (number)

Lock_time (number)