There are some configuration options that are universally available to all Heka output plugins. These will be consumed by Heka itself when Heka initializes the plugin and do not need to be handled by the plugin-specific initialization code.
Boolean expression, when evaluated to true passes the message to the filter for processing. Defaults to matching nothing. See: Message Matcher Syntax
The name of the message signer. If specified only messages with this signer are passed to the filter for processing.
Frequency (in seconds) that a timer event will be sent to the filter. Defaults to not sending timer events.
Connects to a remote AMQP broker (RabbitMQ) and sends messages to the specified queue. The message is serialized if specified, otherwise only the raw payload of the message will be sent. As AMQP is dynamically programmable, the broker topology needs to be specified.
Config:
An AMQP connection string formatted per the RabbitMQ URI Spec.
AMQP exchange name
AMQP exchange type (fanout, direct, topic, or headers).
Whether the exchange should be configured as a durable exchange. Defaults to non-durable.
Whether the exchange is deleted when all queues have finished and there is no publishing. Defaults to auto-delete.
The message routing key used to bind the queue to the exchange. Defaults to empty string.
Whether published messages should be marked as persistent or transient. Defaults to non-persistent.
Whether published messages should be fully serialized. If set to true then messages will be encoded to Protocol Buffers and have the AMQP message Content-Type set to application/hekad. Defaults to true.
Example (that sends log lines from the logger):
[AMQPOutput]
url = "amqp://guest:guest@rabbitmq/"
exchange = "testout"
exchangeType = "fanout"
message_matcher = 'Logger == "/var/log/system.log"'
CarbonOutput plugins parse the “stat metric” messages generated by a StatAccumulator and write the extracted counter, timer, and gauge data out to a graphite compatible carbon daemon. Output is written over a TCP or UDP socket using the plaintext protocol.
Config:
An IP address:port on which this plugin will write to. (default: “localhost:2003”)
New in version 0.5.
“tcp” or “udp” (default: “tcp”)
if set, keep the TCP connection open and reuse it until a failure; then retry (default: false)
Example:
[CarbonOutput]
message_matcher = "Type == 'heka.statmetric'"
address = "localhost:2003"
protocol = "udp"
Specialized output plugin that listens for certain Heka reporting message types and generates JSON data which is made available via HTTP for use in web based dashboards and health reports.
Config:
Specifies how often, in seconds, the dashboard files should be updated. Defaults to 5.
Defaults to “Type == ‘heka.all-report’ || Type == ‘heka.sandbox-output’ || Type == ‘heka.sandbox-terminated’”. Not recommended to change this unless you know what you’re doing.
An IP address:port on which we will serve output via HTTP. Defaults to “0.0.0.0:4352”.
File system directory into which the plugin will write data files and from which it will serve HTTP. The Heka process must have read / write access to this directory. Relative paths will be evaluated relative to the Heka base directory. Defaults to $(BASE_DIR)/dashboard.
File system directory where the Heka dashboard source code can be found. The Heka process must have read access to this directory. Relative paths will be evaluated relative to the Heka base directory. Defaults to ${SHARE_DIR}/dasher.
Example:
[DashboardOutput]
ticker_interval = 30
Output plugin that serializes messages into JSON structures and uses HTTP requests to insert them into an ElasticSearch database.
Config:
ElasticSearch cluster name. Defaults to “elasticsearch”
Name of the ES index into which the messages will be inserted. Supports interpolation of message field values (from ‘Type’, ‘Hostname’, ‘Pid’, ‘UUID’, ‘Logger’, ‘EnvVersion’, ‘Severity’, a field name, or a timestamp format) with the use of ‘%{}’ chars, so ‘%{Hostname}-%{Logger}-data’ would add the records to an ES index called ‘some.example.com-processname-data’. Defaults to ‘heka-%{2006.01.02}’.
Name of ES record type to create. Supports interpolation of message field values (from ‘Type’, ‘Hostname’, ‘Pid’, ‘UUID’, ‘Logger’, ‘EnvVersion’, ‘Severity’, field name, or a timestamp format) with the use of ‘%{}’ chars, so ‘%{Hostname}-stat’ would create an ES record with a type of ‘some.example.com-stat’. Defaults to ‘message’.
Interval at which accumulated messages should be bulk indexed into ElasticSearch, in milliseconds. Defaults to 1000 (i.e. one second).
Number of messages that, if processed, will trigger them to be bulk indexed into ElasticSearch. Defaults to 10.
Message serialization format, either “clean”, “logstash_v0”, “payload” or “raw”. “clean” is a more concise JSON representation of the message, “logstash_v0” outputs in a format similar to Logstash’s original (i.e. “version 0”) ElasticSearch schema, “payload” passes the message payload directly into ElasticSearch, and “raw” is a full JSON representation of the message. Defaults to “clean”.
If the format is “clean”, then the ‘fields’ parameter can be used to specify that only specific message data should be indexed into ElasticSearch. Available fields to choose are “Uuid”, “Timestamp”, “Type”, “Logger”, “Severity”, “Payload”, “EnvVersion”, “Pid”, “Hostname”, and “Fields” (where “Fields” causes the inclusion of any and all dynamically specified message fields. Defaults to all.
Format to use for timestamps in generated ES documents. Defaults to “2006-01-02T15:04:05.000Z”.
When generating the index name use the timestamp from the message instead of the current time. Defaults to false.
New in version 0.5.
Allows you to optionally specify the document id for ES to use. Useful for overwriting existing ES documents. If the value specified is placed within %{}, it will be interpolated to its Field value. Default is allow ES to auto-generate the id.
Example:
[ElasticSearchOutput]
message_matcher = "Type == 'sync.log'"
cluster = "elasticsearch-cluster"
index = "synclog-%{field1}-%{2006.01.02.15.04.05}"
type_name = "sync.log.line-%{field1}"
server = "http://es-server:9200"
format = "clean"
flush_interval = 5000
flush_count = 10
id = %{id}
Writes message data out to a file system.
Config:
Full path to the output file.
Output format for the message to be written. Supports json or protobufstream, both of which will serialize the entire Message struct, or text, which will output just the payload string. Defaults to text.
Whether a timestamp should be prefixed to each message line in the file. Defaults to false.
File permission for writing. A string of the octal digit representation. Defaults to “644”.
Permissions to apply to directories created for FileOutput’s parent directory if it doesn’t exist. Must be a string representation of an octal integer. Defaults to “700”.
Interval at which accumulated file data should be written to disk, in milliseconds (default 1000, i.e. 1 second). Set to 0 to disable.
Number of messages to accumulate until file data should be written to disk (default 1, minimum 1).
Operator describing how the two parameters “flush_interval” and “flush_count” are combined. Allowed values are “AND” or “OR” (default is “AND”).
Example:
[counter_file]
type = "FileOutput"
message_matcher = "Type == 'heka.counter-output'"
path = "/var/log/heka/counter-output.log"
prefix_ts = true
perm = "666"
flush_count = 100
flush_operator = "OR"
Logs messages to stdout using Go’s log package.
Config:
If set to true, then only the message payload string will be output, otherwise the entire Message struct will be output in human readable text format.
Example:
[counter_output]
type = "LogOutput"
message_matcher = "Type == 'heka.counter-output'"
payload_only = true
Specialized output plugin that listens for Nagios external command message types and delivers passive service check results to Nagios using either HTTP requests made to the Nagios cmd.cgi API or the use of the send_ncsa binary. The message payload must consist of a state followed by a colon and then the message e.g., “OK:Service is functioning properly”. The valid states are: OK|WARNING|CRITICAL|UNKNOWN. Nagios must be configured with a service name that matches the Heka plugin instance name and the hostname where the plugin is running.
Config:
An HTTP URL to the Nagios cmd.cgi. Defaults to http://localhost/nagios/cgi-bin/cmd.cgi.
Username used to authenticate with the Nagios web interface. Defaults to empty string.
Password used to authenticate with the Nagios web interface. Defaults to empty string.
Specifies the amount of time, in seconds, to wait for a server’s response headers after fully writing the request. Defaults to 2.
Must match Nagios service’s service_description attribute. Defaults to the name of the output.
Must match the hostname of the server in nagios. Defaults to the Hostname attribute of the message.
New in version 0.5.
Use send_nsca program, as provided, rather than sending HTTP requests. Not supplying this value means HTTP will be used, and any other send_nsca_* settings will be ignored.
New in version 0.5.
Arguments to use with send_nsca, usually at least the nagios hostname, e.g. [“-H”, “nagios.somehost.com”]. Defaults to an empty list.
New in version 0.5.
Timeout for the send_nsca command, in seconds. Defaults to 5.
New in version 0.5.
Specifies whether or not SSL/TLS encryption should be used for the TCP connections. Defaults to false.
New in version 0.5.
A sub-section that specifies the settings to be used for any SSL/TLS encryption. This will only have any impact if use_tls is set to true. See Configuring TLS.
Example configuration to output alerts from SandboxFilter plugins:
[NagiosOutput]
url = "http://localhost/nagios/cgi-bin/cmd.cgi"
username = "nagiosadmin"
password = "nagiospw"
message_matcher = "Type == 'heka.sandbox-output' && Fields[payload_type] == 'nagios-external-command' && Fields[payload_name] == 'PROCESS_SERVICE_CHECK_RESULT'"
Example Lua code to generate a Nagios alert:
output("OK:Alerts are working!")
inject_message("nagios-external-command", "PROCESS_SERVICE_CHECK_RESULT")
New in version 0.5.
Outputs a Heka message in an email. The message subject is the plugin name and the message content is controlled by the payload_only setting. The primary purpose is for email alert notifications e.g., PagerDuty.
Config:
If set to true, then only the message payload string will be emailed, otherwise the entire Message struct will be emailed in JSON format. (default: true)
The email address of the sender. (default: “heka@localhost.localdomain”)
An array of email addresses where the output will be sent to.
Custom subject line of email. (default: “Heka [SmtpOutput]”)
SMTP host to send the email to (default: “127.0.0.1:25”)
SMTP authentication type: “none”, “Plain”, “CRAMMD5” (default: “none”)
SMTP user name
SMTP user password
Output plugin that serializes messages into the Heka protocol format and delivers them to a listening TCP connection. Can be used to deliver messages from a local running Heka agent to a remote Heka instance set up as an aggregator and/or router.
Config:
An IP address:port to which we will send our output data.
Specifies whether or not SSL/TLS encryption should be used for the TCP connections. Defaults to false.
New in version 0.5.
A sub-section that specifies the settings to be used for any SSL/TLS encryption. This will only have any impact if use_tls is set to true. See Configuring TLS.
Specifies how often, in seconds, the output queue files are rolled. Defaults to 300.
Example:
[aggregator_output]
type = "TcpOutput"
address = "heka-aggregator.mydomain.com:55"
message_matcher = "Type != 'logfile' && Type != 'heka.counter-output' && Type != 'heka.all-report'"
WhisperOutput plugins parse the “statmetric” messages generated by a StatAccumulator and write the extracted counter, timer, and gauge data out to a graphite compatible whisper database file tree structure.
Config:
Path to the base directory where the whisper file tree will be written. Absolute paths will be honored, relative paths will be calculated relative to the Heka base directory. Defaults to “whisper” (i.e. “$(BASE_DIR)/whisper”).
Default aggregation method to use for each whisper output file. Supports the following values:
Default specification for new whisper db archives. Should be a sequence of 3-tuples, where each tuple describes a time interval’s storage policy: [<offset> <# of secs per datapoint> <# of datapoints>] (see whisper docs for more info). Defaults to:
[ [0, 60, 1440], [0, 900, 8], [0, 3600, 168], [0, 43200, 1456]]
The above defines four archive sections. The first uses 60 seconds for each of 1440 data points, which equals one day of retention. The second uses 15 minutes for each of 8 data points, for two hours of retention. The third uses one hour for each of 168 data points, or 7 days of retention. Finally, the fourth uses 12 hours for each of 1456 data points, representing two years of data.
Permission mask to be applied to folders created in the whisper database file tree. Must be a string representation of an octal integer. Defaults to “700”.
Example:
[WhisperOutput]
message_matcher = "Type == 'heka.statmetric'"
default_agg_method = 3
default_archive_info = [ [0, 30, 1440], [0, 900, 192], [0, 3600, 168], [0, 43200, 1456] ]
folder_perm = "755"