syslog-ng v1.6 reference manual

http://www.balabit.com/products/syslog_ng/reference-1.6/syslog-ng.html/index.html

Balázs Scheidler

Version 1.6.9

This manual is free software; you may redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2, or (at your option) any later version.

This is distributed in the hope that it will be useful, but without any warranty; without even the implied warranty of merchantability or fitness for a particular purpose. See the GNU General Public License for more details.


Table of Contents

1. Introduction to syslog-ng
2. Global objects
Sources
Filters
Destinations
Log paths
Options
3. Reference
Source drivers
internal()
unix-stream() and unix-dgram()
tcp() and udp()
file()
pipe()
sun-streams() driver
Destination drivers
file()
pipe()
unix-stream() & unix-dgram()
udp() & tcp()
usertty()
program()
Filter functions
Options reference
4. Performance tuning in syslog-ng
Setting garbage collector parameters
gc_idle_threshold()
gc_busy_threshold()
Setting output queue size
Setting the sync parameter

List of Tables

2.1. Communication method between syslogd and its clients
2.2. Available source drivers in syslog-ng
2.3. Available destination drivers in syslog-ng
2.4. Log statement flags
3.1. Available options for unix-stream() and unix-dgram()
3.2. Available options for udp() and tcp()
3.3. Available options for file()
3.4. Available options for pipe()
3.5. Available options for sun-streams
3.6. Available macros in filename expansion
3.7. Available options for file()
3.8. Available options for pipe()
3.9. Available options for unix-stream() & unix-dgram()
3.10. Available options for udp() & tcp()
3.11. Additional options for tcp()
3.12. Available options for usertty()
3.13. Available options for program()
3.14. Available filter functions in syslog-ng
3.15. List of supported global options in syslog-ng

List of Examples

2.1. Source statement on a Linux based operating system
2.2. A filter statement finding the messages containing the word deny coming from the host blurp
3.1. Using the internal() driver
3.2. Using the unix-stream() and unix-dgram() drivers
3.3. Using the udp() and tcp() drivers
3.4. Script to feed a growing logfile into syslog-ng
3.5. Using the file() driver
3.6. Using the pipe() driver
3.7. Using the sun-streams() driver
3.8. Using the file() driver
3.9. Using the file() driver with macros in the file name and a template for the message
3.10. Using the pipe() driver
3.11. Using the unix-stream() driver
3.12. Using the tcp() driver
3.13. Using the usertty() driver
3.14. Using the program() destination driver

Chapter 1. Introduction to syslog-ng

One of the most neglected area of Unix is handling system events. Daily checks for system messages is crucial for the security and health conditions of a computer system.

System logs contain much "noise" - messages having no importance - and important events which should not be lost in the tide of messages. With the current tools it is difficult to select which messages are interesting.

A message is sent to different destinations based on the assigned facility/priority pair. There are 12+8 (12 real and 8 local) predefined facilities (mail, news, auth etc.), and 8 priorities (ranging from alert to debug).

One problem is that there are facilities which are too general (e.g.: daemon), and are used by many programs, even if they do not relate each other. It is difficult to find the interesting bits from the enormous amount of messages.

A second problem is that there are very few programs which allow setting their "facility code" to use for logging. It is at best a compile time parameter.

Consequently, using facilities as a means of filtering is not an optimal approach. A better solution would be to make the syslog facility a runtime option for all applications, and add the ability to create new facilities in syslogd. Neither of these are available; the first one is not even feasible.

One of the design principles of syslog-ng was to make message filtering much more finegrained. syslog-ng is able to filter messages based on the contents of messages in addition to the priority/facility pair. This way only the really important messages are sent to a specific destination. Another design principle was to make log forwarding between firewalled segments easier using long hostname format, which makes it easy to find the originating and the chain of forwarding hosts - even if a log message traverses several computers. The last principle was a clean and powerful configuration file format.

Chapter 2. Global objects

In syslog-ng a message path (or message route) consist of one or more sources, one or more filtering rules and one or more destinations. A message is entered to syslog-ng in one of its sources, if that message matches the filtering rules it is sent to the specified destinations. Note that a message goes to all matching destinations by default, although this behavior can be changed.

Sources

A source is a set of source drivers collecting messages using a given method. For instance, there is a source driver for AF_UNIX, SOCK_STREAM style sockets, used by the Linux syslog() call.

To declare a source, the source statement has to be used in the configuration file with the following syntax:

source <identifier> { source-driver(params); source-driver(params); ... };

The identifier has to uniquely identify the given source, and may not clash with any of the reserved words (in case of a name clash, simply enclose the identifier in quotation marks).

You can control exactly which drivers are used to gather log messages, thus you have to know how your system and its native syslogd communicate. Below is an introduction to the inner workings of syslogd on some of the tested platforms:

Table 2.1. Communication method between syslogd and its clients

PlatformMethod
LinuxA SOCK_STREAM unix socket named /dev/log; some of the distributions layers switched over to using SOCK_DGRAM, though applications still work with either method.
BSD flavorsA SOCK_DGRAM unix socket named /var/run/log.
Solaris (2.5 or below)An SVR4 style STREAMS device named /dev/log.
Solaris (2.6 or above)In addition to the STREAMS device used in earlier versions, 2.6 uses a new multithreaded IPC method called door. By default the door used by syslogd is /etc/.syslog_door.

Each possible communication mechanism has the corresponding source driver in syslog-ng. For instance, to open a unix socket with SOCK_DGRAM style communication use the driver unix-dgram, the same with SOCK_STREAM style - as used under Linux - is called unix-stream.

Example 2.1. Source statement on a Linux based operating system

source src { unix-stream("/dev/log"); internal(); udp(ip(0.0.0.0) port(514)); };

Each driver may take parameters; some of them are required, others are optional. The required parameters are positional, meaning that they must be specified in a defined order. A unix-stream() driver has a single required argument, the name of the socket to listen to, and several optional parameters, which follow the socket name. Optional arguments can be specified in any order using the option(value) syntax.

Table 2.2. Available source drivers in syslog-ng

NameDescription
internal()Messages generated internally in syslog-ng.
unix-stream()Opens the specified unix socket in SOCK_STREAM mode and listens for messages.
unix-dgram()Opens the specified unix socket in SOCK_DGRAM mode and listens for messages.
file()Opens the specified file and reads messages.
pipe(), fifoOpens the specified named pipe and reads messages.
udp()Listens on the specified UDP port for messages.
tcp()Listens on the specified TCP port for messages.
sun-stream(), sun-streams()Opens the specified STREAMS device on Solaris systems and reads messages.

For a complete descriptions on the above drivers, see the section called “Source drivers”.

Filters

Filters perform log routing within syslog-ng. You can write a boolean expression using internal functions: a message passes if the expression is true.

Filters also have a unique identifying name that can be referenced in log statements.

Syntax for the filter statement:

filter <identifier> { expression; };

An expression may contain parentheses, the boolean operators "and", "or" and "not", and any of the functions listed in Table 3.14, “Available filter functions in syslog-ng”.

Example 2.2. A filter statement finding the messages containing the word deny coming from the host blurp

  filter f_blurp_deny { host("blurp") and match("deny"); };

For a complete description on the above functions, see the section called “Filter functions”.

Note

Earlier revisions of syslog-ng included a special filter identifier ("DEFAULT") which matched all not-yet-matched messages. This made a configuration much simpler and easier to manage. This feature was removed in syslog-ng 1.5.x, and a more powerful idea was introduced. For more details consult the section called “Log paths”.

Destinations

A destination is where a log is sent if the filtering rules match. Similarly to sources, destinations are comprised of one or more drivers, each defining how messages are handled. Destinations can be declared in the configuration file via a destination statement using the syntax below:

destination <identifier> { destination-driver(params); destination-driver(params); ... };

Table 2.3. Available destination drivers in syslog-ng

NameDescription
file()Writes messages to the specified file.
fifo(), pipe()Writes messages to the specified named pipe.
unix-stream()Sends messages to the specified unix socket in SOCK_STREAM style (Linux).
unix-dgram()Sends messages to the specified unix socket in SOCK_DGRAM style (BSD).
udp()Sends messages to the specified host and UDP port.
tcp()Sends messages to the specified host and TCP port.
usertty()Sends messages to the specified user's terminal if logged in.
program()Forks and launches the specified program, and sends messages to its standard input.

For detailed description of the supported drivers, see the section called “Destination drivers”.

Log paths

The previous chapters described how to define sources, filters and destinations. These components have to be connected together using log statements. The syntax of log statements is described below:

log { source(s1); source(s2); ... 
   filter(f1); filter(f2); ... 
   destination(d1); destination(d2); ... 
   flags(flag1[, flag2...]); };

Any message coming from any of the listed sources, matching all the filters is sent to all listed destinations. Log statements are processed in the order they appear in the config file.

By default, all matching log statements are processed, therefore a single log message might be sent to the same destination several times, provided the destination is listed in several log statements.

This default behavior can be changed using the flags() parameter.

Table 2.4. Log statement flags

FlagDescription
finalThis flag means that the processing of log statements ends here. Note that this does not necessarily mean that matching messages will be stored only once, as there can be matching log statements processed prior the current one.
fallbackThis flag makes a log statement 'fallback'. Being a fallback statement means that only messages not matching any 'non-fallback' log statements will be dispatched.
catchallThis flag means that the source of the message is ignored, only the filters are taken into account when matching messages.

Options

There are several options that can modify the behavior of syslog-ng. For an exact list of possible options see the section called “Options reference”. Each option may have parameters, similarly to driver specifications. The general syntax is:

options { option1(params); option2(params); ... };

Chapter 3. Reference

This chapter documents the drivers and options that can be used in the configuration file.

Source drivers

The following drivers may be used in source statements, as described in the section called “Sources”. The option log_msg_size() is available in each source: it specifies the maximum length of incoming log messages in bytes. If not specified, the value of the global option is used (see the section called “Options reference”).

internal()

All internally generated messages "come" from this special source. To collect warnings, errors and notices from syslog-ng itself, include this source in one of your source statements.

Declaration: internal()

syslog-ng will issue a warning upon startup if this driver is not referenced.

Example 3.1. Using the internal() driver

source s_local { internal(); };
  

unix-stream() and unix-dgram()

These two drivers behave similarly: they open the given AF_UNIX socket and start listening on it for messages. unix-stream() is primarily used on Linux and uses SOCK_STREAM semantics (connection oriented, no messages are lost); while unix-dgram() is used on BSDs and uses SOCK_DGRAM semantics: this may result in lost local messages if the system is overloaded.

To avoid denial of service attacks when using connection-oriented protocols, the number of simultaneously accepted connections should be limited. This can be achieved using the max-connections() parameter. The default value of this parameter is quite strict, you might have to increase it on a busy system.

Both unix-stream and unix-dgram has a single required positional argument, specifying the filename of the socket to create, and several optional parameters.

Note

syslogd on Linux originally used SOCK_STREAM sockets but this was changed in some distributions to SOCK_DGRAM at around 1999. The change was a fix to a possible DoS problem, however, this might not have been a proper solution. On Linux you can choose to use whichever driver you like as syslog clients automatically detect the socket type being used.

Declaration: 
   unix-stream(filename [options]);
   unix-dgram(filename [options]); 

The following options can be specified for these divers:

Table 3.1. Available options for unix-stream() and unix-dgram()

NameTypeDescriptionDefault
owner()stringSet the uid of the socket.root
group()stringSet the gid of the socket.root
perm()numberSet the permission mask. For octal numbers prefix the number with '0', e.g.: use 0755 for rwxr-xr-x.0666
keep-alive()yes or noSelects whether to keep connections open when syslog-ng is restarted; can be used only with unix-stream(). yes
max-connections()numberLimits the number of simultaneously open connections. Can be used only with unix-stream().10

Example 3.2. Using the unix-stream() and unix-dgram() drivers

# source declaration on Linux
source s_stream { unix-stream("/dev/log" max-connections(10)); };

# source declaration on BSD
source s_dgram { unix-dgram("/var/run/log"); };

  

tcp() and udp()

These drivers enable to receive messages from the network. As the name of the drivers implies, both UDP and TCP can be used to transport messages.

UDP is a simple datagram oriented protocol, which provides "best effort service" to transfer messages between hosts. It may lose messages, and no attempt is made at the protocol level to retransmit such lost messages.

TCP provides connection-oriented service, which basically means a flow-controlled message pipeline. In this pipeline each message is acknowledged, and retransmission is done for lost packets. Generally it is safer to use TCP, because lost connections can be detected, and no messages get lost. However, the syslog protocol traditionally uses UDP.

The tcp() and udp() drivers do not require any positional parameters. By default they bind to 0.0.0.0:514, which means that syslog-ng will listen on all available interfaces, port 514. To limit accepted connections to only one interface, use the localip() parameter as described below.

Note

The tcp port 514 is reserved for use with rshell, so select a different port if syslog-ng and rshell is used at the same time.

Declaration:
   tcp([options]);
   udp([options]);

The udp() and tcp() have the following options:

Table 3.2. Available options for udp() and tcp()

NameTypeDescriptionDefault
ip() or localip()stringThe IP address to bind to. Note that this is not the address where messages are accepted from. 0.0.0.0
port() or localport()numberThe port number to bind to.514
keep-alive()yes or noAvailable for tcp() only; specifies whether connections should be closed upon the receipt of a SIGHUP signal. yes
tcp-keep-alive()yes or noAvailable for tcp() only; specifies whether TCP keep alive messages using the SO_KEEPALIVE socket option should be enabled. no
max-connections()numberSpecifies the maximum number of simultaneous connections.10

Example 3.3. Using the udp() and tcp() drivers

source s_tcp { tcp(ip(127.0.0.1) port(1999) max-connections(10)); };
source s_udp { udp(); };
  

file()

Usually the kernel presents its messages in a special file (/dev/kmsg on BSDs, /proc/kmsg on Linux), so to read such special files the file() driver is needed. Please note that this driver cannot follow a file like tail -f does. To feed a growing logfile into syslog-ng (e.g.: an HTTP access.log), use a script like this:

Example 3.4. Script to feed a growing logfile into syslog-ng

#!/bin/sh
tail -f logfile | logger -p local4.info
    

The file driver has a single required parameter specifying the file to open, and the log_prefix() option.

Table 3.3. Available options for file()

NameTypeDescriptionDefault
log_prefix()stringThe string to prepend to log messages. Useful for logging kernel messages as they are not prefixed by kernel: by default. empty string
Declaration:
   file(filename);

Example 3.5. Using the file() driver

source s_file { file("/proc/kmsg" log_prefix("kernel: ")); };
  

Note

On Linux, the klogd daemon can be used in addition to syslog-ng to read kernel messages and forward them to syslog-ng. klogd used to preprocess kernel messages to resolve symbols etc., but as this is deprecated by ksymoops there is really no point in running both klogd and syslog-ng in parallel. Also note that running two processes reading /proc/kmsg at the same time might result in dead-locks.

pipe()

The pipe driver opens a named pipe with the specified name and listens for messages. It is used as the native message delivery protocol on HP-UX.

The pipe driver has a single required parameter, specifying the filename of the pipe to open.

Table 3.4. Available options for pipe()

NameTypeDescriptionDefault
pad_size()numberSpecifies input padding. Some operating systems (such as HP-UX) pad all messages to block boundary. This option can be used to specify the block size. (HP-UX uses 2048 bytes). 0
log_prefix()stringThe string to prepend to log messages. Useful for logging kernel messages as they are not prefixed by kernel: by default. empty string
Declaration:
   pipe(filename);

Note

You have to create this pipe using mkfifo(1).

Example 3.6. Using the pipe() driver

source s_pipe { pipe("/dev/log" pad_size(2048)); };
  

sun-streams() driver

Solaris uses its STREAMS framework to send messages to the syslogd process. syslog-ng has to be compiled with this driver enabled for sun-streams() to be usable (see ./configure --help).

Newer versions of Solaris (2.5.1 and above), use a new IPC in addition to STREAMS, called door to confirm the delivery of a message. syslog-ng supports this new IPC mechanism via the door() option (see below).

The sun-streams() driver has a single required argument specifying the STREAMS device to open, and the door() option.

Example 3.7. Using the sun-streams() driver

source s_stream { sun-streams("/dev/log" door("/etc/.syslog_door"); };
    

Table 3.5. Available options for sun-streams

NameTypeDescriptionDefault
door()stringSpecifies the filename of a door to open, needed on Solaris above 2.5.1. none

Destination drivers

Destination drivers output log messages to somewhere outside syslog-ng: a file or a network socket.

file()

The file driver is one of the most important destination drivers in syslog-ng. It allows to output messages to the specified file, or to a set of files.

The destination filename may include macros which gets expanded when the message is written, thus a simple file() driver may result in several files to be created. Macros can be included by prefixing the macro name with a '$' sign (without the quotes), just like in Bourne compatible shells.

If the expanded filename refers to a directory which does not exist, it will be created depending on the create_dirs() setting (both global and a per destination option).

Warning

Since the state of each created file must be tracked by syslog-ng, it consumes some memory for each file. If no new messages are written to a file within 60 seconds (controlled by the time_reap global option), it is closed, and its state is freed.

Exploiting this, a DoS attack can be mounted against the system. If the number of possible destination files and its needed memory is more than the amount available on the logserver.

The most suspicious macro is $PROGRAM, where the number of possible variations is quite high, so in untrusted environments $PROGRAM should not be used.

Table 3.6. Available macros in filename expansion

NameDescription
FACILITYThe name of the facility from where the message originates.
PRIORITY or LEVELThe priority of the message.
TAGThe priority and facility encoded as a 2 digit hexadecimal number.
PRIThe priority and facility encoded as a 2 or 3 digit decimal number as it is present in syslog messages.
DATEDate of the message using the BSD-syslog style timestamp format (month/day/hour/minute/second, each expressed in two digits).
FULLDATEDate of the message using the same format as DATE, but including the year as well.
ISODATEDate of the message in the ISO standard timestamp format (yy-mm-ddThh:mm:ss+-ZONE). If possible, it is recommended to use ISODATE for timestamping.
YEARThe year the message was sent. Time expansion macros can either use the time specified in the log message, e.g.: the time the log message is sent, or the time the message was received by the log server. This is controlled by the use_time_recvd() option (see the section called “Options reference”).
MONTHThe month the message was sent.
DAYThe day of month the message was sent.
WEEKDAYThe 3-letter name of the day of week the message was sent, e.g.: 'Thu'.
HOURThe hour of day the message was sent.
MINThe minute the message was sent.
SECThe second the message was sent.
TZOFFSETThe time-zone as hour offset from GMT; e.g.: '-0700'.
TZThe time zone or name or abbreviation; e.g.: 'PDT'.
HOSTThe name of the source host where the message originates from. If the message traverses several hosts and the chain_hostnames() option is on (see the section called “Options reference”), the first host in the chain is used.
FULLHOSTThe full FQDN of the host name chain, including the domain name.
HOST_FROM Name of the host that sent the message to syslog-ng. If the message traverses several hosts, this is the last host in the chain.
FULLHOST_FROM FQDN of the host that sent the message to syslog-ng. If the message traverses several hosts, this is the last host in the chain.
SOURCEIPIP address of the host that sent the message to syslog-ng. (I.e. the IP address of the host in the FULLHOST_FROM macro.)
PROGRAMThe name of the program sending the message.
MSG or MESSAGEMessage contents including the program name and pid.
MSGONLYMessage contents without the program name.

Note

The macros related to the time of the message (e.g.: ISODATE, HOUR, etc.) have two further versions each: one with the S_ and one with the R_ prefix (e.g.: S_DATE and R_DATE ). The S_DATE macro represents the date found in the log message, i.e. when the message was sent by the original application. R_DATE is the date when syslog has received the message. DATE equals either S_DATE or R_DATE, depending on the global option set in use_time_recvd() (see the section called “Options reference”).

Table 3.7. Available options for file()

NameTypeDescriptionDefault
log_fifo_size()numberThe number of entries in the output fifo. Use global setting.
fsync()yes or noForces an fsync() call on the destination fd after each write. Note: this may seriously degrade performance.no
sync_freq()numberThe logfile is synced when this number of messages has been written to it. Use global setting.
owner()stringSet the owner of the created file to the one specified. root
group()stringSet the group of the created file to the one specified. root
perm()numberThe permission mask of the file if it is created by syslog-ng. For octal numbers prefix the number with '0', e.g.: use 0755 for rwxr-xr-x. 0600
create_dirs()yes or noEnable creating non-existing directories. no
dir_perm()numberThe permission mask of directories created by syslog-ng. Log directories are only created if a file after macro expansion refers to a non-existing directory, and directory creation is enabled (see the create_dirs() option below). For octal numbers prefix the number with '0', e.g.: use 0755 for rwxr-xr-x. 0600
dir_owner()stringThe owner of directories created by syslog-ng. root
dir_group()stringThe group of directories created by syslog-ng. root
template()stringSpecifies a template which defines the logformat to be used in this file. Possible macros are the same as for the file() destination. A format conforming to the default logfile format.
template_escape()yes or noTurns on escaping ' and " in templated output files. This is useful for generating SQL statements and quoting string contents so that parts of the log message are not interpreted as commands to the SQL server. yes
remove_if_older()numberIf set to a value higher than 0, before writing to a file, syslog-ng checks whether this file is older than the specified amount of time (specified in seconds). If so, it removes the existing file and the line to be written is the first line of a new file having the same name. In combination with e.g.: the $WEEKDAY macro, this can be used for simple log rotation, in case not all history has to be kept. Never remove existing files; use append instead ( = 0).

Example 3.8. Using the file() driver

destination d_file { file("/var/log/messages" ); };
  

Example 3.9. Using the file() driver with macros in the file name and a template for the message

destination d_file {
        file("/var/log/$YEAR.$MONTH.$DAY/messages"
                template("$HOUR:$MIN:$SEC $TZ $HOST [$LEVEL] $MSG $MSG\n")
                template_escape(no)
        );
};
  

pipe()

This driver sends messages to a named pipe like /dev/xconsole.

The pipe driver has a single required parameter, specifying the filename of the pipe to open.

Declaration:
  pipe(filename);
        

Note

You have to create this pipe using mkfifo(1).

Table 3.8. Available options for pipe()

NameTypeDescriptionDefault
owner()stringSet the owner of the pipe to the one specified. root
group()stringSet the group of the pipe to the one specified. root
perm()numberThe permission mask of the pipe. For octal numbers prefix the number with '0', e.g.: use 0755 for rwxr-xr-x. 0600
template()stringSpecifies a template which defines the logformat to be used. Possible macros are the same as for the file() destination. A format conforming to the default logfile format.
template_escape()yes or noTurns on escaping ' and " in templated output files. This is useful for generating SQL statements and quoting string contents so that parts of the log message are not interpreted as commands to the SQL server. yes

Example 3.10. Using the pipe() driver

destination d_pipe { pipe("/dev/xconsole"); };
  

unix-stream() & unix-dgram()

This driver sends messages to a unix socket in either SOCK_STREAM or SOCK_DGRAM mode.

Both drivers have a single required argument specifying the name of the socket to connect to.

Declaration: 
  unix-stream(filename [options]);
  unix-dgram(filename [options]); 

Table 3.9. Available options for unix-stream() & unix-dgram()

NameTypeDescriptionDefault
template()stringSpecifies a template which defines the logformat to be used. Possible macros are the same as for the file() destination. a format conforming to the default logfile format.
template_escape()yes or noTurns on escaping ' and " in templated output files. This is useful for generating SQL statements and quoting string contents so that parts of the log message are not interpreted as commands to the SQL server. yes

Example 3.11. Using the unix-stream() driver

destination d_unix_stream { unix-stream("/var/run/logs"); };
  

udp() & tcp()

This driver sends messages to another host on the local intranet or internet using either UDP or TCP protocol.

Both drivers have a single required argument specifying the destination host address, where messages should be sent, and several optional parameters. Note that this differs from source drivers, where local bind address is implied, and none of the parameters are required.

Declaration:
  tcp(host [options]);
  udp(host [options]);

Table 3.10. Available options for udp() & tcp()

NameTypeDescriptionDefault
localip()stringThe IP address to bind to before connecting to target. 0.0.0.0
localport()numberThe port number to bind to.0
port() or destport()numberThe port number to connect to.514
template()stringSpecifies a template which defines the logformat to be used. Possible macros are the same as for the file() destination. A format conforming to the default logfile format.
template_escape()yes or noTurns on escaping ' and " in templated output files. This is useful for generating SQL statements and quoting string contents so that parts of the log message are not interpreted as commands to the SQL server. yes
spoof_sourceyes or noEnables source address spoofing. This means that the host running syslog-ng generates UDP packets with the source IP address matching the original sender of the message. It is useful when you want to perform some kind of preprocessing via syslog-ng then forward messages to your central log management solution with the source address of the original sender. This option only works for UDP destinations though the original message can be received by TCP as well. This option is only available if syslog-ng was compiled using the --enable-spoof-source configure option. no
log_fifo_size()numberThe number of entries in the output fifo. Use global setting.

Table 3.11. Additional options for tcp()

NameTypeDescriptionDefault
sync()numberThe messages are sent to the remote host when this number of messages have been collected. 0
tcp-keep-alive()yes or noAvailable for tcp() only, and specifies whether to enable TCP keep alive messages using the SO_KEEPALIVE socket option. no

Example 3.12. Using the tcp() driver

destination d_tcp { tcp("10.1.2.3" port(1999); localport(999)); };
  

usertty()

This driver writes messages to the terminal of a logged-in user.

The usertty() driver has a single required argument, specifying a username who should receive a copy of matching messages.

Declaration: 
  usertty(username);

Table 3.12. Available options for usertty()

NameTypeDescriptionDefault
template()stringSpecifies a template which defines the logformat to be used. Possible macros are the same as for the file() destination. a format conforming to the default logfile format.
template_escape()yes or noTurns on escaping ' and " in templated output files. This is useful for generating SQL statements and quoting string contents so that parts of the log message are not interpreted as commands to the SQL server. yes

Example 3.13. Using the usertty() driver

destination d_usertty { usertty("root"); };
  

program()

This driver executes the specified program with the specified arguments and sends messages to the standard input (stdin) of the child.

The program() driver has a single required parameter, specifying a program name to start. The program is executed with the help of the current shell, so the command may include both file patterns and I/O redirection, they will be processed.

Declaration: 
  program(commandtorun);

Note

The program is executed once at startup, and kept running until SIGHUP or exit. The reason for this is to prevent starting up a large number of programs for messages, which would imply an easy DoS.

Table 3.13. Available options for program()

NameTypeDescriptionDefault
template()stringSpecifies a template which defines the logformat to be used. Possible macros are the same as with destination file(). a format conforming to the default logfile format.
template_escape()yes or noTurns on escaping ' and " in templated output files. This is useful for generating SQL statements and quoting string contents so that parts of the log message are not interpreted as commands to the SQL server. yes

Example 3.14. Using the program() destination driver

    destination d_prg { program("/bin/cat >/dev/null"); };
  

Filter functions

The following functions may be used in the filter statement, as described in the section called “Filters”.

Table 3.14. Available filter functions in syslog-ng

NameSynopsisDescription
facilityfacility(facility[,facility])Match messages having one of the listed facility code.
level() or priority()level(pri[,pri1..pri2[,pri3]])Match messages based on priority.
program()program(regexp)Match messages by using a regular expression against the program name field of log messages
host()host(regexp)Match messages by using a regular expression against the hostname field of log messages.
match()match(regexp)Tries to match a regular expression to the message itself.
filter()filter(filtername)Call another filter rule and evaluate its value
netmask()netmask(ip/mask)Check the sender's IP address whether it is in the specified IP subnet

Options reference

The following options can be specified in the options statement, as described in the section called “Options”.

Table 3.15. List of supported global options in syslog-ng

NameAccepted valuesDescriptionDefault
time_reopen()numberThe time to wait in seconds before a dead connection is reestablished.60
time_reap()numberThe time to wait in seconds before an idle destination file is closed.60
sync()numberThe number of lines buffered before they are written to file.0
stats()numberThe period between two STATS messages in seconds. STATS are log messages sent by syslog-ng, containing statistics about dropped log messages.600
log_fifo_size()numberThe number of lines fitting to the output queue.100
chain_hostnames()yes or noEnable or disable the chained hostname format.yes
keep_hostname()yes or noEnable or disable hostname rewriting.no
check_hostname()yes or noVerifies that the hostname contains only valid characters. no
bad_hostname()regular expressionA regexp containing hostnames which should not be handled as hostnames. empty string
create_dirs()yes or noEnable or disable directory creation for destination files.no
owner()useridThe uid of the objects.root
group()groupidThe gid of the objects.root
perm()permission valueThe permission mask of objects. For octal numbers prefix the number with '0', e.g.: use 0755 for rwxr-xr-x.0600
dir_owner()useridThe owner of the directories.root
dir_group()groupidThe owner group of the directories.root
dir_perm()permission valueThe permission mask of the directories. For octal numbers prefix the number with '0', e.g.: use 0755 for rwxr-xr-x.0700
use_time_recvd()yes or noUse the time a message is received instead of the one specified in the message.no
use_dns()yes or noEnable or disable DNS usage. syslog-ng blocks DNS queries, so enabling DNS may lead to a Denial of Service attack. To prevent DoS, protect the syslog-ng network endpoints with firewall rules, and make sure that all hosts, which may get to syslog-ng are resolvable.yes
dns_cache()yes or noEnable or disable DNS cache usage.yes
dns_cache_size()numberNumber of hostnames in the DNS cache.1007
dns_cache_expire()numberNumber of seconds while a successful lookup is cached.3600
dns_cache_expire_failed()numberNumber of seconds while a failed lookup is cached.60
log_msg_size()numberMaximum length of a message in bytes.2048
use_fqdn()yes or noAdd Fully Qualified Domain Name instead of short hostname.no
gc_idle_threshold()numberSets the threshold value for the garbage collector when syslog-ng is idle. GC phase starts when the number of allocated objects reaches this number. See also the section called “Setting garbage collector parameters”.100
gc_busy_threshold()numberSets the threshold value for the garbage collector when syslog-ng is busy. GC phase starts when the number of allocated objects reach this number. See also the section called “Setting garbage collector parameters”.3000
sanitize_filenames()yes or noReplace control characters in filename with the dot ('.') character.yes

Chapter 4. Performance tuning in syslog-ng

There are several settings available for finetuning the behavior of syslog-ng. The defaults should be adequate for a single server or workstation installation, but for a central loghost receiving the logs from multiple computers it may not be enough.

Setting garbage collector parameters

Syslog-ng uses a garbage collector internally, and while the garbage collector is running it does not accept messages. This may cause problems if some non-connection oriented transport protocol is used, like unix-dgram() or udp(). There are two settings which control the garbage collection phase:

gc_idle_threshold()

With this option you can specify the idle threshold of the gc. If the number of allocated objects reaches this number and the system is idle (no message arrived within 100msec ), a gc phase starts. Since the system is idle, presumably no messages will be lost if the gc is run. Therefore, this value should be low, but higher than the minimally allocated objects. The minimum number of objects allocated depends on the configuration; exact numbers can be obtained by specifying the -v command line option.

gc_busy_threshold()

This threshold is used when syslog-ng is busy accepting messages (this means that within 100msec an I/O event occurred). However, to prevent syslog-ng from consuming all the available memory, gc should be run in these cases as well. Set this value high, so that the log bursts do not get interrupted by the gc.

Setting output queue size

syslog-ng always reads its incoming log channels to prevent the running daemons from blocking. This may result in lost messages if the output queue is full, it is therefore important to set the output queue size (termed in number of messages). The size of the output queue can be set either globally, or on a per destination basis.

options { log_fifo_size(1000); };
      

or

destination d_messages { file("/var/log/messages" log_fifo_size(1000)); };
        

You should set your fifo size to the estimated number of messages in a message burst. If bursts extend the bandwidth of your destination pipe, syslog-ng can feed messages into the destination pipe after the burst has collapsed.

Of course syslog-ng cannot widen the network bandwidth, so if the destination host lives on a noisy network and the logtraffic extends the bandwidth of this network, syslog-ng cannot do anything. However, it will do its best.

Setting the sync parameter

The sync parameter does not exactly do what its name implies. As you have seen messages to be sent are buffered in an output queue. The sync parameter specifies the number of messages held in this buffer before anything is written.

Note that it does not write all buffered messages into a single chunk; each distinct message is written with a single write() system call.