PostgreSQL csvlog

Available in AxoSyslog version 4.5.0 and later.

This parser processes messages in the PostgreSQL csvlog format. The following sample message is a multi-line message with embedded NL characters. This is a single, multi-line log entry that starts with the timestamp.

2023-08-08 12:05:52.805 UTC,,,22113,,64d22fa0.5661,1,,2023-08-08 12:05:52 UTC,23/74060,0,LOG,00000,"automatic vacuum of table ""tablename"": index scans: 0
pages: 0 removed, 4 remain, 0 skipped due to pins, 0 skipped frozen
tuples: 114 removed, 268 remain, 0 are dead but not yet removable, oldest xmin: 149738000
buffer usage: 97 hits, 0 misses, 6 dirtied
avg read rate: 0.000 MB/s, avg write rate: 114.609 MB/s
system usage: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s",,,,,,,,,""

The postgresql-csvlog-parser() extracts the information from this message into a set of name-value pairs. By default, the name-value pairs have the .pgsql prefix.

@version: current

log {
    source { file("/var/log/pgsql.log" follow-freq(1) flags(no-parse)); };
    parser { postgresql-csvlog-parser() };
    destination { ... };
};

The postgresql-csvlog-parser() driver is actually a reusable configuration snippet configured to parse log messages using the csv-parser(). For details on using or writing such configuration snippets, see Reusing configuration blocks. You can find the source of this configuration snippet on GitHub.

on-type-error()

Description: Specifies what to do when casting a parsed value to a specific data type fails. Note that the flags(drop-invalid) option and the on-error() global option also affects the behavior.

Accepts the same values as the on-error() global option.

prefix()

Description: Insert a prefix before the name part of the parsed name-value pairs to help further processing. For example:

• To insert the my-parsed-data. prefix, use the prefix(my-parsed-data.) option.

• To refer to a particular data that has a prefix, use the prefix in the name of the macro, for example, ${my-parsed-data.name}.

• If you forward the parsed messages using the IETF-syslog protocol, you can insert all the parsed data into the SDATA part of the message using the prefix(.SDATA.my-parsed-data.) option.

Names starting with a dot (for example, .example) are reserved for use by AxoSyslog. If you use such a macro name as the name of a parsed value, it will attempt to replace the original value of the macro (note that only soft macros can be overwritten, see Hard versus soft macros for details). To avoid such problems, use a prefix when naming the parsed values, for example, prefix(my-parsed-data.)