elasticsearch2: DEPRECATED - Send messages directly to Elasticsearch version 2.0 or higher

Warning This destination is deprecated and will be removed from a future version of AxoSyslog. We recommend using the elasticsearch-http: Send messages to Elasticsearch HTTP Bulk API destination instead.

Starting with version 3.7 of AxoSyslog can directly send log messages to Elasticsearch, allowing you to search and analyze your data in real time, and visualize it with Kibana.

Note the following limitations when using the AxoSyslog elasticsearch2 destination:

Since AxoSyslog uses Java libraries, the elasticsearch2 destination has significant memory usage.

Declaration:

   @include "scl.conf"
    
    elasticsearch2(
        index("syslog-ng_${YEAR}.${MONTH}.${DAY}")
        type("test")
        cluster("syslog-ng")
    );

Example: Sending log data to Elasticsearch version 2.x and above

The following example defines an elasticsearch2 destination that sends messages in transport mode to an Elasticsearch server running on the localhost, using only the required parameters.

   @include "scl.conf"
    
    destination d_elastic {
        elasticsearch2(
            index("syslog-ng_${YEAR}.${MONTH}.${DAY}")
            type("test")
        );
    };

The following example sends 10000 messages in a batch, in transport mode, and includes a custom unique ID for each message.

   @include "scl.conf"
    
    options {
        threaded(yes);
        use-uniqid(yes);
    };
    
    source s_syslog {
        syslog();
    };
    
    destination d_elastic {
        elasticsearch2(
            index("syslog-ng_${YEAR}.${MONTH}.${DAY}")
            type("test")
            cluster("syslog-ng")
            client-mode("transport")
            custom-id("${UNIQID}")
            flush-limit("10000")
        );
    };
    
    log {
        source(s_syslog);
        destination(d_elastic);
        flags(flow-control);
    };

Example: Sending log data to Elasticsearch using the HTTP REST API

The following example send messages to Elasticsearch over HTTP using its REST API:

   @include "scl.conf"
    
    source s_network {
        network(port(5555));
    };
    
    destination d_elastic {
        elasticsearch2(
            client-mode("http")
            cluster("es-syslog-ng")
            index("x201")
            cluster-url("http://192.168.33.10:9200")
            type("slng_test_type")
            flush-limit("0")
        );
    };
    
    log {
        source(s_network);
        destination(d_elastic);
        flags(flow-control);
    };

Verify the certificate of the Elasticsearch server and perform certificate authentication (this is actually a mutual, certificate-based authentication between the AxoSyslog client and the Elasticsearch server):

   destination d_elastic {
        elasticsearch2(
            client-mode("https")
            cluster("es-syslog-ng")
            index("x201")
            cluster-url("http://192.168.33.10:9200")
            type("slng_test_type")
            flush-limit("0")
            http-auth-type("clientcert")
            java-keystore-filepath("&amp;lt;path-to-your-java-keystore&amp;gt;.jks")
            java-keystore-password("password-to-your-keystore")
            java-truststore-filepath("&amp;lt;path-to-your-java-keystore&amp;gt;.jks")
            java-truststore-password("password-to-your-keystore")
        );
    };

To install the software required for the elasticsearch2 destination, see Prerequisites.
For details on how the elasticsearch2 destination works, see How AxoSyslog interacts with Elasticsearch.
For the list of options, see Elasticsearch2 destination options (DEPRECATED).

The elasticsearch2() driver is actually a reusable configuration snippet configured to receive log messages using the Java language-binding of AxoSyslog. For details on using or writing such configuration snippets, see Reusing configuration blocks. You can find the source of the elasticsearch configuration snippet on GitHub.

Note If you delete all Java destinations from your configuration and reload syslog-ng, the JVM is not used anymore, but it is still running. If you want to stop JVM, stop syslog-ng and then start syslog-ng again.

elasticsearch2: DEPRECATED - Send messages directly to Elasticsearch version 2.0 or higher

Declaration:

Example: Sending log data to Elasticsearch version 2.x and above

Example: Sending log data to Elasticsearch using the HTTP REST API

Prerequisites

How AxoSyslog interacts with Elasticsearch

Client modes

Search Guard

Elasticsearch2 destination options (DEPRECATED)