Prerequisites

To publish messages from AxoSyslog to Apache Kafka, complete the following steps.

Steps:

1. If you want to use the Java-based modules of AxoSyslog (for example, the Elasticsearch, HDFS, or Kafka destinations), you must compile AxoSyslog with Java support.

   • Download and install the Java Runtime Environment (JRE), 1.7 (or newer). You can use OpenJDK or Oracle JDK, other implementations are not tested.

   • Install gradle version 2.2.1 or newer.

   • Set LD_LIBRARY_PATH to include the libjvm.so file, for example:LD_LIBRARY_PATH=/usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server:$LD_LIBRARY_PATH

     Note that many platforms have a simplified links for Java libraries. Use the simplified path if available. If you use a startup script to start AxoSyslog set LD_LIBRARY_PATH in the script as well.

   • If you are behind an HTTP proxy, create a gradle.properties under the modules/java-modules/ directory. Set the proxy parameters in the file. For details, see The Gradle User Guide.

2. Download the latest stable binary release of the Apache Kafka libraries (version 0.9 or newer) from http://kafka.apache.org/downloads.html.

3. Extract the Apache Kafka libraries into a single directory. If needed, collect the various .jar files into a single directory (for example, /opt/kafka/lib/) where AxoSyslog can access them. You must specify this directory in the AxoSyslog configuration file.

4. Check if the following files in the Kafka libraries have the same version number: slf4j-api-<version-number>.jar, slf4j-log4j12-<version-number>.jar. If the version number of these files is different, complete the following steps:

   1. Delete one of the files (for example, slf4j-log4j12-<version-number>.jar).

   2. Download a version that matches the version number of the other file (for example, 1.7.6) from the official SLF4J distribution.

   3. Copy the downloaded file into the directory of your Kafka library files (for example, /opt/kafka/lib/).