Send data to Google Pub/Sub
Starting with version 4.5.0, AxoSyslog can send data to Google Cloud Pub/Sub using its HTTP REST API.
Prerequisites
- A Google Pub/Sub subscription.
- An IAM service account that AxoSyslog uses for authentication.
- A Google Cloud project that has the Pub/Sub API enabled.
For details, see the Google Pub/Sub tutorial.
To configure AxoSyslog, you’ll need the name of the project and the topic where you want to send your data.
Minimal configuration:
destination d_pubsub {
google-pubsub(
project("syslog-ng-project")
topic("syslog-ng-topic")
auth(
service-account(
key("/path/to/service-account-key.json")
)
)
);
};
This driver is actually a reusable configuration snippet configured to send log messages using the http() driver using a template. You can find the source of this configuration snippet on GitHub.
Options
The following options are specific to the google-pubsub() destination. But since this destination is based on the http() destination, you can use the options of the http() destination as well if needed.
Note: The
google-pubsub()destination automatically configures some of thesehttp()destination options as required by the Google Pub/Sub API.
attributes()
| Type: | string |
| Default: | "--scope rfc5424,all-nv-pairs --exclude MESSAGE" |
Description: A JSON object representing key-value pairs for the Pub/Sub Event, formatted as AxoSyslog value-pairs. By default, the google-pubsub() destination sends the RFC5424 fields as attributes. If you want to send different fields, override the default template. By default, the message part is sent in the data() option.
auth()
Options for cloud-related authentication. Currently only the GCP Service Account authentication is supported.
Specify the JSON file storing the key to the service account like this:
auth(
service-account(
key("/path/to/service-account-key.json")
)
)
service-account()
Authenticate to a service account using Service Account Key-Based Authentication. This method works both inside and outside GCP It uses a service account key generated and downloaded through the GCP IAM & Admin console. The long-term service account key is used to generate short-term tokens for authentication (also called self-signed JWT).
audience()
| Type: | string |
| Default: |
google-pubsub() destination, the audience() option is set to https://pubsub.googleapis.com/google.pubsub.v1.Publisher. Don’t change it.
key()
| Type: | string (path) |
| Default: |
Path to the service account key.
token-validity-duration()
| Type: | integer (seconds) |
| Default: | 3600 |
user-managed-service-account()
Available in AxoSyslog version 4.6 and later.
user-managed-service-account() method is only available for VMs running within GCP.
Authenticate to a user-managed service account of a GCP virtual machine using the VM Metadata Server Method. AxoSyslog interacts with the internal GCP metadata server, which provides an OAuth2 token for authentication. You can attach the default service accounts as well.
metadata-url()
| Type: | string |
| Default: | http://metadata.google.internal/computeMetadata/v1/instance/service-accounts |
The URL of the metadata server. When specifying the port, use the URL:port format.
name()
| Type: | string |
| Default: | default |
Name of the service account to use.
data()
| Type: | string/template |
| Default: | "${MESSAGE}" |
Description: The template to use as the data part of the Google Pub/Sub message.
project()
| Type: | string |
| Default: | - |
Description: The ID of the Google Cloud project where AxoSyslog sends the data. The Pub/Sub API must be enabled for the project.
topic()
| Type: | string |
| Default: | - |
Description: The name of the Google Pub/Sub topic where AxoSyslog sends the data.