Categories

Using Logstash with Bonsai

Bonsai offers support for the data processing tool Logstash.
Last updated
July 7, 2023

Logstash is a data processing tool for ingesting logs into Elasticsearch. It plays a prominent role in the Elastic suite, and a common question is whether Bonsai offers support for it.

The answer is a qualified “yes.” Logstash is a server-side tool, meaning it runs outside of Bonsai’s infrastructure and Bonsai is not involved in its configuration or management. But as a host, Bonsai is not opinionated about where your cluster’s data comes from. So if you have Logstash running on your servers, you can configure an output to your Bonsai cluster, and it will work.

Connecting your Logstash instance to a Bonsai cluster is as easy as adding an output to the Logstash configuration file like so:

<div class="code-snippet w-richtext"><pre><code fs-codehighlight-element="code" class="hljs language-javascript">output {
   elasticsearch {
       # https://randomuser:randompass@something-12345.us-east-1.bonsai.io
       # would be entered as:
       hosts     => ["something-12345.us-east-1.bonsai.io:443"]
       user      => "randomuser"
       password  => "randompass"
       ssl       => true
       index     => ""
   }
}</code></pre></div>

Autocreation and Bonsai

If an application sends data to an index which does not exist, Elasticsearch will create that index and assume its mappings from the data in the payload. This feature is called autocreation, and it is supported in a limited capacity on Bonsai. Certain base names can be used for autocreation. Those base names are:

  • .kibana
  • events
  • filebeat
  • kibana-int
  • logstash
  • requests

This means your Logstash index must start with one of these index names, or it will not be automatically created.

We have made a number of tests and verified that we are fully compatible with Logstash, as of version 1.5+. Older versions of Logstash don’t have support for SSL/TLS or HTTP Basic Auth; these older versions can work with Bonsai, but only without the benefits of encryption or authentication.

If you have any issues getting Logstash to pass data along to Bonsai, check the documentation to make sure it’s set up correctly. If that doesn’t help, feel free to reach out to us at support@bonsai.io and we’ll do our best to get you pointed in the right direction.

View code snippet
Close code snippet