Filebeat is a lightweight shipper for forwarding and centralizing log data. It monitors the log files or locations that you specify, collects log events, and forwards them to Elasticsearch for indexing. A common question is whether Bonsai offers support for it.
The answer is a qualified “yes.” Filebeat is a server-side tool, meaning it runs outside of Bonsai’s infrastructure and Bonsai is not involved in its configuration or management. But as a host, Bonsai is not opinionated about where your cluster’s data comes from. So if you have Filebeat running on your servers, you can configure an output to your Bonsai cluster, and it will work.
To connect Filebeat to a Bonsai cluster you just need to add your Bonsai URL to the filebeat.yml file like this:
<div class="code-snippet-container"><a fs-copyclip-element="click-2" href="#" class="btn w-button code-copy-button" title="Copy"><img class="copy-image" src="https://global-uploads.webflow.com/63c81e4decde60c281417feb/6483934eeefb356710a1d2e9_icon-copy.svg" loading="lazy" alt=""><img class="copied-image" src="https://assets-global.website-files.com/63c81e4decde60c281417feb/64839e207c2860eb9e6aa572_icon-copied.svg" loading="lazy" alt=""></a><div class="code-snippet"><pre><code fs-codehighlight-element="code" fs-copyclip-element="copy-this-2" class="hljs language-javascript">output.elasticsearch:
hosts: ["wp-play-8646224217.us-east-1.bonsaisearch.net:443"]
protocol: "https"
username: "aaa" # The randomly-generated username for your cluster
password: "xxx" # The randomly-generated password for your cluster</code></pre></div></div>
Autocreation and Bonsai
If an application sends data to an index which does not exist, Elasticsearch will create that index and assume its mappings from the data in the payload. This feature is called autocreation, and it is supported in a limited capacity on Bonsai. Certain base names can be used for autocreation. Those base names are:
This means your Filebeat index must start with one of these index names, or it will not be automatically created.
It is important to note that Filebeat requires the OSS distribution of Elasticsearch, so for this process to work the OSS version of Filebeat needs to be used.
