Managing Your Cluster


This guide covers the basic features of managing with your Bonsai cluster. Some features, like changing the plan and deprovisioning, are only available to users of Heroku and Manifold users perform these actions in a slightly different way. Those differences are covered in the Getting Started with Heroku and Getting Started with Manifold documents, respectively.

Once you have successfully provisioned a Elasticsearch cluster, Bonsai provides a clean, workable interface for managing it. This interface will vary slightly in appearance depending on whether or not you signed up through or one of our partnerships, such as Heroku or Manifold. Despite these UI differences, the functionality is the same.

Here are the different overview pages you might see: Dashboard:

Heroku Dashboard:

Manifold Dashboard:

For the purposes of this documentation, subsequent screenshots will show the version of the UI. Users of Heroku and Manifold should be able to follow along, as the cosmetic differences do not impact functionality.


The overview page shows the basic details of your cluster. You can see your cluster’s status (which will be green, yellow or red), the region the cluster is provisioned in, the version of Elasticsearch it’s running, and a link out to your cluster's Kibana instance. If there is an available version upgrade, you can take advantage of it by deleting your data and clicking the button to upgrade to the next version.

You will also see a section about usage:

The display shows your actual usage vs the limits of your plan for a number of metrics:

  • Docs: This is the total number of documents you have in your index. We’re counting all documents, which can sometimes lead to confusion when nested documents are involved. If you have a parent document with three child documents, that counts as four documents, not one. This can be a source of confusion, as Elasticsearch may report different counts based on the endpoint queried.
  • Data: This is the disk footprint of your cluster, or the amount of data your cluster is occupying on the servers.
  • Shards: This is the number of shards in your cluster. We’re counting both primary and replica shards in this display.
  • Memory: This is the amount of memory your cluster is occupying on the server.

Below that, you will see a section marked “Performance”:

These boxes indicate a number of statistics about your cluster’s performance:

  • Number of Requests: This is the total number of requests your cluster has served in the past 24 hours. If there is a -, then there is no data for it. The total is broken out by READ (which is search traffic), and WRITE (which is any operation that creates, updates or deletes data).
  • Request Median: This indicates your median latency. The first number is the median response time for all requests, READ is for search traffic, and WRITE is for any operation that creates, updates or deletes data. The median is an important metric because it’s more resistant to long tail effects, and gives a better picture of overall performance than averages.
  • 99% of Requests (may also be 95%): This shows percentiles for response times for all requests, as well as for READ and WRITE traffic. A percentile indicates how much of the data set falls below a particular value. For example, if the p99 time for a cluster is 100ms, that means 99% of all requests are occurring in 100ms or less. This is an important metric for benchmarking, especially with high traffic volumes.

Heroku and Manifold users will see some additional information:

This area shows what plan level the cluster is subscribed to, and what limits (if any) come with that subscription. There is a tool tip (the "?" in the circle) which links out to the documentation for changing your plan.


This tab shows detailed analytics about your cluster’s traffic. It will look something like this:

There is a lot to unpack here, and we’ve drafted an entire documentation page for it here.

Manage Users Only

This section does not apply to Heroku or Manifold users. If you're a user of one of these platforms, check out:

This tab allows you to administer your cluster.

The first section is the Settings view, which shows the cluster’s region, current plan, and URL.

The next section is Plan management. If you haven’t entered billing information to your account yet, you’ll see a blue button to Enter Billing Information. Clicking this will take you to your Account Management page.

If you have entered billing information, you’ll see a few different options for a new plan level:

Each option will give you some important metrics about what the new plan offers in terms of capacity. Select a new plan, then click the blue Upgrade button.

We’re Pros at Proration

When you upgrade or downgrade a cluster’s plan, you will not be billed for a month of service on each plan. We will issue a service credit for time unused on the old plan, and a prorated charge for the remainder of your billing period on the new plan. This ensures that your hard earned cash isn’t wasted in the process of growing or shrinking the cluster.

The next section is Log Trimming:

If you’re using Bonsai to host logs, or some application that creates regular indices, you can specify a pattern here and Bonsai will automatically purge the oldest index when your plan limits are reached.

For example, assume you’re indexing time-series data, say, number of support tickets in a day. And you’re putting these in time-series indices like support_tickets-201801010000, support_tickets-201801020000, and so on.

With this feature, you could specify a pattern like support_tickets-*, and we’ll auto-prune the oldest index first when you reach your plan limits.

The final section is Datadog Integration:

Datadog is a monitoring service that allows customers to see real time metrics related to their application and infrastructure, as well as receive alerts for predefined events. Datadog offers an Elasticsearch integration for monitoring clusters, and Bonsai supports this integration.

Users wishing to integrate with Datadog can use the documentation here.


The Interactive Console is a web-based UI for interacting with your cluster. Think of it like a user-friendly version of curl. If you want to try out some queries or experiment with the Elasticsearch API, this is a great place to start:

The UI allows the user to select an HTTP verb, enter an endpoint and run the command. The results are shown in the navy console on the right. The box below the verb and endpoint boxes can be used to create a request body.


Some places in the Elasticsearch documentation suggest using a GET request even when passing a request body. An example would be something like:

  GET /_search
      "query" : {
          "term" : { "user" : "kimchy" }

Without getting bogged down in pedantry, GET is a questionable verb to use in this case. A POST is more appropriate. It’s what the UI is assuming anyway. If you paste in a request body and use a GET verb, the body will be ignored.


The next tab is Streaming Logs. This will show a real-time stream of all the data hitting your cluster. There is also a tab for the Top 20 Slow Requests. This will begin to populate if requests slow down measurably.


This feature is still in beta and can sometimes have issues. It also does not show a history. We’re working on making it better though!


This section of the cluster dashboard offers a handful of data protection tools. These are to keep your data and your cluster safe.

Allowing Read-Only Access

The first section allows you to enable read-only access to your cluster (production grade plans only).

By enabling read-only access, anyone with your un-authed cluster URL can read the data, but creating, updating and destroying data will still require the randomized credentials.

The primary use case is if your application is using client-side searching. For example, a JavaScript that searches your cluster based on user actions (like autocomplete). Because this happens on the client side, any user could theoretically see your cluster URL. If your authentication credentials are in that URL, a malicious user could modify or delete your data.

This feature allows you to safely use your cluster URL in client-side applications. Even if a user sniffs the URL, he or she cannot do anything to your data.

Managing Credentials

The next section is Credential Management. This section allows you to create additional credentials and regenerate your cluster’s default credentials.

In addition to your default credentials, you can create a few other username/password pairs for specific purposes. For example, if you have a short term developer working on your application, you may want to give them their own token. This allows you to revoke it at any time without impacting your application, and if there is a rogue request to your cluster, the logs will show which key was used.

Managing Supplemental Tokens

To create a new token, give it a name and click on “Generate New Token”

For security purposes, the fully-qualified URL will only be displayed once. Copy it, paste it to where ever you’d like to share it, and then click “Continue.”

Back at the Access page, you will now see the new credential listed:

Supplemental credentials can be revoked at any time by clicking on the Revoke button. This action takes effect immediately.

Regenerating Your Default Credentials

You would want to regenerate your default credentials if your fully-qualified URL has been linked (say, if somehow was copy-pasted into an email or, perish the thought, StackOverflow). To do that, simply click the “Regenerate Credentials” button. This will instantly regenerate a new, randomized authentication pair.

The old credentials will remain active for two minutes or so. After that time the old keys are revoked. The purpose of the two minute warning is to give administrators the opportunity to update their application with the new credentials before the old ones expire.

Kibana URL
This section shows your Kibana URL and provides a link to open it in another window.

Deprovision Users Only

This section does not apply to Heroku or Manifold users. If you're a user of one of these platforms, check out:

If you need to deprovision (destroy) a cluster, all you need is your account password:

Think Carefully!

Once a cluster has been deprovisioned, it is destroyed immediately. The data is deleted, it will instantly stop responding, and all requests will return an HTTP 404. This can not be undone unless you create a new cluster, and then reindex.

We’re Pros at Proration

When you upgrade or downgrade a cluster’s plan, you will not be billed for a month of service on each plan. We will issue a service credit for time unused on the old plan, and a prorated charge for the remainder of your billing period on the new plan. This ensures that your hard earned cash isn’t wasted in the process of growing or shrinking the cluster.