Managing Your Cluster

trlNote

This guide covers the basic features of managing with your Bonsai cluster. Some features, like changing the plan and deprovisioning, are only available to users of Bonsai.io. Heroku and Manifold users perform these actions in a slightly different way. Those differences are covered in the Getting Started with Heroku and Getting Started with Manifold documents, respectively.

Once you have successfully provisioned a Elasticsearch cluster, Bonsai provides a clean, workable interface for managing it. This interface will vary slightly in appearance depending on whether or not you signed up through Bonsai.io or one of our partnerships, such as Heroku or Manifold. Despite these UI differences, the functionality is the same.

Here are the different overview pages you might see:

Bonsai.io Dashboard:

Heroku Dashboard:

Manifold Dashboard:

For the purposes of this documentation, subsequent screenshots will show the Bonsai.io version of the UI. Users of Heroku and Manifold should be able to follow along, as the cosmetic differences do not impact functionality.

Overview

The overview page shows the basic details of your cluster. You can see your cluster’s status (which will be green, yellow or red), the region the cluster is provisioned in, the version of Elasticsearch it’s running, and a link out to your cluster's Kibana instance. If there is an available version upgrade, you can take advantage of it by deleting your data and clicking the button to upgrade to the next version.

You will also see a section about usage:

The display shows your actual usage vs the limits of your plan for a number of metrics:

  • Docs: This is the total number of documents you have in your index. We’re counting all documents, which can sometimes lead to confusion when nested documents are involved. If you have a parent document with three child documents, that counts as four documents, not one. This can be a source of confusion, as Elasticsearch may report different counts based on the endpoint queried.
  • Data: This is the disk footprint of your cluster, or the amount of data your cluster is occupying on the servers.
  • Shards: This is the number of shards in your cluster. We’re counting both primary and replica shards in this display.
  • Memory: This is the amount of memory your cluster is occupying on the server.

Below that, you will see a section marked “Performance”:

These boxes indicate a number of statistics about your cluster’s performance:

  • Number of Requests: This is the total number of requests your cluster has served in the past 24 hours. If there is a -, then there is no data for it. The total is broken out by READ (which is search traffic), and WRITE (which is any operation that creates, updates or deletes data).
  • Request Median: This indicates your median latency. The first number is the median response time for all requests, READ is for search traffic, and WRITE is for any operation that creates, updates or deletes data. The median is an important metric because it’s more resistant to long tail effects, and gives a better picture of overall performance than averages.
  • 99% of Requests (may also be 95%): This shows percentiles for response times for all requests, as well as for READ and WRITE traffic. A percentile indicates how much of the data set falls below a particular value. For example, if the p99 time for a cluster is 100ms, that means 99% of all requests are occurring in 100ms or less. This is an important metric for benchmarking, especially with high traffic volumes.

Heroku and Manifold users will see some additional information:

This area shows what plan level the cluster is subscribed to, and what limits (if any) come with that subscription. There is a tool tip (the "?" in the circle) which links out to the documentation for changing your plan.

Metrics

This tab shows detailed analytics about your cluster’s traffic. It will look something like this:

There is a lot to unpack here, and we’ve drafted an entire documentation page for it here.

Manage

Bonsai.io Users Only

This section does not apply to Heroku or Manifold users. If you're a user of one of these platforms, check out:

This tab allows you to administer your cluster.

The first section is the Settings view, which shows the cluster’s region, current plan, and URL.

The next section is Plan management. If you haven’t entered billing information to your account yet, you’ll see a blue button to Enter Billing Information. Clicking this will take you to your Account Management page.

If you have entered billing information, you’ll see a few different options for a new plan level:

Each option will give you some important metrics about what the new plan offers in terms of capacity. Select a new plan, then click the blue Upgrade button.

We’re Pros at Proration

When you upgrade or downgrade a cluster’s plan, you will not be billed for a month of service on each plan. We will issue a service credit for time unused on the old plan, and a prorated charge for the remainder of your billing period on the new plan. This ensures that your hard earned cash isn’t wasted in the process of growing or shrinking the cluster.

The next section is Index Trimming:

If you’re using Bonsai to host logs, or some application that creates regular indices, you can specify one or more prefix patterns here and Bonsai will automatically purge the oldest indices in the group of indices that match each pattern until the group size is below the limit specified. The pattern will only match from the start of the index name.

For example, assume you’re indexing time-series data, say, number of support tickets in a day. And you’re putting these in time-series indices like support_tickets-201801010000, support_tickets-201801020000, and so on.

With this feature, you could specify a pattern like support_tickets-, and we’ll auto-prune the oldest indices first when you reach the size limit specified for the pattern. Indices scheduled for removal will be highlighted in red.

Please note we will not purge the last remaining index that matches the pattern even if the size is above the limit.

The final section is Datadog Integration:

Datadog is a monitoring service that allows customers to see real time metrics related to their application and infrastructure, as well as receive alerts for predefined events. Datadog offers an Elasticsearch integration for monitoring clusters, and Bonsai supports this integration.

Users wishing to integrate with Datadog can use the documentation here.

Console

The Interactive Console is a web-based UI for interacting with your cluster. Think of it like a user-friendly version of curl. If you want to try out some queries or experiment with the Elasticsearch API, this is a great place to start:

The UI allows the user to select an HTTP verb, enter an endpoint and run the command. The results are shown in the navy console on the right. The box below the verb and endpoint boxes can be used to create a request body.

Beware

Some places in the Elasticsearch documentation suggest using a GET request even when passing a request body. An example would be something like:

  GET /_search
  {
      "query" : {
          "term" : { "user" : "kimchy" }
      }
  }

Without getting bogged down in pedantry, GET is a questionable verb to use in this case. A POST is more appropriate. It’s what the UI is assuming anyway. If you paste in a request body and use a GET verb, the body will be ignored.

Logs

The next tab is Streaming Logs. This will show a real-time stream of all the data hitting your cluster. There is also a tab for the Top 20 Slow Requests. This will begin to populate if requests slow down measurably.

Beta!

This feature is still in beta and can sometimes have issues. It also does not show a history. We’re working on making it better though!

Access

This section of the cluster dashboard offers a handful of data protection tools. These are to keep your data and your cluster safe.

Allowing Read-Only Access

The first section allows you to enable read-only access to your cluster, scoped to production clusters.

Deprecation Notice

Read-only functionality has been deprecated and replaced with advanced credential management. Bonsai no longer supports read-only urls without credentials for new clusters. All current clusters that have this feature enabled will continue to be able to use the read-only url. However, if you disable it, the enable form will no longer be available. See Credential Access Management for more information on how to create readonly credentials.

Kibana URL
This section shows your Kibana URL and provides a link to open it in another window.

Deprovision

Bonsai.io Users Only

This section does not apply to Heroku or Manifold users. If you're a user of one of these platforms, check out:

If you need to deprovision (destroy) a cluster, all you need is your account password:

Think Carefully!

Once a cluster has been deprovisioned, it is destroyed immediately. The data is deleted, it will instantly stop responding, and all requests will return an HTTP 404. This can not be undone unless you create a new cluster, and then reindex.

We’re Pros at Proration

When you upgrade or downgrade a cluster’s plan, you will not be billed for a month of service on each plan. We will issue a service credit for time unused on the old plan, and a prorated charge for the remainder of your billing period on the new plan. This ensures that your hard earned cash isn’t wasted in the process of growing or shrinking the cluster.