{"_id":"5873bd3f0953ef19007342cb","project":"5633ebff7e9e880d00af1a53","user":"5633ec9b35355017003ca3f2","category":{"_id":"574d127f6f075519007da3d0","project":"5633ebff7e9e880d00af1a53","__v":0,"version":"5633ec007e9e880d00af1a56","sync":{"url":"","isSync":false},"reference":false,"createdAt":"2016-05-31T04:26:39.925Z","from_sync":false,"order":3,"slug":"versions","title":"Versions"},"parentDoc":null,"version":{"_id":"5633ec007e9e880d00af1a56","project":"5633ebff7e9e880d00af1a53","__v":15,"createdAt":"2015-10-30T22:15:28.105Z","releaseDate":"2015-10-30T22:15:28.105Z","categories":["5633ec007e9e880d00af1a57","5633f072737ea01700ea329d","5637a37d0704070d00f06cf4","5637cf4e7ca5de0d00286aeb","564503082c74cf1900da48b4","564503cb7f1fff210078e70a","567af26cb56bac0d0019d87d","567afeb8802b2b17005ddea0","567aff47802b2b17005ddea1","567b0005802b2b17005ddea3","568adfffcbd4ca0d00aebf7e","56ba80078cf7c9210009673e","574d127f6f075519007da3d0","574fde60aef76a0e00840927","57a22ba6cd51b22d00f623a0"],"is_deprecated":false,"is_hidden":false,"is_beta":false,"is_stable":true,"codename":"","version_clean":"1.0.0","version":"1.0"},"__v":0,"updates":[],"next":{"pages":[],"description":""},"createdAt":"2017-01-09T16:41:35.474Z","link_external":false,"link_url":"","githubsync":"","sync_unique":"","hidden":false,"api":{"results":{"codes":[]},"settings":"","auth":"required","params":[],"url":""},"isReference":false,"order":0,"body":"Most guides to upgrading Elasticsearch in the wild focus on the operational details - this stuff can be interesting to note, but if you have a Bonsai cluster you don’t need to worry about ops - we’ve got that covered. But what about best practices for updating your app’s version of Elasticsearch? Read on to prepare for your version upgrade. We’ve streamlined this process with some new tools, so open your Bonsai cluster dashboard and let’s dive in!\n\n# Best Practices\n_Or, helpful tips to avoid breaking your app and having a really bad day._\n\n- First, use one Elasticsearch cluster per environment. One for production, and another for staging.\n\n- Test full deletion and reindexing in staging before upgrading. Reindexing is something you should be familiar with anyway as a part of normal usage of Elasticsearch, such as changing analyzer settings, backfilling a new field, or - in this case - upgrading to a new major version.\n\n- Research, implement and test breaking changes. More on this in a bit, we’ve got some tools and tips that will help make this process less time-consuming.\n\n# Test in Staging\n\nIf you don’t already have a staging app and cluster, create one.\n\n### 1. Create a new cluster (on the same version as your prod cluster) and reindex the same data and settings as your production cluster.\n\nWe’ll be using this new cluster in the steps below to run tests and get it working perfectly with our app before making a hot-switch for zero downtime for your users. If your current cluster isn’t used for production, skip this step and follow along with your current cluster. New Bonsai clusters automatically provision with version 2.4.\n\n### 2. Run the migration checker tests.\n\nThe Migration Checker will run a test suite against any cluster to confirm compatibility with the version you’re upgrading to. Navigate to your new cluster dashboard and click the ‘Run Migration Tests’ button. You can run the checker on any cluster, but we’ll be using the new testing cluster you created in step 1.\n[block:image]\n{\n  \"images\": [\n    {\n      \"image\": [\n        \"https://files.readme.io/3040905-01.png\",\n        \"01.png\",\n        1490,\n        844,\n        \"#f4f5ea\"\n      ]\n    }\n  ]\n}\n[/block]\nA new window will open up and automatically run tests against your indices. \n[block:image]\n{\n  \"images\": [\n    {\n      \"image\": [\n        \"https://files.readme.io/b998c26-02.png\",\n        \"02.png\",\n        1490,\n        736,\n        \"#235170\"\n      ]\n    }\n  ]\n}\n[/block]\n### 3. If you have failing tests, check the docs, makes changes, and try again.\n\nIf your migration checker yields failing tests you should correct each issue before upgrading. The checker provides links to documentation to help you make changes to your data and index mappings prepare for migration. Click on the \"docs\" link next to each line-item test to read about break changes and how to fix them in your integration.\n\nA good practice for managing your Elasticsearch indices is to version control your mappings and index settings. If you’re not already doing it, this is a good time to start.\n\n### 4. If your checks are all green, you’re clear for upgrade!\n\nIf all checks pass, you can start your upgrade process. We advise replacing your staging cluster first and deploying your new version of ES in staging. Once this final check does well, create a second ‘new version’ cluster to replace your production cluster.\n\n### 5. Launch to production!\n\nOnce you have your new production cluster ready to go, it’s time to switch your .env cluster URLs for production. Simply replace your old Bonsai URL in your app for your new one, and deprovision your old cluster(s).\n\n# For Our Heroku Amigos\n\nThe Bonsai Heroku addon allows multiple Bonsai resources to be attached to one app, and upgrading versions is a perfect use-case of this feature. Heroku users should follow the same steps above, only they should provision a Bonsai resource through Heroku and attach it to the same app as their current staging app.\n\nAdd a new add-on cluster to your staging app using the CLI tool or the UI.\n\n## Through the CLI Tool\n\nRun the `addons:create` command.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"$ heroku addons:create bonsai:PLAN_NAME -a APP_NAME\",\n      \"language\": \"text\",\n      \"name\": \"\"\n    }\n  ]\n}\n[/block]\nThe CLI should respond with a confirmation and further instructions.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"$ heroku addons:create bonsai:staging -a fragrant-cedar-4321\\nCreating bonsai:staging on ⬢fragrant-cedar-4321... free\\n    \\nThanks for adding Bonsai! Please log in to the Bonsai dashboard to finalize your account setup.\\n    \\nUse `heroku addons:open bonsai` to view the dashboard.\\nCreated bonsai-circular-12345 as BONSAI_ROSE_URL\\nUse heroku addons:docs bonsai to view documentation\",\n      \"language\": \"text\",\n      \"name\": \"Terminal\"\n    }\n  ]\n}\n[/block]\n\n## Through the UI\n\nNavigate to your app’s addons. In the example below, we've already have a cluster added to a staging app. To add a new one, I’ll go to “Find more add-ons”.\n[block:image]\n{\n  \"images\": [\n    {\n      \"image\": [\n        \"https://files.readme.io/dffd6a2-03.png\",\n        \"03.png\",\n        1245,\n        161,\n        \"#e5e8ed\"\n      ]\n    }\n  ]\n}\n[/block]\nThis will take me to the Heroku addons marketplace:\n[block:image]\n{\n  \"images\": [\n    {\n      \"image\": [\n        \"https://files.readme.io/31406bc-04.png\",\n        \"04.png\",\n        1265,\n        666,\n        \"#51478b\"\n      ]\n    }\n  ]\n}\n[/block]\nSearch for Bonsai, and click through to our addons page:\n- https://elements.heroku.com/addons/bonsai\n\nPick a plan that’s appropriate for your staging needs (remember the name), and click “Install Bonsai”\n[block:image]\n{\n  \"images\": [\n    {\n      \"image\": [\n        \"https://files.readme.io/60c9e5c-05.png\",\n        \"05.png\",\n        1214,\n        647,\n        \"#73598e\"\n      ]\n    }\n  ]\n}\n[/block]\nIt will ask you which app to install to:\n[block:image]\n{\n  \"images\": [\n    {\n      \"image\": [\n        \"https://files.readme.io/e96861a-06.png\",\n        \"06.png\",\n        1267,\n        666,\n        \"#63626a\"\n      ]\n    }\n  ]\n}\n[/block]\nIt will route you back to your app, and you’ll need to choose the plan you picked before for capacity, and click “Provision”.\n[block:image]\n{\n  \"images\": [\n    {\n      \"image\": [\n        \"https://files.readme.io/9b17a4d-07.png\",\n        \"07.png\",\n        1270,\n        665,\n        \"#63636b\"\n      ]\n    }\n  ]\n}\n[/block]\nAfter a few seconds you’ll notice you now have two addons:\n\n[block:image]\n{\n  \"images\": [\n    {\n      \"image\": [\n        \"https://files.readme.io/9e60a82-08.png\",\n        \"08.png\",\n        1242,\n        259,\n        \"#f4f6f5\"\n      ]\n    }\n  ]\n}\n[/block]\nYour new addon will have a random color associated with it. This shows up in your config as BONSAI_COLOR_URL. For example, BONSAI_CYAN_URL in the output below:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"$ heroku config -a fragrant-cedar-4321 \\n\\n=== fragrant-cedar-4321 Config Vars\\nBONSAI_CYAN_URL: https://key:secret:::at:::apple-12345678.us-east-1.bonsaisearch.net\\nBONSAI_URL:      https://key:secret@cherry-12345678.us-east-1.bonsaisearch.net\\nLANG:            en_US.UTF-8\\nRACK_ENV:        production\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nAs of this article’s writing, all new clusters are provisioned as ES version 2.4 by default. In this example, I would do 4 things:\n\n\n1. Run my Migration Checker tests on my old cluster’s dashboard.\n  As it runs tests, I will make notes, read docs that I needed, and make changes to my settings for the new cluster I just created (CYAN).\n2. Making appropriate index changes, then reindex my new settings and data to CYAN, also making adjustments in my app’s ES clients for the major version change if needed.\n3. Launch staging to test a few queries, and ask my product team to try to break it :)\n4. When all works out, provision a new Bonsai resource on my production app. This will follow the same process for the staging app, only now since all of our testing was done in staging all we’d need to do is set up the indices and settings, reindex, and launch to production.\n\n# Catch-Me’s\n\nWe have two last things to take note of while upgrading your version.\n\n***1. Launch your production version change during off-peak hours.***\n***2. Be explicit about which .env variables you’re using in your initializers.***\nSince these multiple Heroku add-on naming conventions aren’t tied to anything in your environment, make sure you create some internal documentation for fellow team members about the new config vars. A newbie looking at Bonsai URL’s in the .env is going to have no idea why one URL is is named Gold, Black, or Purple, and these names have much less significance that a name like BONSAI_STAGING_2_URL, etc. You can make it clear to others by writing very explicit notes in initializers or creating some tests that break production rollout to prevent unexpected changes. In my example, I’m using Rails and our gem bonsai-elasticsearch-rails - and important thing to note for those using this gem is that by default it looks for BONSAI_URL in the config to set everything up in production. Since I have a new environment variable, I need to explicitly set up my client with the new URL in an initializer to override the gem’s default settings. This goes for any other app.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"\\nif env.staging?\\n  BONSAI_URL_2 = ENV['BONSAI_CYAN_URL']\\n  \\n  if BONSAI_URL_2\\n    Elasticsearch::Model.client = Elasticsearch::Client.new(\\n      url:BONSAI_URL_2,\\n      log: true\\n    )\\n  end\\nelse\\n  BONSAI_URL = ENV['BONSAI_URL']\\n  \\n  if BONSAI_URL\\n    Elasticsearch::Model.client = Elasticsearch::Client.new(\\n      url:BONSAI_URL,\\n      log: true\\n    )\\n  end\\nend\",\n      \"language\": \"ruby\",\n      \"name\": \"initializers/elasticsearch.rb\"\n    }\n  ]\n}\n[/block]\n# Enjoy your new version!\n\nUpgrading your dependency versions can be difficult, so we hope this article and the Migration Checker have made it easier. If you need any help or tips, please open a ticket at support@bonsai.io.\n\nCheers,\nThe Bonsai Team","excerpt":"In this post, we’ll walk through the best way to handle migrating your cluster from 1.7 to 2.4 (or any major version upgrades), erring on the side of over-explaining.","slug":"upgrading-your-version","type":"basic","title":"Upgrading Major Versions"}

Upgrading Major Versions

In this post, we’ll walk through the best way to handle migrating your cluster from 1.7 to 2.4 (or any major version upgrades), erring on the side of over-explaining.

Most guides to upgrading Elasticsearch in the wild focus on the operational details - this stuff can be interesting to note, but if you have a Bonsai cluster you don’t need to worry about ops - we’ve got that covered. But what about best practices for updating your app’s version of Elasticsearch? Read on to prepare for your version upgrade. We’ve streamlined this process with some new tools, so open your Bonsai cluster dashboard and let’s dive in! # Best Practices _Or, helpful tips to avoid breaking your app and having a really bad day._ - First, use one Elasticsearch cluster per environment. One for production, and another for staging. - Test full deletion and reindexing in staging before upgrading. Reindexing is something you should be familiar with anyway as a part of normal usage of Elasticsearch, such as changing analyzer settings, backfilling a new field, or - in this case - upgrading to a new major version. - Research, implement and test breaking changes. More on this in a bit, we’ve got some tools and tips that will help make this process less time-consuming. # Test in Staging If you don’t already have a staging app and cluster, create one. ### 1. Create a new cluster (on the same version as your prod cluster) and reindex the same data and settings as your production cluster. We’ll be using this new cluster in the steps below to run tests and get it working perfectly with our app before making a hot-switch for zero downtime for your users. If your current cluster isn’t used for production, skip this step and follow along with your current cluster. New Bonsai clusters automatically provision with version 2.4. ### 2. Run the migration checker tests. The Migration Checker will run a test suite against any cluster to confirm compatibility with the version you’re upgrading to. Navigate to your new cluster dashboard and click the ‘Run Migration Tests’ button. You can run the checker on any cluster, but we’ll be using the new testing cluster you created in step 1. [block:image] { "images": [ { "image": [ "https://files.readme.io/3040905-01.png", "01.png", 1490, 844, "#f4f5ea" ] } ] } [/block] A new window will open up and automatically run tests against your indices. [block:image] { "images": [ { "image": [ "https://files.readme.io/b998c26-02.png", "02.png", 1490, 736, "#235170" ] } ] } [/block] ### 3. If you have failing tests, check the docs, makes changes, and try again. If your migration checker yields failing tests you should correct each issue before upgrading. The checker provides links to documentation to help you make changes to your data and index mappings prepare for migration. Click on the "docs" link next to each line-item test to read about break changes and how to fix them in your integration. A good practice for managing your Elasticsearch indices is to version control your mappings and index settings. If you’re not already doing it, this is a good time to start. ### 4. If your checks are all green, you’re clear for upgrade! If all checks pass, you can start your upgrade process. We advise replacing your staging cluster first and deploying your new version of ES in staging. Once this final check does well, create a second ‘new version’ cluster to replace your production cluster. ### 5. Launch to production! Once you have your new production cluster ready to go, it’s time to switch your .env cluster URLs for production. Simply replace your old Bonsai URL in your app for your new one, and deprovision your old cluster(s). # For Our Heroku Amigos The Bonsai Heroku addon allows multiple Bonsai resources to be attached to one app, and upgrading versions is a perfect use-case of this feature. Heroku users should follow the same steps above, only they should provision a Bonsai resource through Heroku and attach it to the same app as their current staging app. Add a new add-on cluster to your staging app using the CLI tool or the UI. ## Through the CLI Tool Run the `addons:create` command. [block:code] { "codes": [ { "code": "$ heroku addons:create bonsai:PLAN_NAME -a APP_NAME", "language": "text", "name": "" } ] } [/block] The CLI should respond with a confirmation and further instructions. [block:code] { "codes": [ { "code": "$ heroku addons:create bonsai:staging -a fragrant-cedar-4321\nCreating bonsai:staging on ⬢fragrant-cedar-4321... free\n \nThanks for adding Bonsai! Please log in to the Bonsai dashboard to finalize your account setup.\n \nUse `heroku addons:open bonsai` to view the dashboard.\nCreated bonsai-circular-12345 as BONSAI_ROSE_URL\nUse heroku addons:docs bonsai to view documentation", "language": "text", "name": "Terminal" } ] } [/block] ## Through the UI Navigate to your app’s addons. In the example below, we've already have a cluster added to a staging app. To add a new one, I’ll go to “Find more add-ons”. [block:image] { "images": [ { "image": [ "https://files.readme.io/dffd6a2-03.png", "03.png", 1245, 161, "#e5e8ed" ] } ] } [/block] This will take me to the Heroku addons marketplace: [block:image] { "images": [ { "image": [ "https://files.readme.io/31406bc-04.png", "04.png", 1265, 666, "#51478b" ] } ] } [/block] Search for Bonsai, and click through to our addons page: - https://elements.heroku.com/addons/bonsai Pick a plan that’s appropriate for your staging needs (remember the name), and click “Install Bonsai” [block:image] { "images": [ { "image": [ "https://files.readme.io/60c9e5c-05.png", "05.png", 1214, 647, "#73598e" ] } ] } [/block] It will ask you which app to install to: [block:image] { "images": [ { "image": [ "https://files.readme.io/e96861a-06.png", "06.png", 1267, 666, "#63626a" ] } ] } [/block] It will route you back to your app, and you’ll need to choose the plan you picked before for capacity, and click “Provision”. [block:image] { "images": [ { "image": [ "https://files.readme.io/9b17a4d-07.png", "07.png", 1270, 665, "#63636b" ] } ] } [/block] After a few seconds you’ll notice you now have two addons: [block:image] { "images": [ { "image": [ "https://files.readme.io/9e60a82-08.png", "08.png", 1242, 259, "#f4f6f5" ] } ] } [/block] Your new addon will have a random color associated with it. This shows up in your config as BONSAI_COLOR_URL. For example, BONSAI_CYAN_URL in the output below: [block:code] { "codes": [ { "code": "$ heroku config -a fragrant-cedar-4321 \n\n=== fragrant-cedar-4321 Config Vars\nBONSAI_CYAN_URL: https://key:secret@apple-12345678.us-east-1.bonsaisearch.net\nBONSAI_URL: https://key:secret@cherry-12345678.us-east-1.bonsaisearch.net\nLANG: en_US.UTF-8\nRACK_ENV: production", "language": "text" } ] } [/block] As of this article’s writing, all new clusters are provisioned as ES version 2.4 by default. In this example, I would do 4 things: 1. Run my Migration Checker tests on my old cluster’s dashboard. As it runs tests, I will make notes, read docs that I needed, and make changes to my settings for the new cluster I just created (CYAN). 2. Making appropriate index changes, then reindex my new settings and data to CYAN, also making adjustments in my app’s ES clients for the major version change if needed. 3. Launch staging to test a few queries, and ask my product team to try to break it :) 4. When all works out, provision a new Bonsai resource on my production app. This will follow the same process for the staging app, only now since all of our testing was done in staging all we’d need to do is set up the indices and settings, reindex, and launch to production. # Catch-Me’s We have two last things to take note of while upgrading your version. ***1. Launch your production version change during off-peak hours.*** ***2. Be explicit about which .env variables you’re using in your initializers.*** Since these multiple Heroku add-on naming conventions aren’t tied to anything in your environment, make sure you create some internal documentation for fellow team members about the new config vars. A newbie looking at Bonsai URL’s in the .env is going to have no idea why one URL is is named Gold, Black, or Purple, and these names have much less significance that a name like BONSAI_STAGING_2_URL, etc. You can make it clear to others by writing very explicit notes in initializers or creating some tests that break production rollout to prevent unexpected changes. In my example, I’m using Rails and our gem bonsai-elasticsearch-rails - and important thing to note for those using this gem is that by default it looks for BONSAI_URL in the config to set everything up in production. Since I have a new environment variable, I need to explicitly set up my client with the new URL in an initializer to override the gem’s default settings. This goes for any other app. [block:code] { "codes": [ { "code": "\nif env.staging?\n BONSAI_URL_2 = ENV['BONSAI_CYAN_URL']\n \n if BONSAI_URL_2\n Elasticsearch::Model.client = Elasticsearch::Client.new(\n url:BONSAI_URL_2,\n log: true\n )\n end\nelse\n BONSAI_URL = ENV['BONSAI_URL']\n \n if BONSAI_URL\n Elasticsearch::Model.client = Elasticsearch::Client.new(\n url:BONSAI_URL,\n log: true\n )\n end\nend", "language": "ruby", "name": "initializers/elasticsearch.rb" } ] } [/block] # Enjoy your new version! Upgrading your dependency versions can be difficult, so we hope this article and the Migration Checker have made it easier. If you need any help or tips, please open a ticket at support@bonsai.io. Cheers, The Bonsai Team