HTTP 502: Bad Gateway
An HTTP 502: Bad Gateway error is rare, but when it does happen, there are really only two root causes: a problem with the load balancer, or Elasticsearch is returning a high number of deprecation warnings.
If you are seeing occasional, generic messages about HTTP 502 errors, then the most likely cause is the load balancer. The short explanation is that there are a few cases where the proxy software hits an OOM error and is restarted. This causes the load balancer to send back an HTTP 502. The error message will be very generic, and it will not say anything about Bonsai.io. The easiest solution is to simply catch and retry these HTTP 502's.
If you are seeing frequent, repeated HTTP 502 messages, and those messages say something like "A problem occurred with the Elasticsearch response. Please check status.bonsai.io or contact firstname.lastname@example.org for assistance", then it's likely due to Elasticsearch's response overwhelming the load balancer with HTTP headers. These headers might look something like this:
"The [string] field is deprecated, please use [text] or [keyword] instead on [my_field]"
This usually happens when the client sends a request using a large number of deprecated references. Elasticsearch responds with an HTTP header for each one. If there is a large number (many thousands) of headers, the load balancer will simply close the connection and respond with an HTTP 502 message. The solution is to review your client and application code, and either: A) use smaller bulk requests, or B) update the code so that it's no longer using deprecated features.
If you are having trouble with this resolving this issue, please let us know.