Skip to main content

Django/Haystack

Rob Sears

Users of Django/Haystack can easily integrate with Bonsai Elasticsearch! We recommend using the official Python client, as it is being actively developed alongside Elasticsearch.

Note

Haystack does not yet support Elasticsearch 6.x or 7.x, which is the current default on Bonsai. Heroku users can provision a 5.x cluster by adding Bonsai via the command line and passing in a --version flag like so:

heroku addons:create bonsai --version=5

Note that some versions require a paid plan. Free clusters are always provisioned on the latest version of Elasticsearch, regardless of which version was requested. See more details here.

Let’s get started:

Adding the Elasticsearch library

You’ll need to add the elasticsearch-py and django-haystack libraries to your requirements.txt file:

elasticsearch>=1.0.0,<2.0.0
django-haystack>=1.0.0,<2.0.0

Connecting to Bonsai

Bonsai requires basic authentication for all read/write requests. You’ll need to configure the client so that it includes the username and password when communicating with the cluster. We recommend adding the cluster URL to an environment variable, BONSAI_URL, to avoid hard-coding your authentication credentials.

The following code is a good starter for integrating Bonsai Elasticsearch into your app:

ES_URL = urlparse(os.environ.get('BONSAI_URL') or 'http://127.0.0.1:9200/')

HAYSTACK_CONNECTIONS = {
    'default': {
        'ENGINE': 'haystack.backends.elasticsearch_backend.ElasticsearchSearchEngine',
        'URL': ES_URL.scheme + '://' + ES_URL.hostname + ':443',
        'INDEX_NAME': 'haystack',
    },
}

if ES_URL.username:
    HAYSTACK_CONNECTIONS['default']['KWARGS'] = {"http_auth": ES_URL.username + ':' + ES_URL.password}

Note about ports

The sample code above uses port 443, which is the default for the https:// protocol. If you’re not using SSL/TLS and want to use http:// instead, change this value to 80.

Common Issues

One of the most common issues users see relates to SSL certs. Bonsai URLs are using TLS to secure communications with the cluster, and our certificates are signed by an authority (CA) that has verified our identity. Python needs access to the proper root certificate in order to verify that the app is actually communicating with Bonsai; if it can’t find the certificate it needs, you’ll start seeing exception messages like this:

Root certificates are missing for certificate

The fix is straightforward enough. Simply install the certifi package in the environment where your app is hosted. You can do this locally by running pip install certifi. Heroku users will also need to modify their requirements.txt, per the documentation:

certifi==0.0.8

I can’t install root certificates or certifi

If certifi and root cert management isn’t possible, you can simply bypass verification by modifying your HAYSTACK_CONNECTIONS like so:

HAYSTACK_CONNECTIONS = {
    'default': {
            'ENGINE': 'haystack.backends.elasticsearch_backend.ElasticsearchSearchEngine',
            'URL': ES_URL.scheme + '://' + ES_URL.hostname + ':443',
            'INDEX_NAME': 'documents',
            'KWARGS': {
              'use_ssl': True,
              'verify_certs': False,
            }
     },
}

This instructs Haystack to use TLS without verifying the host. This does allow for the possibility of MITM attacks, but the probably of that happening is pretty low. You’ll need to weigh the expediency of this approach against the unlikely event of someone eavesdropping, and decide whether there is an acceptable security risk of leaking data in that way.

Related articles