Hugo is a static site generator written in Go. It is conceptually similar to Jekyll, albeit with far more speed and flexibility. Hugo also supports generating output formats other than HTML, which allows users to pipe content directly into an Elasticsearch cluster.
In this guide, we are going to use this feature to tell Hugo to generate the exact format needed to submit the file to the _bulk endpoint of Elasticsearch.
In order to make use of this documentation, you will need Hugo installed and configured on your system
Use the URL for your cluster. A Bonsai URL looks something like this:
<div class="code-snippet w-richtext"><pre><code fs-codehighlight-element="code" class="hljs language-javascript">curl -XPUT https://user123:pass456@my-awesome-cluster-1234.us-east-1.bonsai.io/hugo</code></pre></div>
Hugo’s configuration settings live in a file called <span class="inline-code"><pre><code>config.toml</code></pre></span> by default. This file may also have a <span class="inline-code"><pre><code>.json</code></pre></span>. or <span class="inline-code"><pre><code>.yaml</code></pre></span>/<span class="inline-code"><pre><code>yml</code></pre></span>yml extension. Add the following snippet based on your config file format:
TOML:
<div class="code-snippet-container"><a fs-copyclip-element="click-1" href="#" class="btn w-button code-copy-button" title="Copy"><img class="copy-image" src="https://global-uploads.webflow.com/63c81e4decde60c281417feb/6483934eeefb356710a1d2e9_icon-copy.svg" loading="lazy" alt=""><img class="copied-image" src="https://assets-global.website-files.com/63c81e4decde60c281417feb/64839e207c2860eb9e6aa572_icon-copied.svg" loading="lazy" alt=""></a><div class="code-snippet"><pre><code fs-codehighlight-element="code" fs-copyclip-element="copy-this-1" class="hljs language-javascript">[outputs]
home = ["HTML", "RSS", "Bonsai"]
[outputFormats.Bonsai]
baseName = "bonsai"
isPlainText = true
mediaType = "application/json"
notAlternative = true
[params.bonsai]
vars = ["title", "summary", "date", "publishdate", "expirydate", "permalink"]
params = ["categories", "tags"]</code></pre></div></div>
JSON:
<div class="code-snippet-container"><a fs-copyclip-element="click-2" href="#" class="btn w-button code-copy-button" title="Copy"><img class="copy-image" src="https://global-uploads.webflow.com/63c81e4decde60c281417feb/6483934eeefb356710a1d2e9_icon-copy.svg" loading="lazy" alt=""><img class="copied-image" src="https://assets-global.website-files.com/63c81e4decde60c281417feb/64839e207c2860eb9e6aa572_icon-copied.svg" loading="lazy" alt=""></a><div class="code-snippet"><pre><code fs-codehighlight-element="code" fs-copyclip-element="copy-this-2" class="hljs language-javascript">
{
"outputs": {
"home": [
"HTML",
"RSS",
"Bonsai"
]
},
"outputFormats": {
"Bonsai": {
"baseName": "bonsai",
"isPlainText": true,
"mediaType": "application/json",
"notAlternative": true
}
},
"params": {
"bonsai": {
"vars": [
"title",
"summary",
"date",
"publishdate",
"expirydate",
"permalink"
],
"params": [
"categories",
"tags"
]
}
}
}</code></pre></div></div>
YAML:
<div class="code-snippet-container"><a fs-copyclip-element="click-3" href="#" class="btn w-button code-copy-button" title="Copy"><img class="copy-image" src="https://global-uploads.webflow.com/63c81e4decde60c281417feb/6483934eeefb356710a1d2e9_icon-copy.svg" loading="lazy" alt=""><img class="copied-image" src="https://assets-global.website-files.com/63c81e4decde60c281417feb/64839e207c2860eb9e6aa572_icon-copied.svg" loading="lazy" alt=""></a><div class="code-snippet"><pre><code fs-codehighlight-element="code" fs-copyclip-element="copy-this-3" class="hljs language-javascript"> outputs:
home:
- HTML
- RSS
- Bonsai
outputFormats:
Bonsai:
baseName: bonsai
isPlainText: true
mediaType: application/json
notAlternative: true
params:
bonsai:
vars:
- title
- summary
- date
- publishdate
- expirydate
- permalink
params:
- categories
- tags</code></pre></div></div>
This snippet defines a new output called “Bonsai”, and specifies some associated variables.
Hugo needs to have a template for rendering data in a way that Elasticsearch will understand. To do this, we will define a JSON template that conforms to the Elasticsearch Bulk API.
Create a template called <span class="inline-code"><pre><code>layouts/_default/list.bonsai.json</code></pre></span> and give it the following content:
<div class="code-snippet-container"><a fs-copyclip-element="click-7" href="#" class="btn w-button code-copy-button" title="Copy"><img class="copy-image" src="https://global-uploads.webflow.com/63c81e4decde60c281417feb/6483934eeefb356710a1d2e9_icon-copy.svg" loading="lazy" alt=""><img class="copied-image" src="https://assets-global.website-files.com/63c81e4decde60c281417feb/64839e207c2860eb9e6aa572_icon-copied.svg" loading="lazy" alt=""></a><div class="code-snippet"><pre><code fs-codehighlight-element="code" fs-copyclip-element="copy-this-7" class="hljs language-javascript">{{/* Generates a valid Elasticsearch _bulk index payload */}}
{{- $section := $.Site.GetPage "section" .Section }}
{{- range .Site.AllPages -}}
{{- if or (and (.IsDescendant $section) (and (not .Draft) (not .Params.private))) $section.IsHome -}}
{{/* action / metadata */}}
{{ (dict "index" (dict "_index" "hugo" "_type" "doc" "_id" .UniqueID)) | jsonify }}
{{ (dict "objectID" .UniqueID "date" .Date.UTC.Unix "description" .Description "dir" .Dir "expirydate" .ExpiryDate.UTC.Unix "fuzzywordcount" .FuzzyWordCount "keywords" .Keywords "kind" .Kind "lang" .Lang "lastmod" .Lastmod.UTC.Unix "permalink" .Permalink "publishdate" .PublishDate "readingtime" .ReadingTime "relpermalink" .RelPermalink "summary" .Summary "title" .Title "type" .Type "url" .URL "weight" .Weight "wordcount" .WordCount "section" .Section "tags" .Params.Tags "categories" .Params.Categories "authors" .Params.Authors) | jsonify }}
{{- end -}}
{{- end }}</code></pre></div></div>
When the site is generated, this will result in creating a file called public/bonsai.json, which will have the content stored in a way that can be pushed directly into Elasticsearch using the Bulk API.
To get the site’s data into Elasticsearch, render it by running <span class="inline-code"><pre><code>hugo</code></pre></span> on the command line. Then send it to your Bonsai cluster with <span class="inline-code"><pre><code>curl</code></pre></span>:
<div class="code-snippet-container"><a fs-copyclip-element="click-4" href="#" class="btn w-button code-copy-button" title="Copy"><img class="copy-image" src="https://global-uploads.webflow.com/63c81e4decde60c281417feb/6483934eeefb356710a1d2e9_icon-copy.svg" loading="lazy" alt=""><img class="copied-image" src="https://assets-global.website-files.com/63c81e4decde60c281417feb/64839e207c2860eb9e6aa572_icon-copied.svg" loading="lazy" alt=""></a><div class="code-snippet"><pre><code fs-codehighlight-element="code" fs-copyclip-element="copy-this-4" class="hljs language-javascript">curl -H "Content-Type: application/x-ndjson" -XPOST "https://user123:pass456@my-awesome-cluster-1234.us-east-1.bonsai.io/_bulk" --data-binary @public/bonsai.json</code></pre></div></div>
You should now be able to see your data in the Elasticsearch cluster:
<div class="code-snippet-container"><a fs-copyclip-element="click-5" href="#" class="btn w-button code-copy-button" title="Copy"><img class="copy-image" src="https://global-uploads.webflow.com/63c81e4decde60c281417feb/6483934eeefb356710a1d2e9_icon-copy.svg" loading="lazy" alt=""><img class="copied-image" src="https://assets-global.website-files.com/63c81e4decde60c281417feb/64839e207c2860eb9e6aa572_icon-copied.svg" loading="lazy" alt=""></a><div class="code-snippet"><pre><code fs-codehighlight-element="code" fs-copyclip-element="copy-this-5" class="hljs language-javascript">$ curl -XGET "https://user123:pass456@my-awesome-cluster-1234.us-east-1.bonsai.io/_search"{"took":1,"timed_out":false,"_shards":{"total":2,"successful":2,"failed":0},"hits":{"total":1,"max_score":1.0,"hits":[{"_index":"hugo","_type":"doc","_id":...</code></pre></div></div>
