Table of Contents
1. Introduction
There are situations when indexed documents do not contain all required fields and it is faster to quickly update them rather than run load workflow that might involve running external jobs – all depends on your particular case.
I want to show you how to update documents in Elasticsearch using Update API and Painless scripting so your documents can be change after being indexed already in document database.
2. Start Elasticsearch
First step as usual is to have Elasticsearch instance up and running somewhere. Easiest way for you will be to run Docker container so please use below command
docker run --rm \
--name elk01 \
-d \
-e node.name="elk01" \
-p 9200:9200 \
docker.elastic.co/elasticsearch/elasticsearch:8.11.3
After a while you can run another command that let you setup password for ‘elastic’ user
docker exec -it elk01 bash -c "(mkfifo pipe1); ( (elasticsearch-reset-password -u elastic -i < pipe1) & ( echo $'y\n123456\n123456' > pipe1) );sleep 5;rm pipe1"
3. Load test data
ELK server is running but it’s empty. In order to practice you need to load some sample data so please run below. Last line is empty intentionally.
curl -k -u elastic:123456 -XPOST "https://localhost:9200/fruits/_bulk" \
-H 'content-type: application/json' -d'
{"index":{"_id":"1"}}
{"name_base64":"QXBwbGU=","color_hex":"477265656e"}
{"index":{"_id":"2"}}
{"name_base64":"QW5hbmFz","color_hex":"59656c6c6f77"}
{"index":{"_id":"3"}}
{"name_base64":"Q2hlcnJ5","color_hex":"526564"}
'
Check if data is loaded properly by running search request
curl -k -u elastic:123456 -XGET "https://localhost:9200/fruits/_search?pretty"
4. Update one document
To update one document at the time you need to specify index, document ID and painless script that will perform changes.
curl -k -u elastic:123456 -XPOST "https://localhost:9200/fruits/_update/1?_source=name_base64&pretty" \
-H 'Content-Type: application/json' -d'{
"script": {
"source": "def x=$(params.field,\"\"); ctx._source[params.target]=x.decodeBase64();",
"params": {
"field": "name_base64",
"target": "name_decoded"
}
}
}'
_source parameter in POST request will allow to retrieve only ‘name_base64’ field for update which is visible in response
{
"_index": "fruits",
"_id": "1",
"_version": 2,
"result": "updated",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 3,
"_primary_term": 1,
"get": {
"_seq_no": 3,
"_primary_term": 1,
"found": true,
"_source": {
"name_base64": "QXBwbGU="
}
}
}
Please search for data to check if documents were updated
curl -k -u elastic:123456 -XGET "https://localhost:9200/fruits/_search?filter_path=hits.hits._source.name_decoded&pretty"
filter_path limits response to fields that you are interested in
{
"hits": {
"hits": [
{
"_source": {
"name_decoded": "Apple"
}
}
]
}
}
5. Update multiple documents
Updating multiple documents with ID specified in URL is not elegant even if scripted. Bulk API comes into picture to cure situation. For rest 2 documents run below
curl -k -u elastic:123456 -XPOST "https://localhost:9200/fruits/_bulk?pretty" \
-H 'Content-Type: application/json' -d'
{ "update" : { "_id" : "2"} }
{ "script" : { "source": "def x=$(params.field,\"\"); ctx._source[params.target]=x.decodeBase64();", "params": {"field": "name_base64","target": "name_decoded"}}}
{ "update" : { "_id" : "3"} }
{ "script" : { "source": "def x=$(params.field,\"\"); ctx._source[params.target]=x.decodeBase64();", "params": {"field": "name_base64","target": "name_decoded"}}}
'
which will return
{
"errors": false,
"took": 10,
"items": [
{
"update": {
"_index": "fruits",
"_id": "2",
"_version": 2,
"result": "updated",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 4,
"_primary_term": 1,
"status": 200
}
},
{
"update": {
"_index": "fruits",
"_id": "3",
"_version": 2,
"result": "updated",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 5,
"_primary_term": 1,
"status": 200
}
}
]
}
now check if all docs are updated and new fields present
curl -k -u elastic:123456 -XGET "https://localhost:9200/fruits/_search?filter_path=hits.hits._source.name_decoded&pretty"
response
{
"hits": {
"hits": [
{
"_source": {
"name_decoded": "Apple"
}
},
{
"_source": {
"name_decoded": "Ananas"
}
},
{
"_source": {
"name_decoded": "Cherry"
}
}
]
}
}
6. Summary
In this tutorial you have learned how to start Elasticsearch docker container, load sample documents and then you practice updating document using Update API and using bulk alternative. For update operation you have used Painless scripting. Now you can start using these techniques in your project.
Have a nice coding!
