Table of Contents
1. Introduction
Have you ever wonder how to insert picture into Elasticsearch document database? This is not straightforward as Elasticsearch does not support blob at the moment and if you try to load such data it will make blob.toString() just before. Fortunately there are text forms to represent binary data like base64 so whatever you have in form of PNG or JPG you can convert into base64 representation and load as text.
Second thing is what to do with the image. Of course you can extract metadata from it or use AI to do some text description and store together in one document but this time I have an idea to send it as wishes card due to upcoming Christmas Eve.
2. Start Elasticsearch
In order to use Watcher you have to enable trial or use your license. I also recommend disable sanitization or you can mention explicitly elements that will be allowed in HTML but this will require more work, disabling it is quicker.
docker volume create --opt type=tmpfs --opt device=tmpfs --opt o=size=800m elkdata
docker run --rm \
--name elk01 \
-d \
-e xpack.license.self_generated.type=trial \
-e xpack.notification.email.html.sanitization.enabled=false \
-e node.name="elk01" \
-p 9200:9200 \
-e ES_JAVA_OPTS="-Xms2g -Xmx2g" \
-v elkdata:/usr/share/elasticsearch/data \
docker.elastic.co/elasticsearch/elasticsearch:8.11.1
before connecting you have to set password for ‘elastic’ user
docker exec -it elk01 bash -c "(mkfifo pipe1); ( (elasticsearch-reset-password -u elastic -i < pipe1) & ( echo $'y\n123456\n123456' > pipe1) );sleep 5;rm pipe1"
3. Load image into Elasticsearch
To load image first you need to have one. I uploaded ready to use Christmas tree image into IPFS network so you can get it from there.
3.1. Get image from IPFS
docker run --rm -it \
-e IPFS_GATEWAY="https://ipfs.filebase.io/" \
-v "$PWD:/tmp" \
curlimages/curl:8.5.0 ipfs://Qmduew22DvwZUk6gPkN5hFcyML49Rs4p4LsBLybzpymWGn -o /tmp/xmasCard.png
3.2. [Optional] Define index settings
curl -k -u elastic:123456 -XPUT "https://localhost:9200/cards" \
-H 'content-type: application/json' -d'
{
"settings": {
"number_of_shards": 1,
"number_of_replicas": 0
}
}'
3.3. Convert image into base64 and load into index
Along with base64 text representation of image you can add wishes and name for the image so it is easy to find it later with search.
someData=`cat xmasCard.png |base64`
echo -n '' > fileToLoad.json
echo '{"index": {"_id": 1}}' >> fileToLoad.json
echo '{"name":"xmas tree in the snow" ,"wishes":"Merry Christmas and Happy New Year" ,"image": "'$someData'"}' >> fileToLoad.json
curl -k -u elastic:123456 -XPOST "https://localhost:9200/cards/_bulk" -H 'Content-Type: application/x-ndjson' --data-binary @fileToLoad.json
4. Configure email account
4.1. Save password for email account
Here you put password into keystore of smtp account
docker exec -it elk01 bash -c "bin/elasticsearch-keystore add xpack.notification.email.account.email_account.smtp.secure_password"
secure re-loadable setting needs to be reloaded
curl -k -u elastic:123456 -XPOST "https://localhost:9200/_nodes/reload_secure_settings"
4.2 Prepare connection settings
Replace email address with email that you have access to 🙂
curl -k -u elastic:123456 -XPUT "https://localhost:9200/_cluster/settings" \
-H 'content-type: application/json' -d'
{
"persistent": {
"xpack.notification.email": {
"default_account": "email_account",
"account": {
"email_account": {
"email_defaults": {
"from": "xmass2023@toughcoding.net"
},
"profile": "standard",
"smtp": {
"auth": true,
"starttls.enable": true,
"host": "smtp.titan.email",
"port": "587",
"user": "xmass2023@toughcoding.net"
}
}
}
}
}
}'
4.3. Email vendor
If you want to use exactly the same email client like in this tutorial you can use Hostinger hosting service. I am using them without any downtime so can recommend as of now.
5. Create watch that will send image with wishes
Run command to create watch and schedule it to run every 10 seconds which means that you will generate a lot of SPAM for target mailbox. Before executing consider changing schedule into
"schedule" : {
"yearly" : { "in" : "december", "on" : 24, "at" : "noon" }
}
otherwise for testing purpose keep values like below
curl -k -u elastic:123456 -XPUT "https://localhost:9200/_watcher/watch/getting_cards" \
-H 'content-type: application/json' -d'
{
"trigger": {
"schedule": {
"interval": "10s"
}
},
"input": {
"chain": {
"inputs": [
{
"search": {
"search": {
"request": {
"indices": [
"cards"
],
"body": {
"query": {
"match": {
"name": "xmas tree in the snow"
}
}
}
}
}
}
},
{
"currentyear": {
"transform": {
"script": {
"source": " return [ '\''_value'\'' : ctx.trigger.triggered_time.getYear() ]",
"lang": "painless"
}
}
}
},
{
"nextyear": {
"transform": {
"script": {
"source": " return [ '\''_value'\'' : ctx.trigger.triggered_time.getYear() + 1 ]",
"lang": "painless"
}
}
}
}
]
}
},
"condition": {
"compare": {
"ctx.payload.search.hits.total": {
"gt": 0
}
}
},
"actions": {
"log": {
"logging": {
"text": "sending image {{ctx.payload.search.hits.hits.0._source.name}}"
}
},
"send_email": {
"email": {
"profile": "standard",
"to": "xmass2023@toughcoding.net",
"subject": "Happy Xmas {{ctx.payload.currentyear._value}}",
"body": {
"html": "<html>\n <body>\n <h1>\n {{ctx.payload.search.hits.hits.0._source.wishes}} {{ctx.payload.nextyear._value}}\n </h1>\n <br />\n <img src=\"data:image/png;base64, {{ctx.payload.search.hits.hits.0._source.image}}\" alt=\"Image\">\n </body>\n</html>"
}
}
}
}
}'
6. Check emails
Messages with magic card should filling mailbox now…
When you open one you will see card with wishes:
7. Deactivate watch and delete
If you want to deactivate
curl -k -u elastic:123456 -XPUT "https://localhost:9200/_watcher/watch/getting_cards/_deactivate"
or delete
curl -k -u elastic:123456 -XDELETE "https://localhost:9200/_watcher/watch/getting_cards"
8. Final thoughts
In this knowledge article you practice how to start Elasticsearch, load image file into it and then schedule mailing of Christmas wishes together with that image. All data stored in Elasticsearch, cool right?
2 Comments