Search
Close this search box.

Sent Christmas Card with Elasticsearch Watcher

Table of Contents

1. Introduction

Have you ever wonder how to insert picture into Elasticsearch document database? This is not straightforward as Elasticsearch does not support blob at the moment and if you try to load such data it will make blob.toString() just before. Fortunately there are text forms to represent binary data like base64 so whatever you have in form of PNG or JPG you can convert into base64 representation and load as text.

Second thing is what to do with the image. Of course you can extract metadata from it or use AI to do some text description and store together in one document but this time I have an idea to send it as wishes card due to upcoming Christmas Eve.

 

2. Start Elasticsearch

In order to use Watcher you have to enable trial or use your license. I also recommend disable sanitization or you can mention explicitly elements that will be allowed in HTML but this will require more work, disabling it is quicker.

				
					docker volume create --opt type=tmpfs --opt device=tmpfs --opt o=size=800m elkdata

docker run --rm \
--name elk01 \
-d \
-e xpack.license.self_generated.type=trial \
-e xpack.notification.email.html.sanitization.enabled=false \
-e node.name="elk01" \
-p 9200:9200 \
-e ES_JAVA_OPTS="-Xms2g -Xmx2g" \
-v elkdata:/usr/share/elasticsearch/data \
docker.elastic.co/elasticsearch/elasticsearch:8.11.1
				
			

before connecting you have to set password for ‘elastic’ user

				
					docker exec -it elk01 bash -c "(mkfifo pipe1); ( (elasticsearch-reset-password -u elastic -i < pipe1) & ( echo $'y\n123456\n123456' > pipe1) );sleep 5;rm pipe1"

				
			

3. Load image into Elasticsearch

To load image first you need to have one. I uploaded ready to use Christmas tree image into IPFS network so you can get it from there.

3.1. Get image from IPFS

				
					docker run --rm -it \
-e IPFS_GATEWAY="https://ipfs.filebase.io/" \
-v "$PWD:/tmp" \
curlimages/curl:8.5.0 ipfs://Qmduew22DvwZUk6gPkN5hFcyML49Rs4p4LsBLybzpymWGn -o /tmp/xmasCard.png
				
			

3.2. [Optional] Define index settings

				
					curl -k -u elastic:123456 -XPUT "https://localhost:9200/cards" \
-H 'content-type: application/json' -d'
{
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 0
  }
}'
				
			

3.3. Convert image into base64 and load into index

Along with base64 text representation of image you can add wishes and name for the image so it is easy to find it later with search.

				
					someData=`cat xmasCard.png |base64`

echo -n '' > fileToLoad.json
echo '{"index": {"_id": 1}}' >> fileToLoad.json
echo  '{"name":"xmas tree in the snow" ,"wishes":"Merry Christmas and Happy New Year" ,"image": "'$someData'"}' >> fileToLoad.json

curl -k -u elastic:123456 -XPOST "https://localhost:9200/cards/_bulk" -H 'Content-Type: application/x-ndjson' --data-binary @fileToLoad.json
				
			

4. Configure email account

4.1. Save password for email account

Here you put password into keystore of smtp account

				
					docker exec -it elk01 bash -c "bin/elasticsearch-keystore add xpack.notification.email.account.email_account.smtp.secure_password"
				
			

secure re-loadable setting needs to be reloaded

				
					curl -k -u elastic:123456 -XPOST "https://localhost:9200/_nodes/reload_secure_settings"
				
			

4.2 Prepare connection settings

Replace email address with email that you have access to 🙂

				
					curl -k -u elastic:123456 -XPUT "https://localhost:9200/_cluster/settings" \
-H 'content-type: application/json' -d'
{
    "persistent": {
        "xpack.notification.email": {
            "default_account": "email_account",
            "account": {
                "email_account": {
                    "email_defaults": {
                        "from": "xmass2023@toughcoding.net"
                    },
                    "profile": "standard",
                    "smtp": {
                        "auth": true,
                        "starttls.enable": true,
                        "host": "smtp.titan.email",
                        "port": "587",
                        "user": "xmass2023@toughcoding.net"
                    }
                }
            }
        }
    }
}'
				
			

4.3. Email vendor

If you want to use exactly the same email client like in this tutorial you can use Hostinger hosting service. I am using them without any downtime so can recommend as of now.

 

5. Create watch that will send image with wishes

Run command to create watch and schedule it to run every 10 seconds which means that you will generate a lot of SPAM for target mailbox. Before executing consider changing schedule into

				
					    "schedule" : {
      "yearly" : { "in" : "december", "on" : 24, "at" : "noon" }
    }
				
			

otherwise for testing purpose keep values like below

				
					curl -k -u elastic:123456 -XPUT "https://localhost:9200/_watcher/watch/getting_cards" \
-H 'content-type: application/json' -d'
{
    "trigger": {
        "schedule": {
            "interval": "10s"
        }
    },
    "input": {
        "chain": {
            "inputs": [
                {
                    "search": {
                        "search": {
                            "request": {
                                "indices": [
                                    "cards"
                                ],
                                "body": {
                                    "query": {
                                        "match": {
                                            "name": "xmas tree in the snow"
                                        }
                                    }
                                }
                            }
                        }
                    }
                },
                {
                    "currentyear": {
                        "transform": {
                            "script": {
                                "source": " return [ '\''_value'\'' : ctx.trigger.triggered_time.getYear() ]",
                                "lang": "painless"
                            }
                        }
                    }
                },
                {
                    "nextyear": {
                        "transform": {
                            "script": {
                                "source": " return [ '\''_value'\'' : ctx.trigger.triggered_time.getYear() + 1 ]",
                                "lang": "painless"
                            }
                        }
                    }
                }
            ]
        }
    },
    "condition": {
        "compare": {
            "ctx.payload.search.hits.total": {
                "gt": 0
            }
        }
    },
    "actions": {
        "log": {
            "logging": {
                "text": "sending image {{ctx.payload.search.hits.hits.0._source.name}}"
            }
        },
        "send_email": {
            "email": {
                "profile": "standard",
                "to": "xmass2023@toughcoding.net",
                "subject": "Happy Xmas {{ctx.payload.currentyear._value}}",
                "body": {
                    "html": "<html>\n  <body>\n    <h1>\n      {{ctx.payload.search.hits.hits.0._source.wishes}} {{ctx.payload.nextyear._value}}\n    </h1>\n    <br />\n <img src=\"data:image/png;base64, {{ctx.payload.search.hits.hits.0._source.image}}\" alt=\"Image\">\n  </body>\n</html>"
                }
            }
        }
    }
}'
				
			

6. Check emails

Messages with magic card should filling mailbox now…

When you open one you will see card with wishes:

christmass card with wishes

7. Deactivate watch and delete

If you want to deactivate

				
					curl -k -u elastic:123456 -XPUT "https://localhost:9200/_watcher/watch/getting_cards/_deactivate"
				
			

or delete

				
					curl -k -u elastic:123456 -XDELETE "https://localhost:9200/_watcher/watch/getting_cards"
				
			

 

8. Final thoughts

In this knowledge article you practice how to start Elasticsearch, load image file into it and then schedule mailing of Christmas wishes together with that image. All data stored in Elasticsearch, cool right?

Follow me on LinkedIn

2 Comments

Leave a Reply

Your email address will not be published. Required fields are marked *

Share the Post:

Enjoy Free Useful Amazing Content

Related Posts