Table of Contents
1. Intro
If you went through Elasticsearch documentation and find out enroll node API without idea what exactly it is and how to use it then I am happy for you right now. This is because I want to show you in details how to use it.
First I will show you how to call this API and use output to create certificates for new node. Let’s start
2. Own implementation of enroll node API
2.1. Theory
API calls
GET /_security/enroll/kibana
GET /_security/enroll/node
are reserved for Kibana and Elasticsearch security auto-configuration. Kibana I described you already.
Now it’s time to talk about Elasticsearch node.
2.2. Start
2.2.1. Starting 1st node with autoconf
docker network create elkai
docker run --rm \
--name elkprivate \
-e "cluster.name=private-cluster" \
-e "node.name=elkprivate" \
-p 9201:9200 \
-d \
--network elkai \
-e ES_JAVA_OPTS="-Xms2g -Xmx2g" \
docker.elastic.co/elasticsearch/elasticsearch:8.15.2
docker exec -it elkprivate bash -c "(mkfifo pipe1); ( (elasticsearch-reset-password -u elastic -i < pipe1) & ( echo $'y\n123456\n123456' > pipe1) );sleep 5;rm pipe1"
When you analyse class AutoConfigureNode.java you can conclude
1. Initial Node Configuration: In this mode (when no enrollment token is provided), AutoConfigureNode generates its own self-signed CA certificates, node certificates, and keys. It then stores these certificates and keys in PKCS#12 keystore files (transport.p12 and http.p12). The passwords for these keystores are generated randomly and stored in the Elasticsearch keystore. No enrollment token is involved in this process.
2. Enrollment Mode: In this mode (when an enrollment token is provided), AutoConfigureNode uses the provided token to enroll into an existing cluster. The token contains an API key and the addresses of existing nodes. AutoConfigureNode uses this information to connect to any node and retrieve the CA certificates, node certificates, and keys. Crucially, these certificates and keys are sent as response to the enrollment request, encoded as Base64 strings. AutoConfigureNode then decodes these Base64 strings, create the certificates/keys, and stores them in the keystore files, just like in the initial node configuration. The keystore passwords are again generated randomly.
2.2.2. Use API enroll node to get certs
extract and convert certificates/keys from DER (base64) to PEM
mkdir -p ./certs
curl -XGET 'https://localhost:9201/_security/enroll/node' -u 'elastic:123456' \
-k -o ./enroll_response.json && \
jq -r '.http_ca_key' enroll_response.json | base64 -d | openssl rsa -inform DER -outform PEM > ./certs/http_ca_key.key && \
jq -r '.http_ca_cert' enroll_response.json | base64 -d | openssl x509 -inform DER -outform PEM > ./certs/http_ca_cert.crt && \
jq -r '.transport_ca_cert' enroll_response.json | base64 -d | openssl x509 -inform DER -outform PEM > ./certs/transport_ca_cert.crt && \
jq -r '.transport_key' enroll_response.json | base64 -d | openssl rsa -inform DER -outform PEM > ./certs/transport_key.key && \
jq -r '.transport_cert' enroll_response.json | base64 -d | openssl x509 -inform DER -outform PEM > ./certs/transport_cert.crt
2.2.3. Start second node and join cluster
docker run --rm \
--name elkprivate2 \
--network elkai \
-e "node.name=elkprivate2" \
-e "cluster.name=private-cluster" \
-e "discovery.seed_hosts=elkprivate" \
-e "xpack.security.enabled=true" \
-e "xpack.security.http.ssl.enabled=true" \
-e "xpack.security.transport.ssl.enabled=true" \
-e "xpack.security.transport.ssl.verification_mode=certificate" \
-e "xpack.security.transport.ssl.key=/usr/share/elasticsearch/config/certs/transport_key.key" \
-e "xpack.security.transport.ssl.certificate=/usr/share/elasticsearch/config/certs/transport_cert.crt" \
-e "xpack.security.transport.ssl.certificate_authorities=/usr/share/elasticsearch/config/certs/transport_ca_cert.crt" \
-e "xpack.security.http.ssl.key=/usr/share/elasticsearch/config/certs/http_ca_key.key" \
-e "xpack.security.http.ssl.certificate=/usr/share/elasticsearch/config/certs/http_ca_cert.crt" \
-e "xpack.security.http.ssl.certificate_authorities=/usr/share/elasticsearch/config/certs/http_ca_cert.crt" \
-v ./certs:/usr/share/elasticsearch/config/certs \
-p 9202:9200 \
docker.elastic.co/elasticsearch/elasticsearch:8.15.2
You have just used enroll node API in your own custom way. Let me explain you also how Elasticsearch is using this API internally.
3. Delegate enrollment process (ENROLLMENT_TOKEN)
elasticsearch-create-enrollment-token -s node
You will get base64 response like
eyJ2ZXIiOiI4LjE0LjAiLCJhZHIiOlsiMTcyLjMxLjAuMjo5MjAwIl0sImZnciI6IjI0NThkMWRlMWYwNjFlYzIxMWZlNzEzNWRkNjEyYWIwOWE1ZDU4YmQ1M2RmYzQ1MjJkNzE3MjlmZmJlMzQzZGYiLCJrZXkiOiJxQnp4ZUpZQkg0Z3pwMjEtcm9NTjpyMi1vWXJ1TVRnT3FkUmVZMTJDblFBIn0=
after decoding
echo -n "" | base64 -d
you can see
{
"ver":"8.14.0",
"adr":[
"172.31.0.2:9200"
], "fgr":"2458d1de1f061ec211fe7135dd612ab09a5d58bd53dfc4522d71729ffbe343df",
"key":"qBzxeJYBH4gzp21-roMN:r2-oYruMTgOqdReY12CnQA"
}
Where:
- ver: current token version decoupled from node version
- adr: list of addresses (host:port of cluster nodes)
- fgr: CA fingerprint
- key: id and api_key joined by a colon (:)
This give you possibilities to
- Contact the listed address (adr)
- Verify TLS using the fingerprint (fgr)
- Authenticate using the API key (key)
Elasticsearch registered mentioned API key (encoded together with id)
{
"api_keys": [
{
"id": "qBzxeJYBH4gzp21-roMN",
"name": "enrollment_token_API_key_4wvxeJYBy24CeoVsrk4A",
"type": "rest",
"creation": 1745785826830,
"expiration": 1745787626830,
"invalidated": false,
"username": "autogenerated_mmIMlxDW",
"realm": "default_file",
"realm_type": "file",
"metadata": {},
"role_descriptors": {
"create_enrollment_token": {
"cluster": [
"cluster:admin/xpack/security/enroll/node"
],
"indices": [],
"applications": [],
"run_as": [],
"metadata": {},
"transient_metadata": {
"enabled": true
}
}
}
}
]
}
With cluster level permission to call API _security/enroll/node
To authenticate you need to encode back key into base64 format.
echo -n 'qBzxeJYBH4gzp21-roMN:r2-oYruMTgOqdReY12CnQA' | base64
Therefore commands
token=`docker exec -it elkprivate elasticsearch-create-enrollment-token -s node | tr -d '\r\n'`
apikey=$(echo -n $token | base64 -d | jq -r '.key' | tr -d '\n' | base64)
curl -k -H "Authorization: ApiKey $apikey" "https://localhost:9201/_security/_authenticate"
Will return response of successful auth
{
"username":"autogenerated_8BrYzZyu",
"roles":[
],
"full_name":null,
"email":null,
"metadata":{
},
"enabled":true,
"authentication_realm":{
"name":"_es_api_key",
"type":"_es_api_key"
},
"lookup_realm":{
"name":"_es_api_key",
"type":"_es_api_key"
},
"authentication_type":"api_key",
"api_key":{
"id":"qhwEeZYBH4gzp21-sYPb",
"name":"enrollment_token_API_key_aL0EeZYBy5_T02FEsbLN"
}
}
Enrollment token basically allow new node to run API call to get all certs in base64 format, decode( base64 DER ASN.1) them and save with password protection inside keystore p12.
| Format | Encoding | File Extension | Contents |
|---|---|---|---|
| PEM | Base64 | .pem, .crt, .key | Human-readable, includes headers like -----BEGIN CERTIFICATE----- |
| DER | Binary | .der, .cer | Binary-encoded X.509 certificate |
| PKCS#12 | Binary | .p12, .pfx | Certificate + Private Key + Chain |
Passwords are type of secure setting and thus stored in elasticsearch-keystore. This is all handled automatically when starting new node with ENROLLMENT_TOKEN as Environment variable. Therefore this API call seems to be internal rather than used by end users unless you want to do different implementation. For example you may choose different certs format like key & crt in separate files instead of combined keystore p12.
curl -k -H "Authorization: ApiKey $apikey" "https://localhost:9201/_security/enroll/node?pretty"
response
{
"http_ca_key" : "MIIJQQIBADANBgkqhkiG9w0BAQEFAASCCSswggknAgEAAoICAQDlSHkUDvtwLZrE2WXM4mWpZjB3+5+C2hdRNv3Jcu2y/fQ/Z8iqFZsRrVzuiFLsmB5U3MxZp5JELaKEantyqJ/rR5oMT3hiEqOwg6L1/vFA4ifQZBHe1yC0c3s5i2c0ae3C3A9bAgRkyp+cXhw/7Tm8m08iKPZRyDzLJzvzY3Ic8zAaanJvdR1lkZDyGuZyyOQ0Kpw3bMG8bYpwuqbzNw3JiJkGjRzJlFb/zp5Cfu8UGu+bwr6UulJzLR6IIMcY7kSfwbN/w1KxLS3FeHZM9xCYDAlchZqf5H9y2LlFt6qA5BXVBwbZdM+tleG9LcNnns8HpBWr4rQMHggylfCVIzR/RGnKG3xOrsARpIGrr6nNTnH7VP1BIUpi0Rh6OOH93cJZf21mW8Jg/lQt6mXCiGFqYybBCQD9kOcAwcuE+69lolaWmD6eid3tyeQecmWtvv3v4e3zpnUiO9QGfizwZR36pBq7pNR29g/O3lGKNG6CpjZ1+2GUBOFeRx7vHHmIch04C17SCsvXbdJh1DafzzwbnV1005o1sg7/znHgTREUTJoGe4ti4+j3x+eySzDN+kuaznmWXR/oO4kCMsI/GwwdvRGw6wOqhmOPIffK3kVbLMulggqxIiPeR1l5QqCfxy3gXELhOFWin3PEM7g/uKOZCAXvFUXnamSJmywQZ6XXbwIDAQABAoICAANmGSFifXiyvF4ZqOgqHP6vXJ2StdEfQYQ7L/TqBrsGB7Ze5/sZeeR/sOZ6T9xg/uYcJc1YbhMjqAqVd4ICHOjEdXSkQvEVPKbztJk378SZ9aQhr6AhiUMTiSqXte5xeYxPbczYEU+bL1WGkZ2i+x2gIcKsX8ZjlP8f9EQN39WtBzQFCvA7CLYGO+NS4cKm1rctQIaTzqVn7ErygWIOmV4476fItLoKQGXnXrI2pn2iTpX9A40529oIO7Eh50Gn2o2RtgL5VsL4m/qW6A4JBY5zczoTYvDm1rrRmqF9cQIbH3W6dnAHysJUe/WtdX2WPZRr6sRTBn89fLqfPWNAk9kd76a65UaBrpRLD8oCiPDX6+VKn5lsnseAjz9Q/QZET0irJO7z0lEwrsolaErARYbGlcuaiVJu8rYCm4TVOXwZLj88c/zwh/feYaIqMnD8J0fFrMhpWD3lrU6S8bXRVlPW5N+ZscL253DDV7PHYYvGPrDeCzRFQEDnanISQ45k/s9Z3miVxMUHyeyzRs234525ZQhwt5JJnitaZka4lYhgOi7TPtsKDtP5qx4to9B75jxheFybVrEjeNHx6FV/AU0SJ9xvVyXaWniJfMzgYOiPUQfRAFisWNNvBGJC3LAbQ7FlJastk8EHeaF01YOPVFO/5G4O2TG8eZY8lvllQ4hNAoIBAQD4MiaQsmNpKBorJkQvN55g1tJfhn+Hv59f61imMZzYDpsnxM9TAyTmxYyQWBj7TJ7JLYjCAnV8en2rr9F5AC+XrUDQfdkY+8A2s5ALrUInrlz6dB17nsuuD5JXQfLB+HGwtmURUioVXgkjZIEmUQjhFEoKqL5uYTYRVoYFe+oqkVwbu+HRUKL8uL5UqDfVlgVVdz7FgToLbM1rL3yUKtWA+ef27yW37umx9nN7GV8zQr7Y2mtANiJc/m0rS9epHIqq37Bpw137hXjpqqrmaoLxtUct1yF8eVkjGIe8BB3iW6EuTsetJsJdzCykQszT+LNO6ylGf55N/PIIIUncF1J1AoIBAQDsfhV/dNTAww6gM/2d+N52Tgb2BQoB1p9AfjF0CDE9KFo6QBUwM9W36IhEMsaFPAu8mzqPvhx/G35kGXoiBjdQy768fqiU4H3ad/SOWcWvv1JbJG/ClCIi/fbrKHOLp+0GtC7ma1VEMctSMJf8xQbvTeJ9rlfGDs6Scj8BN641T2yOVTcP4o3BnUPKhhtf7y5job312gC9TopAC/FlTMLgjxvm2ha425xhzweKIa1OBouqltj6DRba/q4iZheDHhBhqVkfd5XMQ7LjOvqArFnzQdAGbE0xECk3zTBwnyaZpFqgbovXnnHJDoLAnHnq48tyLE8npIi0O6lyT6/zyz3TAoIBACFhLUqckz4bRJm1BcqL4mMHwTdTsWciYF7YGg4P8hYksL1CfNefPqFCxErNbl6tyVUpKJFfH0nkd25VsQhi/AOcK3Fe24m+ofU5ZRAM9y67Boowf56WlrIMKhROLEXmEfRAM4uGz14cTYJTDAOJNnZ+8g5I74OM02a1ikTGi5G6Bvc753ztKV5vwjlEfm3dRU5fQTPy00miEmZt9oU+1YQJGCdftmTsXsWubY2KtgEA4wXrKi1ymmCDX09JXfHCguvENcejnMotAzv51g4zGPVE+hOyMTC6aUCNFgQ1UYeV5zGBKt2grtdKlF5rFX0Un5jni5+Nnk7CzBJH19Uf1dkCggEAZnt89W67g40DmYjuLrbaMIo/mf01CPborBoDdDTYoZgLAZMjm2/a/YdXBba7MKGEtIbQKdpInwxbCSXBN11aOzkPvr9kbp+Z6kJxR/6/HBncpoJzq+5lnKRRjDZLXAm5PV53tIJuwM6TPVqxgmgfSTSHIc+bWciw2+WGSBDI/XEdqdBjvA6BP5XW+ryCwF/1ylcc6p0+FMskPfzu7ucEzCs3/CImFpWUfw4oRkOxxk4v2AzmmfVyIhSbgWycY8Vzc97fMDOyRoJP0wiL5ZbKpA+xBheIm+pU8kmI5EUThSEj5MIC971Bsc8H+k/UxWVRlHh/FL7IPtJb6518sjkw4QKCAQBpnF/Yj+ezcE4Zi44rpgGNWvhEXi5DL8XeXpYPKEYi8C5YY1GdegACzPnA+SUQT0jrquVArW2i8MOrAWYssJsfT0X3WDitShulPVDZliFKG+mIAWTzJEUiZvrXNkmn3fL19Hv09N4YTRsVU7ULNuB3tuJTx0fHrte5/v+3knTGYtwDo9jaifikvolteRXUvunaP5Z2L1z+DDtguk8avahLeakNG177fDazmXFwdVdbq/bCVVsby8vtAMC8OfYjHUn6NOJbUJ0dozf0NzAQ/SlPRX67irWEzn/BJoxvYVQIr3vu9o/sdR+f+X3XdI00wZZ2lKaywMxwKadQ6inQ8BDp",
"http_ca_cert" : "MIIFWTCCA0GgAwIBAgIUPc854BiP6jqbP3myZyxXMcZSY8UwDQYJKoZIhvcNAQELBQAwPDE6MDgGA1UEAxMxRWxhc3RpY3NlYXJjaCBzZWN1cml0eSBhdXRvLWNvbmZpZ3VyYXRpb24gSFRUUCBDQTAeFw0yNTA0MjcxMTMwNDFaFw0yODA0MjYxMTMwNDFaMDwxOjA4BgNVBAMTMUVsYXN0aWNzZWFyY2ggc2VjdXJpdHkgYXV0by1jb25maWd1cmF0aW9uIEhUVFAgQ0EwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQDlSHkUDvtwLZrE2WXM4mWpZjB3+5+C2hdRNv3Jcu2y/fQ/Z8iqFZsRrVzuiFLsmB5U3MxZp5JELaKEantyqJ/rR5oMT3hiEqOwg6L1/vFA4ifQZBHe1yC0c3s5i2c0ae3C3A9bAgRkyp+cXhw/7Tm8m08iKPZRyDzLJzvzY3Ic8zAaanJvdR1lkZDyGuZyyOQ0Kpw3bMG8bYpwuqbzNw3JiJkGjRzJlFb/zp5Cfu8UGu+bwr6UulJzLR6IIMcY7kSfwbN/w1KxLS3FeHZM9xCYDAlchZqf5H9y2LlFt6qA5BXVBwbZdM+tleG9LcNnns8HpBWr4rQMHggylfCVIzR/RGnKG3xOrsARpIGrr6nNTnH7VP1BIUpi0Rh6OOH93cJZf21mW8Jg/lQt6mXCiGFqYybBCQD9kOcAwcuE+69lolaWmD6eid3tyeQecmWtvv3v4e3zpnUiO9QGfizwZR36pBq7pNR29g/O3lGKNG6CpjZ1+2GUBOFeRx7vHHmIch04C17SCsvXbdJh1DafzzwbnV1005o1sg7/znHgTREUTJoGe4ti4+j3x+eySzDN+kuaznmWXR/oO4kCMsI/GwwdvRGw6wOqhmOPIffK3kVbLMulggqxIiPeR1l5QqCfxy3gXELhOFWin3PEM7g/uKOZCAXvFUXnamSJmywQZ6XXbwIDAQABo1MwUTAdBgNVHQ4EFgQUUXzuyeESWGpgbdEIM3qAgl8NVk8wHwYDVR0jBBgwFoAUUXzuyeESWGpgbdEIM3qAgl8NVk8wDwYDVR0TAQH/BAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAgEAfvK9o/pa+Mqzwe6hyZ49Z/t2zCJo76X/vG820BT/o34DFK32yZEWNfGkVOd5SEwx+6NlbHI6ekjmXIaMxwxi9PnicYCKXlD6JTdaBC8d5XhAY+49FUKh3k8zkEIkInidsBwG+HOKm3g4a7GDptHXIg43hzn26qQlPX1e3d8ru+YGSOljlc3CtU5krjA8URVFce4Mofr31nUqzVBySj/2V5FRmKSRb3rwNOPraFwQNtPeYsLACfqtuOCesQeCwoY9FqRpmcBoeIiWu7NGUbJ2CHyXHf5wyjPwE4pVEt007v/7yug6aEr/gtbVspoSVcfOLvczs+afOQIBQrePzyZPdhhwZsicofjWHTvO5KyTXrmZ6oPFU6r2SPXzc/Wuf5Y1dS2cv0f4EQ8MEYDfChNYAi8+ysajq4AD6/+LtbbMxLYMDlSYYqN4FCafwrCEz5stDcf5n5lHE/Wghy5joi5JhHog3OQ/2jooq3ED9PwF6T5K0nBiGZy7z9z2hkPiWg4/SSHEe2C5MikSHrxcDX6VSD4LfF3b0wBVNnEWOKq0ouTBccTq9YtfzKr1bE7aZMOUcOOtOh7bv8OHzY8o7oqlDgHLUzekaWHyy4hKivtTDIIcccu4+4RFjH/UzLs0+9zvEloC8ShiwJyx9wxh0W4NPeWai/tLnc6dIHWYdPqjS/A=",
"transport_ca_cert" : "MIIFXDCCA0SgAwIBAgIVAOmLUvmARZHv9rK94HYPQrHyGfMIMA0GCSqGSIb3DQEBCwUAMDwxOjA4BgNVBAMTMUVsYXN0aWNzZWFyY2ggc2VjdXJpdHkgYXV0by1jb25maWd1cmF0aW9uIEhUVFAgQ0EwIBcNMjUwNDI3MTEzMDM4WhgPMjEyNDA0MDMxMTMwMzhaMDwxOjA4BgNVBAMTMUVsYXN0aWNzZWFyY2ggc2VjdXJpdHkgYXV0by1jb25maWd1cmF0aW9uIEhUVFAgQ0EwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQDQJxfGsfJmpVkkVBGYkfRcninY6bhFNrEFMmR6zKcmodoWm7kA/74rKyAkzwq7dy3hj3lqXmYrdpjwue/XNmnBTDholxO6XDdUH4SqSbWIIILVXlRq9p5eQ7VXyRyB7SzqMhM0fdGuY48tza0WNcto3LR6nnVdu9IdG4v77ozzbo0HlQcF7qLDRUKbWXv3HuNbisdzh4gHbwOoBeJSJpwzd7Aa9ps4qtkxY9qGoQvfZdc7INp7dFHOL40vDyqzNyOIOABRnGB/qy7zcdj7uB0Sjp9VuodVUxmlhx3UXiArRp4g94Whito1Rhd/Qt5GtIwbAG7qJRvYnCif/KytXq0iphsXRe5S/BP5DPHrqJFKQ2ihdgspgPgzJnGoENjZoK4y6C2/DMrO8tOyfZ9r76dC6H4ak+OQ65jS1tHWtqwdV5ZtDpU42FuzKjyKxyqhAqG1zME5rrwMiaLCTdh/eLRQ3Zwd94Jd6WPB83hSAkZTqt2o5cfnoVTH/OeNrAsbVrc9uy2+7sutWhjCJC1fQV4AkpBYuUMUxOLpyBMeR1/kBH/AqO2U7ChJpG1qORybfoOJF7PobVUDws19xFKPVjYOfnGprm8rDUryyeZVYGJGZGdGH/s80pB/+3XwXVt8Zx/pb7UR7WkYepA5WjEyVYizoUTjhEccyYyFGKyY/DzthQIDAQABo1MwUTAdBgNVHQ4EFgQU6msC0Hq2DMg9jyCYApb7iUHl1PcwHwYDVR0jBBgwFoAU6msC0Hq2DMg9jyCYApb7iUHl1PcwDwYDVR0TAQH/BAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAgEAyIqp007gJTPYfzbJeWyX3R7j5CqIaOCKT++4/rk8KaE7WHUAIUJsSkuT+apYBokC5YUfEl+IparYh370pEVudBeoa71Mb4yPz1QtBpkkHurHwxuLSXwyjHG9eqvdFeKZG6M2tcg20lAp5iNAoFK7ch1WRo08Q9pn4WVjlT7DImbFT1WdNdEFU3lf1eLl3DCuPgXaP5wWowMKZjqXFJMZLQysDS54wLM3jLtt4gEZq0TbIL70Guj4XE24zAFJyV4A9uGECui/elqSZRjWSQSv4knE1m7wY8fUeaecENUTMNx64trEUPtyjFoSN7lwdOuMg6izYdAMP25Em2+HlwQsRmT7oTOHoB7b9g/Df7es2B5C+NrKZQBNB4KrJfQhzSY3VL5k3vjWrkWVxzTXjlV4/pMYAmucuooQQ9rENJciY5KCwOGyuNpxrKeymOK03PMW+GRWcT2IG4IEIgVob5Cj/Du1+ugFRx8mBXH1JyVHU7h221WD4ZOqkyc6R2eiybN4ODjLehQXBmtG3Jb3YdFPWxAaF7GhDSESFIyw9NJHBKX24gHvW0nJiUL8OkKK+Drl0x0YsplOl4HJSVHBcWsp0TlOIjMjfZRgaerwtEL8+9IdcyHS5Gh05v5bPITRJBeRvVPvRPKq/0fM/rc+6mum6BXikc4hYpjwJirbihtKC3s=",
"transport_key" : "MIIJQgIBADANBgkqhkiG9w0BAQEFAASCCSwwggkoAgEAAoICAQC0M56cXUGFI47kVnsRiMS/JMJIfixzouUqMcAOteVC6lUiKX/jd7aF1hrvhfSccIuuh+gZLEPPTYorxZ1rS1zap7/n+l1l5QzGsSbGh3N5y7IBVSH/ZEwLMnVcYk1Ucn9RSgjbCCEgH8KQ0CboeG93zZOfptXTenLa06AMxa+a5yAvJ0/awGZMNAFxw57IE8t5UXTBGaV8rWSonOCo7YAwkB96JbGT/qdp/9EC2sul6jhVIz7RQOP02GNC0u3ioWgaLtznwRx/ZAtB9u7z7DVy4Q/Dgyw0mMeJhQGwtRBTl5hpXUIGfmexZr4d1DpwVChjcWDOAvqOzaL5sDqpJAwHesetdB7aPYG0p2OL8UWIAPpSZITRmyKJKgUefoTXJOzaNaaGFxrkvfGjz9hqnWcAohQTONo5pu4mEOsPLTuM9fdutc8STsjD0rF24u6yNzL+2y+M2cTvBEpaMlOVYPg6joExhGGsmofi0aGMD2Wx/yQot0/F8HDu1PlBqiyhSP1+A/oF4S9AcRwYylRNvCbvBJnczQ75sqjLMmyGzwHpKj1Sj9vxJ6TmIIfJRKNCQ/t7c/XRN2Au6ozXM9Gozh1hRz7vb/AqlkfXaQ8eNyQqViZ68ayXpBBzGCLd/K5Nsl395IL9gUYoGfpVHdgk0/xd7tua7hphCdEiEcpz0IuPFwIDAQABAoICAFhNdTtVPV6TCpncwOjaNuMdjbC9V2YA3EA9acormyEbuEUJxff+uSvDO6bE4xVn1yLayULQnJfwfLUvXJDvuJwTGKiB2JOuVoDk3UXqqQz1YpBhmcvO566S9syjBRbNFpOMbmN6i9T0i0CwJ2jCX4j67YyN0fkT50VQ73xiwMbs3CoPHwPoRRixYUXQoLWVcZ9M6aQYP64lP46adUKCQ09IyIvv/aEyZ/I6376A3cs7R4h8wXMD09Cu2pEr0FM+Wcd+0FQmnqLUMOoNqZFQ3e4h2DorxJUvipPaX/Yp+8qinhZ/kvOgrIpTePUgAtT5gGMec1vwPLyOiv7EuzIYDmYmBwOd9wsV9SAI0OAm4At9Z4eZHUetSD5UT+SYZBQKAGKnbZwJVy0D+EZWDABcL3Ee/6Ij9wsDhDg5/YXpeTPg5KDkzBzDMxqqWVsokykj8lecwZpjvuFmIVAZ0SjsNg5OLkycNvYytbAoIG75i4AyOkvuxBeN0C8AzJHL3n+KxL+nkAxwB0YZf7Zh0lFF0e56Kn3ZHe0g1zc4yWQ1+ItgdPxU+7ufKixj8fauXTKrGnWnnUvDtCI8xMJ8QDJsqm7zodK68gQnOqNNcrehLUhJQbGhlakJzqdPaVaHss8izVUI8rZ2KT6U37DODz/BmJ4FsBrGSzyMgfTOXGUU1FnFAoIBAQDwu9YubsycSiJFndoJHLuIjuVS+ckglwlmhoyVVclLWJQKHjoFEP3XdpdSM/IT7m7KP+aQ//6iIkMnoa17gQXQyl8Sog0IDkrY8n3TrCZFELfyLQw4EMNtszcUjyijG4q+wOr0P7mBIp7OdFJBFvPWOypXe+gollRD2bYSAnJbotg4pYZSuKhsHU0nG7rchCYlO3QZflzbRKMx71pivmI9JELBWMcaRuhTNdTGu+i5gdefJkItWF3b5RJP8yz+xqUtqQVVazROon8oBF8rorD5V2LdiGuHWaIgJ5m6Ki9hrWnKlGCBi291iQFrDf+4WRJklFfR6PPb09xFInXkV6bLAoIBAQC/oRTtgosuNf1R25/u5MmCZ1xOrmqLFhw3JJCTztKvWtmvCCLRhx5683q711l7jOT/fitHoZv7uh2zAcI4jJ8RHaFH5wO+Rxgvr1AXvjfq6nXSUTc1JF8jw3f80KrDxu2xNSCQGJ6j2V0DLxUU2vib3OVJn3OJn2i3pxyIQ2jlyW6NrVMOG9yIPTumBXjJK64x+TT9VbGKihqeJuPpVY9L0/qG7pVD8gyYnSguJF8E1x2NGYn8Oo44nwfESN2W0nmeB42OfIkPilEz3nO/SYTB6EnvHpNKjnrL5uyEejQGTd54DaakNhDjv/77x6/tsPgdv+sximS2RQmQfm7cWyNlAoIBAEX5s7Znip5bhj7KNZi0e2akctB9vxL0FE1zCuZVu19lbhLK2n6ig8bft/izFMjqY9XyYSkA4JlscCzS+ESQKoqYaK5X4IXY8bTCOAURY0ZsJ6wDoCuj48q9b2NQgbV3ygrMtP8ujtOUpqjIcyhsbZB2PkaTw9YYbIIUhLITd/5fBY7hvnvJJVvX4Jdfnh9/Yj8sWwEWFBPAjdd0BcfyI0g2hBkDn0xqFBor6Z+i+3EA5xZh3pajuxSyL3KB8zBTuaveF6jOsrOw1Flje+6JKfwwhJJ0lx0O1uuV1z+gB4nTCI9UNgEx8MIvgvQFa7HAYnJOFID1v/zERqDp9W7cc6UCggEAN2gWyfmVzn2zyehLnOIv1XGXQfSyP5bb/6Gl3+bMCgGlsTMUBisgSAKAOTGx4MRDLhBH6UUz+Zu9nJVkl78o+uTrTgPglDKwLpFtAgBw4I58FJyA7u+eRpPs7H9U8Jhi/3rR+Hf13oApoZMKcGZDvaUn8pqU6HTb7UX3PProqJAOVA6KVij+IbI6ve0VGG11x2M7zAfr9pepzJKIEV44uX8ID92J8QRaWgIOrRC9HIQqOjLpL7Gqj3qD32AGGeavXV7nsgwWARVIF0w5gVR7eylzvxzDD6qUnMIfHsrKogv/yAyCYzZyubc4vYnmNz6U7t0f3soaBo19j3bPDQ2Y9QKCAQEAru4KypPemzHlsZ+nxDKzImCSQWyHc1cAJxlo6wYN6T78cVhedhz/psTvK6+TMo7icbzYCuDhdbETPm70ZEnX+UiNHxIYAb82fqsr1nd9DLpvtxnVfL9QSqbLNRqcTIMeBkiNxPkXnQvez5e/7AF5Hi9GISk16YNESuSsUk/ZNSUMDC2TyzSitFUDXV0tJDe+EoXjkJa2s54xJkxahWqeInxCZIXduEFDqk6rJMqFN1zNKNwC7RTWKPoeTCFh5jPrO2hykHFa0EKp4o3bCdlvtyUZgxsf15SEHNC7RpDmOl3H/J47ElQKtO8bg1rZex82AI8nQzI/RstrttomHB5WFg==",
"transport_cert" : "MIIFLjCCAxagAwIBAgIULN+3GdK/8aY3d9n1JyndJdeXe8cwDQYJKoZIhvcNAQELBQAwPDE6MDgGA1UEAxMxRWxhc3RpY3NlYXJjaCBzZWN1cml0eSBhdXRvLWNvbmZpZ3VyYXRpb24gSFRUUCBDQTAgFw0yNTA0MjcxMTMwNDBaGA8yMTI0MDQwMzExMzA0MFowFTETMBEGA1UEAxMKZWxrcHJpdmF0ZTCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBALQznpxdQYUjjuRWexGIxL8kwkh+LHOi5SoxwA615ULqVSIpf+N3toXWGu+F9Jxwi66H6BksQ89NiivFnWtLXNqnv+f6XWXlDMaxJsaHc3nLsgFVIf9kTAsydVxiTVRyf1FKCNsIISAfwpDQJuh4b3fNk5+m1dN6ctrToAzFr5rnIC8nT9rAZkw0AXHDnsgTy3lRdMEZpXytZKic4KjtgDCQH3olsZP+p2n/0QLay6XqOFUjPtFA4/TYY0LS7eKhaBou3OfBHH9kC0H27vPsNXLhD8ODLDSYx4mFAbC1EFOXmGldQgZ+Z7Fmvh3UOnBUKGNxYM4C+o7NovmwOqkkDAd6x610Hto9gbSnY4vxRYgA+lJkhNGbIokqBR5+hNck7No1poYXGuS98aPP2GqdZwCiFBM42jmm7iYQ6w8tO4z19261zxJOyMPSsXbi7rI3Mv7bL4zZxO8ESloyU5Vg+DqOgTGEYayah+LRoYwPZbH/JCi3T8XwcO7U+UGqLKFI/X4D+gXhL0BxHBjKVE28Ju8EmdzNDvmyqMsybIbPAekqPVKP2/EnpOYgh8lEo0JD+3tz9dE3YC7qjNcz0ajOHWFHPu9v8CqWR9dpDx43JCpWJnrxrJekEHMYIt38rk2yXf3kgv2BRigZ+lUd2CTT/F3u25ruGmEJ0SIRynPQi48XAgMBAAGjTTBLMB0GA1UdDgQWBBS2CfV9Ap2sdJdsZic/Zuu9w6ItAzAfBgNVHSMEGDAWgBTqawLQerYMyD2PIJgClvuJQeXU9zAJBgNVHRMEAjAAMA0GCSqGSIb3DQEBCwUAA4ICAQBmnUZFTaYKnHLwDh5ubuaXEt8rTcYfK4STHV3YZVNymg2P3YWsD/q/CrFLqc12W3NzkdnrsQY0WEkzwOBCyzZWC2jK/VGBRmBm5OZ5jORQhXE5w+UVHN94HqVG0gs0XTdwmVgP4yTlZ+sNf96LQZmQaQhBW5WlsfLEiemDn6Gsd+FJ0aotQo/nOOLELQoO0l5vEKmGmTfK/Zb1+gI0ehvdOzpjdz8RBB8sybd1tWjneZK5Vw9i7pokoeJoT4fSL6m8/9vW8EJnnhi6AtBCpLZAct5SVLYXxcEduI0hip26h8NbE9qeusGHyrcRynbypZ0UyBDJyYTH1M5uZV2/Od9QAHRJqvroTykXHPlLiXeLlZmLDh9LKXpUCFsW1R/V9OhJzN5XRHq4SoiAW2/lTlMN+NRds9nbz5VrNIHDtthChpgZK3yNVSeysALLXKYyKesbHDPDTxQCTO7TSV9dANla73yX5epp0Bx9vtBLOkZj87iDOhmBfdFXfs4MAJDEalThoxfwVf0VPj6YpNPkbkQKrrHWHm2L6JRwxpRYzrNx8RELaft/dc/GcYjnGlY3SXlJPpX9K6tBpk6OUEyM4Q+QK9YZYfs5Uzu+Y3ZcDkVL8GJna9pNrcpm2PGcQ7JY/C1W4fVWSN2jjfl7m7rdZFEb0AKa/MCdSPObYovbHDclow==",
"nodes_addresses" : [
"172.31.0.2:9300",
"172.31.0.6:9300",
"172.31.0.5:9300",
"172.31.0.4:9300",
"172.31.0.3:9300"
]
}
Response objects:
- http_ca_key
- http_ca_cert
- transport_ca_cert
- transport_key
- transport_cert
- nodes_addresses
3.2.Start
below will have xpack.security.enrollment.enabled: true
docker run --rm \
--name es01 \
-d \
-e ES_JAVA_OPTS="-Xms2g -Xmx2g" \
-e "cluster.name=private-cluster" \
-e "node.name=es01" \
-p 9201:9200 \
--network elkai \
docker.elastic.co/elasticsearch/elasticsearch:8.15.2
docker exec -it es01 bash -c "(mkfifo pipe1); ( (elasticsearch-reset-password -u elastic -i < pipe1) & ( echo $'y\n123456\n123456' > pipe1) );sleep 5;rm pipe1"
token=`docker exec -it es01 elasticsearch-create-enrollment-token -s node | tr -d '\r\n'`
You don’t need to manually copy certificates or key files if you’re using the enrollment token. The process will automatically handle the certificates and key configuration for you.
for i in {2..6}; do
docker run --rm -d \
--name es0$i \
--net elkai \
-e "node.name=es0$i" \
-e "cluster.name=private-cluster" \
-e ENROLLMENT_TOKEN="$token" \
-e ES_JAVA_OPTS="-Xms2g -Xmx2g" \
docker.elastic.co/elasticsearch/elasticsearch:8.15.2
done
http_ca.crt
The CA certificate that is used to sign the certificates for the HTTP layer of this Elasticsearch cluster.
http.p12
Keystore that contains the key and certificate for the HTTP layer for this node.
transport.p12
Keystore that contains the key and certificate for the transport layer for all the nodes in your cluster.
http.p12 and transport.p12 are password-protected PKCS#12 keystores
By running below command you can inspect that all are password protected with credentials stored in keystore
for x in es0{1..6} ;docker exec -i $x elasticsearch-keystore list
You can check fingerprint of CA cert
for x in es0{1..6} ;docker exec -i $x openssl x509 -in /usr/share/elasticsearch/config/certs/http_ca.crt -noout -fingerprint -sha256
Http cert
for x in es0{1..6} ;docker exec -i $x bash -c '
PASS=$(elasticsearch-keystore show xpack.security.http.ssl.keystore.secure_password) && \
openssl pkcs12 -in /usr/share/elasticsearch/config/certs/http.p12 -passin pass:$PASS -clcerts -nokeys -nodes | \
openssl x509 -noout -fingerprint -sha256'
Transport cert
for x in es0{1..6} ;docker exec -i $x bash -c '
PASS=$(elasticsearch-keystore show xpack.security.transport.ssl.keystore.secure_password) && \
openssl pkcs12 -in /usr/share/elasticsearch/config/certs/transport.p12 -passin pass:$PASS -clcerts -nokeys -nodes | \
openssl x509 -noout -fingerprint -sha256'
Above executions gave exactly same finger prints for respective certs across nodes.
You can check details about certs via API
for x in es0{1..6} ;docker exec -i $x curl -k -u elastic:123456 "https://localhost:9200/_ssl/certificates?filter_path=path,expiry&pretty"
4. Manually copying certs
4.1. Theory
As you have seen in previous example all certificates are same indeed. They cannot be used in full certification mode when Elasticsearch is checking hostname inside Subject Alternative Name (SAN). Therefore alternatively you can reuse them after first node is bootstrapped.
4.2. Start
docker run --rm \
--name es01 \
-d \
-e ES_JAVA_OPTS="-Xms2g -Xmx2g" \
-e "cluster.name=private-cluster" \
-e "node.name=es01" \
-p 9201:9200 \
--network elkai \
docker.elastic.co/elasticsearch/elasticsearch:8.15.2
set password for elastic user
docker exec -it es01 bash -c "(mkfifo pipe1); ( (elasticsearch-reset-password -u elastic -i < pipe1) & ( echo $'y\n123456\n123456' > pipe1) );sleep 5;rm pipe1"
Now simply copy config/certs content for later
docker cp es01:/usr/share/elasticsearch/config/certs ./es01-certs
Autoconfig makes keystores password protected so you have to save passwords for later provisioning
HTTP_PASS=$(docker exec es01 elasticsearch-keystore show xpack.security.http.ssl.keystore.secure_password)
TRANSPORT_KEY_PASS=$(docker exec es01 elasticsearch-keystore show xpack.security.transport.ssl.keystore.secure_password)
Now in the loop run rest of the nodes
for i in {2..6}; do
docker run --rm -d \
--name es0$i \
--network elkai \
-e ES_JAVA_OPTS="-Xms2g -Xmx2g" \
-e "node.name=es0$i" \
-e "cluster.name=private-cluster" \
-e "discovery.seed_hosts=es01" \
-e "xpack.security.enabled=true" \
-e "xpack.security.enrollment.enabled=true" \
-e "xpack.security.http.ssl.enabled=true" \
-e "xpack.security.transport.ssl.enabled=true" \
-e "xpack.security.transport.ssl.verification_mode=certificate" \
-e "xpack.security.transport.ssl.keystore.path=certs/transport.p12" \
-e "xpack.security.transport.ssl.truststore.path=certs/transport.p12" \
-e "xpack.security.http.ssl.keystore.path=certs/http.p12" \
-e "xpack.security.http.ssl.certificate_authorities=certs/http_ca.crt" \
-e "xpack.security.transport.ssl.keystore.password=$TRANSPORT_KEY_PASS" \
-e "xpack.security.transport.ssl.truststore.password=$TRANSPORT_KEY_PASS" \
-e "xpack.security.http.ssl.keystore.password=$HTTP_PASS" \
-e "xpack.security.http.ssl.truststore.password=$HTTP_PASS" \
-v ./es01-certs:/usr/share/elasticsearch/config/certs \
docker.elastic.co/elasticsearch/elasticsearch:8.15.2
done
4.3. Examine
Check CA fingerprint
for x in es0{1..6} ;docker exec -i $x openssl x509 -in /usr/share/elasticsearch/config/certs/http_ca.crt -noout -fingerprint -sha256
Check http cert fingerprint
for x in es0{1..6} ;docker exec -i $x bash -c "cat /usr/share/elasticsearch/config/certs/http.p12 | \
openssl pkcs12 -passin pass:$HTTP_PASS -clcerts -nokeys -nodes | \
openssl x509 -noout -fingerprint -sha256"
Check transport cert fingerprint
for x in es0{1..6} ;docker exec -i $x bash -c "cat /usr/share/elasticsearch/config/certs/transport.p12 | \
openssl pkcs12 -passin pass:$TRANSPORT_KEY_PASS -clcerts -nokeys -nodes | \
openssl x509 -noout -fingerprint -sha256"
Above executions gave exactly same finger prints for respective certs across nodes.
5. Summary
In this long knowledge article you have learn how to use enroll node API with Elasticsearch for your custom deployment. You have also explore how this API is used internally by Elasticsearch to allow smooth enrollment of new and new nodes.
Have a nice coding!
