Table of Contents
1. Introduction
Cannot find example docker project with Kibana and Elasticsearch running together fully encrypted?
There are already projects with Elasticsearch running with TLS enabled so it is encrypting communication between Elasticsearch nodes and clients. Somehow Kibana is still run only with http.
I have solution for you. I prepared sample project that starts secure Kibana along with Elasticsearch. Here you are!
2. Project structure
I have prepared ready to use project specially for you. Just download, extract and run it. Simple, that’s it. But before you start let me explain components.
2.1. What is included
This is docker compose based project that orbits around compose definition in yaml file format “compose.yml”. Two little wrappers in shell script to start and stop cluster using additional flags for docker compose up/down commands to let you avoid creating ad hoc unix aliases and make it compact.
Both Elastic and Kibana nodes got external configuration file, mainly to organize space in compose definition file but also to separate logic. These two are included in self descriptive catalogs elkConfig01 and kibanaConfig01.
You can also see one extra yaml config fileelasticsearch-certutil-instances.yml. This is for certificate creation by Elastic CA. Not widely recognize by web browser but for develepment purpose it is enough. In PROD you will likely use your corporate CA or certbot. Same elastic image is used to run this utility.
There is also file containing environment variables like elk stack version, ports etc. called “env_variables.sh”
Finally you got special files with secrets where you keeping passwords.
In the following sections I will describe for you each of file.
elk-kibana-secure
- compose.yml
- stop_docker_compose_cluster.sh
- start_docker_compose_cluster.sh
- elkConfig01
- elasticsearch.yml
- kibanaConfig01
- kibana.yml
- elasticsearch-certutil-instances
- elasticsearch-certutil-instances.yml
- env_variables.sh
- secrets
- kibana_system_pass2.txt
- forcurl.txt
- kibana_system_pass.txt
- elk_password.txt
2.1.1. docker compose file
I will start from most important compose.yml file. There are 5 services
installation_container:
volumes:
- certs:/usr/share/elasticsearch/config/certs
es01:
depends_on:
installation_container:
condition: service_healthy
passwordsetter:
depends_on:
es01:
condition: service_healthy
kibanakeystore:
depends_on:
passwordsetter:
condition: service_completed_successfully
kibana01:
depends_on:
kibanakeystore:
condition: service_completed_successfully
installation_container creating all certificates for Elasticsearch and Kibana and keeping them in volume certs.
es01 is Elasticsearch node and is starting as second, once it’s having GREEN cluster status then 3rd service can start
passwordsetter is 3rd one. it’s basically curl image that call elastic to set password for kibana_system user
kibanakeystore is 4th service and it’s sole role is to create kibana-keystore file and put password there that was setup in previous step.
kibana01 is last one. It is using kibana_system user to create internal kibana indexes like .kibana. Password is read from keystore.
Elastic and Kibana have their custom yml file.
configs:
elasticsearch-certutil-instances:
file: ${PWD}/elasticsearch-certutil-instances/elasticsearch-certutil-instances.yml
elk-config-yml-01:
file: ${PWD}/elkConfig01/elasticsearch.yml
kibana-config-yml-01:
file: ${PWD}/kibanaConfig01/kibana.yml
all password are kept as secrets and bind mounted under read-only location
secrets:
elk_password:
file: secrets/elk_password.txt
curl_password:
file: secrets/forcurl.txt
kibana_password:
file: secrets/kibana_system_pass.txt
kibana_password_2:
file: secrets/kibana_system_pass2.txt
2.1.2. Kibana config file
server.host: kibana01
server.publicBaseUrl: https://kibana01:5601
elasticsearch.hosts: https://es01:9200
server.ssl.enabled: true
server.ssl.certificate: /usr/share/kibana/certs/kibana01/kibana01.crt
server.ssl.key: /usr/share/kibana/certs/kibana01/kibana01.key
server.ssl.certificateAuthorities: /usr/share/kibana/certs/ca/ca.crt
elasticsearch.ssl.certificate: /usr/share/kibana/certs/es01/es01.crt
elasticsearch.ssl.key: /usr/share/kibana/certs/es01/es01.key
elasticsearch.ssl.certificateAuthorities: /usr/share/kibana/certs/ca/ca.crt
elasticsearch.ssl.verificationMode: full
all referenced certs and keys are created and placed in volume.
2.1.3. Elastic Config File
network.host: 172.16.238.10
node.name: es01
node.roles: data,master
cluster.initial_master_nodes: es01
discovery.seed_hosts: es01
bootstrap.memory_lock: true
xpack.security.enabled: true
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.key: certs/es01/es01.key
xpack.security.http.ssl.certificate: certs/es01/es01.crt
xpack.security.http.ssl.certificate_authorities: certs/ca/ca.crt
xpack.security.http.ssl.verification_mode: full
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.key: certs/es01/es01.key
xpack.security.transport.ssl.certificate: certs/es01/es01.crt
xpack.security.transport.ssl.certificate_authorities: certs/ca/ca.crt
xpack.security.transport.ssl.verification_mode: full
2.1.4. Elasticsearch certutil input file
instances:
- name: "es01"
dns:
- "es01"
- "localhost"
ip:
- "172.16.238.10"
- name: "kibana01"
dns:
- "kibana01"
- "localhost"
ip:
- "172.16.238.11"
2.1.5. Starting script
You can start and stop project using simple wrappers
stop_docker_compose_cluster.sh
start_docker_compose_cluster.sh
2.1.6. Environment variable file
Few variables you can noticed are setup in that file
COMPOSE_PROJECT_NAME=elastic-kibana-secure
CLUSTER_NAME=docker-compose-elk
STACK_VERSION=8.15.2
MEM_LIMIT=2g
ES_PORT=9200
KIBANA_PORT=5601
LICENSE=trial
2.2. Extract project
You have to run
curl -XGET -O https://toughcoding.net/WWoKjKhPdqNcDf7MHDcWVkA9/elk-kibana-secure.tar.bz2
tar -xpvjf elk-kibana-secure.tar.bz2
In docker compose secret files has to be with 400 or 600 access. Therefore I tried to preserve their permission settings.
3. Startup cluster
Run start command to have Elasticsearch and Kibana up and running
./start_docker_compose_cluster.sh
3.1. verify TLS connection with Kibana
download CA cert
docker cp elastic-kibana-secure-es01-1:/usr/share/elasticsearch/config/certs/ca/ca.crt ./http_ca.crt
connect to kibana
curl -vv -u elastic:123456 -XGET "https://localhost:5601/" --cacert ./http_ca.crt
response
* Trying [::1]:5601...
* Connected to localhost (::1) port 5601
* ALPN: curl offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* CAfile: ./http_ca.crt
* CApath: none
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN: server accepted http/1.1
* Server certificate:
* subject: CN=kibana01
* start date: Jan 16 17:52:15 2025 GMT
* expire date: Jan 16 17:52:15 2028 GMT
* subjectAltName: host "localhost" matched cert's "localhost"
* issuer: CN=Elastic Certificate Tool Autogenerated CA
* SSL certificate verify ok.
* using HTTP/1.1
* Server auth using Basic with user 'elastic'
> GET / HTTP/1.1
> Host: localhost:5601
> Authorization: Basic ZWxhc3RpYzoxMjM0NTY=
> User-Agent: curl/8.4.0
> Accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
< HTTP/1.1 302 Found
As you can see you are connected with TLS 1.3 to Kibana. Good!
4. Summary
In this tutorial you have study structure of example docker compose project that let you start automatically Elasticsearch and Kibana. Both password protected with encrypted connection. Now you can enjoy secure Kibana.
Have a nice coding!
