TL;DR

We use a Docker cluster to launch an Elasticsearch instance as a global service. This will scale our Elasticsearch instance with each machine we add to the cluster. We will set a certain parameters in the Elasticsearch container to launch an Elasticsearch cluster within our Docker cluster.

Context

We created a Docker cluster earlier. We even created an Elasticsearch instance on that cluster. We will be removing that deployment and creating a new instance with clustering functionality.

Removing the current instance

To remove the existing Elasticsearch instance we will be using the following command:

docker service rm elasticsearch

Creating a clustered Elasticsearch instance

To create a new Elasticsearch instance with clustering we will be using the following command:

docker service create \
    --mode global \
    --name elasticsearch \
    --network log_net \
    --mount "type=volume,source=elasticsearch_index,destination=/usr/share/elasticsearch/data" \
    --mount "type=volume,source=elasticsearch_config,destination=/usr/share/elasticsearch/config" \
    --env "cluster.name=docker-cluster" \
    --env "bootstrap.memory_lock=true" \
    --env "ES_JAVA_OPTS=-Xms1g -Xmx1g" \
    --endpoint-mode "dnsrr" \
    --restart-condition any \
    elasticsearch:5-alpine \
    -E "transport.host=0.0.0.0" \
    -E "discovery.zen.ping.unicast.hosts=elasticsearch" \
    -E "discovery.zen.minimum_master_nodes=1"

Here we are adding a few environment variables to tell Elasticsearch that we want to create a cluster. The --mode global option is used to make the instace scalable with the cluster. Another important parameter is endpoint-mode "dnsrr" witch change the container networking mode to DNS round robin. The container parameters after the image configures Elasticsearch to work in cluster mode.

Verification

We can confirm our Elasticsearch cluster by examining the logs of the container. By following these commands we get the logs:

docker service ls # to list the services

docker ps # to list the container on the host

docker container logs <container name> | grep master # to show the logs

#output of the above command
[2017-05-01T14:56:08,806][INFO ][o.e.c.s.ClusterService   ] [T9mVqck] new_master {T9mVqck}{T9mVqcktQ0mb_z7VO18X_w}{u1y7v6lTQYmHCkMHDiQhTQ}{10.0.1.22}{10.0.1.22:9300}, reason: zen-disco-elected-as-master ([0] nodes joined)

We can see the new_master message and nodes joined message that confirms Elasticsearch is working in cluster mode.