Learn with me: Consul – A Service Networking Solution

Pre-requisites

• Basic understanding of differences between Monolithic and Microservices architecture
• Comfortable in using docker and docker-compose container tools
• Microservices used in this article are written in python

The code used for this demo can be cloned from this github repo.

What we cover?

The main aim of this article is to do the end-to-end setup of a mircroservice cluster registered with Consul. So, I’ll cover:

• Consul installation
• Basic Consul commands
• Service discovery – running two microservices and registering them with Consul
• Using Consul DNS service to resolve the service names

Note: All the setup is for demo and dev purpose only, don’t use any of the config discussed on production without digging more

I’ll not be explaining each and every component of Consul instead I’ll include Consul documentation links wherever I find is necessary. Let’s start mining gold and get our hands dirty 😉

What is Consul?

Consul is service networking solution which solves the major problems such as common configuration, service discovery, complex firewall rules and segmentation introduced while moving from monolithic to microservices architecture. Below features of Consul and how they address the newly introduced problems:

• Configuration solved by Key/Value store feature
• Segmentation & Complex networking solved by Service Mesh
• Service Discovery solved by Consul Connect

The video below gives a good introduction of these features

Installation

Consul can be installed in many ways as mentioned in this documentation. In this article, I’ll be using docker. It is easier and faster to setup using docker.

Follow the below steps:

$ git clone https://github.com/umessh/consul_example.git
$ cd consul_example
$ docker-compose build
$ docker-compose start consul_server

After running the above commands, the docker container with Consul will start running. To get inside the container use the below command:

$ docker exec -it consul_examples_customer_service_1 sh

Now, you will be able to run and see what Consul components/services are running inside the container. Before proceeding further, I suggest going through the glossary doc, getting started tutorials and CLI commands doc.

Below are few commands, you might want to try to see what’s going on inside the Consul container:

$ ps

$ netstat -ntlp

$ consul members
$ consul catalog services
$ consul catalog datacenters
$ consul reload
$ consul services register 

You can also access the Consul UI, by opening the http://localhost:8500/ui link in the browser.

Note: To proceed further you might want to stop the running Consul container, because in the next section we will be starting all the containers including the microservices. Use docker-compose stop to the consul container

Demo

I intentionally didn’t include few config steps in the Dockerfile and docker-compose file, so that we can go through them by manually executing the missing steps. They will give better understanding of how the Consul works. They also make you feel bit comfortable with the troubleshooting.

There are two microservices built using python-flask in the source repo under services folder. One is customer and other is orders. They are just stubs don’t have any implementation. We will use these two services and register them in Consul and configure the Consul DNS. That will enable the communication between the two services without knowing where each of those services are deployed.

Let’s first start the container cluster using the below command:

$ docker-compose up

Once the cluster is up and running, you can test whether everything is running as expected by opening the below urls in the browser:

• Customer service – http://localhost:5001
• Orders service – http://localhost:5002
• Consul UI – http://localhost:8500/ui

We need to capture the IP addresses of the containers running Customer service, Orders service and Consul. Use the below commands to capture the IPs

$ docker inspect -f \
'{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' consul_examples_customer_service_1
$ docker inspect -f \
'{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' consul_examples_orders_service_1
$ docker inspect -f \
'{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' consul_examples_consul_server_1

Now, let’s register Customer and Orders service with Consul. Both the service definitions are included in the repo consul->config folder. Use the service definitions and replace the IP addresses captured in the above step. Follow the below steps for service registration:

$ docker exec -it consul_examples_consul_server_1 sh
# cd /consul/config
# vi customer.json 
# vi order.json 
# consul reload

# consul catalog services

# apk update && apk add bind-tools
# dig @127.0.0.1 -p 8600 customer.service.consul
# dig @127.0.0.1 -p 8600 order.service.consul

$ docker exec -it consul_examples_customer_service_1 sh
# dig orders.service.consul

# vi /etc/resolv.conf 
# vi /etc/dnsmasq.d/consul 
server=/consul/#8600
# dnsmasq -qRh --config-dir=/etc/dnsmasq.d/
dnsmasq service will be started.. now test the dig
# dig orders.service.consul

Run the same steps on orders service container to access the customer service container.

With this we completed the goals listed under “What we cover?” section. In the next post, I’ll cover the Key/Value Store usage, auto registering, side car proxies, interactions and more advanced multi-server/client config.

Thanks for reading and Happy coding!! 😀