RabbitMQ with Java, Spring and Docker, asynchronous communication between microservices

In this article we will go over RabbitMQ practical implementation and try to look to the theoretical concepts meanwhile. We will be using docker and docker-compose so you can follow with any Operational System.

Project on Github:

Domain

We will be sendings user informations from a producer microservice to a consumer microservice, for simplicity.

Projects setup

Let’s create them by going to start.spring.io

producer-microservice

consumer-microservice

Both projects need the following extra dependencies inside pom.xml file.

on producer

on consumer

RabbitMQ docker setup

On the root of one of the project, it doesn’t matter wich one(or both), let’s create a docker-compose.yml file, containing the required configurations to run RabbitMQ in a docker container.

container_name: can be any name, the name of our docker container

image: oficial rabbitmq docker image to be pulled

ports: exposed ports that will be used to access rabbitmq broker and rabbitmq console manager

Now go to the root directory of one of the microservices, and run

docker-compose up

It will run the rabbitmq container, that can be verified running

docker ps

RabbitMQ management UI

Go to localhost:15672

We will log in to RabbitMQ user interface. By default rabbit uses “guest” as username and password, so go ahead and log in with them.

Go ahead, take a look at the UI. As I said earlier, it isn’t the purpose of this article to detail this interface, I only went this far without coding because we will be using it while we develop our projects.

Messaging concepts

What is message and why we need it ?

• SOAP protocol has message, header and body
• asynchronous, different for a synchronous http request
• example:
• customers messages, orders coming in a huge load
• we can process them one by one
• we use advanced messaging to encapsulate any kind of information
• queues, topics, channels, exchanges, depends on the protocol definition

How does this message goes from the producer to consumer ?

Exchanges

• first to receive the message
• routes a message to one or more queue
• routing algorithms depends on exchange types and “bidings”
• “biding” is nothing more than a configuration to bind exchanges to queues
• types:
• direct = empty string and amq.direct
• fanout = amq.fanout
• topic = amq.topic (user purchased, for example)
• headers = amq.match and amq.headers

Queues

• messages arrive from exchanges to queues
• where messages go before reaching a subscriber
• properties:
• name: name of the queue
• durable: either to persist to disk or not
• exclusive: delete the queue if not used anymore
• auto-delete: delete the queue when consumer unsubscribes

Bidings

• rules that exchanges use to route messages to queues
• from exchange E to queue K, K has to be bound to E
• may have an optional routing key attribute used by some exchange types
• routing_key acts like a filter
• imagine something like:
• queue is a destination city
• exchanges are like the airport of this city
• bidings are the routes to arrive from the airport to the city
• there can be zero or many routes possible to reach it

Producer Microservice

producer-microservice architecture

1. Connect to broker with username, password and host.
2. Create JsonConverter bean, to convert json objects to be sent as messages
3. Create RabbitTemplate bean, to make the conversion of the given object and send it to the broker

environment variables

resources/application.yml

• message: hard coded response value
• host: host to connect(previously localhost…)
• port: host port
• password: default password as mentioned
• username: default username as mentioned
• exchange: exchange name to send messages
• queue: queue name to send messages
• routingKey: routing key to bing exchange to queue

config/RabbitMQConfig.java

• it will handle creating our connection
• and will create the RabbitTemplate and MessageConverter beans

domain/User.java

controller/ProducerController.java

service/ProducerService.java

Once the user object arrives to the service layer, we need a RabbitTemplate as shown in the diagram to

1. convert the object into a proper message.
2. inform exchange name and routing key to wich the message will be sent.

If we run the project, and send a request to our producer end-point

Our application will throw

RabbitMQ docker container will throw

Because we didn’t create the exchange to send the message, neither the queue to recieve the message.

Both of them will be created as Spring Beans as our other configurations. It can be done either on consumer or producer, and the reason is because we could do it manually in the management console, but we don’t want to be doing it manually. So let’s do it dinamically.

Consumer Microservice

consumer-microservice architecture

1. Create Queue bean
2. Create Exchange bean
3. Create binding bean
4. Connect to broker with username, password and host.
5. Create JsonConverter bean, to convert json objects to be sent as messages
6. Create RabbitTemplate bean, to make the conversion of the given object and send it to the broker

environment variables and database config

resources/application.yml

• queue: different than before here we provide the queue name that we consume messages from

resources/application.properties

• h2 database and spring data jpa configs

config/RabbitMQConfig.java

• it will create the queue(s), exchange(s) and it’s bindings that we need

domain/User.java

• user entity to be persisted in database

repositories/UserRepository.java

• repository responsible for persisting user entity to database

service/ConsumerService.java

• here we will listen to the queue
• convert the messages
• persist them to database

Running this application we will be able to see in the RabbitMQ management interface that the exchange and queue were created, and correctly binded.

user.exchange

user.queue

Exchange and queue correctly binded with the provided routing key

Running the Microservices

Sending our User payload now, will trigger the whole messaging proccess

Then it will be handled on ProducerController from producer-microservice

And consumed on ConsumerService from consumer-microservice

And then persisted on database, that can be accessed by going to localhost:8080/h2-console, press ok and click in the user table

Conclusion

The main idea here was to make a project to implement two microservices communicating through RabbitMQ, so you can implement this solution inside a much more complex system.

If this article helped you in some way, consider giving it a clap.

Project on Github:

Find me on: Github profile | Linkedin