Using RabbitMQ in building the system of leads generation
A small entry for a start:
RabbitMQ is a queue server which allows different applications to interact with each other according to AMQP protocol, widely used by such corporations as Google, Microsoft, NASA, Red Hat, VMware, AT&T.
RabbitMQ server is employed for building service architecture when the following selected services process resource intensive and dormant tasks:
Xero – service for online bookkeeping
Secure Trading – service for payment processing
RabbitMQ is used to establish communication between servers; perhaps, it is one of the most efficient solutions for this purpose. In our company, we use this proven technology in different projects, for example, in messenger development.
In this post, we will tell you about using the RabbitMQ queue server in our project, specifically in building the system of leads generation and their sell to the clients.
Also, you can read our article about RabbitMQ usage in chat application and mobile messenger development.
The customer was a company, which had the following problem: with an increasing number of clients a lot of time was spent on financial servicing operations like invoice creation, payment request, payment verification and confirmation.
So a manager had to manually gather data on payments from CRM, make invoices in Xero based on this information, prepare bills for customers via Secure Trading, and only then send messages to clients with invoices and payments status. This process looked as follows:
What was our main task:
To automate processes of creating invoices via third-party Xero API and delivering invoices via email; make automatic debiting from user card via Secure Trading API.
Solution with cron
The first pancake was a failure. We created the script, which runs two times a week and made necessary actions within a single process.
This imposed constraints on the overall performance (long answer waits from API). Also, any mistake on one of the stages led to finishing work of the whole process and required a complete script restart.
So we came to a conclusion that we needed to divide the code into several independent processes with the ability of communication between them.
Solution – using RabbitMQ server
The task was divided into stages that in turn were transformed into independent processes:
1. Collecting information about invoices needed to be created on the Xero side.
2. Delivering invoices in Xero.
3. Delivering invoices to user emails.
4. Automated debiting from user card via Secure Trading.
5. Email with notification of debiting or inability of debiting (no attached card, insufficient means).
Such architecture offers an error-tolerant solution, which can be easily scaled (RabbitMQ allows processes to communicate between several servers).
Convenient monitoring tools for RabbitMQ enable to track errors and test each process separately, and using of several independent processes provides the ability to divide tasks and minimize the time of their processing.
Also an additional benefit was the possibility to move all resource intensive tasks (report generation, mailing) in postprocessing and reduce response wait time to the user.
RabbitMQ is a very flexible system that works on a standardized protocol AMQP basis, which allows completely different applications to communicate and exchange data: Web, Mobile, Desktop, Embedded.
In this case, RabbitMQ is an efficient solution for IoT system also, as it supports a lightweight communication protocol MQTT, which is widely used to exchange messages between devices.
For communication between services and frontend, we integrated RabbitMQ queue management program which helped us to move all complicated tasks to the background (mailing, report generation, etc.), and this greatly increased the speed of user interface.
The project was successfully deployed to Bitbucket and we used Git for source control.