How to launch
a technology
product with
100K+ RPS throughput

Initially, the VK Process Mining process analytics platform appeared as a custom solution for a specific project. None of the existing products met the customer’s requirements, so the development of the solution was delegated to VK. After the project was implemented, VK Process Mining became part of the VK Digital Technologies product portfolio.

One of the tasks that the developers faced from the very beginning was horizontal scaling of the solution. When implementing a platform, customers often start with one process, such as procurement analytics or logistics, and then expand the solution to others.

As the project progresses, it is necessary to break down processes into different data models over different periods and observe dynamic changes over time. At the same time, corporate information systems constantly generate new data that also needs to be analyzed. Therefore, VK Process Mining needed a cluster solution that would meet the following requirements:

• Horizontal scaling
• Processing large amounts of data
• Analytics based on thousands of attributes from various systems
• Real-time data loading

Architectural
solutions

VK Process Mining has a classic web application with frontend, backend, and a data storage. Tarantool works as a web server, and provides high performance. It can withstand a load of hundreds of thousands of RPS. Additionally, there is an HTTP balancer between the system and the user.

The solution uses a hybrid storage. The main data with ready-made chains of events is stored in RAM for quick access. Tarantool saves the data pumped through ETL into a disk storage system.