Reenginnering of a ETL Banking process

In the context of one of the main Italian banks, the Digital Inside project was developed in order to satisfy the strengthening of the transmission channel of heterogeneous flows, from the Subsidiaries to the Parent Company and the automation of the normalization and standardization phases of the data, following the quality check phases on data previously loaded into the system.

Context

Digital innovation in the banking sector is essential for any bank to stay up-to-date. In the era of big data, it has become crucial to develop innovative solutions that can automate processes which would otherwise slow down the production chain of any large enterprise.

Within the context of one of the leading Italian banks, the Digital Inside project was developed to meet a dual purpose:

  • Strengthening the transmission channel from Subsidiaries to the Parent Company of heterogeneous flows.
  • Automating the stages of data normalization and standardization, following quality checks on the data previously loaded into the system

Solution

The data management and transmission process is divided into several phases:

  • Flow Upload : Subsidiary users have a FE page at their disposal for uploading files subject to a series of preparatory checks.
  • Multi Data Layer: The data is stored in an intermediate staging layer reflecting the original flow structure.
  • Data quality: Alongside normalization, control rules are applied to generate potential alerts.
  • Monitoring and Modification of normalized tracks and, analysis through BI tools.

The solution adopts a microservices architecture (Java Spring) with a PostgreSQL and MongoDB DB component and an Angular FE layer.

  • Next Innovative Steps

    Upcoming developments include innovative initiatives leveraging the integration of AI and NLP to enhance the ingestion platform's efficiency and user experience

    In this context, the following use cases have been identified:

    - Creation of reports for clustering and consultation of normalized traces via natural language, thanks to the use of NLP-based RAG techniques.

    - Assistance with application settings through voice or text interaction with a chatbot.

    - Optimization of data quality processes using AI techniques that detect data inconsistencies through user interaction.

    - Assistance in monitoring flows by integrating information on reminders to Subsidiaries.

    strip-0

Conclusions

Previously, incoming practices were manually managed , resulting in a high risk of human error. The introduced automation has significantly reduced these errors and optimized the production of BI reports as complete as possible.

This digital transformation represents a crucial step, ensuring more informed and timely decisions and laying a solid foundation for a technologically advanced future in the banking sector, thanks also to future evaluations on AI integration, aimed at optimizing UX and making processes related to the platform more secure.