Case Study
Building a Data Lake
Industry
-
Banking & Finance
(a major French commercial bank)
Challenge
-
Building a data lake for an instant payment project
Solution
- Design and implementation of the data lake architecture;
- Implementation of data pipelines using Hadoop ecosystem technologies;
- Development of the data storage layer using Kassandra;
- Setting up the data visualisation and analysis platform with ELK;
- Technical environment: the project was developed using the Hadoop, Java, Kassandra and ELK ecosystem. It was implemented in a hybrid cloud environment.
Technologies
-
Java
-
Springboot
-
ELK
-
HDFS
-
Kafka
-
Spark
-
Kassandra
Keys to success
- Mastery of the Hadoop ecosystem: our expertise in data processing and storage technologies, particularly Hadoop, enabled us to design and implement a reliable and scalable system for the bank's data lake;
- Use of hybrid cloud: we implemented a hybrid cloud architecture to enable the bank to deploy large-scale data processing and analysis services quickly and easily. This approach optimised costs while guaranteeing data security and compliance;
- Close collaboration with business teams: we worked closely with the bank's business teams to understand their data processing needs and goals. This collaboration resulted in the creation of a data lake tailored to the bank's needs in terms of performance, capacity and functionality, and facilitated the adoption of the solution by end users.