endorsedCompleted

Big Data Infrastructure Revamp and Cleanup for FinTech

Services Covered:

Big Data Analytics and Business Intelligence • Cloud Consulting and Services • Solution Architecture

Industry

Financial Services

Duration

4 months

Budget

10K - 49K

Start Date

1 October 2019

Team Size

1-5 Employees

Engagement Model

Onshore

Project Capability Score (PCS)

71/100

Good

Project Capability Score (PCS) estimates how capable Oneture Technologies is of successfully delivering a project like this, based on its past experience, track record, and ability to handle similar work. Learn more about PCS

Client

Confidential

New Delhi, India

Project Summary

Oneture Technologies restructured and archived large, organically grown data sets for a fintech company and helped finalize a new architecture with an AWS team. They built partitioning scripts, integrated Power BI, and improved query performance. The engagement delivered under a strict timeline and enabled significant reductions in query time and improved data access for product teams.

Key Challenges

Consolidate disparate data sources and provide historical access for product teams
Revamp infrastructure to handle large and growing data volumes using Hadoop
Lack of in-house data engineering resources and a tight rollout timeline

Project Deliverables

Structured and archived databases (including analysis of ~800+ tables)
Data partitioning scripts for regular updates
Architecture finalized with the AWS team
Power BI integration for reporting

Project Solution

Oneture analyzed and restructured disparate databases (including an ~800+ table database), archived legacy data, wrote scripts to partition data and automate updates, and integrated the data into Power BI for reporting. They spent initial weeks finalizing pipelines and architecture and collaborated with an AWS team to finalize the cloud architecture. The vendor prioritized open-source tools and custom scripting to meet the client's requirements.

Project Outcome

Reduced long-running MySQL query times from more than 15 minutes to 3.8 seconds
Delivered three of four planned deliverables within a strict timeline
Changed how teams access and view data, enabling improved data consumption
Generated opportunities for follow-on engagements

Platforms

Cloud

Tech Stack

Apache Hadoop
Amazon Web Services (AWS)
MySQL

Client Endorsement

Overall Review Rating

4.5

Timeliness

Cost Rating

Willing to Refer

Quality of Deliverables

“The team at Oneture Technologies met expectations. They delivered a customized solution within a strict timeline. Their work reduced query time significantly. Their openness spurred the opportunity for future engagements.”
Anonymous
Head Data Science
Date: 08/27/2020