endorsedCompleted

Data Engineering for SaaS Platform

Services Covered:

AI and Machine Learning

Industry

Information Technology

Duration

7 months

Budget

10K - 49K

Start Date

1 April 2020

Team Size

1-5 Employees

Engagement Model

Nearshore

Project Capability Score (PCS)

75/100

Good

Project Capability Score (PCS) estimates how capable Geniusee is of successfully delivering a project like this, based on its past experience, track record, and ability to handle similar work. Learn more about PCS

Client

Factmata

London, United Kingdom

Project Summary

Geniusee delivered data engineering and MLOps work for a SaaS platform, using Kubeflow and Kubernetes and improving the client's AWS stack. The engagement included daily meetings, platform scoping, and established reporting and QA processes. As a result, the client's data engineering system now handles multiple social data types and integrates with other social listening tools via custom schemas.

Key Challenges

Reduce AWS platform costs
Ensure the data pipeline was able to process millions of documents per day through a natural language pipeline

Project Deliverables

Data engineering pipeline capable of processing multiple data types
Kubeflow/Kubernetes-based MLOps components and orchestration
Improvements and management of AWS stack including Lambda functions
API integration with other social listening tools using a custom schema

Project Solution

Geniusee was selected via YouTeam for their data engineering and Python/MLOps expertise. They participated in daily standups and collaborative planning to spec the platform and roadmap. The team implemented Kubeflow- and Kubernetes-based data engineering components, set processes for check-ins, reporting, and user QA, and provided dedicated staff (1 DevOps architect and 1 data engineer) to manage the work.

Project Outcome

Our data engineering system can now handle 8 different data types including Reddit and Twitter
Integration via API to other social listening tools using a custom schema
Very effective reporting and daily sprint management between teams

Platforms

SaaS
Web

Tech Stack

Python
Amazon Web Services (AWS)

Client Endorsement

Overall Review Rating

4.88

Timeliness

Cost Rating

Willing to Refer

Quality of Deliverables

“Thanks to Geniusee, the client's data engineering system can now handle multiple data types. The team's workflow was effective in reporting and daily management of Sprints.”
Dhruv Ghulati
CEO
Date: 03/18/2021