endorsedOngoing

Data Collection Platform and NLP Components for Retail Data Provider

Services Covered:

Custom Software Development • AI and Machine Learning • Big Data Analytics and Business Intelligence

Industry

E-commerce & Retail

Duration

131 months

Budget

50K - 199K

Start Date

1 September 2015

Team Size

Confidential

Engagement Model

Offshore

Project Capability Score (PCS)

76/100

Good

Project Capability Score (PCS) estimates how capable Kindgeek is of successfully delivering a project like this, based on its past experience, track record, and ability to handle similar work. Learn more about PCS

Client

Datasembly

Washington, United States of America

Project Summary

KindGeek served as ongoing development partners to build and improve a large-scale data collection platform for a retail/CPG-focused client. They developed core backend components—product matching and brand extraction—worked in Scala and Akka, and integrated storage and aggregation using Amazon RedShift and RethinkDB. Their work supported business development efforts and improved internal processes and project management.

Key Challenges

Build out a data collection platform to collect and manage large volumes of pricing data from across the country
Develop product-matching capabilities for items lacking UPCs using NLP and large-scale search
Implement brand extraction to attribute product strings to manufacturers

Project Deliverables

Data collection and processing pipeline
Product-matching component (natural language processing/text analysis)
Brand extraction component
Backend services implemented in Scala (concurrency/distributed systems with Akka)
Data storage and aggregation using Amazon RedShift and RethinkDB

Project Solution

KindGeek improved the client's large-scale data collection platform by writing back-end pipeline components for collecting and processing hundreds of millions of pricing records daily. They developed a product-matching system that uses natural language processing and text analysis to match products without UPCs, and a brand-extraction component to determine manufacturers from product text. The team implemented backend services primarily in Scala (using Akka), and integrated data storage/aggregation on Amazon RedShift and real-time collection with RethinkDB.

Project Outcome

Enabled a sales process with General Mills, contributing to negotiations for a potentially large contract
Improved project management and processes (introduced Blossom for task management)
Integrated with the client’s systems and operated as full-time development team members supporting scaling work

Platforms

Tech Stack

Scala
Amazon RedShift
RethinkDB
Amazon Web Services (AWS)

Client Endorsement

Overall Review Rating

Timeliness

Cost Rating

Willing to Refer

Quality of Deliverables

“KindGeek’s work has facilitated a high-level sales negotiation with the client’s largest customer to date. They adapted well to existing systems, are engaged in process improvement, and employ efficient, structured project management. They provide personal attention and serve as true team members.”
Dan Gallagher
CTO
Date: 07/15/2016