endorsedCompleted

RAG System Development on Azure for Legal AI Assistant

Services Covered:

AI and Machine Learning • Large Language Model (LLM) Development • Software QA and Testing • Performance Testing • Cloud Consulting and Services

Industry

Legal

7 months

Budget

50K - 199K

Start Date

1 May 2025

Team Size

1-5

Engagement Model

Offshore

Project Capability Score (PCS)

67/100

Good

Project Capability Score (PCS) estimates how capable Abstracta Inc is of successfully delivering a project like this, based on its past experience, track record, and ability to handle similar work. Learn more about PCS

Client

Iberley Informacion Legal SL

Spain

Project Summary

Abstracta Inc. was engaged to design, implement, and QA a Retrieval-Augmented Generation (RAG) system on Microsoft Azure integrated with GPT-4 to support a legal AI assistant for a repository of 500,000 Spanish legal documents. The vendor initially followed an Agile approach and delivered structured sprints, but in the second phase the engagement deteriorated: communication and methodology collapsed, product stability and QA were poor, and the client reports the vendor ultimately abandoned the project after charging €70,000, leaving an incomplete product.

Key Challenges

Develop a Retrieval-Augmented Generation (RAG) system on Azure integrated with ChatGPT-4
Process and retrieve information from a repository of 500,000 legal documents efficiently
Provide testing, quality assurance, and overall system performance optimization

Project Deliverables

Architecture design of the RAG system
Data ingestion and indexing of 500,000 legal documents
Testing strategy and QA implementation
Performance monitoring and optimization
Collaboration and technical guidance to internal team

Project Solution

Abstracta was contracted to design and implement a Retrieval-Augmented Generation (RAG) architecture hosted on Microsoft Azure and integrated with GPT‑4. The scope included defining and implementing document ingestion and indexing for 500,000 legal documents (Azure Cognitive Search compatibility), delivering an architecture design, building testing strategies (automated and manual), and implementing performance monitoring and optimization. The vendor was also expected to provide technical guidance and collaborate with the client's internal team.

Project Outcome

Project lost structure and communication deteriorated in the second half of the engagement
Product was unstable and testing/QA were poorly managed, forcing the client to take over critical work
Vendor abandoned the project after charging €70,000, leaving an incomplete and unreliable product
There were no measurable outcomes that reflected the original scope or justified the investment

Platforms

Cloud
AI/ML Platform

Tech Stack

Azure

Client Endorsement

Overall Review Rating

Timeliness

Cost Rating

Willing to Refer

Quality of Deliverables

“Abstracta Inc. has abandoned the project after charging the client €70,000 and delivering an incomplete product. The team has failed to deliver the project as per the scope, and the client is disappointed with the partnership. The team has used an Agile methodology during the project's first phase.”
Anonymous
Executive
Date: 04/04/2025