Is the RAG ecosystem heading down the same chaotic path as early API development

Building a reliable RAG (Retrieval-Augmented Generation) application feels like piecing together a jigsaw puzzle—lots of tools, frameworks, and decisions to juggle.

Back in 2011, API developers had to cobble together frameworks for tasks like authentication and rate limiting. Without platform teams, this chaos led to a proliferation of tools—an issue many organizations are still untangling today as they consolidate on platforms like Kong.

RAG development is facing a similar challenge. Developers must stitch together frameworks like LangChain and LlamaIndex for data retrieval and processing, observability tools like Langfuse, evaluation frameworks such as DeepEval and Ragas, and APIs to create practical applications. Each piece is critical, yet the ecosystem lacks the cohesion needed for streamlined development.

This brings me to some pressing questions for RAG developers and platform teams:

• Is agent and RAG sprawl becoming a problem?

• Is it time for a RAG platform?

• Like API management, should observability and evaluations of RAG applications exist as separate capabilities?

• How equipped are current platform teams to handle RAG applications and platforms?

I’d love to hear your thoughts: Are we heading toward a RAG platform era, or is this ecosystem destined for sprawl? Please DM me if you would like to discuss.