pathwaycom/llm-app
Ready-to-run cloud templates for building real-time RAG, AI pipelines, and enterprise search applications that synchronize with various live data sources.
Awesome Infra for AI › Vector Databases & Retrieval Infrastructure
activeloopai/deeplake
Deep Lake is an AI Data Runtime designed to manage and store diverse data types — including embeddings, audio, text, videos, images, and DICOM files — optimized for deep learning and AI agent applications. It functions as a serverless multimodal datalake, offering scalable retrieval and efficient data streaming during model training. Key features include native compression with lazy NumPy-like indexing, multi-cloud support (S3, GCP, Azure), and built-in dataloaders for popular frameworks like PyTorch and TensorFlow. The platform simplifies the deployment of enterprise-grade LLM-based products by providing robust storage for all AI-relevant data, querying capabilities, and integrated vector search. It also supports data versioning and lineage, which is crucial for MLOps. Deep Lake offers direct integrations with tools such as LangChain and LlamaIndex for vector store functionalities in LLM applications, and Weights & Biases for data lineage during model training. While it emphasizes data management for training, its primary utility for this list lies in its role as a vector-enabled data store for LLMs and agentic RAG, facilitating the serving and operational aspects rather than just pure model training. It enables users to store large datasets in their own cloud environments and provides a visualizer for instant data validation.
https://github.com/activeloopai/deeplake
Also in Vector Databases & Retrieval Infrastructure
run-llama/llama_index
LlamaIndex is an open-source data framework for building LLM applications by connecting custom data sources to large language models, focusing on data ingestion, indexing, and retrieval augmented g...
milvus-io/milvus
Milvus is a high-performance, cloud-native vector database designed for scalable vector Approximate Nearest Neighbor (ANN) search, efficiently organizing and searching vast amounts of unstructured ...
VectifyAI/PageIndex
PageIndex is a vectorless, reasoning-based RAG system that builds hierarchical tree indexes from documents and uses LLMs to reason over them for context-aware retrieval.
qdrant/qdrant
Qdrant is an open-source, high-performance vector similarity search engine and vector database designed specifically for AI applications, enabling fast storage, search, and management of vectors wi...
Tencent/WeKnora
WeKnora is an open-source, LLM-powered knowledge framework for enterprise document understanding, semantic retrieval, and autonomous reasoning, featuring RAG, ReAct agents, and an auto-maintaining ...
topoteretes/cognee
Cognee is an open-source AI memory platform that provides AI agents with persistent long-term memory through a self-hosted knowledge graph, combining vector embeddings and graph reasoning.
RyanCodrai/turbovec
TurboVec is a Rust-based approximate nearest neighbor (ANN) vector index with Python bindings, built on Google Research's TurboQuant algorithm for efficient, memory-optimized vector similarity search.