Awesome Infra for AI › LLM Gateways & Proxies

Kenza-AI/sagify

⭐ 442 Python repository created 2018-03-03

Sagify provides a streamlined interface for managing machine learning workflows and LLM operations on AWS SageMaker. Its core offering is an LLM Gateway, which acts as a unified entry point for interacting with both proprietary models like OpenAI's GPT series and open-source LLMs such as Llama-2 and various embedding models. This gateway abstracts away the complexities of deployment and integration, allowing users to focus on leveraging LLMs for chat completions, image generation, and embeddings through a simplified API. Sagify supports deployment of these models as SageMaker endpoints, handling the underlying infrastructure setup. Users can configure and deploy services like chat completions, image creations, and embeddings, choosing from a variety of supported models. The tool also facilitates the local deployment of a FastAPI-based LLM Gateway, connecting to the deployed SageMaker endpoints or external APIs like OpenAI. It aims to reduce the operational overhead typically associated with bringing ML and LLM models into production, making them more accessible for developers and ML engineers by automating much of the AWS SageMaker integration and LLM service orchestration. While it interacts with the AWS SageMaker training capabilities, its primary value proposition lies in simplifying the serving and inference aspects of models, particularly LLMs, through its gateway module.

https://github.com/Kenza-AI/sagify

llmopsllm-inferencellm-gatewaymodel-servingsagemakeropenaiopen-source-llmgenerative-ailangchain