deepchecks/deepchecks
Deepchecks is an open-source platform providing continuous validation for AI and ML models and data from research to production, focusing on testing, CI, and monitoring.
Awesome Infra for AI › Model & Data Drift Monitoring
squaredev-io/whitebox
Whitebox is an open-source, end-to-end machine learning monitoring platform designed for production environments. It provides comprehensive monitoring capabilities for deployed ML models, including classification and regression metrics. Key features include tracking accuracy, F1-score, recall, and precision for classification models, as well as relevant metrics for regression tasks. A central aspect of Whitebox is its ability to detect and alert on data and model drift, which is crucial for maintaining model performance over time in dynamic real-world scenarios. The platform emphasizes ease of setup and use, offering a Pythonic SDK for building custom monitoring infrastructure. It is built to integrate seamlessly with Kubernetes, providing a robust and production-ready MLOps solution. Whitebox supports PostgreSQL and SQLite for data storage and offers deployment options via Docker and Helm charts, making it flexible for various operational setups. While the project initially focused broadly on ML monitoring, it has announced a pivot towards monitoring Large Language Models (LLMs), indicating an evolving focus on contemporary AI operational challenges.
https://github.com/squaredev-io/whitebox
Also in Model & Data Drift Monitoring
NannyML/nannyml
NannyML is an open-source Python library for post-deployment ML model monitoring, offering performance estimation, data drift detection, and intelligent linking of drift alerts to performance changes.
MAIF/eurybia
Eurybia is a Python library for detecting data and model drift, validating data, and generating comprehensive HTML reports for AI governance and model monitoring.