Awesome Infra for AI › Model & Data Drift Monitoring

deepchecks/deepchecks

⭐ 4061 Python repository created 2021-10-11

Deepchecks is an open-source solution designed for continuous validation of AI and ML models and their associated data throughout the entire lifecycle, from research and development to production deployment. Its core functionality spans three main areas: Deepchecks Testing, CI & Testing Management, and Deepchecks Monitoring. Deepchecks Testing allows users to run built-in and custom checks and suites for validating tabular, NLP, and computer vision models and data. This component helps in identifying issues like data integrity problems, model performance degradation, and data/model drift early in the development cycle. It provides detailed reports, enabling easy inspection of check results. The CI & Testing Management aspect facilitates collaboration over test results, ensuring that models are production-ready through efficient iteration. This can be integrated into CI/CD pipelines to automate validation processes. Deepchecks Monitoring focuses on tracking and validating deployed models' behavior in production environments. It helps detect model and data drift, anomalies, and performance issues in real-time. The monitoring solution is available as both an open-source and managed offering, with the open-source version supporting the monitoring of a single model per deployment. The platform supports various data types, ensuring comprehensive validation across different AI/ML applications.

https://github.com/deepchecks/deepchecks

data-driftdata-validationmodel-monitoringmodel-validationmlopsai-validationcontinuous-integrationproduction-ml