Evidently
Evidently is an open-source Python library developed for machine learning and evaluation, testing and monitoring of LLM-based systems. The tool, which can work with tabular and text data, provides both offline analysis and live monitoring processes, offering over 100 built-in metrics. Users can export data in JSON or HTML format and configure their system by defining custom metrics.
evidentlyai · MLOps & AI Infrastructure
Common commercial products in this category: Weights & Biases, Amazon SageMaker, Databricks
- You can create data quality and performance reports for predictive and productive AI models.
- By defining test sets, you can perform automatic pass/stay checks in the CI/CD processes or data verification stages.
- Using the tracking interface, you can visualize the metrics of the ML and LLM systems with their changes over time.
- • NeutralHacker News
Launch HN: Evidently AI (YC S21) – Track and Debug ML Models in Production
“Thanks for the support! For the moment the workflows you describe are built externally: - To set notifications you can send the output from Evidently to other tools like Grafana and then build a notification workflow around it. If you have a batch model, you can use some…”
- • NeutralHacker News
Launch HN: Evidently AI (YC S21) – Track and Debug ML Models in Production
“True, we do not want to force users into a rigid workflow. We build the tool so it can integrate with other parts of the ML stack. Right now you can for example use Evidently to calculate if the model has drifted, and then log the results in a JSON format, or take only some…”