Skip to content

Conversation

@thegovind
Copy link

Description

This PR adds a comprehensive workshop notebook demonstrating Azure AI's evaluation and observability capabilities, developed in collaboration with @changliu2 and Alex Sutton for TechConnect. This is the first iteration of the workshop material, which has been fully tested and provides hands-on examples of:

  • Safety evaluation using ContentSafety, Violence, and HateUnfairness evaluators
  • Security testing with DirectAttackSimulator and adversarial testing
  • Quality assessment using Relevance, Coherence, and Groundedness evaluators
  • Cloud-based evaluation workflows with azure-ai-projects
  • Observability setup with OpenTelemetry and Application Insights
  • Best practices for responsible AI governance

The workshop includes:

  • A Jupyter notebook with detailed explanations and working code
  • Sample evaluation test data in JSONL format
  • Environment configuration templates

This serves as a practical guide for developers to implement comprehensive evaluation and monitoring in their Azure AI applications.

Checklist

  • I have read the contribution guidelines
  • I have coordinated with the docs team (mldocs@microsoft.com) if this PR deletes files or changes any file names or file extensions.
  • This notebook or file is added to the CODEOWNERS file, pointing to the author or the author's team.

@thegovind thegovind requested a review from a team as a code owner February 10, 2025 05:01
@thegovind thegovind had a problem deploying to external-contribution February 10, 2025 05:02 — with GitHub Actions Failure
@thegovind thegovind had a problem deploying to external-contribution February 12, 2025 16:18 — with GitHub Actions Failure
@thegovind thegovind had a problem deploying to external-contribution February 12, 2025 19:16 — with GitHub Actions Failure
@thegovind thegovind had a problem deploying to external-contribution February 12, 2025 20:48 — with GitHub Actions Failure
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants