Human-in-the-Loop Design

Format: Design a workflow for humans to review AI output.

When human review is needed:

• AI's decision affects real people (loan approvals, hiring screening)
• AI errors are costly (medical advice, legal documents)
• AI is in its early learning phase (insufficient data, low accuracy)

Exercise: Design Human-in-the-Loop workflows for the following scenario:

Scenario: AI Content Moderation System

Fully automated (high confidence):
  AI confidence > 95% "safe" → Auto-approve
  AI confidence > 95% "violation" → Auto-remove + notify user

Human review (low confidence):
  AI confidence 50-95% → Enter human review queue
  AI confidence < 50% → Priority human review

User appeal:
  Removed content → User can appeal → Human re-review