An AI agent plays physician, gathering patient information sequentially via tool calls, forming diagnoses, reviewing clinical notes for errors, and planning treatment. This environment tests real-world diagnostic reasoning, not just medical knowledge recall.
Case Information
Phase 1
Gather patient info via tools
Phase 2
Review SOAP note for errors
Phase 3
Finalize care plan
1. Start Episode
2. Take Action
3. Check State
View current episode state including case progress, scores, and phase information.
Workflow: Gather patient info → form differential → confirm diagnosis
Available tools: chart.history, chart.vitals, chart.labs, chart.imaging, calculate, ddx.add, ddx.remove
Transition to Phase 2: Call ddx.confirm with {"diagnosis": "..."}
Workflow: Review SOAP note → identify error → correct it
Available tools: note.review, note.correct
Step 1: Call note.review with {} to see the note
Step 2: Call note.correct with {"sentence_id": "N", "correction": "..."}
Transition to Phase 3: Call note.flag_none with {}
Workflow: Finalize care plan → end episode
Only tool: plan.finalize
Call: plan.finalize with {"plan": "..."} or {}
Result: Episode ends, episode reward calculated
📊 Patient Data (Phase 1)
- chart.history — Patient HPI, PMH, meds
- chart.vitals — Vital signs (BP, HR, temp)
- chart.labs — Lab panels (BMP, CBC, ABG)
- chart.imaging — Imaging reports (CXR, CT)
🔬 Clinical Analysis (Phase 1)
- calculate — Run formulas (anion_gap, qsofa, Wells, etc)
- ddx.add — Add diagnosis to differential
- ddx.remove — Remove diagnosis
- ddx.confirm — Confirm final diagnosis → Phase 2
📝 Documentation & Planning (Phase 2-3)
- note.review — View SOAP note (Phase 2)
- note.correct — Fix error in note (Phase 2)
- note.flag_none — Mark complete → Phase 3
- plan.finalize — Finalize care plan (Phase 3)
Phase 1 - Diagnosis (40%): Accuracy of ddx.confirm diagnosis (0.0-1.0 scale)
Phase 2 - Note Correction (25%): Identifying & fixing SOAP note error (0.0-1.0 scale)
Phase 3 - Calculations (20%): Accuracy of medical formulas used (±5% tolerance = 1.0)
Efficiency (15%): Avoiding unnecessary test calls (1.0 - redundant_tests × 0.02)
Final Reward: Sum of (Phase 1 × 0.40 + Phase 2 × 0.25 + Phase 3 × 0.20 + Efficiency × 0.15) = 0.0 to 1.0
Base URL: https://pratinavseth-meddiagnostic-env.hf.space
POST /reset: Start new episode with task name
POST /step: Execute action and get reward + observation
GET /state: Get current episode state
See inference.py for complete agent integration example.
An AI agent plays physician, gathering patient information sequentially via tool calls, forming diagnoses, reviewing clinical notes for errors, and planning treatment. This environment tests real-world diagnostic reasoning.
README: Complete overview at root README.md
Demo Guide: Interactive demo examples in docs/DEMO.md
Evaluator Guide: API integration patterns in docs/EVALUATOR_GUIDE.md
Design Rationale: Technical design decisions in docs/DESIGN_RATIONALE.md