cloudops.theater-audit
Type: commands | Track: Enterprise
Audit testing theater signals in the runbooks project and produce a theater_score (0-100). Measures four signals: orphan test files, coverage omit entries, mock density, and wheel install gate presence. Delegates to qa-engineer.
Quick Start
# Full audit (default: runbooks project)
/cloudops:theater-audit
# Specific project path
/cloudops:theater-audit /path/to/project
Theater Score
| Signal | Weight | Description |
|---|---|---|
| Orphan test files | 30 pts | Test files outside pytest testpaths discovery |
| Coverage omit count | 25 pts | Entries in [tool.coverage.report] omit list |
| Mock density | 25 pts | Ratio of MagicMock/patch lines to assert lines |
| Wheel install missing | 20 pts | No pip install dist/*.whl step in publish workflow |
| Score Range | Verdict |
|---|---|
| < 20 | PASS — acceptable |
| 20–50 | WARNING — address before next release |
| > 50 | FAIL — BLOCKING, must resolve before publish |
Benefits
- Prevents
TESTING_THEATERanti-pattern — identifies test suites with high pass counts but zero real coverage - Finds orphan test files outside
testpathsthat CI never discovers - Detects mock density (MagicMock/patch without real assertions) that validates mock behavior, not code paths
- Verifies wheel install gate exists in publish workflow (Gate 0)
When to Use
| Attribute | Detail |
|---|---|
| Persona | QA Engineer |
| Trigger | Test quality concerns — before a PyPI release, when CI passes at 8% coverage while HITL target is 58%, or when a new engineer adds many test files without corresponding real coverage growth |
| Business Value | Theater score (0-100) measuring four signals — surfaces the gap between "tests pass" and "tests validate real behavior"; March 2026 incident: 9,826 functions passed at 8% coverage while theater signals were at FAIL level |
| Frequency | Sprint |
Example: As a QA Engineer, I need to audit testing theater signals before the sprint release because the published wheel must not fail at consumer pip install even when all internal tests pass. I run /cloudops:theater-audit which scores orphan files (30pts), coverage omit count (25pts), mock density (25pts), and wheel gate presence (20pts), and produces a theater_score with specific remediation actions for each FAIL signal.
Enterprise-only. Contact sales for licensing details.