Security & Quality Lifecycle
For every $1 you spend on developing a digital solution, plan to spend at least another $1 to ensure full adoption and scaling.
AI agents build governed. Humans ship trusted. 80% autonomy, 100% accountability.
Golden Path: From Shift-Left to Sustained Quality
Phase 1: Scan (2 min)
Who: devops-security-engineer scans. security-compliance-engineer reviews findings.
What: Static analysis, container scanning, secret detection. Shift-left.
Why: 10x cheaper to fix in dev than prod. Zero critical/high vulnerabilities before merge.
What-if skip: Vulnerabilities ship to production, post-deployment remediation costs 10x more.
How
/security:sast
Output
- SAST scan results (code + container + secrets)
- Vulnerability report ranked by severity
- Remediation recommendations
Quality Gate: Zero CRITICAL/HIGH. All findings acknowledged.
Phase 2: Test (variable)
Who: qa-engineer validates test quality. HITL reviews coverage.
What: 3-tier testing with real APIs. Battle tests, not mock theater.
Why: DRYRUN_OVER_READONLY prevented — real API validation with READONLY profiles. 6,298 real tests.
What-if skip: TESTING_THEATER — inflated pass rates, production surprises despite green CI.
How
/cloudops:theater-check # Quick theater health check
/cloudops:theater-audit # Full audit with theater score
Output
- Theater score (0-100): mock density, orphan count, coverage omit entries
- Test execution results with real API validation
- Coverage report against pyproject.toml fail_under
Quality Gate: Theater score under 20 (low theater). Coverage meets fail_under.
Phase 3: Comply (1 hour)
Who: security-compliance-engineer gates. HITL reviews evidence package.
What: SOC2, APRA CPS 234, ISO 27001 evidence generation. Security posture check.
Why: Regulatory risk caught at design time. Audit evidence auto-generated, not manually assembled.
What-if skip: Compliance surprises at quarterly audit, manual evidence taking weeks.
How
/aws:security-posture # Security Hub findings ranked by severity
/security:cert-inventory # Multi-cloud certificate expiry triage
Output
- Security Hub findings: CRITICAL/HIGH/MEDIUM/LOW with SOC2 mapping
- Certificate inventory with 30/60/90 day risk triage
- Compliance evidence package for auditors
Quality Gate: All CRITICAL findings have remediation plan. Cert expiry monitored.
Phase 4: Harden (per release)
Who: devops-security-engineer hardens. Hooks enforce deterministically.
What: Supply chain SBOM, Docker registry enforcement, signed container images.
Why: SLSA Level 2+ provenance. Chainguard Wolfi base images are sigstore-signed.
What-if skip: Unsigned images, dependency confusion, supply chain compromise.
How
/devcontainer:validate-registry # Docker registry compliance scan
Output
- Registry compliance score (nnthanh101/* only)
- SBOM generated (CycloneDX format)
- Trivy scan clean (zero CRITICAL/HIGH)
Quality Gate: Registry 100% compliant. SBOM attestation attached.
Phase 5: Sustain (per sprint)
Who: meta-engineering-expert extracts patterns. HITL decides improvements.
What: Anti-pattern catalog grows. Testing theater audits. Quality ratchet.
Why: Each anti-pattern cost real sessions. The catalog (64 patterns) prevents repeat failures.
What-if skip: Same mistakes repeated, testing theater creeps back, quality degrades.
How
/speckit.retrospective # Sprint retrospective with action items
/ceremony:review # Sprint review with DORA actuals
Output
- Updated anti-pattern catalog entries
- Quality metrics trend (coverage, theater score, governance score)
- Improvement actions with owners and deadlines
Quality Gate: Anti-patterns documented. Coverage ratchet maintained.
LEAN/5S Applied to Security & Quality
| Principle | Application | Evidence |
|---|---|---|
| Sort | 64 anti-patterns catalogued, not ad-hoc | .claude/rules/anti-patterns-catalog.md |
| Set in Order | 3-tier testing: snapshot / LocalStack / AWS live | testing/battle-conftest skill |
| Shine | 104K lines of mock theater deleted | conftest 1,157 → 99 lines |
| Standardize | Hook-enforced gates (exit 2 = blocked) | detect-testing-theater.sh |
| Sustain | Coverage ratchet: fail_under measured, not estimated | pyproject.toml SSOT |
By Persona
Security Engineer
Path: /security:sast → /devcontainer:validate-registry → /aws:security-posture
Time to Value: First SAST scan in under 2 minutes.
QA Lead
Path: /cloudops:theater-check → /cloudops:theater-audit → /ceremony:review
Time to Value: Testing theater score in under 5 minutes.
Compliance Officer
Path: /aws:security-posture → /security:cert-inventory → bash scripts/governance-score.sh
Time to Value: Audit evidence package in 1 hour.
Common Mistakes (Anti-Patterns)
| Mistake | Why It Fails | Fix |
|---|---|---|
| TESTING_THEATER | Mocks without assertions inflate pass rates | detect-testing-theater.sh hook |
| DRYRUN_OVER_READONLY | --dry-run when READONLY profiles exist | Real execution with READONLY |
| ADJUSTED_METRIC_EXCLUSION | Expanding coverage omit to inflate rates | detect-testing-theater.sh blocks |
| MATURITY_SCORE_WITHOUT_TEST | Scoring based on file counts, not test results | Cite test execution evidence |
| PHANTOM_MODULE_IMPORT | Import passes --help but crashes at runtime | AST-based test_cli_imports.py |
| COVERAGE_GATE_MISMATCH | CI gate differs from pyproject.toml | Single SSOT in pyproject.toml |
Quick Reference: Command Cheat Sheet
# Scan (shift-left)
/security:sast
# Test (real, not theater)
/cloudops:theater-check
/cloudops:theater-audit
# Comply (evidence generation)
/aws:security-posture
/security:cert-inventory
# Harden (supply chain)
/devcontainer:validate-registry
# Sustain (continuous improvement)
/speckit.retrospective
/ceremony:review
Agent Team
| Agent | Role in This Path | Phase/Stage | Talent Bench |
|---|---|---|---|
| security-compliance-engineer | Threat modeling + compliance mapping (APRA CPS 234, SOC2, NERC CIP-013) | Scan/Comply/Sustain | Profile |
| devops-security-engineer | SAST scanning (semgrep, bandit) + secrets scanning (gitleaks/trivy) | Scan/Shift-Left | Profile |
| qa-engineer | Quality gate enforcement + testing rigor validation (no testing-theater) | Test/Verify | Profile |
| cloud-architect | Security architecture design + threat modeling for new features | Design/Harden | Profile |
| infrastructure-engineer | Supply chain hardening (container registry validation, signed artifacts) | Harden/Deploy | Profile |
7 Skills Coverage
| Skill | Coverage in This Path | Implementation |
|---|---|---|
| S1 System Design | Shift-left security pipeline (Scan→Test→Comply→Harden→Sustain), secure SDLC integration | Pipeline architecture, security gates, phase gating |
| S2 Tool Design | SAST tool schemas (bandit, semgrep) + container scanner schemas (trivy, checkov) | Tool integration, rule configuration, output parsing |
| S3 Retrieval | CVE databases (NVD), Security Hub findings, supply chain SBOMs (CycloneDX), compliance requirement repositories | Vulnerability intelligence, threat data, compliance rules |
| S4 Reliability | Scan retry logic + circuit breaker for third-party vulnerability APIs + timeout enforcement | Resilience, timeout guards, fallback paths |
| S5 Security | This IS the security path — full coverage of Principle V (Governance). READONLY AWS access for posture collection, HITL gates before remediation | Comprehensive security controls, human-in-the-loop mutations |
| S6 Evaluation | Vulnerability scoring (CVSS + business impact) + compliance mapping (CVE→CPS234 req) + audit trail generation | Risk quantification, compliance evidence, decision support |
| S7 Product Thinking | Audit reports for APRA CPS 234 / SOC2 certification + timeline transparency (roadmap for remediations) + board-level risk summary | Regulatory compliance, executive communication, risk transparency |
Last Updated: March 2026 | Status: Active | Maintenance: security-compliance-engineer