AI-written code passes its tests far more often than its security tests
In Semgrep's run, Claude Opus 5.5 wrote working code for 93.5% of 186 real-CVE tasks and working, secure code for 54.8%. Why tests miss it, what to check.
Model evaluation
Every post on this topic, newest first.
Updated 2 OCT 2026
Typed decision models: what early tests show
Models that answer typed questions with a probability in one pass: what they are, what the first independent tests found, and how to test one.