78
Reproduced
Leaderboard
CyberGym Level 1 ranking: agents receive a vulnerability description and unpatched codebase, then must reproduce the target with a working PoC.
Type:
Trials:
| # | Agent / Model | Success Rate | Date | Source |
|---|---|---|---|---|
| 1 |  Gcsa Agent Agent | 91.3% | 2026-08-26 | GCSA |
| 2 |  Crystalline Agent | 89.6% | 2026-06-08 | Independent researcher |
| 3 |  MDASH Agent | 88.5% | 2026-05-12 | Microsoft |
| 4 |  GPT-5.5-Cyber Model | 85.6% | 2026-06-22 | OpenAI |
| 5 |  Xuanwu Atuin AI Agent | 84.8% | 2026-07-22 | Tencent Xuanwu Lab |
| 6 |  Claude Mythos Preview Model | 83.1% | 2026-04-07 | Anthropic |
| 7 | GPT-5.5 Model | 81.8% | 2026-04-23 | OpenAI |
| 8 | GPT-5.4 Model | 79.0% | 2026-04-23 | OpenAI |
| 9 |  MopMonk Agent Agent | 73.1% | 2026-06-28 | MopMonk AI |
| 10 |  Whitzard Agent | 68.9% | 2026-07-21 | Whitzard Team |
| 11 |  GLM-5.1 Model | 68.7% | 2026-04-07 | Zhipu AI |
| 12 | Claude Opus 4.6 Model | 66.6% | 2026-02-05 | Anthropic |
| 13 | GPT-5.4 Model | 66.3% | 2026-04-07 | Zhipu AI |
| 14 | Claude Sonnet 4.6 Model | 65.2% | 2026-02-17 | Anthropic |
| 15 |  SageAgent Agent | 60.2% | 2026-02-09 | OpenSage Team |
| 16 |  Muse Spark 1.1 Model | 59.0% | 2026-07-09 | Meta AI |
| 17 |  DeepSeek-V4-Pro Model | 57.7% | 2026-07-13 | XDxAI |
| 18 | Claude Opus 4.5 Model | 50.6% | 2026-02-05 | Anthropic |
| 19 | Muse Spark Model | 43.5% | 2026-04-14 | Meta AI |
| 20 | GLM-5 Model | 43.2% | 2026-02-12 | Zhipu AI |
| 21 |  Kimi K2.5 Model | 41.3% | 2026-02-02 | Kimi |
| 22 | GPT-5 Model | 39.4% | 2025-12-05 | CyberGym Team |
| 23 |  Gemini 3.1 Pro Model | 38.8% | 2026-04-07 | Zhipu AI |
| 24 | Claude Sonnet 4.5 Model | 28.9% | 2025-09-29 | Anthropic |
| 25 | Claude Opus 4.1 Model | 25.0% | 2025-09-29 | Anthropic |
| 26 | GLM-4.7 Model | 23.5% | 2026-02-12 | Zhipu AI |
| 27 | Claude Sonnet 4 Model | 22.6% | 2025-09-29 | Anthropic |
| 28 | Claude Sonnet 4 Model | 17.9% | 2025-05-23 | CyberGym Team |
| 29 | Claude Sonnet 3.7 Model | 14.5% | 2025-09-29 | Anthropic |
| 30 | Claude Sonnet 3.7 Model | 11.9% | 2025-05-15 | CyberGym Team |
| 31 | GPT-4.1 Model | 9.4% | 2025-05-15 | CyberGym Team |
| 32 | GPT-4.1 Model | 9.0% | 2025-05-15 | CyberGym Team |
| 33 | GPT-4.1 Model | 7.4% | 2025-05-15 | CyberGym Team |
| 34 | GPT-4.1 Model | 7.2% | 2025-05-15 | CyberGym Team |
| 35 | Gemini 2.5 Flash Preview Model | 4.8% | 2025-05-15 | CyberGym Team |
| 36 | DeepSeek-V3 Model | 3.6% | 2025-05-15 | CyberGym Team |
| 37 | o4-mini Model | 2.5% | 2025-05-15 | CyberGym Team |
| 38 | R2E-Gym-32B Model | 2.0% | 2025-05-15 | CyberGym Team |
| 39 |  Qwen3-235B-A22B Model | 1.9% | 2025-05-15 | CyberGym Team |
| 40 | OpenHands-LM-32B Model | 1.7% | 2025-05-15 | CyberGym Team |
| 41 | SWE-Gym-32B Model | 0.1% | 2025-05-15 | CyberGym Team |
Data synced from CyberGym · snapshot 2026-08-26
The vulnerability list.
CyberGym n=100 real-world task pool — 78 vulnerabilities reproduced with zero human steps across 42 open-source projects. Click Start to browse the project index, then pick a project to view findings.
42
Projects
11
CWE classes
78%
Success rate
cryptidsmith://vulns/
WE SECURE THE SOFTWARE THE WORLD RUNS ON
Click a row to expand details. Success is based on CyberGym ASAN exit_code only. HOT = recovered after upgrade.