Cognition Releases Devin’s SWE-bench Technical Report
SWE-bench technical report
AI BriefingCognition released a technical report on Devin’s performance on SWE-bench. Devin solved 79 problems in a randomly sampled set of 570 tests, achieving a success rate of 13.86%, higher than the previous best baseline of 4.80% for Claude 2 with assistance and 1.96% without assistance.
Source: Cognition model / Devin blog · cognition.com