OpenRouter’s Hands-On Measurement of Jev 1.13 and Claude Opus 5 on Accuracy, Latency, and Cost for the Banking77 Classification Task
Is Jev as Accurate as Frontier Models at Classification?
AI BriefingOpenRouter compared the intent-classification performance of Jev 1.13 and Claude Opus 5 using 3,080 customer-service utterances from the Banking77 test set. Jev's accuracy was 81.0%, 3.3 percentage points below Opus's 84.4%, but its median latency was 175 ms, approximately 1/13 of Opus's 2,266 ms. The cost per 1,000 requests was $0.11 versus $2.42 with prompt caching enabled.
Source: OpenRouter: Announcements · openrouter.ai
Event follow-up1
View all event coverage- OpenRouter’s Hands-On Measurement of Jev 1.13 and Claude Opus 5 on Accuracy, Latency, and Cost for the Banking77 Classification TaskOpenRouter: Announcements · September 23