Ai2 releases Old Ideas, Novel Problems research and benchmark, revealing instability in LLM novelty judges
Can we trust LLMs to judge whether a research idea is novel?
AI BriefingAi2 releases the study Old Ideas, Novel Problems, along with a benchmark, data, code, and an automated pipeline, to test the reliability of LLMs as judges of research idea novelty.
Also reported by 1 other source
- Ai2 Releases Old Ideas, Novel Problems Research and Benchmark to Test the Reliability of LLMs in Judging the Novelty of Research IdeasHugging Face Community Blog · 20 days ago
Source: Ai2 / Allen Institute for AI · allenai.org
Event follow-up1
View all event coverage- Ai2 releases Old Ideas, Novel Problems research and benchmark, revealing instability in LLM novelty judgesAi2 / Allen Institute for AI · 20 days ago