InferenceBench: A Benchmark for Open-Ended LLM Inference Optimization by AI Agents
AI agents are increasingly used to automate research and development tasks, yet existing benchmarks typically evaluate them on prescribed workflows or narrow action spaces. Even nominally open-ended tasks can often be solved by retrieving a well-known recipe and tuning a few hyperparameters, making it unclear whether strong results reflect genuine optimization or memorized solutions. We introduce InferenceBench, where an agent must deploy an OpenAI
By Jehyeok Yeon, Ben Rank, Maksym Andriushchenko