AI Benchmarks Miss How Models Fail Under Pressure

A preprint link that appears to promise a benchmark of ten frontier large language models instead leads to a different study entirely—one that tested 21 models on whether they fabricate missing safety data in hospital injury records. At the same time, a separate arXiv paper, OPT-Engine: Benchmarking the Limits of LLMs in Optimization Modeling via Complexity Scaling, systematically exposes a “critical robustness gap” when large language models are pushed to scale optimization problems from simple linear programs to mixed-integer programming. Together, the two sources illustrate a shift in AI evaluation: away from static accuracy scores and toward stress-testing how models behave under missing data, rising complexity, and real-world constraints.

For executives and specialists deploying large language models in logistics, finance, supply chain management, or clinical safety, the findings matter because they expose failure modes that standard leaderboards rarely capture. A model can appear accurate on a benchmark while still inventing information when a data field is absent, or while failing to maintain global constraints as an optimization problem grows. These are not abstract concerns; they directly affect whether an AI system can be trusted to extract injury data or to formulate industrial planning problems without human intervention.

A preprint link that does not match its title

The medRxiv URL provided under a title about benchmarking ten frontier large language models does not lead to that claimed study. Instead, it redirects to a Zenodo repository for a different paper: “Do language models invent the safety data they cannot find? A multi-model benchmark of large language model abstention and fabrication in injury-narrative extraction”, by Amir Rafe and Subasish Das from Texas State University, published August 27, 2026.

That study evaluated 21 large language models on their ability to extract bicycle helmet status from 1,464 human-adjudicated NEISS emergency-department injury narratives. Its focus was not general accuracy but two specific behaviours: abstention, where a model correctly states that the information is not present, and fabrication, where a model invents a plausible answer when the data is absent. The dataset includes 57,096 raw model calls with latency and token counts, which supports reproducibility and further analysis. No specific performance percentages or rankings of models were confirmed in the available metadata.

The distinction between abstention and fabrication is important because it separates a model that acknowledges uncertainty from one that silently produces a plausible but unsupported answer. In safety-related data extraction, that difference can determine whether a downstream analysis is based on verified information or on invented details. The mismatch between the link title and the underlying repository is also a reminder that preprint metadata can be unreliable. Researchers and decision-makers should verify the actual content of a preprint before citing it, especially when a title suggests a different scope or model count than the deposited paper.

OPT-Engine: stress-testing optimization reasoning

A separate paper, OPT-Engine: Benchmarking the Limits of LLMs in Optimization Modeling via Complexity Scaling, published on arXiv as 2601.19924v2 on May 14, 2026, takes a different approach. Developed by researchers from Cardinal Operations (Shanghai), Shanghai University of Finance and Economics, University of Chicago, and Shanghai Jiao Tong University, the framework evaluates large language models on ten canonical Operations Research problems, scaling systematically from Linear Programming to Mixed-Integer Programming.

The paper distinguishes between two modes of model reasoning. Pure-Text Reasoning (PTR) relies on chain-of-thought text generation without external tools. Solver-integrated Reasoning (SIR) allows the model to call an external solver for computational steps. The key findings are:

• Pure-Text Reasoning via Chain-of-Thought shows a critical robustness gap as complexity increases.
• Integrating external computational tools helps with local calculations, but models still fail to respect global optimization constraints.
• The primary bottleneck identified is the automated formulation of constraints within the current state-of-the-art Solver-integrated Reasoning paradigm.

The primary bottleneck identified is the automated formulation of constraints within the current state-of-the-art Solver-integrated Reasoning (SIR) paradigm.

This means that even when a model can call a solver to handle arithmetic or local steps, it may still produce a formulation that violates the overall problem structure. The failure is not in the solver but in the model’s ability to translate a real-world problem into a complete and correct set of constraints. Mixed-integer problems are common in logistics and supply chain planning, where discrete decisions are often required. The complexity scaling in OPT-Engine is therefore directly relevant to industrial applications that cannot tolerate silent constraint violations.

Why these failures matter beyond leaderboards

The two studies point to the same conclusion from different angles: current evaluation methods often miss how models fail when conditions become harder. The Zenodo study tests whether models can say “not present” instead of inventing an answer. OPT-Engine tests whether models can maintain correctness as problem complexity scales. Both move beyond static accuracy metrics to probe scalability and robustness in high-stakes domains like logistics, finance, and supply chain management.

For executives and specialists, the practical implication is clear: current large language models are not yet reliable for complex, real-world optimization without human oversight in constraint formulation. A model that performs well on simple problems may break down as complexity increases, as the OPT-Engine findings show. The framework offers a roadmap for developing more robust, industrial-grade AI systems by identifying where that breakdown occurs and what needs to be improved.

The injury-narrative study similarly treats abstention as a measurable behaviour distinct from fabrication, which is a useful distinction for safety-critical applications. A model that refuses to answer when data is missing may be less impressive in a demo, but it is safer in practice than a model that fills the gap with fluent but false output.

What this means for AI evaluation and deployment

The mismatch between the medRxiv link title and the Zenodo repository also carries a methodological lesson. Preprint metadata can be incomplete or misleading, and a title that mentions ten models may not match the deposited paper that evaluated 21 models. Teams that rely on preprint summaries without opening the underlying record risk citing the wrong scope, model count, or findings. In this case, the available metadata confirms no specific performance percentages or rankings, which means any downstream claim about model superiority would be unsupported.

More broadly, the shift toward stress-testing is likely to accelerate. Benchmarks that only report aggregate accuracy can hide the difference between a model that knows what it does not know and a model that fills gaps with fluent but false output. The OPT-Engine framework contributes a structured way to measure that difference as complexity scales, while the Zenodo dataset provides raw model calls, latency, and token counts for reproducibility. Both resources give researchers and practitioners a clearer view of where current systems need human oversight.