[{"data":1,"prerenderedAt":54},["Reactive",2],{"news-story-news-1792366853834-y802u":3},{"id":4,"title_zh":5,"title_en":6,"summary_zh":7,"summary_en":8,"takeaways_zh":9,"takeaways_en":14,"content_zh":19,"content_en":20,"external_links":21,"source":31,"category":38,"impact_score":39,"views_count":40,"likes_count":41,"related_entities":42,"tags":46,"image_url":53},"news-1792366853834-y802u","南加州大学与 UCLA 推出离散空间状态追踪基准 Grid-Bench：8,928 组可执行测试揭示大模型长链推演严重失真，64 步长轨迹开闭源断崖式分化","USC, UCLA, and CMU Release Grid-Bench for Multi-Step Spatial Reasoning: 8,928 Executable Instances Reveal Severe State Drift Across 64-Step Trajectories","当前 LLM 能否在复杂连续空间操作中精确预测离散物理世界的演变？南加州大学（USC）、UCLA 与 CMU 联合发布 Grid-Bench（arXiv:2610.22312，GitHub: grid-bench/Grid-Bench）。该基准基于 8 种离散几何变换规则，测试 2D/3D/4D 网格在 0 至 64 步长事件链下的全状态精确预测。对 GPT-5.6 Sol、Claude 5 Sonnet、GLM-5、Kimi K2.5 等 6 款顶尖模型的评测表明：在 64 步长轨迹下，开源与中型模型完整状态精确率几乎归零，唯闭源旗舰维持断层领先。","Can LLMs reliably maintain spatial state representations across sequential operations? USC, UCLA, and CMU introduced Grid-Bench (arXiv:2610.22312, GitHub: grid-bench/Grid-Bench), an executable benchmark with 8,928 instances testing 2D/3D/4D grid state transitions across history lengths up to T=64. Evaluations across 6 frontier models show steep degradation: while GPT-5.6 Sol and Claude 5 Sonnet retain strong cell accuracy, open-weight and mid-tier models collapse to near-zero exact world reconstruction over long trajectories.",[10,11,12,13],"南加州大学、UCLA 与 CMU 联合发布可执行多维空间推理基准 Grid-Bench（arXiv:2610.22312），跨越 2D/3D/4D 网格与 8 大几何规则","涵盖 8,928 组可执行测试实例，评测历史步长从 T=0 延伸至极端复杂的 T=64 步连续离散世界演变","在 T=16 步中程推演中，GPT-5.6 Sol（98.0% 单元准度）与 Claude 5 Sonnet（96.8%）保持领先，中型模型精确世界命中率跌至个位数","在 T=64 步极限长链下，GLM-5 与 Kimi K2.5 完全精确世界命中率归零，证实长程离散空间推演必须外挂确定性状态机",[15,16,17,18],"USC, UCLA, and CMU introduced Grid-Bench (arXiv:2610.22312), an executable spatial tracking benchmark across 2D, 3D, and 4D discrete manifolds","Evaluates 8,928 instances across 8 spatial transformation rule families and trajectory lengths up to T=64 steps","At intermediate length T=16, GPT-5.6 Sol (98.0% cell accuracy) and Claude 5 Sonnet (96.8%) lead while mid-tier models drop sharply","At extreme length T=64, exact world recovery drops to 0 for GLM-5 and Kimi K2.5, demonstrating that long-horizon agents must rely on external state engines","### 核心背景与行业痛点\n随着 Coding Agent 与物理仿真智能体承接诸如自动化 CAD 建模、网格图算法优化与游戏环境模拟等复杂任务，模型必须能够理解一系列连续的空间指令（如移动局部区域、对称翻转、连通域重着色、局部求和汇聚），并在脑海中精准维护随着时间不断演变的空间状态配置。然而，现有的多模态评测多局限于单步问答或静态图像理解，无法系统量化大模型在多步长连续空间推演中的**状态漂移（State Drift）与累积误差**。为此，南加州大学（USC）、加州大学洛杉矶分校（UCLA）与 CMU 等团队联合推出了全可执行的多维空间状态评测基准 **`Grid-Bench`（`arXiv:2610.22312`）**。\n\n### 架构亮点与底层机制\n1. **8 大离散几何操作族与多维网格世界构建**：\n   - 基准定义了 8 种严格可执行的数学与几何变换操作：`PAINT`（曼哈顿球内着色）、`REFLECT`（轴对齐双射翻转）、`STENCIL`（卷积模板模运算加和）、`MIX`（有界连通域汇聚）、`RELABEL`（最大连通域重标）、`LAYER`（边界分层扩展）、`MOVE`（方向平移）与 `COMPRESS`（类 2048 几何滑块压缩）；\n   - 跨越 2D（16×16）、3D（6×6×7）与 4D（4×4×4×4）三大多维离散流形（单元数均为 252 或 256 格），并设立 $T = 0, 1, 8, 16, 32, 64$ 六级历史长度，构建了 8,928 个独立可复现测试例。\n2. **确定性参考执行器与逐单元诊断矩阵**：\n   - 每个测试案例均具备 Python 确定性参考运行结果，支持像素级、单元级精确匹配率（Exact-Match）、修改区域准确率（C-Acc）与非修改区域准确率（U-Acc）的多维断错分析。\n\n### 权威 Benchmark 与实测跑分对比\n团队在统一提示词与 128,000 Token 上下文限额下对 **GPT-5.6 Sol、Claude 5 Sonnet、GPT-5.4 Mini、GLM-5、Kimi K2.5、MiniMax M2.5** 六款前沿模型展开了深度实测（Table 10/11/13/15）：\n- **短程推演阶段（$T=1$ 与 $T=8$）**：\n  - 在 $T=1$ 时，GPT-5.6 Sol 取得 **183/192 完全精确命中**，Kimi K2.5 达到 148，MiniMax M2.5 达到 150，Claude 5 Sonnet 达到 137；\n- **中程状态追踪分水岭（$T=16$）**：\n  - GPT-5.6 Sol 保持 **98.0% 单元准确率（87/192 完全精确命中）**；Claude 5 Sonnet 维持 **96.8% 单元准确率（61/192 完全精确）**；\n  - 中型与开源模型出现明显掉队：GPT-5.4 Mini（82.4% 单元准确率，仅 5 题精确）、GLM-5（80.9%，8 题精确）、Kimi K2.5（80.6%，18 题精确）、MiniMax M2.5（64.3%，11 题精确）；\n- **长程极端推演考验（$T=64$）**：\n  - 在长达 64 步的连续离散世界演变后，中型模型完全精确预测能力全线崩溃：**GLM-5 为 0/192，Kimi K2.5 为 0/192，GPT-5.4 Mini 与 MiniMax M2.5 仅 1 题精确**；\n  - 仅顶级推理旗舰能维持有效预测：GPT-5.6 Sol 斩获 **25 题完全精确（81.6% 单元准确率）**，Claude 5 Sonnet 斩获 **14 题完全精确（90.5% 单元准确率）**；\n  - 累计正确解数量对比（Table 11 Combined /768）：GPT-5.6 Sol 以 **314 题**断层领先，Claude 5 Sonnet 取得 **201 题**，远超 Kimi K2.5（57 题）、GLM-5（28 题）与 MiniMax M2.5（48 题）。\n\n### 开发者实战落地与开箱指南\n- **开源代码与基准套件**：代码、生成流水线与评测工具均已在 GitHub 开源（`https://github.com/grid-bench/Grid-Bench`）。\n- **智能体架构设计启示**：实测数据证实，即使参数高达千亿级别的现代大模型，在纯依靠内部自注意力长上下文追踪离散空间演变时，超过 16 步即会累积致命的状态漂移。在设计涉及 CAD 几何编辑、图网络迁移或多步骤物理仿真的 Coding Agent 时，**绝对不能依赖 LLM 的隐式记忆，必须挂载外置状态机（External State Machine）或确定性内存快照**，由模型输出增量操作指令、由外置解释器维护确定性真实状态。","### 核心背景与行业痛点\nAs LLM-based coding agents and simulation frameworks take on spatial computing tasks—such as procedural CAD generation, grid-based graph processing, and game state forecasting—they must maintain consistent spatial world representations across multi-step instructions (reflecting regions, moving components, aggregating neighborhoods). However, existing vision-language and reasoning benchmarks evaluate static question-answering rather than tracking how internal representations evolve over long action sequences. Researchers from USC, UCLA, and CMU introduced **`Grid-Bench` (`arXiv:2610.22312`, GitHub: `grid-bench/Grid-Bench`)**, an executable spatial benchmark spanning 2D, 3D, and 4D discrete manifolds.\n\n### 架构亮点与底层机制\n1. **Eight Grounded Spatial Rule Families Across Multi-Dimensional Manifolds**:\n   - Defines 8 mathematically rigorous spatial transformation rules: `PAINT`, `REFLECT`, `STENCIL` (modular neighborhood aggregation), `MIX`, `RELABEL` (connected-component coloring), `LAYER`, `MOVE`, and `COMPRESS` (tile sliding);\n   - Tests across 2D (16×16), 3D (6×6×7), and 4D (4×4×4×4) grids (252–256 cells) across 6 history lengths ($T = 0, 1, 8, 16, 32, 64$), generating 8,928 independently executable evaluation instances.\n2. **Deterministic Python Execution & Diagnostic Matrix**:\n   - Because reference execution provides an exact ground truth, performance is broken down into complete-state exact matches, cell-wise accuracy on modified regions (C-Acc), and cell-wise accuracy on static regions (U-Acc).\n\n### 权威 Benchmark 与实测跑分对比\nBenchmarked across six frontier models (GPT-5.6 Sol, Claude 5 Sonnet, GPT-5.4 Mini, GLM-5, Kimi K2.5, MiniMax M2.5) with 128k output tokens and no external tools (Tables 10, 11, 13, 15):\n- **Short Trajectories ($T=1$)**: GPT-5.6 Sol yields **183/192 exact world matches**, MiniMax M2.5 reaches 150, Kimi K2.5 hits 148, and Claude 5 Sonnet scores 137;\n- **Intermediate History Divergence ($T=16$)**:\n  - GPT-5.6 Sol maintains **98.0% cell accuracy (87/192 exact worlds)**, and Claude 5 Sonnet posts **96.8% cell accuracy (61/192 exact worlds)**;\n  - Mid-tier and open models suffer sharp drops: GPT-5.4 Mini falls to 5 exact matches (82.4% cell acc), GLM-5 to 8 exact (80.9%), Kimi K2.5 to 18 exact (80.6%), and MiniMax M2.5 to 11 exact (64.3%);\n- **Long-Horizon Breakdown ($T=64$)**:\n  - Over 64 consecutive transitions, mid-tier models completely break down: **GLM-5 and Kimi K2.5 hit 0/192 exact matches**, while GPT-5.4 Mini and MiniMax M2.5 recover just 1;\n  - Frontier reasoning flagships retain non-trivial tracking: GPT-5.6 Sol achieves **25 exact world recoveries (81.6% cell acc)** and Claude 5 Sonnet yields **14 exact recoveries (90.5% cell acc)**;\n  - Total Exact Matches across $T=8..64$ (Table 11, /768): GPT-5.6 Sol dominates with **314 exact matches**, Claude 5 Sonnet achieves **201**, outstripping Kimi K2.5 (57), MiniMax M2.5 (48), and GLM-5 (28).\n\n### 开发者实战落地与开箱指南\n- **Open-Source Benchmark**: The evaluation harness and datasets are accessible at `https://github.com/grid-bench/Grid-Bench`.\n- **Architectural Implications**: Findings demonstrate that even 100B+ parameter LLMs fail to track spatial discrete dynamics past 16 steps without external aids. When engineering Coding Agents for CAD refactoring or discrete state machines, developers should never rely on the model's internal scratchpad memory; instead, anchor execution to an **external deterministic state engine** that executes model-proposed atomic commands.",[22,25,28],{"title":23,"url":23,"type":24},"https://grid-bench.github.io/","other",{"title":26,"url":26,"type":27},"https://github.com/grid-bench/Grid-Bench","github",{"title":29,"url":29,"type":30},"https://arxiv.org/abs/2610.22312","arxiv",{"platform":32,"author_name":33,"author_handle":34,"author_avatar":35,"tweet_url":36,"published_at":37},"twitter","USC / UCLA / CMU","@huggingface","https://unavatar.io/x/huggingface","","2026-10-18T23:40:53.834Z","Research",91,37,0,{"models":43,"agents":44,"vendors":45},[],[],[],[47,48,49,50,51,52],"Grid-Bench","Spatial Reasoning","USC","UCLA","Benchmark","State Tracking","https://arxiv.org/html/2610.22312v1/revision-rule-matrix.png",1793165123393]