[{"data":1,"prerenderedAt":48},["Reactive",2],{"news-story-news-1792424548151-emeeh":3},{"id":4,"title_zh":5,"title_en":6,"summary_zh":7,"summary_en":8,"takeaways_zh":9,"takeaways_en":14,"content_zh":19,"content_en":20,"external_links":21,"source":25,"category":32,"impact_score":33,"views_count":34,"likes_count":35,"related_entities":36,"tags":40,"image_url":47},"news-1792424548151-emeeh","小米 AI 实验室发布细粒度多模态模型 Xiaomi-Lens-0：4B 参数刷新 RefCOCO 语义分割纪录，掩码自监督打通像素级空间定位","Xiaomi AI Lab Introduces Xiaomi-Lens-0: 4B MLLM Built on Qwen3.5 Sets SOTA on RefCOCO with Native Pixel-Level Mask Decoding","针对多模态大模型普遍依赖纯文本监督而欠缺像素级细粒度空间定位能力的瓶颈，小米 AI 实验室（Xiaomi AI Lab）发表技术报告 Xiaomi-Lens-0（arXiv:2610.23279）。该模型基于 Qwen3.5 4B 打造，通过轻量级掩码解码器将视觉编码器 Patch 特征、隐层表征与掩码 Token 状态深度融合，在 RefCOCO/RefCOCO+/RefCOCOg 目标表达理解（REC 达 94.6%/92.1%/92.5%）与表达语义分割（RES cIoU 达 83.4%/79.9%/81.1%）上全面超越 Florence-2 与 LISA，同时保持 69.7% 的通用多模态感知高分。","Multimodal LLMs excel at holistic image captioning but struggle with pixel-level spatial grounding due to lack of spatial supervision. Xiaomi AI Lab unveiled Xiaomi-Lens-0 (arXiv:2610.23279), a 4B MLLM based on Qwen3.5 that natively predicts pixel masks. A lightweight decoder integrates patch embeddings, visual latents, and mask-token activations, driving RefCOCO comprehension (REC Prec@0.5: 94.6% / 92.1% / 92.5%) and segmentation (RES cIoU: 83.4% / 79.9% / 81.1%) past Florence-2 and LISA-7B while preserving a 69.7% MME-RW score.",[10,11,12,13],"小米 AI 实验室发布 4B 参数细粒度多模态大模型 Xiaomi-Lens-0（arXiv:2610.23279），以 Qwen3.5 4B 为基座原生输出像素掩码","在 RefCOCO/RefCOCO+/RefCOCOg 目标表达理解（REC）测试中分别斩获 94.6%、92.1% 与 92.5%，全面超越 Florence-2 与 Grounding-DINO-L","在表达语义分割（RES）任务上取得 83.4% / 79.9% / 81.1% 的 cIoU 高分，大幅碾压 7B 级的 LISA 与 GSVA","在 MME-RW-en 真实世界通用评测中取得 69.7%，超越 Qwen2.5-VL 7B，证实细粒度空间监督能够反哺通用多模态理解",[15,16,17,18],"Xiaomi AI Lab released Xiaomi-Lens-0 (arXiv:2610.23279), a 4B MLLM based on Qwen3.5 that natively generates pixel-level masks","Outperforms Florence-2 and Grounding-DINO-L on RefCOCO/RefCOCO+/RefCOCOg REC benchmarks with 94.6%, 92.1%, and 92.5% Prec@0.5","Achieves 83.4% / 79.9% / 81.1% cIoU on RES segmentation, substantially surpassing 7B models like LISA and GSVA","Maintains superior general perception, scoring 69.7% on MME-RW-en and outperforming Qwen2.5-VL 7B","### 核心背景与行业痛点\n当前主流多模态大语言模型（MLLM，如 Qwen-VL、LLaVA）在整图问答与宏观场景描述上表现优异，但在需要像素级定位的细粒度视觉任务（如端侧精准抠图、局部瑕疵检测、具身机器人操作锚定）中往往捉襟见肘。其核心瓶颈在于：绝大多数 MLLM 采用纯文本或简易边界框（Bounding Box）进行弱监督对齐，语言模型内部表征从未与底层像素掩码（Pixel Mask）建立直接反向传播梯度。既有像素级增强方案（如 LISA、PixelLM）大多参数冗重、推理缓慢且严重牺牲了模型的通用对话能力。小米 AI 实验室团队在 **`arXiv:2610.23279`** 中正式发布了全新轻量级旗舰 **`Xiaomi-Lens-0`**。\n\n### 架构亮点与底层机制\n1. **端到端原生掩码解码头（Native Pixel Mask Decoder）**：\n   - 架构以 **Qwen3.5 4B** 为基座，设计了一个极低开销的跨模态掩码解码器。该解码器同时接入三路信息流：① 视觉编码器的原始 Patch 特征金字塔；② 视觉投影隐向量；③ 语言模型输出的专用 `[MASK]` Token 状态。三者在隐空间进行交叉注意力聚合，直接输出高分辨率像素级二值分割掩码；\n2. **多阶段渐进式空间自监督对齐配方**：\n   - 第一阶段冻结视觉与语言基座，仅预训练掩码投影头；第二阶段联合微调多模态主干与解码器，在常规图文对话损失之外叠加动态 Focal Loss 与 Dice Loss，在完全保留通用感知能力的同时赋予模型像素级物理世界锚定能力。\n\n### 权威 Benchmark 与实测跑分对比\n在细粒度目标表达理解（REC）、表达语义分割（RES）与通用多模态评测中（Table 1, 3, 6, 8）：\n- **目标表达理解（REC, Prec@0.5）**：\n  - 在 `RefCOCO` 验证集上斩获 **94.6%**（超越 Florence-2 的 93.4% 与 Grounding-DINO-L 的 90.6%）；\n  - 在干扰项极多的 `RefCOCO+` 上达到 **92.1%**（大幅超越 Florence-2 的 88.3% 与 Grounding-DINO 的 82.8%）；\n  - 在复杂长句 `RefCOCOg` 上达到 **92.5%**（超越 Florence-2 的 91.2%）；\n- **表达语义分割（RES, cIoU 像素交并比）**：\n  - `RefCOCO`、`RefCOCO+` 与 `RefCOCOg` 分别斩获 **83.4%、79.9% 与 81.1%**，相较 7B 级基线 `LISA LLaVA-7B`（74.9% / 65.1% / 67.9%）与 `GSVA Vicuna-7B`（77.2% / 65.9% / 72.7%）实现跨代际碾压；\n- **通用多模态感知无损（General Perception）**：\n  - 在 `MME-RW-en` 真实世界基准中取得 **69.7%**，`MME-RW-cn` 达到 **67.5%**，`CV-B` 达到 **88.1%**，超越 7B 参数规模的开源模型 `Qwen2.5-VL 7B`（60.8% / 58.3%）。\n\n### 开发者实战落地与开箱指南\n- **论文与技术细节**：完整架构与消融数据详见 `arXiv:2610.23279`。\n- **端侧智能体选型建议**：Xiaomi-Lens-0 证明了 4B 级端侧模型结合专用掩码解码头即可在定位与分割上超越通用 7B~13B 模型。面向手机端相册智能编辑、具身机械臂视觉抓取或桌面 GUI 元素像素级识别的开发者，可直接借鉴其 `Patch Features + Mask Token` 融合解码范式，在毫秒级延迟下实现像素级视觉交互。","### 核心背景与行业痛点\nWhile modern Multimodal Large Language Models (MLLMs, e.g., Qwen-VL, LLaVA) perform impressively on holistic image question answering, they struggle with fine-grained spatial grounding and pixel-level segmentation. Conventional architectures rely on text-only or coarse bounding-box supervision, leaving token embeddings largely disconnected from spatial image structures. Existing segmentation-augmented models (such as LISA and PixelLM) typically append heavy external decoders that drastically increase latency and degrade conversational abilities. Researchers at Xiaomi AI Lab unveiled **`Xiaomi-Lens-0` (`arXiv:2610.23279`)**, a compact 4B multimodal model solving spatial grounding natively.\n\n### 架构亮点与底层机制\n1. **Native Pixel-Level Mask Decoder**:\n   - Built upon **Qwen3.5 4B**, Xiaomi-Lens-0 integrates a lightweight cross-attention mask decoder that combines three signals: vision-encoder patch embeddings, visual projection latents, and the language model's `[MASK]` token hidden states to directly output fine-grained pixel masks;\n2. **Multi-Stage Progressive Spatial Supervision**:\n   - Pretrains mask projection layers before jointly optimizing the full backbone with a combination of autoregressive next-token prediction, Focal loss, and Dice loss, preserving general vision-language conversational fluency while acquiring pixel-exact spatial precision.\n\n### 权威 Benchmark 与实测跑分对比\nEvaluated on Referring Expression Comprehension (REC), Referring Expression Segmentation (RES), and general perception benchmarks (Tables 1, 3, 6, 8):\n- **Referring Expression Comprehension (REC, Prec@0.5)**:\n  - Scores **94.6%** on `RefCOCO` (beating Florence-2 at 93.4% and Grounding-DINO-L at 90.6%);\n  - Hits **92.1%** on `RefCOCO+` (outperforming Florence-2 at 88.3% and Grounding-DINO-L at 82.8%);\n  - Reaches **92.5%** on `RefCOCOg` (vs Florence-2 at 91.2%);\n- **Referring Expression Segmentation (RES, cIoU)**:\n  - Achieves **83.4% on RefCOCO**, **79.9% on RefCOCO+**, and **81.1% on RefCOCOg**, decisively outperforming 7B-parameter architectures like `LISA LLaVA-7B` (74.9% / 65.1% / 67.9%) and `GSVA Vicuna-7B` (77.2% / 65.9% / 72.7%);\n- **General Multimodal Perception**:\n  - Achieves **69.7% on MME-RW-en**, **67.5% on MME-RW-cn**, and **88.1% on CV-B**, surpassing larger 7B models including `Qwen2.5-VL 7B` (60.8% / 58.3%).\n\n### 开发者实战落地与开箱指南\n- **Research Details**: Full architectural proofs and dataset splits are available in `arXiv:2610.23279`.\n- **Edge Deployment Takeaway**: Demonstrates that a 4B-parameter model with an integrated patch-and-mask-token decoder surpasses 7B–13B models in spatial grounding. Engineers building on-device photo editing, GUI element segmentation, or robotic pick-and-place systems can adopt this design to achieve sub-50ms pixel-level interactive segmentation.",[22],{"title":23,"url":23,"type":24},"https://arxiv.org/abs/2610.23279","arxiv",{"platform":26,"author_name":27,"author_handle":28,"author_avatar":29,"tweet_url":30,"published_at":31},"twitter","Xiaomi AI Lab","@huggingface","https://unavatar.io/x/huggingface","","2026-10-19T15:42:28.141Z","Research",93,62,0,{"models":37,"agents":38,"vendors":39},[],[],[],[41,42,43,44,45,46],"Xiaomi","Xiaomi-Lens-0","Qwen3.5","Multimodal","Segmentation","RefCOCO","https://arxiv.org/html/2610.23279v1/versatile_tasks.png",1793150087718]