Google's Gemma team says Gemini 4 Argon wrote vLLM optimizations that speed up gemma-4-E2B-it 2.85x on an A100

Background and pain points

After an open model ships, specialized kernels in inference engines often lag by weeks or months, and small models suffer most from generic code paths. On Oct 26, Google's official @googlegemma account said the team used Gemini 4 Argon to build a set of Gemma 4 optimizations for vLLM, as a demonstration that a frontier coding model can take on low-level performance engineering.

Architecture and mechanism

The PRs were filed by Luciano Martins of Google DeepMind. PR #60674 adds fused Triton kernels for QKV norm/RoPE, pre/post-FFN norms, PLE and MoE routing. PR #60689 optimizes MTP speculative decoding with a sparse gather-GEMV kernel that cuts intermediate memory allocation. PR #60445 auto-enables KV-sharing fast prefill and adds piecewise CUDA Graph capture.

Benchmarks

Google says gemma-4-E2B-it on a single A100 went from about 37,900 to 108,000+ tokens/sec, roughly 2.85x, with 100% token parity. That figure covers the whole set. PR #60674 alone self-reports up to 1.90x prefill and about 1.06x decode on E2B, and only 1.04x to 1.05x on larger Gemma 4 models. All numbers are the author's own measurements with no third-party reproduction yet.

Getting started

As of Oct 27, only 3 of the 10 PRs in the linked list are merged, so release builds of vLLM do not yet get the full gain. Track the PRs, or test the branches in a staging environment and compare throughput and output parity on your own workload before production.