Dustin: Sparse Verification for Efficient Long-Context Speculative Decoding

A new technique called Dustin, published on arXiv, addresses a key bottleneck in speculative decoding for large language models. While speculative decoding improves throughput for multi-batch long-context scenarios, its efficiency is often limited by KV cache loading dominating verification latency.

The paper identifies that existing compression methods fail in this regime: static eviction causes accuracy loss due to saliency shift, while dynamic selection introduces prohibitive computational overhead. Dustin's draft-augmented sparse verification strikes a new balance between accuracy and efficiency.

Published on arXiv cs.CL on June 25, 2026, this work is relevant as long-context LLMs become more common in applications like document analysis and codebase understanding, where inference efficiency is critical for practical deployment.