Hugging Face Delivers Native-Speed vLLM Backend for Transformers Models

Hugging Face announced on July 8 a major performance milestone: the Transformers library's vLLM modeling backend now matches or exceeds the speed of vLLM's hand-written native implementations. Model authors can automatically leverage their Transformers implementations to get ultra-fast vLLM inference without any porting effort.

The performance claims are backed by rigorous benchmarks across three very different Qwen3 models: a 4B dense model on a single GPU, a 32B dense model with tensor parallelism, and a 235B-parameter FP8 Mixture-of-Experts model running data+expert parallelism on an 8×H100 node. In every case, the Transformers backend met or beat native throughput.

Using the Transformers modeling backend requires just a single flag: --model-impl transformers. It composes with all of vLLM's usual parallelism options, so no serving configuration changes are needed.

The Transformers library supports over 450 architectures through consistent APIs, with the design principle that model implementations are self-contained and easy to understand. Last year, Hugging Face first integrated Transformers as a modeling backend in vLLM, allowing model authors to run Transformers models inside vLLM without porting. This update closes the performance gap.

Models using linear attention are not currently supported, though support is coming soon. Custom models with code living in Hub repos are unlikely to work as they may not be written compliantly.

This advancement has significant implications for the AI inference infrastructure ecosystem. vLLM is one of the industry's most popular high-performance inference engines, and Transformers is the framework where most new models are first implemented. Their seamless high-performance integration dramatically reduces the cost of moving from model research to production deployment.