French AI Startup ZML Open-Sources LLMD to Accelerate Inference Across AI Chip Clusters

ZML, a French AI startup that has earned an endorsement from Turing Award winner Yann LeCun, has officially released ZML/LLMD, an open-source software product designed to accelerate AI inference across large clusters of AI chips. The release marks a shift from concept to product for the company.

ZML/LLMD targets the core bottleneck in large-scale AI inference: the efficiency of communication and scheduling across hundreds or thousands of AI chips. ZML claims its software can better harness the distributed computing power of AI accelerators, reducing both inference latency and operational costs for large language model deployments.

By releasing ZML/LLMD as free open-source software, ZML aligns with the broader industry trend where software optimizations are key to unlocking hardware performance. The approach could lower the barrier for enterprises and developers to run large models in production environments.

ZML has been one of the most closely watched AI startups in France, and LeCun's public backing gave it early credibility. With LLMD now publicly available, the company faces the real test of delivering on its performance promises.

As AI inference demand grows exponentially, any technology that meaningfully reduces per-token costs carries strategic value. ZML/LLMD's success will hinge on benchmark results across real-world hardware configurations.

Watch for independent benchmarks comparing ZML/LLMD against established inference engines like vLLM and TensorRT-LLM.