ZML, a Paris-based AI startup endorsed by Turing Award winner Yann LeCun, has unveiled ZML/LLMD — a new large language model inference server designed to run open-source models across a wide range of chips. The supported hardware includes offerings from Nvidia, AMD, Google's TPU, Apple Metal, and Intel Arc.
The company's founder, Steeve Morin, told TechCrunch that the goal is to dismantle existing silos that restrict AI workloads to specific hardware. By enabling different chips to operate at their maximum available speed — and sometimes faster — ZML aims to give enterprises and cloud providers the freedom to build systems using a mix of processors that may be less expensive or more energy-efficient.
The launch comes at a time when inference — the processing of user prompts — has been growing more critical than model training. As AI becomes embedded in everyday workflows, the software and architecture barriers that lead to vendor lock-in have become increasingly problematic, according to Morin.
A Potential Market Disruptor
The promise of achieving peak performance across diverse chip architectures represents both a technological achievement and a potential market disruptor. With AI-related costs mounting, the ability to mix and match hardware could reshape how organizations approach infrastructure.
Morin noted that ZML's software could also benefit emerging AI chipmakers, many of which are based in Europe. He cited companies including Axelera, Fractile, Kalray, OLIX, Q.ANT, SiPearl, SpiNNcloud, and VSORA as potential collaborators. However, he emphasized that geographic origin matters less than the opportunity to work on unprecedented innovations.
Despite positioning ZML as a cross-platform solution, Morin clarified that he remains positive about Nvidia. The startup maintains a good relationship with the chip giant, which has been preparing for the growing demand for inference. Nvidia's existing supply advantage is also a factor, he acknowledged.
Competing in the Inference Gold Rush
Inference has become the focus of such intense investment that observers have dubbed the trend an "inference gold rush." ZML faces competition from several well-funded players. Baseten was recently valued at $13 billion, while Inferact was launched by the creators of the open-source project vLLM. RadixArk, the commercial entity behind SGLang, is another contender.
Both vLLM and SGLang partially overlap with ZML/LLMD's functionality, but Morin envisions a broader scope. He revealed that ZML has reached the point of co-designing silicon, signaling ambitions that extend well beyond inference optimization.