AI news ·
Liquid AI releases experimental draft model for LFM2.5-VL-3B that accelerates vision-language inference up to 3.13x on device
Liquid AI's LFM2.5-VL-DSpark boosts its 3B vision-language model decoding up to 3.13x on-device by adding 280 million parameters, an 8.9% memory increase, with no output quality change.
Liquid AI released LFM2.5-VL-DSpark on September 24, 2026, an experimental speculative decoding drafter for its 3-billion-parameter vision-language model. The addition trades a small memory increase for decode speedups up to 3.13x on device and 2.66x on an H100 GPU, without altering output quality.
The drafter adds 280 million parameters - an 8.9% increase over the base model - and ships with day-one support in llama.cpp, MLX-VLM, and SGLang. End-to-end latency gains reach 2.62x on Apple Silicon and 2.27x on H100, depending on the workload.
How speculative decoding works for vision-language models
The vision drafter uses the same architecture as Liquid AI's text DSpark drafters. It captures the target model's hidden states at a fixed set of tapped layers and conditions on them to draft a block of candidate tokens. Image patches and text tokens are projected into a shared representation before those layers, so the drafter operates on hidden-state vectors of identical dimensionality regardless of input modality. The inference algorithm is unchanged from the text models.
Training and architecture choices
The team followed the DSpark recipe with a mixture of vision-language supervised fine-tuning data, weighted toward expected production workloads. After testing configurations with 3, 4, and 5 layers, they settled on a simplified attention-only drafter with 4 layers and a block size of 9. Training ran for 10 epochs on the final mixture, with acceptance rates improving before hitting diminishing returns. At inference, Liquid AI recommends a block size of 8 or 9 depending on hardware.
The parameter breakdown: the decoder stack accounts for 193 million parameters, the hidden-state projection adds 21 million, and the Markov head contributes 65.5 million. Norms and the confidence head total just 6,400 parameters.
Performance across CPU and GPU
Measurements used a DSpark block size of 8 across six vision tasks from the MMSpec benchmark: general VQA, text VQA, image captioning, chart VQA, complex reasoning, and multi-turn conversation.
On an M5 Max with MLX, decoding ran 2.30x to 3.13x faster by task, with end-to-end latency improving 1.56x to 2.62x. On an M3 Ultra with llama.cpp, decoding improved 1.57x to 2.14x and end-to-end by 1.30x to 1.77x. On an H100 GPU, decoding speedups ranged from 2.04x to 2.66x, with end-to-end gains of 1.64x to 2.27x.
Where speculation hits its limits
Speculative decoding accelerates only the decode phase - not vision encoding or prefill. VLMs add overhead because the image must pass through a vision encoder, after which the language backbone processes hundreds of visual tokens alongside the text prompt. On edge devices with less compute than datacenter GPUs, prefill consumes a larger share of end-to-end latency. This is Amdahl's law at work: even a large decode speedup yields only a modest overall gain when other stages dominate wall time.
Running the model
For SGLang, users launch the target model with the drafter attached using the --speculative-algorithm DSPARK flag and pointing --speculative-draft-model-path to the DSpark weights. The block size is read from the draft model's config.json. In llama.cpp, the server command specifies the draft model with -md and sets --spec-type draft-dspark. MLX-VLM requires passing --draft-model to the server command. Speculative decoding is exact: the target verifies every proposed token, so greedy output matches the target model alone.
Why this matters for developers and product teams
Faster vision-language inference with no quality trade-off changes what's practical on-device. A 2.62x end-to-end speedup on Apple Silicon means real-time VQA and captioning can run locally without cloud latency. For teams building multimodal features, the 8.9% parameter overhead is small enough to fit within existing memory budgets. The day-one support across three inference stacks also removes the usual wait for ecosystem catch-up. Anyone integrating VLMs into production applications can test the speedup immediately with a drop-in model swap.