Liquid AI LFM2.5-24B-A2B Review: A 2B-Active Model Built for On-Device Agents
Liquid AI's largest and last LFM2.5 model trades knowledge for speed. We read the model card, the pricing and the benchmark gaps so you know where it fits.
Jump to 11 sections
Liquid AI's LFM2.5-24B-A2B is a 24-billion-parameter mixture-of-experts model that activates about 2.45 billion parameters per token and targets fast, private on-device agents. The model card reports about 184 tokens per second on an Apple M5 Max. It leads on instruction following and tool calling but trails Qwen3.6-35B-A3B on knowledge and coding benchmarks.
Model Drop reviews new models and tools against what their makers claim. This review covers the numbers on the Hugging Face model card, the pricing across providers and the limits Liquid AI itself states. It is for developers building local or low-latency agents who need to know whether this model deserves a test. We did not run our own benchmarks, so every score below comes from Liquid AI.
What Is LFM2.5-24B-A2B?
It is the largest and final model in Liquid AI's LFM2.5 family, according to the Hugging Face model card. The card describes a text-only reasoning model with about 2.45 billion active parameters per token, built for fast, private on-device agents. It supports 16 languages, including English, Chinese, Japanese and Hindi.
The architecture has 40 layers: 30 double-gated short convolution blocks and 10 grouped-query attention blocks. Each mixture-of-experts block has 64 experts with top-4 routing. The context window is 131,072 tokens with a 128K vocabulary.
The license is LFM 1.0, which you should read before shipping a product. The model always produces a reasoning step before its answer, so every response includes thinking tokens. That affects both latency and cost.
How Fast Is It?
Liquid AI reports about 184 tokens per second on an Apple M5 Max and 114 tokens per second on an AMD Ryzen AI Max+ 395. On a single NVIDIA B300 at high concurrency, it reports up to 21,000 output tokens per second. These are vendor numbers, and speed varies with quantization, prompt length and software.
The speed comes from the small active size. Only about 2.45 billion parameters compute for each token, though all 24 billion must fit in memory. Our explainer on mixture-of-experts versus dense models covers the tradeoff.
The model card lists FP8, GGUF, ONNX and MLX-4bit builds. A speculative-decoding draft model called DSpark is listed as coming soon. Our guide to speculative decoding explains why a draft model can raise speed further.
How Does It Score Against Qwen3.6-35B-A3B?
It wins on agent and instruction benchmarks and loses on knowledge and coding. The model card compares it with Qwen3.6-35B-A3B, a larger mixture-of-experts model. The split tells you where to use it.
| Benchmark | LFM2.5-24B-A2B | Qwen3.6-35B-A3B |
|---|---|---|
| LiveClaw (agentic) | 48.09 | 38.19 |
| IFEval (instruction following) | 96.75 | Not stated |
| BFCLv4 (tool calling) | 65.01 | Not stated |
| GPQA Diamond (knowledge) | 74.42 | 83.69 |
| SWE-Bench Verified (coding) | 51.00 | 66.20 |
The IFEval and BFCLv4 entries are listed as leading, but the card summary we read gives no Qwen figure for them. Treat the gaps on GPQA Diamond and SWE-Bench Verified as the clearest comparison. The Qwen model is ahead by about nine points on GPQA Diamond and about 15 on SWE-Bench Verified.
These are company-reported numbers. Independent results will appear as the community tests it, and they may differ.
What Does the Model Card Say It Is Bad At?
Liquid AI says the model is not recommended for knowledge-heavy tasks on its own. It suggests pairing it with retrieval or web search. That is a candid limit, and it matches the benchmark gap on GPQA Diamond.
The always-on reasoning step is another cost. Every answer includes thinking tokens, which add latency and, on hosted APIs, output cost. For simple lookups or short classifications, a non-reasoning model may be faster.
It is also text-only. If your agent needs images, screens or documents with layouts, you need a different model. Our look at on-device models for mobile covers options with other strengths.
What Does It Cost?
Pricing depends on the provider. LLM Reference lists Liquid AI's own API at $0.10 per million input tokens, $1.00 per million output tokens and $0.05 for cache reads. OpenRouter and Vercel AI Gateway list $0.05 input and $0.70 output.
Those prices are low next to frontier models, but reasoning tokens count as output. A response that thinks for 1,000 tokens and answers in 200 bills 1,200 output tokens. At $0.70 per million, that is about $0.00084 per response.
Local use has no per-token fee, but it has a hardware and electricity cost. If you run thousands of short agent steps a day, local inference can pay for itself quickly. For occasional use, a hosted API is simpler.
Who Should Use It?
Use it if you build agents that call tools, follow strict formats and need low latency or privacy. The strong BFCLv4 and IFEval results point to dependable structured output. Our comparison of function calling and structured outputs explains why that matters.
Skip it if your task depends on broad world knowledge, heavy coding or multimodal input. A larger model, or this one paired with retrieval, fits those better. Teams with strict data rules may like that the weights run on a laptop, since prompts and files never leave the device. That privacy benefit is one of the clearest reasons to pick a local model over a hosted one.
Model Drop's editors would test it on a narrow, well-defined agent step first. Pick one tool-calling task you run today, and compare accuracy, latency and cost with your current model.
How Should You Set It Up?
Follow the model card's recommended generation settings: temperature 0.1, top-k 50 and a repetition penalty of 1.05. Those low-temperature settings suit tool calling, where you want consistent output. Use the Pythonic or JSON function-call formats it was trained on.
Start with the quantized build that fits your hardware. The GGUF and MLX-4bit versions are the most practical for laptops. Keep at least 24GB of memory free for comfortable local runs of a 24B model.
Watch the thinking tokens. Set a maximum output length so the reasoning step does not run long, and log how many tokens each step uses. That is the main lever on both speed and cost.
Why Does the Architecture Matter?
The design mixes two block types to keep memory and compute low. Short convolution blocks handle local patterns cheaply, while a smaller number of attention blocks handle long-range links between tokens. Only 10 of the 40 layers use attention, which cuts the memory a long context needs.
That matters on laptops. Attention layers store a growing cache as the conversation lengthens, and that cache is often the limit on local runs. A model with fewer attention layers can keep a long context in less memory than a standard transformer of similar size.
The mixture-of-experts design adds the other half. With 64 experts and top-4 routing, each token passes through only a few of them. You get the knowledge capacity of a 24B model with the per-token compute of a much smaller one, as long as all the weights fit in memory.
How Does It Compare With Other Small Models?
The model card compares it with only one rival, so context from our earlier testing helps. Our roundup of small models that run on one GPU shows that most small models give up either speed or accuracy. LFM2.5-24B-A2B leans hard toward speed and format discipline.
Its training adds weight to that focus. Liquid AI says post-training combined supervised fine-tuning, on-policy distillation and agentic reinforcement learning inside agent harnesses such as OpenClaw and Pi. That is practice on the kind of loops where the model will be used.
Total training reached about 36 trillion tokens by the card's count, though third-party listings give a different figure for pretraining alone. Use the model card as the source of truth and check it again if the numbers matter for your work.
What Should You Check Before Deploying It?
Check five things before you ship. Confirm the license terms fit your business. Measure speed on your own hardware. Test accuracy on your real tool schemas. Set a cap on thinking tokens. Plan a fallback for tasks that need broader knowledge.
Build a small evaluation set from real agent traces. Include easy steps, edge cases and tasks where the right answer is to refuse or ask a question. A model that scores well on public benchmarks can still fail on your own formats.
Keep the evaluation running after launch. Models behave differently when quantized, and a new build can shift results. A weekly run of the same test set catches regressions early. Record the model build, quantization and settings with every run so you can explain any change in results to your team.
Verdict
LFM2.5-24B-A2B is a focused tool, not a general replacement for a large model. It is fast, cheap and strong at following instructions and calling tools, and it is honest about its weak spots in knowledge and coding. For on-device agents, it earns a test.
Next step: download a quantized build, run ten of your real tool-calling tasks, and compare accuracy and latency with your current model before you commit.
