NVIDIA Dory 1.0-4B Review: A Looped Model With Adjustable Depth
NVIDIA lets you choose how many times Dory reuses its core layers at serving time. Here is what one, two, and four loops buy you.
Jump to 8 sections
NVIDIA Dory 1.0-4B is an open-weight, 4-billion-parameter language model that reuses a shared block of layers several times per token. You pick one, two, or four loops at serving time, with no retraining. NVIDIA recommends two loops, and its numbers show the biggest gains on coding and agent tasks.
This review is for developers who run small models locally or on modest GPUs. Model Drop tests launches against what the vendor claims, so we compared the published numbers across loop counts and flagged where the evidence is thin.
What Is a Looped Model, and How Does Dory Work?
A looped model repeats part of its network instead of stacking more unique layers. MindStudio's overview says Dory splits its layers into three groups: eight input layers, a recurrent block of 20 layers, and eight output layers. The recurrent block runs multiple times per forward pass, and the original input is injected again on every pass.
The parameter count stays fixed at 4 billion, while compute per token scales with the loop count. Attention alternates between sliding-window and full attention, and NVIDIA pretrained the model natively in NVFP4, a 4-bit format, before continuing in BF16.
The practical idea is simple. Instead of choosing a bigger model for a harder task, you give the same model more thinking passes. Easy requests can use one loop, and hard ones can use four.
Looping is not brand new as a research idea, but shipping it as an open model with a runtime dial is unusual. Most small models give you one fixed cost per token. Dory gives you a setting, and that changes how you think about deployment. You can match compute to the request instead of paying the same price for every prompt.
How Do One, Two, and Four Loops Compare?
More loops help most on coding and agent work, and they do little for knowledge tests. The numbers below come from NVIDIA's reported results, as summarized by DataStudios.
| Benchmark | 1 loop | 2 loops | 4 loops |
|---|---|---|---|
| SWE-bench Verified | 50.3% | 59.5% | 61.8% |
| Terminal-Bench 2.1 | 12.5% | 18.1% | 22.2% |
| GPQA Diamond | 69.19% | 72.41% | 72.16% |
| IFBench | 74.29% | 77.94% | 79.00% |
| AA-Omniscience accuracy | 10.03% | 10.08% | 10.03% |
The jump from one loop to two is the big one. SWE-bench gains 9.2 points there and only 2.3 more at four. GPQA Diamond and Humanity's Last Exam peak at two loops and dip slightly at four, while AA-Omniscience, a factual-recall test, barely moves.
The lesson is that looping adds reasoning depth, not knowledge. A model cannot loop its way into facts it never learned. For tasks that depend on recall, a larger model or retrieval will help more than extra loops.
Think of the loop count as a quality knob for agent work. At one loop, the model scores 12.5% on Terminal-Bench 2.1, which tests command-line tasks. At two loops it scores 18.1%, and at four it reaches 22.2%. That is a near doubling from the lowest to the highest setting, which is a large move for a model that never changes size.
What Does Extra Looping Cost?
Extra loops cost compute. DataStudios counts total layer executions of 36 at one loop, 56 at two, and 96 at four. Four loops involve about 2.67 times the layer executions of one, but that is a structural count, not a measured latency ratio.
NVIDIA also says higher loop counts let the model solve the same tasks with fewer generated output tokens. If that holds on your workload, some of the extra compute is repaid. MindStudio notes, however, that whether total latency and cost rise or fall depends on the workload and serving stack.
Our advice matches NVIDIA's. Measure end-to-end latency and cost on your own prompts at one, two, and four loops before you choose. A pretty benchmark gain is not worth it if your users wait twice as long.
A simple routing policy follows from this. Use one loop for short, low-risk requests such as classification or formatting. Use two loops as the default for general work. Reserve four loops for long agent runs, code repair, or multi-step tool use, where a wrong step is expensive. A router can pick the setting from the task type or from a quick difficulty estimate. If you already use tricks such as speculative decoding to speed up generation, test them alongside loop counts, since both change the speed and cost balance.
How Does Dory Stack Up Against Other Small Models?
Dory is strong for its size on agentic work and weaker on broad knowledge. At four loops it beats Qwen3.5-4B, which scores 54.7% on SWE-bench Verified, and edges Qwen3.5-9B at 61.0%. It does not catch the bigger mixture-of-experts models.
| Model | SWE-bench Verified | Terminal-Bench 2.1 |
|---|---|---|
| Dory 4B, 4 loops | 61.8% | 22.2% |
| Qwen3.5-4B | 54.7% | 15.0% |
| Qwen3.5-9B | 61.0% | 24.0% |
| Gemma 4 26B A4B | 57.4% | 35.0% |
| Qwen3.6-35B-A3B | 73.4% | 43.0% |
Qwen3.5-9B leads Dory on Terminal-Bench 2.1, and Gemma 4 and Qwen3.6-35B-A3B lead it by a wide margin. DataStudios notes that some comparator figures were reported rather than measured under identical conditions. Read the table as a rough position, not a ranking.
For wider context on small models, see our guide to small models that fit on a single GPU. Dory fits that category, with an unusual way to spend extra compute.
Can You Run Dory Locally?
Yes, with some setup. NVIDIA published BF16, MXFP8, NVFP4, and Q5_K_M GGUF builds. DataStudios lists checkpoint sizes of about 7.97 GB for BF16, 4.77 GB for MXFP8, and 3.67 GB for the GGUF build.
Runtime memory is larger than file size. It also includes KV caches, activations, and engine overhead, and it grows with context length and concurrent requests. Dory's last-loop KV caching keeps only the final pass's cache for recurrent attention layers, so memory stays near the one-loop level. NVIDIA reports accuracy stays close to per-loop caching.
Serving support is uneven. MindStudio says llama.cpp supports Dory, including a build path for Apple Silicon, while vLLM support was pending an official release and could be installed from an open pull request. If you run a server stack on vLLM, plan for a custom build. Our comparison of local AI inference servers can help you pick a runtime.
Hardware planning is easier for Dory than for most larger models. The 4-bit and 8-bit builds are small enough for a single consumer GPU or a recent laptop, and the theoretical weight size for a 4-billion-parameter model is about 2 GB at 4-bit and 4 GB at 8-bit, before overhead. That makes it a practical choice for teams that want to keep data on their own machines. Our guide to on-device AI models for mobile covers the wider trade-offs of running models locally.
Long context is the main memory risk. Running at the full 256K window can need far more memory than a short chat, because the KV cache grows with the number of tokens. If you plan to use long documents, test memory at your real context length, not at a short demo prompt.
What Are the Caveats?
The caveats are real. All the benchmark figures are NVIDIA's own, and independent testing is still needed to confirm production gains. DataStudios points out that the 256K context window does not guarantee uniform retrieval across the whole span.
Native 4-bit pretraining is also described as harder to stabilize than post-training quantization. That is a research caution more than a user problem, but it is a reminder that the recipe is new.
The license is OpenMDW 1.1, which DataStudios says covers commercial and non-commercial use subject to its terms. It does not detail those terms, so read the license before you ship a product.
Model Drop's editors also want to see cost-per-success numbers measured by an independent party. DataStudios provides some, but they rest on invented cost assumptions rather than NVIDIA measurements, and under those assumptions one loop has the lowest cost per success. That is a useful reminder that a higher benchmark score does not automatically mean a better bill.
Who Should Try Dory?
Developers building local coding helpers, agents, or tool-calling workflows should try it. The adjustable depth suits mixed workloads, where most requests are simple and a few are hard. You can route by difficulty and save compute on the easy ones.
Teams that need strong factual recall should look elsewhere. Dory's own results show no gain on AA-Omniscience from extra loops. Pair it with retrieval, or choose a larger model.
Start with a small test plan:
- Run your own tasks at one, two, and four loops.
- Record quality, latency, and memory at each setting.
- Try last-loop KV caching to hold memory flat.
- Compare against your current small model on the same prompts.
Conclusion
Dory 1.0-4B is a useful experiment in spending compute on demand. Two loops looks like the sweet spot, four loops helps on hard agent tasks, and nothing here fixes missing knowledge. The serving story is still maturing.
Your next step is a short local test on your own prompts. Model Drop will revisit Dory when independent benchmarks and stable serving support arrive.
