AI tool
Soup CLI
Soup CLI is a command-line tool for fine-tuning large language models on computers with limited GPU memory. It streams frozen base model layers from system RAM during LoRA training, allowing models that exceed GPU capacity to be trained. It is int...
About Soup CLI
Soup CLI is a command-line tool for fine-tuning large language models, launched this week. It targets users who want to train models on hardware with limited GPU memory, specifically citing a 4 GB laptop GPU. The tool is open source under Apache-2.0 and supports several post-training methods.
Review
Soup CLI takes a specific approach to a common bottleneck in LLM fine-tuning: GPU memory. Because LoRA keeps the base model frozen and read-only, Soup stores that model in system RAM and streams it into the GPU one decoder layer at a time. This keeps peak VRAM usage down to a single layer rather than the entire model.
The developer reports a measured result on an RTX 3050 Laptop 4 GB: Llama-3.1-8B trains at 119.6 tok/s in 3.32 GB peak VRAM. The project's stated focus is on correctness as much as speed, with a protocol that compares streamed runs against resident runs and requires logits to match exactly.
Key Features
- Layer-streaming for LoRA fine-tuning, moving the frozen base model from GPU to system RAM
- Multiple training methods in one YAML configuration: SFT, DPO, GRPO, and KTO
- Built-in evaluation, gating, and export functionality
- Apache-2.0 license with all measurements published in the repository, including failed runs
- Correctness verification protocol that matches streamed logits against a resident run
Pricing and Value
The tool is listed as free and open source under Apache-2.0. There is no paid tier or subscription model mentioned in the available information. The value for users comes from the ability to run fine-tuning on consumer-grade hardware, avoiding cloud GPU costs for local iteration. The project also hosts a web presence at trysoup.dev on Vercel, though pricing for that service is not defined in the reference material.
Pros
- Runs 8B parameter fine-tuning within 4 GB VRAM, a practical threshold for many laptops
- Publishes all benchmark numbers, including those that revealed bugs in released code
- Supports multiple training objectives (SFT, DPO, GRPO, KTO) from a single YAML file
- Includes a reproducibility check that catches silent autograd failures
Cons
- Correctness validation has been tested against full-precision base models; compatibility with 4-bit quantized bases is not yet confirmed
- Streaming adds complexity compared to standard fine-tuning workflows, and the developer has already found and fixed one gradient bug above a certain layer size
- Not well suited for users who need to fine-tune very large models (100B+ parameters) or who lack familiarity with command-line YAML configuration
Soup CLI fits users who want to iterate locally on LLM fine-tuning without cloud GPU spend, particularly those on 4 GB laptops. The published correctness protocol and open measurements make it a reasonable choice for researchers who need to verify training integrity. Teams already comfortable with CLI tools and Hugging Face ecosystems will find the workflow straightforward, while those expecting a graphical interface or turnkey solution should look elsewhere.
Jobs Soup CLI suits
We checked Soup CLI against 500+ jobs; these get the most out of it.
Alternatives to Soup CLI
Other tools that do the same main job. Our AI compared Soup CLI with similar tools and kept the ones you could use instead.
