Soup CLI

About Soup CLI

Soup CLI is a command-line tool for fine-tuning large language models, launched this week. It targets users who want to train models on hardware with limited GPU memory, specifically citing a 4 GB laptop GPU. The tool is open source under Apache-2.0 and supports several post-training methods.

Review

Soup CLI takes a specific approach to a common bottleneck in LLM fine-tuning: GPU memory. Because LoRA keeps the base model frozen and read-only, Soup stores that model in system RAM and streams it into the GPU one decoder layer at a time. This keeps peak VRAM usage down to a single layer rather than the entire model.

The developer reports a measured result on an RTX 3050 Laptop 4 GB: Llama-3.1-8B trains at 119.6 tok/s in 3.32 GB peak VRAM. The project's stated focus is on correctness as much as speed, with a protocol that compares streamed runs against resident runs and requires logits to match exactly.

Key Features

• Layer-streaming for LoRA fine-tuning, moving the frozen base model from GPU to system RAM
• Multiple training methods in one YAML configuration: SFT, DPO, GRPO, and KTO
• Built-in evaluation, gating, and export functionality
• Apache-2.0 license with all measurements published in the repository, including failed runs
• Correctness verification protocol that matches streamed logits against a resident run

Pricing and Value

The tool is listed as free and open source under Apache-2.0. There is no paid tier or subscription model mentioned in the available information. The value for users comes from the ability to run fine-tuning on consumer-grade hardware, avoiding cloud GPU costs for local iteration. The project also hosts a web presence at trysoup.dev on Vercel, though pricing for that service is not defined in the reference material.

Pros

• Runs 8B parameter fine-tuning within 4 GB VRAM, a practical threshold for many laptops
• Publishes all benchmark numbers, including those that revealed bugs in released code
• Supports multiple training objectives (SFT, DPO, GRPO, KTO) from a single YAML file
• Includes a reproducibility check that catches silent autograd failures

Cons

• Correctness validation has been tested against full-precision base models; compatibility with 4-bit quantized bases is not yet confirmed
• Streaming adds complexity compared to standard fine-tuning workflows, and the developer has already found and fixed one gradient bug above a certain layer size
• Not well suited for users who need to fine-tune very large models (100B+ parameters) or who lack familiarity with command-line YAML configuration

Soup CLI fits users who want to iterate locally on LLM fine-tuning without cloud GPU spend, particularly those on 4 GB laptops. The published correctness protocol and open measurements make it a reasonable choice for researchers who need to verify training integrity. Teams already comfortable with CLI tools and Hugging Face ecosystems will find the workflow straightforward, while those expecting a graphical interface or turnkey solution should look elsewhere.

Open 'Soup CLI' Website