BaseRT

About BaseRT

BaseRT is an LLM runtime built for Apple Silicon. It installs with a single command and runs local models directly on the user's device, removing per-token costs and keeping data private. The tool is open source and free.

Review

BaseRT focuses on speed for local LLM inference on M-series chips. The team reports up to 6.3× faster prompt processing (prefill) compared to llama.cpp and up to 3.9× faster than MLX on the M5 Pro, using the Metal 4 tensor API. These gains target the initial input-understanding phase, not necessarily all aspects of token generation.

Key Features

• Performance tuned for Apple Silicon, with Metal 4 tensor API optimizations yielding up to 6.3× faster prefill versus llama.cpp on M5 Pro.
• Single-command installation through the terminal.
• Local execution keeps all data on-device and removes per-token costs.
• Built-in CLI benchmarking tool measures tokens per second for different models on your specific hardware.
• Integrated model downloader fetches open-weight models directly from the command line.

Pricing and Value

BaseRT is free and open source. Because inference happens locally, there are no per-token charges and data stays on the user's machine.

Pros

• Measurable prefill speed gains on Apple Silicon, especially with M5 Pro and newer tensor cores.
• Straightforward setup via a one-line install command.
• No per-token billing and complete data privacy with on-device processing.
• Includes a benchmarking tool to assess performance on your specific hardware.
• Built-in model downloader simplifies fetching models without manual file handling.

Cons

• Exclusive to Apple Silicon; it won't run on Intel-based Macs, Windows, or Linux machines.
• Speed improvements center on prompt processing-token generation latency for real-time use cases may not see the same level of uplift.
• Lacks built-in model orchestration for dynamically switching between models based on latency or quality requirements; this is a planned feature, not yet available.

BaseRT fits developers and researchers who run LLMs on Apple Silicon and want faster prompt processing without per-token costs. It's a practical runtime for on-device agents or privacy-sensitive projects, though those needing automatic model switching or cross-platform support will find it limited today.

Open 'BaseRT' Website