What is Rapid-MLX?
Rapid-MLX is an open-source inference server for running language models locally on Apple Silicon Macs. It provides OpenAI- and Anthropic-compatible APIs and is built to work with coding agents such as Codex, Claude Code, and OpenCode, as well as other compatible clients. A command-line launcher can select a model suited to the Mac and connect an agent, while a separate macOS desktop app offers a graphical way to chat and manage models.
Rapid-MLX Features:
Local Model Server: Run supported open models on an Apple Silicon Mac through a server built on Apple's MLX framework.
Compatible APIs: Expose OpenAI-compatible chat and responses endpoints alongside an Anthropic Messages-compatible endpoint for local clients.
Agent Launcher: Launch supported coding agents from the command line with a local model, using a private profile that can be removed without changing existing Codex settings.
Model Selection: Choose from models covering text, vision, speech, and video, with a one-command option that picks a model suited to the Mac.
Request Handling: Use continuous batching to process concurrent requests, with a radix prefix cache that can save cached state to disk at shutdown and restore it at startup.
Desktop App: Use the macOS app to chat locally, manage models, and work with vision, files, voice, and image generation.
Rapid-MLX Benefits:
Keep Work Local: Running models on the Mac keeps model use on the user's own machine rather than requiring a hosted inference service.
Reuse Existing Clients: Compatible API formats let users connect supported agents and apps without replacing their existing client workflow.
Less Setup: The launcher handles model choice and agent connection in one command, reducing the amount of manual configuration.
More Responsive Serving: The project reports faster decoding than Apple's MLX on measured tasks, which can make local model responses arrive sooner.
Flexible Access: A command-line server and a desktop app give users a choice between agent-driven workflows and a graphical chat interface.
Use Cases:
Local Coding Agents: Run Codex, Claude Code, or OpenCode against a model on a Mac for coding-agent workflows.
App Prototyping: Point an application that accepts an OpenAI- or Anthropic-compatible endpoint at a local model while developing or testing.
Private Model Chat: Use the desktop app to chat with a local model and work with supported files, voice, vision, or image generation.
Mac Model Evaluation: Compare local model-serving speed on Apple Silicon using Rapid-MLX's published benchmarks and measured results.
Concurrent Requests: Serve multiple compatible clients or sessions from one local server using its continuous-batching support.