How to run local LLMs on Mac with Apple Silicon and MLX

If you use a Mac with an M1, M2, M3, or M4 chip, you're no longer stuck relying on ChatGPT, Claude, or Gemini in the cloud to leverage AI. Apple's ecosystem has matured enough that you can run large language models (LLMs) directly on your machine, with very decent performance, good speed, and, most importantly, with your data always under control.

The key to this leap lies in MLX, Apple's machine learning framework, and how it integrates with tools like mlx-lm, Ollama, LM Studio, and even FileMaker 2025. Thanks to Apple Silicon's unified memory and an optimized inference engine, it's now perfectly feasible to set up code assistants, local RAG systems, text-to-speech, and entire enterprise workflows without sending a single byte to external servers.

What is MLX and why does it matter in Apple Silicon?

MLX is an open-source framework created by Apple and designed specifically to leverage the capabilities of Apple Silicon architecture . Unlike PyTorch or TensorFlow, which attempt to be cross-platform and adapt to a wide range of configurations, MLX focuses on the M1-M4 chips and takes advantage of two key features: unified memory and Metal GPU acceleration.

Unified memory means that the CPU and GPU share the same RAM space, so the data loads of a model don't have to jump from one place to another. This translates to fewer data copies, lower latency, and better memory utilization —crucial when dealing with models that can occupy tens of gigabytes.

The model format associated with MLX typically uses .npz (NumPy Zip) files to store compressed weights, along with a config.json file and tokenizer files. Popular models like Mistral, Phi-2, or LLaMA 3 can be converted to this format and run directly on your Mac without relying on external APIs.

Apple has made a serious commitment to MLX: it has released it under an open license, integrated it with Apple Intelligence in macOS, iOS, and iPadOS , and connected it to Swift via a native API. At WWDC 2025, MLX was presented as the official foundation for working with custom models in the Apple ecosystem, making it clear that it is not a passing experiment, but a strategic component.

In practice, this approach makes MLX the most logical choice for those seeking a well-integrated, local AI stack on Mac , with long-term viability and without relying on a third party to change the rules of the game or the prices of its APIs.

Model Formats: MLX vs. GGUF and the Role of Conversion

In the world of local LLMs, the weight format makes a significant difference in performance. One of the de facto standards is GGUF , the successor to the older GGML, which is used by Ollama, llama.cpp, and LM Studio. GGUF is optimized for local execution and aggressive quantization, and it works wonderfully in llama.cpp-based environments.

However, GGUF is not designed for the MLX memory model. Users who have been experimenting with models on M-chip Macs for some time have observed that converting these GGUF models to the native MLX format significantly improves both throughput (tokens per second) and memory usage, especially in long-running scenarios where bandwidth is critical.

The typical workflow for converting a model from GGUF to MLX includes several steps: analyzing the GGUF header (magic bytes, number of tensors, metadata), mapping or reordering the weights to fit the MLX tensor layout, generating the configuration files (config.json, tokenizer, etc.), and finally, using the mlx_lm.convert utility for architectures that MLX natively supports.

Since 2026, Ollama has begun integrating MLX as its preferred backend on Macs with Apple Silicon, confirming a clear ecosystem shift towards convergence between optimized formats (GGUF) and native execution with MLX . The conversation in technical communities increasingly revolves around which models convert best, which quantizations can handle quality loss, and how to leverage MLX to minimize overhead.

Meanwhile, both worlds continue to coexist: GGUF remains the convenient standard for downloading ready-to-use templates from Hugging Face or Ollama, while the MLX format prevails when seeking maximum performance on Mac and being willing to take an extra step in configuration.

Current situation: Apple Intelligence, MLX, and Apple Silicon hardware

With the arrival of Apple Intelligence in 2024, the company took a significant step by bringing AI capabilities directly to the operating system : writing assistance, email summaries, image editing, semantic searches, and more. Many of these tasks run entirely locally on devices with an M1 chip or higher, leveraging the integrated inference engines and unified memory.

These advancements are not available for Intel-based Macs, marking a clear division between the older generation and the Apple Silicon era. On M1-M4 systems, however, Apple is deploying its own foundational models within the system, while promoting MLX as the official platform for developers and advanced users to work with their own custom models.

At the hardware level, the combination of an efficient CPU, integrated GPU, and unified memory makes Macs a very solid option for local inference, although they don't compete with high-end dedicated GPUs for massive training. Unofficial community benchmarks show 20-30% improvements in generation speed when using MLX versus llama.cpp on Metal on the same machine, something that is especially noticeable in projects that require fast interaction, such as code assistants.

Furthermore, Apple has been introducing new generations of chips (up to and including the M4), with performance improvements specifically designed for machine learning workloads. On machines with 64GB, 96GB, or even 128GB of unified memory, it's possible to run large models that would require very expensive dedicated GPUs on a PC, making a Mac mini or a powerful laptop a viable option as a local AI server for a small team.

Looking ahead, everything suggests that MLX will continue to gain importance in the Apple ecosystem, both in terms of system functionalities and developer tools, establishing itself as the de facto standard for local AI on Mac.

How to use MLX in practice: installation and first steps

Using MLX today doesn't require being a machine learning guru, but it does require at least a basic level of comfort with the Terminal and Python. The good news is that the basic workflow can be summarized in just a few commands, and once configured, the experience is quite straightforward.

The first step is to meet the minimum requirements: a Mac with Apple Silicon (M1 or later), macOS 13 Ventura or later , an internet connection to download the templates, and some disk space (around 8-10 GB for small templates, considerably more for large ones). Terminal comes pre-installed (in Applications > Utilities).

MLX relies on Python. Many macOS installations already include Python 3, but it's a good idea to check with `python3 --version` . If no version is listed or it's too old, the usual solution is to install Homebrew and, with it, the appropriate version of Python. Once you have Python ready, you can install the mlx-lm package with a simple `pip3 install mlx-lm`.

From that point on, mlx-lm takes care of almost everything: it downloads compatible models from Hugging Face, manages quantization, and launches chat sessions from the command line. For example, to test a Mistral model configured for instructions, simply run something like mlx_lm.chat –model mlx-community/Mistral-7B-Instruct-v0.3-4bit , and the system will download the necessary files, convert them to the appropriate format, and run the model.

Once downloaded, the model is cached locally, making subsequent runs much faster and, best of all, eliminating the need for an internet connection. From there, you can integrate MLX into Python scripts, Swift projects, or even build it behind your own API to connect it with other applications.

Comparison: MLX versus Ollama, llama.cpp and LM Studio

MLX doesn't exist in isolation. In the landscape of local LLMs for Mac, it coexists with popular tools like Ollama, llama.cpp, and LM Studio , each with its own personality, advantages, and limitations. Understanding how they fit together helps you choose the right "engine" for your use case.

Ollama has become the easiest gateway to local LLMs . With just a few commands, you can download models in GGUF format, run them, and expose an OpenAI-compatible API on localhost. For developers, it's incredibly convenient: any code that already communicates with the OpenAI API can be pointed to Ollama simply by changing the base URL.

Llama.cpp is the C/C++ inference engine that powers Ollama and many other tools. It is extremely flexible and highly optimized, but its direct use is often more cumbersome, and it is best suited for those who need fine-grained control over compilation, advanced options, or deployments on very limited devices.

LM Studio, for its part, offers a native graphical interface for Mac that simplifies the downloading, management, and execution of models . It's ideal for those who don't want to struggle with the terminal: you choose a model, an engine (including MLX on Apple Silicon), adjust basic parameters, and chat in a user-friendly window. It has long supported MLX as a backend, so you can benefit from Apple's optimizations without leaving the app.

MLX differs from all of them in that it's the core framework, specifically tuned for Apple Silicon. When Ollama or LM Studio use MLX as their engine instead of pure Metal, they gain in memory efficiency, inference speed, and reduced friction with the unified memory architecture . This doesn't mean that MLX replaces Ollama or LM Studio; rather, they complement each other. Many users use MLX directly for demanding tasks and Ollama to serve models to applications via APIs.

Specific advantages of MLX compared to GGUF-based solutions

GGUF-centric solutions, such as Ollama or llama.cpp, have one major advantage: they work on almost any platform , from PCs with dedicated GPUs to Linux servers and modest devices. But when we narrow our focus to Macs with Apple Silicon, MLX begins to stand out for several very specific technical reasons.

First, memory management . MLX leverages the unified memory of the M chips at a low level, reducing copying and improving how model weights are mapped in RAM. This is especially noticeable in long contexts (for example, when using large context windows for RAG or analyzing extensive documents), where other engines suffer from the constant movement of data.

Secondly, MLX is aligned with Apple's roadmap . As Apple expands Apple Intelligence, refines compilers, and improves ML kernels, MLX directly receives those improvements. This makes MLX a future-proof bet for anyone developing software in the Apple ecosystem or looking to integrate AI with frameworks like Swift, AppKit, or SwiftUI.

Extensibility also plays a role. MLX allows you to work not only with inference, but also with training and fine-tuning (including LoRA) and quantification within the framework itself, which facilitates setting up complete experimentation cycles: you download a model, adapt it with your data, serve it locally and, if necessary, integrate it into an ERP or custom applications.

Finally, because it's released under an open license, the technical community can inspect, contribute to, and create tools for MLX, as mlx-lm already does. This is generating an emerging ecosystem of pre-converted models, guides, and utilities that are bringing MLX closer to the "plug & play" experience many users expect.

From demo to real-world use: how local AI is being used on Mac

For a while, running local LLMs on a Mac felt like a lab experiment: something you showed in a demo, but didn't actually use every day. That's changing rapidly as Ollama leverages MLX and the software stack matures.

The stories you read in communities like r/LocalLLaMA or developer forums all point in the same direction: users who have gone from having a "gaming PC for AI" to relying almost exclusively on a MacBook with an M chip , because the day-to-day experience is more pleasant: less noise, less power consumption, more integration with the rest of their work.

One of the key factors is that unified memory changes the way you think. Instead of obsessing over the gigabytes of VRAM available on a dedicated GPU, on a Mac with 64GB of unified memory you can load sizable models without battling artificial graphics card limitations. Obviously, there's a physical ceiling, and bandwidth also matters, but the bottlenecks are different and, in many developer workflows, more manageable.

Added to this is the constant improvement in quantization techniques, such as well-tuned Q4 variants or approaches like TurboQuant, which allow models with 20 or 30 parameters to fit into much more modest environments without excessively degrading quality. When good quantization is combined with MLX, a "compound effect" emerges: more speed, less memory, and larger models than seemed reasonable just a couple of years ago.

In practical terms, this translates to local coding assistants, private RAG systems for internal documents, data processing pipelines, and even offline personal assistants that work seamlessly on a MacBook, turning the laptop into a kind of "portable AI server." All this with the assurance that the data never leaves the machine.

Local text-to-speech with MLX: Murmur's example

MLX isn't just for text models. A very illustrative real-world example is Murmur, a native macOS application that performs text-to-speech (TTS) conversion entirely locally using MLX as its inference engine. The idea is simple: you paste or type text, press a button, and get a WAV audio file generated on your Mac.

The beauty of Murmur is that it breaks with the typical model of cloud-based TTS services: there are no monthly subscriptions, no character limits , no internet connection required, and your text isn't uploaded to any server. It's a one-time purchase, and processing happens entirely on the computer, leveraging the GPU via Metal and unified memory.

In terms of performance, machines with M2 Pro or M3 Max offer quite comfortable speeds: around 150 words per second , with the fan almost always silent and without blocking the rest of the system. The voice quality falls into the "convincing narrator" category, falling short of highly specialized giants like ElevenLabs, but more than sufficient for listening to articles, reviewing drafts, or generating test recordings.

Its limitations are clear: it's not intended for real-time streaming, English works better than other languages, and for now, it's only available for Apple Silicon . Even so, it perfectly illustrates how MLX allows complex models to be packaged into "human-friendly" desktop apps that deliver real value without exposing sensitive content to third parties.

Furthermore, this type of project opens the door to additional features: importing complete documents (PDF, EPUB, DOCX), dialogues with multiple voices, or even local voice cloning from short samples, all taking advantage of the infrastructure that MLX already provides on Mac.

Local AI and enterprise: from MLX and Ollama to FileMaker and ERP systems

In professional environments, the true power of local AI lies not only in the model itself, but also in its integration with business systems . This is where tools like FileMaker 2025, customized ERP solutions, and hybrid architectures that combine local execution with cloud services when truly needed come into play.

FileMaker 2025, developed by Claris (an Apple subsidiary), has taken a significant step forward by allowing MLX models to be hosted directly on FileMaker Server . This means that an enterprise database can expose script commands that call a local language model, hosted on the server itself, without any critical data leaving the organization's perimeter.

One of the most interesting applications is fine-tuning using LoRa. Instead of manually managing datasets, scripts, and model versions, FileMaker offers a structured interface where training logs, parameters, and versions can be centrally organized. The result is a reproducible workflow where model tuning ceases to be a manual task and becomes a standard business process.

In parallel, ERP systems act as the "glue" that binds data, processes, and AI results together. Local AI (whether using MLX or Ollama) connects via APIs, Python services, or specific integrations, so the models don't exist in isolation but are embedded in workflows such as invoicing, document analysis, internal support, or report generation.

For organizations with strong compliance requirements (data protection, industry regulations, confidentiality), this combination of local execution, version control and traceability , along with the security of using AI on-premises , is much more attractive than relying solely on external services, whose terms can change and which, by design, require taking information off-site.

Powerful use cases: assistants, private RAG, and code co-pilots

Beyond the purely technical aspects, what's interesting is what can actually be built on top of this stack. Some of the strongest use cases for local LLMs on Mac with Apple Silicon and MLX are coding assistants, private RAG systems, and productivity copilots.

A particularly useful pattern involves separating the logic into layers: an interface (for example, a code editor or a desktop app), a local server that translates requests into model calls and handles inference , and the language model itself. The model never directly interacts with the file system; instead, it proposes structured actions (tool calls) that the application executes in a controlled manner, returning the result as context.

Tool calls are essential for the wizard to be truly useful. The model generates a structured instruction (for example, "read this file," "run these tests," "search for this text in the project"), the server interprets it, the application executes it, and the results are reinjected into the model's context. All of this requires a proper tokenizer and robust parsers that understand the model's output format; if the configuration fails, the tool calls can break, leading to an erratic experience.

Another key area is that of private Retrieval-Augmented Generation (RAG) systems. With a local LLM on MLX or Ollama, plus a vector database (ChromaDB, for example), you can index internal documents, contracts, manuals, or ticket histories and query them in natural language, without exposing that data to a remote API. For sectors like healthcare, legal, or public administration, this makes all the difference.

Finally, local code copilots connected to tools like Continue.dev or Tabby allow a development team to leverage AI for autocompletion, refactoring, or test generation without their repository leaving the corporate environment. With programming-optimized models (such as Qwen or code-tuned variants of LLaMA 3) and MLX as the engine on Mac, the experience closely resembles that of cloud-based copilots, but with a radically different privacy profile.

Ultimately, the decision is rarely "all on-premises" or "all cloud," but rather a hybrid approach: sensitive and routine tasks (summaries, classification, internal reviews) are handled with on-premises models, while queries requiring extreme reasoning or specific needs can be delegated to external APIs. The key takeaway is that, with Apple Silicon and MLX, the on-premises side of the equation is becoming increasingly attractive and less of a sacrifice.

The combination of Apple Silicon, unified memory, and MLX has gone from a technical curiosity to a very serious option for running local language models on Mac. With tools like mlx-lm, LM Studio, Ollama, and enterprise platforms like FileMaker 2025, it's possible to build everything from personal assistants to complex ERP workflows, keeping data in-house, with controlled costs, and performance that no longer feels second-rate compared to high-end GPUs. For anyone working with sensitive information, seeking vendor independence, or wanting to push their Mac beyond its limits, opting for local LLMs with MLX has become a very sensible move.