Transcription
If you've bought a Mac laptop in the last few years, it could already outperform a $2,000 Nvidia graphics card for a lot of real AI work. And yet, when people talk about AI hardware, Apple barely gets a mention and Nvidia gets the attention. So, what's going on?
The reason might surprise you, and it's called CUDA. Every AI tool that you've heard of, Chat, GPT, Stable Diffusion, the whole ecosystem, it was all built on Nvidia's platform called CUDA. And it's been the industry standard now for nearly 20 years, and it only runs on Nvidia hardware. Apple has been effectively locked out of CUDA entirely.
But instead of trying to get back in, they're building something better, and it's called Metal. It's now at the point where it doesn't just match CUDA for certain workloads, it actually beats it. So today, I'm going to explain why that's the case and what that means for you if you're a developer. Because with Metal, you can get started faster and cheaper than ever before.
Think of CUDA as a language that tells the GPU how to perform heavy compute, like training AI models or processing video. NVIDIA invented CUDA in 2006, and since then, the entire AI ecosystem has been built on top of it. Back in 2018, Apple and Nvidia had a huge public falling out, and Apple dropped support for Nvidia GPU drivers in Mac OS entirely, and Nvidia never came back. And then, when Apple launched their M-series chips in 2020, the GPU was built directly into the Apple silicon chip. There's no slot for an Nvidia card, even if you wanted one. So Apple built Metal instead.
Launched in 2014 and has been refined ever since. It runs on every Apple device: iPhone, iPad, and the M-series Macs. But unlike CUDA, Metal handles compute, graphics, AI, and video processing all in the same chip, in the same framework, with the same memory. Apple didn't just get lucky here. This was a deliberate architectural decision for their entire stack. They control the chip, the GPU, the framework, and the operating system, and they optimize all of them together. Now, Nvidia is brilliant at making GPUs, but Apple is brilliant at making everything around the GPU talk to each other seamlessly. Those are different skills, and they lead to very different outcomes depending on the job.
CUDA is generally exceptional at what it does. If you're training a large language model from scratch, running multi-GPU clusters in a data center, CUDA is the gold standard. NVIDIA has spent 20 years building the deepest ecosystem in computing. PyTorch, TensorFlow, and the entire modern AI stack was built assuming CUDA exists, and that won't change overnight. But ecosystems are built to solve problems that existed at the time. And CUDA was built for a world where big data centers do all the heavy lifting, and it's extraordinary at that job. It's not designed for the world I believe we're moving into, where AI runs on device, on laptops, on the edge, in real time, and using battery power. That's a whole new set of constraints, and that's exactly where Metal wins.
There are three reasons why. One, bandwidth. Bandwidth is how quickly the chip moves data around internally. And Apple's M4 Pro does that more than four times faster than a typical Nvidia card can receive data from the rest of the system. So even if the Nvidia GPU is more powerful, it spends a surprising amount of time just waiting, whereas Metal doesn't wait. Two, memory. Nvidia GPUs have a hard memory ceiling. If you hit that, everything stops, whereas Apple silicon doesn't have that problem. The GPU shares the same pool of memory with the rest of the chip, which means more room, no ceiling, and no crashes. Number three, power. An Nvidia GPU can draw as much power as a small heater. An M4 MacBook Pro does comparable AI work on power that's less than a light bulb. So if you're running AI all day, the difference shows up pretty quickly on your battery life and your electricity bill.
I think of CUDA as a specialist tool, very powerful but narrow. It was designed for researchers and engineers running serious hardware in serious facilities. Getting started with CUDA means buying expensive gear, configuring drivers, managing dependencies, and hoping nothing breaks when you next update your operating system. There's a reason why AI development has been typically a game for well-funded organizations.
But Metal flips that. It's a unified framework. It's one API that covers GPU compute, graphics, AI acceleration, and video processing across every Apple device: every iPhone, iPad, MacBook, Mac Studio. The same framework, the same code, and the same model. Apple ships it as a part of the operating system. You don't install it. You don't even configure it. It's just there. That's a game-changer for developers because the traditional path to shipping AI was to build it in the cloud, pay for GPU compute, and pray that your costs don't blow up when you start getting users. But Metal removes that layer entirely. The GPU is already in your users' pockets, and you don't have to pay to use it. And that changes the economics of building completely.
So let's talk about who benefits from that. If you're building AI into your app, you don't need to pay for external GPU. There are tools like Orama which can now run full AI models locally on a MacBook. So your laptop is your AI development machine from day one. And when you're ready to ship, those same models can run directly on users' phones with no cloud or API fees. That's a direct path to a billion devices in people's pockets. And CUDA just can't give you that.
NVIDIA has built an empire on CUDA. 20 years of dominance with every AI lab, every cloud platform, and every research institution running on their hardware. It's not just market leadership. It's a monopoly. And like every monopoly in tech history, it looks unbeatable right up to the point where it isn't. Apple didn't set out to beat Nvidia at their own game. They decided to play a different game entirely. Then they made it really easy for developers because GPU programming is notoriously hard. But Metal takes away the complexity and gives developers a clean, powerful framework that works across every Apple device without needing a computer science degree in GPU architecture.
And so here we are, Nvidia, the Goliath of GPU computing versus an Apple framework that already runs on a billion devices. CUDA built the AI era, and that's not in dispute. But I believe the AI era is moving on. It's moving from data centers to devices and from cloud dependencies to local intelligence. With the M5 Ultra arriving later this year, I think we're going to see this get very interesting. And my money is on Apple.
But one last thing I will say is I've never worked for Apple, never been paid for Apple, and I don't even own any shares in Apple. And I have no affiliation with Nvidia either. But I do believe that technology gets better when ideas can compete. Monopolies don't innovate, they consolidate. And the fact that Apple is forcing this conversation in the AI world is good for everyone, regardless of which platform you're building on.
If you find this content useful, please consider subscribing to my channel so you'll never miss an update. Thank you and see you in the next.