Transcription
A 27-billion parameter model running on an iPhone. That should be impossible. A Caltech spin-off called PrismML just pulled it off. Not with normal compression, but with patented math. Six tricks make it fit. Counting down.
Number six. They're one-bit bonsai models, 14 times less memory, eight times faster, five times less energy, and on standard reasoning encoding benchmarks, they match full precision models the same size.
Number five. But is it lobotomized like every other one-bit model? A top reply on Reddit's local llama community put it plainly, far better than any other one-bit model. Almost as good as the uncompressed original.
Number four. Here's the trick. It is not normal quantization. 128 weights get bundled together. Each one kept as a single sign, plus or minus one, sharing one float 16 scale. It is stored as a single bit, but it still does the math in float 16. That costs a little more compute, but language models are limited by memory, not compute, so it barely matters.
Number three. Over 10 times the intelligence density of a full precision model the same size. They ship it at 8 billion, 4 billion, and 1.7 billion parameters. More brain per byte.
Number two. They took Alibaba's Qwen 3.6, 27 billion parameters, 54 gigabytes, and shrank it to four. The whole model. Caltech holds the patents, and PrismML has already had first talks with Apple.
Number one. And you don't even need an iPhone. The bonsai demo runs one-bit models on your Mac, your GPU, even your CPU. One command. The 8-billion model loads in about 2 and 1/2 gigabytes. Fully local, zero cloud calls. PrismML thinks that in about three years, 95% of artificial intelligence runs on your own devices, and only 5% touches the cloud. 4 gigabytes, one bit, local. Did you test the one-bit model yet?