What happened
A startup called PrismML has released Bonsai 27B, an on-device AI model that packs 27 billion parameters into a footprint small enough to run directly on an iPhone. Founded by a team of Caltech researchers, PrismML built Bonsai on top of Alibaba's Qwen3.6-27B, then compressed it aggressively while keeping most of its original capabilities: multistep reasoning, tool use, image understanding, and agent-based tasks.
That's a big deal, because models this size normally live in the cloud. A standard, uncompressed Qwen3.6-27B model takes up roughly 54 GB of storage — more than double the base storage of most iPhones. Even after conventional compression, it still needs around 18 GB, which is unworkable for a phone that also has to run everything else on the device.
According to a CNBC report, PrismML is already in early talks with Apple about the compression technology behind Bonsai. PrismML CEO Babak Hassibi confirmed that Apple, along with other companies, is testing the models for speed, power draw, and overall performance. Hassibi described the discussions as "very early" but said "things are progressing nicely."
Why it matters
The pitch behind Bonsai 27B is simple: AI agents are getting chattier, and cloud inference doesn't scale well with that. An agent handling a real task might make hundreds of model calls in sequence, each one carrying context, producing structured output, and feeding the next step. Every one of those calls costs money in the cloud, adds network latency, and often means sending private data — screen content, documents, personal messages — off the device.
Run the model locally instead, and the marginal cost of each step in that loop drops to zero, while sensitive data never leaves the phone. PrismML frames this as the foundation for always-on agents and offline assistants, plus hybrid setups where simple, privacy-sensitive tasks stay on-device and only the genuinely hard steps get routed to frontier models in the cloud.
The compression trick is aggressive: instead of storing each neural network weight at the standard 16 bits, Bonsai uses roughly one to two bits per weight. In the most extreme variant, each weight only has two possible states; in the slightly larger version, three. PrismML's white paper also flags a labeling issue common across the industry — a Qwen3.6-27B-IQ2_XXS build marketed as "2-bit" actually averages 2.8 bits per weight. By comparison, Bonsai's 1-bit variant reaches an intelligence density of 0.530 per GB, well ahead of ternary and full-precision (FP16) models on the same measure.
How to use it today
PrismML ships Bonsai in two sizes, aimed at different hardware:
- The quality-focused variant is about 5.9 GB and targets laptops, though shipping packages can run larger depending on the runtime — PrismML's white paper lists roughly 7.2 GB for the llama.cpp build and 8.49 GB for the MLX build.
- The smaller variant comes in around 3.9 GB, small enough to fit inside the limited storage of an iPhone 17 Pro Max. PrismML notes that even a 12 GB RAM iPhone only makes about 6 GB available to a single app, split between the model itself and its cache — which is why the compression work matters so much.
On an iPhone 17 Pro Max, the smaller Bonsai model generates about 11 tokens per second, and PrismML's battery testing found roughly 672 tokens generated per 1 percent of battery drained. Bonsai isn't publicly available as a consumer app yet — it's positioned as a foundation model for developers building agent and assistant apps. If you want to experiment with AI tools in the meantime without waiting on hardware rollouts, a lighter starting point is a browser-based toolkit like [mykreatool.com](https://mykreatool.com), which offers free AI tools for quick tasks without any local install.
Who benefits
Developers building agent-style apps stand to gain the most immediately: anything that chains many small model calls together — a research assistant, a coding helper, a workflow automation tool — gets cheaper and faster once inference happens on-device instead of over the network. Startups building privacy-sensitive products, like tools that read screen content or personal documents, also benefit, since nothing has to be transmitted to a server to get an answer.
Apple's reported interest points to a bigger opportunity: baking compressed, capable reasoning models directly into iOS could let Siri and other system features run agent-like tasks locally, without the cost or latency of cloud calls. Enterprise teams working with sensitive data — legal, healthcare, finance — could also use on-device models like Bonsai to keep processing entirely within their own hardware, sidestepping data-residency concerns tied to cloud AI.
Risks
Compression isn't free. PrismML's own testing across 15 benchmarks shows the larger variant retains about 95 percent of the original Qwen3.6-27B's performance, while the smaller, phone-ready variant keeps around 90 percent. Math and coding held up well under compression, but the more aggressive 1-bit version showed sharper drops in image understanding, instruction following, and agent-based tool use — exactly the capabilities agentic apps rely on most.
There's also a trust gap in how these models get marketed. PrismML itself points out that a "2-bit" competitor model actually averages 2.8 bits per weight, meaning labeled compression ratios don't always reflect reality. Buyers and developers evaluating on-device models should look at independent benchmark scores rather than headline bit-counts. And since Bonsai's talks with Apple are still described as "very early," any near-term expectation of it shipping inside iOS should be treated as speculative rather than confirmed.
Conclusion
Bonsai 27B shows that a full-scale, 27-billion-parameter reasoning model no longer has to live exclusively in the cloud — PrismML's compression work gets it down to under 4 GB while keeping roughly 90 percent of its original performance. With Apple reportedly testing the underlying technology, on-device AI agents that reason, use tools, and understand images without sending data to a server may be closer than most people expect. For now, the practical move for builders is to watch how PrismML's talks with Apple progress while experimenting with the lightweight AI tools already available today.
Comments 0