Key Points:
- Apple is evaluating technology from Khosla-backed startup PrismML to shrink and run server-sized AI models natively on iPhones.
- PrismML compressed a 27-billion-parameter model (Alibaba’s Qwen 3.6) from 54 GB to under 4 GB with no performance loss.
- The startup’s 1-bit and ternary compression architectures reduce neural network footprints by up to 15 times.
- Running larger AI models locally cuts data center costs, reduces latency, and reinforces Apple’s commitment to user privacy.
Apple has initiated preliminary evaluations of an advanced AI model-shrinking technology that could soon allow iPhones to run massive, server-class artificial intelligence models directly on local hardware. The initiative involves discussions with PrismML, a Caltech-spun startup backed by Khosla Ventures. The startup’s newly developed mathematical compression algorithms allow high-performance neural networks to run locally on mobile devices using up to 15 times less memory than traditional configurations, potentially triggering a major milestone in on-device mobile intelligence.
The core achievement driving this evaluation is the successful miniaturization of one of the world’s most powerful open-source models. The startup recently demonstrated the compression of Alibaba’s Qwen 3.6 model, a massive 27-billion-parameter neural network. Standard deployment of a model this size normally requires a 54 GB storage footprint, restricting its operation exclusively to high-powered cloud servers in data centers. Through advanced mathematical compression, the technology shrank the 54 GB file to less than 4 GB, allowing all 27 billion parameters to run natively and simultaneously on an iPhone 15 or newer.
Traditional large language models use 16-bit floating-point precision, resulting in massive memory footprints and demanding heavy processing bandwidth at inference time. The startup’s approach bypasses this physical bottleneck by pioneering 1-bit and ternary weight architectures. This technology simplifies how the model stores internal numerical values, reducing each figure to essentially a 1-bit or ternary state of -1, 0, or +1. Storing neural weights at extremely low-bit precision decreases the memory footprint of the models by an order of magnitude without severely sacrificing logical reasoning or generative capabilities.
This extreme compression is critical because mobile devices operate under highly strict RAM limitations. For instance, while a premium smartphone may feature 12 GB of total RAM, the operating system never exposes the full memory budget to a single application. On-device AI models typically receive a strict maximum budget of about 6 GB, which they must share with key-value caches and system activations. Because no conventional build of a 27-billion-parameter model can clear this memory gate, the compressed 1-bit model is the first of its size to fit on a smartphone with comfortable room to operate.
Most current on-device AI models run on sparse architectures with only a few billion parameters active at a time to stay within memory limits. The on-device flagship model introduced during the Worldwide Developers Conference, for example, features 20 billion parameters but only keeps 1 billion to 4 billion parameters active during runtime. In contrast, the newly released Bonsai 27B model developed by the startup keeps all 27 billion parameters simultaneously active during execution, enabling highly complex on-device tasks like advanced software coding and autonomous agent reasoning.
For Apple, the ability to run larger AI models locally represents a vital strategic goal. The company recently opened public beta testing for iOS 27, which features a highly anticipated, long-awaited redesign of its Siri digital assistant powered by Apple Intelligence. As the firm works to make its assistant competitive with premier cloud-based rivals, keeping processing local reduces latency and avoids the massive, ongoing costs associated with operating high-powered cloud data centers.
In addition to cost savings, local execution reinforces the company’s core marketing commitment to user privacy. By processing complex queries directly on the device rather than transmitting sensitive data to external servers, the tech giant can guarantee that user inputs remain entirely private. This on-device architecture also allows sophisticated features to function offline, ensuring that users can access advanced reasoning, document summarization, and coding assistance even in environments with zero internet connectivity.
The technology behind this model-shrinking breakthrough grew out of research at the California Institute of Technology (Caltech). The startup’s chief executive officer, Babak Hassibi, also serves as a professor of electrical engineering at the university. While Caltech owns the foundational patents behind the low-bit neural network architectures, the academic institution has licensed them exclusively to the startup. This exclusive partnership secures a robust intellectual property moat around the startup’s compression techniques as it moves to shrink even larger, trillion-parameter models.
The launch of the open-source Bonsai 27B model has provided the first public benchmarks of this compression technology across various hardware configurations. Running natively on desktop hardware, the 1-bit model reaches up to 163 tokens per second on an NVIDIA GeForce RTX 5090 GPU, dropping slightly to 134 tokens per second in Ternary mode. On Apple’s own M5 Max silicon, the model achieves 87 tokens per second in 1-bit and 58 tokens per second in Ternary, proving that low-bit kernels can deliver high-speed performance across both consumer graphics cards and custom system-on-chip architectures.
Ultimately, the preliminary discussions between the technology giant and the Caltech spinout demonstrate a major transition in the economics of artificial intelligence. By demonstrating that massive, server-class models can run natively on standard smartphone hardware, the technology represents a clear path forward for private, local intelligence. As the evaluation of speed, energy efficiency, and performance continues, this model-shrinking breakthrough will likely redefine what consumers can expect from their mobile devices over the next several years, bringing the power of the cloud directly into the palm of their hand.
