Perplexity charts path to run advanced AI models on regular computers

Perplexity AI has unveiled an experimental approach to deploying large language models on standard consumer devices, potentially democratising access to advanced artificial intelligence capabilities. The company's work centres on running a 35-billion-parameter model—a substantial AI system—on hardware typically found in home computers and laptops, rather than requiring expensive data centre infrastructure.

The technical foundation rests on three key optimisations: streamlined inference engines that process AI queries more efficiently, quantisation techniques that reduce the computational demands of models without significantly compromising performance, and memory management strategies that maximise available resources. These methods work in concert to eliminate the traditional bottleneck where only organisations with substantial computing budgets could deploy powerful language models.

Inference—the process of running a trained model to generate responses—has traditionally consumed far more resources than the initial model training phase. By refining how inference operates, Perplexity's approach reduces the processing power and memory required at the point of use, making deployment on consumer machines viable.

The implications extend across South and Southeast Asia, where access to computational resources remains uneven. Researchers, small businesses, and individual developers currently unable to afford cloud-based AI services could potentially leverage these optimised models locally, without relying on external servers or internet connectivity for every query.

While the experiment remains in early stages, the results suggest that the next generation of AI tools may not require prohibitively expensive infrastructure, potentially reshaping how artificial intelligence becomes integrated into everyday computing across the region.