What is an NPU?
A neural processing unit is also known as an AI accelerator, and is hardware inside a computer specialised for machine learning tasks. Running an AI task through an NPU can in theory make for a much more power-efficient operation than running it through a central or graphics processor, because the NPU cores are specially designed for AI.
NPU power is measured in TOPS, or trillions of operations per second, and those numbers are recently climbing, given mainstream interest in AI. While most recent PC laptop chips sit at around 10 TOPS, the Qualcomm NPUs in Microsoft’s new laptops produce 40 or 45. Similarly, while Apple’s most powerful M3 from last year hits 18 TOPS, the M4 in the latest iPad Pro achieves 38.
Later this year, Apple will roll out Apple Intelligence, a suite of AI capabilities that are built into its computer, tablet and smartphone operating systems. Examples it has given include proofreading text to identify errors beyond what standard spell-checkers can do, generating images on the fly based on prompts, drawings or on-screen context, and passing everything it knows about you to Siri so the assistant can help you more easily, without needing to upload any of your data.
Copilot+ PCs run a special version of Windows 11 that includes some unique AI apps. One allows the user to draw with a stylus or mouse while an AI helper constantly tries to interpret the drawing, with the user then able to integrate the suggestions into their creation.
Another keeps track of everything that’s been displayed on the screen so users can search their history semantically rather than having to remember file names or web addresses (for example, a search for “dog fights tree” will return the Reddit video you were watching earlier this morning, even if none of those words were present). However, this feature, called Recall, has effectively been delayed after some backlash. Microsoft says it wants to make extra sure users understand the safeguards in place to prevent abuse or privacy issues.
Recall, which will work Copilot+ PCs, remembers everything you do on the PC so you can scroll back through time.
Both ecosystems also support more general AI features, either entirely on device or in a hybrid model that sees them work securely in tandem with cloud servers, but the end result will supposedly be that many laborious quasi-creative tasks will be quicker and easier. Need to write a two-line bio about yourself? Just point to your CV and ask the AI to do it. Need a quick illustration to stick in your presentation about green waste? It takes around two seconds.
The future of AI on laptops
But it’s not just about the AI experiences that come with the device; having an NPU on board means developers can use it with their own programs, too. Both Microsoft and Apple have a new set of tools developers can use to integrate their software or build brand-new apps, and on Copilot+ PCs, the NPU is already being leveraged in Photoshop, DaVinci Resolve and others.
Microsoft has also shown off how the processor is used by Cephable, an assistive tech platform that allows people to control their PC with head movements or facial expressions. By moving its camera pipeline to the NPU, the app works more efficiently and no longer hogs a large chunk of the CPU bandwidth, meaning those with accessibility needs can use it without slowing down their computer.
“What’s exciting is this is just the start. You’re building that platform, and then you’re really enrolling your developers to say we’ve got this new technology on a PC with a huge addressable base,” said Dan Johnson, Microsoft’s Australia and New Zealand Windows lead.
Loading
“If you’re an author, and there’s a small language model that’s developed just for that audience, and you’re actually on a plane without any internet, and you can still do that work. I think that will be a wow moment where people sort of get it.”
Johnson said there was so much to explain to people about why AI on PCs will be such a big deal that Microsoft is breaking the rollout down into phases. The first is the introduction of Copilot+, marking a new class of PC with unique experiences that show what the technology can do.
While AI capabilities will become more generally available on PCs in the future, the tech represents a sudden jump for laptops, Johnson said.
“It has been more like sort of evolution rather than revolution [over the past decade],” he said.
“But now for the superuser that is really productive, that wants the latest, there’s a new reason to go out and buy a new sort of PC. The NPU really is a jump.”
Whether or not NPUs will wind up integrated into every device, powering AI processes that would have been unthinkable even with room-sized servers a few decades ago remains to be seen.
Get news and reviews on technology, gadgets and gaming in our Technology newsletter every Friday. Sign up here.