Why I hope Apple keeps investing in on-device AI

Recent reports say Apple may use artificial intelligence models from OpenAI or Anthropic to provide the smarter Siri experience it promised us over a year ago. That’s good in the sense that it means we’ll get yesterday’s jam tomorrow, but it may mean the company ceases to invest as much as it should in the development of genAI models that run on the device.

This concerns me because I think on-device, edge-based intelligence has a huge part to play in the future evolution of AI services. I think there are lots of reasons for this to be the case — security and privacy, obviously, but also for another good reason: the network.

The network, don’t forget the network

Switched-on tech purchasers are making huge investments in network infrastructure to support the AI services they hope to deploy across their companies.

A recent Cisco survey tells us that 97% of IT leaders see the network as critical to rolling out rolling out AI, IoT, and cloud, and 91% of them plan to increase the amount of money they spend on networking as a result. They’re also investing in data centers, and all of them seem to think that the networks themselves need to become smarter.

“AI is changing everything — and infrastructure is at the heart of that reinvention. The network has powered every wave of digital transformation, accelerating the convergence of IoT, cloud, hybrid work, and defending against rising security threats,” said Chintan Patel, CTO and Vice President Solutions Engineering, Cisco EMEA in a press release.

“IT leaders know the network they build today will shape the business they become tomorrow. Those who act now will be the ones who lead in the AI era.” 

The thing is, when it comes to network resources, we already know that the best way to optimize network capacity is to offload traffic to other services where possible. That’s why phones like to use Wi-Fi for calls, for example. Why would it be any different for AI? 

Making AI mundane again

Once you accept that optimizing access to these resources is what’s happening, it becomes easier to accept that one way to reduce demand is to create AI models that run on the device itself. Apple’s devices are, after all, equipped with super powerful low-energy processors and should be more than equal to a range of AI-driven tasks. That’s why it makes sense for the company to invest and continue to invest in genAI models that can work on the device, as so many Apple Intelligence models already do.

The conservation of network resources isn’t predicated only on cost efficiency, but also response. Look at it this way: as AI is inevitably more widely deployed in mission-critical environments, any kind of lag between an AI request and resolution of that request is unacceptable.

Just as you don’t want an AI-powered vehicle to suffer from lag as it approaches a pedestrian crossing, you don’t want lag to hit a rail traffic management system as two express trains speed toward each other on the same track. In some situations, network-derived lag costs lives, and while the drip-fed TV broadcast images of human misery we see so regularly today suggests lives don’t matter as much now as they did at the end of the last century, it still makes sense to offload mundane requests like spelling, summarization, and transcription so more critically important needs can be met within the context of network congestion and scarcity.

This also means it makes sense to continue to invest in edge intelligence. 

Eyes on the prize(s)

Doing so answers another burning need in enterprise deployment for privacy and security. It simply seems better to put more intelligence on the edge device. That means creating focused AI models capable of running on the device.

Doing so dramatically reduces the attack surface, eliminates network-derived lag, and ensures better privacy.

That’s why intelligence at the edge will inevitably become more important over time. Apple’s own on-device Apple Intelligence tools are likely to be the first in a larger suite, though building out that suite may take a while. That time frame is reflected in Apple’s purported decision to open Siri up to additional AI services.

In the end, as you can see, a confluence of factors make intelligence at the edge vital to the overall AI ecosystem. As more and more services and systems become network-reliant, all stakeholders will seek to offload some of those demands elsewhere (just as mobile telcos already shift traffic to Wi-Fi when they can), and the most logical place to run at least the most commonplace demands will be on the devices themselves. Every genAI transaction that can be handled on the device means one less whirl on the server and the preservation of the fragment of power it takes to send the instruction there and back again. 

So, who will make the mobile AI infrastructure?

Of all the AI firms I’m reading about, and all the Big Tech firms working with them, only Apple seems to have made significant investment in delivering such services in this way. That doesn’t mean it is the only one, nor does it mean it will succeed — it may already have declared failure internally — but the direction remains the same: networks will become smarter and devices more capable of handling more complex models natively.

With that in mind, despite the relatively short-term obstacles Apple seems to face, in the longer term it still makes sense for it to invest in on-device AI, because that is the direction of travel. And that’s why I hope Apple continues to invest in it.

You can follow me on social media! Join me on BlueSky,  LinkedIn, and Mastodon.US lets China buy semiconductor design software again – ComputerworldRead More