As AI agents go mainstream, companies lean into confidential computing for data security

Companies need to stop ignoring data security as AI agents take over internal data movement in IT environments, analysts and IT execs said.

“Regulation of AI use cases — such as in healthcare and financial services, [which] are already subject to extensive regulatory oversight and compliance monitoring — will require auditability of the specific AI models or agentic software deployed,” said Richard Searle, chief AI officer at Fortanix.

With those kinds of issues playing out in the real world, some top tech players are embracing the concept of “confidential computing,” which has existed for years but is now finding new life with the rise of generative AI (genAI).

Confidential computing creates a hardware boundary in which AI models and data are locked. Information is released only to those models and agents with proper access to prevent unauthorized use of protected data.

For enterprises concerned about AI security, putting confidential computing into action can alleviate some of those fears, said Craig Matsumoto, a contributing research analyst at Futuriom. “It goes hand-in-hand with enterprises’ preference for private-cloud AI. They want an aspect of control,” Matsumoto said.

Google is now allowing companies to take its proprietary Gemini models and operate them in-house — the first time Google has allowed that to be done. In short, companies don’t need to be connected to the internet or Google Cloud to run Gemini.

One way that’s now possible: confidential computing technology on Nvidia GPUs allows Google to put its AI model on untrusted hardware outside the Google Cloud infrastructure. Gemini is designed to run on the company’s TPU, which is found only in Google Cloud. But in this case, the exported model can run in a confidential virtual machine on Nvidia’s GPU.

“Nvidia’s GPU hosting the model protects the IP of Google Gemini when it’s running in the data center, and also protects the enterprise IP used in these models,” said Justin Boitano, vice president of Enterprise AI products at Nvidia.

Specific technology attests that a user is authorized and able to receive information or access the model.

There is growing interest in this technology for applications that want “local data and local decision making with low latency,” said Sachin Gupta, vice president of infrastructure and solutions at Google.

“A combination of latency and data residency data compliance drives this kind of use case,” Gupta said.

GPUs combine high performance with robust security, which makes them ideal for regulated industries such as healthcare, finance, and government, said Steven Dickens, principal analyst at Hyperframe Research. “Compliance with regulations such as HIPAA and GDPR is essential,” he said.

Another place confidential computing has come into play involves WhatsApp, which recently got genAI tools that can generate quick summaries of a user’s latest messages. The summaries are private and not visible to Meta or other parties.

The feature represents Meta’s earliest use of confidential computing —the company calls it Private Processing — to secure user information. Meta has struggled for decades with protecting user data, but is now using confidential computing to regain user trust.

Meta essentially built a private computing environment with AMD and Nvidia GPUs; private WhatsApp information is fed into the environment, which generates the summaries without being visible to anyone else.

The private computing model minimizes interception of the data as it moves from WhatsApp to the cloud.

This is the “first use case where Meta applies Private Processing, we expect there will be others where the same or similar infrastructure might be beneficial in processing user requests,” the company said in a post detailing the technology.

Anthropic last month announced “Confidential Inference,” which provides security guarantees to customers using its Claude genAI technology. The feature creates a chain of trust as data moves up and down the AI chain and takes into account AI agents that are increasingly becoming brokers in inferencing.

And, on the Apple side of the equation, there’s the company’s Private Cloud Compute ecosystem.

(Confidential computing can also be used for non-AI applications; AMD and Intel have confidential computing technologies for CPUs that are available via virtual machines.)

Despite the increasing use of confidential computing, there remain concerns about its arrival in cloud environments, where CPUs check system-level attestation and GPUs authenticate data. For one, data travels to GPUs only through CPUs, and any vulnerability will leave a giant gap for hackers to steal data.

“In such an environment, the reliability of confidential computing and attestation becomes very fragile,” said Alex Matrosov, a security expert and CEO of Binarly. 

CPU-based technologies are also susceptible to side-channel attacks, raising concerns about their reliability, Hyperframe’s Dickens said.

A Google-disclosed vulnerability last December affected AMD confidential computing  and required microcode updates.As AI agents go mainstream, companies lean into confidential computing for data security – ComputerworldRead More