At WWDC, Apple infuses AI into the Vision Pro

Sluggish sales of Apple’s Vision Pro mixed reality headset haven’t dampened the company’s enthusiasm for advancing the device’s 3D computing experience, which now incorporates AI to deliver richer context and experiences.

Apple has upgraded the newly renamed visionOS 26 operating system to improve usability, blending AI into the background to enhance images and user interaction, Apple executives said at the company’s Worldwide Developer Conference on Monday.

Apple calls Vision Pro a “spatial computer” in which users can watch movies or interact with digital experiences in the real world using hands, eyes, and other senses.

A new feature called “spatial scenes” uses AI to give a 3D feel to traditional two-dimensional photos, which will be available in the Photos app. The AI model uses computational depth to add more perspectives to images, Haley Allen, senior director of visionOS program management, said during the presentation.

“Spatial scenes will make your web browsing experience more engaging, too,” Allen said.

In Safari, users will be able to select a feature called “spatial browsing” to view two-dimensional inline photos in supported articles in a three-dimensional format. Apple also introduced a curated app called “spatial gallery” to showcase curated spatial image content.

And the company is making major AI-driven changes to its Persona feature, which will now create more realistic avatars of users for the FaceTime video chat application. Previous avatars had lifelike renditions, but the updated Personas will be more personal and familiar, Allen said.

“Hair, lashes, complexion, all look remarkably accurate,” Allen said.

Apple also added many AI-driven improvements to apps such as Image Playground, which are supported across Apple devices. Apple has updated Image Playground to create more creative emojis by tapping into OpenAI’s ChatGPT AI application, which supports text and image creation.

The company also introduced a variety of non-AI features.

Vision Pro users will now be able to access widgets using their headset. The widgets will remain fixed within the visual field and will be clearly accessible at all times, Mike Rockwell, vice president of the Vision Products Group, said during a pre-recorded Apple video detailing the new features.

Widgets such as calendar and battery indicator are already popular in macOS. The company also introduced new widgets, such as a circular clock, music, real-time weather, and photos that can show snapshots of pictures from personal collections.

The widgets are blended into a user interface built around the new “liquid design” color scheme, which focuses on sharp colors and window edges.

Users can also now share Vision Pro experiences with others, which could help enterprises collaborate on three-dimensional engineering and design applications. Teams in hundreds of companies already use Vision Pro headsets for design, sales, training, and more, Rockwell said.

With that in mind, companies can also now create a pool of Vision Pro devices that can be shared among team members and guest users. Users can save their hand, eye, accessibility, and prescription information to their iPhone and activate a shared headset.

Apple also added new APIs such as “Protected Content,” which can show confidential materials to authorized users.

The company is also adding new input options such as the Logitech Muse, a pen-like device that can be used to draw within the 3D interface, and support for the Sony PlayStation VR2 Sense Controller; the latter can be worn on the hand to play motion-controlled video games within the headset.At WWDC, Apple infuses AI into the Vision Pro – ComputerworldRead More