Apple Inc. revealed several previews of its upcoming software updates today during its Worldwide Developer Conference, including new features coming in its next-generation macOS Sequoia, iOS 18, iPadOS 18 and VisionOS, the company’s mixed-reality operating system.

The updates to macOS, iOS and iPadOS will roll out with a new feature called Apple Intelligence, a generative artificial intelligence-powered capability coming from Apple later this year that will change the way that users interact with their devices. It’s deeply integrated with these operating systems and uses Apple silicon to power itself to allow users to rewrite text, create images and take action within their apps.

What’s in macOS Sequoia

MacOS Sequoia focuses on what Apple calls “Continuity,” or the ability to move from one device to another without having to lose data or focus on whatever app that’s currently being used. In this case, the company is adding the ability to use their iPhone directly from Mac.

The new feature called iPhone Mirroring allows users to call up their iPhone’s screen directly onto their Mac and access it right on their screen, launch and browse apps and interact directly with them using the keyboard, trackpad and mouse – even with their phone is across the room in their bag. Users can even drag and drop between the iPhone and the Mac. When this is happening, the iPhone screen remains locked, and nobody else can access or see what’s happening, keeping the interaction private.

In addition, users can see iPhone notifications on the Mac and even interact with them just like Mac notifications. When they pop up the user is taken to the mirrored iPhone screen.

Safari, Apple’s web browser, also received big updates that will highlight information from the web, pulling out directions, summaries and quick links in an easy-to-read guide from web pages to minimize distractions. If the browser detects a video, Viewer pops it out with playback controls.

A new Passwords app, built on Keychain, now seamlessly synchronizes passwords, passkeys, Wi-Fi passwords and other credentials in one place safely all in one place. This means that users can make certain that they don’t lose their passwords for logins. They can also back them up to iCloud with end-to-end encryption across Apple devices, and Windows with the Microsoft app.

iPadOS 18 and the power of handwriting

IPads take their power in that they’re tablets for writing and with an Apple Pencil they get even better. With iPadOS 18 Apple introduced two machine learning capabilities that take writing to the next level, one that involves math and another that involves handwriting.

Math Notes on Calculator now allows users to write out mathematical equations and expressions in their own handwriting and it will be able to solve them. This includes variables that they can assign as if they were writing on a whiteboard, such as calculating a budget, physics problems and more. Using internal AI capabilities, Calculator can understand what is on the screen and solve equations.

Math Notes can also automatically understand equations and even produce graphs on demand, making it a powerful scientific tool for even the savviest users.

Using an Apple Pencil, Smart Script in Notes can make a user’s handwritten notes smoother and easier to read, which means that they can jot faster and keep the look and feel of their writing style without worry. It will also allow them to edit their already-written text by moving it aside, adding more text, or even deleting by scratching out sentences, paragraphs will automatically reflow to fit the new content.

With iPadOS 18, the tab bar got an upgrade where it now floats about app content and will morph into the sidebar when a user needs to dive into an app’s full functionality. As a result, apps now are fully edge-to-edge and don’t need sidebars, instead, the app bar takes that functionality and stays out of the way.

IOS 18 brings increased connectivity and privacy

Apple introduced the upcoming iOS 18, which will include a redesigned Control Center, providing better access to third-party apps, content from social media and other controls such as the ability to unlock a vehicle or turn on the flashlight.

The new controls gallery displays all available options, and users will find it easy to customize how controls are laid out, including adjustments on how they are sized and grouped. Users can also switch controls at the bottom of the Lock Screen and set controls to work from the action button.

Messages is getting support for RCS, which will finally enable reliable message exchanges with non-Apple devices. Support for RCS has been expected since late 2023 when Apple signaled that the update was coming sometime this year. Support for the protocol will help standardize communication between Apple and Android devices, which already support RCS, allowing for a better messaging experience.

“This integration marks a significant stride toward the standardization of messaging protocols, fostering better interoperability and a consistent experience across different devices,” said Inderpal Singh Mumick, chief executive of Dotgo LLC, an RCS for business solutions company. “For the business sector, the adoption of RCS on Apple devices unlocks new possibilities for enhanced communication, including features like branded messaging, verification, encryption, two-way chat and recognizable sender IDs.”

iOS 18 will also introduce Messages via satellite for times when cellular and Wi-Fi are not available. Powered by the same technology that enables iPhone satellite capabilities, a user will be prompted if they want to use a satellite to send a message and then the app will send messages.

New privacy features coming to the update include even more tools to protect user safety including locked and hidden apps. It’s not uncommon for people to hand their phones over to others so that they can play with them, watch videos or enjoy content, but they don’t want them snooping around. For additional privacy, apps can be locked or hidden, and content like messages or emails inside apps can be hidden from search or notifications so that it requires a biometric unlock to reveal them.

VisionOS 2 brings new mixed-reality features to Vision Pro

Vision Pro, Apple’s mixed-reality headset and “spatial computing platform,” will roll out its first major update to its device this fall, VisionOS 2, for free bringing more features designed to enhance its customers’ experience with the headset and transform their entertainment and productivity experience.

New gesture support is coming to Vision Pro to make it easier to navigate menus. The headset already has numerous cameras on the outside that can see the user’s hands, which makes it easy for the internals to understand the world. Now, users can hold their hand up and tap their forefinger and thumb to open the Home View. Flipping their hand over brings up time and battery level, and tapping again opens the Control Center.

Later this year, Mac Virtual Display is getting an upgrade from a virtual 4K monitor in mixed-reality to a higher-resolution display. This virtual display allows users to take their MacBook and bring it into mixed reality, coming soon, it will also be able to get an ultra-wide display, equivalent to two 4K monitors side-by-side.

“Apple said it has some 2,000 native apps for the Vision Pro thus far, in about four months of being a consumer product in the United States. By comparison, tens of thousands of apps are released every month into the App Store (or the Google Play store for Android devices),” Rolf Illenberger, CEO of VRdirect, an enterprise VR solutions company, told SiliconANGLE. “As VisionOS 2 continues to evolve, an emphasis on development is a major priority for Apple. The more native experiences that can be built for the Vision Pro, the greater it benefits the long-term usability of the device.”

Reaching out to developers, Apple opened up advanced volumetric application programming interfaces to allow advanced 3D apps to share the same space and TabletopKit that can anchor to flat surfaces, such as tables, for card games or other virtual experiences. The company also featured enterprise-specific APIs that will enable developers to build apps for surgical training in healthcare, equipment maintenance tool software for manufacturing and more.

Illenberger noted that Apple has leaned into enterprise use cases, as the API keys into. “So development to further improve those users and their experience will lead to greater corporate investment and adoption on the business-to-business level,” he added.

Images: Apple