Over 34 million developers are part of Apple's developers community with access to great frameworks including AR specific ones like ARKit and RealityKit. Here is an overview of what we know so far from this year's WWDC when it comes to augmented reality, with more details to unravel this week after all the engineering sessions. You can find a list of recommended AR sessions and labs to check out from this year's WWDC at the end of the article.
RoomPlan
Image source: Apple
A great use of the LiDAR scanner to create 3D floor plans for rooms in real-time. What is great about this new Swift API is that it also provides characteristics like dimensions and what types of furnitures and appliances are included in the scanned environment. Instructions and structural highlights are available to help guide the users in the scanning process.
My colleague from Berlin, Cristian Diaz was super fast in testing this out and sharing his results. RoomPlan can provide the content in USD and USDZ file formats, which can be imported and further developed in compatible tools like Cinema 4D or AutoCAD. Opening the scanned room file and seeing the individual components is really cool. The walls, chairs, tables and even the refrigerator and oven were identified and named accordingly as you can see in the example below.
Applications in the real estate and e-commerce sectors are quite obvious, but I am looking forward to see all the creative ways developers will make use of it!
Available on iOS 16.0+, iPadOS 16.0+ and Mac Catalyst 16.0+.
More information for developers can be found on the new RoomPlan framework documentation page.
ARKit 6
ARKit was first released in 2017, as Apple's go-to AR framework. In 2019, RealityKit came along with improvements as it was built from the ground up specifically for AR experiences. Here are the main updates announced this year with ARKit 6, while no news have been announced for RealityKit:
4K Video Capture
Available on iPhone 11 or later and iPad Pro (5th generation), this new feature will allow better quality video capture, especially great for applications that outputs video content.
Image and video capture improvements
"With ARKit 6, you can capture a high-resolution background image during an ARKit session, which is great for enabling higher-quality virtual object reconstruction and for photography apps that want to guide users to the perfect shot while maximizing final image quality. ARKit 6 also introduces support for HDR video and adds EXIF tags, such as exposure and white balance, to every ARKit frame. And you can now directly control the setting exposure, white balance, and focus of the camera during an ARKit session." - Apple
Location Anchors (in more cities)
Location anchors were introduced in 2020 but with a limited availability in major United States cities and London. The update we are getting this year is availability in the following additional cities:
- Australia: Sydney, Melbourne
- Canada: Vancouver, Montreal, Toronto
- Japan: Fukuoka, Hiroshima, Kyoto, Nagoya, Osaka, Tokyo, Yokohama
- Singapore
Improvements to Motion Capture
The Motion Capture feature lets you capture the motion of a person in real time just with the use of your device. The new ARKit 6 highlights that this feature now includes tracking for your left and right ears, with improved overall body pose detection. Great potential for face filters and health applications with those additional improvements.
More information for developers can be found on the updated ARKit framework documentation page.
USD advancements
Wednesday is all about USD in WWDC's AR schedule with sessions on the latest advancements in working with those assets. Understanding the core concepts of this file format is crucial if you are an Apple AR developer.
WWDC22 AR sessions
Tuesday June 7:
- Bring your world into augmented reality
- Create parametric 3D room scans with RoomPlan
- Qualities of great AR experiences
Wednesday June 8:
Thursday June 9:
WWDC22 1-on-1 Developer Labs
Tuesday June 7:
Wednesday June 8:
Thursday June 9:
- RoomPlan lab
- ARKit lab
- RealityKit and Reality Composer lab
- Core Motion lab
- Object Capture lab
- RoomPlan lab
Friday June 10
WWDC22 Digital lounges
Last year Apple introduced Digital Lounges, which is a Slack workspace you can register based on the topics you are interested in. This year "Augmented Reality" has been added as a topic which will have Q&A sessions on ARKit, RealityKit, Reality Composer, Object Capture, USD and RoomPlan.
One more thing... or not?
The iOS augmented reality community has a bit of mixed feelings about this year's WWDC. Especially when companies like Snap and Niantic are really evolving in this space and catching developers interest. Nothing new on Reality Composer and RealityKit as far as we can see, which seems odd and it just makes me wonder: is Apple saving those for a special AR-focused event coming up?
Resources
- Augmented Reality - Apple Developer
- ARAnchors - Bringing virtual objects into the real world
- Augmented reality face filters with iOS - Borderless Engineering Conference 2021
- Building Augmented Reality experiences with iOS - Engineering Awesome Conference 2020
- Easy web augmented reality with AR Quick Look
- How to convert 3D models to USDZ files using Apple's Reality Converter
- Using USDZ for a better AR experience
- How to make an augmented reality decorating experience app with AR Quick Look
Article Photo by Apple
Author
You may also like
Introducing UCL Max AltPlay, a turn-by-turn real-time Football simulation
At this year's MonstarHacks, our goal was to elevate the sports experience to the next level with cutting-edge AI and machine learning technologies. With that in mind, we designed a unique solution for football fans that will open up new dimensions for wa...
Agorametrics: Data democratisation from Sports Science for the benefit of all
As part of Monstarlab's 4th edition of the internal hackathon, MonstarHacks, the theme was to leverage cutting-edge technologies to innovate within the sports industry by developing solutions that enhance fan engagement, optimise athletic performance, and...
