Over 34 million developers are part of Apple's developers community with access to great frameworks including AR specific ones like ARKit and RealityKit. Here is an overview of what we know so far from this year's WWDC when it comes to augmented reality, with more details to unravel this week after all the engineering sessions. You can find a list of recommended AR sessions and labs to check out from this year's WWDC at the end of the article.
A great use of the LiDAR scanner to create 3D floor plans for rooms in real-time. What is great about this new Swift API is that it also provides characteristics like dimensions and what types of furnitures and appliances are included in the scanned environment. Instructions and structural highlights are available to help guide the users in the scanning process.
My colleague from Berlin, Cristian Diaz was super fast in testing this out and sharing his results. RoomPlan can provide the content in USD and USDZ file formats, which can be imported and further developed in compatible tools like Cinema 4D or AutoCAD. Opening the scanned room file and seeing the individual components is really cool. The walls, chairs, tables and even the refrigerator and oven were identified and named accordingly as you can see in the example below.
Applications in the real estate and e-commerce sectors are quite obvious, but I am looking forward to see all the creative ways developers will make use of it!
Available on iOS 16.0+, iPadOS 16.0+ and Mac Catalyst 16.0+.
ARKit was first released in 2017, as Apple's go-to AR framework. In 2019, RealityKit came along with improvements as it was built from the ground up specifically for AR experiences. Here are the main updates announced this year with ARKit 6, while no news have been announced for RealityKit:
4K Video Capture
Available on iPhone 11 or later and iPad Pro (5th generation), this new feature will allow better quality video capture, especially great for applications that outputs video content.
Image and video capture improvements
"With ARKit 6, you can capture a high-resolution background image during an ARKit session, which is great for enabling higher-quality virtual object reconstruction and for photography apps that want to guide users to the perfect shot while maximizing final image quality. ARKit 6 also introduces support for HDR video and adds EXIF tags, such as exposure and white balance, to every ARKit frame. And you can now directly control the setting exposure, white balance, and focus of the camera during an ARKit session." - Apple
Location Anchors (in more cities)
Location anchors were introduced in 2020 but with a limited availability in major United States cities and London. The update we are getting this year is availability in the following additional cities:
Australia: Sydney, Melbourne
Canada: Vancouver, Montreal, Toronto
Japan: Fukuoka, Hiroshima, Kyoto, Nagoya, Osaka, Tokyo, Yokohama
Improvements to Motion Capture
The Motion Capture feature lets you capture the motion of a person in real time just with the use of your device. The new ARKit 6 highlights that this feature now includes tracking for your left and right ears, with improved overall body pose detection. Great potential for face filters and health applications with those additional improvements.
Wednesday is all about USD in WWDC's AR schedule with sessions on the latest advancements in working with those assets. Understanding the core concepts of this file format is crucial if you are an Apple AR developer.
Last year Apple introduced Digital Lounges, which is a Slack workspace you can register based on the topics you are interested in. This year "Augmented Reality" has been added as a topic which will have Q&A sessions on ARKit, RealityKit, Reality Composer, Object Capture, USD and RoomPlan.
One more thing... or not?
The iOS augmented reality community has a bit of mixed feelings about this year's WWDC. Especially when companies like Snap and Niantic are really evolving in this space and catching developers interest. Nothing new on Reality Composer and RealityKit as far as we can see, which seems odd and it just makes me wonder: is Apple saving those for a special AR-focused event coming up?
AndroidX Bluetooth is a new addition to the Jetpack Suite of libraries. While currently in its alpha stage, the system already provides robust safety measures addressing common pitfalls in Android BLE development. Moreover, it establishes a clear trajecto...
The Secure Enclave
The Secure Enclave is a pivotal part of most modern Apple devices. On your iPhone, it's what keeps secure your most sensitive information. Information such as your biometric data, to use with Face & Touch ID and your payment inform...