Starting in iOS 14 and macOS Big Sur, developers will be able to add the capability to detect human body and hand poses in photos and videos to their apps using Apple's updated Vision framework, as explained in this WWDC 2020 session.
This functionality will allow apps to analyze the poses, movements, and gestures of people, enabling a wide variety of potential features. Apple provides some examples, including a fitness app that could automatically track the exercise a user performs, a safety-training app that could help employees use correct ergonomics, and a media-editing app that could find photos or videos based on pose similarity.
Hand pose detection in particular promises to deliver a new form of interaction with apps. Apple's demonstration showed a person holding their thumb and index finger together and then being able to draw in an iPhone app without touching the display.
Additionally, apps could use the framework to overlay emoji or graphics on a user's hands that mirror the specific gesture, such as a peace sign.
Another example is a camera app that automatically triggers photo capture when it detects the user making a specific hand gesture in the air.
The framework is capable of detecting multiple hands or bodies in one scene, but the algorithms might not work as well with people who are wearing gloves, bent over, facing upside down, or wearing overflowing or robe-like clothing. The algorithm can also experience difficulties if a person is close to edge of the screen or partially obstructed.
Similar functionality is already available through ARKit, but it is limited to augmented reality sessions and only works with the rear-facing camera on compatible iPhone and iPad models. With the updated Vision framework, developers have many more possibilities.
At today's WWDC 2025 keynote event, Apple unveiled a new design that will inform the next decade of iOS, iPadOS, and macOS development, so needless to say, it was a busy day. Apple also unveiled a ton of new features for the iPhone, an overhauled Spotlight interface for the Mac, and a ton of updates that make the iPad more like a Mac than ever before.
Subscribe to the MacRumors YouTube channel ...
Apple today announced that iPadOS 26 will be compatible with the iPad models listed below.
iPadOS 26 features a new Liquid Glass design, a menu bar, improved app windowing, and more.
iPadOS 26 supports the following iPad models:iPad Pro (M4)
iPad Pro 12.9-inch (3rd generation and later)
iPad Pro 11-inch (1st generation and later)
iPad Air (M2 and later)
iPad Air (3rd generation and...
Wednesday June 11, 2025 7:14 am PDT by Tim Hardwick
Apple at WWDC previewed a bunch of new features coming in its updated operating systems, but certain changes will have been met with dismay by third-party developers who already offer apps with equivalent or similar features. In other words, their product has been "sherlocked" by Apple.
When Apple creates an app or a feature that has functionality found in a third-party app, it is referred...
In 2020, Apple added a digital car key feature to its Wallet app, allowing users to lock, unlock, and start a compatible vehicle with an iPhone or Apple Watch. The feature is currently offered by select automakers, including Audi, BMW, Hyundai, Kia, Genesis, Mercedes-Benz, Volvo, and a handful of others, and it is set to expand further.
During its WWDC 2025 keynote today, Apple said that 13...
Apple has announced iOS 26, and the upcoming software update includes a long list of new features and changes for iPhones.
The first iOS 26 developer beta is now available, and a public beta will follow next month. The update will be released later this year.
iOS 26 is compatible with the iPhone 11 and newer.
Below, we have provided a high-level overview of 100 new features and changes ...
Wednesday June 11, 2025 4:22 pm PDT by Juli Clover
iOS 26 features a whole new design material that Apple calls Liquid Glass, with a focus on transparency that lets the content on your display shine through the controls. If you're not a fan of the look, or are having trouble with readability, there is a step that you can take to make things more opaque without entirely losing out on the new look.
Apple has multiple Accessibility options that ...
Thursday June 12, 2025 4:53 am PDT by Tim Hardwick
With iOS 26, Apple has introduced some major changes to the iPhone experience, headlined by the new Liquid Glass redesign that's available across all compatible devices. However, several of the update's features are exclusive to iPhone 15 Pro and iPhone 16 models, since they rely on Apple Intelligence.
The following features are powered by on-device large language models and machine...
iOS 26 suggests that Apple is indeed working on a new version of the AirPods Pro that could be coming in the not too distant future. Code in iOS 26 discovered by MacRumors contributor Steve Moser references the "AirPods Pro 3" alongside the "AirPods Pro" and the "AirPods Pro 2."
There is no AirPods Pro 3 at the current time, but rumors suggest that Apple plans to introduce a new model this...
Honestly, this seems like the kinda stuff that'd make Apple AR compelling—being able to draw in midair means you’d also be able to navigate an interface in midair with just your hands.
Using AR/VR without bringing a controller everywhere seems analogous to what set the iPhone apart from other touchscreen phones in 2007; you didn’t need a stylus.
The framework is capable of detecting multiple hands or bodies in one scene, but the algorithms might not work as well with people who are wearing gloves, bent over, facing upside down, or wearing overflowing or robe-like clothing.
I don't think the wizarding community is going to be too happy about this...
