Apple Vision Framework Identify face

Question

Is it possible in the Apple Vision Framework to compare faces and recognise if that person is in a picture compared to a reference image of that person?

Something like Facebook Face recognition.

Thomas

Answer 1

From the Vision Framework Documentation:

The Vision framework performs face and face landmark detection, text detection, barcode recognition, image registration, and general feature tracking. Vision also allows the use of custom Core ML models for tasks like classification or object detection.

So, no, the Vision Framework does not provide face recognition, only face detection.

There are approaches out there to recognize faces. Here is an example of face recognition in an AR app: https://github.com/NovatecConsulting/FaceRecognition-in-ARKit

They trained a model that can detect like 100 persons, but you have to retrain it for every new person you want to recognize. Unfortunately, you can not just give two images in and have the faces compared.

Answer 2

According to Face Detection vs Face Recognition article:

Face detection just means that a system is able to identify that there is a human face present in an image or video. For example, Face Detection can be used to auto focus functionality for cameras.

Face recognition describes a biometric technology that goes far beyond a way when just a human face is detected. It actually attempts to establish whose face it is.

But...

In a case you need an Augmented Reality app, like Facebook's FaceApp, the answer is:

Yes, you can create an app similar to FaceApp using ARKit.

Because you need just a simple form of Face Recognition what is accessible via ARKit or RealityKit framework. You do not even need to create a .mlmodel like you do using Vision and CoreML frameworks.

All you need is a device with a front camera, allowing you detect up to three faces at a time using ARKit 3.0 or RealityKit 1.0. Look at the following Swift code how you can do it to get ARFaceAnchor when a face has detected.

And additionally, if you wanna use reference images for simple face detection – you need to put several reference images in Xcode's .arresourcegroup folder and use the following Swift code as additional condition to get a ARImageAnchor (in the center of a detected image).