Most people feel comfortable with facial recognition for their use in Instagram and Face ID filters. But this relatively new technology may seem a bit complicated. Your face is like a fingerprint, and the technology behind facial recognition is complex.
As with any new technology, facial recognition has drawbacks. These disadvantages become more evident as the military, the police, the advertisers and the creators of deep fakes find new fraudulent ways to take advantage of facial recognition software.
Now, more than ever, it is essential that people understand how this technology works. It is also important to know its limitations and how it will develop in the future.
Before entering the different media that uses this technology, it is important to understand how the facial recognition process works. Here are three applications for this software and a simple explanation of how faces are recognized or identified:
There are different types of recognition
The first is basic facial recognition; used in Animoji filters and also on Instagram. Your mobile camera "searches" for the defining characteristics of a face, specifically a pair of eyes, nose and mouth.
Then, use algorithms to fix a face and determine in which direction you are looking, if you have your mouth open, etc. It is worth mentioning that this is not facial identification, it is only software that looks for faces.
The second is the Face ID and similar programs; When setting up Face ID and other programs on your smartphone, take a picture of your face and measure the distance between your facial features. Then, every time you are going to unlock your mobile, you must see through the camera to measure and confirm your identity.
The third is the identification of a stranger; When an organization wants to identify a face for security, advertising or surveillance reasons, it uses algorithms to compare that face with an extensive database of faces. This process is almost identical to Apple's Face ID but on a larger scale.
In theory, any database (ID cards, Facebook profiles) could be used, but a clear and pre-identified photo database is ideal.
All right, let's get into the essentials. Because the "basic facial recognition" used for Instagram filters is such a simple and harmless process, we will focus completely on facial identification and the many different technologies that can be used to identify a face.
Most facial recognition is based on 2D images
As expected, most facial recognition software relies entirely on 2D images. But this is not done because 2D facial images are super accurate, but are made for convenience.
So the vast majority of cameras take photos without depth, and public photos that can be used for facial recognition databases (Facebook profile images, for example) are in 2D.
Why is 2D facial image not super accurate? It is because a flat image of your face lacks identifying features, such as depth. With a flat image, a computer can measure pupillary distance and mouth width, among other variables. But it cannot determine the length of its nose or the prominence of your forehead.
In addition, 2D facial images are based on the visible light spectrum. This means that these images do not work in the dark, and may be unreliable in low light conditions.
Clearly, the way to avoid some of these deficiencies is to use 3D facial images. But how is it possible? Do you need a special team to see a 3D face?
IR cameras add depth to your identity
While some facial recognition applications are based solely on 2D images, it is not uncommon for facial recognition to also be based on 3D images. In fact, your experience with facial recognition probably involves a bit of 3D.
This is achieved through a technique called lidar, which is similar to sonar. Essentially, facial scanning devices, such as your iPhone, shoot a harmless IR matrix on your face. This matrix (a wall of lasers) is then reflected on your face and is captured by an IR or ToF camera on your mobile.
Where does 3D magic happen? The IR camera on your phone measures how long it takes for each beam of IR light to bounce on your face and return to the mobile. Naturally, the light that is reflected in your nose will have a shorter trip than the light that is reflected in your ears, and the IR camera uses this information to create a unique depth map of your face.
When used in conjunction with basic 2D images, 3D images can significantly increase the accuracy of facial recognition software.
Lidar images are a strange concept that can be difficult to understand. If this helps, try to imagine that the IR mesh of your smartphone or any facial recognition device is a board toy with pins. Like a board toy, your face leaves a slit in the IR mesh, where the nose is noticeably deeper than, say, your eyes.
Thermal imaging allows facial recognition to work at night
One of the shortcomings of 2D facial recognition is that it is based on the visible spectrum of light. In simple terms, basic facial recognition does not work in the dark. But this can be solved by using a thermal camera.
But: does the thermal image not depend on the IR light? Yes, it depends. But thermal cameras do not send bursts of IR light; They simply detect the IR light emitted by the objects. Hot objects emit a ton of IR light, while cold objects emit an insignificant amount of IR light.
The expensive thermal cameras can even detect subtle temperature differences on a surface, so the technology is ideal for facial recognition.
Thermal imaging could solve recognition problems
There are several different ways to identify a face with thermal images. All these techniques are incredibly complicated, but they share some fundamental similarities, so we will try to simplify things with a list:
1.- Several photos are needed: a thermal camera takes several photos of a person's face. Each photo focuses on a different spectrum of IR light (long, short and medium waves). Usually, the long wave spectrum provides the greatest amount of facial details.
2.- The blood vessel maps are useful: these IR images can also be used to extract the formation of blood vessels in a person's face. It's scary, but blood vessel maps can be used as unique facial fingerprints.
They can also be used to find the distance between the facial organs; if the thermal image produces images of poor quality or to identify bruises and scars.
3.- The person can be identified: a composite image is created; that is, a set of data using multiple IR images. This composite image can be compared with a facial database to identify that person.
Facial recognition is used by armed forces
Of course, thermal facial recognition is generally used by the military, it's not something you'll find in the corner store. And it is not something that will come with your next smart mobile.
In addition, thermal imaging does not work well during the day or in generally well lit environments. So they do not have many potential applications outside the army.
Limitations of facial recognition
We have spent a lot of time talking about the deficiencies of facial recognition. As we have seen in IR and thermal imaging, it is possible to overcome some of these limitations. But there are still some problems that have not yet been resolved:
- Obstruction: As expected, sunglasses and other accessories can fire and block facial recognition software.
- Poses: facial recognition works best with a neutral image facing forward. A tilt or turn of the head can make it difficult. Even for IR-based recognition software. In addition, a smile, swollen cheeks or any other pose can change the way a computer measures your face.
- Light: all forms of facial recognition depend on light, be it visible spectrum or IR light. As a result, strange lighting conditions may decrease the accuracy of identification. This may change, as scientists are currently developing sonar-based facial recognition technology.
- The database: without a good database, facial recognition cannot work. In this same order of ideas, it is impossible to identify a face that has not been correctly identified in the past.
- Data processing: Depending on the size and format of a database, computers may take some time to identify faces correctly. In some situations, such as surveillance, limitations in data processing restrict the use of facial identification for everyday applications.
This technology could overcome many inconveniences
As of now, the best way to avoid these limitations is to use other forms of identification along with facial recognition. Your mobile will ask you for a password or a fingerprint if it cannot identify your face.
And the Chinese government uses identification cards and tracking technology to close the margin of error that exists in its facial recognition network.
In the future, scientists will surely find a way to get around these problems. They can use probe technology together with LIDAR to create 3D facial maps in any environment.
And they can find ways to process facial data and identify strangers in an incredibly short period of time. Either way, this technology has great potential, so it is worth keeping up.