Here’s Why Your Video Doorbell Sucks at Recognizing You

Of all the features included in video doorbells, facial recognition might be the most underappreciated. There’s nothing like hearing your smart speakers call out “mom is at the door” before she can even knock. Unfortunately, people have a good reason to dismiss this feature—it doesn’t work.

Visit a smart brand’s support forum, and you’ll see a ton of people complaining that their smart doorbell misidentifies every guest. Video doorbells think that toddlers are grandfathers, or that every pizza boy is your spouse. And there isn’t much you can do to solve this problem. Smart doorbells just suck at facial recognition.

Doorbell Cameras Don’t Capture Enough Detail

The facial recognition technology utilized by smart doorbells is pretty cutting-edge, as it employs some of the most advanced AI ever created. But even with great AI, video doorbells don’t have the hardware to really support facial recognition—they just can’t capture enough detail.

Like most facial recognition systems, smart doorbells capture and analyze 2D images. These flat images contain a lot of useful info, such as the width of your mouth, your skin tone, and the space between your eyes. But this data may not be unique to your face. In fact, this data may not be all that accurate, as video doorbells capture fairly low-res images of moving subjects.

More advanced facial recognition systems, such as the one that’s in your iPhone, capture “3D” images using infra-red TOF cameras. Here’s the gist; they shoot invisible lasers at your face and measure the time it takes each laser to bounce back. The data captured by these TOF cameras contributes to a “depth map,” which contains measurements like the length of your nose or the angle of your ears.

RELATEDThe Problem With AI: Machines Are Learning Things, But Can’t Understand Them

That 3D data is a lot more useful than what your video doorbell captures, for reasons that should be obvious. But hardware isn’t the only problem here. In the grand scheme of things, the advanced AI used by your video doorbell is actually pretty rudimentary.

Facial Recognition Algorithms Need Training

The facial recognition systems offered in smart doorbells are “self-learning.” They may ask you to identify a new person, but for the most part, they build and organize a facial database without user input. And that can present some problems.

See, self-learning facial recognition systems are always trying to improve their accuracy. That means collecting a ton of data; it’s hard for your doorbell to identify someone if it’s only seen that person once. So, every time your doorbell sees “mom,” it adds to its collection of “this is what mom looks like.”

But when a plumber comes to your door and gets incorrectly identified as “mom,” the facial recognition system becomes less accurate. Your doorbell doesn’t know it made a mistake, and suddenly, guests with a mustache could be “mom.” This leads to a downward spiral—the loss of accuracy creates more false positives, and “mom” now comes in every size, shape, and skin color. In the eyes of your doorbell, everyone is “mom.”

It’s like when a student learns a math equation incorrectly. They may be confident in what they learned, but until they bomb an exam, they won’t realize that they screwed up. Students need someone to check that they’re learning stuff correctly, and the same goes for AI.

Unfortunately, you’re the teacher in this situation.

How to Improve Your Doorbell’s Facial Recognition

Improving your doorbell’s facial recognition system is a chore. There’s no permanent fix here—you need to actively keep up with the facial recognition system to address its mistakes.

First thing’s first, you need to ensure that your smart doorbell can clearly see guests’ faces. That may mean repositioning the doorbell, regularly cleaning its lens, or adding some lights outside your front door.

Once you know that your doorbell can see what it’s supposed to see, you need to clean up its facial recognition database. This process will differ for each doorbell, but in most cases, you can find a list of faces in your smart doorbell’s companion app. (If you own a Nest Doorbell, go into the Nest Aware settings in your Google Home app. I don’t know why Google hides this stuff.)

Delete any misidentified faces that your doorbell has captured, and be sure to tell your doorbell the names of any unidentified people that you expect to return to your home. If you regularly curate this database, your doorbell should get a lot better at identifying guests.

RELATEDThe 6 Best Smart Lights of 2022

Here’s the bad news; even if you try to improve the accuracy of your doorbell’s facial recognition system, it’ll never be perfect. In fact, it may always suck. Facial recognition technology is still pretty rudimentary, and smart doorbells use very basic hardware to “see” people.

If you hate the idea of curating a facial recognition database, maybe you should just disable the feature. Replacing your doorbell isn’t worth the money, as every brand’s facial recognition systems suffer from the same problems.