Revelations about NSA spying have left people wondering about the privacy of their digital data. But what about the privacy of their faces?
The movies make facial recognition look easy: In the 1998 film Enemy of the State, a team of NSA agents simply freeze a surveillance tape, tap some keys and identify the face a few computer beeps later.
But Neeraj Kumar, an expert in computer vision at the University of Washington, says we're nowhere close to just grabbing anybody's face off a security camera and coming up with a name.
Facial recognition has become pretty good at one-to-one comparisons — for instance, checking your face against the photo on your company ID. The accuracy is up to 95 percent.
But that's not so good if you're trying to come up with a name, and you're comparing one photo against many possible matches.
"Each time you do a comparison, there's 5 percent chance that it's wrong," Kumar says. "And that adds up. In fact, it multiplies up. Very quickly, you find that a 95 percent accuracy leads to pretty terrible results when you're actually trying to answer the question of, 'Who is this person?' "
Universal Challenges
That's a hard question for a computer. It's even harder when you ask it to identify a face in what experts call an "image from the wild."
"You're looking at surveillance-type images from low-resolution cameras," says Manuel Cueva, a Los Angeles County sheriff's deputy who trains officers in their facial recognition system. "If the image is such poor quality, you may not get any results, period."
The software can get confused by shadows and weird angles. Even a goofy smile can throw it. So that's problem one.
Problem two in this quest for universal facial recognition is that the software needs to compare one picture to another. In LA County, for instance, officials will run the image against their booking photos — about 6 million mug shots.
But if you've never been arrested in LA County, their system won't name you. And that's usually the end of it.
"There really isn't a set standard that we follow to be able to extend our searches into other jurisdictions," Cueva says.
Some agencies and states are working to increase sharing of mug shots and department of motor vehicles photos. But as of right now, we simply don't have a universal database of faces.
Or ... do we?
A Billion Custom Facial Models
Look at Facebook, says Amie Stepanovich, director of the domestic surveillance project at the Electronic Privacy Information Center in Washington, D.C.
Facebook has the largest biometric database in the world — "and it's all been formed by people voluntarily submitting pictures to Facebook and identifying who they belong to," she says.
Theoretically, every time you label faces by tagging a picture, you're chipping away at those two big challenges for universal facial recognition. First, you're helping to build a super-database of labeled faces. Second, you're uploading multiple versions of each person's face, which can improve a system's accuracy.
"If you had lots of photos of each person ... you could build a model for Martin, a model for me, a model for other people. Now you have a custom-tuned model for each person," Kumar, from the University of Washington, says.
Multiply that by a billion — a billion custom-tuned facial "models."
Facebook would not answer NPR's questions about what it does with facial recognition information; social media companies rarely talk about their internal systems.
But they're surely aware of their huge database's potential. Last year, Facebook bought Face.com, whose company's founders had published a paper titled "Leveraging Billions of Faces to Overcome Performance Barriers in Unconstrained Face Recognition."
Transparency And Possibility
Stepanovich wants social media companies to explain how facial models can be used. On Facebook, for example, you can't identify faces of people who aren't already your "friends," but she wonders if, behind the scenes, Facebook can do broader searches — say, at the request of the government.
"As we're seeing specifically over the past few months, no matter how much a company attempts to protect your privacy, if they're collecting information about you, that information is vulnerable to government search," Stepanovich says.
Facebook won't say whether this is technically possible; Google, which offers the competing Google Plus service, also won't comment on the record about the feasibility of broader face searches.
Kumar doubts anyone is doing universal searches of Facebook faces. He says the numbers are just too big.
However, if social media companies are able to narrow the search — say, if they can compare a photo with the facial models of everybody who "likes" NPR, or everybody who lives in Des Moines — then, Kumar says, you'd have the makings of a useful search tool.
Copyright 2021 NPR. To see more, visit https://www.npr.org.
