Peering into the state of facial recognition

If you’re worried about whether the FBI overreaches when it uses facial-recognition technology to identify and track suspected criminals, you’re not alone. An investigative arm of Congress is now voicing concerns long expressed by advocates of civil liberties and privacy.

According to a study by the nonpartisan U.S. Government Accountability Office published last week, the FBI needs to “better ensure privacy and accuracy” in its implementation of facial-recognition technology. The federal agency can search nearly 412 million photos across all “searchable repositories” to which it has access, the report says. This includes 30 million mug shots culled from its primary facial-recognition database, the Next Generation Identification-Interstate Photo System.

“The vast majority of them are noncriminal photos,” says Jennifer Lynch, a senior staff attorney at the Electronic Frontier Foundation, an advocate of online privacy and civil rights. The GAO report found that the FBI’s photo database uses driver’s license photos from 16 states, as well as photos from passport and visa applications to the United States.

The database, says Lynch, who specializes in EFF cases involving facial recognition, is “being used to try to find suspects in crimes. The problem with that is that face recognition is notoriously inaccurate,” she says.

That assertion was reiterated by several people The Parallax spoke with for this report, including Reza Derakhshani, an associate professor of electrical engineering at the University of Missouri at Kansas City.

“Facial-recognition technology has its blind spots,” says Derakhshani, who also serves as chief scientist at EyeVerify, a biometric-security startup. “Not all technologies are created equal. A lot depends on how well faces are being captured” by devices ranging from low-end security cameras to those equipped with 4K high-definition and an optical zoom. “It also depends on what you’re trying to do: identify one person from millions of people, or one person from a short list?”

The FBI declined to comment.

Facial-recognition technology has been instrumental in creating a virtual panopticon where people can be tracked across the “real world” and the Internet. It’s spread to nearly every part of our lives, from government surveillance to enhancing consumer experiences, including our most personal devices: our smartphones.

While advocates of the technology swear by its abilities to improve society, the GAO’s report reflects the suspicions of its critics: Its uneven accuracy jeopardizes civil liberties and privacy, and despite its widespread and ever-expanding uses, it is not ready for prime time.

Organizations far and wide have been sounding alarms over the technology’s use and inconsistent accuracy. Besides government agencies such as the FBI, tech companies ranging from startups to titans such as Apple, Google, and Facebook use facial recognition to make their services more engaging.

They’re betting that customers won’t mind helping improve their identification algorithms if they enjoy seeing themselves tagged in their friends’ photos, Derakhshani says.

Not quite accurate

Accurate algorithms—the mathematical formulas used to calculate facial recognition—are difficult to develop for facial recognition, Derakhshani says. A program “could pick something that is not a face. You look at clouds, and sometimes you can see faces in the clouds, even though there are no faces there. Those types of errors do occur.”

In 2011, a Massachusetts man found out he was misidentified as a terrorist suspect, thanks to inaccurate facial-recognition technology. And just last year, Google tagged two African-Americans as gorillas.

So how accurate are today’s most prevalent facial-recognition technologies? While it depends on the algorithms and the size of the image databases, EyeVerify CEO Toby Rush says most are “95 percent” accurate.

“For recognizing people walking down a hallway, it’s pretty effective,” Rush says. “But if you’re talking about security,” he adds, “that’s orders of magnitude off…As a security mechanism, we need 99.998 percent accuracy.”

Advances in facial-recognition algorithms have begun to focus on small details on the face, such as wrinkles around the eyes, to improve accuracy, says Mika Rautiainen, an expert in computational linguistics and artificial intelligence at the University of Oulu in Finland.

“The challenge is in getting the identity correct in the way that the human would,” he says. While a computer may correctly identify 95 percent or more of a person’s face, that last 5 percent, depending on the technology’s use, could represent the difference between jail or freedom.

A spread in official use

The FBI, in facing condemnation from privacy advocates and federal colleagues alike for a perceived lack of internal oversight of its facial-recognition program, is fighting to make the biometric data it collects exempt from Privacy Act rules.

Police departments around the world, meanwhile, are eagerly deploying the technology.

In preparation for the World Cup in 2014, Brazil bought facial-recognition goggles that can take up to 400 photos of faces per second. Officials are expected to use them there again at this year’s Summer Olympics. Ireland is expanding its use of facial-recognition technology for its national police force, and China has developed police cars equipped with facial-recognition technology.

Getting better all the time

Data scientists’ constant algorithmic iterations are gradually improving the accuracy of facial recognition. Improved accuracy, for better or worse, is enabling broader uses of the technology.

Using facial-recognition technology, for example, users of the Russian-language 2chan imageboard recently outed porn stars to their families. Google enabled Android users to log into their phone or tablet using their face. A university in China developed an ATM that uses facial-image capture to verify a user’s identity, and Uber developed plans to use the technology to help users identify drivers.

“When the technology is done right,” Rautiainen says, “it can be something that helps the weaker get justice more efficiently.” But when it hasn’t been properly built, it could be just the opposite.

The accuracy of facial-recognition programs, Rautiainen says, largely depends on the quality of their algorithms and image databases. False identification is a common problem. Among other things in his field of expertise he says he avoids in part because of a high failure rate: using his face to log in to his Android phone.

And Lynch, of the EFF, remains skeptical about the technology’s effectiveness.

“I have not seen any examples of face recognition being used to thwart any kind of terrorist activity,” she says. “To identify the Boston Marathon bombers, the FBI relied on eyewitness identification.”