A Scary Demonstration of What Unchecked Face Recognition Could Look Like

A face recognition and video analytics company has created a product that provides a stark reminder of the power of these technologies and how they are likely to be used over time by law enforcement, powerful corporations, and others, if we as a society allow it.

The technology in question involves video search, which we described in our 2019 video analytics report. In the past, video operators looking for something would have to manually scroll through many hours of video, but technology is increasingly automating such searches. In a presentation for subscribers of the surveillance research group IPVM, a company called Vintra presented on its technology for quickly searching through large stores of video footage

The relevant three-minute part of the full presentation is worth watching. In it, a company executive searches through a month’s worth of video footage captured by around 10 fixed cameras, plus body cameras, in a transit center in San Jose, California. He feeds the system with the photograph of a male subject, and the system does a face recognition search through all the stored video from that month and produces 23 snapshots of the man from the center’s cameras. Clicking on any of the snapshots plays the video in which he was captured.

Already, that’s a demonstration of the stunning new power that surveillance camera systems create when combined with face recognition and today’s search capabilities.

But there’s more. The Vintra executive then presses a button called “Find associates.” He selects a time period — he uses 10 minutes but it could have been shorter or longer — and then runs a new search. This search yields snapshots of 154 other people, each of whom was seen on camera within 10 minutes of the subject.

In other words, this system allows face recognition to be used to track not just one person, but to map out people’s associations with each other.

Of the people spotted with the subject in the demo, 150 appeared on camera with him only once, and another three appeared with him twice. One man, however, had 14 “co-appearances” with the subject — clearly not coincidence, but a result of some association between the two men. The system displayed snapshots of the 14 co-appearances, and clicking on them instantly played the video of the two of them together.

The men could be anything from co-workers to commuting partners to lovers. Perhaps clicking through to view their joint appearances would shed light on which. But whatever the case, their association has now been revealed to the prying eyes of this camera network and its operators. One of Vintra’s mottos is “Know what the cameras know,” and if this product lives up to the demo, it’s a spookily accurate slogan, not least because it captures the way that AI is allowing video cameras to “wake up” — rendering them able not just to dumbly record video, but increasingly to understand what they’re seeing.

With this kind of technology, as the Vintra pitchman put it, “You can really start building out a network. You may have one guy, that showed up a few times, that you’re interested in — you can start looking at windows of time around him to see who else is there at the same time, and build out the networks of those people.”

Too many conversations about surveillance focus on how information could be used in isolation against a specific individual. But analytics is a powerful tool, and when information is collected not about just one suspect, but about large numbers of people, we often forget that such data can be cross-referenced to create maps of associations. I wrote this piece in 2013 to try to hammer home that often non-intuitive point, but maps of people’s associations (called “social network analysis”) have long been a product of mass surveillance. It has been done using cellphone data by the NSA, and by the U.S. military overseas using wide-area aerial surveillance, for example.

Now, face recognition and other analytic techniques appear to have brought social network analysis to video surveillance. And who knows what purposes such mining could be used for. The Vintra pitchman told his security audience that his product “will plug in to BI tools” — referring to Business Intelligence, a catch-all buzzword referring to non-security uses of data such as competitive research and marketing: “You may be using the cameras for security, but 94, 96 percent of the time there’s no event that security’s interested in — but there’s always information that the system is generating on those that you can plug into your BI.”

The bottom line is that when we see a video camera today, we need to update our intuitions about what it’s capable of. It may no longer be just collecting inert and unused video, but, especially if that camera is part of a larger network, the data it collects could be mined for insights about our lives across space and time. Communities and policymakers considering the installation of surveillance cameras — especially camera networks — should take heed.