Commercial Face Recognition: Is It Time To Freak Out?
I recently watched the latest Mission Impossible movie, Mission Impossible 23: More Impossibler, and of course they used the tired Hollywood face recognition trope where they hack into a global network of surveillance cameras and find the person they are looking for instantly. There are so many problems with this from a technical standpoint, it is hard to know where to start. However, I will just hit on two of the most simple. The cameras used in the movie are relatively low-resolution, and mounted on the ceilings. That means they get small number of pixels, and at a terrible angle. Here is an example that is actually from a surveillance vendor's page, which presumably shows off the best they can do:
As you can see, there is very little detail on the faces, maybe 20 pixels at best. There is no technology, current or future, 2D or 3D that is going to be able to do anything useful with that. Pose angle and pixels are the most important part of making face matching work.
A recent Wired article is a little hysterical: "Companies are making money off your face!", so I wanted to address a couple of points from it. First of all, the fact that companies are making money off of anything and everything is just part of how our economic system works, so that doesn't bother me.
"After creating a “faceprint” from a photo of you, companies can then seek out that visage in their databases or in live video feeds from cameras in stores, airports, or border checkpoints. Security firm Safran, for example, can search faces in video footage against watch lists or track where a person has been and when they went there."
This is technically true, but misleading. A faceprint (technically called a template) is used to compare faces. However, this statement implies a "Hollywood style" connectivity of databases that just doesn't exist in real life. For example, Wal-Mart is allegedly experimenting with using face matching to identify shoppers at their stores. I'm not posting any links to reference this because all of the links I found are from conspiracy sites. But it is not far-fetched at all, so let's assume they are really doing this. If so, they can't really cross-reference that with, say footage from the airport. Those are different databases. In a similar fashion, the police cannot get a "live" feed from Wal-Mart's system. They would have to subpoena for footage from a particular date with probable cause, etc.
"Other software assesses your state of mind from your facial expression. Affectiva, for example, builds models that take into account everything from eye creases to lip quivers. With 11 billion data points from nearly 3 million faces, the company says it has translated emotions into information."
This is probably one of the biggest face-recognition snake oil sales I have seen in the past few years. Human emotions are super-complex things. Can you determine that someone is enraged and shouting with a red face? Sure. Could you detect that someone is mildly annoyed at their shopping experience? Maybe in some cases, but not most. It is just too complex of a thing to measure. The first time an automated kiosk tries to console me, I am probably just going to punch its digital face.
"Some companies use cameras embedded in street ads or hidden in a store to judge your age, gender, and interest level to decide what products or services to serve up."
This one is actually the most plausible, mainly because it gets around the pose angle problem. By embedding the camera at eye level in screens, ads or mannequins you get more pixels and a better angle. Here is a vision of what this might look like from the movie Minority Report. In the movie, they use iris matching, but the basic idea is the same:
You can certainly see some embarrassing situations arising out of this. "Mr. Anderton, how did that Viagra work for you last night?". But in reality I don't see this exact kind of scenario playing out like in the movie - just too much cacophony. I see it more working as an alert on your phone or as a private alert on something like the next generation of Google Glass.
Am I worried about the commercial use of face matching? A little. But not all that much. Companies will always do what they can to optimize advertising because it directly affects their bottom line. And they will always go up to the line of what people will accept and will (generally) not step over it. We'll get the level of privacy we as a society are willing to accept. I'm much more worried about government surveillance than private surveillance, because the government doesn't have to answer to customers in the same way as the private sector.