Is Snap Inc. Building a Wearable Face Recognition Device for the National Security Agency?

For folks too busy having a real-world life to keep up with this stuff, Snap is the recently created parent company of double-decacorn Snapchat (think Google/Alphabet) which, deciding it would be more than just the app, rolled out video recording "Spectacles" to do image capture and upload:

They would now like you to think of them as a "digital lifestyle" platform, thank you very much.
They've also hired bookrunners for a potentially blockbuster IPO early next year.

From Hackernoon, Oct. 3:

Could Spectacles be an elaborate distraction?

Last week, Evan Spiegel of Snap Inc. unveiled his first hardware product Spectacles to a few journalists. Wall Street Journal author Seth Stevenson recalls how Spiegel invited him into “a small conference room” where he “draped a towel over a mysterious object sitting on a table” calling Spiegel “eager to the point of jitters”.

Perhaps “eager” isn’t the right adjective.

So a white-hot, consumer-focused company with a $20-Billion valuation reveals its first-ever spiffy gadget via a cramped conference room… with a towel.

What a complete and utter lack of fanfare.

What changed in Silicon Valley?

Were all the stages and conference halls booked in Palo Alto that day?

This is the flagship hardware launch of one of the hottest entities in the valley. Surely they want a buzz around their new device.

Yet… no spotlights or smoke machines? No music? No crowd? Not even a black turtleneck?

None in sight.

Additionally, the product which Spiegel refers to as a “toy” doesn’t seem meaningfully distinguishable from Epiphany Eyewear — video-recording glasses developed by Vergence Labs whom Snapchat acquired in 2014.

Tech journalists know that Snap Inc. has been working on a secret project for months, possibly years, all-the-while hiring up all the best electronics and robotics talent in the industry. This massive effort finally culminated to build… a toy?

Could the recent press release about Spectacles be an elaborate distraction to take attention off of a more unsavory product?

Acquisitions, Hires & a Patent

Snapchat may have indicated in a July patent published at the United States Patent and Trademark Office that they have developed facial-recognition device which displays personal information within seconds of a facial scan. The patent details a means of “executing a facial recognition technique against an individual face within the image to obtain a recognized face”.

This patent comes after their recent acquisition of Vergence Labs, known for developing Epiphany Eyewear — a product similar to Google Glass, as well as a string of high-profile hires in the consumer electronics industry. These newly-hired hardware specialists reportedly joined a secret research and development lab according to a March article by CNET. Their previous work ranges from wireless-video doorbells, security cameras, robotic Star Wars toys, Google Glass, GoPro, and the Oculus VR headset according to a recent Financial Times article.

Also, they reported Snap Inc. was “looking at pretty much every AR startup with computer vision skills” as a target for a possible acquisition.

Up until last Saturday, Snapchat still had not publicly announced any plans to develop hardware. It wasn’t teased at all until rumors started circulating. The conclusion was pretty much a slam dunk when Financial Times journalists discovered Snap Inc.’s move to pay and join the Bluetooth consortium which they called a “clear signal of intent” to develop hardware.

So, if press about their secret operation was the catalyst for them to pass off Spectacles’ 2-year-old product as something new, what is it that they’re really working on?

Gathering Facial Profiles With “Lenses” Feature?

In order to use the silly-cartoon-face-making “Lenses” feature on Snapchat, the interface instructs the user to tap on their face which initiates a facial scan. This captures the user’s face for a seemingly-temporary period so they can apply silly dog ears and rainbow barf to their heart’s content and send it to their friends.

How “Lenses” Could Train a System to Recognize a Face

Adam Geitgey’s Medium article Modern Face Recognition with Deep Learning explains how accurate facial recognition relies on a system’s ability to “pick out unique features of the face that you can use to tell it apart from other people — like how big the eyes are, how long the face is, etc”. And the system must also be able to “compare the unique features of that face to all the people you already know to determine the person’s name.”

One method of face recognition is to program a system to compare measurements of obvious facial landmarks like the outside edges of eyes or top-of-the-chin to mouth etc. but the most accurate way for a system to reliably recognize a face is to let it decide which measurements matter most by feeding it millions of faces.

Determining these mysterious measurements is resource-intensive but highly accurate. Luckily, services like OpenFace have processed the millions of face images necessary to discover the 128 unique measurements that make for an accurate result. Using a service like this, any 10 different pictures of the same person should give roughly the same measurements.

In Machine Learning, capturing these vital 128 facial measurements is called “embedding”. These measurements are unique to almost every human being.

To capture a person’s facial signature, an algorithm must first encode their facial features using a method called HOG (Histogram of Oriented Gradients) which outputs a simplified image that is basically a flattened-and-centered set of the subject’s primary facial features. That output is then passed through a neural network that knows which 128 measurements to make and saves them.

With our face captured, all the system has to do to identify someone is compare the measurements to those of all the facial measurements captured for other people and figure out which person’s measurements are the closest to find a match....MORE