by Barry Chudakov on March 12th, 2010

Face Recognition

Face perception is so hardwired into our brains, it comes as no surprise that we describe social status in terms of face: saving face, losing face, face-off. Showing your face and owning it—owning the right to present it on your own terms, even to sell it if you’re blessed with beauty—is the height of personal empowerment. But our features are also the focus of emerging facial recognition systems that will significantly alter how we think of owning and presenting our faces.

Source: Flickr, Simone Kesh, all rights reserved.


Historically the Metalife of face has played out one way behind the burqas of the Middle East, and quite another way in the Life The Movie culture of the West. Today we can see that face is an emerging new currency, a way of creating both monetary and personal value. Hidden and forbidden, the face in hijab becomes clandestine, mysterious; in visual culture the face is idolized, adored, painted, Botoxed, surgically rearranged, and becomes the most lucrative chip at the table of image poker.

Against this historical and political backdrop arrive the augmented reality technologies of facial recognition. Although we all intuitively recognize someone from his or her facial features, facial recognition is a biometric that makes human recognition a computerized process. New tools like TAT Recognizr and PhotoTagger—augmented reality tools that enable others to find and tag you by your face—create a counter logic to our intuition and elicit a new narrative about faces and what they mean. The word recognition comes from the Latin recognito, to know again. These technologies are compelling us to re-cognize or know anew what the human face means.



One man who has spent a lifetime deciphering faces is Dr. Paul Ekman, co-author of Emotion in the Human Face (1971), Unmasking the Face (1975), Facial Action Coding System (1978), and author of Face of Man (1980), and What the Face Reveals (1997), among other titles. Dr. Ekman is the inspiration behind (and scientific advisor to) the hit Fox TV show, Lie To Me. Using video capture to reveal techniques for understanding so-called micro-expressions, in Lie To Me Dr. Cal Lightman (Tim Roth) directs a supporting cast who seek to understand and then apprehend criminals and others who are lying to our faces.


Lie To Me facial expressions. Source:


Here is Dr. Paul Eckman explaining lie detection, using Kato Kaelin as his subject:



As Dacher Keltner writes in Born To Be Good, Ekman and Wallace Friesen devoted seven years without funding or promise of publication to developing the Facial Action Coding System (FACS), an anatomically based method for identifying every visible facial muscle movement. They used frame-by-frame analysis to study human facial expressions: the tools of lens and video captured and froze expressions, then enabled an elaborate coding system.


Facial Action Coding System, Flickr


Where Ekman was interested in studying and cataloging facial expressions to discover a science of emotion that could find the truth behind a verbal smokescreen, newer augmented reality tools take your physiognomy at face value, turning your likeness into a visual link and identifier, a de facto connector, a kind of Tinkertoy hub that fits you into a larger universe of networked connections.

In a UX Matters post entitled, “Playing Well with Others,” Joe Lamantia describes how facial recognition is likely to show up soon in existing social networks:

“A user, holding a smartphone or other device camera, focuses on a person’s face, in close view, long enough to register and recognize the person’s facial image, then waits while the service interrogates that person’s digital identity profile—which he or she has possibly defined, but may be ad-hoc and aggregated. The user then lowers the phone and visually scans the profile or other information the facial-recognition application has retrieved, perhaps choosing to focus on a specific pool of Facebook or another service’s shared-lifestream content, perusing it to get a sense of a person’s status, interests, and usual activities. Finally, the user may decide to engage in conversation with that person—or may perhaps choose not to do so.”


Face scan used in facial recognition systems. Source: Endtimes Worldnews.


Reality augmented by face. Your face. Never before in human history has something so essentially ours been so essentially not ours. This is the new Metalife of face, the sudden divorce of who you think you are from what others see in you or intend for you. These AR tools are just the latest reminders that your face is sliding off its customary perch at the front of your self-presentation, winding up in places you never dreamed or in fact you know nothing of. This essential badge of youness is speedily being adopted by mere strangers as evidenced by technologies that are already in use around the world:

  • The London Borough of Newham, in the UK, has tested a facial recognition system built into their borough-wide CCTV system.
  • The German Federal Police use a facial recognition system to allow voluntary subscribers to pass fully automated border controls at the Frankfurt Rhein-Main international airport.
  • Since 2005 the German Federal Criminal Police Office has provided centralized facial recognition on mugshot images for all German police agencies.
  • In Las Vegas and elsewhere, casinos use facial recognition systems to catch card counters and other blacklisted individuals.
  • The Australian Customs Service has an automated border processing system called SmartGate that uses facial recognition. The system compares the face of the individual with the image in an e-passport microchip, certifying that the holder of the passport is the rightful owner.
  • The Pennsylvania Justice Network searches crime scene photographs and CCTV footage in their mugshot database of previous arrests. A number of cold cases have been resolved since the system became operational in 2005.
  • Law enforcement agencies in the US and abroad use arrest mugshot databases in their forensic investigative work.
  • The U.S. Department of State operates one of the largest face recognition systems in the world: over 75 million photographs are actively used for visa processing.
  • “Because a person’s face can be captured by a camera from some distance away, facial recognition has a clandestine or covert capability (i.e., the subject does not necessarily know he has been observed). For this reason, facial recognition has been used in projects to identify … shoplifters in stores, criminals and terrorists in urban areas.” [Source: “Biometrics, A Look at Facial Recognition”]
  • Surveillance cameras are set up in areas known as “FaceTraps.” These are places where the target’s face can be readily captured to create a high-quality image. Such FaceTraps might include an airline check-in counter or a shopping mall escalator with a red flashing light situated above a clock at the top of the escalator.
  • To identify potentially dangerous individuals, the Transportation Security Administration has stationed specially trained behavior-detection officers at 161 U.S. airports. The officers may be positioned anywhere, from the parking garage to the gate, trying to spot passengers who show an unusual level of nervousness or stress. [source: Ken Kaye, Washington Post, 9 Nov 09]



In Sonnet 94, Shakespeare wrote:

    They that have power to hurt and will do none,
    That do not do the thing they most do show,
    Who, moving others, are themselves as stone,
    Unmoved, cold, and to temptation slow,
    They rightly do inherit heaven’s graces
    And husband nature’s riches from expense;
    They are the lords and owners of their faces,

But what does it mean today to be the lord and owner of your face?

Even Dr. Paul Ekman cannot tell us what this new Metalife of face means because our adoption of facial recognition technologies—and our response to that adoption—is still too early, too new. What we can see from our present vantage point is that facial recognition is part of a larger picture that I call the deconstruction of you. This deconstruction is currently relegated to the scientific province of biometrics, defined as “the automatic recognition of a person using distinguishing traits.” These biometrics include:

  • Iris scan
  • Retinal scan
  • Facial recognition
  • Speaker/voice
  • Fingerprint
  • Hand/finger geometry
  • Signature verification
  • Keystroke dynamics
  • Gait
  • Ear shape
  • Body odor

In essence, face becomes an element or artifact in the larger deconstruction and revaluing of you for purposes you have not deliberately intended—for example facial profiling at an airport. (Even though as a US citizen you might agree that better airport security is desirable.) The key is that our faces are so intimately ours, so basic to our sense of identity that there is hardly anything more personal or uniquely our own. How then do we countenance allowing others to use that face, often for purposes we have not agreed to or may not even know about?


A fingerprint being compared to an iris scan. Source: Flickr, tampaempire, all rights reserved.


I am suggesting that we consider the face as a gateway to a larger discussion of how biometric captures and artifacts create Metalife incarnations. This discussion starts by asking, how do we assert agency, how do we manage (or assert a measure of control over) our own faces and—by biometric extension—our own bodies in newer virtual environments? Clearly it is time to start framing discussions around emerging—and often morphing—technologies that have the potential affect us in ways we are only beginning to imagine. We must define the issues that will galvanize us; we must press for more and better answers.

As TAT Recognizr and PhotoTagger and other augmented reality tools enable finding, surveillance and tagging identity via the face, a host of new questions arise. Assuming that passive spectating from the sidelines is not an option, here are some of the topics inherent in widespread adoption of facial recognition technologies that will change how others see us, how we will see ourselves, and ultimately may change how we face the future:


Face Recognition: So What, Now What?

1. Any artifact of you has the potential to create a Metalife—especially due to the enlivening and networking of information.

2. Given newer and emerging technologies, your face can now be considered an artifact.

3. Once your face becomes an artifact, it is functionally prepared to become an object in the “Internet of Things.” As an object your face can be conveyed and manipulated: it can be an actuator of information about you, linked through wired and wireless networks.

4. While our notions of privacy and anonymity are undergoing profound changes as we port our identity to ever more elaborate and involving virtual environments, face detection and recognition can occur without our knowledge or consent. At what point will facial recognition applications cross the line to violate legally protected privacy rights? Does your Metalife have the same rights to privacy that the physical you does?

    a. For example, if your face has been captured and scanned in a FaceTrap, do you have a right to know? [United States v. Miller, 425 U.S. 435 (1976) suggests we do not have a legal right of privacy in the facial features we show in public.]
    b. Even though an application like TAT Recognizr only works when both persons have subscribed to the service, will privacy be eventually compromised by hackers and others who seek to circumvent safeguards?

5. As a result of the above, your face is now an actor in the drama of the drag-and-drop self. Your Metalife, your persona—and now your face—can be captured and then show up almost anywhere. While this may be an alarming development to some, the emerging reality of face recognition demands that we stay vigilant to privacy concerns and take on the new role of managing our face and self in a host of digital environments.

6. Face recognition and related technologies effectively deconstruct you. As David Weinberger wrote, now everything is miscellaneous. This has broad Metalife implications. Via various forms of capture and transmission, these become artifacts that can also become objects:

    a. Your body
    b. Your body’s (medical) history
    c. Your family history (birth, death, geneaology)
    d. Your devices (iPhone, iPad, computer)
    e. Your story (in the “database of intentions” found by search)
    f. Your representations (driver’s license, green card, etc.)
    g. Your biometric identifiers

7. As our faces and personal artifacts all become miscellaneous bits of information that then become objects, our Metalife extends indefinitely.


Related Information



F.A.C.E. Training

Google wants your face

Google Goggles

Lie To Me


Polar Rose

Playing Well with Others

Rand Facial Recognition


The Internet of Things


Email This Post Email This Post     Print This Post Print This Post

From Trends

  1. After reading this I can see low tech head scarves becoming a fashion trend…and a way of controlling access to our visages. I suspect the negative aspects of face recognition will be coming soon.

  2. This is very good blog very good information on CCTV Security products,its really awesome, We have Similar website you can visit us to

    Cctv Face Recognition

Trackbacks & Pingbacks

  1. Heartbeats | Metalifestream
  2. Search and Recognition | Metalifestream

We invite your thoughts and comments about this post. Leave a reply here.

Note: XHTML is allowed. Your email address will never be published.

Subscribe to this comment feed via RSS