Facial recognition and emotion detection in environmental installation and social media applications

---

Russell Pensyl

Northeastern University, USA

: J Comput Eng Inf Technol

Abstract

---

Facial recognition technology is a growing area of interest, where researchers are using these new applications for study in psychology, marketing and product testing and other areas. There are also applications where the use of facial image capture and analysis can be used to create new methods for control, mediation and integration of personalized information into web based, mobile apps and standalone system for media content interaction. Our work explores the application of facial recognition with emotion detection, to create experiences within these domains. For mobile media applications, personalized experiences can be layered personal communication. Our current software implementation can detect smiles, sadness, frowns, disgust confusion, and anger. In a mobile media environment, content on a device can be altered, to create a fun, interactive experience, which is personally responsive and intelligent. By intersecting via direct communication between peer to peer mobile apps, moods can be instantly conveyed to friends and family – when desired by the individual. This creates a more personalized social media experience. Connections can be created with varying levels of intimacy, from family members, to close friends, out to acquaintances and further to broader groups as well. This technique currently uses a pattern recognition to identify shapes within an image field using Viola and Jones Open CV Haar-like features application and a “feret” database of facial image and support vector machine (LibSVM) to classify the capture of the camera view field and identify if a face exists. The system processes the detected faces using an elastic bunch graph mapping technique that is trained to determine facial expressions. These facial expressions are graphed on a sliding scale to match the distance from a target emotion graph, thus giving an approximate determination of the user’s mood.

Biography

---

Russell Pensyl (MFA 88, BFA 85) is an American Media Artist and Designer. His work maintains a strategic focus on communication, narrative, and user centric design processes for interactive and communication media. He is currently Full Professor in the Department of Art+Design at Northeastern University. Previous posts include, Director of Research and Graduate Studies at Alberta College of Art+Design, Director of the Interaction and Entertainment Research Center, Executive Vice Dean of the School of Art, Design and Media at Nanyang Technological University in Singapore, Chair of the Department of Digital Art and Design at Peking University. His current work includes the creation of location based entertainment several areas of technology in the application of content delivery in environmental spaces including facial recognition, positioning and localization, gesture recognition. His recent research in the use of facial recognition technology, positioning and augmented reality annotation is resulting in commercially viable communication technologies as well as user centric, autonomously responsive systems using biometric data in interactive installations. In 2010, his recent work explored the “subtle presence” autonomously responsive media in an interactive installation that presents a dynamic time lapse still-life painting that shifts subtly, caused by sensing personal characteristics of the viewer in the exhibition space. In 2011, this installation was featured in the International Sarajevo Winter Festival. In 2008, his mixed reality installation “The Long Bar” was a Curator Invited Installation into the SIGGRAPH Asia Synthesis – Curated Show/Art Gallery in Singapore. His exhibition credits include international exhibitions in China, USA, Japan, and Europe.