Facial Analysis: Wall Street's Next Big-Data Tool

9 Google Apps Tips: Productivity Boosters

Facial expressions can provide a window into a person's unspoken thoughts. But can these nonverbal tells provide real-time insights for financial traders?

That's the goal of New Jersey Institute of Technology (NJIT) researchers, who are developing ways to filter and analyze corporate data to forecast future market trends. This big-data effort encompass a variety of information streams: including video of a CEO's presentation during a quarterly analyst call, in which analytics software scrutinizes the chief executive's facial or tonal changes; real-time speech-to-text conversion of the CEO's comments; and a live feed of the company's stock price and other market data.

This effort includes the work of Dr. James Cicon, an assistant professor of finance at NJIT's School of Management. Cicon focuses primary on empirical corporate finance.

[Google's revised version of CAPTCHA promises to make life easier for users. Read Google Gets Better At Spotting Humans.]

"Specifically, I look at boards of directors, CEOs, things like that," said Cicon in a phone interview with InformationWeek. "That's my big interest. I try to see if I can get more information to put in our financial models than perhaps [companies are] explicitly providing."

During a quarterly analyst call, for instance, real-time facial analysis would require a meticulous and speedy examination of a speaker's rapidly changing expressions, a task the NJIT system is designed to do.

"We track certain areas of the face," Cicon explained. "We look at the eyes -- the shape the eyes take during the course of the interview. We look at the eyebrows. We look for a downturned or widening mouth, or upturned lips."

NJIT's software extracts measurements of primitive emotional states, such as disgust, fear, and surprise. Its analytical capabilities are based in part on research conducted decades ago by renowned American psychologist Paul Ekman, who developed a comprehensive system for describing observable facial movements for emotions. "We can measure those facial expressions," said Cicon. "It's kind of straightforward if you have the machinery to look at a face and compare [it] with what Paul Ekman says a fearful face looks like."

Real-time measurement of emotions could prove useful in many fields, such as law enforcement and stock trading. A fearful-looking CEO on a conference call, for instance, might interest Wall Street traders hoping to profit from the manager's facial clues.

"You could take these primitive Ekman measures and combine them into higher-level measures as well," said Cicon. "For example, what we'd like to do is have a… lie detector running, or something like that. Does the manager believe what he's saying? This would be very useful to an investor."

Wall Street types could combine facial analysis with data from other sources as well. "You might do a live transcription where you take the manager's voice… and create a text stream from the words being said," Cicon said. "We can do content analysis of someone's speech to determine emotional states as well."

Given its ability to detect emotion states from a distance, facial analysis might have a future in law enforcement, too. "You can't hook someone up to a lie detector without their permission, so what we see is some type of a lie detector application. We wouldn't necessarily call it a lie detector. We might call a 'measure of sincerity.'

"Sincerity is the difference between what you're saying -- your words -- and what your face is expressing. A person who is saying one thing, but whose face appears to be telling us something else, might be considered 'insincere.' "

Facial analysis will soon find its way into real-world applications, Cicon predicted. "You have an academic paper written about something, and a couple of months later [stock] traders are using it as fast as they can. They recognize that academic research adds a lot of value to their work and gives them an advantage."

Apply now for the 2015 InformationWeek Elite 100, which recognizes the most innovative users of technology to advance a company's business goals. Winners will be recognized at the InformationWeek Conference, April 27-28, 2015, at the Mandalay Bay in Las Vegas. Application period ends Jan. 16, 2015.