Computer being developed to read people's lips

A computer is being taught to interpret human emotions based on lip pattern, according to research published in the International Journal of Artificial Intelligence and Soft Computing.

 The system could improve the way we interact with computers and perhaps allow disabled people to use computer-based communications devices, such as voice synthesisers, more effectively and more efficiently.

How it was done

Karthigayan Muthukaruppanof Manipal International University in Selangor, Malaysia, and co-workers have developed a system using a genetic algorithm that gets better and better with each iteration to match irregular ellipse fitting equations to the shape of the human mouth displaying different emotions.

They have used photos of individuals from South-East Asia and Japan to train a computer to recognise the six commonly accepted human emotions - happiness, sadness, fear, angry, disgust, surprise - and a neutral expression. The upper and lower lip is each analysed as two separate ellipses by the algorithm.

Interaction between human and computers

"In recent years, there has been a growing interest in improving all aspects of interaction between humans and computers especially in the area of human emotion recognition by observing facial expression," the team explains. Earlier researchers have developed an understanding that allows emotion to be recreated by manipulating a representation of the human face on a computer screen.

Such research is currently informing the development of more realistic animated actors and even the behaviour of robots. However, the inverse process in which a computer recognises the emotion behind a real human face is still a difficult problem to tackle.

It is well known that many deeper emotions are betrayed by more than movements of the mouth. A genuine smile for instance involves flexing of muscles around the eyes and eyebrow movements are almost universally essential to the subconscious interpretation of a person's feelings. However, the lips remain a crucial part of the outward expression of emotion.

The team's algorithm can successfully classify the seven emotions and a neutral expression described. The researchers suggest that initial applications of such an emotion detector might be helping disabled patients lacking speech to interact more effectively with computer-based communication devices, for instance.

(EurekAlert, September 2012)

Read more: 

Computers and your eyes

Why do vivid memories feel so real

No more PC headaches

We live in a world where facts and fiction get blurred
In times of uncertainty you need journalism you can trust. For 14 free days, you can have access to a world of in-depth analyses, investigative journalism, top opinions and a range of features. Journalism strengthens democracy. Invest in the future today. Thereafter you will be billed R75 per month. You can cancel anytime and if you cancel within 14 days you won't be billed. 
Subscribe to News24
Voting Booth
Zama zama crackdown: What are your thoughts on West Village residents taking the law into their own hands?
Please select an option Oops! Something went wrong, please try again later.
Results
Authorities should bring in the army already
8% - 411 votes
Illegal miners can't be scapegoated for all crime
58% - 2962 votes
What else did we expect without no proper policing
31% - 1551 votes
Vigilante groups are also part of the problem
3% - 146 votes
Vote
Rand - Dollar
16.59
+0.2%
Rand - Pound
20.03
+0.2%
Rand - Euro
16.94
+0.0%
Rand - Aus dollar
11.55
+0.5%
Rand - Yen
0.12
+0.2%
Gold
1,794.52
+0.3%
Silver
20.54
-0.7%
Palladium
2,228.10
-0.7%
Platinum
936.01
-1.0%
Brent Crude
96.65
+1.8%
Top 40
63,771
0.0%
All Share
70,266
0.0%
Resource 10
65,045
0.0%
Industrial 25
85,829
0.0%
Financial 15
15,741
0.0%
All JSE data delayed by at least 15 minutes Iress logo
Editorial feedback and complaints

Contact the public editor with feedback for our journalists, complaints, queries or suggestions about articles on News24.

LEARN MORE