EmoHeart
With EmoHeart (”Emotional Heart”), avatars in Second Life can visually express the emotions which users convey in their (English) in-world instant messaging text.
Get your own EmoHeart from the following Second Life landmark!
Installation
The distributor of the EmoHeart object is located inside a (fictitious) Starbucks cafe of the Second Life replica of National Center of Sciences building in Tokyo, which also hosts the National Institute of Informatics (NII).
- Click on the heart and agree to take EmoHeart object.
- The object will appear in your Inventory/Objects: right-click on it, and select “wear” (object will be attached to the chest).
After you type an emotion-bearing message in the chat channel of Second Life, you will see the EmoHeart and a facial expression for a few seconds after you sent the chat message. If no emotion is detected in your message, the EmoHeart will remain invisible and the facial expression will remain neutral.
Please watch our EmoHeart video!
Research Background
Emotional expression is natural (and important!) in real life, but currently rather cumbersome in Second Life, where expressions have to be selected and activated manually. With the heart-shaped object of EmoHeart, we provide an additional channel for visualizing emotions in a vivid and intense way.
In order to achieve truly natural communication in virtual worlds, we set a two-fold focus in our research:
- Recognition (”sensing”) of affective content in text.
- Visualization of emotional expression for animated agents such as avatars in virtual worlds.
We introduced a novel syntactical rule-based approach to affect recognition from text. The key feature of the developed Affect Analysis Model is that it is designed to handle not only grammatically and syntactically correct textual input, but also informal messages written in abbreviated or expressive manner. “Sensing” of affect in text means that each sentence is automatically annotated with one of nine basic emotions (’anger’, ‘disgust’, ‘fear’, ‘guilt’, ‘interest’, ‘joy’, ’sadness’, ’shame’, ’surprise’), or neutral, and a numerical value indicating the degree of emotional intensity.
The control of the conversation in Second Life is implemented through the EmoHeart object (invisible in case of `neutral’ state) attached to the avatar’s chest. This object listens to each message of the avatar, (1) sends it to the Affect Analysis Model, (2) receives the result (dominant emotion and intensity), and (3) visually reflects the sensed affective state through the avatar’s facial expression and EmoHeart’ texture (indicating the type of emotion), and size of the texture (indicating intensity).
EmoHeart was developed by Alena Neviarouskaya of Ishizuka Lab at the University of Tokyo, as part of a joint research project with Prendinger Lab at the National Institute of Informatics. We would like to acknowledge the Stanford NLP Group for providing the Stanford Parser under GNU GPL. EmoHeart uses the Stanford Parser.
Here is a reference to the main publication on the Affect Analysis Model (AAM), the intelligent component underlying the EmoHeart application.
A. Neviarouskaya, H. Prendinger, and M. Ishizuka. User study on AffectIM, an avatar-based Instant Messaging system employing rule-based affect sensing from text. Int’l J of Human-Computer Studies, Vol. 68, No. 7, 2010.7, pp 432-450 [Online Version]
Other relevant papers can be found on the homepages of Helmut Prendinger and Ishizuka Lab.
