03 About IMIimi.ntu.edu.sg/About_IMI/Documents/AAR/IMI_Annual... · IMI is an incubator of multidisciplinary, cutting-edge and media-related research ideas. ... Through an easy to

1

2

03 About IMI

03 IMI Missions

07 Research @ IMI

17 PhD Programme in New Media

33 International Events organized @ IMI

41 Industrial Collaboration

45 BeingThere Centre (BTC)

61 Publications

3

About IMI

IMI Missions

To develop cutting-edge new media research within

IMI.

To lead the international collaboration BeingThere

Centre on 3D Telepresence at NTU.

To create synergy and interactions with NTU schools

to empower New Media interdisciplinary research

through integrated projects and joint PhD

supervisions.

To promote and develop industrial applications in

Singapore and elsewhere in the world.

Dissemination& IndustrialApplications

InterdisciplinaryResearchProjects

BeingThereCentre

IMI Core Research

IMI is an incubator of multidisciplinary, cutting-edge and media-related research ideas. It is directed by Professor Nadia Magnenat

Thalmann who is a world pioneer in Interactive Digital Media (IDM) research, particularly in all aspects of Virtual Humans simulation. IMI

strives to build a strong reputation as a global leading IDM hub. In response to Singapore’s national priority to develop IDM as a strategic

R&D area, IMI aspires to facilitate and promote cross-disciplinary collaboration in media research at NTU. At the same time, it also seeks

to provide a one-stop interface to the outside community by forging partnerships with the Singapore government, industry players, and

other international entities. Another important role of IMI is to enrich the educational experience of students by exposing them to the

multifaceted field of IDM.

The Institute is dedicated to creating an environment where technology and creativity can coexist and develop. The general objective of

IMI core research is to create true synergies between the real and the virtual worlds and to develop interactions between real people and

virtual or artificial creatures like virtual humans, virtual animals and robots in a way of creating a real social relationship.

IMI attracts talent researchers from over 15 countries working in this organization.

4

“...This research institute is dedicated to creating an environment where technology and creativity

can coexist and develop.”

NTU President and IMI Founding Chairman

Professor Bertil ANDERSSON

5

“IMI strives to be an incubator of multidisciplinary cutting edge media related research ideas and

establish Singapore as a key player at the forefront of the global interactive digital media

innovation.”

Director, Institute for Media Innovation

Professor Nadia MAGNENAT THALMANN

6

IMI Core Group

1st row from the left: Mr RIVERA Xerxes Marlon Jr. N, Mr Qui C.T. TRAN, Mr Derek HO, Mr KOH Kian Hua, Dr ZHANG Zhijun, Dr Aryel

BECK, Dr XIAO Yang, Ms KHIN Yaminn

2nd row from the left: Prof Daniel THALMANN, Dr Frank Yunqing GUAN, Ms Jess LIM, Mr Roman VLASOV, Prof Nadia MAGNENAT

THALMANN, Mr William LAI, Dr Samuel LEMERCIER, Dr Zerrin YUMAK, Dr Jun LEE

7

8

Core Research Areas

The general objective of IMI Core Research framework is to create true synergies between the real and the virtual worlds. This means the

development of natural interaction between real people and virtual or artificial creatures like virtual humans, virtual animals and robots in

a way of creating a real social relationship. IMI core research areas include 3D Telepresence, Virtual Humans and Social Robots, 3D

Serious Games, 3D Crowd Simulation, 3D Immersion, 3D Medical Simulation and 3D Fashion.

9

Virtual humans are social agents in the virtual world, while social robots belong to the real, physical world. They are meant to interact with

each other as well as with real humans. In order to create believable and long-lasting relationships, they must be empowered with

personality and memory, and be able to express artificial emotions and moods. To achieve autonomous behaviour, they must be aware of

their social environment, understand what is going on around them, who is doing what, what are the relationships between people, and

how humans feel.

Virtual Humans and Social Robots

Nadine is one of the most realistic female humanoid robots around the

world and is the first to be modelled after a western female, Prof Nadia

Magnenat Thalmann. It has been constructed by Kokoro’s Actroid

technology. This robot has a realistic artificial skin and is able to display

very realistic facial and body expressions. It also has a controllable

upper-body capable of realistic human movements. Nadine has a total of

26 DoF for facial expressions and upper body movements. Nadine can

speak, display emotions, understand some situations and express natural

gestures. This makes it ideal for the study of Human-Robot Interaction.

10

IMI is working together with MIRALab at the University of Geneva on a Virtual Try On application which combines virtual models, virtual

fashion and garment simulation technologies into a single interactive application. Research has been led on fast physics based

simulation models and interactive intuitive interface. In the application, the user has the possibility to dress a virtual mannequin by

choosing a garment from a selection of clothes. The garment will automatically adapt to the 3D mannequin shape by means of a

physical simulation. Through an easy to use touch-screen manipulation, the user can interactively change the fabric colour, fabric types,

modify the design and select other options which will be immediately visualized. Once the mannequin has been dressed, the user can

see her walk around in the garments he/she selected. Knowing that the creation of a real dress needs less than one week, this rapid

prototyping technology will dramatically shorten the time to design clothes according to the customers’ taste.

3D Fashion

11

Crowd simulation is the process of simulating the movement of a large number of virtual humans or characters. Virtual cities have

become common in video games and other virtual applications. However, most of them are uninhabited or with only a few people.

Benefiting from crowd simulation technology, it becomes possible that large-scale virtual environments are populated with up to tens of

thousands of virtual humans. Research in crowd simulation has been widely explored in a number of fields such as computer graphics,

civil engineering, sociology and robotics. Crowds have wide applications for both real-time and non-real-time simulations. Virtual crowds

are used to simulate epic battles in the movie industry. Crowds are exploited to populate virtual worlds in computer games. In safety

training, trainees can participate in the evacuation scenarios to practice controlling crowds in emergency situations like earthquake, fire,

flood, etc.

The research of crowd simulation in IMI focuses on the interaction and immersion of the virtual crowds. The immersive 3D display

system has been successfully built, which has great potential for the training and evaluation of emergency evacuation and other real-

time applications of crowd simulation with interaction.

3D Crowd Simulation

12

3D Serious Games

We develop educational games for learning and training purpose with the aid of virtual

tutors or virtual trainers. This research is cross-school and interdisciplinary with inputs

from School of Mechanical & Aerospace Engineering, School of Computer

Engineering, National Institute of Education, School of Art, Design and Media. The

research program also partners with local companies including Underwater World

Singapore, PEC Ltd and AWWA Special School. Part of the projects under the

program received funding support from industry.

IMI is working on 3D medical simulation to develop novel 3D interactions and

immersive techniques for medical simulations. The project will allow medical students

in their early training to access realistic information (anatomic and medical) in a

manner of low cost, fidelity and easy to repeat. They also can learn how to solve

problems in medical emergencies e.g. recurrent Obstetric Management

Mistakes(OMM). With the collaboration of the Lee Kong Chian School of Medicine,

we apply various surgical simulations such as a baby birth simulation. Students can

have surgical training using virtual patients with different parameters to perform

surgical operations using interactive devices (e.g., cybergloves) and predict optimal

surgical outcome. This study is very interdisplinary as it implies medicine, biology,

visualisation and interaction techniques. New Media is essential to this research as it

allows visualizing realistic medical simulation with proper user interactions.

3D Medical Simulation

13

Professors Associated to IMI through Interdisciplinary Seed Projects

Assoc Prof HE

Ying, SCE

Asst Prof Ina

Conradi CHAVEZ,

ADM

Asst Prof Justin

DAUWELS, EEE

Asst Prof YUAN

Junsong, EEE

Prof Daniel

THALMANN, IMI/SCE

Assoc Prof CAI

Yiyu, MAE

14

Interdisciplinary Seed Projects

Integrating a User within a Group or a Crowd using Computer Vision and

Synthetic Vision

Period of Funding: 1 Dec 2012 - 30 Jun 2014

PI: Asst Prof YUAN Junsong (EEE)

Co-PI: Prof Daniel THALMANN (SCE)

The project consists in integrating smoothly a user (avatar) inside a moving Virtual

Group in front of a large 3D screen (or if possible in the IMI Immersive room). The

Virtual Group will be composed of autonomous Virtual Humans (agents). Gestures

and actions of the user will be analysed using Computer Vision, e.g. regular and depth

cameras. One original aspect of the project is to combine the classical Computer

Vision with Synthetic Vision as first introduced by Renault et al [[1]]. Agents will be

able to navigate and avoid collisions using the information coming from a camera

(Computer Vision) [2,3,4] and using a synthetic vision rendered from their point-of-

view.

3D Interactive Visualization of the Human Body in an Immersive Situation

Period of Funding: 1 Mar 2013 - 28 Feb 2014

PI: Assoc Prof CAI Yiyu (MAE)

Co-PI: Prof Nadia MAGNENAT THALMANN (IMI)

Clinical Collaborator: Dr TEO Yee Hong (TTSH)

International Collaborator: Prof Franz-Erich WOLTER (Leibniz University Hannover,

Germany)

This project aims to develop novel 3D interactive and immersive technique for medical

education application. The project if developed will allow medical students in their

early learning journey to access realistic information (anatomic and medical) in a

manner of low cost, fidelity and easy to repeat. Based on this project, we will apply to

various surgical simulations such as obstetric management and knee recovery after

orthopaedic surgery. Students can have surgical training using virtual patients with

different parameters to perform surgical operation using interactive devices (e.g.,

cybergloves) and predict optimal surgical outcome.

15

Robust and Accurate Face Segmentation and Expression Transfer for

Autonomous Avatars

Period of Funding: 1 Jul 2012 - 28 Feb 2014

PI: Assoc Prof HE Ying (SCE)

We will be using a completely new paradigm for 3D facial expression acquisition and

retargeting. It envisions a suite of new algorithms and tools that allow users to acquire

analyze process and visualize 3D facial expressions more easily and effectively than

conventional methods. It will provide high resolution facial motion data without

requiring the subject to wear cumbersome suits or tracking devices and markers and

different expressions of the same subject can be mapped to the same canonical 2D

domains. As a result, the captured data are stored in Geometry Video, a highly

compact representation that can be easily visualized and steamed. Therefore, the

captured data can be widely applied, with the potential to revolutionize a variety of

real-world applications in the entertainment industry, biomechanics, mass

communication, medical applications, etc.


Virtual Pink Dolphins – Phase II

Period of Funding: 1 Oct 2010 - 31 Jan 2014

PI: Assoc Prof CAI Yiyu (MAE)

Co-PIs: Asst Prof CHIA Kok Hwee (NIE)

Prof Daniel THALMANN (IMI)

Assoc Prof ZHENG Jianmin (SCE)

Prof Nadia MAGNENAT THALMANN (IMI)

Mr Norman Kiak Nam KEE (NIE)

Asst Prof Kenneth FEINSTEIN (ADM)

This project is interested to investigate the modeling, simulation and visualization of

virtual pink dolphins for the purpose to design serious games to replace the real ones

for applications in special education, therapy, and entertainment. Virtual Reality

technology will be studied aiming to equip the virtual pink dolphins with multi-sensorial

functions (visual, audio, and tactile) for human-dolphin communication. Special efforts

are being made to study the virtual pink dolphin assisted therapy (VPDAT) and

human-mediated learning for Autism Spectrum Disorder (ASD) children.

16


Monitoring of Group Discussions: A Socio-engineering Approach

Period of Funding: 1 Dec 2011 - 31 Dec 2013

PI: Asst Prof Justin DAUWELS (EEE)

Co-PIs: Prof Nadia MAGNENAT THALMANN (IMI)

Prof Daniel THALMANN (IMI)

Asst Prof Martin CONSTABLE (ADM)

Human group interaction is very complex and rich. Monitoring any group can be done

using non-verbal (facial expressions, body languages, gestures, etc ) or/and verbal

(voice tone, voice pitch,...etc) signals. Despite the impressive advancements in video-

audio signal processing, analyzing different aspects of human interaction as empathy,

hostility, (dis-)agreement, flirting, dominance, superiority, inferiority, etc. remains very

challenging. Investigating these aspects is crucial for building intelligent robots that

can participate in conversations in a natural way, or to fulfill certain social

responsibilities in our life. As our ultimate goal is to improve the current computer

systems and social robots that suffer from a lack of social skills, we will explore the

social signals and social behaviours, including social interactions (like turn taking and

backchannelling), social attitude (like alliance), and social relations/ roles to build more

socially intelligent model for robots.

Unframed

Period of Funding: 15 Aug 2011 - 31 Dec 2013

PI: Asst Prof Ina Conradi CHAVEZ (ADM)

Co-PI: Asst Prof Mark Joseph CHAVEZ (ADM)

The project goal is to use IMI extraordinary creative platform (The Immersive Room)

as a space to author new media art experimentation. The works will explore surreal,

abstract and realistic way of painting using medium of 3D stereo animation.

17

EEE29%

MAE13%

HSS3%

SCE47%

WKWSCI8%

18

PhD Programme in New Media

“IMI offers doctoral students a vibrant multidisciplinary research environment. In particular, they have

an opportunity to work in the area of cutting-edge interactive 3D simulation with top equipment as the

IMI Immersive Room and real-time motion capture in a multidisciplinary team. Through their personal

work, seminars, courses, and interaction with research experts, PhD candidates are prepared for

research positions in leading academic institutions as well as private and public organizations.”

The PhD Program, along with the Interdisciplinary Graduate

School (IGS) is at the core of IMI's effort to promote New Media.

Playing a crucial part in ensuring the future of New Media in

Singapore, IMI aims to educate the next generation of scientists

and technologists and encourage them to share and benefit from

their knowledge of engineering, art, design, business,

educational and behavioral research. The 38 PhD students

pursuing their PhD study in IMI come from different NTU schools

and are specialized in a wide spectrum of research disciplines.

Professor Daniel THALMANN

PhD Coordinator

Distribution of IMI PhD Students jointly supervised with NTU schools

19

Congratulations to Dr Pravin KAKAR, 1st PhD Graduate from IMI

“I am really feeling privileged to be a part of this institute which has really enabled my research and I hope that IMI scales greater high

along with times and all the best for the future.”

Dr Pravin KAKAR

20

PhD Interdisciplinary Seminars

11 April 2013

Presented by

Dr Jingwen DAI

Dr ZHANG Zhijun

Subbarao Nikhil NARAYAN

26 September 2013

Presented by

HOU Junhui

FANG Hui

LIANG Hui

26 November 2013

Presented by

Claudia KUSTER

CHENG Peng

ZHANG Yuzhe

Shakeel AHMAD

29 October 2013

Presented by

DONG Lu

GUO Guibing

TAN Jiat Chow

LI Qiaohong

21

PhD Interdisciplinary Seminars

12 March 2013

Presented by

LAI Danbo

LI Bingbing

Jaroslaw Slawomir

KOCHANOWICZ

25 February 2013

Presented by

EE Changshun Andrew

Yasir TAHIR

CHEN Yong

15 January 2013

Presented by

Dr LIU Gengdai

DONG Lu

Athirai Aravazhi IRRISSAPPANE

GU Yuanlong William

22

Assoc Prof ANG

Pei-Hui, Rebecca,

HSS

Assoc Prof CAI

Jianfei, SCE

Assoc Prof CAI

Yiyu, MAE

Assoc Prof CHAN

Kap Luk, EEE

Assoc Prof CHAU

Lap Pui, EEE

Assoc Prof CHEN

I-Ming, MAE

Asst Prof Mark

Joseph CHAVEZ,

ADM

Assoc Prof CHEN

Shen-Hsing,

Annabel, HSS

Asst Prof CONG

Gao, SCE

Asst Prof Martin

CONSTABLE,

ADM

Asst Prof Justin

DAUWELS, EEE

Assoc Prof HE

Ying, SCE

Asst Prof HU

Guoqiang, EEE

Asst Prof JUNG

Younbo, WKWSCI

Asst Prof Andy

KHONG Wai

Hoong, EEE

Prof KOH Soo

Ngee, EEE

Assoc Prof Vladimir

Vladimirovich

KULISH, MAE

IMI PhD Supervisors

Asst Prof FU

Chi Wing, SCE

Asst Prof CHEW

Lock Yue, SPMS

Asst Prof Kenneth

FEINSTEIN, ADM

23

Prof LUKE Kang

Kwong Kapathy,

HSS

Assoc Prof May

Oo LWIN,

WKWSCI

Assoc Prof Pina

MARZILIANO,

EEE

Assoc Prof MIAO

Chun Yan, SCE

Assoc Prof SEET

Gim Lee, Gerald,

MAE

Assoc Prof Alexei

SOURIN, SCE

Dr OLGA

Sourina, EEE

Assoc Prof TAN

Ah Hwee, SCE

Assoc Prof THENG

Yin Leng, WKWSCI

Assoc Prof WANG

Libo, EEE

Asst Prof YUAN

Junsong, EEE

Asst Prof

ZHANG Jie, SCE

Asst Prof ZHANG

Wei, SCE

Assoc Prof ZHENG

Jianmin, SCE

Assoc Prof SOON

Ing Yann, EEE

IMI PhD Supervisors

Prof Daniel

THALMANN,

IMI/SCE

Assoc Prof James

Patrick WILLIAMS,

HSS

Assoc Prof ZHU

Ce, EEE

Assoc Prof LIN

Feng, SCE

Assoc Prof LIN

Weisi, SCE

24

Current PhD Projects

Vision Based Optimal Path Formation Control and Obstacle Avoidance for Multiple Mobile

Robots

Student: Shakeel AHMAD

Supervisor: Asst Prof HU Guoqiang (EEE)

Co-supervisor: Prof Nadia MAGNENAT THALMANN (IMI)

The optimal path control and obstacle avoidance problem of vision-guided mobile robot with dynamical

and sensing constraints will first be investigated.

Secure Multi-modal Interface with Augmented Reality

Student: DINH Quang Huy

Supervisor: Assoc Prof SEET Gim Lee, Gerald (MAE)


Mentor: Assoc Prof LIN Weisi (SCE)

Memetic Stigmergy: A Multi-method Approach in Studying Decentralized Loci across Social

Networks

Student: ENG Weiwen Herbert

Supervisor: Assoc Prof Patrick Williams (HSS)

Co-supervisor: Asst Prof Chew Lock Yue (SPMS)

Mentor: Asst Prof Kenneth Feinstein (ADM)

Simulation and Automatic Navigation of an Intravascular System

Student: CHEN Yong

Supervisor: Assoc Prof CAI Yiyu (MAE)

Co-supervisor: Prof Daniel THALMANN (SCE)

To make the vascular surgery easier and more efficient, the project aims to design an automatic

navigation system for catheter manipulation. The system consists of catheter control, navigation and

simulation. The coordinates of the catheter tip is obtained by an embedded sensor, and the whole

system works in real time.

25


The Role of Embodiment and Presence on the Development of Leadership Capabilities in

Massively Multiplayer Online Role-playing Games

Student: EE Changshun Andrew

Supervisor: Asst Prof JUNG Younbo (WKWSCI)


Given the advent and prevalent use of virtual environments in New Media, such as in social media (e.g.

Second Life), users – especially the young and adolescent – may acquire and master skills essential for

proficient navigation of these environments. Specifically, I am interested in the role of virtual

embodiment and presence on leadership development among users of these virtual environments.

Study of Nonverbal Communication in Telepresence Robot

Student: GU Yuanlong William

Supervisor: Assoc Prof SEET Gim Lee, Gerald (MAE)


The research topic focuses on the use of nonverbal communication to improve the interactive

experience in human-robot interaction (HRI). The objective is to improve the interpersonal relation

between the system and interactant, by showing emotion, degree of involvement and etc, through the

mode of nonverbal communication.

Three Dimensional Content Processing and Delivery over Communication Network

Student: HOU Junhui

Supervisor: Assoc Prof CHAU Lap-Pui (EEE)


This project aims to improve the quality and speed of real-time 3D processing and transmission. The

results obtained from this project would be useful for areas requiring interactive 3D visual output, e.g.

computer games, and virtual reality.

Developing a Testbed for Evaluating Trust and Reputation Systems in E-commerce

Environment

Student: Athirai Aravazhi IRRISSAPPANE

Supervisor: Asst Prof ZHANG Jie (SCE)


My research focus is to develop a comprehensive testbed to evaluate the robustness of the Trust and

Reputation systems, so that it would be beneficial for the researchers in the field to analyse and

compare their trust models with the purpose of improving their performance.

26

Robotic Enhanced Tele-communication System

Student: LI Bingbing

Supervisor: Assoc Prof CHEN I-Ming (MAE)


This project is aim at developing a robot system which enhance the current telecommunication system

in terms of delivering non-verbal information such as body movement and emotional information.

Quality Assessment in Computer Graphics

Student: LI Qiaohong

Supervisor: Assoc Prof LIN Weisi (SCE)


The research aims to combine human vision system and 3D scene information into the objective quality

assessment metrics enable automatic detection and elimination of rendered images of unacceptable

quality.


Automated Thyroid Nodule Detection using 2D Ultrasound Images

Student: Subbarao Nikhil NARAYAN

Supervisor: Assoc Prof Pina MARZILIANO (EEE)


The main aim of this project is to eliminate the subjectivity involved when measuring the volume of a

thyroid gland/nodule and hence reduce the high inter-observer variation during identifying a thyroid

nodule. The prognosis for thyroid carcinoma is based on the maximum dimension of the nodule under

observation hence eliminating subjectivity will significantly improve the chances for accurate diagnosis.

Monitoring of Group Discussions: a Socio-engineering Approach

Student: Yasir TAHIR

Supervisor: Asst Prof Justin DAUWELS (EEE)


The research is focused on developing a real time automated system that can monitor group discussion

using audio and video signals, which can provide suggestion in real time to improve group interaction.

The ultimate goal is to improve the current computer systems and social robots that suffer from a lack

of social skills, we will explore the social signals and social behaviours, including social interactions,

social attitude, and social relations/ roles to build more socially intelligent model for robots.

27

Acoustic Source Localization in Room Environment

Student: WU Kai

Supervisor: Asst Prof Andy KHONG W. H. (EEE)

The knowledge of the acoustic source location is of much importance in several applications including

hands-free communication systems, teleconferencing, automatic camera steering, etc. We are currently

working on speech source localization in this reverberant and noisy environment.

Neuroimaging as a Surrogate Marker to Assess Cognitive Substrates in Cognitive Training

Software

Student: TAN Jiat Chow

Supervisor: Assoc Prof THENG Yin Leng (WKWSCI)

Co-supervisor: Assoc Prof Annabel CHEN (HSS)

Brain training software has experienced rapid growth in recent years. These brain training software

make claims of improving cognitive or brain functions and the goal of the proposed work is to evaluate

these claims using neuroscience methods e.g. near infra-red optical imaging (NIRS),

electroencephalography (EEG).


Learning Based Scene Understanding

Student: WANG Anran

Supervisor: Assoc Prof CAI Jianfei (SCE)

Co-supervisor: Asst Prof WU Jianxin (SCE)

The objective of this research is to investigate the existing interactive image segmentation algorithms

and then design a better solution. We aim at designing and developing more intuitive and intelligent

image segmentation algorithms using learning technology. We are particularly interested in how to

combine the interactive image segmentation and learning technology to fast segment a large set of

images.

Distributed and Constrained Conformal Mapping

Student: ZHANG Minqi

Supervisor: Assoc Prof HE Ying (SCE)

The research is focused on creating efficient conformal mapping algorithm by proposing a distributed

and constrained conformal mapping. The distributed feature makes it favorable for the GPU and

become very efficient.

28

Markerless Motion Capture of Human Based on Monocular Camera

Student: BIAN Zhenpeng

Supervisor: Assoc Prof CHAU Lap Pui (EEE)


Motion capture of human is a highly active research area due both to the number of potential

applications and its inherent complexity. The research focuses on real-time markerless motion capture

of human, especially based on monocular camera.


Resolving Data Sparsity and Cold Start in Recommender Systems

Student: GUO Guibing


Co-supervisor: Prof Daniel THALMANN (IMI)

The main research problems we desire to address are the two severe issues that original

Collaborative filtering inherently suffers from: data sparsity and cold start. Data sparsity arises from the

phenomenon that users in general rate only a limited number of items; Cold start refers to the difficulty

in bootstrapping the Recommender systems for new users. We aims to solve these issues in both 2D

and 3D environments.

Group Structure and Social Behaviour Simulation Based on Implementation of Psychological

Models of Emotionality and Motivation

Student: Jaroslaw Slawomir KOCHANOWICZ

Supervisor: Assoc Prof TAN Ah Hwee (SCE)

Co-supervisor: Prof Daniel THALMANN (IMI)

This specific research aims to identify the certain significant concepts within psychology and sociology,

to enable implementation of a generic model of autonomous agents with individualised needs, goals,

expression styles, capabilities, group identities, culture, coping strategies, history, world model, private

social map and other characteristics.

Depth Camera Based Hand Gesture Recognition

Student: LIANG Hui

Supervisor: Asst Prof YUAN Junsong (EEE)

Co-Supervisor: Prof Daniel THALMANN (IMI)

The goal of this research is to design and implement algorithms for the recognition of postures and

gestures of a user immersed in a Virtual Environment with Virtual Humans. This research will develop

novel real-time recognition methods with trade-off between accuracy and speed in order to solve the

problem efficiently.

29

Design of a Reputation Mechanism for 3D E-commerce

Student: FANG Hui



The research attempts to design a feedback-based reputation mechanism to solve the inherited trust

problems in 3D e-commerce which is getting more feasible and popular. With 3D technology, the

constructing processes such as feedback provision, feedback computation, feedback presentation and

user decision making will be changed.

Modeling and Animating 3D Chinese Water Color Paintings

Student: CHENG Peng

Supervisor: Assoc Prof MIAO Chun Yan (SCE)


The research is focused on simulating the behavior of Chinese ink and analysing various artistic effects

of colour ink diffusion. The study also looks how a scheme presents can be simulated automatically for

their computer generated simulation.


Modeling Virtual Human with Episodic Memory

Student: ZHANG Juzheng

Supervisor: Assoc Prof ZHENG Jianmin (SCE)


The research goal of this project is to refine the existing methods; episode representation, indexing

scheme, matching methods and storage policy , if any, or develop new ones to find a more effective

way to encode, store and retrieve episodic memory.

On Developing High-quality Immersive Rendering for Real-time 3D Teleconference

Student: ZHAO Mengyao

Supervisor: Asst Prof FU Chi-Wing (SCE)

Co-supervisor: Assoc Prof CAI Jianfei (SCE)

This project aims to investigate time-critical methods to analyze and model the input image-based

information on real-time 3D object reconstruction and rendering from real scenes.

30

Automatic 3D Clothing: from 2D Patterns to Garment

Student: ZHANG Yuzhe



This project aims to develop algorithms and solutions for automatic modeling and prepositioning of

garment from 2D pattern. And it mainly focuses on the process of converting 2D pattern into 3D

garment and its application in virtual try-on.

3D Simulation of Articulation in Physiological Human

Student: ZHANG Wenjing



The research is to model the articulation of human to help with the diagnosis and treatment of certain

disease. The research is mainly divided into two parts to build an accurate geometry model of human

articulation from MRI.

Interactive Image Segmentation

Student: ZHU Hongyuan

Supervisor: Assoc Prof CAI Jianfei (SCE)


Image segmentation plays an important role in many fields of Computer Vision and Computer Graphics.

Interactive image segmentation is an ad-hoc topic in recent years. The main objective of this research

is to improve the segmentation with higher-level semantics provided by user input.


Research on Multimedia Community-based Question Answering

Student: YUAN Quan

Supervisor: Asst Prof CONG Gao (SCE)


This research focuses on Multimedia Community-based Question Answering (MCQA), a new type of

Community-based question-answering (CQA) that enables users to post questions with photos and

video. This research involves with question organization, multimedia question retrieval, expert user

discovery and recommendation, and QA in 3D virtual space.

31

Cyber-learning in Cyberworlds

Student: LAI Danbo

Supervisor: Assoc Prof Alexei SOURIN (SCE)

Co-supervisor: Assoc Prof ZHAO Dongsheng (NIE)

This project proposes to immerse the learners in 3D AI-assisted shared virtual collaborative

environments where mathematical formulas are used for immediate interactive definition and multi-

sensory rendering of geometry, appearance and physical properties of the shapes.

Adolescents' Information Disclosure on Facebook: Impact of Their Parents and Personality

Student: LIU Cong

Supervisor: Assoc Prof ANG Pei-Hui, Rebecca (HSS)

Co-supervisor: Assoc Prof May Oo LWIN (WKWSCI)

Personality traits (i.e., narcissism and social anxiety) and parental mediation on adolescents’

information disclosure on Facebook was studied. Parental mediation influences risky online behaviors

of their children in both direct and indirect ways. Implications on promoting adolescents’ online safety

will be discussed.


Perceptually Based Selective Rendering

Student: DONG Lu

Supervisor: Assoc Prof LIN Weisi (SCE)

Co-supervisor: Assoc Prof ZHU Ce (EEE)

In this project, a perceptual model is to be exploited for image rendering (especially in illumination and

shadow determination). Visual attention, contrast sensitivity and masking will be investigated.

Experiments will be conducted to validate the effectiveness of the proposed method.

CBAN: An Embedded Patient Monitoring Body Area Network

Student: Saurabh RASTOGI

Supervisor: Assoc Prof LIN Feng (SCE)

Co-supervisor: Asst Prof Mark CHAVEZ (ADM)

After examining various approaches for this purpose we will describe the design methodology for the

proposed Body Area Network (BAN) and explore the configuration space looking for the most effective

solution, performance, cost and energy-wise. This BAN solution provides higher levels of user

satisfaction as expected from novel healthcare delivery scenarios (e.g., mobility and customization)

while the architecture in its heart achieves these targets by complex hardware methods (such as

multitasking, parallel processing) and performs beyond the expectations of accurate real time

diagnosis, analysis and monitoring of diseases.

32

Development of Aesthetic Algorithms for Acquired and Generated Digital Image

Student: ZHANG Xiaoyan

Supervisor: Assoc Prof Kap Luk CHAN (EEE)

Co-supervisor: Asst Prof Martin CONSTABLE (ADM)

This project explores the structural models behind visual appeal and the development of algorithms to

apply them to acquired and/or generated digital images. Operations such as post-exposure refocusing,

re-rendering of the scene from an alternate viewpoint, single-image based on matting and composition

and automatic scene segmentation will be explored.

Computer-aided Evaluation of Prosody Evaluation for Language Learning

Student: ZHAO Sixuan

Supervisor: Prof KOH Soo Ngee (EEE)

Co-supervisor: Prof K. K. LUKE (HSS)

In our project, a reference-independent prosody evaluation method using Support Vector Machine

(SVM) is proposed to evaluate the learner’s prosody without reference utterances.


Statistical Machine Learning Techniques with Applications to Image Retrieval

Student: ZHANG Lining

Supervisor: Assoc Prof WANG Libo (EEE)

Co-supervisor: Assoc Prof LIN Weisi (SCE)

In this work, we study several statistical machine learning techniques, e.g., supervised learning, semi-

supervised learning, subspace learning and active learning in a synergistic way to maximize the

effectiveness of a learning task, and thus enhance the performance of the image retrieval systems.

Quantification of Brain Responses for Neurofeedback Game in 3D Virtual Environments

Student: WANG Qiang

Supervisor: Asst Prof OLGA Sourina (EEE)

Co-supervisor: Assoc Prof Vladimir Vladimirovich KULISH (MAE)

The main objective of our project is exploiting new algorithms and new features for the brain state

recognition for real-time applications with as fewer electrodes as possible.

33

34

Real-time Interaction with Nadine Humanoid Robot at Swissnex Singapore

End of 2013 Party │ 5 December 2013

The Ambassador of Switzerland in Singapore, Thomas

Kupfer (2nd from the left)

Presentation of Nadine Humanoid Robot online

Nadine sings a song of Adele Dr. Suzanne Hraba-Renevey, Executive Director of

Swissnex Singapore (2nd from the right)

35

19th ACM Symposium on Virtual Reality Software and Technology

(VRST 2013) │ 6-8 October 2013

The ACM Symposium on Virtual Reality Software and Technology (VRST) is an international forum for the exchange of experience and

knowledge among researchers and developers concerned with virtual reality software and technology. This year, VRST 2013 was

organized by the Institute for Media Innovation (IMI) from 6-8 October 2013 on the Nanyang Technological University (NTU) campus.

VRST 2013 provided an opportunity for VR researchers to interact, share new results, show live demonstrations of their work, and

discuss emerging directions for the field. The event was sponsored by ACM SIGCHI and SIGGRAPH.

Papers have been selected among 89 submissions by an International Program Committee following a strict peer review. 20 full papers

and 18 short papers have been selected for the Conference.

The technical program was composed of 8 sessions: medical VR and Virtual Rehabilitation, Human Factors, 3D Interaction, Navigation,

Avatars and Robots in TelePresence, Simulation, Rendering, and VR Display Technologies.

36

Institute for Media Innovation (IMI)

organized Public Lecture by Prof Petteri

Nisunen and Tommi Grönlund at the NTU

Reality Theatre, Research Techno Plaza on

1 October 2013

The NTU President Professor Bertil Andersson (1st row, 2nd from the right at the public

lecture)

Public Lecture by Prof Petteri Nisunen and Tommi Grönlund │ 1 October 2013

37

Perception and Behaviour of Virtual Humans Workshop │ 10 May 2013

Distinguished Lecture: The e-Documentation of the Past: Risks and Challenges │ 26 September 2013

Distinguished Lecturer: Dr. Marinos Ioannides, Cyprus University of Technology, Digital Heritage Lab, Cyprus

Assoc Prof Miriam Reiner, Technion Touch Lab,

Technion University, Israel

38

In the year 2013, IMI accommodated 34 visits (441 visitors from more than 13 countries).

Ms Anne-Catherine Lyon, State Councillor and Head

of Department of Education and Youth Development,

Canton of Vaud, Switzerland

Delegates from global offices of Temasek Holdings

Dr Lykke Friis and Dr Esben Lunde Larsen, members of

the Danish Parliament, accompanied by Prof Er Meng

Hwa, NTU Vice President (International Affairs)

Visitors to IMI

Mr Lim Chin Siang, Director of IDMPO from MDA

39

Journalists from Nature, The Economist and Le Monde

Delegates from Beijing Institute of Fashion Technology,

China

Final Year Medical students from Imperial College

London

The finalists for Nanyang Assistant Professorship Scheme

Visitors to IMI

40

Prof Franz-Erich WOLTER

Welfenlab

Leibniz University of Hanover

Germany

Assoc Prof Miriam REINER

Technion Touch Lab

Technion University

Israel

Assoc Prof ZHANG Hong

Wuhan University of Science and Technology

China

Prof Hector OROZCO

Autonomous University of Mexico

State

Mexico

Asst Prof Arjan EGGES

Utrecht University

The Netherlands

Sybren A. Stuvel

PhD Student

Utrecht University

The Netherlands

Visiting Professors, Researchers and PhD Students

41

42

PEC Ltd is a specialist engineering group servicing the oil and gas, petrochemical, oil and chemical terminal, and pharmaceutical

industries. Their two core business activities are project works and maintenance services in the region.

IMI and PEC Ltd tied up to develop simulation technology for engineering applications.

NTU-PEC Research Collaboration Project Kickoff Meeting │ 31 January 2013

43

Signing of NTU-NVIDIA Collaboration MOU │ 28 January 2013

The MOU signing ceremony between the Institute for Media Innovation, NTU and NVIDIA was held on 28 January 2013.

Desirous of promoting and developing cooperation in the field of GPU technologies, the MOU aims to foster closer ties between the two

parties and to promote cooperative activities including exchange of information, organization of joint workshops, seminars and

conferences.

44

Industrial Collaboration

Collaboration between IMI and Asian Women’s Welfare Association (AWWA)

Special School

“The collaboration with IMI has benefited both my ASD and Multiple Disabilities students.

It is amazing to see them interacting with the 3D dolphins through the use of hand

signals. Best of all, they were able to transfer that learning, out of the Immersive Room

into their classroom.”

Mrs Ruby SEAH

Principal, AWWA School

Collaboration between IMI and Lee Kong Chian School of Medicine

To explore the use of a highly immersive augmented reality environment (HIARE) to

provide a technological frame-work for uniformity of Phase 3 medical student safe and

controlled exposure to acute medical emergencies.

Virtual Pink Dolphin-assisted Therapy for Children with Autism

Funding Supported by Temasek Trust-funded Singapore Millennium Foundation

Research on autism and development of therapy for autism are of worldwide interest

today. Several studies show dolphin-assisted-therapy (DAT) has positive effect to children-

with-autism in terms of anxiety/stress relief, communication/learning improvement, and

cognitive-performance/self-confidence increase.

45

46

About BeingThere Centre

BeingThere Centre is an International Research Centre for Telepresence and

Telecollaboration mainly based at the Institute for Media Innovation (IMI) in Nanyang

Technological University (NTU), Singapore.

It is funded by Media Development Authority of Singapore with joint effort from ETH

Zurich (Switzerland), UNC Chapel Hill (USA) and NTU (Singapore). The objective of

the BeingThere Centre is to make major technological and systems-level advances

leading to a 3D credible experience of Telepresence in a fitted room using a mobile

platform. The result will be a breakthrough in the quality of interpersonal

communication at a distance allowing for eye contact and proper motion parallax

among a group of users.

BeingThere Centre

(IMI)

Est 15 Dec 2010

NTU

UNCETH

47

“BeingThere Centre based at the Institute for Media Innovation in Nanyang Technological University,

Singapore is a centre of breakthrough in Virtual Reality.”

Chair, BTC Steering Committee

NTU Deputy President and Provost

Professor BOEY Yin Chiang, Freddy

48

BeingThere Centre Steering Committee

Prof Freddy BOEY

Chair of Steering Committee

Deputy President and Provost

NTU, Singapore

Prof Anselmo LASTRA

Chair, Department of Computer Science

UNC Chapel Hill, USA

Mr Chun Cheng YEO

Assistant Chief Executive (Industry)

Executive Director, IDMPO

Media Development Authority,

Singapore

Dr. Silke MEYNS

Head of Team Patents and Licensing

ETH Zurich, Switzerland

49

BeingThere Centre International Advisory Board

Prof Susanto RAHARDJA

Chair of International

Advisory Board

Institute for Infocomm

Research, AStar, Singapore

Dr Mod MARATHE

Cisco Research and

University Relations,

USA

Dr.-Ing Aljoscha SMOLIC

Disney Research, Zurich

Prof Craig GOTSMAN

Technion, Israel

Prof Mel SLATER

University of Barcelona,

Spain

BeingThere Centre Co-Directors

Prof Nadia MAGNENAT THALMANN

Co-director

NTU, Singapore

Prof Markus GROSS

Co-director

ETH Zurich, Switzerland

Prof Henry FUCHS

Co-director

UNC Chapel Hill, USA

50

Midterm Review Panel Members

“The review panel believes the Centre is a great return on investment for MDA … The BeingThere Centre has encouraged

multidisciplinary collaboration at NTU. It brings faculty members and students from Computer Science, Electronic Engineering,

Mechanical Engineering, Social Science, and others, to work together within the same framework. The Centre has made worldwide

Impact of Talent. 6 former research fellows were offered faculty/top company positions in US, Canada, France and China. Clearly,

the Centre has substantially increased the international visibility of NTU. We congratulate the co-directors and members of the

BeingThere Centre for their impressive work.“

- From BeingThere Centre Midterm Review Panel

Dr ZHANG Zhengyou

Chair of Midterm Review Panel

Microsoft Research, Redmond, USA

Prof Michitaka HIROSE

The University of Tokyo, JAPAN

Prof Werner PURGATHOFER

Vienna University of Technology, AUSTRIA

BeingThere Centre Midterm Review │ 11 - 12 November 2013

51

KPI Fulfilment of BeingThere Centre as of 30 September 2013

BeingThere Centre Midterm Review │ 11-12 November 2013

BTC Midterm Review Meeting Chair of Midterm Review Panel trying out

the BTC Project

Midterm Review Panel Meeting with

Steering Committee Members

52

BeingThere Centre Associated Professors

Assoc Prof CAI

Jianfei, SCE

Assoc Prof CAI

Yiyu, MAE

Assoc Prof CHAU

Lap Pui, EEE

Asst Prof FU

Chi Wing, SCE

Asst Prof YUAN

Junsong, EEE

Assoc Prof ZHENG

Jianmin, SCE

Assoc Prof CHAM

Tat Jen, SCE

Assoc Prof CHEN

I-Ming, MAE

Asst Prof Andy

W. H. KHONG,

EEE

Assoc Prof SEET

Gim Lee, Gerald,

MAE

Prof Daniel

THALMANN,

IMI/SCE

Prof SUN

Chengzheng,

SCE

Assoc Prof HE

Ying, SCE

Asst Prof Justin

DAUWELS, EEE

Asst Prof Michael

Donald PATTERSON,

HSS

53

BeingThere Centre Engaged Researchers and PhD Students

NTU Researchers

Dr Zerrin YUMAK

Dr Aryel BECK

Dr Xiao YANG

Dr ZHANG Zhijun

Dr WONG Choon Yue

Dr Jun LEE

Mr BURHAN

Mr DENG Teng

Mr ISKANDARSYAH

Ms PANG Wee Ching

Mr REN Jianfeng

Mr TAN Fuwen

Mr XU Di

Mr Guo Yu

NTU PhD Students

Ms ZHAO Mengyao

Ms WANG Anran

Mr LI Bingbing

Mr GU Yuanlong, William

Mr HOU Junhui

Mr LIANG Hui

Mr WU Kai

Mr ZHANG Minqi

Mr ZHANG Juzheng

UNC Researchers

Prof Miriam REINER

Mr Andrei STATE

Dr Renjie CHEN

Dr Jingwen DAI

UNC PhD Students

Mr Andrew MAIMONE

Mr Peter LINCOLN

Mr Ryan SCHUBERT

Mr DOU Mingsong

ETH Researchers

Dr Luca BALLAN

Dr Jean-Charles BAZIN

Dr Tobias MARTIN

ETH PhD Students

Ms Claudia CLUSTER

Mr Nicola RANIERI

54

PIs: Assoc Prof CHAM Tat Jen (NTU)

Prof Henry FUCHS (UNC)

Team Members: Assoc Prof CAI Jianfei (NTU)

Assoc Prof Philip FU (NTU)

Mr Andrei STATE (UNC)

Mr TAN Fuwen (NTU)

Mr DENG Teng (NTU)

Mr XU Di (NTU)

Dr Renjie CHEN (UNC)

Prof Miriam REINER (UNC)

This project is aimed at exploring and developing technical and research methodologies for scene

acquisition and display to enable the room-based Telepresence system. The grand vision for the

Telepresence system is to simulate a highly realistic experience for users whereby distant remote

rooms appear to be separated only by transparent glass walls. This entails that as a user walks

about within the local room, her view of the remote room not only appears 3D, but is also

geometrically consistent with her positional point of view, and rendered with sufficient perceptual

fidelity to make the user believe she is physically there – right next to the remote room.

Novel Scene Acquisition and Display Technologies for a Next-generation Room-based Telepresence System

BeingThere Centre Project 1

• A. Maimone, X. Yang, N. Dierk, A. State, M. Dou and H. Fuchs, General-purpose Telepresence with Head-worn Optical See-

through Displays and Projector-based Lighting, IEEE Virtual Reality 2013, Orlando, USA, March 16 - 23, 2013

• M. Zhao, F. W. Tan, C. W. Fu, C. K. Tang, J. F. Cai and T. J. Cham, High-quality Kinect Depth Filtering for Real-time 3D

Telepresence, International Conference on Medical Education (ICME 2013), San Jose, California, USA, July 15 - 19, 2013

55


PIs: Prof Markus GROSS (ETH)

Prof Marc POLLEFEYS (ETH)

Team Members: Dr Luca BALLAN (ETH)

Dr Tobias MARTIN (ETH)

Dr Jean-Charles BAZIN (ETH)

The project aims to design and build and entirely novel, mobile display platform enabling

autostereoscopic display of collaborators at remote locations. The core novelty is a humanized,

transparent OLED display combined with an LCD layer as an adaptive parallax barrier. A set of

cameras and time-of-flight depth sensors captures the three dimensional appearance, head

position, and distance of the remote communication partner in real-time. This information will be

utilized to control the parallax barrier and to steer the holonomic motion platform.

Mobile, Autostereoscopic 3D Display and Telepresence Platform

• C. Kuster, N. Ranieri, Agustina, H. Zimmer, J. C. Bazin, T. Popa and M. Gross, Towards Next Generation 3D Teleconferencing

Systems, 3DTV Conference 2012 Special Session on “Telepresence”, Zurich, October 15 - 17, 2012

• J. C. Bazin and M. Pollefeys, 3-line RANSAC for Orthogonal Vanishing Point Detection, IEEE/RSJ International Conference on

Intelligent Robots and Systems 2012, Algarve, Portugal , October 7 - 12, 2012

56

PIs: Assoc Prof CHEN I-Ming (NTU)

Assoc Prof SEET Gim Lee, Gerald (NTU)

Research Prof Greg WELCH (UNC)

Team Members: Dr WONG Choon Yue (NTU)

Mr BURHAN (NTU)

Ms PANG Wee Ching (NTU)

Dr Jingwen DAI (UNC)

The objective of the project is to improve on the state of art technology of deploying Telepresence

robots for telecommunication. The plan is to address the need for Telepresence robot technology

which provides rich interactivity without requiring the robot’s user to devote significant cognitive

activity to the operation of the robot.

Specifically, the intention is to develop a mobile Telepresence robot capable of navigating

autonomously or semi-autonomously (directed but not controlled) in a typical office environment

while allowing safe and relaxed interaction without the need to be consciously concerned about

preventing collisions, and maintaining a comfortable distance and pose with respect to the other

humans involved in the interactions.

Animatronic Avatar: Development of a Robotic Avatar for Telepresence Interaction and Communication


• G. Seet, P. W. Ching, Burhan, V. Lastrebov, W. Gu and W. C. Yue, A Design For A Mobile Robotic Avatar – Modular Framework,

3DTV Conference 2012 Special Session on “Telepresence”, Zurich, October 15 - 17, 2012

• W. C. Pang, G. Seet and X. Yao, A Multimodal Person-following System for Telepresence Applications, ACM Sym. on Virtual

Reality Software and Technology (VRST2013), Singapore, October 6 – 8, 2013

57


PIs: Prof Nadia MAGNENAT THALMANN (NTU)

Asst Prof YUAN Junsong (NTU)

Prof Daniel THALMANN (NTU)

Team Members: Assoc Prof HE Ying (NTU)

Asst Prof Andy W. H. KHONG (NTU)

Assoc Prof Lap-Pui CHAU (NTU)

Assoc Prof Jianmin ZHENG (NTU)

Assoc Prof CAI Yiyu (MAE)

Dr Aryel BECK (NTU)

Dr Zerrin YUMAK (NTU)

Dr ZHANG Zhijun (NTU)

Dr Xiao YANG (NTU)

Dr Jun LEE (NTU)

Mr Jianfeng REN (NTU)

The general objective of this project is to partially replace a real participant by its virtual

counterpart. The virtual participant is supposed to give a partial illusion that the real person is

present. This implies that the Virtual Human (VH) should look the same as the real human, speaks

with the same intonation, and be somehow aware of the real situation, the real participants, and the

task currently performed. The VH should react at the right time based on the perception he/she has

from the real participants. It implies to evaluate what each real participant is doing. Perception will

be obtained by visual and audio input and recognition. The virtual participant reacts according to

the input and its current knowledge. Its reactions encompass animation (body and facial gestures)

and speech synthesis.

Autonomous Virtual Humans and Social Robots for Telepresence

• J. Hou, L.P. Chau, Y. He, M. Zhang and N. Magnenat Thalmann, Rate-distortion Model Based Bit Allocation for 3-D Facial

Compression Using Geometry Video, IEEE Transactions on Circuits and Systems for Video Technology, No. 99, February 25, 2013

• Z. Yumak, N. Magnenat Thalmann, Multi-party Interaction with a Virtual Character and Human-like Robot, The 19th ACM

Symposium on Virtual Reality Software and Technology (VRST2013), Singapore, October 6-8, 2013

58

PIs: Assoc Prof CHAM Tat Jen (NTU)

Prof Henry FUCHS (UNC)

Prof Markus GROSS (ETH)

Team Members: Prof Miriam REINER(UNC)

Dr Jean-Charles BAZIN (ETH)

Dr Tobias MARTIN (ETH)

Mr ISKANDARSYAH (NTU)

Mr GUO Yu (NTU)

In this research project, broad experimental investigations are conducted into advanced hardware

and software technologies to support future systems for immersive 3D Telepresence with multiple

users per site, unburdened by any head gear, supporting full bidirectional eye contact. 3D

capture/acquisition, 3D display, as well as the related domain of 3D data encoding and

representation in the context of Telepresence are investigated.

This project particularly focus on the feasibility of approaches for the ambitious and challenging

tasks. With a focus on advanced research, rather than complete prototypes, the next generation of

3D acquisition and display, far beyond the capabilities of existing systems is believed to be

reached.

Advanced Technologies for 3D Capture, Communication, and Display


• J. C. Bazin, H. Li, I. S. Kweon, C. Demonceaux, P. Vasseur and K. Ikeuchi, A Branch-and-Bound Approach to Correspondence

and Grouping Problems, The IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), pp. 1565 – 1576, July, 2013

• C. Kuster, T. Popa, J. C. Bazin, C. Gotsman and M. Gross, Gaze Correction for Home Video Conferencing, SIGGRAPH Asia 2012,

Singapore, November 29 – December 1, 2012

59

BeingThere Centre Achievements Highlights in 2013

CGI Achievement Award, Germany, 2013

Professor Nadia Magnenat Thalmann received CGI Achievement Award in Germany in recognition of her

outstanding accomplishment, academic excellence and service to the Computer Graphics Community in CGI 2013

Conference held in Germany, 2013.

2013 VR Career Award, USA, 2013

Prof. Henry Fuchs received the "2013 VR Career Award in recognition of his lifetime contributions to research and

practice in virtual environments, telepresence and medical applications." in IEEE Virtual Reality 2013 Conference

held in Orlando, FL, USA, March 16-23, 2013.

IEEE Computer Society Award in IEEE VR 2013 Conference, USA, 2013

Prof Greg Welch received IEEE Computer Society award for “Outstanding Performance as Co-Chairman of the

IEEE VR 2013 Conference,” in IEEE Virtual Reality 2013 Conference held in Orlando, FL, USA, March 16-23,

2013.

Best Short Paper Award in IEEE VR 2013 Conference

The paper titled "General-Purpose Telepresence with Head-Worn Optical See-Through Displays and

Projector-Based Lighting“ by Andrew Maimone, Xubo Yang, Nate Dierk, Andrei State, Mingsong Dou and

Henry Fuchs was given the "Best Short Paper Award" in IEEE Virtual Reality 2013 Conference held in

Orlando, FL, USA, March 16-23, 2013.

Business Development

A spin-off company was registered from the research work by the BeingThere Centre’s researcher Rajan

Sobhana Rashobh and professor Andy Khong from NTU.

Best Paper Award (second place) in SIAM Conference on Geometric and Physical Modeling (GD/SPM’13)

The paper titled "A Simple and Local Method for Computing Quasi-conformal Map on 3D Surfaces“ by Zhang

Minqi and He Ying was given the "Best Paper Award (second place)" in Society for Industrial and Applied

Mathematics (SIAM) conference on Geometric & Physical Modeling (GD/SPM 13) held in Denver, Colorado,

USA, November 11 - 14, 2013.

60

BeingThere Centre Co-organized Workshop on Virtual Humans and Cognitive

Modeling │ 26 April 2013

Prof Matthias Scheutz, Tufts University, USAProf Andrew Ortony, Northwestern University, USA

Director, Programme in Computational Social

Cognition IHPC/A*STAR, Singapore

61

62

Selected Keynotes & Invited Talks

N. Magnenat Thalmann, Cloth Simulation, Virtual Try On and Fashion Show, Xiamen University, China, October 14, 2013

N. Magnenat Thalmann, Interacting with Virtual Characters and Social Robots, Virtual Humans and Cognitive Modeling Workshop,

Institute for Media Innovation, Singapore, April 26, 2013

N. Magnenat Thalmann, Interacting with Social Robots and Virtual Humans, Workshop on 3D Displays and Related Technologies,

Nanyang Executive Centre Auditorium, NTU, Singapore, February 4, 2013

Books & Book Chapters

S. Sarda , M. Constable, J. Dauwels, S. Dauwels, M. Elgendi, M. Zhou, U. Rasheed, Y. Tahir, D. Thalmann and N. Magnenat Thalmann,

Real-time Feedback System for Monitoring and Facilitating Discussions, Natural Interaction with Robots, Knowbots and

Smartphones - Putting Spoken Dialog Systems into Practice, Lecture Notes in Computer Science 2013, Springer, 2013

Journal Papers

H. Zhu, J. Zheng, J. Cai, and N. Magnenat Thalmann, Object-level Image Segmentation Using Low Level Cues, IEEE Transactions

on Image Processing (TIP), October 2013

S. Zhao, S. N. Koh, and K.K. Luke, Reference-independent Prosody Evaluation Based on Prosodic Unit Segmentation, IEICE

Transactions on Information and Systems, Vol. E96-D, No. 9, September 2013

C. Liu, R.P. Ang and M.O. Lwin, Cognitive, Personality, and Social Factors Associated with Adolescents’ Online Personal

Information Disclosure, Journal of Adolescence, Elsevier, No. 1467, August 2013

L. Zhang, L. Wang, and W. Lin, Geometric Optimum Experimental Design for Collaborative Image Retrieval, IEEE Transactions on

Circuits and System for Video Technology, August 2013

J.C. Bazin, H. Li, I.S. Kweon, C. Demonceaux, P. Vasseur and K. Ikeuchi, A Branch-and-Bound Approach to Correspondence and

Grouping Problems, The IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), pp. 1565 – 1576, July 2013

63

Journal Papers

X.Q. Wu, J.M. Zheng, C.L. Wu and Y.Y Cai, Variational Structure–texture Image Decomposition on Manifolds, Signal Processing,

Elsevier, Vol. 93, pp. 1773–1784, July 2013

H. Liang, J. Yuan, D. Thalmann and Z. Zhang, Model-based Hand Pose Estimation via Spatial-temporal Hand Parsing and 3D

Fingertip Localization, The Visual Computer, Vol.29, No. 6-8, pp 837-848, June 2013

J. Ren, X. Jiang and J. Yuan, Noise-Resistant Local Binary Pattern with an Embedded Error-correction Mechanism, IEEE

Transactions on Image Processing, vol. 22, issue 10, pp. 4049-4060, June 2013

H. Zhou, J. Zheng and L. Wei, Texture Aware Image Segmentation using Graph Cuts and Active Contours, Journal of Pattern

Recognition, Vol. 46, Issue 6, pp. 1719-1733, June 2013

P. Chiang, Y.Y. Cai, K.H. Mak and J.M. Zheng, A B-spline Approach to Phase Unwrapping in Tagged Cardiac MRI for Motion

Tracking, Magnetic Resonance in Medicine, Vol. 69, pp. 1297–1309, May 2013

G.D. Liu, S. Choudhary, J.Z. Zhang and N. Magnenat Thalmann, Let's Keep in Touch Online: A Facebook Aware Virtual Human

Interface, The Visual Computer, May 2013

Y.B. Wang, R. Dubey, N. Magnenat Thalmann and D. Thalmann, An Immersive Multi-agent System for Interactive Applications, The

Visual Computer, Volume 29, Issue 5, pp. 323-332, May 2013

S. R. Carvalho, R. Boulic, C.A. Vidal and D. Thalmann, Latent Motion Spaces for Full-body Motion Editing, The Visual Computer,

Vol.29, No.3, pp.171-188, March 2013

G. Guo and M. Elgendi, A New Recommender System for 3D E-commerce: An EEG Based Approach, 23rd International Joint

Conference on Artificial Intelligence (IJCAI2913), Journal of Advanced Management Science, Vol. 1, No. 1, pp. 61-65, March 2013

Q, Wang and O. Sourina, Real-time Mental Arithmetic Task Recognition From EEG Signals, IEEE Transactions on Neural Systems

and Rehabilitation Engineering, Vol. 21, No. 2, pp. 225-232, March 2013

J. Hou, L.P. Chau, Y. He, M. Zhang and N. Magnenat Thalmann, Rate-distortion Model Based Bit Allocation for 3-D Facial

Compression Using Geometry Video, IEEE Transactions on Circuits and Systems for Video Technology, No. 99, February 2013

S.Q. Xin, X. Wang, J. Xia, W. Mueller-Wittig, G.J. Wang and Y. He, Parallel Computing 2D Voronoi Diagrams Using Untransformed

Sweepcircles, Computer-Aided Design Journal, Vol. 45, Issue 2, pp. 483-493, February 2013

Y. Cai, N.K.H. Chia, D. Thalmann, N.K.N. Kee, D. Lu, J. Zheng and N. Magnenat Thalmann, Design and Development of A Virtual

Dolphinarium To Increase Functional Developmental Learning in Children with Autism Spectrum Disorder, IEEE Transactions on

Neural Systems & Rehabilitation Engineering, Vol.21, No. 2, pp. 208-217, 2013

64

A. Wang, H. Zhu, J. Cai and J. Wu, Multi-class Cosegmentation with Pairwise Active Learning, 2013 Pacific-Rim Conference on

MultiMedia (PCM 2013), Nanjing, China, December 13-16, 2013

T. Deng, H. Li, J. Cai, T.J. Cham and H. Fuchs, Kinect Shadow Detection and Classification, ICCV 2013 Workshop: Big Data in 3D

Computer Vision, Sydney, Australia, December 8, 2013

J. Hou, L.P. Chau, Y. He and N. Magnenat Thalmann, Expression-invariant and Sparse Representation for Mesh-based

Compression for 3D Face Models, IEEE Conference on Visual Communication and Image Processing (VCIP 2013), Sarawak,

Malaysia, November 17-20, 2013

X. Ying, Zhenhua and Y. He, A Parallel Algorithm for Improving the Maximal Property of Poisson Disk Sampling, Proceedings of

the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games, Society for Industrial and Applied Mathematics (SIAM)

conference on Geometric & Physical Modeling (GD/SPM 13), Denver, Colorado, USA, pp. 179-179, November 11-14, 2013

M. Zhang and Y. He, A Simple and Local Method for Computing Quasi-conformal Map on 3D Surfaces, Society for Industrial and

Applied Mathematics (SIAM) conference on Geometric & Physical Modeling (GD/SPM 13), Denver, Colorado, USA, November 11-14,

2013

Y. Tahir, U. Rasheed, K. Hui, S. Dauwels, J. Dauwels, D. Thalmann, N. Magnenat Thalmann, NAO Robot as a Social Mediator: A User

Study, International Conference on Social Robotics (ICSR2013), Bristol, UK, October 27-29, 2013

U. Rasheed, Y. Tahir, S. Dauwels, J. Dauwels, D. Thalmann and N. Magnenat Thalmann, Real-Time Comprehensive Sociometrics for

Two-Person Dialogs, 4th International Workshop on Human Behavior Understanding (HBU2013), Barcelona, Spain, October 22, 2013

G. Guo, J. Zhang, D. Thalmann and N. Yorke-Smith, Prior Ratings: A New Information Source for Recommender Systems in E-

Commerce, ACM Sym. on Virtual Reality Software and Technology (VRST2013), Singapore, October 6–8, 2013

W.C. Pang, G. Seet and X. Yao, A Multimodal Person-following System for Telepresence Applications, ACM Sym. on Virtual

Reality Software and Technology (VRST2013), Singapore, October 6–8, 2013

A. Wang, H. Zhu, J. Cai and J. Wu, Multi-class Cosegmentation with pairwise active learning, ACM Sym. on Virtual Reality Software

and Technology (VRST2013), Singapore, October 6–8, 2013

Y. Xiao, J. Yuan and D. Thalmann, Human-Virtual Human Interaction by Upper Body Gesture Understanding, ACM Sym. on Virtual

Reality Software and Technology (VRST2013), Singapore, October 6–8, 2013

Conference Papers

65

Z. Yumak and N. Magnenat Thalmann, Multi-party Interaction with a Virtual Character and Human-like Robot, The 19th ACM

Symposium on Virtual Reality Software and Technology (VRST2013), Singapore, October 6-8, 2013

Z. Yuzhe, Z. Jianmin and N. Magnenat Thalmann, Virtual Try-on with Kinect, ACM Sym. on Virtual Reality Software and Technology

(VRST2013), Singapore, October 6–8, 2013

J. Zhang, J. Zheng, N. Magnenat Thalmann, Design Mood Dynamics for Virtual Human, ACM Sym. on Virtual Reality Software and

Technology (VRST2013), Singapore, October 6–8, 2013

L. Liu, B. Li, C. Sun, I. Chen, M. Sung and T. Goh, Motivated Learning Embodied Implementation in Autonomous Systems - A Case

Study in NAO Robot, International Symposium on Robotics and Mechatronics (ISRM 2013), Singapore, October 2-4, 2013

M.S. Dou, H. Fuchs and J.M. Frahm, Scanning and Tracking Dynamic Objects with Commodity Depth Cameras, International

Symposium on Mixed and Augmented Reality (ISMAR 2013), Adelaide, Australia, October 1-4, 2013

A. Maimone and H. Fuchs, Computational Augmented Reality Eyeglasses, International Symposium on Mixed and Augmented Reality

(ISMAR 2013), Adelaide, Australia, October 1-4, 2013

C. Chen, J. Cai, J. Zheng, T.J. Cham and G. Shi, A Color-guided, Region-adaptive and Depth-selective Unified Framework for

Kinect Depth Recovery, IEEE Multimedia Signal Processing (MMSP) Workshop 2013, Italy, September 30 - October 2, 2013

J. Hou, L.P. Chau, Y. He and N. Magnenat Thalmann, Human Motion Capture Data Recovery via Trajectory-based Sparse

Representation, Proceedings of IEEE International Conference on Image Processing (ICIP '13), Melbourne, Australia, September 15-18,

2013

J. Ren, X.D. Jiang and J. Yuan, Learning Binarized Pixel-difference Pattern for Scene Recognition, IEEE International Conference

on Image Processing (ICIP), Melbourne Australia, September 15-18, 2013

J. Ren, X. D. Jiang and J. Yuan, Relaxed Local Ternary Pattern for Face Recognition, IEEE International Conference on Image

Processing (ICIP), Melbourne Australia, September 15-18, 2013

Q. Yuan, G. Cong, Z. Ma, A. Sun and N. Magnenat Thalmann, Who, Where, When and What: Discover Spatio-temporal Topics for

Twitter Users, Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2013),

pp 605-613, Chicago, United States, August 11-14, 2013

Conference Papers

66

H. Fang, Trust Modeling for Opinion Evaluation by Coping with Subjectivity and Dishonesty, in Proceedings of the 23rd

International Joint Conference on Artificial Intelligence (IJCAI 2013), Beijing, China, August 3-9, 2013

H. Fang, Y. Bao and J. Zhang, Misleading Opinions Provided by Advisors: Dishonesty or Subjectivity, in Proceedings of the 23rd

International Joint Conference on Artificial Intelligence (IJCAI 2013), Beijing, China, August 3-9, 2013

G. Guo, Improving the Performance of Recommender Systems by Alleviating the Data Sparsity and Cold Start Problems, 23rd

International Joint Conference on Artificial Intelligence (IJCAI2013), Beijing, China, August 3-9, 2013

G. Guo, J. Zhang and N. Yorke-Smith, A Novel Bayesian Similarity Measure for Recommender Systems, 23rd International Joint

Conference on Artificial Intelligence (IJCAI2013), Beijing, China, August 3-9, 2013

H. Wang, B. Zou and G. Guo, Qualitative and Quantitative Preferences Based Web Service Composition by Integrating with Trust,

23rd International Joint Conference on Artificial Intelligence (IJCAI2013), Beijing, China, August 3-9, 2013

W. Gu, G. Seet and N. Magnenat Thalmann, Improved Human Interaction in Telepresence Robot using Real-time Face

Segmentation, 32nd Chinese Control Conference (CCC2013), Xi'an, China, July 26-28, 2013

M. Zhao, F.W. Tan, C.W. Fu, C.K. Tang, J.F. Cai and T.J. Cham, High-quality Kinect Depth Filtering for Real-time 3D Telepresence,

International Conference on Multimedia and Expo (ICME 2013), San Jose, California, USA, July 15-19, 2013

N.S. Narayan, P. Marziliano, and C.G.L. Hobbs, Automatic Removal of Manually Induced Artefacts in Ultrasound images of

Thyroid Gland, in 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE EMBC 2013),

Osaka, Japan, July 3-7, 2013

Rajan S. Rashobh and Andy W.H. Khong, A Multichannel Time-domain Subspace Approach Exploiting Multiple Time-delays for

Acoustic Channel Equalization, The 38th IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), May

26–31, 2013

J. Ren, X. D. Jiang and J. Yuan, Dynamic Texture Recognition Using Enhanced LBP Features, The 38th IEEE International

Conference on Acoustics, Speech, and Signal Processing (ICASSP), Vancouver, Canada, May 26–31, 2013

K. Wu, S. T. Goh and A.W.H. Khong, Speaker Localization and Tracking in the Presence of Sound Interference by Exploiting

Speech Harmonicity, in Proc. IEEE Int. Conf. Acoust. Speech Signal Process (ICASSP), Vancouver, Canada, May 26-31, 2013

Conference Papers

67

Conference Papers

S. Zhao, S.N. Koh, I. Y. Soon and K.K. Luke, Feedback Utterances for Computer-aided Language Learning Using Accent

Reduction and Voice Conversion Method, in Proc. 38th International Conference on Acoustics, Speech, and Signal Processing

(ICASSP), Vancouver, Canada, 2013 (Oral), pp. 8208-8212, May 26–31, 2013

S. Zhao, S.N. Koh, I. Y. Soon and K.K. Luke, Phonetic Segmentation Using Statistical Correction and Multi-resolution Fusion, in

Proc. 38th International Conference on Acoustics, Speech and Signal Processing (ICASSP), Vancouver, Canada, 2013 (Oral), pp. 6694-

6698, May 26–31, 2013

L. Dong, W. Lin, C.W. Deng, C. Zhu and H.S. Seah, To Exploit Uncertainty Masking for Adaptive Image Rendering, IEEE

International Symposium on Circuits and Systems, Beijing, China, May 19-23, 2013

H. Zhu, J. Cai, J. Zheng, J. Wu and N. Magnenat Thalmann, Salient Object Cutout Using Google Images, IEEE International

Symposium on Circuits and Systems (ISCAS 2013), Beijing, China, May 19-23, 2013

H. Fang, J. Zhang and N. Magnenat Thalmann, A Trust Model Stemmed from the Diffusion Theory for Opinion Evaluation, in

Proceedings of the 12th International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS 2013), May 6-10, 2013

J. Kochanowicz, A.H Tan and D. Thalmann, Enabling Generative, Emergent Artificial Culture, International Conference on

Autonomous Agents and Multiagent Systems (AAMAS 2013), Saint Paul, Minnesota, USA, May 6-10, 2013

L. Zhang, L. Wang and W. Lin, A Semantic Subspace Learning Method to Exploit Relevance Feedback Log Data for Image

Retrieval, IEEE Symposium Series on Computational Intelligence (SSCI2013), Singapore, April 16-19, 2013

A. Maimone, X. Yang, N. Dierk, A. State, M. Dou and H. Fuchs, General-purpose Telepresence with Head-worn Optical See-through

Displays and Projector-based Lighting, IEEE Virtual Reality 2013, Orlando, USA, March 16-23, 2013

Agustina and C.Z. Sun, Dependency-conflict Detection in Real-time Collaborative 3D Design Systems, The ACM Conference on

Computer Supported Cooperative Work and Social Computing (ACM CSCW 2013), Texa, USA, February 23-27, 2013

Agustina and C.Z. Sun, XPointer: an X-ray Telepointer for Relaxed-space-time WYSIWIS and Unconstrained Collaborative 3D

Design Systems, The ACM Conference on Computer Supported Cooperative Work and Social Computing (ACM CSCW 2013), Texa,

USA, February 23-27, 2013

S. Zhao, I.Y. Soon, S.N. Koh and K.K. Luke, Automatic Phonetic Segmentation, Audio Day workshop (ADSC), Singapore, January,

2013