Cognitive Assistance for the Aging

IBM Research - Tokyo

Eldercare Research

Hironobu TakagiIBM Research – TokyoAging Strategic Initiative


Perc

enta

ge o

f Pop

ulat

ion

65 y

ears

and

old

er

JapanItalyGermany

IrelandChina AustraliaBrazilUS

IndiaEgypt

2017

2017

27%

World Population Aging


62 87

28%Dementia 15% + MCI 13%estimation by Japanese Government, 2012

of people over the age of 65 are at the risk of dementia.

70%of people want to work over the age of 70 (and forever).

Life expectancy of women in 1950 and 2015.

* numbers in Japan

3%Total social cost per year in Japan is $145B or

of the GDP


We need a new model for active life-long society.


Eldercare Research: Focus Areas

Social

Intelligent assist

Sensing

Accessibility

Brain

2003 - 2012

2010 -

2015 -

2016 -

Physical assist

Japan Post

Japan Post


Eldercare Pilot Project with Japan Post• Pilot with 1000 elderlies completed (Oct. 2015 – Oct. 2016)• Medicine reminders, video telephony, online shopping, photo sharing, etc.


Global Apple + IBM Apps for Elder Services

Care givers

Family members

Elderly people


Italian Post



Social

Intelligent assist

Sensing

Accessibility

Brain

2003 - 2012

2010 -

2015 -

2016 -

Physical assist

Japan Post

Japan Post


Voice-base natural user interface for daily watchover• Analyze life patterns, feelings, interests and issues from daily conversation (e.g. cognitive decline detection)• Share information with families and care givers• Provide support that can keep the elderly self-sufficient

Conversation as Sensors

User

WeatherEventsMealsActivities

Life PatternsCognitive issuesFeelingsInterests

Family & Friends

Daily Engagement Alerts & ReportsSupport workers

WATSON


11

Conversational Eldercare - DEMO


Questionnaire Intent“What did you have last night?” Dementia (Neuro Cognitive Disease) Risk

Assessment

“Did you go out today?” Activity checking“You mentioned about gardening in diary. Are you interested in community gardening?” Recommendation of activities

”Which city do you want to travel?” Marketing“The shop, you mentioned last week, is now on sale!” Promotion

“We delivered a package, but you were not there. Please contact post office" Notification (business related)

Intent of each questionnaire


Emotion and Physical Condition Recognition

Voice data Features

988 acoustic features for emotion recognition.

• Intensity• Loudness• 12 MFCC• Pitch (F0)• Probability of voicing,

F0 envelope, • 8 LSF, • Zero-Crossing Rate

Labeled voice

dataset

ML-basedRegression

Estimation

Feeling good or bad

40

0

100

Physical condition

Ex: Good night

*Each circle represents one greeting.*One participant data

Emotion

NegativePositive

Confusion FrustrationDelight Flow

Physical Condition

EmotionYorktown – Speech team

Tokyo Research

Happy

Sad


Dashboard for Family and Care Givers

Condition

Emotion (voice sound analysis)

Personality analysis (Watson personality insights)

Daily activities

Eating habits

Dementia questionnaire scoresand more ….



Social

Intelligent assist

Sensing

Accessibility

Brain

2003 - 2012

2010 -

2015 -

2016 -

Physical assist

Japan Post

Japan Post


th Leading Cause of Death6But, interventions during MCI have been shown to be effective

Of the top 10 killers, AD is the only one thatcannot be prevented, cured, or even slowed

50-95%of the people with dementia never have screening• High income countries: 40-50% (e.g. US 45%)• Low/Middle income countries: 5-10%

Societal Impact of DimentiaPoor gait performance can be used for detection and also

prediction of dementia and mild cognitive impairments before up to 7-12 years

[Buchman & Bennett, Expert Rev. Neurother., 2011]

[McGough EL et al . Phys Ther. 2011; Beauchet O et al., J Nutr Health Aging. 2013]


• Gait speed & its variability

• Step frequency

• Stride time variability

• Step-length & its variability

• Foot swing velocity

• Stance and stride time

• Motor function• Fall risks

• Cognitive decline• Episodic memory• Executive function

• Diseases• Parkinson disease• MCI• Alzheimer’s disease

Feature extraction of gait

Features Detection



Brain Simulation: Understanding Alzheimer Disease (2016)

19

APOE-4 carriers

APOE-4 non-carriers

Investigate whether and how an abnormality in inter-areal structural connectivity observed in APOE-4 carriers (major risk factors in Alzheimer Disease) affects cortical information processing

Structural connectivity of APOE-4 carriers might reduce cortical information propagation and lead to negative effects in information integration.


Social

Intelligent assist

Sensing

Accessibility

Brain

Physical assist



Thank youHiro [email protected]: @hirotakagi

Technology

Cognitive Assistance for the Aging