Pepper and Watson Speech related API

Pepper and Watson Speech related API

(a.k.a Failure story of making Translation App on Pepper)

Forex Robotics Co., Ltd

Copyright ©2015 – 2016 Forex Robotics Co. Ltd. All rights Reserved.

At first,

This is failure sotry of development,

_|￣|○

Please take easy to listening …



MLResearch and Development

Marketing prediction system

RobotDevelopment Robot

App and systems

Fin-techDevelopment MT4

EA App and systems

KazTakahashiForex Robotics CO., Ltd.

CEO

１＋１1 Pepper, 1 Person（as of May 2016）

PreviousDeveloper and

researcher Trend Micro Inc, about consumer product and security

２０１５Established since

Garage Entrepreneurs

IBM Join Global

Entrepreneur Program

Get award of Pepper related hackathon

２times

Certified Official Robo App Partner （Basic)

Bluemixuse

Node-REDWatson API

DashDB

Official member of Robot Revolution

Initiative

Copyright 2015 Forex Robotics Co. Ltd. Allright Reserved.

Introduced Communication Robot Industory Map by Robot Startsorce：Communication Robot Industy Map / 2016 Q1 / Japan robot start inc.

Main Issue: One day, One customer ask me that …

“Can Pepper translate between Japanese and

Chinese without additional device?”


Customer Insight

•My customer’s store already sets up Pepper.•The store come many Chinese customer at one time. (By

sight seeing bus, guess around 50 people)•But only 2 persons about Chinese speaker staff.• If Pepper translate between JP Chinese, JP speaker

staff may support Chinese customers.•No additional cost, because my customer already have

bought Pepper.


However,


Speech to text Text to speechLanguage Translation

Pepper’s speech recognition APIsare not realized “free word” recognition. (need to define wording)

Are Pepper + Watson API able to do that?

But the reality was not so sweet …


音声認識Speech to text

音声合成Text to speech

テキスト翻訳Language Translation

EN

Portuguese

Spanish

French

Arabic

English （UK)

Portuguese(Brazil)

English （US)

Japanese

Chinese(北京語）

Arabic

Spanish

Spanish

English （US,UK）

Portuguese(Brazil)

French

German

Italian

Japanese

Functions don’t connect. ( ﾟДﾟ)

TO make matters worse…

Pepper’s microphone is easy to pick up noise

• Conjecture• Pepper has microphone on head top. (there are 4 microphones)• Also Pepper has CPU fan on head top, so easy to pick up the noise.• The sound data has disadvantage for Watson Speech Recognition API.


It should be silent.

So, from Pepper is

Difficult to recognition speech by Watson API

•千売り場はどこで

• うん無理はどこ

• チェン氏はどこです

•支援おりはどこで

• D_エー売り場とか

•遅延売りはどこで

• チェーン売り場とこです

• D_エー売りはどこです


（例）「チェーン売り場はどこですか？」

e.g. “Where is car chain section?”

To begin with,

What is the speech recognition? (in Japanese)


O H A

1. Identified vowels and consonants based on spike.

千チェン遅延チェーン

2. Analogy words based on identified sounds.

• Conjecture• When pickup wording, I guess that selected high frequency appearance word. • If so, terminology is low rate to pick up, because low frequency appearance wording.• In the first place, there is possibility of not listed wording .(e.g. special terminology or

coined word)

Summary

• Pepper’s Speech recognition APIs are difficult about free word recognition.• Watson Speech recognition API is support free word

recognition and grate support function for Pepper.• However, Japanese Chinese translation function is not

realize only Watson API. （as of May 2016）• Request of Watson Speech Recognition API

• Need frequency control and add word function for terminology.

• It's super, if improve recoginition by noises!


Finally, I made proto type!

• I’ll do demo at Robot Forum 2016 in Forex Robotics booth. (July 1st, Aug 2nd)


http://youtu.be/tTufpC5xReo

At the end,


Wanted business opportunity about cooperation of the robot and IT systems!

Also wanted partner company! (no matter JP or not)

Boost up robot industry together!

[email protected]://www.facebook.com/forexrobotics.jp/

Feel free contact me!

mailto:[email protected]

https://www.facebook.com/forexrobotics.jp/

Thank you for your attention!


Technology

Pepper and Watson Speech related API