Introduction of Rakuten Institute of Technology, Reality Domain Group, Rakuten

Rakuten Institute of Technology

Jul/14/2014

Rakuten Institute of Technology

http://rit.rakuten.co.jp/

Value Proposition

Third Reality

Vision Locations

Rakuten’s Strategic R&D Organization

RIT: Rakuten Institute of Technology

Tokyo, New York,

and Paris

Rakuten (now)

Positioning

Basic

research

Applied study

Seeds study

Planning

By seeds

Planning

By needs Development Release Service

Rakuten (before)

R.I.T.

Academic

RIT connects technology seeds with service development, and provide

an engine to let Rakuten grow in the long term.

3 months to 1 year 1 to 3 years 3 to 5 years

Tokyo (40), NY (10) & Paris

New York Tokyo Paris

Global R&D Teams

Scientists Developing novel theories for novel technologies

Rakuten Institute of Technology

There are three roles in R.I.T. to develop/introduce advanced

technologies to Rakuten Services.

Developing Department / Business Units

Technologists Developing novel and efficient technologies

Coordinators Identifying and integrating technological needs from business units

Organizational Roles for Innovation

Research Area Focus

Rakuten Institute of Technology

Reality Domain

Introduction

Jul/14/2014

Reality Domain Group

Rakuten Institute of Technology

http://rit.rakuten.co.jp/

8

Provide exciting new discoveries to customers via innovative presentation or interaction.

Deliver/extract more “Reality” via/from multimedia data.

Merge offline and online experiences to bring more value and “Entertainment” to shopping.

Online & Offline

Image Processing & Computer Vision

Next Generation UI/UX

Reality Domain Research Area

9

Current Research Topics

10

Image Processing and Computer Vision

Image/movie processing, multimedia, computer vision

11

Poor quality Plural objects Wrong photo Text added image No image

Selected as the

‘best image’ Images evaluated as unsatisfactory

Recommended as the ‘Best’ image

Text-free image sorting focusing on quality resolution

Text Image Detection

新発売！

12

MS EXCEL MS PPT

Web interface gives gray image on demand

Other old methods

Convert photos from color to gray while maintaining perceptual color contrast

Human perception focused - reduce colors to gray

keeping high contrast

2014 ICPRAM participated

Color Photo to Gray

13

Next-Generation User Interface/User Experience

UI / UX

14

Visualize the popularity and their attributes using human icons

Also you can watch this movie on YouTube

HITOKE: A Study of Queue Visualization of Internet Purchase Information

http://www.youtube.com/watch?v=cxVcTcZMXPs

Sync purchase information and visualize them as

human icons. Colors tell buyers’ attribute, so we

can know which clustered person likes this item.

HITOKE

15

Fashion coordination interface for mobile device

swipe

give a function to try many coordination easily

by flicking smart phone screen, also changing

size and background functions are provided.

Published@CHI2013

Also you can watch this movie on YouTube

KiTeMiROOM: A Fashion-Coordination System for Mobile Devices

http://www.youtube.com/watch?v=S1VisIV28VY

Quick browse

Change situation (background)

Flexible sizing

Tuck-in/out coordination

KiTeMiROOM

16

Picture based search interface for Rakuten Ichiba

Provide a novel interface for Rakuten Ichiba

search result, only showing item images to

bring a new discovery - serendipity.

Also you can watch this movie on YouTube

PicSer - Picture Based Search for Rakuten Ichiba

http://www.youtube.com/watch?v=QwJQbmCjCDk

Rich Image Window Shopping

Color Filtering

Similar Item Search

Image Grouping

Compare Items

Keyword addition & reduction

UIUX

NLP ImageProcessing

NLP

ImageProcessing

UIUX

UIUX

Providing many functions based on several basic technology

*PicSer

17

Search interface for hand gesture

Also you can watch this movie on YouTube

KooSHO: an environment for Japanese text input based on aerial hand gestures.

http://www.youtube.com/watch?v=xurR3sYT08w

Recognize the letter

which a user gestured by

kinect

Natural Language

Proecssing

NLP engine provides

candidate queries with

inputted letter

“Randoseru”

“Reinco-to”

“Rryukkusakku”

User can select candidate

search keyword by touch

After selecting keyword,

user can watch particular

item by touch, of course

can buy it.

“R”

KooSHO

18

Online <-to-> Offline (O2O)

Ubiquitous Computing, Augmented Reality, Virtual Reality

19

Visualize the popularity of each items by human icons utilizing AR Technology

Prototype – show the popularity by

human icons in smartphone. Each

colored icon means the attribute of

buyer, men or women, young or old, and

so on.

Advanced version – for food exhibition

In this advanced version, users can get

information on the screen by putting REAL

flyer in front of the camera. The number of

human icons are calculated from POS data. Also you can watch this movie on YouTube

“AR HITOKE: visualize the popularity of REAL shops”

http://www.youtube.com/watch?v=RF1Rh7DGW6c

Original version

AR HITOKE

20

Show the items and shops information utilizing digital signage

Also you can watch this movie on YouTube

Enriching Event Information Display with Touch Interaction

https://www.youtube.com/watch?v=OzB2J9wp_AE

The NET The REAL

Detailed item information,

Online user reviews, and so on

Access the website via QR code

R-WallSHOP

21

Business Contribution

22

Brighten and liven up a color of a dish

Yummy filter

Trim image into square (for list page) keeping important area

trim

Make dish images

more attractive

Find important area

Yummy Filter and Image Trimmer for Rakuten Recipe

23

Visitors enjoyed fashion coordination easily.

It’s first time to introduce our technology to

general users directly.

Shared the coordination on facebook

Fashion Coordinate System for Events (March 2013)

KiTeMiROOM, fashion coordination

system to empower people to check

many coordination easily, speedy.

24

HITOKE, show the popularity of each items by human icons

After applying, majority

product got more 2

times click/day and the

other reduce it by 30%.

HITOKE provided ‘bias’

It must be

popular!

HITOKE meets Rakuten24 (April-May 2013)

25

Show demonstration at Nagoya Station, Umeda-Hanshin, Namba Takashimaya

The number of human icons

means each shop’s popularity.

This number is depended on the

real POS data.

Icon sometimes

speaks online

review.

AR HITOKE at department store events (Sep-Oct ‘13)

26

Shop map in the

department store

Extracted text from user

reviews (picked by NLP

technology)

Lead to the

website

Show demonstration at Hakata Hankyu department store

Touch interesting item and detailed

information appears

R-WallSHOP at department store events

27

Cross over the border of NET & REAL