Crowdsoucing Based Mobile Image Translation

A Crowdsourcing Based Mobile Image Translation and

Knowledge Sharing Service

Department of Computer ScienceWaseda University, Tokyo, Japan

1Helsinki Institute for Information Technology

2Eindhoven University of Technology

[email protected]

Yefeng Liu, Vili Lehdonvirta1, Mieke Kleppe2, Todorka Alexandrova, Hiroaki Kimura, Tatsuo Nakajima

mailto:[email protected]


A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus

• Introduction

• Human Mobile Image Translation

• Preliminary Study

• Discussion

• Future Directions

Outline

2


Introduction

A menu board outside a restaurant, Tokyo

“...I can’t wear tie here?? Should I

take off my tie?..”

3


Real World Problem

• Digital pocket translators or online translation services are useless if you donʼt know how to input the characters.

4


(Typical) Mobile Image Translation

Image Text

OCR Optical Character Recognition

MT Machine Translation

Irregular fonts or formats, handwriting, etc. Poor

performance

English Text

5


Image Text

Question of the image

TranslatorCommunityOutsourcing

English Text

• Better quality in text recognition and translation

• Human worker can provide richer interpretations and responses in addition to literal answers.

Our Solution: Human Mobile Image Translation

Crowdsourcing

6


Image Based Translator + Mobile Q&A

• NOT only a translator

• But also a knowledge broker that allows users to share high level information pertinent to the situation at hand, e.g.

• advice

• explanations

• instructions

• suggestions

7

Q: “what’s the difference between 1 and 2 in my electricity bill?”

A: “1is basic charge, 2 is additional fee ”


Basic work-flow overview

Open call

etc.

Requester Translators Requester

ScoringEnglish

Kanji

Best answer

8

A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 9

Translation Expert/Social Search Mobility Quality

(OCR based) Mobile Image Translation

Yes No Yes So-so

Human-based Search Yes & No Yes No Good

Proposed Solution Yes Yes Yes Good

Comparison


Preliminary study

• A preliminary study and design research aims to

• verify the feasibility of the design

• identify real user requirements and design issues

10


Preliminary study - Method

Collected pictures/questions from potential users

Fifteen characteristic cases were selected from the collected images

Interviewed the requesters what kind of answers they were expecting

Assigned questions to invited translators

Interviewed translators for their feedbacks

Compared the results with the requesters’ expectations

11

we interviewed the translator for collecting their feedback.


Preliminary Study Cases - Example

“...how long do I have to wait?”

Information in the picture is insufficient to answer this question.

However, most of the repliers can still suggest an approximate waiting time according to their life experiences.

12



“What are the events between 5th and 8th?”

Poor question text.

Some translators misunderstood the question, thus provided useless answers.

Preliminary Study Cases - Example (2)



Preliminary Study - Implication

1. Communication between requester and worker.

Better communication Better understanding Better result

2. Question/Answer style

• Short, but clear (e.g clarify to what level of details is wanted);

• Question with choices is better;

• Asking for links (of image/web page/etc) is a good way to lower the difficulty and faster the response time.


Discussion (1)

1. Quality of outcome

Misunderstanding between requester and worker strongly affects quality of outcome.

- Requesters may use unclear or too complicated English.- Workers often are not native English speakers.

- People always make mistakes.- Malicious replies.


Discussion (2)

An additional proofreading phase.

open call

Requester Translators Requester

ScoringEnglishKanji

Best answerProofreaders


Discussion (3)2. Different user types (user requirements)

Client Users

Short-term stay Long-term stay

Need immediate answer

Need immediate answer

Waitable Waitable

A B C D

may have different preference on the accuracy vs. timeliness trade-off


Future Directions (1)

1. Dynamical task allocation with real time requirement

i. capable for the task

ii. available for the task

Not only about if the worker is free, but also involves other factors like expertise, properties of question, etc.

• Task is better be assigned to worker who is:


Future Directions (2)

2. Motivation and Incentive

Social and Intrinsic incentive: game play

A location-based mobile game is designed


• Current Status

20

Conclusion

Early Test

Prototype Implementation

Redesign Usability/On field studyRedesign

Preliminary StudyDesign

• Human-based mobile image translation system

• Preliminary study

• Conclusion

• Findings and future directions

Thank you for your attention!

Distributed & Ubiquitous Computing Lab.Depart. of Computer Science, Waseda University

http://www.dcl.info.waseda.ac.jp/

Yefeng Liu, PhD candidate

[email protected]

http://www.dcl.info.waseda.ac.jp

http://www.dcl.info.waseda.ac.jp





“keywords” style answer is preferred

- Many translators use English as 2nd or 3rd language, they oftenface the problem of being unable to explain in long sentence.

a). “Pork, spicy, famous chinese food”

b). “Twice cooked pork (huiguo rou)”

- meaningless if don’t know the name



“is it a show or training course?”“what’re these two? can you provide links of pics”



what divination result I got here?I wanna buy the ticket for swim!

Documents

Crowdsoucing Based Mobile Image Translation