Nokia @ Skunkworkspeople.csail.mit.edu/ledlie/papers/skunkworks-feb10.pdf · Nokia @ Skunkworks...

Nokia @ SkunkworksNokia Research Center, Cambridge, US and Nairobi, Kenya

Jonathan Ledlie, Billy Odero, Brian Omwenga, Einat Minkov, Imre Kiss, Joseph Polifroni, Jay Chen, Pauline Githinji, Mokeira Masita-Mwangi, Lucy Macharia, Jussi ImpioFebruary 2010

Outline

OutlineKenya Projects• Tafsiri (aka Crowd Translator) • Tangaza (“announce”)• SMS Find‣ Next talk: Jay Chen

Other Projects• Indoor Localization/Positioning (w/MIT)• Internet Overlay Routing (w/Imperial College London)

Other Nokia Projects

Interrupt!(please)

Tafsiri (1/2)

Tafsiri (1/2)Problem: have speech-enabled service in English• e.g. on-device or server-side mobile money transfer like MPESA• How do you translate (“localise”) it for e.g. Luo, or even Swahili

‣ Can’t purchase :-(

Tafsiri (1/2)Problem: have speech-enabled service in English• e.g. on-device or server-side mobile money transfer like MPESA• How do you translate (“localise”) it for e.g. Luo, or even Swahili

‣ Can’t purchase :-(

Our Approach: Tafsiri (aka Crowd Translator)• Cheaply create speech recognizer for local, low-corpus languages• Like Nathan Eagle’s txteagle, Amazon’s Mechanical Turk• Programmer standpoint‣ Like a “localisation” file

Tafsiri (2/2)Like Mechanical Turk: pay users for validated work (i.e. speech contributions)

MobileOperator

Worker

30 min0 min

C D D’ ZA’E

0.86, 0.93, ..., ....

Users are not translating; Tafsiri translates (“localises”) speech service

Tafsiri (2/2)

1. User flashes CX• Gets callback

Like Mechanical Turk: pay users for validated work (i.e. speech contributions)

MobileOperator

Worker

30 min0 min

C D D’ ZA’E

0.86, 0.93, ..., ....

Tafsiri (2/2)

MobileOperator

Worker

30 min0 min

C D D’ ZA’E

0.86, 0.93, ..., ....

2. Selects his native language

Tafsiri (2/2)

MobileOperator

Worker

30 min0 min

C D D’ ZA’E

0.86, 0.93, ..., ....3. Mimics voice prompts• “new transaction”

Tafsiri (2/2)

MobileOperator

Worker

30 min0 min

C D D’ ZA’E

4. Automatic verification

Tafsiri (2/2)

MobileOperator

Worker

30 min0 min

C D D’ ZA’E

4. Automatic verification

5. Payment

OutlineKenya Projects• Tafsiri• Tangaza• SMS Find‣ Next talk: Jay Chen

Tangaza (1/2)

Tangaza (1/2)Tangaza (“announce” in Swahili)• Send voice messages to friends, family, and groups‣ e.g., Nairobi taxi drivers, tomato farmers in Uganda

• “Twitter” (social net, status updates) for users with basic phones

Problem• Create a useable group speech and text messaging service• Be nice if everyone would just upgrade to my S40 application• Instead make it reasonable on all GSM phones

How it works• Text messages for creating and manipulating groups‣ “create group slot”‣ create #skunkworks 1 or join #skunkworks 1

• Call in to create spoken messages‣ Skunkworks spoken messages now linked with slot 1‣ Also listen and forward spoken messages

Tangaza (2/2)

Tangaza (2/2)Pilots• Strathmore‣ They use Facebook‣ Since November

• Huruma‣ They don’t‣ Much more Ksh conscious‣ Fewer snazzy phones‣ Starting this week‣ Nokia subsidizing, doesn’t scale

Tangaza (2/2)Pilots• Strathmore‣ They use Facebook‣ Since November

• Huruma‣ They don’t‣ Much more Ksh conscious‣ Fewer snazzy phones‣ Starting this week‣ Nokia subsidizing, doesn’t scale

Eventual Goals...?• Link with on-phone premium version, free minutes• 98% without new device could use Tangaza vanilla

OutlineKenya Projects• Tafsiri• Tangaza• SMS Find‣ Next talk: Jay Chen

Indoor Localization / Positioning (1/2)

0x6d20xbc40x5fe

Indoor Localization / Positioning (1/2)Main idea: Build mapping from RF to location

0x6d20xbc40x5fe

Two methods:• Both take ambient RF sources‣ wi-fi, cell towers‣ use GPS hints if available 0xa3b

0x6d20xbc40x5fe

Two methods:• Both take ambient RF sources‣ wi-fi, cell towers‣ use GPS hints if available

Method #1• Survey: DB of guesses of RF source x,y(,z)• Use: Triangulate observed RF sources• Works outdoors, more scalable than #2

0x6d20xbc40x5fe

Two methods:• Both take ambient RF sources‣ wi-fi, cell towers‣ use GPS hints if available

Method #1• Survey: DB of guesses of RF source x,y(,z)• Use: Triangulate observed RF sources• Works outdoors, more scalable than #2

Method #2• Scrap method #1 b/c poor accuracy indoors due to multipath• Survey: map RF signatures (aka fingerprints) to spaces• Use: lookup closest signature‣ We assume the granularity of a space is a medium-sized room

0x6d20xbc40x5fe

Indoor Localization/Positioning (2/2)

Indoor Localization/Positioning (2/2)Problem with both methods• Expert surveyor is costly and must be repeated

Our Approach: Users become Surveyors• Organic collection and update

Challenges• Erroneous data• Malicious data• Knowing what you don’t know• Motivation?• Applications?

OutlineKenya Projects• Tangaza (“announce”)• Tafsiri (aka Crowd Translator)• SMS Find‣ Next talk: Jay Chen

Other Nokia ProjectsNokia Money• Currently piloting in a state in India• Client like MPESA; outlets are Nokia shops; real bank• Interesting w.r.t. operator, bank, regulatory/gov

“Discover Maemo” in Kenya• High-end mobile programming: N900• ~July; two days; part of MIT AITI @ local univ• Details from King’ori in May

$1m App Competition• Requirement: helps people living on less than $5 a day• Deadline: April 18• callingallinnovators.com

The End

Thanks!Jonathan Ledlie

jonathan.ledlie@nokia.com

Extra Slides

Our Approach(a) Make Canonical Recordings (c) Verify Input

(b) Gather User Input (d) Expand Corpus

English Swahili Gold Std.Utterance

carboatplane

garimashuandege

“gari”“mashua”“ndege”

“...”

carboatplane

garimashuandege

“...”

Prompt User1 Utterance

ndegeg

mashuag

...ggarig

...gmashuag

ndege1

mashua1

...1gari1′

...1mashua1′

carboatplane

garimashuandege

“...”

ndegeg

mashuag

...ggarig

...gmashuag

ndege1

mashua1

...1gari1′

...1mashua1′

Intra-session Agreement?

gari1 ⋲ gari1′mashua1 ⋲ mashua1′

carboatplane

garimashuandege

“...”

ndegeg

mashuag

...ggarig

...gmashuag

ndege1

mashua1

...1gari1′

...1mashua1′

Intra-session Agreement?

gari1 ⋲ gari1′mashua1 ⋲ mashua1′

Word Utterance

carcarcarcar...

boatboat

gari1′gari2

...mashuag

mashua1

How is a speech recognizer built?

1. Expert creates dictionary

bangers b-ae-N-s@r-z batter b-ae-t-s@r ... ...

2. Collect corpus from native speakers

3. Build phoneme matcher

Output: bangers (98%) batter ( 1%) ...

1. Determine target phrases (no expert)2. Collect corpus from native speakers

“new transaction” “cancel transaction” “cancel”

3. Build phrase matcher

Output: cancel transaction (99%) cancel (1%)

Type 1: PHONEME Type 2: PHRASE

b (0.8)

bH (0.2)

ae (0.98)

ai (0.02)

“cancel transaction” (.99)

“cancel” (.01)

- Expensive (>$10m/language)+ Grammar expandable+ Memory: |phonemes|

+ Cheap ($10k/language)- Corpus not expandable without more collection~ Memory: |vocabulary| Good enough for C&C on devices w/small vocab

How is a speech recognizer built?

1. Expert creates dictionary

bangers b-ae-N-s@r-z batter b-ae-t-s@r ... ...

2. Collect corpus from native speakers

3. Build phoneme matcher

Output: bangers (98%) batter ( 1%) ...

1. Determine target phrases (no expert)2. Collect corpus from native speakers

“new transaction” “cancel transaction” “cancel”

3. Build phrase matcher

Output: cancel transaction (99%) cancel (1%)

Type 1: PHONEME Type 2: PHRASE

b (0.8)

bH (0.2)

ae (0.98)

ai (0.02)

“cancel transaction” (.99)

“cancel” (.01)

- Expensive (>$10m/language)+ Grammar expandable+ Memory: |phonemes|

+ Cheap ($10k/language)- Corpus not expandable without more collection~ Memory: |vocabulary| Good enough for C&C on devices w/small vocab

Automatic VerificationGoal:• Discard low quality work• Tolerate noise to improve trust

Previous work (Turk, txteagle, Sarmenta)

• Give k users same task‣ Slow payment

Our approach: Intra-session Agreement• Make a small fraction of user’s queries redundant• Measure acoustical similarity between each pair• Examine distribution of similarity scores

‣ Like same user saying same word? Accept‣ Else: Reject

30 min0 min

C D D’ ZA’E

D = s(a,a′), s(d,d′), ..., s(k,k′)

s(x,y)⇒acoustical similarity

Our approach: Intra-session Agreement• Make a small fraction of user’s queries redundant• Measure acoustical similarity between each pair• Examine distribution of similarity scores

‣ Like same user saying same word? Accept‣ Else: Reject

30 min0 min

C D D’ ZA’E

D = s(a,a′), s(d,d′), ..., s(k,k′)

s(x,y)⇒acoustical similarityCan be augmented with other methods• vs. Gold Standard, vs. Corpus

NRC/Cambridge ProjectsThree server-side services; prototyping in East Africa (Audio/SMS-based)

• Our focus: User-generated content

Tangaza (“announce” in Swahili)

• Send voice messages to friends, family, and groups‣ e.g., Nairobi taxi drivers, tomato farmers in Uganda

• “Twitter” (social net, status updates) for emerging markets

Crowd Translator• Apply mechanical turk model to generate input for speech recognizer

‣ Micropayments in exchange for small tasks• Improve device/service localization through speech in many more local languages

Mosoko (“mobile marketplace” in Swahili)

• Post and query advertisements for jobs, apartments, and goods• “Craigslist” for the Next Billion

Nokia @ Skunkworkspeople.csail.mit.edu/ledlie/papers/skunkworks-feb10.pdf · Nokia @ Skunkworks...

Documents

Crowd Translator: On Building Localized Speech Recognizers through Micropayments Nokia Research Center, Cambridge US Jonathan Ledlie, Billy Odero, Einat

Nokia IP Security Platforms Technical Specifications Guide · Nokia IP Security Platforms Technical Specifications Guide Nokia Enterprise Solutions Nokia IP60 Nokia IP260 Nokia IP265

Die Commander DeLuxe Collection....Nokia C 3 Nokia C 6 Nokia C 7 Nokia E7-00 Nokia N8 Nokia N 97 Mini Nokia X6 Samsung B 2100 Samsung Galaxy Blackberry Bold 9700 Blackberry Bold 9780

Ledlie Construction REIMAGINE REBUILD RELOVE · Ledlie Construction came to us in desperate need of a fresh, new brand identity. Something ... and Twitter are not just passing trends,

Nokia Severance Plan - BenefitAnswers Plus, or the Nokia Group with respect to information and/or intellectual property; (b) ... Nokia Severance Plan Nokia . Nokia Nokia Nokia

The Secrets To Creating A Killer Skunkworks

Nokia Bell Labs• michel.le_pallec@nokia-bell-labs.com • Ludovic.Noirie@nokia-bell-labs.com, Remi.Varloot@Nokia-bell-labs.com –The presenter: • Fabien.Mathieu@nokia-bell-labs.com

Skunkworks projects

Nokia 5800 XpressMusic -käyttöohjedownload-fds.webapps.microsoft.com/supportFiles/phones/... · 2016. 6. 29. · Nokia, Nokia Connecting People, Nokia Care ja XpressMusic ovat Nokia

Roaming Guard · XpressMusic, Nokia 5800 XpressMusic, Nokia C5-03, Nokia C5-04, Nokia C5-05, Nokia C5-06, Nokia C6-00, Nokia N97, Nokia N97 mini, Nokia X6 Aplikace Roaming Guard je

LB simulator by Jonathan Ledlie. Model

New Nokia in 2006 · 2019. 5. 23. · » Nokia Eseries ﬁ rst shipments – Nokia E60, Nokia E61, Nokia E70, Nokia E50 and Nokia E62 – a range of devices designed for business

Новинки сервисов Nokia - Nokia Карты

Technologies to Develop on Nokia Platforms E61i Nokia E62 Nokia E70 Nokia N70* Nokia N91 Nokia N72* Nokia N80 Nokia N92 Nokia N93 Nokia N71 Nokia N75 Nokia 6125 Nokia 6136 Nokia 6233

Gebruikershandleiding Nokia E72 - Belsimpelhandleiding.belsimpel.nl/nokia-e72-metal-grey...Nokia, Nokia Connecting People, Eseries, Nokia E72 en Navi zijn handelsmerken of gedeponeerde

User Guide for Nokia 2600 - Specialty Answering Serviceringtones.specialtyansweringservice.net/wp.../nokia... · Nokia tune is a sound mark of Nokia Corporation. Nokia operates a

2016 09-06v3 skunkworks q&a information session public

Nokia 6120 classic Nokia 6121 classic Guía del …Nokia, Nokia Connecting People, Nokia Care, Navi y Visual Radio son marcas comerciales o registradas de Nokia Corporation. Nokia

Nokia Marketing Strategies of Nokia

Skunkworks Case Study: BMW M GmbH