Symbol Dragoman

Nawar Halabi, Chaohai Ding, Yunjia Li, Lama Al-Zaben developing with Alaa Mashat, Fadwa M.Al Rowais and

Kewalin Angkananon

Mentors: Dr Mike Wald and E.A. DraffanWAIS Research Group

Present Situation

Pictograms / symbols are used by those who cannot speak or have poor language skills. Symbols can also be used to provide literacy skill support but they can be ambiguous in terms of their meaning.

Example of Bliss symbols

Example of a Communication Aid with symbols and wordsWords are often placed below or above the symbols - question and answer sessions can often reveal what is/was actually meant in a conversation.

http://www.assistiveware.com/product/proloquo2go

Multilingual situation

English Put on the coat

Arabic Dress the coat –

femaleألبسي المعطف Dress the coat –

maleألبس المعطف

Need to think about the morphology – where the building blocks of a language change the make up of words

Making choices regarding the number and type of symbols offered to suit individual skills and abilities.

Where the user, parents, carers, therapists and teachers speak different languages here

Systems already available

Those that allow symbol to word translation without

grammatical corrections – Grid Playersingle word/multiword to symbol

translation – Widgit Pointsentence creation when swiping across a

symbol set – Symbol Path Android appsymbol to quick phrase/sentence selection

with machine translation

Requirements

A system that will allow Arabic symbol users to work in both languages with the symbols and words appearing from right to left for Arabic and left to right for English

A system that works out what will possibly be the meaning of a collection of symbols chosen by the user in any order

Grammatically correct sentences / phrases presented in two languages.

Development Process - Sentence Generation

Two methods were used one for English and one for Arabic with TATOEBA. English: Using  a large corpus of sentences taken from

http://www.manythings.org/anki/ .... Indexed by elastic search (http://www.elasticsearch.org/) and we look for it by keywords

Arabic: Added a rule-based generator inspired by the Subsumption architecture

Development Process – English sentence Generation

Development Process – Arabic Sentence Generation

Subsumption Architecturehttp://cs.brown.edu/~tld/courses/cs148/02/architectures.html

Sample screenshots Home Page

Responsive Design

Symbol Dragoman http://symbolmanager.ecs.soton.ac.uk:8080/

Further Information:https://access.ecs.soton.ac.uk/events/

ss12

Code Available from BitBucket https://bitbucket.org/nhalabi/symbol-dragoman

Acknowledgements

Symbols from ARASAAC http://www.catedu.es/arasaac/index.php Other icons from the Noun Project http://thenounproject.com/

Creative Commons – Attribution (CC BY 3.0) Speak designed by Mister Pixel from the Noun ProjectRefresh designed by Jardson A. from the Noun ProjectHouse designed by John Caserta from the Noun ProjectPeople designed by useiconic.com from the Noun ProjectAnimals designed by Pete Fecteau from the Noun ProjectSchool designed by Mike Wirth from the Noun ProjectFood designed by Johannes Ben Jurca from the Noun ProjectClothes designed by Daniel Hanly from the Noun ProjectTime designed by Wayne Middleton from the Noun ProjectPaint Brush designed by Björn Andersson from the Noun ProjectEmotion Dice designed by Martin Vanco from the Noun ProjectPlaying designed by jon trillana from the Noun Project

Thank you to all those who participated along the way.