SIGN LANGUAGEPresented by

Harash Pal01211805309

Under the Guidance ofDr. P. R. Gupta

Facts about Sign Language• Sign languages are NOT the same all over the world.• Sign languages are NOT just gestures and pantomime, but do have

their own grammar.• Sign languages are NOT dependent on spoken languages and do not

resemble spoken languages used in the same region.• Sign languages are NOT “languages of the hands” only, but use non-

manual expressions as well.• Sign languages have NOT been invented by hearing people.• Western sign languages (e.g. American Sign Language) are NOT

better than Indian Sign Language.• Signed codes for spoken languages (e.g. Signed English) are NOT

better than Indian Sign Language.

Some Misconceptions• All Deaf Users are Written-English Literate• We Can Generate SL Text as Output• SL is Just Manually Performed English where English and SL have

the same linguistic structure – that one language is merely a direct encoding of the other.

• SL Can Easily Use Written-Language MT Technology• It’s OK to Ignore Classifier Predicates• Many software designers incorrectly assume that written English

text in a user-interface is always accessible to deaf users• Many designers believe that if audio information is also presented

as written English text, then the needs of a deaf user are met

Research Areas of Sign Language

• Sign Language Recognition• Statistical Machine Translation• English Text-to-Sign Language Animation(3D)• Speech-to-Sign Language Animation(3D Avtar)• Sign Language HCI & Computer Applications• Sign Language into Speech

Sign language vs. English Language• Sign Language has a grammar that is independent of and quite

distinct from English• SL allows much freer word order compared to English• English marks tense morphologically on verbs, whereas SL (like

many languages e.g. ASL) expresses tense lexically via temporal adverbs

• ASL and English also differ in structure at the level of phonology• Signed languages, like spoken languages, exhibit a level of

sublexical structure that involves segments and combinatorial rules, but phonological features are manual rather than oral

• English and ASL differ quite dramatically with respect to how spatial information is encoded

• English, like many spoken languages, expresses locative information with prepositions, such as “in,” “on,” or “under.”

• In contrast, SL encodes locative and motion information with verbal classifier constructions where handshape morphemes specify object type, and the position of the hands in signing space schematically represents the spatial relation between two objects

• Thus, English and ASL are quite distinct from each other within phonological, morphological, and syntactic domains

Sign language vs. English Language

Sign language vs. English Language• A natural language with a linguistic structure distinct from

English• Many deaf adults with English reading difficulty are fluent in

Sign Language• A majority of deaf 18-year-olds in the United States have an

English reading level below that of a typical 10-year-old student

• Studies have shown that the majority of deaf high school graduates in the United States have only a fourth grade English reading level (Holt, 1991).

Example #1

For example, after saying “pipe,” a Signer produced an ASL classifier construction indicating a vertically oriented thin cylinder without any accompanying speech. In this ASL construction, two “F” handshapes (thumb and index finger form a circle) overlapped with contact, and the right hand moved upward. This ASL expression does not have an exact English translation and describes the spatial orientation and general size of the pipe

Example #2English sentence and corresponding glosses:

(1) “So Sylvester who’s on the ledge [jumps into] the apartment.” JUMP

(2) P7: “I [don’t] [think] he would [really] [live].” NOT THINK REALLY LIVE

By convention, words in upper case represent English glosses (the nearest equivalent translation) for ASL signs.

Example #3• “He goes over and sees the gutter going up the side of the

building. [Happen] [what]? HAPPEN WHAT

• Right [next to] [the window]!” NEXT-TO WINDOW

• In example (4), the phrase “Happen what?” is a word-for-sign translation of an ASL conjunction phrase that could be translated in this context as “And guess what?”. This rhetorical question is followed by the answer “[it’s] right next to the window!” . In the answer, the English expletive “it” and copula “is” are deleted, which could also be considered an example of ASL-influenced English.

Example #4“[An] [old] [woman] [seem] [her] [bird] [she] [protect].”

A-N OLD WOMAN SEEM POSS BIRD. PRO PROTECT • Note:– A-N = fingerspelled article;– PRO = pronoun, a point to a location in signing space;– POSS = possessive pronoun, a B handshape (fingers

extended and together) directed toward the same location in space

• This example is another word-for-sign translation of grammatical ASL. The ASL phrases could be translated as “There’s this old woman, and it seems it’s her bird. She protects it.”

• Single handed lexical• Double handed sign• Facial Expression• Head Tilt• Signing Space• Rotation• Orientation• Pace(rate of motion or change) and Timing

Elements of Sign Language

Common Sign Language Terms

Some signing communication systems

• Sign Languages, e.g.:– American Sign Language(ASL)– Indian Sign Language(ISL)

• Signed English(SE)• Fingerspelling

ASL: Widely Used Sign Language• The primary means of communication for an estimated one-

half million to two million deaf people in the United States.• American Sign Language (ASL), a full natural language with a

linguistic structure distinct from English (Lane et al., 1996, Neidle et al., 2000, Liddell, 2003a; Mitchell, 2004).

Signed English (SE)

• Signed English (SE) is another form of manual signing communication that is distinct from ASL.

• It is not a full natural language.• There are several different styles of SE communication, but all

of them encode an English sentence into a set of signs performed by the signer’s hands.

• SE uses many of the same signs as ASL (and some additional signs of its own).

• SE retains English sentence structure and word order, and it is most commonly used in educational settings for deaf students who are learning English.

Fingerspelling

• Fingerspelling is a method of communicating the letters of the English alphabet using special handshapes to spell words during signing.

• It is usually reserved for titles, proper names, and other specific situations.

• In an English-to-ASL interpretation/translation context, fingerspelling can be more common

• An interpreter will often use fingerspelling to directly mimic terminology or specialized vocabulary being used in the original English language signal.

Gloss• Since there is no written form for ASL signs, transcriptions

frequently associate each ASL sign with some approximate English translation, called a “gloss.”

• Since glosses are used as class labels, two signs are assigned the same gloss if and only if they correspond to the same ASL lexical item.

Visual Space in Sign Language• Signing Space

– This is the area of space in front of the signer’s torso where the majority of classifier predicates and other SL signs are performed. During an ASL conversation, signers can associate locations in space around their body with people, objects, or concepts under discussion.

• Depicting Space– Sometimes the way in which objects are arranged in space is

meant to topologically indicate the 3D layout of those objects in some scene under discussion. When the signing space is used in this manner, it is referred to as Depicting Space (Liddell, 2003a).

• Token Space– Sometimes the locations associated with objects in space is not

meant to indicate how the objects are arranged in a 3D scene. In this case, we say that the space around the signer is being used as a Token Space

The Lexical Signing (LS)• A signer constructs most SL sentences by syntactically combining

individual SL lexical items into complete sentences and add grammatically meaningful facial expressions and other non-manual signals (NMS) to the performance. The exact performance of each lexical item (each sign) may also need to be modified to accommodate syntactic, morphological, or phonological requirements of particular SL.

• This form of signing, since it is fundamentally the generation of sentences based on individual lexical signs, will be called Lexical Signing (LS)

• During LS signing, the Signing Space is primarily used as a Token Space. This means that the locations chosen for entities during LS signing are not topologically meaningful; that is, one entity being positioned to the left of another in the signing space does not indicate the entity is to the left of the other in the real world

Classifier Predicate (CP)

• CP, a phenomenon in which signers use special hand movements to indicate the location and movement of invisible objects (representing entities under discussion) in space around their bodies.

• CPs are frequent in ASL and are necessary for conveying many concepts.

• During a CP, the signers’ hands represent an entity in space in front of them, and they position, move, trace, or re-orient this imaginary object to indicate the location, movement, shape, or other properties of some corresponding real world entity under discussion.

(CP) Continued……..

• A CP consists of two simultaneous components:– (1) the hand in a semantically meaningful hand shape and – (2) a 3D movement path that the hand travels through

space in front of the signer.• To produce a complete ASL sentence, a CP will typically follow

a noun phrase that indicates which object in the conversation is being described by the CP.

Creating CPs• A classifier predicate is created by first selecting one of a closed

set of handshapes based on the characteristics of the entity in the noun phrase (whether it be a vehicle, upright animate figure, squat four-legged object, etc.) and what aspect of the entity the signer wishes to discuss (its surface, size, position, motion, etc).

• The signer then produces a three-dimensional movement for the hand which communicates a contour, a position in the space around the signer, a motion through 3D-space, a physical/abstract dimension, and/or some other property of the object which needs to be communicated.

• Classifier predicates are therefore ideal for describing scenes, articulation of tools, movements, sizes, and other information of a visual/spatial or scene/process nature.

Example: CPsFor example, to express “the car parked between the cat and the house,” the signer could use a combination of three classifier predicates•The non-dominant hand in a “Spread C” handshape would indicate a location in space where a miniature invisible house could be envisioned. The signer’s eye gaze would aim at the location assigned to the ‘house.’•The dominant hand in a “Hooked V” handshape would indicate a location in space where a miniature invisible cat could be envisioned. The signer’s eye gaze would aim at the location assigned to the ‘cat.’•The dominant hand in a “Number 3” handshape would trace a path in space corresponding to a car driving and stopping in between the ‘house’ and ‘cat’ locations in space. As the car nears the end of its motion path, the open palm ofthe non-dominant hand (in a “Flat B” handshape) would form a platform on which the car parks. The signer’s eye gaze would follow the motion of the car.

Continued……

•Before each of these classifier predicates, the signer would perform an ASL noun phrase which indicates which object is being referred to: HOUSE, CAT, or CAR. During this noun phrase, the signer’s eye gaze would aim at the audience.

ASL Representation: Timing and Coordination

• In addition to under-specifying the behavior of some of the articulators of the signing performance, the ASL representations used by some previous MT systems have oversynchronized

• the timing relationships in the ASL performance. Fundamentally, all four

• of these systems have treated the ASL performance as a string of individual glosses

• (where a “gloss” is an English word that represents a particular ASL sign). Within this

• linear one-dimensional representation, these systems have tried to encode some of the

• other parallel/overlapping elements of the ASL performance.

Multi-Path SL MT Architecture

Limitations while Translation

• Lack of SL writing system• SL’s visual modality introduces unique

complexities into the translation problem. For example, the space around the signer can be used for many communicative purposes

Indian Sign LanguageIndia now has its own national association for sign language interpreters, the Association of Sign Language Interpreters (ASLI). This organization was formed in 2007 by Arun C. Rao and others. Interpreting in India is still growing, although at this time it has not been recognized as a profession by India's government.Unlike other sign languages the Indian sign language starts with emphasize to grammar rather than vocabulary first.