Taking a introductory “look” at eyegaze

Lesley Iacona, M. Ed., CCC-SLP (lesley.iacona@nau.edu)

Nikkol Anderson, MA, OTR/L, ATP (nikkol.anderson@nau.edu)

Assistive Technology Center

Institute for Human Development

Northern Arizona University

Who uses eyegaze for communication?

An eyegaze board allows persons with severe speech and motor impairments to communicate by focusing their gaze on selected items displayed on the board. Children and adults with the following conditions often use light tech and high tech eyegaze for communication.

ALS (Lou Gehrig’s disease)

Brain injury

Cerebral palsy

Locked-in Syndrome (brainstem stroke)

Multiple Sclerosis

Muscular Dystrophy

SMA (spinal muscular atrophy)

Spinal cord injury

Stroke

Rhett Syndrome

Light Tech Eyegaze Communication

A light-tech eye gaze communication board is a vertically mounted board with a window cut in the middle. When the eye gaze board is held up at eye level to the child, the child and his/her communication partner should be able to make eye contact through the open window. The user then learns to scan to the choices on the board, fixate their eyegaze on the desired choice, then look back at the communication partner to confirm the selection.

Information taken from the Bridge School website at http://www.bridgeschool.org. There are photos and videos of children using light-tech eyegaze on the Bridge School website.

Pros and Cons of Light-Tech

Pros: Cheap, portable, and require no calibration.

Cons: This method of communication is limiting in that there is only so much room on the board to place choices. You have to have enough space between the items to allow the communication partner to interpret the user’s gaze accurately. This can be adjusted to some degree by changing the size of the icons, the size of the eyegaze board, or using encoding. You also need a communication facilitator.

Encoding

To prevent limiting the user to only a few choices at a time, choices can be encoded.

Encoding is the use of a pattern to represent a message. Encoding is meant to visually simplify systems and reduce the demand for access so that students can focus on content. It generally involves a row-column pattern. The location of each message is represented by 2 variables. Colors and numbers are commonly used because they are easy to see and communicate. Using partner assisted scanning, the partner would first offer the color choice for each row. When the student makes a selection, the partner then offers the number coordinate for each item. Alternatively, the partner may offer choices using a key.

Information taken from the Bridge School website at http://www.bridgeschool.org/transition/multimodal/encoding.php. There are photos and videos of children using encoding on the Bridge School website.

Light- Tech Solutions can be used to prepare for device trials with high-tech eyegaze systems.

Can be used with objects, photographs, icons or words.

Can adjust number and size of icons on the board.

Can adjust size of the eyegaze board.

Visual scanning, fixation, and joint attention are reinforced by looking at the preferred icon and then to the communication partner for acknowledgement.

Can practice learning to use dwell and blink for selection with communication partner.

Get one!

You can make your own eyegaze boards using laminated sheets, baseball card holders, foam core boards, PVC piping, plexiglass or other clear plastic.

If you want to purchase inexpensive eyegaze frames, visit www.enablingdevices.com to check out the Eye-Talks by Enabling Devices. They are $21 for a light plexiglass frame.

Patrick Joyce, an alternative communicator with ALS, designed Speakbook to be a free, light-tech communication tool for use in any environment. You can download it and watch an informative video about how to use it by visiting http://www.speakbook.org/about/.

Mid-Tech Eyegaze

The MegaBee mid-tech option is a little complex at first, but once you get the hang of it can be a good alternative to spelling using a plexiglass board. You can view a video of how the Megabee works athttp://www.megabee.co.uk/megabee_simulation.html.

Comes with software to create your own coded phrases. 

Bluetooth capabilities for data transfer. 

Carry Case

AC Charger and 24 hour run time

Dual LCD screens

In the U.S. the MegaBee is sold by AMDI for $1280 and can be found at http://amdi.net/index.php?route=product/product&filter_name=megabe&product_id=416.

High-Tech Eyegaze Systems

Traditionally Good Candidates for High-Tech Eyegaze Systems Persons who want to communicate without the need for a

facilitator to interpret their eyegaze. Persons with appropriate trunk and head control. Persons who want to use direct selection rather than

scanning for communication. Persons with appropriate vision skills.

Concerns with high-tech eyegaze

Vision problems (evaluator sometimes calibrate)

• CVI

• Droopy eyelids

Visual Fatigue

Poor Positioning

Difficulty Mounting

Poor head control.

Most of the systems are difficult to see outside.

Technical Support

Durability

Cost

Weight

Advanced functions (computer access, phone)

General High-Tech Eyegaze Features

Companies have various sized devices to meet users’ needs.

Users aged 3 to 80 have been reported.

Grids range from simple displays of 4 or 8 icons and progress to complex system that support continued language development.

Many of the devices offer text based displays for literate users.

Offer environmental control features.

Calibration Features

Each system has a different method to access calibration features. You can: Improve and remove calibration points individually. Manually change calibration area (x, y axis

settings…….)When you reduce calibration area, you lose accuracy. This setting moves the edges for calibration in from the corners, which can be difficult for some users.

Make adjustments to size, speed, color, and number of calibration points.

Animate stimuli and/or use a video for calibration. Set the camera to track both eyes together or track them

individually. This is useful if the communicator has one eye that “works” better than the other.

Set calibration to be per user area or global (important if changing user areas……).

You can set the calibration to work when wearing eyeglasses.

Camera Features

Each system has a different method to access camera features. You can: Check your eyetracking status through use of the camera. See a picture of the user that shows if they are positioned

correctly so that the camera can read their eyes.

Selection Method Features

Each system has customizable selection method features. You can: Choose dwell, blink, or use an external switch. You can set

the dwell times. Present the cursor in different formats (shrinking dot,

hand, highlighted keys, clock, or invisible feedback). Include audio feedback in the form of a “click” and adjust

volume of this.

Some devices have a zoom feature that first enlarges a section of the screen and then lets you make a selection based on that enlarged view.

What does high tech eyegaze access “look” like? The simple explanation…….

A user typically needs to be positioned at a specific angle and a specific distance from the camera that is attached to the AAC or computer system. Cameras are mounted below the monitor they are used to control. Software within the device analyzes the user’s eye position to determine where the user is looking on the screen. In order to complete this analysis, camera systems sometimes need to be “calibrated” for a specific user. Other times the system uses a compilation of calibrations from a group of users to create a sort of “average” user. This is useful when the consumer has difficulty calibrating the device themselves. Most calibrations are saved specifically for the user and recalibration is only done to improve accuracy if there has been a change in position or vision status.

To select messages, the user looks at the screen display of programmed buttons or keys. To activate a key on the screen, the user either looks at the key for a predetermined period of time (called dwell), blinks their eyes, or uses an external switch to make the selection.

How does high tech eyegaze work? The more complicated explanation……

“The Eyegaze Edge  uses the pupil-center/corneal-reflection method to determine where the user is looking on the screen. An infrared-sensitive video camera, mounted beneath the System’s screen, takes 60 pictures per second of the user’s eye. A low power, infrared light emitting diode (LED), mounted in the center of the camera’s lens illuminates the eye. The LED reflects a small bit of light off the surface of the eye’s cornea. The light also shines through the pupil and reflects off of the retina, the back surface of the eye, and causes the pupil to appear white. The bright-pupil effect enhances the camera’s image of the pupil so the system’s image processing functions can locate the center of the pupil. The Edge calculates the person’s gazepoint, i.e., the coordinates of where he is looking on the screen, based on the relative positions of the pupil center and corneal reflection within the video image of the eye. Typically the Eyegaze Edge predicts the gazepoint with an average accuracy of a quarter inch or better.”

Information taken directly from the http://www.eyegaze.com/assistive-technology-device/medical-technical-information/how-does-it-work/ website.

DynaVox Industries

http://www.dynavoxtech.com/products/eyemax/

• All the capabilities of the Vmax plus it is:• Accessible using the eyes - blink or dwell.• Integral power supply making the EyeMax ideal for

wheelchair mounting• Allows head movement & retains calibration• Full InterAACt content, including Navigator

• e-Book reader• Email• Environmental control• Internet

EyeMax System

Maestro with EyeMax

19

• Small profile of Maestro with EyeMax supports communication while maintaining the field of vision for driving a wheelchair and interacting with others.

• Easier transportation and setup with the smaller and lighter Maestro!

Language

Gateway

Gateway Pro

AlphaCore

21

• AlphaCore is designed to facilitate communication for literate adults, like those with ALS.

• Developed by Amy Roman, M.S., CCC-SLP, AlphaCore reduces message creation time by more than 50% by combining Core Vocabulary (a small set of high-usage words that account for about 75% of the words we say) with well-organized conversation pages. • AlphaCore is available exclusively on the DynaVox Vmax+ and EyeMax System.

Tobii I-SeriesCommunication Unbound

Tobii I-12 or I-15Built for purposeful 24/7 communication

Built for life 24/7

Robust, accurate Tobii eye tracking technology

Unique Wake-on-Gaze™ including Eye Accessible power plans

2 hot swap batteries, 9 hours non-stop communication, 24 hour operation mode

Stands on its own (no stand required), dual positions with rotatable screen

Purposeful, durable design: IP-43 rating

2 sizes; 12.1” & 15.0”

Multi touch capacitive screen with Corning Gorilla Glass

Voice Optimized High Quality Sound

Tobii I-Series: Purpose-Built Speech Generating Device

IR GEWA™ Environmental control with 2 transmit windows and excellent performance

Automotive Industry Bluetooth phone connection

Fanless with Solid State Drive

Dual cameras (forward for Skype and back-facing as camera)

Durable medical device Class 1

Suitable for funding, full service offering

Fully loaded with Tobii Communicator 4.8 and all AAC vocabularies

Research-based Sono Suite language content

Sono Suite Evaluation Toolkit for evaluation purposes

Tobii I-Series: Purpose-Built Speech Generating Device

Tobii LanguageSono scribe

Sono Lexis

Sono Primo

Award-winning Gaze Selection Relaxed, precise and faster computer

access Intuitive; eliminates unwanted clicks Ability to hit smaller targets Faster computer access Integrated keyboard

Portable computer access in two sizes

Has its own processor

Works on any PC computer

Tobii PCEye Go and PCEye ProFull computer access with the eyes

New Eyegaze Technology at The Prentke Romich Company

Accent 1200 with NuEye

Unity Language

Unique and powerful language system

Focuses on frequently used words that correlate with typical language development and natural communication

Capitalizes on motor planning sequences to improve generation of quick, novel utterances

28

Unity 60 Sequenced core = 2,000 + words (includes derivatives) activities = 2,000 + words pages = 2,500 + words

Unity 84 Sequenced core = 3,000 + words (includes derivatives) activities = 3,000 + words pages = 700 + words

Unity 144 Sequenced core = 7,000 + words (includes derivatives) activities = 2,000 + words pages = 300 + words

29

UNITY sequenced vocabulariesinclude many words:

Accent 1200 with NuEye*for a full list of features, please visit our website at www.prentrom.com*

Windows® 8 operating system powered by a dual core processor

Pre-loaded with NuVoice™, PRC’s exclusive software, that powers both Unity® and Essence™

Context-Sensitive Help built right into the device for fast answers that are easy to access

Front and rear cameras for taking pictures

Wi-Fi connectivity for rapid remote tech support, software upgrading, and online access

Integrated Bluetooth® connectivity

10-12 hour battery life

Language Activity Monitor (LAM)

Built-in user manual

Product Key Code for security and instant upgrades

Dedicated and integrated versions 30

NuEye Highlights*for a full list of features, please visit our website at www.prentrom.com*

Ability to see your eyes and exact positioning for set up with enhanced tracker status that includes leveling features for ease of device placement

On screen tracking box with easy to see green for active status and red for paused status

Ability to save user profiles after completing calibration and save calibration to all user areas

Advanced feature settings that allow for increased body and head movement

Ability to use both eyes together or individual eyes for active tracking

31

LC Technologies

Eyegaze Edge Tablet AAC System

Eyegaze Edge Desktop AAC System

The EyeFollower and the 600 Series EyeTracker are their stand alone eyegaze cameras systems for computer access.

Taken directly from: http://www.eyegaze.com/assistive-technology-device/

LC Technologies, Inc: Eyegaze Edge Tablet and Desktop AAC

The system offers: Keyboard Phrases Games Simple EyeMouse- Single click

program for software such as Speaking Dynamically Pro or Clicker

The Grid AAC software- Create your own icon or text screens with keys of any color or size.

Lights and Appliances- Environmental Control

Computer Access- The Eyegaze Edge acts as a peripheral mouse and keyboard which interfaces with an external Windows computer.

Eyegaze Edge Features

Highly accurate – The Edge predicts a user’s gaze point to within 1/4 inch or less.

Eye fatigue – Eyegaze Edge programs & hardware are designed to reduce or eliminate visual fatigue associated with eye-controlled systems.

Faster response – Eyegaze activation time is user-determined and can respond in as little as 1/10 of a second.

Comfortable – A  a user can operate the Edge from any position, including side-lying or with the head tilted,  without tilting the screen.

Eye variations -  Eyegaze Edge systems require only use of one eye and track accurately even if the use has droopy eyelids (ptosis). Has nystagmus ,  or irregularly shaped pupils.

Multiple positions – The Eyegaze Edge adjustable camera allows the system to be positioned high or low with a focus range of 16 inches to 30 inches..

Low light – The Edge will track a user’s eyes even in the dark.

Off angle tracking – The  user does not need to be parallel to the screen.

EyeTech Digital Systems

EyeTech Digital Systems Eyegaze Hardware

Eyetech Digital Systems sells an eyegaze system for computer access.

VT series designed to work up to 10 feet away from a television display.

Used with external AAC software by companies such as Words+ and Forbes Rehab Services (FRS) Custom Solutions.

DynaVox EyeMax systems used their algorithms for their product.

Craig Cannon in Mesa, AZ is company representative.

EyeTech Digital Systems

“EyeTech’s new hot spot user interface conserves space on a user’s screen and improves the entire user experience with quick and natural control of Windows-based programs. There are four different colored hot spots (blue, yellow, red, and green) which appear on the sides and top corners of the screen. Simply look off the screen over the hot spot. Different mouse and keyboard functions will appear thereafter.”

EyeTech Digital Systems: Quick Access 1.0 Software Eye Scrolling Tool—Just look near the bottom or top of a Web page or other

document and let your eyes control the scrolling. This new feature is a huge time saver for avid Facebook users, blog readers, and for scrolling document where there is a lot of content.

Quick Pause Hot Spot—Simply look over the blue hot spot and click on the pause button to pause the cursor. The cursor stops moving and enables the user to quickly unpause by looking at the blue hot spot again. It is that simple.

Real-time Zoom View—Now users can zoom anywhere and see a real-time view of the screen. This is ideal for zooming in on areas where there may be moving parts, such as YouTube videos or other applications with moving controls.

Animated Calibration—Easier to calibrate than ever before, Quick Access provides an animated calibration with a 5, 9, or 16 point setup. At the end of the calibration a visual score is presented to help the user measure performance.

Hand Mouse and Eye Mouse Now Play Nicely—Simply move the hand mouse and it will smoothly override the eye mouse. The eye mouse automatically resumes control once the hand mouse is not in use for a certain amount of time that you can customize.

Information taken directly from the http://www.eyetechaac.com/quick-access-1-0-eye-tracking-software-for-aac website.

General Positioning Recommendations

The angle of the device and the head should be the same. You may have to raise or lower the device and angle it to match the user’s gaze.

The head should be centered.

Eyes are level and gaze is in the middle to upper part of the display screen.

User within 18-29” of the display screen.

No obstructions between you and the camera.

Lighting and reflections can also affect accuracy so environment is important.

Can be used with glasses but you need to be sure reflections are not affecting accuracy.

Mounting Eyegaze Systems

Decide if you need a wheelchair mount, desk mount, or floor mount.

Positioning is critical to successful use of eyegaze.

Additional safety concerns due to weight of devices. Make sure mount is installed according to directions. Make sure mount is within the footprint of the chair and

does not throw the chair off balance.

Positioning of eyegaze devices can interfere with field of vision for power mobility.

Wheelchair Mounts

Wheelchair mount Advantage: Remains on the consumer’s wheelchair, so

the device is always with the user for communication. (Exception is when being transported in a motor vehicle.)

Disadvantage: If the user needs frequent adjustments for eyegaze positioning, the adjustable pieces can strip out quickly.

Desk Mounts

Desk Mount Advantage: Easily adjustable, lightweight and portable. Disadvantage: Requires a caregiver to move from one

location to another. Requires the user to be at an adjustable desk or table.

Floor Mounts

Floor Mount Advantage: Very adjustable. Can be used in

wheelchair, floor chair, or in bed. Disadvantage: Large and bulky. Not easy to store or

transport.

Mounting Companies

Daessy at www.daessy.com

CJT at www.cjtmounting.com

DynaVox at www.dynavoxtech.com/products/connectit

Mount’n Mover at www.mountnmover.com

REHAdapt at www.rehadapt.de/en/products.php (sold by amdi.net as well in the U.S.)

The future The major companies are continuing to release devices

that smaller and more powerful with faster processing speeds, as well as more intuitive options for computer access and environmental control.

Samsung Galaxy S4: You can tilt your head up or down to scroll on the page. You can also pause video by looking away.

L.C. Technologies sells eyegaze research software called Nyan. This is being used by researchers to analyze data on human cognition, consumer behavior, and oculomotor processes.

Questions?