View
218
Download
1
Category
Tags:
Preview:
Citation preview
Artificial visionTehisnägemine
Mihhail Šubin
Tallinna Tehnikaülikool, 2010
Contents• Historical facts• Different approaches• Examples, technical details• Conclusions• Used materials
Some historical facts
• Cortical implants– William Dobelle
• Retinal implants– Mark Humayun
Dobelle’s cortical implants
• Inspired by Giles Brindley’s research (1968)• 1970-1972. Cortical stimulation of 37 sighted
volunteers• 1972-1973. Stimulation of visual cortex of three blind
volunteers who were temporarily implanted for a few days
• 1974-1978. Four blind volunteers implanted (one retained implant for 3 months, one for 14 years, two for over 20 years)
(continued)
• 2002. Eight people received implants on commercial basis.
• Total of 16 people implanted on commercial basis, last in y. 2005.
• W. Dobelle died in y. 2004, soon after this Dobelle Institute in Lisbon was closed. Development of Dobelle brain implant is continued by Stony Brook University and Avery Biomedical Devices Inc.
Retinal implants• Proof of principle demonstrated in early 90-s by Mark
Humayun• 1998. Second Sight Medical Products, Inc. Was
founded• 2002-2004. 6 people implanted with first generation
implant (16 electrodes). 5 of them still use the device in their homes today.
• 2009. Second Sight announced that the U.S. Food and Drug Administration (FDA) has granted approval for up to 20 people who are blind or who have severely impaired vision to participate in the ArgusTM II Retinal Implant feasibility study in the U.S.
Different approaches
• Cortical (brain) implant • Retinal implant• Other options?
Brain implant approach
• William Dobelle– Surface implant– Electrodes mapped after implantation– Relatively high current needed– Results introduced (several working
systems)
(continued)
• Richard Normann– Utah electrode array (intra-cortical)– Lower stimulation currents– More precision (small groups of neurons
can be stimulated)– Still no working prototype for human
Retinal implant approach• Chip with array of electrodes on the surface of retina
– Stimulates retinal nerve cells– Receives data by wired or wireless (radio/optical) channel– Electrode mapping should be 1:1 (unlike cortical implants
where electrodes should be remapped to produce logically ordered “picture”)
• Chip mounted outside the eyeball with only electrodes connected to retina– No optical data channel option– Less problems with excess heat
Other options
• Optic nerve stimulation
• Non-intrusive approaches– Sonic vision– Tactile vision
Examples, technical details
• Dobelle’s Artificial Vision system• Utah intra-cortical electrode array• vOICe• Tactile systems
Technical detailsDobelle’s artificial vision system
http://biomed.brown.edu/Courses/BI108/2006-108websites/group03retinalimplants/multimedia/article.pdf
Array of electrodes: drawing and x-ray image
Phosphene map in visual space (electrodes matrix implanted to the right occipital lobe produces
phosphenes in left visual field)
• Frame rate 1 to 10 fps (best results @ 4 fps)• System needs to be calibrated before use (stimulation current
levels may vary)• Image from camera is processed
– To map electrodes in more-or less logical order
– To provide more contrast image (e.g. edge detection)
• Phosphene map is stable (stimulation of certain brain area produces a phosphene in certain spot in the visual field)– Phosphene map moves in the visual field following view direction
(eyes tracking system is needed to stabilize its position)
• Active phosphenes flicker. No color.• Some of the patients experienced seisures while using the
system.• One of the latest patients could even drive a car using 144
electrode implant (72+72 at each hemisphere)
Utah intra-cortical electrode arrayhttp://www.bioen.utah.edu/cni/projects/blindness.htm#program
Electrode array used to find out how nail length influences stimulation results
• Biocompatibility research shows good results – 2-5 uM thick capsule forms around each electrode
• Implantation technology is important– Best results achieved when array is rapidly inserted into
cortical tissues (in 200 uS)
• At first electrode arrays are used on animals to record electrical activity of neurons while visual (or auditory*) stimulation is applied.
• Next stage is behavioral experiments
*auditory cortex is chosen for behavioral experiments because of the ease of providing auditory stimuli
vOICe Auditory Displayhttp://www.seeingwithsound.com/etumble.htm
Original camera image (left) and spectrographic reconstruction from vOICe “visual sounds” (right)
• Brightness corresponds to amplitude, position to frequency
• Technically proven effective resolution up to about 4000 pixels (voicels) – limited to some 1000 to 4000 pixels maximum due to
limitations in human hearing (32 by 32 up to a 64 by 64).
• 32 by 32 pixel resolution is more than enough to get recognizable images
• Frame rate from 1 to 8 fps (depends on resolution)• It is proven that auditory stimulation can cause
excitation of visual cortex, especially for blind people, but vOICe usability is still highly individual– Some patients claim that after long term training, adaptation
and practice they actually see the image, and the sound produced by the system turns to barely noticeable background noise
• Some sound masking occurs when using the system• Image enhancement algorythms (edge detection,
high contrast, negative image) can be used to improve perception
• 1st publication in 1992, system available since 1998
Other options, such as tactile vision substitution systems (TVSS) are also being developed
• Veresk (http://www.tactilevision.ru/english/index.php?id=phil_device)
– Electrotactile– 1000 electrodes fixed around the patients torso– Electrode spacing is currently 8mm
• Numerous electrotactile, mechanotactile and thermotactile systems were developed and tested over time
Conclusions• Dobelle’s system is proven to work, but looks frightening.
Normann’s research is not yet resulted in any practical application, but still seems more solid and promising.
• Retinal implants seem to have become more popular (than cortical ones) over last 5-10 years.
• Non-invasive solutions, like sonic vision systems, look perfect (if they really work as described). They also look like the best choice for people who were born blind.
• Pixelated vision simulators demonstrate how human brain can adapt to lower resolution. Other tests and experiments confirm great flexibility and adaptability of human brain (which is an advantage to be used in implementation of artificial vision system).
List of used materials• http://www.seeingwithsound.com/dobelle.htm• http://www.seeingwithsound.com/etumble.htm(vOICe and Dobelle's brain implant comparison)
• http://www.seeingwithsound.com/winvoice.htm (vOICe demo)
• http://www.wired.com/wired/archive/10.09/vision_pr.html (Article in Wired - Dobelle/Normann/Humayun)
• http://www.bioen.utah.edu/cni/projects/blindness.htm (Utah Electrode Array - Richard Normann)
• http://www.2-sight.com/SSMP_ARVO_2009.pdf (Second Sight retinal implant approval by FDA)
• http://biomed.brown.edu/Courses/BI108/BI108_1999_Groups/Vision_Team/Vision.htm
(cortical implants in general)
• http://www.nidek-intl.com/artificial_vision.html#topics• http://biomed.brown.edu/Courses/BI108/2006-108websites/
group03retinalimplants/multimedia/article.pdf(Dobelle’s article)
Recommended