PSY 341K Vision and Action Class hours: Tues, Thurs Room 4.244, SEA Instructor: Professor Mary Hayhoe SEAY Room X Office hours: Anytime by appointment TA: Sariel Li Office hours: by appointment. Web Site:http://homepage.psy.utexas.edu/class/341K/hayhoe/2016 Introductions Organization 1.Lectures + discussion 2.Class demos/experiments + short reports (2pages) 3. Requirements: mid-term & final exams (short answer) = 60%, 3 short reports = 36%, plus attendance/participation = 4%. 4. Readings/lectures etc on web site. 5. Theme: understanding neural basis of simple visually guided actions Text: D.A. Rosenbaum (2010) Human Motor Control. Academic Press Other readings will be available on the class web site The problem: How does the brain control behavior? Phrenology Localization of function Homunculus Traditional approach to understanding brain is by localization of function Eg fusiform face area More recently: organization of large-scale circuits Focus on simple behaviors, rather than abstract attributes. Perception, attention, memory, movement control: usually considered separately. Even simple actions involve many parts of the brain. action plan Size, direction velocity motivation signals to muscles coordinate feedback respiration heart rate memory Initiate sequence targeting But even simple actions involve many parts of the brain. action plan Size, direction velocity motivation signals to muscles coordinate feedback memory Initiate sequence targeting Many parts of the brain are active in making a single eye movement. Want to understand the whole circuit from perception to action. Substantia nigra pc Primary visual cortex Area 46MT/MIT Lets look at eye movements. Why are eye movements interesting? What can be learnt from eye movements? Direction of gaze reveals the current focus of attention. Sometimes called overt attention. Covert attention = attending to information in peripheral visual field. Gaze exclusively on task-relevant objects. Tight linkage between location of gaze and information needed at that moment. Eye movements provide a rich source of data about the locus of attention and how it shifts around. Implication: Seeing is an active process, not a passive registration of world. We need to study the integrated system. The Frog and the Dragonfly Fundamental role of perception is to choose appropriate actions. Actions chosen for future state of the world. Must take into account sensory delays and the time to execute an action. Frogs, babies, and highly skilled tennis players all accomplish amazingly complex goal directed actions, solving similar problems. What visual information is needed to change lanes? Video is speed Whats involved in changing lanes? Current lane location + any cars in front? Is there a car in the left lane? What speed? Where will it be in a few sec? Is left lane clear? Steering commands for lane change. Ie prediction, memory, current sensory data Visual information needs to be integrated across time and space to determine choice of correct action. Ability to look at these complex behaviors in the real world is a result of technological developments in eye tracking, body tracking, and also virtual environments. This allows us to get at the Perception-Action cycle. Date Topic Jan 19 Overview of the course: understanding human actions Jan 21 The Perception Action Cycle: some examples. (Guest lecture: Dr Matthis) Jan 26 Early Visual Processing: the eye and visual cortex Jan 28. Central visual processing: dorsal and ventral streams, posterior parietal cortex. (Consequences of stroke in these areas) Feb 2 Walking and catching: class experiment. ( one versus two eyes ) Feb 4 Somatosensory system: peripheral organs and central projections. Feb 9 Motor system: overview, motor programs Short Report 1 due Feb 11 Motor system: overview, motor programs (ctd) Feb 16 Muscles, spinal networks Feb 18 Motor cortex Feb 23 Basal ganglia (Parkinsons disease etc) Feb 25 Cerebellum (motor learning) Role of somatosensory system in actions: Ian Waterman https://www.youtube.com/watch?v=FKxyJfE831Q Mar 1 Eye movements: characterization and circuitry Mar 3 Eye movements: class demos Mar 8 Review Mar 10 Mid-term Mar 15, 17 Spring Break Mar 22 Eye movement demos: making sandwiches Mar 24. Data analysis Mar 29 Reaching and grasping: neural basis, feedback & feedforward control Short report 2 due. Mar 31 Reaching and grasping: ctd Apr 5 Reaching and grasping: visuo-motor coordination & plasticity Apr 7 Prism adaptation: class experiment Apr 12 Attention: neurophysiological effects, visual search Apr 14 Attention: (ctd) Apr 19 Attention and eye movements in natural environments Apr 21 Walking and texting: class experiment Apr 26 Pre-frontal cortex and executive function Apr 28 Pre-frontal cortex and executive function (ctd) Short Report 3 due May 3 Review May 5 Final Exam Visual Perception: what do we want to explain? How do we get visual information from the world and use it to control behavior? What neural processes underlie visually guided behavior? Traditional sub-areas - visual sensitivity color vision spatial vision temporal vision binocular vision/ depth perception texture perception motion perception surfaces, segmentation object perception attention perceptual learning Visual control of movement eye movements reaching attention The Eye and Retina Photoreceptor density across the retina Cone Photoreceptors are densely packed in the central fovea Visual Acuity matches photoreceptor density Relative visual acuity Receptor density The mapping of objects in space onto the visual cortex Why eye movements are hard to measure. 18mm 0.3mm = 1 deg visual angle xa tan(a/2) = x/d a = 2 tan - 1 x/d Visual Angle d 1 diopter = 1/focal length in meters 55 diopters = 1/.018 A small eye rotation translates into a big change in visual angle Primary Cortical Sub-divisions Visual Projections Eyes predict where the ball will go. This implies that we have learnt how balls are likely to behave and can predict where it will be. Gaze allocation when walking in a real environment Things to do: control direction, avoid obstacles, foot placement, characterize surroundings etc normal vision involves sets of sub-tasks or modules need to allocate attention effectively between sub-tasks. In natural behavior, humans make sequences of sensorimotor decisions that serve behavioral goals. How do humans distribute attention when there are multiple goals? SLO-MO