The Constructivist Learning Architecture: A Model of ... ai-lab/pubs/ Constructivist Learning Architecture: A Model of ... with a vast heart. I am grateful to count her ... A Model of Cognitive Development for Robust Autonomous ...

  • Published on
    26-Apr-2018

  • View
    215

  • Download
    3

Transcript

  • The Constructivist Learning Architecture: A Model of Cognitive Development for

    Robust Autonomous Robots

    Harold Henry Chaput

    Report TR04-34 August 2004

    hhchaput@mac.com http://homepage.mac.com/hhchaput/

    Artificial Intelligence Laboratory The University of Texas at Austin

    Austin, TX 78712

  • Copyright

    by

    Harold Henry Chaput

    2004

  • The Constructivist Learning Architecture:

    A Model of Cognitive Development for Robust Autonomous Robots

    by

    Harold Henry Chaput, B.A., M.S.C.S.

    Dissertation

    Presented to the Faculty of the Graduate School of

    the University of Texas at Austin

    in Partial Fulfillment

    of the Requirements

    for the Degree of

    Doctor of Philosophy

    The University of Texas at Austin

    August 2004

  • This dissertation is dedicated to my parents, Henry and Constance Chaput.

  • v

    Acknowledgements

    This dissertation would not have been possible without the guidance, patience and bril-liance of my amazing dissertation committee. An eternal debt of gratitude goes to Ben Kuipers, whose inspirational words were the genesis of this work; Les Cohen, whose sage and keen insight showed me the value of my own ideas; and Risto Miikkulainen, who showed me the path from cool ideas to compelling science. I also want to thank Bruce Porter and Ray Mooney for their ad-vice and encouragement, and Peter Stone for some great ideas and Stphane Grappelli. I am ever thankful.

    It was really Robert Turknett who goaded me into working on this topic, and many of the ideas in this dissertation came from our lofty conversations and heady debates. Rob is my foil, my straight man, and my big brother. Thanks pal.

    I would have been completely lost in Psychology were it not for Cara Cashon, who was ex-ceptionally nice and patient with me from the moment we met. She has a radiant intellect coupled with a vast heart. I am grateful to count her among my close friends.

    The UTCS department is full of cool people with big brains, but Ken Stanley, Jeff Provost, and Tal Tversky were my brothers in arms and partners in crime. Each one is a genius and an ex-ceptional human being. Somebody give them a job.

    I am blessed with the greatest friends a man could have. I couldnt have written this dis-sertation without Adam Biechlin, Sue Blankenship, Carmen Gonzalez-Sifuentes, Cinqu Hicks, Mark Holzbach, Andy Hundt, Bob Kaeding, Cheyenne Kohnlein, Charles Lewis, Manuel and Davd Quinto-Pozos, Rich Skrenta and Joel Stearns. And a special expression of love to Jennifer Potter, the sister I never had. Drinks all around.

    I am deeply grateful to my comrades at the Austin Museum of Digital Art, especially Todd Simmons, Ginny Blocher, Kat Sauceda and Lisa Lee. AMODA was more than just a distraction from my research, it was nourishment for the other half of my brain. This research might be more complete if it wasnt for AMODA, but it would also be duller. Every AMODA volunteer is an in-spiration, and I am honored to have worked with them. Thanks for playing museum with me.

    Dr. Brian Slator was the man who made me serious about going to grad school in the first place, and he has treated me like a son ever since. He, along with Rita, Audrey and Megan, cared for me and made it impossible to imagine a future without my Ph.D. I couldnt have lucked into a better councillor or role model, and I hope I grow up to be just like him. Rock on.

    All these people got me started and helped me along the way. But nobody is more respon-sible for motivating me to finish than Kazuki Kinjo, the rarest of gems. Ours is a most impractical, improbable, thoroughly modern romance, one that has inspired this author to bridge oceans, chal-lenge states, and move mountains (including the one in your hands). Only Kazuki could convince me that there is life and love after the defense, and I cant wait to get there.

  • vi

    The Constructivist Learning Architecture: A Model of Cognitive Development for Robust Autonomous Robots

    Publication No. _________

    Harold Henry Chaput, Ph.D.

    The University of Texas at Austin, 2004

    Supervisors: Benjamin J. Kuipers and Risto Miikkulainen

    Autonomous robots are used more and more in remote and inaccessible places where they cannot be easily repaired if damaged or improperly programmed. A system is needed that allows these robots to repair themselves by recovering gracefully from damage and adapting to unfore-seen changes. Newborn infants employ such a system to adapt to a new and dynamic world by building a hierarchical representation of their environment. This model allows them to respond ro-bustly to changes by falling back to an earlier stage of knowledge, rather than failing completely. A computational model that replicates these phenomena in infants would afford a mobile robot the same adaptability and robustness that infants have. This dissertation presents such a model, the Constructivist Learning Architecture (CLA), that builds a hierarchical knowledge base us-ing a set of interconnected self-organizing learning modules. The dissertation then demonstrates that CLA (1) replicates current studies in infant cognitive development, (2) builds sensorimotor schemas for robot control, (3) learns a goal-directed task from delayed rewards, and (4) can fall back and recover gracefully from damage. CLA is a new approach to robot control that allows robots to recover from damage or adapt to unforeseen changes in the environment. CLA is also a new approach to cognitive modeling that can be used to better understand how people learn for their environment in infancy and adulthood.

  • vii

    Contents

    Acknowledgements v

    List of Tables x

    List of Figures xi

    1. Introduction 11.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1

    1.1.1 Importance of Flexibility . . . . . . . . . . . . . . . . . . . . . 21.1.2 Fallback and Recovery . . . . . . . . . . . . . . . . . . . . . . 3

    1.2 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .41.3 Overview of the Dissertation . . . . . . . . . . . . . . . . . . . . . .5

    2. Foundations 62.1 Psychological Foundations . . . . . . . . . . . . . . . . . . . . . . .6

    2.1.1 Constructivism . . . . . . . . . . . . . . . . . . . . . . . . . . 62.1.2 Contemporary Studies in Infant Cognition . . . . . . . . . . . . 72.1.3 An Information Processing Approach to Cognitive Development 92.1.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

    2.2 Computer Science Foundations . . . . . . . . . . . . . . . . . . . . 112.2.1 The Self-Organizing Map (SOM) . . . . . . . . . . . . . . . . . 112.2.2 Features of the SOM . . . . . . . . . . . . . . . . . . . . . . . 122.2.3 The SOM for Cognitive Development . . . . . . . . . . . . . . 12

    2.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

    3. The Constructivist Learning Architecture 143.1 The CLA Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143.2 CLA Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . 153.3 CLA Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163.4 Fallback in CLA . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

    4. Cognitive Development Model 194.1 Causal Perception . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

    4.1.1 Causal Perception in Adults . . . . . . . . . . . . . . . . . . . 204.1.2 Causal Perception in Infants . . . . . . . . . . . . . . . . . . . 20

    4.2 Modeling Causal Acquisition with CLA . . . . . . . . . . . . . . . . 214.3 Experiment 1: Acquisition of Causal Perception . . . . . . . . . . . 23

    4.3.1 Training Parameters and Measurement . . . . . . . . . . . . . 234.3.2 Level 1: Component View . . . . . . . . . . . . . . . . . . . . . 234.3.3 Level 2: Causal View . . . . . . . . . . . . . . . . . . . . . . . 25

  • viii

    4.3.4 Comparison to Infant Experiments . . . . . . . . . . . . . . . . 264.3.5 A Causal Continuum . . . . . . . . . . . . . . . . . . . . . . . 264.3.6 Stage-Like Development . . . . . . . . . . . . . . . . . . . . . 274.3.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

    4.4 Experiment 2: Fallback During Overload . . . . . . . . . . . . . . . 284.4.1 Input Vectors for The Noisy Event . . . . . . . . . . . . . . . . 284.4.2 Network Response . . . . . . . . . . . . . . . . . . . . . . . . 284.4.3 Experiment 2 Discussion . . . . . . . . . . . . . . . . . . . . . 29

    4.5 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294.5.1 Feedforward Neural Networks . . . . . . . . . . . . . . . . . . 294.5.2 Reinforcement Learning . . . . . . . . . . . . . . . . . . . . . 324.5.3 Self-Organizing Models . . . . . . . . . . . . . . . . . . . . . . 324.5.4 Evolutionary Systems . . . . . . . . . . . . . . . . . . . . . . . 33

    4.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

    5. CLA for Mobile Robots 345.1 The Schema Mechanism . . . . . . . . . . . . . . . . . . . . . . . . 34

    5.1.1 Implementation Details . . . . . . . . . . . . . . . . . . . . . . 355.2 CLA and the Schema Mechanism . . . . . . . . . . . . . . . . . . . 36

    5.2.1 Similarities between CLA and the Schema Mechanism . . . . . 375.2.2 Efficiency of the Schema Mechanism . . . . . . . . . . . . . . . 375.2.3 Efficiency of CLA . . . . . . . . . . . . . . . . . . . . . . . . . 38

    5.3 Implementing the Schema Mechanism with CLA . . . . . . . . . . . 385.3.1 Implementation Details of CLASM . . . . . . . . . . . . . . . . 39

    5.4 Experiment: CLASM in the Microworld . . . . . . . . . . . . . . . 415.4.1 Experiment Setup . . . . . . . . . . . . . . . . . . . . . . . . . 415.4.2 Learning Parameters . . . . . . . . . . . . . . . . . . . . . . . 425.4.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435.4.4 Experiment Discussion . . . . . . . . . . . . . . . . . . . . . . 48

    5.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

    6. Learning with Delayed Rewards 496.1 CLA and Delayed Rewards . . . . . . . . . . . . . . . . . . . . . . 49

    6.1.1 Constructivist Reinforcement Learning . . . . . . . . . . . . . . 506.1.2 Robust Reinforcement Learning . . . . . . . . . . . . . . . . . 516.1.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

    6.2 Experiment: Foraging . . . . . . . . . . . . . . . . . . . . . . . . . 516.2.1 Robot and Environment . . . . . . . . . . . . . . . . . . . . . . 516.2.2 Learning System . . . . . . . . . . . . . . . . . . . . . . . . . 536.2.3 Action Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . 546.2.4 Training Parameters . . . . . . . . . . . . . . . . . . . . . . . 546.2.5 Experiment Results . . . . . . . . . . . . . . . . . . . . . . . . 546.2.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 546.2.7 Experiment 1 Conclusion . . . . . . . . . . . . . . . . . . . . . 60

    6.3 Experiment 2: Graceful Recovery from Damage . . . . . . . . . . . 61

  • ix

    6.3.1 Experiment Setup . . . . . . . . . . . . . . . . . . . . . . . . . 616.3.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 626.3.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 626.3.4 Experiment 2 Conclusion . . . . . . . . . . . . . . . . . . . . . 64

    6.4 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 646.4.1 Hierarchical Reinforcement Learning . . . . . . . . . . . . . . 646.4.2 Temporal Transition Hierarchies . . . . . . . . . . . . . . . . . 656.4.3 Hierarchy of Abstract Machines . . . . . . . . . . . . . . . . . 656.4.4 Spatial Semantic Hierarchy . . . . . . . . . . . . . . . . . . . . 656.4.5 Learning from Uninterpreted Sensors . . . . . . . . . . . . . . 65

    6.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

    7. Discussion and Future Work 677.1 CLA Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . 67

    7.1.1 Recovery with Re-Learning . . . . . . . . . . . . . . . . . . . . 677.1.2 Alternatives to the SOM . . . . . . . . . . . . . . . . . . . . . 687.1.3 Alternative Integration Methods . . . . . . . . . . . . . . . . . 687.1.4 Using Neighboring Schemas . . . . . . . . . . . . . . . . . . . 687.1.5 Comparison to Reinforcement Learning . . . . . . . . . . . . . 69

    7.2 Robotics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 697.2.1 Physical Robot Applications . . . . . . . . . . . . . . . . . . . 697.2.2 Other Robot Platforms . . . . . . . . . . . . . . . . . . . . . . 69

    7.3 Psychology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 707.3.1 Testable Predictions . . . . . . . . . . . . . . . . . . . . . . . 707.3.2 Understanding Motor Development through Robotics . . . . . . 707.3.3 Adult Cognition and Language . . . . . . . . . . . . . . . . . . 71

    7.5 Neuroscience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 717.5.1 Neurological Confirmation . . . . . . . . . . . . . . . . . . . . 727.5.2 Testable Predictions . . . . . . ....

Recommended

View more >