A WEARABLE INDOOR NAVIGATION SYSTEM FOR ...d- ... A WEARABLE INDOOR NAVIGATION SYSTEM FOR BLIND AND

  • View
    0

  • Download
    0

Embed Size (px)

Text of A WEARABLE INDOOR NAVIGATION SYSTEM FOR ...d- ... A WEARABLE INDOOR NAVIGATION SYSTEM FOR BLIND AND

  • A WEARABLE INDOOR NAVIGATION SYSTEM FOR

    BLIND AND VISUALLY IMPAIRED INDIVIDUALS

    by

    Yicheng Bai

    B.S., China Agriculture University, 2007

    M.S., Beihang University, 2010

    Submitted to the Graduate Faculty of

    the Swanson School of Engineering in partial fulfillment

    of the requirements for the degree of

    Doctor of Philosophy

    University of Pittsburgh

    2014

  • ii

    UNIVERSITY OF PITTSBURGH

    SWANSON SCHOOL OF ENGINEERING

    This dissertation was presented

    by

    Yicheng Bai

    It was defended on

    August 06, 2014

    and approved by

    Mingui Sun, Ph.D., Professor, Departments of Neurological Surgery, Bioengineering and

    Electrical and Computer Engineering

    Steven P. Levitan, Ph.D., Professor, Department of Electrical and Computer Engineering

    Zhi-Hong Mao, Ph.D., Associate Professor, Departments of Electrical and Computer

    Engineering and Bioengineering

    Hassan Karimi, Ph.D., Professor, School of Information Science

    Ervin Sejdi, Ph.D., Associate Professor, Department of Electrical and Computer

    Engineering

    Bernardine Dias, Ph.D., Assistant Research Professor, Robotics Institute, Carnegie Mellon

    University

    Dissertation Director: Mingui Sun, Ph.D., Professor, Departments of Neurological Surgery,

    Bioengineering and Electrical and Computer Engineering

  • iii

    Copyright © by Yicheng Bai

    2014

  • iv

    A WEARABLE INDOOR NAVIGATION SYSTEM FOR

    BLIND AND VISUALLY IMPAIRED INDIVIDUALS

    Yicheng Bai, Ph.D.

    University of Pittsburgh, 2014

    Indoor positioning and navigation for blind and visually impaired individuals has become an

    active field of research. The development of a reliable positioning and navigational system will

    reduce the suffering of the people with visual disabilities, help them live more independently,

    and promote their employment opportunities.

    In this work, a coarse-to-fine multi-resolution model is proposed for indoor navigation in

    hallway environments based on the use of a wearable computer called the eButton. This self-

    constructed device contains multiple sensors which are used for indoor positioning and

    localization in three layers of resolution: a global positioning system (GPS) layer for building

    identification; a Wi-Fi - barometer layer for rough position localization; and a digital camera -

    motion sensor layer for precise localization. In this multi-resolution model, a new theoretical

    framework is developed which uses the change of atmospheric pressure to determine the floor

    number in a multistory building. The digital camera and motion sensors within the eButton

    acquire both pictorial and motion data as a person with a normal vision walks along a hallway to

    establish a database. Precise indoor positioning and localization information is provided to the

    visually impaired individual based on a Kalman filter fusion algorithm and an automatic matching

    algorithm between the acquired images and those in the pre-established database. Motion

    calculation is based on the data from motion sensors is used to refine the localization result.

    Experiments were conducted to evaluate the performance of the algorithms. Our results show

    that the new device and algorithms can precisely determine the floor level and indoor location

    along hallways in multistory buildings, providing a powerful and unobtrusive navigational tool for

    blind and visually impaired individuals.

  • v

    TABLE OF CONTENTS

    TABLE OF CONTENTS ................................................................................................................ v

    List of tables ................................................................................................................................... ix

    List of figures ................................................................................................................................. xi

    Acknowledgement ....................................................................................................................... xvi

    1.0 Introduction ........................................................................................................................... 1

    2.0 Background ............................................................................................................................ 4

    2.1 Existing Research and Related Work ............................................................................... 4

    2.1.1 Wireless network based systems ............................................................................... 4

    2.1.2 Pseudolite GPS based system ................................................................................... 9

    2.1.3 Inertial measurement unit (IMU) based system ...................................................... 10

    2.1.4 Image feature matching approach ........................................................................... 11

    2.1.5 Other methods ......................................................................................................... 12

    2.2 Existing Technologies to Be Used ................................................................................. 13

    2.2.1 Wi-Fi-based positioning system ............................................................................. 14

    2.2.2 Audio assistance and speech recognition technologies .......................................... 17

  • vi

    2.2.3 Optimal path finding methods ................................................................................ 18

    2.2.4 Summary of Existing Technologies ........................................................................ 19

    3.0 Proposed System for Blind and Visual Impaired Individuals ............................................. 20

    3.1 Overview of the proposed system .................................................................................. 20

    3.2 Hardware design ............................................................................................................. 23

    3.2.1 Current version of eButton...................................................................................... 23

    3.2.2 Modified eButton for the proposed system ............................................................. 33

    3.2.3 Hardware features ................................................................................................... 34

    3.2.4 Power management ................................................................................................. 37

    3.3 Barometer based floor detection .................................................................................... 45

    3.3.1 General concept for barometer based floor detection ............................................. 46

    3.3.2 Barometer sensor based floor detection fundamentals ........................................... 47

    3.3.3 Altitude Difference Calculation by Atmospheric Pressure Change ....................... 51

    3.4 Landmark-based indoor positioning............................................................................... 57

    3.4.1 Overview of landmark-based indoor positioning ................................................... 58

    3.4.2 Saliency map based landmark selection ................................................................. 60

    3.4.3 Landmark description ............................................................................................. 71

    3.4.4 Landmark detection based indoor positioning ........................................................ 76

    3.5 Multi-sensor fusion ...................................................................................................... 102

    3.5.1 Data Fusion Algorithm Design ............................................................................. 102

  • vii

    3.5.2 Algorithm Implementation.................................................................................... 105

    4.0 Experimental Results ......................................................................................................... 108

    4.1 Barometer based floor detection .................................................................................. 108

    4.2 Landmark-based Indoor Positioning ............................................................................ 114

    4.2.1 Database establishment ......................................................................................... 115

    4.2.2 Ground truth data acquisition................................................................................ 117

    4.2.3 Motion estimation results ...................................................................................... 119

    4.2.4 Position localization results .................................................................................. 121

    5.0 Contributions and Discussion ............................................................................................ 127

    5.1 Contributions ................................................................................................................ 127

    5.2 Discussion .