Upload
dinhxuyen
View
218
Download
3
Embed Size (px)
Citation preview
2009-2-3 1
Introduction to
Information School of Tsinghua University &
Research Work Conducted in Our Group
Fuchun Sun
Tsinghua UniversityState Key Lab of Intell.Tech. & Syst.
National Univ. of Singapore, 01.19, 2009
2009-2-3 2
Introduction to Tsinghua University
2009-2-3 3
Started as Tsinghua Preparatory SchoolFounded in 1911, the last year of Qing Dynasty, as Tsinghua Preparatory School for sending Chinese students to studyabroad by the “Boxer Indemnity(庚子赔款)”
“Tsinghua” comes from the name of the royal “TsinghuaGarden”, which is currently the headquarter of the university
2009-2-3 4
Became National Tsinghua University in 1928
3 Schools, 15 Depts in 1928HumanitiesSciencesLaw
School of Engineering in 1932CivilMechanical Electrical
The Original GateThe Big Hall The Tsinghua School
2009-2-3 5
Transitions since 1937
Moved to Changsha (1937) and then Kunming (1938) together with Peking Univ. and Nankai Univ. to form Sorthwest Associated Univ. Returned to Beijing in 1946 with 5 Schools
Humanities, Science, Law, Engineering, Agriculture
Became a Polytechnic Univ. in 1952Learned from Russia with restructuring of high education Established new disciplines such as Radio Engineering
2009-2-3 6
Resumed as a Comprehensive Univ since 1980’s
School of Economics and Management (‘84)
Humanities and Social Science (‘93)
School of Law (‘99)
Arts and Design (1999) Public Policy & Administration (‘00)
Journalism and Communication(’02)
2009-2-3 7
Currently, 13 Schools and 54 Departments13 Schools (49 departments)
SciencesArchitectureMechanical EngineeringInformation Science & TechnologyCivil EngineeringAerospace (’04)Medicine (‘06, Peking Union Medical College, Tsinghua University)
Economics and Management (’84)Humanities and Social Science (’93)Law (’99)Arts and Design (‘99)Public Policy & Administration (‘00)Journalism and Communication(’02)
5 Independent DepartmentsEnvironmental Science and Engineering Electrical Engineering (电机工程与应用电子技术)
Engineering Physics Chemical EngineeringMaterials Science and Engineering
1 Research InstituteInstitute of Nuclear and New Energy Technology
Campus (hectares) 405.5Beijing 306others 100
Floor (million square meters) 2.56Beijing 2.23others 0.33
2009-2-3 8
Faculty Members 2,857Professors 1,201
Associate professors 1,061under the age of 45 1,925 (67%)academicians of CAS 34academicians of CAE 30
Post-docs 957Degree Students 31,395
Undergraduates 13,900Master candidates 12,179Academic degree 5,055Professional degree 7,124
PhD Candidates 5,316
Faculty Members and Students (2007)
Foreign Students 2,021 Ph.D Students: 101Master Degree: 254Undergraduates: 784 (5.7%)Training Students: 882 (come from 75 countries)Female 943 (47%)
HK/MC/TW Students 300Ph.D Students: 59Master Degree: 96Undergraduates: 108Training Students: 37
2009-2-3 9
Professional Degrees“Master of Professional” Programs
Tsinghua’s efforts to bridge the gap between universities and industryStudents from industry with working experience
Part-time, off-campus
Intensive courseworkMath, engineering, and also
management, ……Thesis topics from industry
Solve real problems from
engineering practice
Advisors from both Tsinghua and
industry
Degree Programs and Student Distribution (Aug. 2007)
(2717) Master of Professional(616) Master of Business
Administration (MBA)(784) Executive MBA(332) Master of Public
Administration (MPA)(176) Master of Professional
Accounting (MPAcc)(118) Juris Master(30) Master of Fine Arts (MFA)
(4) Master of Lansscape Architecture (MLA)
2009-2-3 10
Research Fund and Patents
3.70 4.15.50
7.308.509.40
10.5011.7
13.814.9
19.3
0
2
4
6
8
10
12
14
16
18
20
1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007
Total
Industry
Year 1998 1998 2000 2001 2002 2003 2004 2005 2006 2007Applied China
Oversea149 189 344 441 583 758 824 872 875 985
200 230Granted China 64 121 135 187 164 501 537 521 568 601
Oversea 57 52
RM
B in 100X
million
Oversea
SingaporeNUS (Student Exchange, Joint Master Program in Logistics & EMBA, Joint Workshop between NUS ECE and Tsinghua CS)NTU (Student Exchange, EMBA)
University-University Collaboration
2009-2-3 12
University-University CollaborationUSA
Harvard (Public Administration, Business Administration, Environmental Engineering, Architecture)MIT (Business Administration, Architecture)Yale (Environmental Engineering)Stanford (Student Exchange, Business Administration)Cornnel University (Scholar Exchange, semi-annual Joint Workshop)
UKCambridge-MIT-Tsinghua Distance Education ProgramSino-UK Network Education Program (Camb.-Tsinghua)Tsinghua-Camb. IT Talents Training Program
JapanKeio (3E, cooperative degree program)Kyoto (environmental engineering, Core University Program)Waseda (Law)Tokyo Institute of Technology (Joint Master Program)
2009-2-3 13
University-University CollaborationGermany
RWTH Aachen (Industrial Engineering, Automotive Engineering)Munich University of Technology (one of the 5 global strategic partner universities
FranceSino-France 4+4 double degree programSino-France Exchange Foundation programSino-France Energy and Environment Center (cooperation between Tsinghua and French Ministry of Education)
The NetherlandsTsinghua-TU_Delft Joint Workshop (annual)Research Collaboration in Micro-electronics area
KoreaTsinghua-Samsung/LG-KAIST Joint WorkshopsAsia Research Center funded by The Korea Foundation for Advanced Studies)
Internationalization in globalization
Joint degree program
Internationalization in globalizationInternational Research Training Group
Cross-modal Interaction in Natural and Artificial Cognitive Systems (CINACS)
Lectures for CINACS
Internationalization in globalization
2009-2-3 17
Introduction to Information School S &T
2009-2-3 18
Milestones of SIST
1933: Electrical Engineering Dept Founded1952: Radio Engineering Dept Founded by merging the telecom labs of Tsinghua and Peking Univ.1955: Automatic Control Discipline Founded in EE Dept1956: Electronic Computer Discipline Founded in EE Dept1958: Automatic Control Dept Founded (including Computer, Control, and Operation Research Disciplines)1970: Computer Science & Technology Dept Founded by merging Automatic Control Dept with the computational mathematics discipline1970: Industrial Automation Dept Founded 1980: Institute of Micro-electronics Founded1994: School of Information Science & Technology Founded 2001: School of Software Founded2003: Research Institute of Information Technology Founded
Current Structure of SIST
19
Information School (SIST)
Electronic Engineering (EE) Tsinghua National Lab (TNList)
Network Research CenterComputer S&T (CS)
Research Institute of IT (RIIT)Micro- and Nano-Electronics
Institute of Micro-ElectronicsAutomation (AU)
Theoretical Computer Science Research CenterSchool of Software (SS)
Departments Institutes
Master of Professionals
Master of Professionals
2009-2-3 20
Dept of Computer Science and Technology
5 Divisions and a State Kay Lab
Computer Software and Computing Theory Division
Human-Computer Interaction and Media Integration Division
High Performance Computing Division
Computer Networks Division
Intelligent Technology and Systems Division
Bo Zhang Sanli Li Andrew Chi-Chih Yaoxue Zhang
2009-2-3 21
Dept of Electronic Engineering
5 Divisions and Two State Kay LabsInformation Opto-electronics DivisionCircuits and Systems DivisionElectromagnetic Field and Microwave DivisionCommunication and Information System DivisionSignal and Information Processing DivisionInformation Network and Complex System Div.
Youshou Wu Bingkun Zhou
2009-2-3 22
Dept of Automation
5 Divisions and 1 National Engineering CenterDetection and Electronics Div.Process Control Engineering Div.Information Processing Div.Control Theory and Technology Div.System Engineering Div.National Engineering Research Center for Computer Integrated Manufacturing System
Yanda Li Cheng Wu
2009-2-3 23
Dept. of Micro-electronics and Nano-electronics (Institute of Micro-Electronics)
3 Divisions and 1 Pilot Line (1) Devices Research Division(2) IC and System Design Division(3) Device Physics and CAD Technology Division(4) Sub-micron Pilot Line (the only one university in China that holds such a pilot line)
Zhijian Li
2009-2-3 24
School of Software Engineering
4 Divisions and 1 National Engineering CenterSoftware theory and systemSoftware engineering and managementInformation system and engineeringImage processing and graphicsNational Engineering Research Center for Enterprise Information Software
Jiaguang Sun
2009-2-3 25
Research Institute of Information Technology (RIIT)
Advisory Committee
Director
Wireless and Mobile Communication Technology R&D Center
Digital Television Technology R&D Center
Operating System and Middleware Technology R&D Center
Web and Software Technology R&D Center
Electronic Packaging Technology R&D Center
Future Information Technology R&D Center
Microprocessor and SoC Technology R&D Center
2009-2-3 26
Tsinghua National Laboratoryof Information Science and Technology (TNList)
Computer Science & Artificial Intelligence
Next-Generation Internet
Communication Systems and Networking
Opto-electronic Devices and Applications
Micro- and Nano-Electronics
Complex Network Systems
Bio-informatics
Academic Committee
Director
Adversary Committee
Academic Committee
Pervasive Computing
Technology Innovation Branch (RIIT)
Managing Board
Academia Programs (1/2)
2009-2-3 27
Ranked #1 (2008)#1 (2006) #1 (2001)
Ranked#1(2008) #1 (2006)#1 (2001)
Academia Programs (2/2)
2009-2-3 28
Ranked #1 2008#1 (2006)#3 (2001)
Ranked #1 (2008)#1 (2006)#2 (2001)
Restructuring for Better Interdisciplinary Researches and Industrial Collaboration
2009-2-3 29
Dept. of Electronics Engineering
Dept. of Computer S&T
Dept. of Automation
School of SoftwareInstitute of Micro-
Electronics
Research Institute of Information
Technology
Tsinghua National Lab of Information Science and
Technology
2009-2-3 30
Faculties and Students (2007)
Total Bachelor Master Doctor
School of Information Science 6971 2701 3078 1192
Electronic Engineering 2245 1021 801 423
Computer Science & Technology 1929 645 868 416
Automation 1463 660 532 271
Micro-electro & Nano-electronic 569 155 332 82
Software Engineering 765 220 545 0
Faculties 486Professors 166Asso. Prof. 178 Postdocs: 116
Oversea Visitors in 2006 (from 30 countries or regions, not include visitors to professors)
Nation(area)
Amount of delegation
Amount of people
America 157 425Japan 44 128
HongKong 20 77France 18 69Holand 13 48Canada 13 18Britain 13 23korea 12 83
Australia 11 23Germany 9 48Singapore 7 12
Taiwan 6 38Italy 5 33
Finland 5 16India 3 4
Norway 3 23Spain 2 9
New Zealand 2 3Philipine 2 20
Brazil 2 3Colombia 1 10
Cyprus 1 1Russia 1 3Cuba 1 4
Sweden 1 1Pakistan 1 3Vietnam 1 8
Switzerland 1 1Portugal 1 6Danmark 1 2
Total 356 1142
Visiting Outside in 2007
EE CS AU ME SE Total
Short term 116 203 57 53 11 440Long term(over 6 months) 1 5 3 1 0 10
Total 117 208 60 54 11 450
Professors visited abroad
Students visited abroadEE CS AU ME SE Total
Short term 119 158 55 27 14 373Long term(over 6 months) 11 11 7 2 0 31
Total 130 169 62 29 14 404
Tsinghua Information ForumAn Industry Sponsored Distinguished Lecture Series on Advanced Topics in IT Field (since 2004)Encourage professors to invite highly distinguished speakers from all over the worldSupported 40+ Distinguished Lectures so far under the name “Tsinghua Information Forum – A/B/C Lectures” (A=NTT DoCoMo ; B=Nokia,C=IBM)Lecture information will go to the sponsor companyVery well attendedExpecting more supports from 2007 academic year!
33
2009-2-3 34
International Student Exchange
THU-Stanford Student Exchange Program (2005~)China-experience for Stanford students and America-experience for Chinese studentsNot only course study, but also (more) industry intern, and further research collaborationMutually waive the tuition fee and recognize the credits10-12 students alternatively per year and stay for 3 months
THU-Cornnel Scholar Exchange (2005~)2 Scholar exchanged in 2006Tsinghua-Cornell Joint Workshop
THU-Waterloo Scholar/Student Exchange Program(2005~)2 Scholars and 2 students exchange each yearDouble degree program in computer science
THU-USC Scholar/Student Exchange Program (2007~)5-7 undergraduate students exchangeJoint workshop (May 24-25, 2007)
THU Teachers and Students in NUS
THU Students in Stanford
THU Students in USC
Special Fund supporting Graduate Students to Int’l Conf
Dept.2004 2005 2006 2007
DoctorMaster Total DoctorMaster Total DoctorMaster Bachelor Total DoctorMaster Total
EE 1 2 3 20 7 27 26 19 45 28 21 49
CS 4 4 26 4 30 53 14 67 54 15 69
AU 2 2 7 2 9 14 6 1 21 16 6 22
ME 1 2 3 6 3 9 7 3 10
SE 7 7 7 7 8 8
RIIT 2 2 1 1 2 2 2
Total 7 2 9 54 24 78 100 50 1 151 105 55 160
Statistic of students sponsored by SIST:
Currently supported by an alumnus, but industry support is needed for sustainable development
Students have Traveled to 30+ Countries/Regions Nation 2004 2005 2006 2007 Total
Austria 2 1 3Australia 3 7 10poland 2 1 3Macao 1 1
Bahamas 1 1Brazil 1 1 2Germany 1 3 4
UAE 1 1France 1 7 4 12Finland 1 1Korea 7 4 1 12Holand 1 1 2Canada 2 4 2 8Czechic 1 1Malaysia 1 1 2Mauritius 1 1America 2 31 52 18 103
Mexico 1 1Norway 2 2Portugal 1 1 2 4Japan 1 2 10 6 19Sweden 2 2
Switzerland 4 4Taiwan 2 6 8Thailand 1 1Turkey 11 11Spain 3 2 5Greece 2 8 10Hongkong 1 6 11 18Singapore 1 5 10 1 17
Italy 1 1 2India 1 1 2
Britain 1 5 3 9Vietnam 4 4China 1 5 1 7
合计 9 78 151 55 293
Research Work Conducted in Our Group
Intelligent System Theory &Space Application Technologies
Prof. Fuchun SunDistinguished Young Scholar in 2006Excellent Dissertation Prize in 2000Associate Editor《IEEE Trans. On Neural Networks》
Member of the Editorial Board of《Soft Computing》and …
Associate Prof. Huaping LiuMore than Papers Published in IEEE Trans. on Automatic Control, Fuzzy Systems, Circuits
and Systems, Automatica, etc.
Intelligent System Theory &Space Application Technologies
Intelligent System Theory &Space Application Technologies
8 Ph.D Candidates
6 Master Candidates
4 Postdoctors
1
一、Spacecraft Dynamics and Control
(1) On-orbiting identifications for space robots or vehicles?
(2) Engineering oriented controller design?(3) Optimal trajectory planning with
vibration reduction?(4) Optimal docking strategy for manned
maneuvering units or two spacecrafts? (5) Position/force hybird control for object
graping?
(1)A Space Robot with Flexible Dual Arms
Stanford Space RobotA Space Robot Test –bed in Tsinghua Univ.
一、Spacecraft Dynamics and Control
(2)A Space Robot with One Arm
A Space Robot with One Arm
一、Spacecraft Dynamics and Control
(3) Manned Maneuvering Unit
Manned Maneuvering Units and Models
一、Spacecraft Dynamics and Control
(1)Modeling the satellite networks and performance evaluation.
(2)Dynamic Routing algorithm.(3)Transmission Protocol and
optimization.(4)Satellite management and control.(5)QoS of satellite networks.
二、 Satellite Networks
In low load(link availability 90%),Transmission delay dominates,single layer network is better than two-layer one.
In high load(link availability 98% ),processing and queue delay dominates,the two-layer network is better than the single-layer one.
三、Tele-operation
目标星服务星
地面站 地面站
卫星
通信网
通信
通信网 测控中心
通信卫星 通信卫星
地面操作中心
Space Teleoperation System
Serving Sat、Target Sat、Communication Sat
Ground-based Control Center,Measuring/Control Center,Ground Station
Function Demonstration 一 Capturing the Client Satellite
Initialization Gripper moves to Client Gripper moves to Client
Grasp the clientThe client rotates 90 degreePull the client to move
三、Tele-operation
Initialization Gripper moves to unit
Grasp the ORUMove the ORUPut the ORU in preassigned place
四、Tele-operation
Function Demenstration一ORU
Gripper moves to unit
THMR-I:Indoor (1986-1990)
THMR-II: Indoor
(1986-1990)
四、Mobile Robots
THMR-V:Outdoor(1999-2005)
Optical coder
Compass
DGPS
Laser radar
CCD camera
Control system
Data radio
THMR-III:Outdoor(1991-1998)
四、Mobile Robots
Key technologies:
(1) Road detecting for mobile robots.(2) Location for mobile robots.(3) Path planning and obstacle
avoidance.(4) Vision serving control.
四、Mobile Robots
5.1 What is the multi-scale model ?
Space scale: The size of the airplane, the distance between the atomsMulti-scale Systems (MUSS)
Scale
Time scale: Vary from the time of flight to the vibrations of the electrons Multi-time scale systems (MTSS)
五、Modeling & Control for multi-time scale systems
Example:Reactor, Neurobiology
5.2 What are multi-time-scale dynamical systems?
A class of systems whose dynamics are described by
mathematical models in different time scales.
For example: A space robot
五、Modeling & Control for multi-time scale systems
Some typical multi-time scale
dynamical systems
Aerospace
Manufacturing
Airplane
Process Industry
RealNetwork
Networked systems
Space Physics
5.3 The description for multi-time-scale systems?
The multi-time scale system can be described as
Whereεis the small parameter. However, in the past
decades, most achievements are restricted to
1 1 2
2 1 2
( ) ( ( ), ( ), ( ), )( ) ( ( ), ( ), ( ), )
t t t tt t t t
εε ε
=⎧⎨ =⎩
x f x x ux g x x u&
&(5.1)
1 1 2
2 1 2
( ) ( ( ), ( ), ( ))( ) ( ( ), ( ), ( ))
t t t tt t t tε=⎧
⎨ =⎩
x f x x ux g x x u&
& (5.2)
五、Modeling & Control for multi-time scale systems
5.4 Problems concerned with multi-time-scale models?
(1) The ill-conditioned property of the multi-time scale dynamics?
(2) Computational complexity?Not suitable for system simulation and controller design.
(3) ε-independent control problem is enough? In some cases,ε-independent control solution do not exist.
Furthermore, for more precise control, ε-dependent control problem is required.
1 1 2
2 1 2
( ) ( ( ), ( ), ( ), )( ) ( ( ), ( ), ( ), )
t t t tt t t t
εε ε
=⎧⎨ =⎩
x f x x ux g x x u&
& (5.3)
五、Modeling & Control for multi-time scale systems
A flexible single-link manipulator, space robot, etc. Linear singularly perturbed model
A flexible multi-link manipulator, large-scale space structure, etc.
Nonlinear singularly perturbed model
T-S fuzzy model
Fuzzy singularly perturbed model (FSPM)
5.5 Fuzzy Singularly Perturbed Models (FSPMs)
五、Modeling & Control for multi-time scale systems
FSPM approach shows the following merits: (1) The FSPM simplifies model representation for a class of
multi-time scale dynamical systems, such as flexible-link manipulator, etc.;
(2) Well-developed techniques for linear singularly-perturbed systems are available for the controller design of a FSPM, so that the upper-bound of the small parameter can be obtained;
(3) The stability and control performance can be guaranteed.
1 1i p ipIf is and and isξ μ ξ μL
11 12 1
21 22 2
( ) ( ) ( ) ( ) ( ) ( ) ( )( ) ( ) ( ) ( ) ( ) ( ) ( )
i i i
i i i
t t t tThen
t t t tε ε ε
ε ε ε ε⎧ = + +⎨
= + +⎩
x A x A z B uz A x A z B u&&
( ) ( ) ( ) ( ) ( )i it t tε ε ε= +E y A y B u&
(5.5)
五、Modeling & Control for multi-time scale systems
Theretical research conducted:
(1) Sufficient conditions and Necessary conditions of FSPM as universal Approximator to multiple scale nonlinear systems.
(2) Controllability of FSPM.
(3) Adaptive and Robust Control for Multiple Scale Nonlinear Systems based FSPM.
(4) FSPMs with markov jump parameters are used to model the networked control systems, some related theoretical results for controller design are hopefully obtained.
(5) ε-dependent Robust control and adaptive control Based FSPM.
五、Modeling & Control for multi-time scale systems
Related works ------ Our group
Controller design for Markovian jumping systems subject to actuator saturation, Automatica, 2006, 42(3): 459-465.
Stability analysis and synthesis of fuzzy singularly perturbed systems, IEEE Transactions on Fuzzy Systems, 2005, 13(2):273-284.
Hinf control for fuzzy singularly perturbed systemsFuzzy Sets and Systems, 2005, 155(2): 272-291.
Comments on "H-infinity fuzzy control design for nonlinear singularly perturbed systems with pole placement constraints: An LMI approach" IEEE Transactions on Systems, Man and Cybernetics, Part B, 2004, 34(6):2422
Comments on ‘constrained controller design of discrete Takagi-Sugeno fuzzy models’, Fuzzy Sets and Systems, 2004, 146(2): 473-476
Reduced-order H-infinity filtering for linear systems with markovian jumpParameters, Systems and Control Letters, 2005, 54(8):739-746
80%
20%
Visual Perception &Brain Intelligence
High Performance of Computers
Visual cognition & Machine Intelligence
Unstructured Information (USI)
Structured Information (SI)
USI
SI
Visual Cognition Principle and Model
Brain Intelligence
五、Unstructured Information Processing
……
Topological PerceptionTheory
Marr Feature Analysis Theory
Feature Space Theory:PCA, ICA
Scale Space, MRF Feature ExtractionFeature SelectionImage Analysis
Visual Path ModelTarget RecognitionTracking & LocationInformation Fusion
What & Where Path
Selective AttentionMechanism
Feature ExtractionTarget Detection
Image Compression
Receptive Field (RF) Model DOG (Difference of Gaussian)
Feedback ConnectionPath
Feature Association Information Interaction
Feature Extraction
Learning Mechanism Self-organized、
Unsupervised
Feature ExtractionTarget Recognition
Visual Cognition Mechanism & Computation Model for USI Processing
Stimulus: on off
五、Unstructured Information Processing
KnowledgeReasoning
Integration Decision
Expression and Computation from USI to SI
Knowledge(SI/ Formalization)
decision-making
Data(USI)
Information(Semi-SI)Content
Understanding
Data Mining
Information Refining
Knowledge Mining
五、Unstructured Information Processing
Multi-modal information Brain-Computer Interaction (BCI) and fusion are important approaches for robust feature processing, target detection, target recognition, Information Fusion, etc.
Multi-modal information BCI
Multi-modal fusion based auto-navigation for intelligent vehicles
Applications for Multi-modal Information and Fusion
Multi-sensor target imaging, fusion and recognition
Evidences for Multi-modal Information Interaction and Fusion
1 Cat’s visual cortex can also receive auditory information;
2 Cat’s epithalamus can receive visual, auditory and touch information;
3 Rattlesnake’s coping area receives visual and infrared information;
4 Hawk’s night vision cells combine the level error and time error of concha.
Multi-modal perceptive information congregates in many traditional areas determined
by single perception signal.
Why multi-modal Fusion?
1 Improve the receiver’s detection, recognition, discrimination and memory ability;
2 Multi-stimulation is more effective than single stimulation.
Physiological Evidences:
五、Unstructured Information Processing
1. Image Feature
Multi-modal Feature
五、Unstructured Information Processing
场景
文本
人工
文本
3. Text FeatureScene Text Artificial Text
2. Audionary Feature
五、Unstructured Information Processing
4. Biological Feature (Face Feature and Gait Feature)
五、Unstructured Information Processing
(Ghazanfar & Schroeder, 2006)
Spontaneous level indicated by the white line
Receptive Fields of a Ventral Intraparietal Area Neuron
Multisensory Integration and Multi-modal Fusion
Superposition ofthe receptive fields from the two modalities
五、Unstructured Information Processing
A Biological Instance of Multi-Modal Information Fusion
Target
Heat Eyes
Light Eyes
Feature Fusion
Feature Extraction
Perceptive Image
………………
……
……
…...
……
……
…...
……
……
…...
……
……
…...
……
……
…...
…………
……
……
…...
……
……
…...
……
……
…...
Reverse Projection
Fusion
Rattlesnakes: Target Recognition through Multi-modal Information Fusion
Reverse Projection
Fusion
五、Unstructured Information Processing
Multi-sensory Remote Sensing Image Fusion Based on Biological Multi-modal Fusion Mechanism
Feedback
Fusion
Recognition
Optical Sensor
SAR
Target
1A
1( )j A 2A
3A
4A
2( )j A 3( )j A
4( )j A
Ω
U
Feature Extraction & Mapping
Feature Extraction & Mapping
Feature Fusion
………………
…..
…..
…..
…..
…..
…..
…..
…..
…..
……
Feedback
Fusion
A Direct Application with Multi-Modal Information Fusion
五、Unstructured Information Processing
Feature Point Matching
Remote Sensing Image Registration and Fusion
五、Unstructured Information Processing
Remote Sensing Image Registration and Fusion
五、Unstructured Information Processing
Optical Image SAR Image Fusion Image
Optical Image SAR Image Fusion Image
Multi-sensory Remote Sensing Image Fusion
五、Unstructured Information Processing
Road Line Detection
Multi-modal Information Fusion for Intelligent Vehicle’s Autonomous Navigation
五、Unstructured Information Processing
Radar and Visual Information Fusion
Multi-modal Information Fusion for Intelligent Vehicle’s Autonomous Navigation
五、Unstructured Information Processing
Feasible Region Detection
Multi-modal Information Fusion for Intelligent Vehicle’s Autonomous Navigation
五、Unstructured Information Processing
Feasible Region Detection
Multi-modal Information Fusion for Intelligent Vehicle’s Autonomous Navigation
五、Unstructured Information Processing
Pedestrian Detection
Pedestrian Detection with Integrated Detector and Shapelet Feature
五、Unstructured Information Processing
Pedestrian Detection with Integrated Detector and Shapelet Feature
Pedestrian Detection
五、Unstructured Information Processing
Vehicle Tracking
Pedestrian Tracking
Target Tracking with Kalman Filter and Sequence Monte Carlo Filter
五、Unstructured Information Processing
Pedestrian Tracking in Multi-camera Environment
五、Unstructured Information Processing
Some Application Instances Based on Multi-modal Video
1. Video Segmentation and Classification
五、Unstructured Information Processing
Some Application Instances Based on Multi-modal Video
2. News Video Processing System
3. Sports Video Analysis
五、Unstructured Information Processing
4. Key Figure Discovery
5. Speaker Detection
五、Unstructured Information Processing
6. Pedestrian Detection, Tracking and Recognition
五、Unstructured Information Processing
Three Aspects for Visual Cognition ComputationStudy
On?
Technical Realization
MechanismModels
Theoretical
Method
Visual cognition mechanism and computational model for USI
processing
Modeling and machine learning for USI
Content Comprehension and semantic representation
Multi-modal information interaction and knowledge reasoning
Feature extraction and conformity for USI
Environmental perception, target detection and target recognition
based on visual information
Target location and tracking based on multi-sensory information fusion
五、Unstructured Information Processing
Anthropopathic analogyHow to interact human-human?
seeing, hearing, taste, touch, smell,reading, speaking, writing …
Our goal: change the computer’s quasi deaf-mute-blind state, and let it
hasseeing, hearing, taste, touch, smell, perception ability learning, memory and emotion cognition abilityspeaking, writing expression abilityproprioception, self-consciousness
Human-Computer Interaction
A Higher Level: Multi-modal Fusion and Interaction
Human-Human Interaction
五、Unstructured Information Processing
Thank you for attention!