View
250
Download
3
Category
Preview:
Citation preview
Microphone Array Design and Beamforming
Heinrich Löllmann
Multimedia Communications and Signal Processing
heinrich.loellmann@fau.de
with contributions from Vladi Tourbabin and Hendrik Barfuss
EUSIPCO Tutorial on Embodied Audition for Robots
August 31, 2015
Overview
Introduction
Microphone Array Design
• measure for array performance
• array design for Spherical Harmonics (SHs)
• new robot head design
Beamforming
• robust least-squares beamformer design
• HRTF-based design for robots
• evaluation for Automatic Speech Recognition (ASR)
Conclusions
H. Löllmann: Microphone Array Design and Beamforming
2
Introduction Microphone Array Design Beamforming Conclusions
Introduction
Beamforming is used by robots to enhance the performance of
the automatic speech recognition (ASR)
Numerous publications about beamformers design and related
approaches, e.g., [Brandstein & Ward, 2001, Van Trees 2002,
Benesty et al, 2008]
• fixed beamformer designs
• adaptive beamforming
• blind source separation (BSS)
...
Which issues are special for a beamformer design in the
context of robot audition?
H. Löllmann: Microphone Array Design and Beamforming
3
Introduction Microphone Array Design Beamforming Conclusions
Design Aspects for Robot Audition
Construction of the robot (head)
• How many microphones are needed/feasible?
• What are the ‘optimal’ microphone positions?
Beamforming with head microphones
• influence of the head (no free-field)
• presence of ego-noise ( tutorial talk on signal enhancement)
• influence of robot movements, especially head rotations
Localization and tracking of the desired speaker
tutorial talks by C. Evers and R. Horaud
H. Löllmann: Microphone Array Design and Beamforming
4
Introduction Microphone Array Design Beamforming Conclusions
Optimal Microphone Positions
Many robot systems use 2 microphones to mimic the human
auditory system [Argentieri et al., 2013]
Can we do better than with only two ears?
How can we determine the optimal microphone placement?
Measure for array performance in dependence of sensor positions
presented in [Tourbabin & Rafaely, 2014]
H. Löllmann: Microphone Array Design and Beamforming
5
Introduction Microphone Array Design Beamforming Conclusions
Model for generalized head-related transfer function (HRTF)
Generalized HRTF Matrix
H. Löllmann: Microphone Array Design and Beamforming
6
complex amplitude of far-field source
complex pressure amplitudes transfer function between source j and sensor l
Introduction Microphone Array Design Beamforming Conclusions
Generalized HRTF Matrix
Representation for D sources, M sensors, K frequency points
Compact matrix formulation
H. Löllmann: Microphone Array Design and Beamforming
7
Introduction Microphone Array Design Beamforming Conclusions
Generalized HRTF Matrix
Singular Value Decomposition (SVD) of matrix with generalized
head related transfer functions (GHRTFs)
Observation
• information to construct from mainly contained by the most dominant
eigenvectors
• high effective rank of GHRTF matrix indicates a good sensor placing
H. Löllmann: Microphone Array Design and Beamforming
8
Introduction Microphone Array Design Beamforming Conclusions
The effective rank of the GHRTF matrix is given by
Optimal microphone positions obtained by
Solution can be found by genetic algorithm optimization
Optimal Sensor Placement
H. Löllmann: Microphone Array Design and Beamforming
9
Introduction Microphone Array Design Beamforming Conclusions
Optimal Sensor Placement
Simulation example
• effective rank of GHRTF matrix for single positions on head surface
Relation between effective rank and beamformer robustness as
well as DOA estimation accuracy derived in [Tourbabin & Rafaely,
2014]
H. Löllmann: Microphone Array Design and Beamforming
10
Introduction Microphone Array Design Beamforming Conclusions
Array Design for Spherical Harmonics (SH)
A nearly ball-shaped robot head with many microphones
motivates array designs in the Spherical Harmonic (SH) domain
An approach related to the previous method developed for this
case
• optimal position founds by minimizing the aliasing level for different
positions
• outline of this concept provided by the appendix, but a detailed
treatment of SH exceeds this tutorial
H. Löllmann: Microphone Array Design and Beamforming
11
Introduction Microphone Array Design Beamforming Conclusions
New Head Array Design for Nao Robot
H. Löllmann: Microphone Array Design and Beamforming
12
Possible regions for microphone
placement (green)
• determined by mechanical
constraints of the manufacturer
Positions considered for the
optimization
• 327 positions on a simulated head
Introduction Microphone Array Design Beamforming Conclusions
New Head Array Design for Nao Robot
H. Löllmann: Microphone Array Design and Beamforming
13
Simulation results (BG University) Layout for new Nao head (Aldebaran)
• slight deviations from optimal position due
to mechanical constraints
Introduction Microphone Array Design Beamforming Conclusions
New Head Array Design for Nao Robot
First prototype head
H. Löllmann: Microphone Array Design and Beamforming
14
Introduction Microphone Array Design Beamforming Conclusions
Beamformer Design for Robot Audition
Review: Filter-and-Sum Beamformer
Beampattern
H. Löllmann: Microphone Array Design and Beamforming
15
Introduction Microphone Array Design Beamforming Conclusions
Robust Least-Squares Frequency-Invariant (RLSFI) Design
[Mabande et al, 2009]
Robust Beamformer Design
H. Löllmann: Microphone Array Design and Beamforming
16
Introduction Microphone Array Design Beamforming Conclusions
Design Example
RLSFI beamformer design for different WNG thresholds
• beampattern for free-field response
• trade-off between spatial selectivity and robustness (WNG)
H. Löllmann: Microphone Array Design and Beamforming
17
Introduction Microphone Array Design Beamforming Conclusions
Design Example
RLSFI beamformer design for
• beampattern for HRTF-based response
• reduced spatial selectivity and distortions in look direction
HRTFs should be considered in the design!
H. Löllmann: Microphone Array Design and Beamforming
18
Introduction Microphone Array Design Beamforming Conclusions
HRTF-based RLSFI Design [Barfuss et al, 2015]
HRTF-Based Beamformer Design
H. Löllmann: Microphone Array Design and Beamforming
19
Introduction Microphone Array Design Beamforming Conclusions
HRTF-based RLSFI design for different WNG thresholds
• distortionless response in look direction
• similar spatial selectivity as for free-field design
Design Example
H. Löllmann: Microphone Array Design and Beamforming
20
HRTF-based design free-field design
Introduction Microphone Array Design Beamforming Conclusions
Experimental Evaluation
Setup
• ASR Pocket Sphinx trained on clean speech of GRID corpus
• test corpus contained 200 utterances
• signal quality evaluated by frequency weighted segmental SNR (fwSegSNR)
[Hu & Loizou, 2008]
• interference at 45o and desired speaker between 0o and 180o
• room impulse responses measured for a T60 of 190ms
H. Löllmann: Microphone Array Design and Beamforming
21
Introduction Microphone Array Design Beamforming Conclusions
Experimental Evaluation
Results
• scenario with speaker
at 30o and 60o most
challenging
• better performance for
HRTF-based design in
almost all cases
H. Löllmann: Microphone Array Design and Beamforming
22
Introduction Microphone Array Design Beamforming Conclusions
Conclusions
Beamformer design for robot audition requires tailored
algorithms and designs
Optimal microphone positions
• can be determined by the effective rank of GHRTF matrix
• can be found for a SH design by minimizing the aliasing level
Beamformer design based on common free-field assumption
leads to inferior results
• knowledge about the HRTF of the robot should be incorporated in the
design
H. Löllmann: Microphone Array Design and Beamforming
23
Introduction Microphone Array Design Beamforming Conclusions
Array Design for Spherical Harmonics (SHs)
How to find the best microphone positions for SHs
► Concept [Tourbabin et al., 2015]
H. Löllmann: Microphone Array Design and Beamforming
24
Appendix
Array Design for Spherical Harmonics
► Example for matrix
• aliasing level for sensor positions given by ratio of highest
off-line element and corresponding diagonal element
► Optimal microphone positions obtained by
H. Löllmann: Microphone Array Design and Beamforming
25
Appendix
Maximal possible WNG
• f or free-field (delay-and-sum beamformer)
• lower maximal WNG for HRTF-based design
White Noise Gain for HRTF-based Design
H. Löllmann: Microphone Array Design and Beamforming
26
Appendix
References
[Argentieri et al., 2013] S. Argentieri, A. Portello, M. Bernard, P. Danés, and B. Gas, “Binaural
Systems in Robotics,” in The Technology of Binaural Listening, J. Blauert, Ed., Modern Acoustics and
Signal Processing, pp.225–253, Springer
[Brandstein & Ward, 2001]: M. Brandstein and D. Ward (Eds.): Microphone Arrays, Springer, 2001
[Van Trees 2002]: H. L. Van Tress: Optimum Array Processing (Detection, Estimation and Modulation
Theory, Part IV), Wiley Intersience
[Benesty et al., 2008] J. Benesty, J. Chen, and Y. Huang: Microphone Array Signal Processing,
Springer
[Tourbabin & Rafaely, 2014] V. Tourbabin and B. Rafaely: “Theoretical Framework for the Optimization
of Microphone Array Configuration for Humanoid Robot Audition”, IEEE Trans. on Audio, Speech, and
Language Processing, vol. 22, no.12
[Mabande et al., 2009] E. Mabande, A. Schad, and W. Kellermann: “Design of Robust Superdirective
Beamformers as a Convex Optimization Problem”, IEEE Intl. Conf. on Acoustics, Speech, and Signal
Processing (ICASSP), Taipei, Taiwan
[Barfuss et al., 2015] H. Barfuss, C. Huemmer, G. Lamani, A. Schwarz, and W. Kellermann: “HRTF-
Based Robust Least-Squares Frequency-Invariant Beamforming”, Workshop on Applications of Signal
Processing to Audio and Acoustics (WASPAA), New Paltz, NY, USA
[Hu & Loizou, 2008] Y. Hu and P.C. Loizou: “Evaluation of Objective Quality Measures for Speech
Enhancement”, IEEE Trans. on Audio, Speech, and Language Processing, vol.16, no.1, pp.229-238
H. Löllmann: Microphone Array Design and Beamforming
27
Appendix
Recommended