The Bernard M. Gordon Center for Subsurface Sensing and Imaging Systems

Miguel Velez-Reyes,Thrust Leader

R2-C: Multi-spectral Discrimination

This work was supported by in part by the Gordon Center for Subsurface Sensing and Imaging Systems, under the Engineering Research Centers Program of the National Science Foundation (Award # EEC-9986821).

Multi-SpectralDiscrimination

(MSD)

Probe

Multi-BandDetectors

The MSD TEAMFaculty/Researchers

M. Velez-Reyes, UPRMS.D. Hunt, V. Manian, N. Santiago, UPRMJ. Goodman, U of Mia RSMAS & UPRMS. Rosario, UPRMB. Roysam, RPIM. Bystrom, BUM. Diem, NEU

MS (9)Andres Alarcon, UPRMCarolina Peña, UPRMLeidy P. Dorado, UPRMAndrea Santos, UPRMNicolas Rey, UPRMOsmarh Martinez, UPRMNestor Díaz, UPRMOrian Tzadik, UPRMKarin Griffis, BU

Ph.D (5)Miguel Goenaga, UPRMMaider Marín, UPRMMaria C. Torres, UPRMAmit Mukherjee, RPIEladio Rodriguez, BUTatiana Cherneko, NEU

UG (6)Suhaili Cardona, UPRMYahayra Gonzalez, UPRMJoralis Sanchez, UPRMChristine Cortes, UPRMYajaira Gonzalez, UPRMLuis Alvarado-Ortiz, UPRM

Spectral ImagingB. Saleh, Intro to SSI

Wavelengthsensitivedetection

Wavelengthsensitivedetection

object

MediumClutter

BroadbandProbe, P

BroadbandProbe, P

Imager-Spectrometer Configuration

BroadbandDetector

BroadbandDetector

object

MediumClutter

Probes at different wavelengths, Pi

Probes at different wavelengths, Piλ1 λ2 .. λn

Spectrometer-Imager Configuration

λ1 , λ

2 , …,λm

( ) ( )( )( ) ( )iiiii λ,wγ,S,λβα,Τλ,Y rrr +=

Sampling the Spectra

Spectral Physics-Based Signal Processing (R2C)

o Crop health o Chemical composition, pH, CO2o Metabolic information o Ion concentrationo Physiological changes (e.g., oxygenation)o Extrinsic markers (dyes, chemical tags)

Examples of β

Detect: presence of a target characterized by its spectral features α

or β

Classify: objects based on features exhibited in α

or β

Understand: object information, e.g., shape or other features based on α or β. Integrating spatial and spectral domains.

Or

Estimate: probed spectral signature {α (x,y,λ)}

physical parameter to be estimated {β(x,y,λ)}αβ

M

Challenge: Complexity of the media

High spectral resolution is needed to resolve the different components

From C.O.Davis, HSI of the Littoral Battle Space, NRL Code 7203

sc

Epidermis

Dermis

Re Rd(λ)

Venous Hb

Arterial Hb

R

100 μm

2 mm

Stratum Corneum

sc

Epidermis

Dermis

Re Rd(λ)

Venous Hb

Arterial Hb

R

100 μm

2 mm

Stratum Corneum

Benthic Habitat Monitoring Biomedical Imaging

Challenge: Small Signal from the Object of Interest

Reflected Bottom Radiance

Water Column Reflected Radiance

Reflected Bottom Radiance

Water Column Reflected Radiance

From NEMO OverviewNemo.nrl.navy.gov

What we Measure

What we want

• Optical Properties of the Water• Bottom Reflectance• Bathymetry

Challenge: Integration of Spectral and Spatial Information

Point-by-point spectroscopy still the major approach for HSI processing

Full HSI exploitation requires integration of spatial-spectral domain information

Pixel spatial coordinates randomly shuffled

Spectral processing Spectral processingSame final per- pixel analysis

results

Challenge: Data Overload Problem

1 Hyper-Spectral Image per sec

105 Gbytes per day

108 Books per day

Fast, Automated, (On-Board) Processing

Only ~2x107 Books inLibrary of Congress

From S. Adler, CenSSIS RICC 2004

MSD Research Across the Center

R2: MultispectralPhysics-Based Signal ProcessingFundamental

ScienceFundamentalScience

ValidatingTestBEDsValidatingTestBEDs

L1L1

L2L2

L3L3 S4

Bio -Med Enviro -Civil

R3: AlgorithmImplementation

Benthic HabitatMapping

R1: Multispectral Sensing

S1 Microscopy,Celular Imaging

R2C Research ProjectsComplexity of the Media

Hyperspectral Image UnmixingUnsupervised Methods using PMFSubsurface Unmixing for Benthic Habitat Mapping

Classification and DetectionSupport vector machinesCurve evolution methods

Small Signal from the targetSignal Enhancement

Denoising of Hyperspectral Imagery using Raman Spectroscopy

Subsurface Unmixing for Benthic Habitat Mapping

R2C Research ProjectsSpectral/Spatial Integration

Scale-Space Representation using Geometric PDEsDenoisingImage segmentation and registration

Vectorial TextureChange Detection

Data Management and ComputingToolboxes: HIAT, and HyCIATHyperspectral image processing using GPUs

Experimental WorkSeaBED – Testbed for coastal remote sensingVIS, MWIR, NIR cameras and spectrometer available on campusOther facilities available from other CenSSIS partners

Hyperspectral Image Registration (R2C.p3)

Image Registration

Reference Image

Sensed Image

Feature DetectionControl Points are

selected

Feature Matching Parameter ModelEstimation

Image Resampling

Transformed Image

SIFT Detector for Grayscale Images (Lowe 2002, 2004)

Threshold

-

-Generation of Scale Space

Gaussian SmoothingDifference-of-Gaussians

DoG

Local Maxima Pixel

Interest Points

Increasingof

Scales

rr

HDetHTr 22 )1(

)()( +

=

( )⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

∂∂

∂∂∂

∂∂∂

∂∂

=

2

22

2

2

2

yDoG

xyDoG

yxDoG

xDoG

yxDoGH ),(

Different scales

Increasingof

Scales

Statistical Decision

Original Image

Band by Band Approach (Mukherjee et.al. TGARS 2009)

Function for combining DoG responses along

spectral dimension

PCAprojection

Threshold

rr

HDetHTr 22 )1(

)()( +

=

Generation of Scale Spaceby Gaussian Smoothing

Local Maxima Pixel

Different scales

Interest Points

Comp. 1

Comp. 2

Comp. M

IncreasingScale

Comp. 1

Comp. M

Comp. 2

Difference-of-GaussiansDoG

( )⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

∂∂

∂∂∂

∂∂∂

∂∂

=

2

22

2

2

2

yDoG

xyDoG

yxDoG

xDoG

yxDoGH ),(

Increasing Scale

-

-

Comp. 1

Comp. 2

Comp. M

IncreasingScale

IncreasingScale

Original Image

New Approach: Multi-channel Approach

Scale Space Representation by Anisotropic Diffusion

Difference of adjacent scales

IncreasingScale

Local Maxima Pixel-vector Vector Ordering

Threshold

( )⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

∂∂

∂∂∂

∂∂∂

∂∂

=

∑∑

∑∑

==

==M

i

iM

i

i

M

i

iM

i

i

yD

xyD

yxD

xD

yxDII

12

2

1

21

2

12

2

),(

( )( ) r

rIIDet

IITr 22 )1( +=Interest Points

Scale t+1

Scale t

Scale t-1

Band Subset Selection

IncreasingScale

-Original Image

Experimental Results

4380 Interest PointsAnisotropic DiffusionVector Ordering andSecond Fundamental Form

2506 Interest PointsMukherjee’s Approach (Gaussian Smoothing and Non-linear function)

Benchmarking Unmixing using cPMF (R2C.p4)

Experimental comparison of cPMF with standard unmixing algorithms that retrieve endmembers from the image pixels using AISA hyperspectral images collected over Vieques Island in Puerto Rico.

Obtain endmembers by

cPMF SMACC Max D

ABUNDANCE ESTIMATION

EndmembersExtraction

EXPERIMENTS AND RESULTSEXPERIMENTS AND RESULTS

Red Mangrove

Bare soil/ dirt roads

High trees (Flamboyan tree)

Black mangrove and palm

Water

Shrubs (zarcillo)

EXPERIMENTS AND RESULTS: WATER ENDMEMBERS

Water

Water (Max D)

0

0.5

1

Water (SMACC)

00.20.40.60.8

Water (cPMF)

0

0.5

1

500 550 600 650 700 750 800 850 9000

100

200

300

400

500

600

700

800

900

1000Spectral Signatures Comparison of Water

Wavelength [Nanometers]

Am

plitu

de

Water (cPMF)Water (SMACC)Water (Max D)

Hyperspectral Image Processing Solutionware

Supervised Classification Module

Unsupervised Classification Module

MATLAB Toolbox Parallel and Distributed Computing

Hardware Implementationin FPGA/DSP and GPUs

New: Hyperspectral Coastal Image Analysis Toolbox (HyCIAT)

Collection of methods for analysis of hyperspectralimages over coastal environments.

Collection of functions that extend the capability of the MATLAB® numeric computing environment.Designed in Macintosh and PC-Windows Systems.

Based in algorithms developed by Dr. James Goodman and from research done in Laboratory of Applied Remote Sensing and Image Processing (LARSIP).

HyCIAT: Processing Scheme

HyCIAT: Sample Results

Abundances and Fractional Maps

1. Low-dimensional representations preserve distance throughout manifold.

2. Only local information is required to produce low-dimensional representation.

3. We can infer other interpoint distances.

1. High-computational cost is required.

2. For Hyperspectral Images, Manifold learning algorithms do not take in consideration spatial information.

Manifold Learning for HSI Exploitation

Manifold Learning using GPU's

Algorithm TaskComputational Complexities

CPU/GPU processing

Isomap

Find k-NN per pixel O(n2) GPUCompute approximate geodesic distance O(n2log(n)) CPUFind optimal t dimensional representation O(n3) GPU

Laplacian

Eigenmaps

Find k-NN per pixel O(n2) GPUConstruct diagonal matrix D O(kn) CPUFind the t dimensional representation O(n3) GPU

Locally Linear Embedding

Find k-NN per pixel O(n2) GPUCalculate matrix C O(kn) GPUCalculate matrix M O(n3) GPU

Cuprite scenario

Execution time as a function of image size

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0 20 40 60 80 100 120

x103

x103Number of pixels

Exec

utio

n tim

e (s

)

CPUGPU

Execution time as a function of k in the k-NN search

0

1

2

3

4

5

6

5 10 15 20 50

k

GPU (40000 pixels)CPU(40000 pixels)GPU (60000 pixels)CPU (60000 pixels)

MSD PostersR2C

R2C p3

Leidy Paola Dorado-Munoz / UPRM Amit Mukherjee / RPI

Miguel Velez-Reyes / UPRM Badrinath Roysam / RPI

"Detection of Interest Point for Multispectral and Hyperspectral Images Using Lowe’s Approach and Anisotopic Diffusion"

R2C p4 Andrea Santos- Garcia / UPRM

Miguel Velez-Reyes / UPRM Samuel Rosario / UPRM Jesus D. Chinea / UPRM

"A Comparison of Unmixing Algorithms for Hyperspectral Imagery"

R2C p5 Carolina Pena Ortega / UPRM

Miguel Velez Reyes / UPRM

"Comparison of Basis-Vector Selection Methods for Target Detection"

R2C p6 Nestor J. Diaz G. / UPRM Vidya Manian / UPRM "Hyperspectral Texture Synthesis by

Multiresolution Pyramid Decomposition"

R2C p7 Karin Griffis / BU Maja Bystrom / BU"A Tunable, Multi-scale, Multi-band Segmentation Procedure for Remotely-Sensed Imagery"

R2C p8 Eladio Rodriguez- Diaz / BU

David A. Castanon / BU Irving J. Bigio / BU

"Pattern Recognition Methods for Spectral Classification in ESS Diagnosis of Cancer"

Related PostersR3

Validating TestBEDsSea p2 Carlos J. Solis Ramirez

/ UPRMRaul E. Torres / UPRM

"Hyperspectral Image Registration and Fusion for Underwater Applications"

Sea p3 Carlos J. Solis Ramirez / UPRM

Raul E. Torres / UPRM

"Modification of the SeaBed Autonomous Underwater Vehicle for Hyperspectral Image Acquisition"

R3A p6 Yajaira Gonzalez Gonzalez / UPRM

Nayda G. Santiago / UPRM

"Analyzing the Use of GPUs for Hyperspectral Image Processing"

R3B and Demo p1

Maria Constanza Torres Madronero / UPRM

Miguel Velez Reyes / UPRM

"HyCIAT: Hyperspectral Coastal Image Analysis Toolbox"

R3B and Demo p2

Samuel Rosario / UPRM Miguel Velez Reyes / UPRM

"Speeding Up the Hyperspectral Image Analysis Toolbox"

Questions?

Suggestions, Advice, …