Multispectral Format from Perspective of Remote Sensing Rulon E. Simmons rulon.simmons@kodak.com...

Multispectral Formatfrom Perspective of

Remote Sensing

Rulon E. Simmons

rulon.simmons@kodak.com

(585) 253-5382

Outline

• Color Issues– Steps to color processing– Color Accuracy in Remote Sensing– Handling color display of non-visible bands

• Format issues– Similarities and differences with other

multispectral applications– XML for Metadata– Compression

Color Issues

Contributing Persons:

Rulon Simmons

Tim Elder

Scott Bennett

Michael Vaughn

Color Processing Approach*

*Note: This is a standard approach used with digital photography.

Display

OutputPreparationRendering

ColorCorrection

Pre-Processing

DRAChannel Balance

RGB’ XYZ XYZ LABTTCMTFCLAB XYZ’

XYZ’ RGB”Gamma

RGB raw

Pre-Processing

Black Point

Calibration Point

Pre-Processing (Calibration from Digital Counts to Reflectance)

• Empirical Line Method*– Using one or more calibration patches– Using best guess of reflectance of one or more

objects

• Atmospheric modeling (e.g., MODTRAN)

*Note: The ELM method of calibration is applicable to any discipline, not just Remote Sensing.

Color Correction(Causes of Color Inaccuracies in Remote Sensing)

• Multispectral band passes in the visible region not equivalent to HVS response functions – IKONOS, QuickBird

• Skylight illumination– Common phenomenon: Materials in

shadow appear bluish

• Dynamic Range Adjustment (DRA)– Varying approaches produce different

color presentations

Color Correction(Problem 1: Different Spectral Sensitivities)

E

E E

II I

I

E = Eye SensitivityI = IKONOS Sensitivity

Color Correction(Ikonos Vs. HVS Color Example)

IKONOS Image Simulated HVS Image

Color Correction (GretagMacbeth Color Checker: Truth & IKONOS Simulation)

Color Correction(Eckerd’s Blue Roof Turns Purple)

HVS IKONOS QuickBird

Color Correction(Image Example and Simulation of IKONOS Color)

Ground Truth:Digital Camera

Simulated IKONOS Color

Color Correction(Problem 2: Shadow Illumination)

• White reference chip in direct sunlight– Used to calibrate the ASD

spectrometer

• White reference chip in shadow– Illuminated by light scattered

by the atmosphere

Rendering

• Convert from XYZ to LAB (to get to a color appearance space)• Apply a Tone Transfer Curve (TTC) to the “L” channel to stretch

midtones while compressing highlights and shadows

• Apply Modulation Transfer Function Compensation to the “L” channel to sharpen image

• Convert back to XYZ

Tone TransferCurve

Luminance (in)

Lum

inan

ce (

out)

Output Preparation

Phosphor Gamut will allow reproduction of most colors seen by IKONOS, but many colors seen by the eye cannot be reproduced by IKONOS.

SO HOW CAN YOU DO COLOR CORRECTION?

Color Correction Approach

• There is no perfect solution, however, all of the problems are addressed by creating a transform that minimizes the Delta E between a set of color patches as seen by the eye and by the sensor

– Use real patches were possible

– Use synthesized data based on system specs

Color Correction(Beyond Visible)

• Near IR and other non-visual spectral regions can be handled in the same way as shown on previous slide. (Note: UV is rarely used in Remote Sensing because it is heavily absorbed by the atmosphere.)

• The transformation seeks to minimize the Delta E between an acquired or synthetically generated image of a set reference panels and some aim colors

• For IKONOS, a “false-color IR” image is produced by mapping the IR channel to the red display, red to the green display, and green to the blue display

Color Correction (for “hyper”-visible imagery)

• Currently in Remote Sensing there is no standard method of mapping color for systems that have more than three spectral channels covering the visible range of the spectrum– Current visualization packages just display

three bands (either user selected or chosen by the program)

– A useful option would be to smartly combine the bands to make an image that is as close as possible to what the human visual system would see

Format Issues

Contributing Persons:

Rulon Simmons

Bernard Brower

Similarities and Differences between Remote Sensing and other

Applications of Multispectral Imaging

TOPIC Textiles Paints Digital Cameras Remote Sensing

Need to Transmit Data Yes Yes Yes YesNeed to Store Data Yes Yes Yes YesNeed for Color Accuracy Yes Yes Yes YesNeed to Compress Data No No Yes YesNeed to Sharpen Data No No Yes YesNeed for Geolocation Data No No No YesNeed for Format Standardization Yes Yes Yes YesCurrent Standards TIFF TIFF TIFF GeoTIFF

JPEG NITF / BIIF (NATO)HDF5

Future Standards J2K J2K J2K J2K

Reconciling Differences• Consider requirements for all disciplines to be a

subset of requirements for Remote Sensing

– Note 1: This approach does not require that all Remote Sensing issues be addressed in the first release of the standard as long as it is extensible at a later date

– Note 2: A basic set of Remote Sensing requirements may not be too difficult to accommodate

• Build upon a standard that will be acceptable to all parties in the future

• Standardize an approach to recording metadata that can be used within any file format

Standard Approach to Metadata (Why Use XML?)

• Standard method of coding metadata

• Can be used regardless of format

• Is human readable as well as machine readable

XML for Pre-Processing

<PRE-PROCESSING>

<DRA>

<PARAM NAME=“GAIN” VALUE=“1.5/>

<PARAM NAME=“OFFSET” VALUE=“10”/>

</DRA>

</PRE-PROCESSING>

XML for Size

<SIZE>

<PARAM NAME=“NBANDS” VALUE=“4”/>

<PARAM NAME=“NROWS” VALUE=“1000”/>

<PARAM NAME=“NCOLUMNS” VALUE=“1000”/>

<PARAM NAME=“NBITS” VALUE=“12”/>

<PARAM NAME=“SEQUENCE” VALUE=“BIP”/>

</SIZE>

XML for Location

<LOCATION>

<PARAM NAME=“COORD PROJECTION” VALUE=“UTM”/>

<PARAM NAME=“LAT” VALUE=“XX.XX”/>

<PARAM NAME=“LONG” VALUE=“YY.YY”/>

</LOCATION>

XML for Color Correction

<COLOR MATRIX>

<PARAM NAME=“RGB TO XYZ” VALUE=“X1, X2, X3, X4, X5, X6, X7, X8, X9”/>

</COLOR MATRIX>

XML for Rendering

<RENDERING>

<PARAM NAME=“XYZ TO LAB” VALUE=“X1, X2, X3, X4, X5, X6, X7, X8, X9”/>

<PARAM NAME=“TTC” VALUE=“X1, Y1, X2, Y2, X3, Y3, …”/>

<PARAM NAME=“MTFC” VALUE=“X1, X2, X3, X4, X5, X6, X7, X8, X9”/>

<PARAM NAME=“LAB TO XYZ” VALUE=“X1, X2, X3, X4, X5, X6, X7, X8, X9”/>

</RENDERING>

XML for Display(In general would not be included in metadata.)

<DISPLAY>

<COLOR MATRIX

<PARAM NAME=“XYZ TO RGB” VALUE=“X1, X2, X3, X4, X5, X6, X7, X8, X9”/>

<PARAM NAME=“LAB TO RGB” VALUE=“X1, X2, X3, X4, X5, X6, X7, X8, X9”/>

</COLOR MATRIX>

<PARAM NAME=“GAMMA” VALUE=“2”/>

</DISPLAY>

Compression

• Compression is required for some, but not all multispectral applications

• JPEG 2000 is an emerging international standard that is gaining wide acceptance (can be used in lossy or lossless mode)

• JPEG 2000 can be used to stream very large data sets in real time to small computing devices such as PDAs (using the feature that the whole image does not have to be decompressed before it can be sent)

• JPEG 2000 can be accommodated within the current NITF/BIIF standard, making it attractive to many currently doing Remote Sensing

Conclusions / Recommendations• Use a four-step approach to color management currently used with digital

photography

• Non-visible channels can be displayed in predictable ways using standard color science

• “Hyper”-visible channels can be combined to give a better representation of true color (from HVS perspective)

• Multispectral format requirements for Remote Sensing are more similar to than different from other applications

• Select a standard that will be acceptable to all disciplines, taking into account widely-used standard formats and one that will likely be used in the future.

• Use XML as a standard for recording metadata