VIPER Tools User Manual v1.5

VIPER Tools User Manual

Version 1.5

11-12-2007

Dar Roberts Kerry Halligan Philip Dennison

This work is licensed under the Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License. To view a copy of this license, visit

http://creativecommons.org/licenses/by-nc-sa/3.0/ or send a letter to Creative Commons, 171 Second Street, Suite 300, San Francisco, California, 94105, USA.

VIPER Tools User Manual Version 1.5 Licensed under the Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License

2

Table of contents INTRODUCTION ............................................................................................................ 4

INTRODUCING VIPER TOOLS ..................................................................................................................... 4 LICENSE...................................................................................................................................................... 4 DEVELOPMENT STATUS.............................................................................................................................. 5 INTENDED USE ........................................................................................................................................... 5 FILE FORMATS AND DATA REQUIREMENTS ................................................................................................ 5 UNITS USED IN VIPER TOOLS.................................................................................................................... 6

INSTALLATION.............................................................................................................. 8 INSTALLATION STEPS ................................................................................................................................. 8

Windows installer executable................................................................................................................ 8 VIPER Tools Binary files .................................................................................................................... 10

ENVI RUNTIME........................................................................................................................................ 10 INSTALLATION TROUBLESHOOTING.......................................................................................................... 11 RUNNING VIPER TOOLS .......................................................................................................................... 12 UNINSTALLING VIPER TOOLS ................................................................................................................. 12

BACKGROUND AND ORGANIZATION .................................................................. 14 INTRODUCTION TO MESMA .................................................................................................................... 14 VIPER TOOLS COMPONENTS ................................................................................................................... 15 NAMING CONVENTIONS............................................................................................................................ 16 THE VIPER TOOLS MENU........................................................................................................................ 17

SPECTRAL LIBRARY TOOLS................................................................................... 19 BEFORE YOU START.................................................................................................................................. 19 CREATE SPECTRAL LIBRARY FROM ROIS ................................................................................................ 19 CREATE METADATA FOR SPECTRAL LIBRARY.......................................................................................... 21 VIEW SPECTRAL LIBRARY........................................................................................................................ 24 MANAGE SPECTRAL LIBRARIES................................................................................................................ 26 CONVERT LIBRARY TO IMAGE.................................................................................................................. 31

OPTIMIZATION TOOLS............................................................................................. 32 CREATE SQUARE ARRAY.......................................................................................................................... 32

Square Array Output Bands................................................................................................................ 35 EAR/MASA/COB (EMC)........................................................................................................................ 37

Calculating EAR, MASA and CoB. ..................................................................................................... 39 VIEW EMC FILE....................................................................................................................................... 41

Selecting optimal spectra .................................................................................................................... 44 CONSTRAINED REFERENCE ENDMEMBER SELECTION (CRES) ................................................................. 47

Calculating CRES:.............................................................................................................................. 47 Examples of CRES .............................................................................................................................. 49

MIXING TOOLS............................................................................................................ 53 RUN SMA/MESMA................................................................................................................................. 53

Running Simple SMA with no fraction constraints ............................................................................. 58 Running MESMA for a two endmember model................................................................................... 60 Running MESMA for a three endmember model ................................................................................ 63

SHADE NORMALIZE SMA RESULTS ......................................................................................................... 69 TERRAIN CORRECTION OF SMA RESULTS (*BETA*) .............................................................................. 71

Running Terrain Correction on SMA Fraction Image........................................................................ 72 REFERENCES................................................................................................................ 74


3

EXAMPLE APPLICATIONS ....................................................................................... 77


4

Introduction Introducing VIPER Tools The VIPER Tools software package is an ENVI add-on that provides a suite of processing tools for multiband passive optical remote sensing data including hyperspectral and multispectral imagery. The major components of the software are tools for creating and managing spectral libraries, for selection of optimal endmembers for Spectral Mixture Analysis (SMA) and for calculating and interpreting SMA and Multiple Endmember Spectral Mixture Analysis (MESMA). Our goal in developing VIPER Tools has been to provide no-cost tools that will enable other users to realize the full potential of MESMA. As part of this package we include detailed instructions for installation and use of VIPER Tools. We also include a detailed User Manual, designed to help users learn how to use VIPER Tools. Examples are drawn from Airborne Visible Infrared Imaging Spectrometry (AVIRIS) data acquired over the Santa Barbara front range in June 2001. These data, and associated reference polygons and metadata have been described in several refereed publications including Roberts et al., (2003), Dennison and Roberts (2003a,b), Dennison et al., (2004) and Ustin et al., (2004). The User Manual is organized into 7 main parts:

1. Installation 2. Background and Organization 3. Spectral Library Tools 4. Optimization Tools 5. Mixing Tools 6. References 7. Example Applications (including examples drawn from areas outside of the Santa

Barbara front range). This user manual focuses on the use of VIPER Tools under the Windows operating system. As of Version 1.5, VIPER Tools may be compatible with ENVI versions on Linux, Unix, or Mac but this has not yet been tested. Installation from binary files (rather than Windows setup executable file) is covered in the Installation section to facilitate installation on non Windows systems, but all other documentation found in this user manual assumes that VIPER Tools is running under Windows.

License Starting at version 1.5, VIPER Tools is licensed under the GNU General Public License GPLv3.0 http://www.gnu.org/licenses/gpl-3.0.html. This means that VIPER Tools is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software


5

Foundation, either version 3 of the License, or (at your option) any later version. VIPER Tools is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

Development Status VIPER Tools operates as an ENVI add-on only, and can not be used as a stand alone IDL, IDL runtime, or IDL Virtual Machine toolset. VIPER Tools was developed using IDL version 6.1 and ENVI version 4.1 and may not be backward compatible with older versions of IDL and/or ENVI. It has been tested and found to be forward compatible with IDL versions 6.2 and 6.3, ENVI 4.2 and 4.3, and with ENVI RT. VIPER Tools has only been tested with Windows 2000 and WindowsXP. VIPER Tools may be compatible with ENVI on Unix, Linux and Mac operating systems but this has not been tested. With the Version 1.5 release of VIPER Tools as open source software we are currently looking for developers interested in porting VIPER Tools into other environments. Please contact [email protected] if you are interested in helping advance the use of VIPER Tools beyond the ENVI user community.

Intended Use VIPER Tools is intended for use in conjunction with ENVI’s built-in capabilities, which are required for visualization, data preparation (including spectral and spatial sub-setting) and assessment of resulting data products.

File Formats and Data Requirements VIPER Tools requires multi-band remote sensing image files in the ENVI file format and spectral libraries in the ENVI spectral library format. While geospatial information and band wavelengths are not required, these parameters will be inherited in all data products produced by VIPER Tools if they are present in the source files. VIPER Tools can be used to process optical data in DN, at-sensor radiance or apparent surface reflectance. Data may be scaled using one of the following scale factors 1, 1000, 10000 and may be stored as byte, integer or floating point data in Band Interleave by Pixel (BIP), Band Interleave by Line (BIL) or Band Sequential (BSQ) format. Two new data formats are introduced as a part of VIPER Tools: the metadata file and the square array. The metadata file is an ASCII Comma Separated Value (CSV) file that is associated with either a spectral library or square array. The metadata file has a single row for each spectrum and multiple columns containing metadata. The first column


6

contains the name for each spectrum, in the exact same order in which the spectra appear in the spectral library. Subsequent columns include user generated metadata. At a minimum, these metadata should include at least one parameter that can be used to group spectra into categories. Commonly this will be a class name (for example, “grass”) and possibly a more general category (for example, “vegetation”). The square array consists of a Band Sequential image that stores all of the information needed to calculate the three endmember selection metrics used in VIPER Tools: EAR, MASA and CoB (these metrics are explained in the Background section). The first band of the square array file stores the Root Mean Square Error (RMSE) of modeled library spectra. The second band stores the spectral angle of modeled spectra. The next two bands record the fractions for the bright (3) and dark (4) endmembers. The last band records the constraint type used, including 0 (no constraint), 1 if the model was partially constrained (exceeded fractions but within RMSE constraint) and a value of 2 if it was fully constrained (exceeded RMSE constraint). Constraints are explained in the Optimization Tools section. ASCII Comma Separated Value (CSV) files are used throughout VIPER Tools to store spectral library metadata, image processing metadata, spectral fit metrics (e.g. EAR, MASA, CoB values), and control files. The CSV format was selected because it is simple, easy to write and read and is human readable. Additionally, on most Windows systems where Microsoft Office software is installed, Excel is the default file reader and automatically parses out the commas to provide quick and easy access to the information in the file.

Units Used in VIPER Tools In an attempt to be consistent for all of the modules in VIPER Tools the following conventions are used for both input and output units. Reflectance Scale Factor: Reflectance Scale Factor is the same as a gain. It is usually used to maximize the dynamic range of a dataset while storing the data in integer format (rather than floating point) to reduce file size. It is divided into a spectral value to create a standardized value for Viper Tools. The only currently allowed scale factors are 1, 1000 and 10000. Thus, if 1000=100% reflectance, a user would specify a gain factor of 1000. In this example, Viper Tools will divide each spectral value by 1000 to ensure that 100% = 1.0. If data are scaled using an alternate scale factor, first convert it to one of these supported scale factors using tools in ENVI. RMSE: Units are the native units of the data: reflectance, DN or radiance. This is regardless of the scale factor, as the scale factor is used when calculating RMSE. For example, output RMSE value of 0.025 for an image containing scaled reflectance with a scale factor of 10,000 means that the RMSE is 2.5% reflectance. The same value from an image with a scale factor of 1 means 2.5% reflectance as well. For a DN image an output RMSE of 2.0 means 2 DNs.


7

SMA Fractions: Units used are fractions. For inputs, such as fraction constraints for SMA or for building a square array, floating point fractions with 2 significant digits are used (e.g., 1.06). The range of allowable fractions for thresholds is from -0.50 to 1.50. For outputs, SMA fractions are always listed as their fractional value in floating point format. Spectral Angle: Units are radians Residuals: Floating point, 1.0 scale factor


8

Installation

Installation Steps VIPER Tools is distributed in 3 different formats: 1) as a Windows installer executable, 2) as IDL binary (*.sav) files, and 3) as IDL source code. The sections below describe the installation using the Windows installer and from the IDL binary files.

Windows installer executable The windows installer executable file enables easy installation of all VIPER Tools software components. It does not install the demonstration data or user manual. The following steps should be followed to obtain and install VIPER Tools. Note the special instructions under ‘ENVI Runtime’ for ENVI Runtime installations of VIPER Tools. Step 1 Download copy of VIPER Tools executable from the VIPER Tools website (www.vipertools.org) by clicking on the ‘Download’ menu item then clicking on the ‘Download VIPER Tools Installer’. Save the file to your desktop or other temporary location. Make sure to download the user manual from this page as well. The user manual contains additional details on installation and use of VIPER Tools. If upgrading VIPER Tools from a previous version you should uninstall the previous version by following the steps below in ‘Uninstalling VIPER Tools’. This is critical if upgrading from a ‘beta’ version to Version 1.4 or higher. Step 2 Double click on the setup file to begin the setup and the following window will appear:


9

Click ‘Next’ to continue the process: Step 3 Select the appropriate installation folder. Unlike some applications, the VIPER Tools add-on must be installed in the correct folder in order for ENVI to locate it. This is because ENVI will need to compile the VIPER Tools components when it launches so that VIPER Tools is available, and ENVI only searches folders in its installation path, including the ‘save_add’ directory, for IDL files to compile. The screen below will appear to inform the user to select the ENVI ‘save_add’ folder.


10

Read this screen and make certain you know where this folder is then click ‘Next’ to continue before continuing. Note: on systems with multiple hard drives this may not be the ‘C:’ drive.

Click the ‘Browse’ tab to map the location of the ‘save add’ folder in the RSI directory. Note that the installer will add ‘VIPER’ to the selected ‘save_add’ folder. This new folder will be created and all VIPER Tools components will be installed to this ‘VIPER Tools’ folder. If a previous version of VIPER Tools has been installed, the VIPER folder may already exist. Take care not to create a new, nested folder by ensuring that the folder to which VIPER Tools will be installed does NOT look like “…\save_add\VIPER\VIPER”. Click ‘Next’ to continue the install process. If using ENVI Runtime, following the directions in the ENVI Runtime section below to complete the installation.

VIPER Tools Binary files For manual installation, or installation on a non-Windows platform, download the binary distribution (e.g., vipertools_binary_v1.5.zip) from www.vipertools.org. Within the distribution path for ENVI, locate the ‘save_add’ folder. Make a subdirectory named ‘VIPER’ in this folder. Unzip the archive file using the appropriate software and save the two IDL binary (*.sav) files to the newly created ‘VIPER’ folder. If using ENVI Runtime, following the directions in the ENVI Runtime section below to complete the installation.

ENVI Runtime


11

ENVI Runtime users must manually edit their ENVI menu file to add a menu button for launching VIPER Tools. To do this open up the ENVI menu file (e.g., C:\RSI\IDL42\products\ENVI62\menu\envi.men) and insert the following line at the end of the 'Spectral' section in the menu: 1 {VIPER Tools} {viper_event} {viper} {separator} This new line should be added just above the line that reads '0 {Map}'. Save the envi.men file then re-start ENVI. The pulldown menu under the 'Spectral' heading should now look like this:

Installation Troubleshooting If installation fails for any reason, uninstall before reinstalling. To uninstall use the windows ‘Add or Remove Programs’ tool (Start>Control Panel>Add or Remove Programs). See Uninstalling VIPER Tools section below for more details. This will


12

remove the VIPER folder and all files associated with VIPER Tools. If a license file is present in the VIPER folder, the folder and the license file should be retained.

Running VIPER Tools To begin using VIPER Tools, start ENVI (if it is already open, quit and restart). All VIPER Tools routines are accessed from the main VIPER Tools menu, which is launched by clicking ‘VIPER Tools’ at the bottom of the ‘Spectral’ main on the main ENVI menu (see image above).

Uninstalling VIPER Tools To uninstall VIPER tools use the Windows Add/Remove Programs utility (Start>Control Panel>Add or Remove Programs). In the Add/Remove Programs utility, scroll down to VIPER Tools entry and click the ‘Remove’ button.

When prompted click ‘Yes’ to verify that you wish to remove VIPER Tools

You will be notified when uninstallation is complete. This indicates that the VIPER Tools software and directories have been removed.


13

If you are uninstalling a ‘beta’ version of VIPER Tools confirm the removal of the VIPER Tools menu item from the ENVI menu bar by clicking ‘Yes’ on the screen below.

Note that if you are uninstalling version 1.1 or higher for ENVI Runtime and you manually edited the ENVI menu file (envi.men) the uninstall routine will not remove this entry from the menu file. You will need to remove this entry manually. If uninstalling version 1.4 or lower and a VIPER Tools license file was present in the ‘VIPER’ folder, both the folder and the license file may be retained. To complete the uninstall process simply delete these items.


14

Background and Organization Introduction to MESMA A spectral mixing technique called Multiple Endmember Spectral Mixture Analysis (MESMA) is at the core of VIPER Tools. MESMA is an extension of simple Spectral Mixture Analysis (SMA). In simple SMA, a spectrum is modeled as the sum of “pure spectra” called endmembers, each weighted by the fraction of an endmember required to produce the mixture (Adams et al., 1993; Roberts et al., 1993; Settle and Drake, 1993). SMA has an advantage over many competing approaches in that it provides physically meaningful measures of cover that account for subpixel mixing. While SMA is a powerful approach, it fails to account for pixel-scale variability in spectral dimensionality, spectral degeneracy between materials and natural variation in the spectra of most materials. As a result, simple SMA underutilizes the potential of most remote sensing data sets for discriminating materials, while at the same time producing fractional errors due to the incorrect type or number of endmembers used to unmix a specific pixel. MESMA extends SMA by allowing the number and types of endmembers to vary on a per-pixel basis (Roberts et al., 1998). MESMA overcomes limitations of SMA by requiring a model to meet minimum fit, fraction and residual constraints while testing multiple models for each image pixel. MESMA is typically implemented by developing a spectral library, then unmixing an image using every possible combination of two, three and four endmembers applied to each pixel. Using this approach, significantly more than four materials can be mapped across an image, while minimizing pixel-scale fraction errors by selecting the best-fit model for each pixel. Applications of MESMA include snow-cover/snow-area mapping (Painter et al., 1998; 2003), plant species mapping in chaparral (Roberts et al, 1998; Dennison and Roberts 2003a/2003b, Roberts et al., 2003), soil mapping in arid lands (Okin et al., 2001), improved estimation of LAI in boreal forests (Sonnetag et al., 2007), landform mapping in North Africa using MODIS (Ballantine et al., 2005), fire temperature mapping (Dennison et al., 2006), urban remote sensing (Rashed et al., 2003; Powell et al., 2007), shape-based temporal compositing (Dennison et al., 2007) and even several planetary applications (Li and Mustard, 2003; Johnson et al., 2006). Quantitative comparisons of reference fractions to estimates using SMA or MESMA have proven to be highly accurate over a diverse number of surfaces (e.g. Elmore et al., 2000; Painter et al., 2003; Powell et al., 2007). Endmembers used in SMA or MESMA can be derived from an image (image endmembers), or from a library of known materials (reference endmembers). Reference endmembers can be derived from the field, laboratory, images or even radiative transfer models. Typical endmembers used in SMA include soil, Green vegetation (GV), Non-photosynthetic Vegetation (NPV) and shade. Identifying a high quality set of reference or image endmembers has been identified as a critical stage of mixture modeling (Tompkins et al., 1997). A number of approaches have been developed for identifying endmembers including the Pixel Purity Index (PPI: Boardman et al., 1995), Manual selection (Bateson


15

and Curtiss, 1996), Endmember Bundles (Bateson et al., 2000), Constrained Reference Endmember Selection (CRES: Roberts et al., 1993; 1998), Count-based Endmember Selection (COB: Roberts et al., 2003), Endmember Average RMSE (EAR: Dennison and Roberts, 2003a) and Minimum Average Spectral Angle (MASA: Dennison et al., 2004). Four techniques used for identifying optimal endmembers are included in VIPER Tools: CoB, EAR, MASA and CRES. While MESMA is a powerful approach, several factors have limited its use. One of the major limitations has been a lack of easily used software. One of the key goals of VIPER Tools is to address this lack of software. Another limitation has been the challenges in selecting endmembers to be used in MESMA. In its earliest implementations, MESMA was typically employed using hundreds of endmembers, resulting in significant challenges in interpretation and computation. More recently, efforts have been made to fine tune the analysis by selecting optimal sets of endmembers from a spectral library prior to running MESMA. A key goal is to identify those spectra that are most representative of a specific class, yet also least likely to be confused with spectra from a different class. To facilitate the development of optimal spectral libraries, several techniques have been developed including:

1) Count-based Endmember Selection (CoB): Using this approach, endmembers are selected that model the greatest number of endmembers within their class (Roberts et al., 2003). As a variation on this approach, a Count-Based Index can be used to rank endmember selection based on maximizing the models selected within the correct class, while minimizing confusion with other classes (Clark, 2005).

2) Endmember Average RMSE (EAR): Using this approach, endmembers are selected that produce the lowest RMSE within a class (Dennison and Roberts, 2003).

3) Minimum Average Spectral Angle (MASA): Using this approach, endmembers are selected that have the lowest average spectral angle (Dennison et al., 2004).

A promising alternative to MESMA, called Automatic Monte Carlo Unmixing is described by Asner and Lobell (2000) and Asner et al., (2003). Using this approach, rather than select a single set of endmembers or predefined models for a pixel, each pixel is unmixed using sets of endmembers randomly selected from “bundles” of similar materials to generate a mean and standard deviation for each fraction. Automatic Monte Carlo Unmixing, while similar to MESMA in principle, has an advantage in that it does not depend on the selection of a single set of endmembers and is readily automated. However, the fractions derived using this approach will be highly dependent on how representative library spectra are of a region and cannot account for species-specific spectral differences as MESMA does.

VIPER Tools Components


16

VIPER Tools includes all of the tools needed to implement MESMA including spectral library development, optimum endmember selection and spectral mixture modeling. The current version of VIPER Tools includes 12 main processing options: Spectral Library Tools Create Spectral Library from ROIs Create Metadata for Spectral Library View Spectral Library (*BETA*) Manage Spectral Libraries Convert Library to Image (*BETA*) Optimization Tools Create Square Array Calculate EAR/MASA/CoB (EMC) View EMC File CRES Mixing Tools Run SMA/MESMA Shade Normalize SMA Results Terrain Correct SMA Results

Naming Conventions As you process data, starting from a spectral library, then following through optimization and mixture models, you are likely to generate a fair number of data files and supporting metadata. While users are free to select their own naming conventions, the following list includes conventions we have devised for our work. VIPER Tool names consist of two components, the base name and suffix (commonly called ‘extension’ on Windows systems). The base name consists of everything that comes before the ‘.’ prior to the suffix. Underscores should be used in place of spaces for increased compatibility with non-Windows systems. Additionally, never use more than one dot (‘.’) in file names, as VIPER Tools parses out filenames using the dot to separate the base and extension names. Naming conventions are extremely important because they help users keep track of where they are in the processing stream, and to relocate appropriate files, especially metadata files, at a later date.


17

Table listing Naming Conventions used in the Tutorial File Type Ending Example from Manual Spectral library *.sli viper_tutorial_train.sli Metadata file *.csv viper_tutorial_train.csv (base name must match the spectral library) Sorted spectral library *_sort viper_tutorial_train_sort.sli Square array *_sqr (no extension) viper_tutorial_train_sort_sqr Ear Masa Cob file (EMC) *_emc.csv viper_tutorial_train_sort_dom_emc.csv Sma or mesma image *_sma (no extension) viper_tutorial_simple_sma Shade normalized images *_sn (no extension) viper_tutorial_simple_sn Terrain corrected image *_tercor (no extension) viper_tutorial_simple_tercor Processing notes: We cannot emphasize enough that you use _ in place of the decimal point in naming conventions. ENVI does not handle the period well, and will often attempt to overwrite a files header if there are two files with the same name, but a different extension after the period. Judicious use of an underscore will avoid many problems and lost headers.

The VIPER Tools Menu After installing the license you should be ready to run VIPER Tools. Start ENVI, click the ‘VIPER Tools’ menu item under the ‘Spectral’ menu. The VIPER Tools Menu should come up with all of the functions now in dark text and available for use:


18

VIPER Tools can be used on a wide variety of data types for a diversity of applications. At its simplest level, VIPER Tools can be used to implement simple or multiple endmember models using existing ENVI spectral libraries. Used in this way, only the Run SMA/MESMA and perhaps the Shade Normalize or Terrain Correct SMA Results options would be used. In its most complete application, VIPER Tools can be used to build new spectral libraries, document them, merge or subset libraries, select optimal endmembers using a variety of measures then run MESMA. The remainder of the tutorial will be divided into the three main activities described above: Spectral Library Tools, Optimization Tools and Mixing Tools.


19

Spectral Library Tools Tutorial:

Before you start The following three sections use examples from Dennison and Roberts (2003b). In this paper, Airborne Visible Infrared Imaging Spectrometer (AVIRIS) data acquired over the Santa Barbara Front Range were used to map vegetation species. To simplify vegetation species class names, species are referred to using four letter abbreviations for their scientific names. For example, the species Adenostoma fasciculatum is abbreviated as adfa. AVIRIS data have been converted to apparent surface reflectance using modified Modtran radiative transfer code described by Green et al., 1993 (See Dennison and Roberts, 2003a and 2003b for details), and scaled to reflectance times 1,000. The following files, available for download at www.vipertools.org, will be used in the examples:

1) 010614r4_4-5_rfl_reg & 010614r4_4-5_rfl_reg.hdr. This is a 750 sample, 550 line 224 channel georectified AVIRIS image.

2) av2001-sub-1-150_half.roi. This contains regions of interest for most dominant cover types in the region. We have divided the original polygons used in Dennison and Roberts, 2003b in half to improve run time.

If you want to follow along with the examples, start ENVI and open the file 010614r4_4-5_rfl_reg using the “Open Image File” option in the File Menu on the ENVI4.2 tool bar. Load which ever band combination you prefer. Click on the “Overlay” menu option on the ENVI Display window, select “Regions of Interest…”. On the ROI Tool window select “File” then “Restore Saved ROI file”. Load the file av2001-sub-1-150_half.roi. *Warning: It is advised to save any supporting data sets in a data subdirectory, not on the Windows ‘desktop’. Placing image data and libraries on the desktop may cause a failure on certain systems. Windows may insert an extra period into the path name, which will cause an error when reading a file base name.

Create Spectral Library from ROIS ENVI does not have the ability to build spectral libraries from ROIS, EVFS or other image based sources while preserving all of the individual spectra that are present in a given image region. Because one goal of VIPER Tools is to preserve spectral diversity, simple averages of all pixels within an ROI are not acceptable. To build a spectral library from ROIs select the ‘Create Spectral Library from ROIs’ module from the main VIPER Tools menu. This will open up the following window:


20

Step 1: Click the first ‘Browse’ button to select the image file that contains the data that you wish to extract. This image must already be open in ENVI and must have ROIs that are defined and open in ENVI. They do not need to be displayed, but must be opened. Step 2: Select the desired ROIs. To select all, or some ROIS use the CTRL and SHIFT keys following ENVI conventions. In this example, all ROIs were selected by first clicking the top ROI, then scrolling to the bottom ROI in the list and while depressing the SHIFT key the last ROI was clicked. Note that in the example data, all of the ROI names begin with “EVF:” because they were originally generated from ENVI vector file (EVF) polygons. Step 3: Click the 2nd ‘Browse’ button to select an output file. Use the ‘.sli’ extension to ensure that the library is recognized as a spectral library by later VIPER Tools modules. Step 4: Click ‘Run’ to execute. A progress bar will appear to show the progress of the library building. This routine can take several minutes or more when a large number of ROIs, or very large ROIs are used to build the library. Tutorial: For the tutorial dataset select 010614r4_4-5_rfl_reg as the input image, select all ROIs, and type the name viper_tutorial_train.sli. If a previous version of the spectral library exists, it should be safe to overwrite it. Pressing the ‘Run’ button will create the spectral library viper_tutorial_train.sli. Open the spectral library viewer in ENVI (Spectral > Spectral Libraries > Spectral Library Viewer) to view some of the spectra.


21

Processing notes:

1. This routine extracts each pixel in each selected ROI, and saves it as a unique spectrum in the output spectral library. It does not calculate a mean spectrum, but rather a complete set of all spectra in a given ROI. This allows for an assessment of the variability within a give ROI and for the removal of outlier spectra.

2. This routine screens for ROI pixels containing “no data” values. If all pixels in a

given ROI have a value of 0 for all bands, data for that ROI are not extracted. If some ROI pixels contain no data, these will be removed from the spectral library.

3. The output file is a standard ENVI spectral library which retains all of the

wavelength information (wavelength, full width half maxima, bad bands list, etc.) but does not have any spatial information. Spectral names are assigned by taking the name of the ROI from which the spectra were extracted and adding the pixel coordinates to create a unique name for each pixel. For example, if the ROI was ‘EVF’ an output spectrum might be named ‘EVF1_X123_Y654’ if the pixel were from the coordinates (123, 654).

Create Metadata for Spectral Library Several components of VIPER Tools require that spectra in a library have associated metadata. Metadata are required to enable VIPER Tools to keep track of the class of each spectrum and calculate EAR, MASA and CoB for each class. Metadata also enable a user to select different criteria for sorting and viewing. Metadata are stored in Comma Separated Value (CSV) format consisting of a minimum of two columns. The first column includes the name of each spectrum. If the spectral library was built using VIPER Tools, this will consist of the name of the ROI followed by the x-y coordinates of the pixel. If you are working with other spectral libraries, the names will be based on whatever names preexisted in the library. While typing in metadata can be challenging, your time is likely to be well spent. Metadata are critical components of later sorting and visualization and ensure that you can share spectral libraries with colleagues. Unlike a database where data types are defined for different columns, the spectral library metadata are treated as string data regardless of actual data type. This means that VIPER tools will treat numbers as characters for sorting purposes (see View Spectral Library, for an exception). The user is free to add as many columns as needed. The metadata can be as simple or as complex as needed (i.e., including sensor, location, lat/long etc or just simply a name). However, VIPER Tools will only use up to two columns of information at a time when sorting. Step 1: Select Create Metadata for Spectral Library. An ENVI window will open requesting a file name. If a metadata file already exists for this library, VIPER Tools will


22

ask whether the metadata file should be overwritten. Before overwriting a metadata file, make sure critical information will not be lost. Step 2: A small window will open in the middle of your screen and read “Open template in default CSV viewer for editing?” Clicking ‘Yes’ will attempt to open up the template metadata file in your default CSV editor (e.g., Excel). Step 3: Enter in metadata. With the metadata CSV file open in your default editor you can now enter in metadata. If the editor is a spreadsheet you should see the following cells for the header line Name Meta1 Meta2 You can rename all but the first column, and add as many columns of information as necessary. If a column is not used, make sure that it does not have a header. It is necessary for all rows in a valid column to have data values. Use a place holder such as ‘NA’ for cells in which metadata is unavailable. Metadata can be added by cutting and pasting from other spreadsheets or databases. Extreme care needs to be taken, however, to preserve the order of the spectra names in the first column, as these must match the order in which the spectra appear in the library. Care must also be taken to ensure that all cells in a given row match the spectra listed in the first column. For these reasons, sorting of metadata in the CSV editor is strongly discouraged. Step 4: When finished adding or editing metadata save the file as a CSV file. As this is not the default for most editors, take care to output the updated version as a CSV and not an *xls or other file type. Tutorial: For the tutorial, we are using metadata fields consisting of up to 12 columns. We are using a hierarchical scheme, starting with the spectral name, followed by the broadest possible class, then progressing to more and more descriptive classes. The final columns are left for additional information, such as data source, supporting photographs and spatial scale. For the library you will develop we have devised the following core metadata fields. Name Perv_Imp Class Form phenology leafform Dominant EVF xy pervious gv shrub evergreen broadleaf ceme EVF xy pervious gv shrub evergreen needleleaf adfa EVF xy pervious gv tree evergreen broadleaf quag EVF xy pervious gv shrub evergreen broadleaf cesp EVF xy pervious gv shrub evergreen broadleaf argl EVF xy pervious npv litter annual grass sgrass


23

EVF xy pervious npv litter annual grass dgrass EVF xy pervious gv shrub deciduous needleleaf arca EVF xy pervious gv shrub evergreen broadleaf qudu Other classes include soil, urban, and the invasive herb Brassica nigra (brni). Only the urban class is considered impervious. Genera and species include: Ceanothus megacarpus big pod Ceanothus ceme Ceanothus spinosus greenbark Ceanothus cesp Ceanothus cuneatus Buckbrush cecu Quercus dumosas scrub oak qudu Quercus agrifolia coast live oak quag Arctostaphylos glandulosa manzanita argl Artemisia californica California sage brush arca sgrass is senescing grass and dgrass is dead grass. Select Create Metadata for Spectral Library. An ENVI window will open requesting a file name. If you are using the example from above, click on “viper_tutorial_train.sli” then click on Open. If a csv file already exists with this name, VIPER Tools will ask whether the metadata file should be overwritten. If you type yes, make sure critical information is not lost. Next a small window will open in the middle of your screen and read “Open template in default CSV viewer for editing?” Click on Yes. If you use Microsoft Excel as your default CSV file editor, an Excel spread sheet will open. For viper_tutorial_train.sli you should see the following cells for the first three lines: Name Meta1 Meta2 EVF:1_X463_Y224 EVF:1_X464_Y224 For this example, I have already created all of the metadata you will need. Open the file viper_tutorial_train_master.csv. Change the first row names so they match, then cut and paste all of the columns from viper_tutorial_train_master.csv into viper_tutorial_train.sli. When you are done entering metadata, save the file in .csv format. Processing notes: 1) Not all computers support the CSV format. For example, when teaching a short course in Germany, we discovered that some German computers did not save CSV files with a comma, but rather used a different symbol. As a result, later VIPER Tool menus, which assume comma delimited formats did not function. There are at least two ways to work around this problem:


24

a) Change the symbol to a comma using notepad or wordpad. b) Modify the system settings on the computer to default to a comma when saving CSV formats. 2) When creating metadata fields, caution should also be used for the field names. Some characters are not allowed by IDL and will cause some elements of VIPER Tools to fail (most notably the View Spectral Library). A good example of a disallowed character in a field name is a “-“. In contrast, a “_” appears to work fine. If View Spectral Library fails with an obscure message about allowed characters, check your metadata fields.

View Spectral Library A significant change between Viper Tools 1.2 and 1.4 was the development of a tool that enables a user to view spectra in a spectral library, select spectra for output (either to a new library or ENVI plot window), sort spectra and link each spectrum to a picture. This tool consists of a number of important features, most notably the ability to view metadata, including new fields added after sorting and after optimization. When using this option, two main windows are opened. The first window lists all of the fields in the metadata and requests that the user tell VIPER Tools which types of data are in each field. The default option is string. Other options include integer, float, double precision etc. This window also requires the user to tell VIPER Tools which symbol is used to account for empty fields in the metadata (ie, 9999, NA etc).

After telling VIPER Tools which types of data are stored in each field, click done. A new menu will open that allows you to display spectra, sort spectra, select and export spectra to either a plot window or new library. If your metadata includes jpegs, such as field photographs, these can also be displayed. An example screen shot is shown on the following page.


25

Using this tool, it is relatively easy to sort and select spectra: Selecting spectra: You can select a specific spectrum by clicking left on the far left column. To display multiple spectra, use the CTRL (single selection) or SHIFT keys (select a range) while you click. Exporting spectra: To export a spectrum, click left in the Export column within the appropriate row (spectral name). This should convert a No to a Yes. Sorting spectra: To sort spectra, click right on the column heading of any category. Changing column width: Often it is difficult to display all of the metadata in a single table. To resize any column width click left on a column boundary. You can widen or shrink a column by moving the bar right or left. Adjusting Plot Bounds: x or y plot bounds can be changed by simply typing in a new value and hitting return.


26

Other options include “Export to ENVI Plot Window” (right of the plot), Select All for Export, Select None for Export and Export to Spectral Library. Processing notes: The View Spectral Library Tool is especially sensitive to the characters you use in names for data fields. If the tool fails, a likely cause is a disallowed character, such as a * or -. Tutorial: Click View Spectral Library (*BETA*) and select the file viper_tutorial_train.sli. At this stage, all metadata elements should be strings, so leave these as defaults. Type NA in the lowest window and hit click on done. After you do this a new window should open that resembles the figure in the tutorial except that the plot window is blank. Experiment with plotting spectra, selecting them, sorting them, modifying plot windows and exportation. Try changing the width of a column. When you are comfortable with this tutorial, move on to the next tool.

Manage Spectral Libraries Often it is necessary to merge one or more spectral libraries. This might be necessary if libraries were derived from multiple sources or, for example, two libraries from the same source needed to be merged. It can also be important to have the ability to sort spectral libraries. This is particularly true for VIPER Tools, in which it is far easier to visualize the relationships between spectra in a square array (see below) if they are sorted by both their category and brightness values. Finally, sorting allows a user to effectively group spectra in a library into similar materials. Thus, if a library consisted of numerous types of soils, live plants or dead plant materials, the user could create a metadata field called “category”, categorize each spectrum as Soil, GV and NPV, then sort the library into these categories independent of their original order when the library was developed. As Manage Spectral Libraries sorts and merges spectral libraries, all of the metadata associated with all files is placed in a new metadata file for the output spectral library. The ‘Manage Spectral Libraries’ module can be used to accomplish the following spectral library tasks:

1. Merging two or more (maximum of 5) spectral libraries 2. Sub-setting spectral libraries to desired spectra 3. Scaling spectral libraries by changing the reflectance scale factor 4. Sorting the libraries by metadata fields or by brightness

To launch the Manage Spectral Libraries module click ‘Manage Spectral Libraries’ on the main VIPER Tools Menu. The following window will appear:


27

Step 1: To select the first spectral library click the ‘Browse’ for 1st library. Step 2: Select desired spectra. If only a subset of these spectra are desired in the output library, click the ‘Subset spectra’ button to open up the subset spectra dialog:

The default is to select all spectra. If all spectra are desired, simply click ‘Done’. If not click the particular spectra that are desired, using the scroll windows to view all files. CTRL and SHIFT keys work here to select multiple spectra, and all spectra can be selected by clicking ‘Select All’. Once the desired spectra are selected click ‘Done’


28

Step 3: If multiple libraries are to be merged, repeat steps above for each input spectral library. Note that if more than one library is used, all libraries must include the exact same bands and the same reflectance scale factor. Also, if at least one of the input libraries has a metadata file, all files must have metadata files and all metadata files must contain the same fields. Step 4: Select input reflectance scale factor using the dropdown box. Possible scale factors are 1, 1000 and 10,000. For radiance or DN data select 1. Note that the default scale factor for ACORN reflectance retrieval outputs is 10,000. All input libraries must use the same reflectance scale factor. Step 5: Select sort fields. Up to two sort fields can be used to sort the output spectral library. These fields include any of the fields in the metadata file as well as a full-wave brightness value (sum of all bands) which is calculated for all spectra if selected. Considering sorting first by the main grouping (e.g., class) and then by brightness to aid in the interpretation of square array results. If only one sort field is desired, select it in both columns (‘1st By:’ and ‘Then By:’). Step 6: Click the ‘Browse’ button in the Output Spectral Library box to select an output file. Use the ‘.sli’ extension to ensure that the library is recognized by later VIPER Tools modules. Step 7: Use the dropdown box in the Output Spectral Library box to select input reflectance scale factor. Possible scale factors are 1, 1000 and 10,000. For radiance or DN data select 1. Setting this to a scale factor that is different from the input scale factor will cause all spectral values to be converted to match the new scale factor. Step 8: Click ‘Run’ to begin processing. Tutorial: For the tutorial we will use the sorting ability to sort the file “viper_tools_train.sli” first by “dominant” (meaning dominant species within the original ROI), then by brightness to aid in the visualization of these data in later steps. We will output this information to a new file called “viper_tools_train_sort.sli”.

1) Select “viper_tools_train.sli” as the 1st spectral library by using the ‘browse’ button

2) Since this library is scaled to 100% reflectance = 1000, select 1000 for the reflectance scale factor.

3) Select dominant in the left sort list and brightness in the right list to sort first by dominant and then on brightness.

4) Select “viper_tutorial_sort.sli” for the output filename, and set the output reflectance scale factor to 1000.


29

5) Click on View Spectral Library and open your sorted library. Note, you have now gained an extra metadata field called brightness. Using the toggle, switch the value from string to floating point for the brightness field. Type NA and click on Done.

6) Within View Spectral Library, rescale column widths so all of the fields are displayed (Dominant or Name are good candidates). Click right on the Brightness column and plot the darkest spectrum (EVF:41_trn_X509_Y171). This should be a member of Ceanothus megacarpus. Click right on the Brightness column to reverse the sorting. Using the CTRL key, click on the left most column for spectrum 1587 (EVF:144_trn_X400_Y440). This is an urban spectrum and should be plotted in red. Based on the spectral shape, it looks to be red tile roof. To verify this, you could use pixel-locator and the x/y coordinates to see where this spectrum is located on the image.

7) Experiment by sorting the spectra by Brightness. Select the brightest member of each Dominant, click on the Export button to convert No to Yes. As you add each spectrum, the number of spectra selected for export should increase. If you are careful, you should end up with more or less 15 spectra. When you are done, export this spectral library to a new one called “viper_tutorial_train_sort_export.sli”. Use the View Spectral Library tool to see what you have created. You should see something similar to the figure on the next page (Note, I erroneously selected two argl). We will use viper_tutorial_train_sort_export.sli again in the following section.


30

Processing Notes: This routine produces a single spectral library that contains all of the input spectra (after subsetting) and has been sorted and converted to the selected output scale factor. A metadata file is produced that transfers the input metadata to all output spectra. This routine is especially useful for converting reference data from reflectance to scaled reflectance to match imagery for SMA. Note that this routine does NOT perform convolution or spectral subsetting. You will have to use ENVI’s built-in tools for that. Several modules of VIPER Tools require that the number of bands be consistent among all spectral libraries and images used. This routine is also useful for sorting libraries into their main groups and by brightness for easy interpretation of square array results.


31

Convert Library to Image

Convert Library to Image was developed to allow a user to convert a spectral library into an image. The primary value of this tool is that it allows a user to process a spectral library using the same tools that might be used on an image. Thus, for example, if a user wanted to apply MESMA to a spectral library and assess how well it discriminated various materials, the user could use this tool to create an image, than apply the mixing tools to the image rendition of the spectral library. The basic components of the tool include: 1) A window that allows a user to select an ENVI Spectral Library. 2) A window that allows a user to choose an output name. 3) A window that asks whether an ROI should be created for each spectrum. 4) In the event that a user opts to create ROIS, a window that asks whether the x/y coordinates should be stripped from the ROI. Tutorial: Click on Convert Library to Image. Select the spectral library you created in the previous section “viper_tutorial_train_sort_export.sli”. Assign an output name of “viper_tutorial_train_sort_export.img”. Say Yes to ROIs but do not strip off the xy coordinates. After the tool is finished, display the spectral library as an image (choose any band combination). Note, how you can navigate the image using pixel locator, plot spectra using the z-profile and identify the original spectrum name using the ROI tool.


32

Optimization Tools Create Square Array Use the Create Square Array module to calculate the fit metrics needed for EAR, MASA and CoB. Square arrays are standard ENVI images that are n by n, where n is the number of spectra in the library. The original format of the square array was proposed by Roberts et al. (1997) and included in several theses published at UCSB (Gardner, 1997; Halligan, 2002). The current format was developed by Kerry Halligan and has been expanded from RMSE to include the spectral angle, bright endmember and shade fractions and a constraint-code band (see below). Columns and rows store the model number, corresponding to one column and row for each spectrum in the library. Each pixel corresponds to a column spectrum used to unmix a row spectrum. Square arrays are stored as an ENVI image, with with the following bands: RMSE, Spectral Angle, EM Fraction, Shade Fraction and a ‘Constrained’ band which indicates if the model met the constraints used (See step 5 and the Square Array Output Bands section below for more detail). The diagonal represents each spectrum modeling itself and is meaningless so it has been zeroed out for all output bands. Creating a square array: Step 1: Launch the Create Square Array module by clicking ‘Create Square Array’ on the main VIPER Tools Module. The following window will appear:


33

Step 2: Select the input spectral library. If more than one group is included in the library, a metadata file must be present. It is best to select a library that has been processed using the ‘Manage Spectral Libraries’ module such that it is sorted first by the main grouping field in the metadata, then by brightness. This will aid in the interpretation of the square array. Step 3: Select the input reflectance scale factor. For radiance or DN data use a scale factor of 1. Step 4: Select an output filename. Since a new metadata file will be written for the square array, the base filename (the part before the extension) must be unique in order to prevent conflicts in metadata files. The recommended naming convention is to add a ‘_sqr’ to the end of the library name, and to optionally add the constraint parameters used (e.g., ‘speclib_sqr_-5_105_2pt5’ would be used for an input file named ‘speclib’ and


34

constraints of -5%, 105% and 2.5%. Note that ‘pt’ is used instead of ‘.’ to avoid conflicts with filename parsing on the ‘.’ elsewhere in VIPER Tools). Step 5: Select the mode to run: unconstrained, constrained, or partially constrained. There are 3 modes for creating Square Arrays, which affect how SMA metrics (fractions and RMSE) are handled:

Unconstrained: SMA is conducted for all pixels in the image regardless of whether it creates super positive (>100%) or negative (<0%) fractions or high RMSE values. All values in the ‘constraint’ code band will be 0, indicating that the models met the constraints.

Constrained: Thresholds are set for minimum and maximum fractions (-50% - 150%) and for RMSE (0%-10%). When either the fractions or the RMSE are exceeded, a value of 2 is stored in the constraint code band for that pixel, indicating that the model did not meet the constraints. A value of 0 will be stored for all models which do not exceed the constraints. Partially Constrained: Thresholds are set for minimum and maximum fractions (-50% - 150%) and for RMSE (0%-10%) as with the constrained mode above. When fractions are exceeded, however, the RMSE is calculated at the fraction threshold value rather than the calculated fraction value. If this ‘constrained’ RMSE is below the RMSE threshold the model is accepted as ‘partially constrained’, and a value of 1 is stored in the constraint code band. If the RMSE exceeds the threshold, then the constraint code band is given a value of 2, indicating that it still exceeded the constraints. Running in partially constrained mode is useful for allowing models with very good fit to be included despite being slightly too dark. For example, spectrum A is a well fit spectrum for modeling spectrum B, but would produce a fraction of 106% (1.06) because it is darker than spectrum B. If running in constrained mode with the maximum allowable fraction set to 105% (1.05), this model would be excluded completely because it exceeds the threshold. In partially constrained mode this model would be run forcing the bright endmember to have a 105% fraction. The resulting RMSE would be slightly higher (due to a slight underestimate of the bright endmember fraction) but would not be excluded. If the RMSE at 105% is below the RMSE threshold it will be considered partially constrained and will be used in further steps of VIPER Tools for calculating EAR, MASA and CoB. It may even be possible that spectrum A will produce a low EAR value for the library, suggesting it is an optimal endmember despite the fact that it is fairly dark.

Step 6: If running in constrained or partially constrained mode, select the desired constraint thresholds by checking the box and adjusting the slider to the desired value. Values are all in percent reflectance. Default values of -0.05, 1.05 and 0.025 for minimum fraction, maximum fraction and RMSE threshold, respectively, represent values used in the literature (Halligan, 2002, Roberts et al., 2003).


35

Step 7: Choose whether to run the program in fast or slow mode. In most cases, users will opt for the fast (memory) mode. However, in cases where users are working with very large spectral libraries (Say tens of thousands of spectra), you will need to run it in slow mode, which writes products to disk as the process runs. Step 8: Click ‘Run’ to create the square array. Tutorial: For the tutorial example, we will work with the file “viper_tutorial_train_sort.sli” we created previously and will create a square array in partially constrained mode.

1) Select viper_tutorial_train_sort.sli and set the reflectance scale factor to 1000. 2) Select “viper_tutorial_train_sort_ sqr” as the output filename. 3) Set the mode to ‘partially constrained’ and set the minimum fraction to -0.05,

maximum to 1.05 and RMSE constraint at 0.025 reflectance. 4) Select fast mode. 5) Click ‘Run’ to create the viper_tutorial_train_sort_sqr square array, which

could take up to 20 minutes.

Square Array Output Bands Output bands in the Square Array module include RMSE, Spectral Angle, EM Fraction, Shade Fraction, and a ‘Constrained’ code. Each of these is described in detail below: RMSE: The RMSE at column A and row B is the root mean square error of spectrum A modeling spectrum B. RMSE is calculated using the following equation:

RMSE images are not symmetrical about the diagonal (see EM fraction description below).

Spectral Angle: Spectral Angle at column A and row B is the angular distance, in radians, between spectrum A and spectrum B. This is the same metric used by ENVI’s Spectral Angle Mapper (SAM) and is calculated in IDL as:

( )2

1λ

1k

2iλ NεRMSE

= ∑

=


36

Spectral angle = acos ( total ( A / sqrt ( total ( A2 ) ) * B / sqrt ( total (B 2 ) ) ) ) Where A and B are vectors containing the spectral data for endmembers A and B For spectral angle, the images will be symmetrical about the diagonal, since spectrum A modeling spectrum B is the same as spectrum B modeling spectrum A. Endmember Fraction: The endmember is the non-shade endmember in a two endmember model. The endmember fraction band at column A and row B is the SMA fraction for endmember A when used to model endmember B. These images are not symmetrical about the diagonal, because brightness differences contribute to differences in SMA fractions and model RMSE. When bright spectrum A models dark spectrum B the SMA fraction will be between 0 and 100%, and the RMSE will be calculated as the difference between the spectra at the brightness of the darker spectrum B. When dark spectrum B models bright spectrum A the SMA fraction will be greater than 100% and the RMSE will be calculated from the difference between the spectra at the brightness of the brighter spectrum A, and thus will be a larger RMSE than the previous case. Shade Fraction: The shade fraction at column A and row B is the SMA shade fraction for endmember A when used to model endmember B. It is calculated as 1 minus the EM fraction, and is included to allow for shade thresholding in the EMC calculations. Constraint Code: The constraint-code band contains a value of 1 if the model was partially constrained (exceeded fractions but within RMSE constraint) and a value of 2 if it was fully constrained (exceeded RMSE constraint). Unconstrained models have a value of 0. See step 5 above for a more details. Example: The following example shows the RMSE, spectral angle and Fraction images for the square array developed for viper_tutorial sort. These represent the results for a partial constrained solution with -0.05, 1.05, and 0.025 for minimum fraction, maximum fraction and maximum RMSE, respectively. The image on the left shows the RMSE with dark values corresponding to a low RMSE and bright values high RMSE. Comparing the RMSE image (left) to the spectral angle image (center) we can see one of the key differences between these two approaches: spectral angle is symmetrical along the diagonal (meaning that the spectral angle is the same between two spectra, regardless of which one is the reference). In contrast RMSE is not. Since we have sorted the library by brightness, we see that a bright spectrum unmixing a darker spectrum produces a low RMSE (upper right triangle of the RMSE image above the diagonal), whereas a dark spectrum unmixing a brighter spectrum produces a much higher RMSE (lower left triangle of RMSE image below the diagonal). Horizontal bright bands in both the spectral angle and RMSE square arrays show the higher values created when one class of spectrum (say GV) is used to unmix another one (say soil or NPV).


37

Square array bands for partially constrained square array are (from left to right): RMSE band, spectral angle band, EM fraction band. The Shade fraction and Constraint-codes are not shown.

EAR/MASA/CoB (EMC) The square array retains all calculations of RMSE, Spectral angle, fractions and constraint type for each possible combination of model (column) and spectrum (row) in the spectral library. The key to VIPER Tools is to use this information to determine which spectrua within a group of spectra are most representative of their class while covering the range of variability within the class. We have developed three approaches to do this. CoB Count based Endmember selection (CoB) was first proposed by Roberts et al. (2003) as a means of selecting optimal endmembers as those members of a library that model the greatest number of spectra within their class. Using this approach, candidate models are assessed by whether they meet fraction, RMSE and residual constraints when unmixing any other spectrum in the library. This information is stored in the square array in the ’Constrained’ band. After calculating each model, the total number of spectra modeled within the class (in_CoB) and the total number of models outside of the class (out_CoB) are recorded for each model. The optimum model is selected as the one that has the highest in_CoB value. Note that ties are not split in this process, so several spectra can have the same in_CoB value. To determine additional CoB selections, VIPER Tools implements the approach described by Clark (2005). Using this approach, once an initial optimum in_CoB spectrum (or spectra, in the case of ties) is selected, all members of the spectral library that were successfully modeled by this spectrum (or by one of the spectra, in the case of ties) are culled from the list of candidate CoB spectra. After culling, a second tier in_CoB is calculated and a second tier, optimal spectrum (or spectra, in the case of ties) is selected base on the the highest in_CoB score. This process continues until all candidate spectra have been eliminated, either by being selected as an optimal


38

spectrum or by being modeled by an optimal spectrum. Out_CoB provides a measure of confusion between classes, with a high value suggesting significant confusion between classes. Ideally, the perfect spectrum will only model members of its class, and not model members outside of its class. This model would have a high in_CoB, but low out_CoB. To assess the generality of a spectrum VIPER Tools includes a third performance metric, the Count Based Index (CoBI: Clark, 2005). In VIPER Tools, CoBI is calculated as the ratio of in_CoB to out_CoB with the denominator multiplied by the number of spectra within a class. Thus a high CoBI and a high in_CoB represents an excellent choice (A specialist as described by Roberts et al., 1997). A high CoBI and moderately low in_CoB may also be a good candidate because it captures a member of a class that is spectrally unique, even if it is not well represented in the library. Low values of CoBI would only be acceptable if they were paired with a high in_CoB (Generalists). EAR Endmember Average RMSE (EAR) was first proposed by Dennison and Roberts (2003a) as a means of locating spectra within a class that provide the best fit. EAR is calculated as the average RMSE produced by a spectrum when it is used to model all other members of the same class. EAR is calculated for each member of a class within the spectral library. In this instance, the optimum spectrum would be the one that produces the lowest average RMSE. MASA Minimum Average Spectral Angle (MASA; Dennison et al., 2004) is similar to EAR in that it is designed to select spectra with the best average fit within a class. It differs from EAR in that the measure of fit used is the spectral angle, not the RMSE. Using this approach each spectrum is used to calculate the spectral angle between itself and all other members of the spectral library. MASA within a class is calculated as the average spectral angle between the reference spectrum (candidate model) and all other spectra within the same class. The best MASA candidate is selected as the one that produces the lowest average spectral angle. EAR and MASA are conceptually very similar. However, the optimum selected by each metric will depend on the overall brightness of the library. For example, for dark objects, subtle spectral differences between the reference and non-reference spectra could result in a fairly large spectral angle. At the same time, these subtle spectral differences would result in a small RMSE. The net result is that MASA will be far more sensitive to spectral differences for dark objects. For bright objects, the opposite is true. In this case, even a fairly large spectral feature may produce a small difference in spectral angle, but a large difference in RMSE, and EAR would be more sensitive to spectral differences. For further discussion of the differences between RMSE and spectral angle as metrics of spectral fit see Dennison et al. (2004). Despite differing sensitivities for darker and brighter endmembers, one endmember will frequently possess both the minimum EAR and MASA values.


39

VIPER Tools includes all three metrics to aid users in selecting optimal spectra. Users are encouraged to try out all three and assess performance based on how well spectra work on actual imagery.

Calculating EAR, MASA and CoB. To calculate these three metrics (and CoBI) from the square array, select Calculate EAR/MASA/CoB (EMC) in the VIPER Tools menu. This will open up a window that requests the name of the square array, a box requesting the group to sort by and an output file name. We suggest using the suffix *_emc.csv, to designate an emc file. The strength of any particular selection technique will vary with metadata field. For example, whereas EAR or MASA may outperform CoB for homogeneous selection criteria, such as dominant, CoB may outperform these measures for heterogeneous selection criteria, such as impervious/pervious. Calculating the EMC file is also critical when choosing the hierarchical level for your selections. For example, if a user were most interested in separating impervious from pervious surfaces, they might opt to create sort on impervious/pervious when creating the EMC file. A different user might be interested in separating urban materials, and thus would sort on the material scale. After running the EMC viewer, you will be asked whether you want to append the new columns on to the existing metadata file associated with the spectral library. Most often the answer will be yes. By doing this, you can use the View Spectral Library Tool but also sort based on COB, MASA or EAR. However, keep in mind that when you do this, this adds additional columns to your original metadata for the source library. Furthermore, these columns will be specific to the Group selected for the EMC viewer. If a user wanted to do this twice (ie, sort on Imperv-perv first, then later sort on Dominant) this creates a problem for VIPER Tools, because it would add new columns to the original metadata file twice, with two sets of columns with the exact same name (but different attributes). The best option in this case, is to say no for the second search, or create a duplicate of the original source library.


40

Steps: Step 1: Select input square array file Step 2: Select the grouping attribute that divides the spectra in the library into classes Step 3: Select the output filename Step 4: Click fast or slow mode. Step 5: Click Run when you are ready to run the file. Step 6: Decide whether to append the EMC information in to the source file metadata. Once the EMC file is done, VIPER Tools will automatically open up the EMC Viewer. If you have previously created an EMC file, you can skip this step and proceed directly to the viewer. Tutorial: For the tutorial example select viper_tutorial_train_sort_ sqr as the input file, sort on ‘Dominant’ and select viper_tutorial_train_sort _dom_emc.csv as the output EMC filename. In the output filename the ‘dom’ was added as metadata to remind the user which category was used for creating the EMC file. When asked whether to append the EMC output on to the source metadata say yes and select viper_tutorial_train_sort.sli. Once Create EMC file is done, the EMC Viewer will open. To see how your metadata have changed in the source file, click on View Spectral Library and select viper_tutorial_train_sort.sli. Note the new metadata fields including “Group”, EAR, MASA, InCoB, OutCoB and COBI. Assign string to Group, float to brightness, EAR, MASA and COBI and Integer to InCoB and OutCoB. Type NA and click Done. Once the View Spectral Library has opened, start by resizing the columns so that EAR, MASA and InCoB are displayed. Start by sorting on InCoB. In this example, the best candidate for each dominant would be the spectrum with the highest InCoB. Where two spectra of the same class have the same InCoB, we can use other measures (such as EAR) to make our choice.


41

Figure showing an example of using the View Spectral Library tool to sort on the best InCoB.

View EMC File The EMC Viewer is a critical tool for aiding the user in identifying and outputting a subset of optimal spectra from a library using the EAR, MASA and CoB values stored in the EMC file. This module is best used in tandem with the ENVI spectral library viewer. We have deliberately chosen not to replicate ENVI tools that already work well, such as the ENVI spectral library viewer.


42

The EMC viewer first opens up showing a seven-column table that includes a row for each spectrum in the library. It also includes three file names above the table, including the source name for the emc file, and two file names that need to be assigned, the Source spectral library and Output spectral library. The table consists of seven columns from left to right including Name, Group, EAR, MASA, In_CoB, Out_CoB and CoBI. The entire spectral library can be viewed, or specific groups viewed by choosing between ALL or the group of most interest. The View EMC module also has the ability to sort spectra based on any of the five metrics and export spectra to a new spectral library. To sort spectra click right on the column header (top of table) of the metric of interest. The first right click sorts the data A-Z, a subsequent right click sorts the data Z-A. To select a spectrum for export right click on the row header (left of table) for line that contains that spectrum. The spectrum will be added to the export list in the right panel. Spectra in the export box can be removed by double clicking the spectrum name in the export panel. Once you are done selecting spectra, these spectra can be exported to a new spectral library, who’s name is selected in the ‘Output spectral library file’ box, by clicking on “Export to Spectral Library”. We have deliberately opted to require extensive user input into this process and avoid automation. We also strongly urge users to use the ENVI spectral library viewer when


43

making selections. No spectrum should be selected as optimal site unseen. A nice trick you can use is to create a temporary spectral library as your output file. As you add new spectra to this library, you can periodically save it and use the ENVI spectral library viewer to look at the spectra you have selected without having to sift through the entire library. When you are done selecting your spectra, you can change the output file name to your final library and save your spectra.

Tutorial: In the following example, we will select a set of optimum spectra using EAR, MASA and CoB from the viper_tutorial_train_sort.sli spectral library. Our objective will be to select the top three candidates for each metric. We will save these spectra to a new library called “viper_tutorial_train_sort_optimal.sli”. However in doing so we should also select candidates from different polygons for EAR and MASA. Thus, if the top three lowest EAR spectra were derived from the same ROI, we would pick the best of that group, than move down the list to find the best EAR in another ROI. CoB requires a different approach, since CoB ensures that the second best spectrum cannot be modeled by the first spectrum. A number of different strategies may be employed. For example Powell (2006), used EAR to select the top 10 candidates within a library, used all 10 in a


44

model, then selected the candidates that in combination had the lowest EAR but also modeled either most of the image, or important components of the image. This approach is necessary to circumvent limitations of a spectral library which may not adequately represent an image. It should also be noted that there are many ways to go about solving this problem. For example, in various instances we have found the following strategy to be highly effective: 1) Sort by group (dominant, class etc) using InCoB. Using this approach we know that the highest InCoB is the spectrum that models the most of that group. We also know that another spectrum within the same group, but with a different InCoB is spectrally unique. 2) Where a group has the same InCoB for multiple candidates, select the spectrum that has the lowest EAR within the spectra that have the same InCoB. 3) In cases where InCoB fails to select a spectrum that has a unique shape (essentially a dark spectrum that can be modeled using a brighter spectrum), use MASA to identify the best candidate. A user can also perform much of this analysis using the View Spectral Library option. This approach has the advantage in that it links the sorting and selection process with the metadata and spectra. However, EMC Viewer can be used to view files created using different sort criteria to generate the EMC file and also includes an option for subsetting the spectra in the table by group (ie, rather than searching through an entire library, just focus on a single group within a category). Thus, if a user is comfortable working with a more cumbersome way of displaying spectra and does not require the metadata, the EMC Viewer may be the tool of choice.

Selecting optimal spectra The following example uses the EMC viewer to select optimal spectra (In contrast to the View Spectral Library approach). Enter the name of the spectral source file (viper_tutorial_train_sort.sli) in the appropriate box and enter the name for the spectral library you wish to create (temp.sli). temp.sli is a spectral library we are going to use as a temporary holding place for spectra as we select them. Use the Select Group subset data option on the View EMC tool and select the first group, adfa. Click right on EAR. Next click right on the left most column at the zero position. This should place the spectrum EVF70_trn_X665_Y80 in the export box. Repeat this process for the next best candidates from EVF57 and EVF51. You should see the screen shown on the previous pages. Use the Spectral Library Viewer to open the ENVI spectral library temp.sli. Using the ENVI Spectral library viewer, locate the spectrum EVF70_X665_Y80. In Options select Plot_Key to display the name of the spectrum on the plot. This spectrum represents the adfa (Adenostoma fasciculatum spectrum) that produced the lowest average RMSE within the adfa class. Display the other two EAR candidates on the same plot.


45

Looking at these spectra, we see mostly subtle differences between the spectra. Our first EAR selection, EVF70 is the brightest across all wavelengths. Our second, EVF57 has intermediate NIR reflectance, the lowest SWIR2 and high visible reflectance. Our third selection, EVF51 is darkest in the visible, NIR and SWIR1, but slightly brighter than EVF57 in the SWIR2. Repeat this process by clicking right on MASA . Locate your top three candidates and click right on the export row to add them to your library. Save a new version of temp.sli and redisplay it using the Spectral Library Viewer. You should note that MASA selected spectra from three different polygons, EVF58, EVF56 and EVF22. You will probably have to close temp.sli first in the Available Bands list, then save a new version. Repeat this process with In_CoB. If two techniques select the same spectrum, only export the spectrum once. In this case EAR and In_CoB selected the same spectrum as the top choice for the top two selections. For my third choice, I opted for EVF68. I also added EVF71, although the low In_Cob of 31 and high Out_Cob of 596 suggests that it may be problematic. Since CoB screens out spectra each iteration, multiple spectra with the same In_Cob represent spectra that modeled the same number of spectra. In this case, the best option is the one with the lowest Out_Cob.


46

Looking at the spectra above, we can see that various flavors of chamise differ either primarily in brightness (with subtle differences in spectral contrast) or dramatically in spectral contrast, with two spectra (EVF68 and EVF71) standing out as having pronounced ligno-cellulose bands, high SWIR2 reflectance and high visible reflectance. These latter two were selected with CoB and are either highly senesced or mixed with rock. Repeat this process for all over major classes (argl, cecu, ceme, cesp, dgrass, quag, sgrass, soil, urban). If you make a mistake, double click in the Export box to remove spectra you selected in error. When you are done, change the name of the output file to a new name and click on the Export Spectral Library box. I would suggest viper_tutorial_train_sort_optimal.sli. Other helpful hints: Some indices will fail to produce markedly different spectra, even if they come from different polygons. This is particularly true of MASA. It may also not be possible to select more than one candidate from CoB. If, for example, the third selection models far more of the wrong class than itself, it may be best left out. In addition, because CoB automatically culls out all spectra from the first selection, even if the second best selection is from the same polygon, it still is likely to be quite different. This was the case with argl, in which the first two candidates came from the same polygon, while the third candidate only had an in_CoB of 16 (but was selected nevertheless). Also, keep in mind that you are working with a research tool. Optimal candidates are likely to differ between cover types and data sources. Thus, simply selecting the top three candidates may not be the best option for your study problem. We do not know all of the answers, but have given you a tool to help you answer yours.


47

Constrained Reference Endmember Selection (CRES) The CRES module in VIPER Tools presents an alternative to the EMC approach to endmember selection. With CRES a user supplies expert knowledge on the expected SMA fractions at a particular location in order to select the optimal endmembers for that site. This approach was first described by Roberts et al. (1993) and later discussed in more detail in Roberts et al. (1998). It has been used extensively in a number of papers published out of the VIPER group to select optimum endmembers for simple spectral mixture analysis (See Roberts et al., 2002; 2004). CRES is a tool that aids the user to see which endmembers will produce SMA fractions that produce the closest match to the estimated fractions. CRES is typically used to select endmembers from a reference library that will produce the expected SMA fractions when applied to an image spectrum.

Calculating CRES: Step 1: Launch CRES by clicking ‘CRES’ on the main VIPER Tools menu bar. This will invoke the CRES module and open the CRES GUI:


48

Step 2: Select the spectral library containing the image spectra you would like to unmix by clicking the ‘Browse’ button. This library should consist of image spectra for which field estimates of fractional abundance of green vegetation (GV), soil, and non-photosynthetic vegetation (NPV) are available. The library can be of any size, but only one spectrum at a time will be assessed with CRES. Once the library has been selected, select the spectrum to be assessed from the dropdown list to the right of the ‘Browse’ button. Step 3: Select the reflectance scale factor. Step 4: Select the spectral libraries containing the GV, soil and NPV spectra to assess. If only a subset of these spectra are desired, click the ‘Subset’ button to select the desired spectra. The GV library (and the soil and NPV libraries) must contain the exact same number and position of bands as the image spectral library. The total number of models to be run can be seen in the box by the text ‘Total number of models:’. This number is calculated as the number of GV spectra * the number of soil spectra * the number of NPV spectra. Each model is used to unmix the single target image spectrum. Step 5: Select the type of shade to use. Photometric shade (zero reflectance) or non-photometric shade can be used for reflectance data. Non-photometric shade should be used for radiance or DN data. If non-photometric shade is used then a shade library needs to be selected (click ‘Browse’ to select library) and a single shade spectrum needs to be identified (in the dropdown box). Step 6: Calculate SMA fractions and RMSE by clicking the ‘Calculate SMA Fractions’ button in the upper right corner of the GUI. This runs an exhaustive set of 4 endmember mixing models using the input GV, soil and NPV spectra. The main table will be propagated with the SMA results for all models. Each row in the table is one of the set of models, and the columns include the component GV, soil and NPV spectra, the endmember fractions for GV, soil, NPV and shade, and the RMSE. The final column, labeled ‘Index’ will contain all zeros at this point because this value has not yet been calculated. Note that SMA fractions and RMSE (Step 6) need to be calculated only once for each image endmember. Step 7: Select SMA Endmember fraction estimates. The 3 text boxes on the right under the label ‘Target Endmember Fractions:’ are for inputting in the user supplied estimates of fractional abundance. These may come from field data, expert knowledge or from some other ancillary data. These values are a critical component of CRES, as the main goal is to find the spectra that produce the closest fit to these endmember fraction estimates.


49

To input endmember fractions start with the shade component. Highlight the text in the text box and enter in a fraction from 0 to 1, then hit the enter key. Next enter values for GV and soil. NPV will automatically be calculated such that the fractions sum to 1. Depending on the quality of your initial estimates, you are likely to adjust the fractions. This can also be a learning process, helping you devise more reasonable estimates of fractions based on your knowledge of what reference spectra should be. Step 8: Choose the CRES endmember to assess by selecting GV, soil or NPV from the ‘CRES EM’ dropdown box to the right of the table. This selects which endmember will be used to calculate the CRES index (see below). Step 9: Select weights for the endmember and RMSE for the CRES index equation by selecting values from the ‘EM Weight’ and ‘RMSE Weight’ dropdown boxes. The general CRES equations are: GV index = rmse_wt*RMSEc+em_wt*abs(GVc-GVt)+abs(SOILc-SOILt)+abs(NPVc-NPVt)+abs(SHADEc-SHADEt) SOIL index=rmse_wt*RMSEc+abs(GVc-GVt)+em_wt*abs(SOILc-SOILt)+abs(NPVc-NPVt)+abs(SHADEc-SHADEt) NPV index=rmse_wt*RMSEc+abs(GVc-GVt)+abs(SOILc-SOILt)+em_wt*abs(NPVc-NPVt)+abs(SHADEc-SHADEt) Where: GVc is the GV calculated fraction and GVt is the target GV fraction, SOILc is the soil calculated fraction and SOILt is the target soil fraction, GVc is the NPV calculated fraction and NPVt is the target NPV fraction, em_wt is the endmember weighting factor (0-10, and rmse_wt is the RMSE weighting (0-10) Some experimentation is needed to assess the best weighting factors. Increasing the endmember weight places the greater importance on matching the fraction of the target endmember. Increasing the RMSE constraint places greater importance on the fit of the model, or the spectral shape. Because RMSE values (e.g., 0.025) are generally small relative to EM fraction values (e.g., 0.3) an RMSE weight of 5 to 10 is generally helpful in ensuring that RMSE (model fit) is sufficiently weighted. Step 10: Calculate the CRES Index by clicking the ‘Calculate Index’ button. The ‘Index’ column in the main table will now be populated with the CRES index value based on the parameters selected and the equation shown in step 9. Smaller index values indicated better models. Step 11: Sort models by right clicking in the column headers. The first right click sorts the data A-Z, a subsequent right click sorts the data Z-A. Sorting A-Z on the ‘Index’ field will place all of the best models to the top. The list of models can be restricted to aid in assessment by typing a maximum RMSE to display at the bottom of the CRES GUI. Tutorial:

Examples of CRES


50

We will start with a spectral library consisting of eight spectra extracted from viper_tutorial_train.sli. These spectra were selected to correspond to nearly pure stands of chamise (adfa: EVF75), big pod Ceanothus (ceme: EVF53), greenbark Ceanothus (cesp: EVF sb poly104), manzanita (argl: EVF52), coast live oak (EVF95), senesced grass (EVF133) and soil (EVF120). In all cases, I have selected the brightest, nearly pure spectra I could find. I have also added one highly senesced chamise to see if we can locate an appropriate NPV that is not senesced grass (EVF57). We have saved these spectra in a library called viper_tutorial_train_iem.sli. The following figure shows these spectra.

Our objective will be to identify the best set of reference endmembers that unmix chamise, oak, manzanita, Ceanothus, soil and senesced grass with physically reasonable fractions. We are likely to target 1-2 gv endmembers, up to 2 NPV and one soil. We will rely on a subset of spectra from WESTUSA for our reference library. This library consists of laboratory and field spectra originally collected in southern California, Washington state and from libraries developed by the University of Washington. For this example, Ie have convolved the library to the 2001 AVIRIS scene, imported a bad bands list and further subset the library to include only spectra appropriate for Southern California. Finally, I have further divided the library into GV, NPV and Soil libraries based on the meta data. The names of the files are 010627_westusa_gv.sli 010627_westusa_npv.sli 010627_westusa_soil.sli We will start with chamise (EVF75). Start CRES and open the image endmember library viper_tutorial_train_iem.sli. Select the spectrum EVF:75_trn_X431_Y64. Open


51

reflectance spectral libraries for GV, Soil and NPV. Leave all spectra selected. This should result in 1,042,695 models. Once this is done, hit “Calculate SMA Fractions”. Given 1 million models, this may take a few minutes. Recall, this step will only be done once for each image endmember. After calculating SMA fractions, you should see the table filled with model numbers, names for the GV, SOIL and NPV spectra and fractions for GV, Soil, NPV and Shade and an RMSE. You can only display five hundred solutions at a time, but all one million combinations have been calculated. Next, our objective will be to estimate fractions for adfa and allow CRES to search the spectral library for Reference Endmembers that will produce those fractions. We will start by estimating fractions of 40% shade (0.40), 50% GV (0.50), 0.00% Soil. As you type in fractions, be sure to hit a return. The NPV fraction should be recalculated each time you add a new value for a fraction. Finally, we will set the model constraints. We will start by searching for GV and set the RMS weight to 10 while leaving the fraction constraint at 1. The following screen shot shows this set up.

In this example, we can see what we would get if we were to unmix the chamise library with an Artemisia californica spectrum and a combination of different soils and NPV spectra. In most cases we see that the fractions do not come close to our estimate, nor are they often physically reasonable (Model40 has a negative 10% GV, -13% Soil, 71% NPV


52

and 52% shade). Click on Calculate Index. After the index has been calculated you will see numbers in the far right column. Click right on this column to sort from low to high or high to low. Given these constraints, our best solution is CCEOLSTCK (GV), DCSMM72.DAT (soil) and cadfa12671 (NPV). This produces 52% GV, -1% Soil (close to 0), 9% NPV and 40% shade. The RMSE is very low at 0.014. Given that CCEOLSTCK is an evergreen chaparral leaf spectrum from leaf stacks and cadfa12671 is chamise stems, this is a very good first model. Since we were only looking for GV, we will start with CCEOLSTCK. As a second example, we will see whether this spectrum might also work for ceme. We change the image endmember selection to EVF53 and recalculate the SMA fractions. After calculating the fractions, we now have to set expected fractions. Looking at EVF53 several things stand out. First, it is much brighter than the other spectra, suggesting a lower shade fraction. Second, it has very low SWIR reflectance, suggesting a low NPV fraction and no soil. We will start with an estimate of 30% shade, 65% GV and 5% NPV. Looking at our first estimate, we find a set of models that produces fractions that match our estimate. However, the GV spectrum is not Ceanothus. In addition, the RMS is pretty high at 0.021. We try a second model, this time lowering the NPV to 0%, decreasing shade to 20% and raising GV to 80%. Using this model, we identify CCEOLSTCK as a viable model (78% GV, 1% soil, -3% NPV and 23% shade). However, we find that the RMSE is still poor at 0.024. This may be our best model, given the library, but we will try one more test. In this last example, we will try 35% shade and 65% GV. Our best selection is this case is SASP12STCK, producing fractions of 64% GV, 0% soil, 2% NPV and 34% shade. The RMSE is still high at 0.02. This is probably the best we can do, given limitations in the reference library. We will conclude our GV search with coast live oak. Select EV95 and Calculate SMA fractions. After calculating the fractions we need to set fraction constraints. Looking at the spectrum, we can see that it is similar to the chamise spectrum in the NIR, but has lower visible and SWIR reflectance, suggesting lower NPV and slightly higher shade content. We will start with a 50/50 mix of GV and shade. As our first choice, we find selected a Quercus douglasii spectrum. However, we also have several problems, including a high RMS (0.025) and 9% soil fraction. Sifting down the list we actually see that CCEOLSTK gives better results, with 51% GV, 6% soil, 0% NPV and 43% shade. The RMSE for model is better at 0.024 but still fairly high. From these three experiments, we would conclude that CCEOLSTCK is our best overall model, but still not ideal for this scene because of high RMS errors. We will conclude by searching for appropriate soil and NPV endmembers. We start with NPV (EVF133). Looking at the spectrum, I would suggest a modest shade fraction (20%), minor soil (5%) and no GV. We switch the index to NPV and hit calculate.


53

As our first selection we get u9litt01-av. This is a spectrum of senesced grass measured in the field using an ASD. This model has a very good fit (0.012 RMSE), but requires 17% soil and 20% shade. While this may be an excellent model, we will try one more by changing the RMSE weight to 5 instead of 10. This has little impact, forcing me to conclude that the soil fraction was probably higher than my initial estimate of 5%. Next, we will see if we can determine a more appropriate NPV for chamise. For this case, we select EVF57, senesced adfa. We calculate the fractions then set fraction constraints. Looking at this spectrum we can see it is obviously highly mixed, including GV, shade, NPV and probably rock or soil. As an initial guess, we will start with a near equal mix of materials with 30% shade, 25% GV, 15% soil and 25% NPV. We will keep the RMSE weight at 5. As our first pass, we select a fairly reasonable model, consisting of stems from Artemisia calfornica, a soil from the Santa Monica Mountains and leaves from Eriogonum cinarium (an evergreen shrub). We also see that u9litt01 produces a reasonable model for this spectrum, with 33%NPV, 22% GV, 15% soil and 30% shade. Given this result, I would stick with u9litt01. We will conclude with a search for a soil spectrum. Select EVF120 and calculation fractions. Given that this soil is fairly bright, we will start with a low shade fraction of 5%. There is also very little evidence of any senesced material, so we will set the NPV to 0%. Our first model turns out to be pretty good. We select DCSMM68.DAT, requiring 4% shade, 3% NPV and 92% soil. The only negative is a higher RMS of 0.019. We can try a second model setting the RMSE weight to 10. In this case we select J9009181. However, this model also requires a slight negative GV fraction, 105% soil and no shade. DCSMM68.DAT is the better model. To summarize, we have one GV, NPV and Soil that appear to be reasonable for all of the image endmembers including CCEOLSTCK (GV), u9litt01-av (NPV) and DCSMM68.DAT (Soil).

Mixing Tools

Run SMA/MESMA Once you have developed a spectral library, you can use the Run SMA/MESMA option to run a range of spectral mixture models. This option gives you the ability to run a simple mixture model by selecting only one model from a spectral library or libraries, or to run multiple mixture models (MESMA) by selecting more than one model . It also gives you the option of constraining output based on fractions, RMSE or residuals. In the


54

case of multiple models, the best single model for each pixel will be selected based on minimum RMSE. The SMA/MESMA module includes two shade options, photometric shade, in which the shade defaults to zero reflectance, or non-photometric shade, which enables a user to provide a shade spectrum from a spectral library. Most data types (byte, integer, floating point) and file times (BIL, BSQ, BIP) are allowed. Reflectance scale factors of 1, 1000 or 10000 are allowed. For original DN or radiance data chose a gain factor of 1. When running this tool, several items should be noted: 1) The number of bands and the band positions of the image and all spectral libraries used must be an exact match. ENVI bad bands lists may be used, and the final set of bands used for the analysis will be only those bands which are listed as ‘good’ in ALL input files (image and libraries). The user will be notified if inconsistencies in the number of bands or the selected bad bands between images. The user will not be notified if the wavelength of band centers differ, so extra care should be taken to ensure that spectra have been properly subset and/or convolved to a common set of wavelengths. 2) It is possible to run the same model on multiple images by selecting multiple, rather than a single, input file. When doing this, a user should turn off the option “Open images for Viewing” to minimize clutter and computational overhead. Output files will be assigned automatically in this case rather than being assigned by the user. To run SMA/MESMA do the following: Step 1: Click on Run SMA/MESMA on the main VIPER Tools menu. The following screen will open


55

Step 2: Select the image or images that will be unmixed. For selecting multiple images use the CTL and SHIFT keys within the select files widget in the typical Windows manner. Step 3: Select the reflectance scale factor (1, 1000, 10000) using a value of 1 for unscaled reflectance, DN or radiance data. Step 4: Select spectral library or libraries by clicking the ‘browse’ button and using the file selection widget. Up to 3 spectral libraries can be selected, and the number of libraries selected will determine the number of endmembers in each model. If one library is selected then one or more 2-endmember models (1 non-shade endmember + shade) can be run. If two libraries are selected then one or more 3-endmember models (2 non-shade endmembers + shade) can be run. If three libraries are selected then one or more 4-endmember models (3 non-shade endmembers + shade) can be run. For each library, select the desired spectra by clicking the ‘Subset spectra’. For single model SMA, select only a single endmember for each selected library. For MESMA, select one or more endmembers from each library. Pay attention to the number of models which will be run. This is calculated each time a new library or spectral subset is selected and is displayed in the box below the label that reads ‘Total number of models:’ Since each pixel in the image must be assessed iteratively with each model, computation costs must be a consideration in designing the scope of the analysis. Consider starting with a reduced subset of endmembers and a reduced image when beginning a new analysis.


56

Note, it is important that you realize that VIPER Tools expects you to enter spectral libraries in order. Thus, if your intent is to run a three endmember model with one endmember as Shade, you would browse first for the first library, then the second one and so on. If you try to run a 3 endmember model, but assign names for Library 1 and 3 with 4 (as shade), the program will crash. Step 5: Select the type of shade that should be used using the toggle buttons. Either photometric shade or non- photometric shade can be used for reflectance data while only non- photometric shade should be used for DN or radiance data. If non- photometric shade is to be used, first select the spectral library containing the shade endmember, then select the single non- photometric shade endmember from the dropdown list. Step 6: Select the constraints to be used. Optional constraints include minimum endmember fraction, maximum endmember fraction, maximum shade fraction, maximum RMSE and residual constraints: Minimum endmember fraction: the lowest allowable fraction value. This can range from -0.5 to 1. Maximum endmember fraction: the highest allowable fraction value. This can range from 1 to 1.5. Maximum shade fraction: the highest allowable fraction value. This can range from 0 to 1. Setting this can help to ensure that water and deep shadows remain unmodeled. RMSE constraint: the maximum allowable model RMSE. Residual constraints: The residual constraint is composed of two components, a constraint threshold and a number of consecutive bands upon which to apply the constraint. Selecting a threshold of 0.025 and a count of 7, for example would exclude all models which produce residual spectra that exceed 0.025 for 7 consecutive bands. Step 7: Select output options. The default outputs are an SMA image which contains endmember fraction(s), shade fraction, model number and model RMSE. An additional default output is metadata file (CSV format) that reports the input and output parameters lists the endmembers comprising each model and lists the proportion of the image modeled by each successful model. Output options include the creation of a classification image, automatic opening of output images in ENVI and creation of a residuals image:

Classification image: The classification image is a single band ENVI classification image in which each class is a model number and class colors are created from a color ramp. This is useful for interpretation of SMA results if you are running more than one mixture model on the same image (i.e., MESMA).


57

When a single model is run this result is meaningless as it contains only two classes (modeled and unmodeled), so this option should be unselected. Automatic opening of images in ENVI: Selecting this option instructs VIPER Tools to load the output SMA and classification image(s) into ENVI’s memory and place the image you create in the Available Bands List of or just save it to a file. In most cases we would leave this checked on. However, it is possible to apply the same model to literally hundreds of images by selecting multiple images to unmix (see step 2 above). To reduce memory demands and clutter in the available bands list deselect this option if more than a few images are being processed. Residuals Image: By default this box is unclicked as residuals images are large files and therefore require extra storage and incure computational costs to write them to file. Residual images contain the model residual spectra stored in floating point data type. For each pixel the residual spectrum is calculated as the image spectrum minus the model spectrum, where the model is the best fit model for that pixel as determined by minimum RMSE. Because the residuals image has the same number of bands as the input image and is a floating point image it is as large or larger than the input image. In the case of a byte input image, the residuals image would be 4 times as larger, for example. Residual images should only be created, therefore, when the user is interested in studying residuals or using them to identify shortcomings of a spectral library. Bad bands will be present in the residual image, with their data values will be set to 0.

Step 8: Select output filename. The recommended naming convention is to add a ‘_sma’ to the end of the input image name, and to optionally add the constraint parameters used (e.g., ‘image_sma_-5_105_2pt5’ would be used for an input file named ‘image’ and constraints of -5%, 105% and 2.5%. Note that ‘pt’ is used instead of ‘.’ to avoid conflicts with filename parsing on the ‘.’ elsewhere in VIPER Tools). If VIPER Tools is unmixing multiple images, it will automatically generate output file names for you. Step 9: Click ‘Run’ to begin processing. Note that if large images and/or a large number of models are used, this may require a long time to complete. Additional options: The ‘Save control file’ button allows the user to save all selected settings to a VIPER Tools SMA/MESMA control file (*.csv). This option is included to capture complicated settings which are time-consuming to input and prone to errors when reproducing. This allows the user to reload the exact same model parameters at a later date to either modify them or to apply them to a different image. The ‘Restore control file’ simply restores a saved control file. Tutorial:


58

For the following examples, we will run several models using the spectral library we previously built for the Santa Barbara front range. We will start with the spectra we selected using CRES from the westusa GV, NPV and Soil libraries (010627_westusa_gv.sli, 010627_westusa_npv.sli, 010627_westusa_soil.sli). After that we will work with viper_tutorial_train_sort_optimal.sli, starting with a series of 2em models, then working up to higher levels of complexity. We will apply these libraries to the AVIRIS image (010614r4_4-5_rfl_reg). For three endmember models we will try a veg-npv mix. We will conclude with a four endmember model with veg-grass-soil. For all MESMA runs we will use standard constraints of 0.025 RMSE, -0.05 and 1.05 minimum and maximum fraction constraints and an 0.80 shade constraint. MESMA outputs a csv file that reports that number of successful models and several images including fractions and a classification image (if selected). We will view these as we progress through this section of the manual.

Running Simple SMA with no fraction constraints To run Simple SMA, you can run up to four endmembers. Most often users will model GV, NPV, Soil and Shade, but you are free to experiment with other endmember combinations (say muddy water vs clean water or impervious vs soil). In most cases, we prefer to run models with only one constraint on the fractions, namely that they sum to 1. Negative or super positive fractions are allowed. To do this, click on the Run SMA/MESMA menu option. You will see a fairly complicated looking screen. You need to provide VIPER Tools with at least four pieces of information, the images(s) to unmix, the scaling factor, at least one spectral library and output file name. In this example, I used the browse button to open 010614r4_4-5.rfl.reg, and have opened the three westusa libraries (010627_westusa_gv.sli, 010627_westusa_npv.sli, 010627_westusa_soil.sli). I have selected CCEOLSTCK for the GV endmember, u9litt01 for NPV and DCSMM68.DAT. I am using a scaling factor of 1000 and will create an out file viper_tutorial_simple_sma.


59

For this example, I will neither Save or Restore a Control file. To run my model, I click turn off the Classification image, leave on Open files for Viewing and click Run. As the model runs, you will see a little progress box. Once the model is done, a metafile is created and opened in notepad. By convention, we add the suffix “_meta.csv” to this file. Thus for my example, VIPER Tools created “viper_tutorial_simple_sma_meta.csv.” You should now be able to open the file, look at the fractions and RMSE. You will note that the fraction image looks good, but has a fairly large number of super positive and negative fractions. The NPV and GV fractions are most reasonable in senesced grasslands and chamise, but tend towards low to negative NPV in riparian areas and oak forests (Clearly a limitation of CCEOLSTCK). The RMSE is generally low (often less than 1%), but highest in urban areas (often over 2.5%). Given that our best model selections were for chamise, grassland and soils, these results are not unexpected. To better model oaks, we would need a better reference endmember than the ones we had in the reference library.


60

Running MESMA for a two endmember model We will now re-run the model. In this case, we will only load one spectral library at a time. However, we will run all possible models in the spectral library. We will use the library we created using the EMC viewer, viper_tutorial_train_sort_optimal.sli .We will also constrain our results using standard sets of constraints, including -0.05, 1.05 fraction constraints, a 0.025 RMSE constraint and 0.80 shade constraint. The latter is useful for screening out very dark pixels, which in this case include 27% of the image which is zero reflectance edge. We will click on the Classification image. Our output image will be viper_tutorial_2em_sma

In this example, we ran 80 2 endmember models unmixing every pixel in the image as a combination of a bright endmember and shade. Looking at the meta file, we can see that these 80 models captured a vast majority of the image, unmixing 67% of the 415220 pixels. Of the 137,689 pixels left unmodeled, most


61

of these are edge pixels. Looking at the metadata output file we see that no model is dominant, although several account for only a small portion of the image. If we flag those models that account for less than one tenth of a percent of the image (< 420 pixels), this equals 15 models, or about one in five (shown in red below). We also see some models that are quite prevalent, accounting for over 10,000 pixels each. These are marked in blue. A quick look at the metadata on the latter models illustrates one of the challenges with MESMA. While models 3 and 7 are adfa, and thus would be expected to be common, model 14 is argl which should not be common and model 33 is cesp. Argl is overmapped in large part because it models urban areas well. Cesp is potentially overmodeled. However, looking at the classification image, it is also mostly restricted to more mesic (wet sites), which fits the ecology of this species. One model that stands out as clearly overmapping is 56, a senescing grass. Looking at the classification image, this model maps edge pixels throughout the image, suggesting that it models mixtures of at least three endmembers well. The most common urban model is also a mixture between urban trees and other materials (73). I have added the class name to the table. I have made likely candidates for three endmember models bold. I have made these selections, in part, by choosing abundant spectra, and in part based on how mixed the spectra appeared.


62

model# #pixels % of image model# #pixels

% of image

1 1522 0.36897 adfa 40 183 0.044364 dgrass 2 564 0.136727 adfa 41 373 0.090424 dgrass 3 7565 1.833939 adfa 43 1022 0.247758 quag 4 2768 0.67103 adfa 44 1066 0.258424 quag 5 7469 1.810667 adfa 45 1780 0.431515 quag 6 14626 3.545697 adfa 46 972 0.235636 quag 7 7094 1.719758 adfa 47 302 0.073212 quag 8 7094 1.719758 adfa 48 1077 0.261091 quag 9 9314 2.257939 adfa 49 1315 0.318788 quag

10 1056 0.256 argl 50 1581 0.383273 quag 11 2113 0.512242 argl 51 5817 1.410182 quag 12 1102 0.267152 argl 52 4264 1.033697 qudu 13 840 0.203636 argl 53 542 0.131394 qudu 14 15949 3.866424 argl 54 5260 1.275152 qudu 16 1086 0.263273 argl 55 7144 1.731879 qudu 17 5086 1.23297 argl 56 23315 5.652121 sgrass 18 2335 0.566061 cecu 57 260 0.06303 sgrass 19 3081 0.746909 ceme 58 195 0.047273 sgrass 20 3823 0.926788 ceme 59 126 0.030546 sgrass 21 2730 0.661818 ceme 60 288 0.069818 sgrass 22 2761 0.669333 ceme 61 3782 0.916849 sgrass 23 4998 1.211636 ceme 62 508 0.123152 sgrass 24 9044 2.192485 ceme 63 251 0.060849 sgrass 25 4136 1.002667 ceme 64 434 0.105212 sgrass 26 6362 1.542303 ceme 65 5925 1.436364 soil 27 4434 1.074909 ceme 66 2078 0.503758 soil 28 1684 0.408242 cesp 67 66 0.016 soil 29 1087 0.263515 cesp 68 44 0.010667 soil 30 4135 1.002424 cesp 69 137 0.033212 soil 31 1345 0.326061 cesp 70 133 0.032242 soil 32 5670 1.374546 cesp 71 612 0.148364 soil 33 17557 4.256243 cesp 72 632 0.153212 urban 34 11522 2.793212 cesp 73 19311 4.681455 urban 35 250 0.060606 dgrass 74 1917 0.464727 urban 36 304 0.073697 dgrass 75 371 0.089939 urban 37 3217 0.779879 dgrass 76 1551 0.376 urban 38 159 0.038546 dgrass 77 830 0.201212 urban 39 86 0.020849 dgrass 79 3379 0.819152 urban

A simple display of the classification image illustrates considerable clustering of models and that many models appear to have modeled the correct material. Good examples include urban areas (orange) and grasslands (pale green and yellow). Ceanothus is mapped correctly to the generic level, but cesp is mapped in place of ceme in many places. Very bright vegetation was not modeled. An obvious next step would be to aggregate models into their original classes and develop a new color scheme (i.e., lump


63

all adfa models into a single adfa class). Several classes are obviously overmapped, include cesp, argl and qudu. This illustrates some of the dangers of using generalist spectra. Overall, this is a very good map.

Figure showing two endmember models selected. The fraction image produces high fractions in most natural vegetation, and lower fractions in most urban areas due to the presence of numerous dark surfaces.

Running MESMA for a three endmember model For a three endmember model, we would use the same approach but add a second library. To avoid running an excessive number of models, we will select a subset of models from the previous example that mapped the correct materials and also covered a reasonable amount of area. For adfa this translates to three models (3, 6 & 7 on the list above). For argl we will choose model 14. For Ceanothus we chose three models for ceme, 23, 24 and 26 22 and two models for cesp, 34 and 35. We will skip cecu and qudu (neither are common in the area) and use model 51 for quag. For the NPV component we will use three senesced grassland spectra (41, 43, and 44). For a three endmember model using soil, we would do well with 65, 66 and 71. We conclude with three urban models, 74, 76 and 79.


64

These selections translate to the following models Model # Name Class Category

3 EVF:56_trn_X521_Y65 adfa gv 6 EVF:57_trn_X530_Y92 adfa gv 7 EVF:70_trn_X665_Y80 adfa gv

14 EVF:52_trn_X499_Y116 argl gv 23 EVF:19_trn_X428_Y100 ceme gv 24 EVF:48_trn_X467_Y122 ceme gv 26 EVF:48_trn_X467_Y119 ceme gv 33 EVF:sb_poly_104_trn_X105_Y138 cesp gv 34 EVF:sb_poly_104_trn_X108_Y136 cesp gv 36 EVF:131_trn_X123_Y223 dgrass npv 37 EVF:132_trn_X126_Y217 dgrass npv 41 EVF:134_trn_X124_Y211 dgrass npv 51 EVF:86_trn_X488_Y327 quag gv 65 EVF:119_trn_X676_Y338 soil soil 66 EVF:115_trn_X134_Y265 soil soil 71 EVF:116_trn_X108_Y239 soil soil 74 EVF:143_trn_X409_Y441 urban soil 76 EVF:146_trn_X440_Y463 urban soil 79 EVF:148_trn_X321_Y395 urban soil

For our three endmember model we select 10 gv endmembers and 3 npv endmembers, totaling 24 models. After selecting spectra, I have saved a control file called viper_tutorial_3em_gv_npv_sma_ctl. A subset of the meta file is shown below: model#, 1st_EM, 2nd_EM, 3rd_EM 1, EVF:56_trn_X521_Y65, EVF:131_trn_X123_Y223, shade 2, EVF:56_trn_X521_Y65, EVF:132_trn_X126_Y217, shade 3, EVF:56_trn_X521_Y65, EVF:134_trn_X124_Y211, shade 4, EVF:57_trn_X530_Y92, EVF:131_trn_X123_Y223, shade 5, EVF:57_trn_X530_Y92, EVF:132_trn_X126_Y217, shade 6, EVF:57_trn_X530_Y92, EVF:134_trn_X124_Y211, shade 7, EVF:70_trn_X665_Y80, EVF:131_trn_X123_Y223, shade 8, EVF:70_trn_X665_Y80, EVF:132_trn_X126_Y217, shade 9, EVF:70_trn_X665_Y80, EVF:134_trn_X124_Y211, shade 10, EVF:52_trn_X499_Y123, EVF:131_trn_X123_Y223, shade 11, EVF:52_trn_X499_Y123, EVF:132_trn_X126_Y217, shade 12, EVF:52_trn_X499_Y123, EVF:134_trn_X124_Y211, shade 13, EVF:19_trn_X428_Y100, EVF:131_trn_X123_Y223, shade 14, EVF:19_trn_X428_Y100, EVF:132_trn_X126_Y217, shade 15, EVF:19_trn_X428_Y100, EVF:134_trn_X124_Y211, shade 16, EVF:48_trn_X467_Y122, EVF:131_trn_X123_Y223, shade 17, EVF:48_trn_X467_Y122, EVF:132_trn_X126_Y217, shade 18, EVF:48_trn_X467_Y122, EVF:134_trn_X124_Y211, shade


65

19, EVF:48_trn_X467_Y119, EVF:131_trn_X123_Y223, shade 20, EVF:48_trn_X467_Y119, EVF:132_trn_X126_Y217, shade 21, EVF:48_trn_X467_Y119, EVF:134_trn_X124_Y211, shade 22, EVF:sb_poly_104_trn_X105_Y138, EVF:131_trn_X123_Y223, shade 23, EVF:sb_poly_104_trn_X105_Y138, EVF:132_trn_X126_Y217, shade 24, EVF:sb_poly_104_trn_X105_Y138, EVF:134_trn_X124_Y211, shade 25, EVF:sb_poly_104_trn_X108_Y136, EVF:131_trn_X123_Y223, shade 26, EVF:sb_poly_104_trn_X108_Y136, EVF:132_trn_X126_Y217, shade 27, EVF:sb_poly_104_trn_X108_Y136, EVF:134_trn_X124_Y211, shade 28, EVF:86_trn_X488_Y327, EVF:131_trn_X123_Y223, shade 29, EVF:86_trn_X488_Y327, EVF:132_trn_X126_Y217, shade 30, EVF:86_trn_X488_Y327, EVF:134_trn_X124_Y211, shade Unclassified pixels: #pixels, % of image 165682, 40.1653328 Number of pixels mapped for each model: model#, #pixels, % of image 1, 1995, 0.4836364 2, 1871, 0.4535758 3, 1569, 0.3803636 4, 1580, 0.3830303 5, 2411, 0.5844849 6, 2799, 0.6785455 7, 481, 0.1166061 8, 639, 0.1549091 9, 1568, 0.3801212 10, 1381, 0.3347879 11, 3840, 0.9309090 12, 39863, 9.6637573 13, 5315, 1.2884849 14, 13663, 3.3122425 15, 3414, 0.8276364 16, 15421, 3.7384243 17, 8728, 2.1158788 18, 10948, 2.6540606 19, 1271, 0.3081212 20, 2232, 0.5410909 21, 511, 0.1238788 22, 3300, 0.8000000 23, 7036, 1.7056969 24, 18653, 4.5219398 25, 15967, 3.8707879 26, 24899, 6.0361214 27, 31820, 7.7139392 28, 2660, 0.6448485 29, 8092, 1.9616969 30, 12891, 3.1250908


66

As we can see, 30 three endmember models mapped slightly less area than our 80 2 endmember models. By comparing the two images, we can see that the majority of unmodeled pixels fall in urban areas, which could neither be modeled as NPV or GV or a mixture of the two. A few areas in the mountains (shown by the top red arrow) could not be modeled with 3 ems, most likely because they were relatively homogeneous and failed the fraction constraints. If we look carefully, we can also see areas that were not modeled by 2 endmembers, but were by three endmembers. These seem to be concentrated in the foot hills and along edges.

Figure showing models selected for a three-endmember model. We will conclude with a four endmember model, in which we include a soil/urban endmember. We run a total of 10*3*6 = 180 models. The number of models is rather large, so we will not include all of them. We see from the meta data that four endmember models model about as much area as three endmember models (197613 unclassified, or 47% of the image). We also see that there are some models that are chosen often. These include:


67

Model Npix Percent Class1 Class2 Class3 Model Name Npix Percent Class1 Class2

3 1130 0.273939 adfa dgrass soil 142 1475 0.3576 cesp dgrass urban 12 1557 0.377455 adfa dgrass urban 143 1348 0.3268 cesp dgrass urban 74 1070 0.259394 ceme dgrass soil 144 1041 0.2524 cesp dgrass urban 77 1253 0.303758 ceme dgrass urban 146 4727 1.1459 cesp dgrass soil 78 1689 0.409455 ceme dgrass urban 147 6018 1.4589 cesp dgrass soil 80 2737 0.663515 ceme dgrass soil 148 5024 1.2179 cesp dgrass urban 81 2363 0.572849 ceme dgrass soil 149 6503 1.5765 cesp dgrass urban 82 1657 0.401697 ceme dgrass urban 150 5345 1.2958 cesp dgrass urban 83 2347 0.56897 ceme dgrass urban 151 1182 0.2865 cesp dgrass soil 84 5083 1.232242 ceme dgrass urban 152 8773 2.1268 cesp dgrass soil 90 1323 0.320727 ceme dgrass urban 153 7478 1.8128 cesp dgrass soil 92 3100 0.751515 ceme dgrass soil 154 7384 1.7901 cesp dgrass urban 93 4475 1.084849 ceme dgrass soil 155 10961 2.6572 cesp dgrass urban 94 1935 0.469091 ceme dgrass urban 156 9391 2.2766 cesp dgrass urban 95 3215 0.779394 ceme dgrass urban 158 4082 0.9896 cesp dgrass soil 96 3786 0.917818 ceme dgrass urban 159 2575 0.6242 cesp dgrass soil 98 3079 0.746424 ceme dgrass soil 160 7332 1.7775 cesp dgrass urban 99 3113 0.754667 ceme dgrass soil 161 7291 1.7675 cesp dgrass urban

100 1451 0.351758 ceme dgrass urban 162 4642 1.1253 cesp dgrass urban 101 2357 0.571394 ceme dgrass urban 164 1017 0.2465 quag dgrass soil 102 2737 0.663515 ceme dgrass urban 165 2797 0.6781 quag dgrass soil 104 1080 0.261818 ceme dgrass soil 166 1868 0.4528 quag dgrass urban 105 1092 0.264727 ceme dgrass soil 167 2547 0.6175 quag dgrass urban 107 1447 0.350788 ceme dgrass urban 168 2112 0.512 quag dgrass urban 108 1164 0.282182 ceme dgrass urban 170 1489 0.361 quag dgrass soil 129 1471 0.356606 cesp dgrass soil 171 1541 0.3736 quag dgrass soil 130 1926 0.466909 cesp dgrass urban 172 2623 0.6359 quag dgrass urban 131 1102 0.267152 cesp dgrass urban 173 3169 0.7682 quag dgrass urban 132 1715 0.415758 cesp dgrass urban 174 4040 0.9794 quag dgrass urban 134 1021 0.247515 cesp dgrass soil 176 2921 0.7081 quag dgrass soil 135 1267 0.307152 cesp dgrass soil 177 1125 0.2727 quag dgrass soil 136 2868 0.695273 cesp dgrass urban 178 2767 0.6708 quag dgrass urban 137 1966 0.476606 cesp dgrass urban 179 5763 1.3971 quag dgrass urban 138 3390 0.821818 cesp dgrass urban 180 2319 0.5622 quag dgrass urban

Looking at the models, we see that the four endmember models are dominated by the broadleaf evergreen shrubs Ceanothus and coast live oak (quag). Only two chamise models have more than 1000 pixels. No manzanita (argl) model had over 1000 pixels. These results make a great deal of sense, since both adfa and argl tend to form mixtures at canopy scales, while Ceanothus and quag can be modeled as leaves and shadows. Looking at the image, we see that the four endmember model fills in more of the urban areas than either the three or two endmember models. Furthermore, many of the riparian areas in the mountains, which were modeled by two or three endmember models, fail with four endmember models. The fact that urban areas are best modeled with four


68

endmembers is consistent with work by Powell (2006), who found that natural systems are best modeled by two endmembers, disturbed landscapes three, and urban areas, four.

Figure showing models selected using a four-endmember model. We can display a fraction image as well. Here we have loaded NPV as red, GV as green and Soil/Urban as blue. We see that the fractions are distributed as we might expect. Furthermore, we know that they have met all of the model constraints. By combining the two, three and four endmember models only the brightest vegetation areas and water would remain unclassified.


69

Figure showing fraction images for NPV, GV and Soil as RGB.

Shade Normalize SMA Results Shade normalization can be performed on SMA fraction images to assess relative abundance of surface materials with the shade component removed. For example, assume we are interested in the relative abundance of green vegetation in a semi-arid region with high topographic relief. In the standard SMA outputs the contribution of topographic shade may cause some confusion when interpreting the green vegetation abundance values. If comparing the green vegetation fractions between a region with low local incidence angle and a region with high local incidence angle, the region of low local incidence will show a higher green vegetation fraction due to the reduced shade component, even if the green vegetation abundance is the same as the other site. This effect can be removed by shade normalizing the SMA output images. To shade normalize VIPER Tools SMA/MESMA output images select ‘Shade Normalize SMA Results’ from the main VIPER Tools menu. This will invoke the shade normalization module and will open the following GUI:


70

Select the input SMA image and an output image filename. The output image will contain one less band the input image since the ‘Shade’ band will no longer be present. The endmember fractions for each pixel will sum to 1. Tutorial: In this example, we will use the output image from CRES (viper_tutorial_simple_sma) as our input image and output a shade normalized file, viper_tutorial_simple_sma.sn. The figures below show the original image and shade normalized version.

Figure showing a simple mixing model with NPV, GV and soil as RGB.


71

SMA results for a simple mixture model developed using CRES. Red is NPV, Green is GV and Blue is soil.

Figure showing Shade normalized results for the simple mixing model shown previously.

Terrain Correction of SMA Results (*BETA*) SMA fraction images can be processed to remove some of the effects of topographic shading using the Terrain Correction Module. This module uses an SMA fraction image, a Digital Elevation Model (DEM) and training locations to develop an empirical relationship between shade fraction and local incidence angle. This relationship can then be used to back out the topographic shade from the SMA Fraction Image, leaving only the canopy shadow component. Before beginning the user must create a DEM that has the same spatial resolution and spatial extent as the SMA Fraction Image to be processed. Next a set of training data must be selected using ENVI ROIs. The goal when selecting training data is to select sites with consistent canopy shadow so that differences in shade fraction can be attributed to differences in the local solar geometry, rather than to differences in canopy shadow. Examples of suitable training sites might be uniform grasslands or young homogeneous conifer forests.


72

The Terrain Correction module can be used to terrain correct any imagery, not just SMA fraction images. For example, a reflectance image could be terrain corrected prior to SMA or image classification to minimize the effects of topography. Studies are currently being conducted to assess the benefits of terrain correction before versus after conducting SMA.

Running Terrain Correction on SMA Fraction Image Begin by openning the Terrain Correction module clicking ‘Terrain Correct SMA Results’ on the main VIPER Tools menu bar.

Step 1: Select the ‘Generate Local Incidence’ tab to generate a local incidence angle image based on the local topography and the solar geometry at the time of the data collect. If a local incidence angle has already been calculated from a previous analysis skip to Step 3. If a local incidence angle has not yet been calculated, select the coregistered DEM image. This DEM image must have the exact same spatial extent and resolution as the image that will be processed. Enter in the solar elevation (in degrees above the horizon) and the solar azimuth. This information is sometimes provided in the header of satellite data. If these values are unknown click the ‘Calculate Solar Geometry’ button. This will open the window below:


73

Click ‘Compute Sun Elevation and Azimuth’ which will launch the next window:

Enter in the date and time that the imagery were collected, and the Latitude and Longitude in degrees, minutes and seconds. Click ‘OK’ to compute geometry then click ‘OK’ again to store these values in main window. Select the filename for the output local incidence angle then click ‘Run’ to generate the cosine of local incidence angle image. Step 2: Select training data. Click on tab 2 ‘Select training data tab’ to access the tools for inputting training data. As mentioned above, training data are supplied in the form of ROIs in the image which represent relatively consistent canopy properties. Begin by selecting either ENVI Standard image or VIPER Tools SMA Image using the toggle on top. Next select the filename of the image and the ROI(s) to use for training. Finally, select the cosine of incidence angle image and then click ‘Run’ to proceed. Step 3: To explore the relationship between image bands and the cosine of the incidence angle click on tab 3 ‘Plot training data’. Select one of the image bands from the dropdown list to plot the empirical relationship between that image band and the cosine of the local incidence angle for the training regions selected in tab 2. By alternating between tab 2 and 3 an optimal set of ROIs can be selected which produce the best relationship for modeling and removing topographic shade. It may be necessary to go


74

back to the ENVI ROI tool and redefine ROIs or to define new ROIs in order to achieve a satisfactory relationship between image response or shade and incidence angle. Note the OLS linear regression equation, coefficient of determination (Rsquare), sample size (n) and p-value (p) which can be useful in assessing the strength of the regression. Step 4: Once a satisfactory relationship between image data and the cosine of the local incidence angle has been established, proceed to tab 4 ‘Apply Correction’. Enter in the output file name and click ‘Apply’ to generate the terrain corrected file. Step 5: Carefully assess the quality of the terrain corrected image and repeat steps above with new or additional training sites if necessary.

References Adams, JB, Smith, MO and Gillespie, AR .1993. Imaging spectrometry: interpretation

based on spectral mixture analysis, In Pieters CM, Englert P, editors. Remote geochemical analysis: elemental and mineralogical composition. New York: Cambridge Univ. Press; 1993: p145-166. (Vol. 7).

Asner, G.P., M.M.C. Bustamante, and A.R. Townsend. 2003. Scale dependence of biophysical structure in deforested lands bordering the Tapajós National Forest, Central Amazon. Remote Sensing of Environment 87:507-520.

Asner, G.P., and Lobell, D. B., 2000, A biogeophysical approach for automated SWIR unmixing of Soils and Vegetation, Remote Sens. Environ., 74, 99-112.

Ballantine, J-A C., Okin, G.S., Prentiss, D.E., and Roberts, D.A. 2005, Mapping African landforms using continental scale unmixing of MODIS imagery, Remote Sensing of Environment, 97, 470-483.

Bateson, A., and Curtiss, B., 1996, A method for manual endmember selection and spectral unmixing, Remote Sens. Environ., 55, 229-243.

Bateson, C.A., Asner, G.P., and Wessman, C., 2000, Endmember Bundles: A new approach to incorporating endmember variability into spectral mixture analysis. IEEE Trans. Geosci. Remote Sens., 38, 1083-1094.

Boardman, J.W., Kruse, F.A., and Green, R.O., 1995, Mapping target signatures via partial unmixing of AVIRIS data in Summaries of the 5th JPL Airborne Earth Science Workshop, JPL Publication 95-1, Vol 1, Jet Propulsion Laboratory, Pasadena, CA pp. 23-26.

Clark, M., 2005, An assessment of Hyperspectral and Lidar Remote Sensing for the Monitoring of Tropical Rain Forest Trees, PhD Dissertation, University of California, Santa Barbara, 319 pp.

Collins, E.F., Roberts, D.A., and Borel, C.C., 2001, Spectral Mixture Analysis of Simulated Thermal Infrared Spectrometry Data: an Initial Temperature Estimate Bounded TESSMA Search Approach, IEEE Trans. Geosci. Remote Sens 39(7) 1435-1446.


75

Dennison, P.E., Charoensiri, K., Roberts, D.A., Peterson, S.H., and Green, R.O., 2006, Wildfire temperature and Land Cover Modeling Using Hyperspectral Data, Remote Sensing of Environment, 100, 212-222.

Dennison, P.E., Halligan, K. Q., Roberts, D.A., 2004, A Comparison of Error Metrics and Constraints for Multiple Endmember Spectral Mixture Analysis and Spectral Angle Mapper, Remote Sens. Environ. 93, 359-367.

Dennison, P.E., and Roberts, D.A.,2003a, Endmember Selection for Multiple Endmember Spectral Mixture Analysis using Endmember Average RSME, Remote Sens. Environ., 87(2-3), 123-135.

Dennison, P.E., and Roberts, D.A., 2003b, The Effects of Vegetation Phenology on Endmember Selection and Species Mapping in Southern California Chaparral, Remote Sens. Environ., 87(2-3), 123-135.

Dennison, P.E., Roberts, D.A., and Peterson, S.H., 2007, Spectral shape-based temporal compositing algorithms for MODIS surface reflectance data. Remote Sens. Environ. 109, 510-522

Elmore, A.J., Mustard, J.F., Manning, S.J., and Lobell, D.B., 2000, Quantifying vegetation change in semiarid environments: precision and accuracy of spectral mixture analysis and the normalized difference vegetation index, Remote Sens. Environ., 73, 87-102.

Gardner, M., 1997, Mapping chaparral with AVIRIS using Advanced Remote Sensing Techniques, Masters Thesis, University of California, Santa Barbara, 58 pp.

Green, R.O., Conel, J.E. and Roberts, D.A., 1993, Estimation of Aerosol Optical Depth, Pressure Elevation, Water Vapor and Calculation of Apparent Surface Reflectance from Radiance Measured by the Airborne Visible-Infrared Imaging Spectrometer (AVIRIS) Using MODTRAN2, SPIE Conf. 1937, Imaging Spectrometry of the Terrestrial Environment, 2-5.

Halligan, K.Q., 2002, Multiple Endmember Spectral Mixture Analysis of Vegetation in the Northeast Corner of Yellowstone National Park, Masters Thesis, University of California, Santa Barbara, 64 pp.

Herold, M., Gardner, M., and Roberts, D.A., 2003, Spectral Resolution Requirements for Mapping Urban Areas, IEEE Trans. Geosci. Remote Sens. 41 (9 Part 1): 1907-1919.

Herold, M., Roberts, D.A., Gardner, M.E. and Dennison, P.E., 2004, Spectrometry for urban area remote sensing – Development and analysis of a spectral library from 350 to 2400 nm, Remote Sens. Environ. Vol 91 (3-4) 304-319.

Johnson, J.R., Staid, M.I., Titus, T.N., and Becker, K., 2006, Shocked plagioclase signatures in Thermal Emission Spectrometer Data of Mars, Icarus, 180: 60-74.

Li, L., and Mustard, J.F., 2003, Highland contamination in lunar mare soils: Improved mapping with multiple end-member spectral mixture analysis, J. Geophys. Res. 108, doi: 10.1029/2002JE001917.5033

Okin, G.S., Roberts, D.A., Murray, B., and Okin, W.J., 2000, Practical limits on hyperspectral vegetation discrimination in arid and semiarid environments, Remote Sens. Env., 77(2): 212-225.


76

Painter, T.H., Dozier, J., Roberts, D.A., Davis, R.E., and Green, R.O., 2003, Retrieval of Subpixel Snow-covered Area and Grain Size from Imaging Spectrometer Data, Remote Sens. Environ., 85, 64-77.

Painter, T.H., Roberts, D.A., Green, R.O., and Dozier, J., 1998, The Effect of Grain Size on Spectral Mixture Analysis of Snow-covered Area from AVIRIS Data, Rem. Sens. Environ. 65:320-332.

Powell, R., 2006, Long-Term Monitoring of Urbanization in the Brazilian Amazon Using Remote Sensing, Unpublished UCSB PhD Dissertation, 257 p

Powell, R., D.A. Roberts, P.E. Dennison and L.L. Hess, 2007. Sub-pixel mapping of urban land cover using multiple endmember spectral mixture analysis: Manaus, Brazil, Remote Sens. Environ., 106: 253-267

Rashed, T., Weeks, J.R., Roberts, D., Rogan, J., and Powell, R., 2003, Measuring the Physical Composition of Urban Morphology using Multiple Endmember Spectral Mixture Models, Photogramm. Eng. Remote Sens. 69(9) 1011-1020.

Roberts, D.A., Adams, J.B., and Smith, M.O., 1993, Discriminating Green Vegetation, Non-Photosynthetic Vegetation and Soils in AVIRIS Data, Rem. Sens. Environ., 44: 2/3 255-270.

Roberts, D.A., Batista, G., Pereira, J., Waller, E., and Nelson, B. 1998, Change Identification using Multitemporal Spectral Mixture Analysis: Applications in Eastern Amazonia, Chapter 9 in Remote Sensing Change Detection: Environmental Monitoring Applications and Methods, (Elvidge, C. and Lunetta R., Eds.), Ann Arbor Press, Ann Arbor, MI, pp. 137-161.

Roberts, D.A., Brown, K.J., Green, R., Ustin, S., and Hinckley, T., 1998, Investigating the relationship between liquid water and leaf area in clonal Populus, Proc. 7th AVIRIS Earth Science Workshop JPL 97-21, Pasadena, CA 91109, 335-344.

Roberts, D.A., Dennison, P.E., Gardner, M., Hetzel, Y., Ustin, S.L., and Lee, C., 2003, Evaluation of the Potential of Hyperion for Fire Danger Assessment by Comparison to the Airborne Visible/Infrared Imaging Spectrometer, IEEE Trans. On Geosci. Remote Sens, 41 (6 Part 1): 1297-1310.

Roberts, D.A., Gardner, M., Church, R., Ustin, S.L., and Green, R.O., 1997, Optimum Strategies for Mapping Vegetation using Multiple Endmember Spectral Mixture Models, in SPIE Conf. Vol 3118, Imaging Spectrometry III, 108-119., San Diego, CA July 27-Aug 1, 1997.

Roberts, D.A., Gardner, M., Church, R., Ustin, S., Scheer, G.,and Green, R.O., 1998, Mapping Chaparral in the Santa Monica Mountains using Multiple Endmember Spectral Mixture Models, Rem. Sens. Environ. 65: 267-279.

Roberts, D.A., Green, R.O., and Adams, J.B., 1997,Temporal and Spatial Patterns in Vegetation and Atmospheric Properties from AVIRIS, Remote Sens. Environ 62: 223-240.

Roberts, D.A. and Herold, M. 2004. Imaging Spectrometry of Urban Materials, in Molecules to Planets: Infrared Spectroscopy in Geochemistry, Exploration Geochemistry and Remote Sensing (P. King, M. Ramsey and G. Swayze, Ed)., Mineral Association of Canada, 155-183.


77

Roberts, D.A., Numata, I., Holmes, K.W., Batista, G., Krug, T., Monteiro, A., Powell, B., and Chadwick, O., 2002, Large area mapping of land-cover change in Rondônia using multitemporal spectral mixture analysis and decision tree classifiers, J. Geophys. Res. Atm., 107 (D20), 8073, LBA 40-1 to 40-18.

Roberts, D.A., Ustin, S.L., Ogunjemiyo, S., Greenberg, J., Dobrowski,S.Z., Chen, J. and Hinckley, T.M., 2004, Spectral and structural measures of Northwest forest vegetation at leaf to landscape scales, Ecosystems, 7:545-562.

Sonnetag, O., Chen, J.M., Roberts, D.A., Talbot, J., Halligan, K.Q., and Govind, A., 2007, Mapping tree and shrub leaf area indices in an ombrotrophic peatland through multiple endmember spectral unmixing, Remote Sens. Environ. 109, 342-360.

Settle, J.J. and Drake, N.A., 1993, Linear mixing and the estimation of ground cover proportions, Int. J. Remote Sens., 14(6), 1159-1177.

Tompkins, S., Mustard J.F., Pieters, C.M., and Forsyth, D.W., 1997, Optimization of endmembers for spectral mixture analysis. Remote Sens. Environ. 59, 472-489.

Ustin, S.L., Roberts, D.A., Gamon, J.A., Asner, G.P., and Green, R.O., 2004, Using Imaging Spectroscopy to Study Ecosystem Processes and Properties, Bioscience, 54(6) 523-534.

Example Applications Overview In the following part we include three sections. In the first section, we provide an overview of spectral libraries we have developed in support of this manual. In the second section, we provide a few brief examples of Viper Tools used to map urban materials. In the final example, we apply Viper Tools to map plant species and functional types in the Pacific Northwest (Wind River). Spectral Libraries Spectral libraries were developed for two general areas and from two majors sources. The first includes the Santa Barbara Urban spectral libraries. These consist of two libraries, one developed from field spectra measured using an ASD Full range spectrometer between May and June, 2001. This library was developed to help us evaluate the potential for imaging spectrometry for discrimination urban surfaces and includes 1089 spectra of roofs, roads, concrete, landscaping materials, paints and other surfaces (see Herold et al., 2004; Roberts and Herold, 2004). Important files include: Sb_all_asd.sli and associated header (*.hdr): ENVI spectral library Sb_all_asd.csv: Metadata file for the spectral library. The original ASD spectra can be found under viper_tools_06\applications\SB\Spectral Libraries\Original ASD


78

The metadata were derived from field notes recorded at the time of acquisition and subsequently stored in an Excel spread sheet by Meg Gardner. In this example, we are employing 6 columns of metadata, plus a column for “source” and “photo”. Source, in this case would be the ASD. The metadata are hierarchical, starting with a division between pervious and impervious, followed by dominant class (ie, water, gv etc), then progressing towards greater levels of specificity (cover type or life form, followed by chemistry of phenology etc).

In this metadata scheme, we have attempted to develop a general approach for a diversity of surfaces. Additional fields I have added include a field for notes (in case the original information is recorded in note books) and spatial scale of source. A second source for urban spectra was derived from a 2000 4 m AVIRIS scene acquired over the cities of Goleta and Santa Barbara (Herold et al., 2003). Files associated with this library include: 000606_av_ref_points.sli (and header) 000606_av_ref_points.csv.


79

The AVIRIS derived spectral library is found in viper_tools_06\applications\SB\Convolved to AVIRIS 2000\Santa Barbara\Reference Points Two associated directories include the field-based spectral library convolved to the AVIRIS 2000 image. The directory Pre-emc-spectra-selection includes spectra consisting of all 224 bands, while Post-EMC-spectra-selection gain adjusted to 1000, contains a spectral subset of 180 bands multiplied by a gain factor of 1000 to match AVIRIS. These directories are likely to be renamed at a future date to simplify them. The Reference Point library consists of 100 spectra of approximately 22 land-cover classes mapped in the field in 2000. These were randomly sampled from larger polygons to create a balanced training data set of 100 pixels for each class. A second set of spectral libraries are included. These were derived from a diversity of sources, including a 2003 AVIRIS high resolution scene over Wind River, field spectra acquired at branch scales in 1999 (See Roberts et al., 2004), ASD field spectra acquired in 1996 either at ground level or from the Wind River Canopy Crane Gondola and a small selection of field spectra acquired from clonal poplars in 1997 (see Roberts et al., 1998b). Washington state spectra are complimented by spectra measured in the laboratory from samples collected in the Santa Monica Mountains using a Varian Cary5 (Roberts et al., 1998a), laboratory spectra of bark, wood, litter and leaves measured using a Beckman DK2a (See Roberts et al., 1997) and a selection of spectra derived from Dr. John Adam’s group at the University of Washington. Where possible, detailed meta-data are included for each spectrum. These spectra are stored in a directory named as follows: viper_tools_06\applications\Washington_state\Spectral Libraries A large number of spectral libraries were generated to support this project including: windriver_all.sli wa99all.sli windriver_all_optimal_2nd.sli wa99all_sub.sli 010627_wa99all_sub_avr.sli WESTUSA_final.sli 030718_wa99all_sub_avr.sli 030718_westusa_final_avr.sli 010627_westusa_final_avr.sli 030718_westusa_final_combined_opt.sli 030718_westusa_final_combined_opt1000.sli030718_westusa_final_avr1000.sli 030718_iem_sub2.sli 030718_iem_sub2_sorted.sli

Windriver_all.sli and its derivatives were derived from AVIRIS data acquired over Wind River in 2003. These data were extracted from approximately 140 polygons distributed over 5 high resolution AVIRIS scenes. Between 3 to 50% of each polygon


80

was sampled depending on size (large polygons were sampled at a lower percentage). This was done in an effort to balance the number of spectra between species. Wa99all.sli and its derivatives are derived from field spectra acquired using an ASD in 1999. This library consists of approximately 500 spectra that cover a majority of the dominant plant species within the Pacific Northwest including replicates. This library also includes some senesced material, soils, gravel, roads and barks. A portion of these spectra were published in Ecosystems (Roberts et al., 2004). Derivatives include spectral subsets (to match Cary-5 spectra acquired between 400 and 2498 nm), and two libraries convolved to 2001 and 2003 AVIRIS. Westusa.sli (WESTUSA) is a combined spectral library consisting of portions of wa99all.sli, and portions of libraries from other sources (as described above). Derived libraries include libraries convolved to match 2001 and 2003 AVIRIS, subsets of optimal spectra selected using ViperTools and libraries multiplied by 1000 to match the gain of AVIRIS reflectance for 2001 and 2003. A final spectral library, 030718_iem_sub2.sli is included. This spectral library was derived from the 2003 Run 7 AVIRIS image and used to select reference endmembers using CRES., Image Analysis: Santa Barbara Urban Case Study The following example illustrates two applications for MESMA to an urban area. The data set we will use is an AVIRIS image acquired by the Twin Otter on June 6, 2000. It was processed to apparent surface reflectance using the Modtran-based retrieval approach described by Green et al., 1993. Minor artifacts in the first pass reflectance were further corrected using a ground spectrum acquired from the tarmack on the Santa Barbara airport. This data set was georectified by Dr. Martin Herold, who subsequently published several papers evaluating spectral and spatial limitations on mapping urban materials (Herold et al., 2003; 2004). There are many problems in which imaging spectrometry is the sensor of choice for urban areas. In this example, we will focus on two research problems in urban areas, mapping impervious surfaces and mapping vegetation cover. The former is critical for quantifying surface runoff and non-point source pollution (Ridd, 1995). Vegetation fractional cover is critical for predicting urban temperatures, ecosystem services (ie, rainfall interception, uptake of carbon) and can be a powerful remotely sensed measure of socioeconomic factors (ie, tree cover tends to increase with property value). Both of these research problems are complicated by the incredible diversity of urban materials, which means that a mixture model has to account for multiple types of vegetation and impervious surfaces. Data used in the analysis included: 000606_aviris.lan: This is a 180 band spectral subset of an AVIRIS mosaic consisting of 1155 lines and and 2003 samples.


81

000606_av_ref_points.sli: This is a spectral library consisting of 2573 spectra of dominant urban materials. The spectra were extracted from the AVIRIS scene from a large number of reference polygons distributed across the image (000606_av_ref_points.roi). 000606_av_ref_points.csv: This is the metadata file for this spectral library. The general analysis procedure included all of the basic steps of Viper Tools including: 1) Create Spectral Library from ROIS 2) Create Metadata for Spectral Library (000606_av_ref_points.csv) 3) Manage Spectral Libraries. We sorted on class and brightness 4) Create Square array (000606_av_ref_points_sqr) 5) Calculate EAR/MASA/CoB At this stage, we departed from the type of analysis used previously. Given our objectives of mapping impervious surfaces and vegetation cover, we did not focus on the material scale. One approach would be to start at the finest level of discrimination, then aggregate classes up to impervious and vegetation cover. An alternative strategy, might be to start at a lower hierarchical level, and sort on broad classes. This is the approach we used, starting with a two class sort on impervious and pervious (level 1), then adding a second level sort on class(GV, NPV, paved, roof, water = level 2). In addition, given that we are up front creating very mixed classes, our selection tool is likely to be different. Given a heterogenous spectral library, neither EAR, nor MASA would be expected to perform well. In this case, In_Cob is likely to be the preferred selection tool. Using the View EMC file, we then selected spectra based on two major criteria: a) Highest In_Cob within a cover category (impervious vs pervious) b) Highest In_Cob within a cover category (class) Where we had ties for In_Cob, we either used Out_Cob (selecting the lowest Out_Cob) or EAR and MASA (Highest In_Cob, lowest EAR and MASA). After building two spectral libraries, we used the Manage Spectral Library tool to merge them into one called 000606_av_ref_points_sort_opt_cob.sli. We selected 4 GV endmembers, 4 NPV endmembers and 43 other urban spectra. Given our objectives, we will present two results, a two-endmember class map and fractions derived from a four endmember model. The latter was generated from a 608 models (4x4x38. We did not use all possible swimming pools). A subset from the full mosaic is shown as a false color composite with 1650, 830 and 675 nm loaded as red green and blue. This subset illustrates a landscape consisting of some natural surfaces (riparian vegetation in green, senesced grasslands as pink), two golf


82

courses (light green) and numerous built-up surfaces including roads, roofs, concrete bridges, parking lots and buildings. Bare soil is evident as lighter pink areas (trails) within grasslands or larger, open dirt parking lots. Red roof types, using this display, are wood-shingle roofs, which are common in this part of Goleta. Water bodies, on the left side of the image, are black.

Figure showing a 1650, 830, 675 nm RGB composite of a subset of Goleta, California. A two endmember model, consisting of 51 models was run using standard constraints (-5%, 105%, 80% shade, 2.5% RMS). This model successfully modeled 82% of the image. The most common model was a GV model, which mapped 15.9% of the image. Overall, GV accounted for 28% of the image. NPV accounted for another 10%, with two of the four NPV models accounting for almost all of the area mapped. Three urban surfaces were the most abundant, including model 22 (5.7%), model 35 (7.16%) and model 44 (4.47%). Prominent unmodeled areas included deeper water bodies, which were not present in the library and very bright metal roofs. Looking at the image, the end results are remarkably good. While we have not done a formal accuracy assessment, knowledge of the area suggests that impervious/pervious surfaces were mapped at accuracies exceeding 90% if we exclude unmodeled areas. The most significant errors appear to be restricted to parking lots (impervious) and dirt (pervious). Brighter parts of parking lots are being mapped as dirt. While this represents an excellent map of impervious surfaces, it is not a good map for discriminating urban cover types. For example, one obvious error is metal roofs, many of


83

which were mapped as concrete. A quick perusal of the spectra explains how this could have happened, since the metal roofs that were misclassified are virtually identical to concrete mixed with shade. Models 22 and 35 are also problematic. These models represent red gravel and red tile roof, neither of which are common in the area. Furthermore, they clearly map the wrong materials – many of the roads are mapped as red tile roofs. To understand this result, we need to consider our original selection criteria. We started by dividing the library into two categories, impervious and pervious. When selecting impervious spectra, we selected members that could model the largest number of spectra in the impervious category. We did not ask the model to tell us whether they were also mapping the right land-cover class. As a result, we selected two spectra, red gravel and red tile, that could be considered generalists. These spectra are excellent candidates for mapping impervious materials, but do not discriminate well between cover types within this class. Red tile roof was selected because it mapped many impervious surfaces well, many of which were actually roads. This result is entirely predictable. Given our objective, of only mapping impervious and impervious classes, it is also a reasonable outcome.

Figure showing two endmember models for the Goleta subset. Dark purple areas are dominated by green vegetation. Light purple areas are dominated by senesced materials. Parking lots and roads tend to be blue while roofs tend to be green. Metal roofs are mapped as green. Wood shingle roofs are mostly aquamarine. The most significant errors include roofs mapped as concrete (lower left, blue) and roads mapped as red tile (green linear features).


84

Our final image shows a 4 endmember model with NPV as Red, GV as Green and Blue as other. Given the constraints on the model, all fractions are between 0 and 100% except the shade fraction, which was only constrained to a maximum value of 80%. While we have not done a formal accuracy assessment of the fractions, the map looks excellent based on local knowledge.

Figure showing fraction images of NPV, GV and other as RGB. Golf courses are light green and senesced grasslands red. Trees/shrubs are also prominent along the south edge of 101 (green near horizontal strip) and in most suburban neighborhoods. The other category includes all urban materials and soils. Image Analysis: Wind River Our final example focuses on Wind River, Washington state. This is an area that has been the focus of extensive research dating back to the mid 1990s. Our earliest AVIRIS images were acquired using the ER2 in 1994. Since that time, data have been acquired in 1995, 1996, 1997, 1998, 2000, 2002, 2003 and most recently, 2005. The latter images were all acquired using the Twin Otter. Initial research objectives focused on the problem of scaling up flux tower data from points to landscapes (Roberts et al., 2004). This research utilized the 1998 ER2 data and focused on species level discrimination at leaf, branch and stand scales and biophysical measures of structure (Equivalent water thickness, albedo, spectral fractions from a simple mixing model). Recent research has focused on the problem of improved


85

measures of carbon stocks and fluxes, partially supported by NASA Carbon Cycle Science research. One objective of that research is to use imaging spectrometry data to map plants species and plant functional types. Another objective is to quantify physiologically significant stand properties. The example we show here deals with mapping plant species and functional types along a portion of one of the five AVIRIS flight lines that cover the area. Data used in the analysis included: 030718r9-35-2_reg_utm-2.sub: This is a 224 band georectified AVIRIS flight line consisting of 4592 lines and 900 samples. windriver_all.sli: This is a spectral library consisting of 2328 spectra of dominant vegetation and non-vegetation in the area. This spectral library was built from approximately 140 polygons distributed across the five georectified AVIRIS flight lines. windriver_all.csv: This is the metadata file for this spectral library. The general analysis procedure included all of the basic steps of Viper Tools including: 1) Create Spectral Library from ROIS. This was done from multiple AVIRIS flight lines. Polygons were statistically subsampled to reduce autocorrelation and 2) Create Metadata for Spectral Library (windriver_all.csv) 3) Manage Spectral Libraries. In this instance we sorted on dominant followed by brightness 4) Create Square array (windriver_all_sli.sqr) 5) Calculate EAR/MASA/CoB In this example, we have used a more standard approach, using dominant as the sorting criterion. Our objective is also different than the previous urban example. In this case, we will start with plant species, than, as needed, aggregate up to plant functional types. Because we are dealing with more subtle spectral differences, we used a combination of EAR, MASA and Cob to select optimal spectra. The end result was a spectral library consisting of 30 endmembers, including Abies grandis (abgr:1), Acer circinatum (acci:2), Acer macrophyllum (acma: 2), Alnus rubra (alru:2), senesced grass (dgrass: 6), Populus trichocarpa (potr: 4), Pseudotsuga menziesii (psme: 3), Pteridium acquilinum (ptaq: 2), rock/soil (3), Thuja plicata (thpl:1), and Tsuga heterophylla (tshe: 4). Selections ranged from a single criterion (Cob for abgr) to multiple criteria (EAR, MASA and CoB for psme). Our final spectral library was windriver_all_optimal_2nd.sli. This spectral library was applied to an AVIRIS subset image to produce fraction and class images for 2, 3 and 4 endmember models. The three endmember model was restricted to mixtures of GV, NPV and shade for simplicity. The two endmember model classified 76% of the image as a member of one of the 51 models. The remaining 24% consisted almost entirely of black edge pixels, so the overall area mapped was quite good.


86

Evaluation of the image results showed that this model was generally good, accurately identifying many of the broad leaved plants, senesced grass and ferns. It also accurately mapped conifers, but appeared to overmap western hemlock. Subsets of two parts of the scene, one towards the north that is dominated by broad leaved plants, and one centered over the Wind River Canopy crane are included. In the southern subset, an area dominated by what appears to be wood chips was modeled as bark, rather than senesced grass.

Figure showing the northern portion of the two endmember model. Dark blue models are poplar or alder, purple is maple, light green is Douglas-fir, cyan is senesced grass, yellow is gravel and orange/red western hemlock.


87

Figure showing the southern portion of the image. In addition to the colors described previously, we have bare soil (blue), fern (very light green). Initial accuracy assessment, using polygons located along this flight line showed that this model produced relatively high accuracies at plot scales for plant species and plant functional types. Overall accuracy at plot scales was 71%, with individual accuracies ranging from a low of 25% (Thuja plicata) to 100% (Acer circinatum, senesced grass, Pseudotsuga menziesii, Pteridium aquilinum). Plot scale accuracies for plant functional types increased to 85% with the greatest confusion between broadleaf trees and shrubs and ferns and Thuja plicata.


88

Table Plot scale species level accuracy. Species include Abies grandis (abgr), Acer circinatum (acci), Acer macrophyllum (acma), Alnus rubra (alru), senesced grass, Populus trichocarpa (potr), Pseudotsuga menziesii (psme), Pteridium acquilinum (ptaq), Thuja plicata (thpl) and Tsuga heterophylla (tshe).

Table showing plot scale and pixel scale accuracies for plant functional types. Column headings match for the two tables (ie, eg has a pixel level accuracy of 98% and plot-scale accuracy of 86%). Column headings include evergreen (eg), broadleaf tree (bl), broadleaf shrub (shrub), annual grass and ferns.

Pixel-scale accuracies were generally high at the plant functional type level at 92%. However, much of this high accuracy can be attributed to dominance of conifers within the landscape (98%). Individual accuracies for broadleaf shrubs and ferns were considerably lower at 8 and 15%, respectively. However, this accuracy may be an underestimate – in many cases ferns and shrubs were mapped in forest gaps, in stands that were called pure Pseudotsuga menziesii or Tsuga heterophylla yet, are likely to have one of these two plant species within gaps.


89

Three endmember models modeled a slightly lower number of pixels, leaving 30% of the image unmodeled. In this case, the surfaces that were not modeled are those that included a significant amount of soil or gravel. Examples include gravel bars and much of the area previously mapped as senesced grass. Another important outcome is that areas that were previously overmapped as western hemlock, are mapped as having a higher amount of Douglas-fir in the three endmember model. For example, in the two endmember model, 26.2% of the image was mapped as Douglas-fir, an underestimate, where as 18.5% was mapped as hemlock, probably an overestimate. When he add a third, NPV endmember to the mix, the area mapped as Douglas-fir increases to 35.55% and area mapped as hemlock decreases to 17.9%. These numbers are much more consistent with field data.

Figure showing models selected for a three endmember model for the northern subset. Gravel bars which were previously mapped in the two endmember case, are now unmodeled because soil/gravel was not included in the model.


90

Figure showing models selected for a three endmember model in the southern subset. Note the large areas in the south that were modeled using two endmembers, but not with three. These areas either require only two endmembers or require a soil endmember to be present in the mix. The final example shows a fraction image derived from the four endmember model. A four endmember model was generated by adding the top soil/gravel endmembers in to the mixture, while keeping the original 20 three endmember models. We added two soil endmembers, resulting in 40 four endmember models. The 4 endmember model successfully modeled a significantly lower proportion of the image, leaving 45% of the image unmodeled. This result is not surprising. For example Powell (2006), while working in the Amazon with Landsat TM data, found that natural cover was best modeled by two endmembers, disturbed landscapes by three and urban areas by four. In our own analysis of Goleta, California using 4 endmembers, we were able to model 83% of the image. While we have to be cautious here, because that model included 608 4 endmember models, and we only used 40 here, the results are consistent with the greater spectral diversity and mixed nature of urban areas. ). Fraction images are shown for this model, showing NPV as red, GV as green and soil as blue. Only the southern subset is shown. In this image, most of the unmodeled areas are simpler surfaces, such as the grasslands, which were modeled well by only two to three endmembers. The fractions for old growth forest, are reasonable and physically realistic.


91

Figure showing fractions for a 4 endmember model, with NPV, GV and soil as red green and blue. Highest soil fractions are restricted to shallow water and roads. GV fractions are highest in forested area or recent clear cuts. Recent clears have the highest NPV fraction of the vegetated areas and lowest shade. These analyses only represent a small fraction of what could be done for either the Santa Barbara urban area or Wind River using Viper Tools.

Documents

VIPER Tools User Manual v1.5