5. Tutorial: spec_mod

5.1. Sample data

Sample imagery captured from a Resonon Pika II VIS-NIR line scanning imager and ancillary sample files can be downloaded from this link.

Before trying this tutorial on your own machine, please download the sample files and place into a local directory of your choosing (and do not change the file names). Indicate the location of your sample files by modifying data_dir:

[1]:
data_dir = r'F:\\nigo0024\Documents\hs_process_demo'

5.2. Confirm your environment

Before trying the tutorials, be sure hs_process and its dependencies are properly installed. If you installed in a virtual environment, first check we are indeed using the Python instance that was installed with the virtual environment:

[2]:
import sys
import hs_process
print('Python install location: {0}'.format(sys.executable))
print('Version: {0}'.format(hs_process.__version__))
Python install location: C:\Users\nigo0024\Anaconda3\envs\hs_process\python.exe
Version: 0.0.4

The spec folder that contains python.exe tells me that the activated Python instance is indeed in the spec environment, just as I intend. If you created a virtual environment, but your python.exe is not in the envs\spec directory, you either did not properly create your virtual environment or you are not pointing to the correct Python installation in your IDE (e.g., Spyder, Jupyter notebook, etc.).


5.3. spec_mod.load_spyfile

Loads a SpyFile (Spectral Python object) for data access and/or manipulation by the hstools class. [API]

Load and initialize the hsio and spec_mod modules

[3]:
import os
from hs_process import hsio
from hs_process import spec_mod

fname_in = os.path.join(data_dir, 'Wells_rep2_20180628_16h56m_pika_gige_7-Radiance Conversion-Georectify Airborne Datacube-Convert Radiance Cube to Reflectance from Measured Reference Spectrum.bip.hdr')
io = hsio(fname_in)
my_spec_mod = spec_mod(io.spyfile)

Load datacube using spec_mod.load_spyfile

[4]:
my_spec_mod.load_spyfile(io.spyfile)
my_spec_mod.spyfile
[4]:
        Data Source:   'F:\\nigo0024\Documents\hs_process_demo\Wells_rep2_20180628_16h56m_pika_gige_7-Radiance Conversion-Georectify Airborne Datacube-Convert Radiance Cube to Reflectance from Measured Reference Spectrum.bip'
        # Rows:            617
        # Samples:        1300
        # Bands:           240
        Interleave:        BIP
        Quantization:  32 bits
        Data format:   float32

5.4. spec_mod.spec_derivative

Calculates the numeric derivative spectra from spyfile_spec. [API]

Load and initialize hsio

[5]:
import os
from hs_process import hsio
from hs_process import spec_mod
fname_hdr_spec = os.path.join(data_dir, 'Wells_rep2_20180628_16h56m_pika_gige_7_plot_611-cube-to-spec-mean.spec.hdr')
io = hsio()
io.read_spec(fname_hdr_spec)
my_spec_mod = spec_mod(io.spyfile_spec)

Calculate the numeric derivative.

[6]:
spec_dydx, metadata_dydx = my_spec_mod.spec_derivative(io.spyfile_spec)  # Be sure this is the spec spyfile and not a full array

Plot the numeric derivative spectra and compare against the original spectra.

[7]:
import numpy as np
import seaborn as sns
sns.set_style("ticks")
wl_x = np.array([float(i) for i in metadata_dydx['wavelength']])
y_ref = io.spyfile_spec.open_memmap()[0,0,:]*100
ax1 = sns.lineplot(x=wl_x, y=y_ref)
ax2 = ax1.twinx()
ax2 = sns.lineplot(x=wl_x, y=0, ax=ax2, color='gray')
ax2 = sns.lineplot(x=wl_x, y=spec_dydx[0,0,:]*100, ax=ax2, color=sns.color_palette()[1])
ax2.set(ylim=(-0.8, 1.5))
ax1.set_xlabel('Wavelength (nm)', weight='bold')
ax1.set_ylabel('Reflectance (%)', weight='bold')
ax2.set_ylabel('Reflectance derivative (%)', weight='bold')
ax1.set_title(r'API Example: `spec_mod.spec_derivative`', weight='bold')
[7]:
Text(0.5, 1.0, 'API Example: `spec_mod.spec_derivative`')
_images/tutorial_spec_mod_13_1.png

Save the derivative spectra to file.

[8]:
fname_spec_der = os.path.join(data_dir, 'spec_mod', 'Wells_rep2_20180628_16h56m_pika_gige_7_plot_611-spec-derivative.spec.hdr')
io.write_spec(fname_spec_der, spec_dydx, df_std=None, metadata=metadata_dydx, force=True)

5.5. spec_mod.spectral_clip

Removes/clips designated wavelength bands from the hyperspectral datacube. [API]

Load and initialize the hsio and spec_mod modules

[9]:
import os
from hs_process import hsio
from hs_process import spec_mod

fname_hdr = os.path.join(data_dir, 'Wells_rep2_20180628_16h56m_pika_gige_7-Radiance Conversion-Georectify Airborne Datacube-Convert Radiance Cube to Reflectance from Measured Reference Spectrum.bip.hdr')
io = hsio()
io.read_cube(fname_hdr)
my_spec_mod = spec_mod(io.spyfile)

Using spec_mod.spectral_clip, clip all spectral bands below 420 nm and above 880 nm, as well as the bands near the oxygen absorption (i.e., 760-776 nm) and water absorption (i.e., 813-827 nm) regions.

[10]:
array_clip, metadata_clip = my_spec_mod.spectral_clip(wl_bands=[[0, 420], [760, 776], [813, 827], [880, 1000]])

Plot the spectra of the unclippe hyperspectral image and compare to that of the clipped image for a single pixel.

[11]:
import seaborn as sns
from ast import literal_eval

spy_hs = my_spec_mod.spyfile.open_memmap()  # datacube before smoothing
meta_bands = list(io.tools.meta_bands.values())
meta_bands_clip = sorted([float(i) for i in literal_eval(metadata_clip['wavelength'])])

ax = sns.lineplot(x=meta_bands, y=spy_hs[200][800]*100, label='Before spectral clipping', linewidth=3)
ax = sns.lineplot(x=meta_bands_clip, y=array_clip[200][800]*100, label='After spectral clipping', ax=ax)
ax.set_xlabel('Wavelength (nm)', weight='bold')
ax.set_ylabel('Reflectance (%)', weight='bold')
ax.set_title(r'API Example: `spec_mod.spectral_clip`', weight='bold')
[11]:
Text(0.5, 1.0, 'API Example: `spec_mod.spectral_clip`')
_images/tutorial_spec_mod_21_1.png

Save the clipped datacube

[12]:
from pathlib import Path

Path(os.path.join(data_dir, 'spec_mod')).mkdir(parents=True, exist_ok=True)
fname_hdr_clip = os.path.join(data_dir, 'spec_mod', 'Wells_rep2_20180628_16h56m_pika_gige_7-clip.bip.hdr')
io.write_cube(fname_hdr_clip, array_clip, metadata_clip, force=True)

5.6. spec_mod.spectral_mimic

Mimics the response of a multispectral sensor based on transmissivity of sensor bands across a range of wavelength values by calculating its weighted average response and interpolating the hyperspectral response. [API]

Load and initialize the hsio and spec_mod modules

[13]:
import os
from hs_process import hsio
from hs_process import spec_mod

fname_hdr = os.path.join(data_dir, 'Wells_rep2_20180628_16h56m_pika_gige_7-Radiance Conversion-Georectify Airborne Datacube-Convert Radiance Cube to Reflectance from Measured Reference Spectrum.bip.hdr')
io = hsio()
io.read_cube(fname_hdr)

my_spec_mod = spec_mod(io.spyfile)

Use spec_mod.spectral_mimic to mimic the Sentinel-2A spectral response function.

[14]:
array_s2a, metadata_s2a = my_spec_mod.spectral_mimic(sensor='sentinel-2a', center_wl='weighted')

Plot the spectra of the hyperspectral image and compare to that of the mimicked Sentinel-2A for a single pixel.

[15]:
import seaborn as sns

spy_hs = my_spec_mod.spyfile.open_memmap()  # datacube before smoothing
meta_bands = list(io.tools.meta_bands.values())
meta_bands_s2a = sorted([float(i) for i in literal_eval(metadata_s2a['wavelength'])])

ax = sns.lineplot(x=meta_bands, y=spy_hs[200][800]*100, label='Hyperspectral (Pika II)', linewidth=3)
ax = sns.lineplot(x=meta_bands_s2a, y=array_s2a[200][800]*100, label='Sentinel-2A "mimic"', marker='o', ms=6, ax=ax)
ax.set_xlabel('Wavelength (nm)', weight='bold')
ax.set_ylabel('Reflectance (%)', weight='bold')
ax.set_title(r'API Example: `spec_mod.spectral_mimic`', weight='bold')
[15]:
Text(0.5, 1.0, 'API Example: `spec_mod.spectral_mimic`')
_images/tutorial_spec_mod_29_1.png

Use spec_mod.spectral_mimic to mimic the Sentera 6x spectral configuration and compare to both hyperspectral and Sentinel-2A.

[16]:
array_6x, metadata_6x = my_spec_mod.spectral_mimic(sensor='sentera_6x', center_wl='peak')
meta_bands_6x = sorted([float(i) for i in literal_eval(metadata_6x['wavelength'])])

ax = sns.lineplot(x=meta_bands, y=spy_hs[200][800]*100, label='Hyperspectral (Pika II)', linewidth=3)
ax = sns.lineplot(x=meta_bands_s2a, y=array_s2a[200][800]*100, label='Sentinel-2A "mimic"', marker='o', ms=6, ax=ax)
ax = sns.lineplot(x=meta_bands_6x, y=array_6x[200][800]*100, label='Sentera 6X "mimic"', marker='o', ms=8, ax=ax)
ax.set_xlabel('Wavelength (nm)', weight='bold')
ax.set_ylabel('Reflectance (%)', weight='bold')
ax.set_title(r'API Example: `spec_mod.spectral_mimic`', weight='bold')
[16]:
Text(0.5, 1.0, 'API Example: `spec_mod.spectral_mimic`')
_images/tutorial_spec_mod_31_1.png

And finally, mimic the Micasense RedEdge-MX and compare to hyperspectral, Sentinel-2A, and Sentera 6X.

[17]:
array_re3, metadata_re3 = my_spec_mod.spectral_mimic(sensor='micasense_rededge_3', center_wl='peak')
meta_bands_re3 = sorted([float(i) for i in literal_eval(metadata_re3['wavelength'])])

ax = sns.lineplot(x=meta_bands, y=spy_hs[200][800]*100, label='Hyperspectral (Pika II)', linewidth=3)
ax = sns.lineplot(x=meta_bands_s2a, y=array_s2a[200][800]*100, label='Sentinel-2A "mimic"', marker='o', ms=6, ax=ax)
ax = sns.lineplot(x=meta_bands_6x, y=array_6x[200][800]*100, label='Sentera 6X "mimic"', marker='o', ms=8, ax=ax)
ax = sns.lineplot(x=meta_bands_re3, y=array_re3[200][800]*100, label='Micasense RedEdge 3 "mimic"', marker='o', ms=8, ax=ax)
ax.set_xlabel('Wavelength (nm)', weight='bold')
ax.set_ylabel('Reflectance (%)', weight='bold')
ax.set_title(r'API Example: `spec_mod.spectral_mimic`', weight='bold')
[17]:
Text(0.5, 1.0, 'API Example: `spec_mod.spectral_mimic`')
_images/tutorial_spec_mod_33_1.png

Save the mimicked datacubes using hsio.write_cube

[18]:
fname_hdr_mimic_s2a = os.path.join(data_dir, 'spec_mod', 'Wells_rep2_20180628_16h56m_pika_gige_7-mimic-s2a.bip.hdr')
fname_hdr_mimic_6x = os.path.join(data_dir, 'spec_mod', 'Wells_rep2_20180628_16h56m_pika_gige_7-mimic-6x.bip.hdr')
fname_hdr_mimic_re3 = os.path.join(data_dir, 'spec_mod', 'Wells_rep2_20180628_16h56m_pika_gige_7-mimic-re3.bip.hdr')
io.write_cube(fname_hdr_mimic_s2a, array_s2a, metadata_s2a, force=True)
io.write_cube(fname_hdr_mimic_6x, array_6x, metadata_6x, force=True)
io.write_cube(fname_hdr_mimic_re3, array_re3, metadata_re3, force=True)

5.7. spec_mod.spectral_resample

Performs pixel-wise resampling of spectral bands via binning (calculates the mean across all bands within each bandwidth region for each image pixel). [API]

Load and initialize the hsio and spec_mod modules

[19]:
import os
from hs_process import hsio
from hs_process import spec_mod

fname_hdr = os.path.join(data_dir, 'Wells_rep2_20180628_16h56m_pika_gige_7-Radiance Conversion-Georectify Airborne Datacube-Convert Radiance Cube to Reflectance from Measured Reference Spectrum.bip.hdr')
io = hsio()
io.read_cube(fname_hdr)

my_spec_mod = spec_mod(io.spyfile)

Use spec_mod.spectral_resample to “bin” the datacube to bands with 20 nm bandwidths.

[20]:
array_bin, metadata_bin = my_spec_mod.spectral_resample(bandwidth=20)

Plot the spectra for both the hyperspectral image and that of the binned image bands (for a single “vegetation” pixel).

[21]:
import seaborn as sns
from ast import literal_eval

spy_hs = my_spec_mod.spyfile.open_memmap()  # datacube before smoothing
meta_bands = list(io.tools.meta_bands.values())
# meta_bands_bin = sorted([float(i) for i in metadata_bin['wavelength']])
meta_bands_bin = sorted([float(i) for i in literal_eval(metadata_bin['wavelength'])])

ax = sns.lineplot(x=meta_bands, y=spy_hs[200][800]*100, label='Hyperspectral (Pika II)', linewidth=3)
ax = sns.lineplot(x=meta_bands_bin, y=array_bin[200][800]*100, label='Spectral resample (20 nm)', marker='o', ms=6, ax=ax)
ax.set_xlabel('Wavelength (nm)', weight='bold')
ax.set_ylabel('Reflectance (%)', weight='bold')
ax.set_title(r'API Example: `spec_mod.spectral_resample`', weight='bold')
[21]:
Text(0.5, 1.0, 'API Example: `spec_mod.spectral_resample`')
_images/tutorial_spec_mod_41_1.png

Save the spectrally resampled datacube using hsio.write_cube

[22]:
fname_hdr_bin = os.path.join(data_dir, 'spec_mod', 'Wells_rep2_20180628_16h56m_pika_gige_7-bin-20.bip.hdr')
io.write_cube(fname_hdr_bin, array_bin, metadata_bin, force=True)

5.8. spec_mod.spectral_smooth

Performs Savitzky-Golay smoothing on the spectral domain. [API]

Load and initialize the hsio and spec_mod modules

[23]:
import os
from hs_process import hsio
from hs_process import spec_mod

fname_hdr = os.path.join(data_dir, 'Wells_rep2_20180628_16h56m_pika_gige_7-Radiance Conversion-Georectify Airborne Datacube-Convert Radiance Cube to Reflectance from Measured Reference Spectrum.bip.hdr')
io = hsio()
io.read_cube(fname_hdr)
my_spec_mod = spec_mod(io.spyfile)

Use spec_mod.spectral_smooth to perform a Savitzky-Golay smoothing operation across the hyperspectral spectral signature.

[24]:
array_smooth, metadata_smooth = my_spec_mod.spectral_smooth(window_size=11, order=2)

Plot the spectra of an individual pixel to visualize the result of the smoothing procedure.

[25]:
import seaborn as sns
from ast import literal_eval

spy_hs = my_spec_mod.spyfile.open_memmap()  # datacube before smoothing
meta_bands = list(io.tools.meta_bands.values())
meta_bands_smooth = sorted([float(i) for i in metadata_smooth['wavelength']])

ax = sns.lineplot(x=meta_bands, y=spy_hs[200][800]*100, label='Before spectral smoothing', linewidth=3)
ax = sns.lineplot(x=meta_bands_smooth, y=array_smooth[200][800]*100, label='After spectral smoothing', ax=ax)
ax.set_xlabel('Wavelength (nm)', weight='bold')
ax.set_ylabel('Reflectance (%)', weight='bold')
ax.set_title(r'API Example: `spec_mod.spectral_smooth`', weight='bold')
[25]:
Text(0.5, 1.0, 'API Example: `spec_mod.spectral_smooth`')
_images/tutorial_spec_mod_49_1.png

Save the smoothed datacube using hsio.write_cube

[26]:
fname_hdr_smooth = os.path.join(data_dir, 'spec_mod', 'Wells_rep2_20180628_16h56m_pika_gige_7-smooth.bip.hdr')
io.write_cube(fname_hdr_smooth, array_smooth, metadata_smooth, force=True)