8.1.3. hsio

class hs_process.hsio(fname_in=None, name_long=None, name_plot=None, name_short=None, str_plot='plot_', individual_plot=False, fname_hdr_spec=None)[source]

Bases: object

Class for reading and writing hyperspectral data files, as well as accessing, interpreting, and modifying its associated metadata. With a hyperspectral data file loaded via hsio, there is simple functionality to display the datacube image as a multi-band render, as well as for saving a datacube as a 3-band geotiff. hsio relies heavily on the Spectral Python package.

Methods Summary

read_cube([fname_hdr, overwrite, name_long, …])

Reads in a hyperspectral datacube using the Spectral Python package.

read_spec(fname_hdr_spec[, overwrite, …])

Reads in a hyperspectral spectrum file using the using the Spectral Python package.

set_io_defaults([dtype, force, ext, …])

Sets any of the ENVI file writing parameters to hsio; if any parameter is left unchanged from its default, it will remain as-is (i.e., it will not be set).

show_img([spyfile, band_r, band_g, band_b, …])

Displays a datacube as a 3-band RGB image using Matplotlib.

write_cube(fname_hdr, spyfile[, metadata, …])

Wrapper function that accesses the Spectral Python package to save a datacube to file.

write_spec(fname_hdr_spec, df_mean[, …])

Wrapper function that accesses the Spectral Python package to save a single spectra to file.

write_tif(fname_tif[, spyfile, metadata, …])

Wrapper function that accesses the GDAL Python package to save a small datacube subset (i.e., three bands or less) to file.

Methods Documentation

read_cube(fname_hdr=None, overwrite=True, name_long=None, name_short=None, name_plot=None, individual_plot=False)[source]

Reads in a hyperspectral datacube using the Spectral Python package.

Parameters

  • fname_hdr (str) – filename of datacube to be read (default: None).

  • overwrite (bool) – Whether to overwrite any of the previous user-passed variables, including name_long, name_plot, and name_short. If variables are already set and overwrite is False, they will remain the same. If variables are set and overwrite is True, they will be overwritten based on typcial file naming conventions of Resonon/Spectronon software. Any of the user-passed variables (e.g., name_long, etc.) will overwrite those that were set previously whether overwrite is True or False (default: False).

  • name_long (str) – Spectronon processing appends processing names to the filenames; this indicates those processing names that are repetitive and can be deleted from the filename following processing (default: None).

  • name_short (str) – The base name of the image file (see note above about name_long; default: None).

  • name_plot (str) – numeric text that describes the plot number (default: None).

  • individual_plot (bool) – Indicates whether image (and its filename) is for an individual plot (True), or for many plots (False; default: False).

Note

hs_process will search for name_long, name_plot, and name_short based on typical file naming behavior of Resonon/ Spectronon software. If any of these parameters are passed by the user, however, that will take precedence over “searching the typical file nameing behavior”.

Example

Load and initialize hsio

>>> from hs_process import hsio
>>> fname_hdr = r'F:\nigo0024\Documents\hs_process_demo\Wells_rep2_20180628_16h56m_pika_gige_7-Radiance Conversion-Georectify Airborne Datacube-Convert Radiance Cube to Reflectance from Measured Reference Spectrum.bip.hdr'
>>> io = hsio()  # initialize an instance of the hsio class (note there are no required parameters)

Load datacube using hsio.read_cube

>>> io.read_cube(fname_hdr)
>>> io.spyfile
Data Source:   'F:\nigo0024\Documents\hs_process_demo\Wells_rep2_20180628_16h56m_pika_gige_7-Radiance Conversion-Georectify Airborne Datacube-Convert Radiance Cube to Reflectance from Measured Reference Spectrum.bip'
    # Rows:            617
    # Samples:        1300
    # Bands:           240
    Interleave:        BIP
    Quantization:  32 bits
    Data format:   float32

Check name_long, name_short, and name_plot values derived from the filename

>>> io.name_long
'-Convert Radiance Cube to Reflectance from Measured Reference Spectrum'

>>> io.name_plot
'7'

>>> io.name_short
'Wells_rep2_20180628_16h56m_pika_gige_7'
read_spec(fname_hdr_spec, overwrite=True, name_long=None, name_short=None, name_plot=None)[source]

Reads in a hyperspectral spectrum file using the using the Spectral Python package.

Parameters

fname_hdr_spec (str) – filename of spectra to be read.

Example

Load and initialize hsio

>>> from hs_process import hsio
>>> fname_hdr = r'F:\nigo0024\Documents\hs_process_demo\Wells_rep2_20180628_16h56m_pika_gige_7_plot_611-cube-to-spec-mean.spec.hdr'
>>> io = hsio()  # initialize an instance of the hsio class (note there are no required parameters)

Load datacube using hsio.read_spec

>>> io.read_spec(fname_hdr)
>>> io.spyfile_spec
Data Source:   'F:\nigo0024\Documents\hs_process_demo\Wells_rep2_20180628_16h56m_pika_gige_7_plot_611-cube-to-spec-mean.spec'
    # Rows:              1
    # Samples:           1
    # Bands:           240
    Interleave:        BIP
    Quantization:  32 bits
    Data format:   float32

Check name_long, name_short, and name_plot values derived from the filename

>>> io.name_long
'-cube-to-spec-mean'

>>> io.name_short
'Wells_rep2_20180628_16h56m_pika_gige_7_plot_611'

>>> io.name_plot
'611'
set_io_defaults(dtype=False, force=None, ext=False, interleave=False, byteorder=False)[source]

Sets any of the ENVI file writing parameters to hsio; if any parameter is left unchanged from its default, it will remain as-is (i.e., it will not be set).

Parameters

  • dtype (numpy.dtype or str) – The data type with which to store the image. For example, to store the image in 16-bit unsigned integer format, the argument could be any of numpy.uint16, ‘u2’, ‘uint16’, or ‘H’ (default=``False``).

  • force (bool) – If hdr_file or its associated image file exist, force=True will overwrite the files; otherwise, an exception will be raised if either file exists (default=``None``).

  • ext (str) – The extension to use for saving the image file; if not specified, a default extension is determined based on the interleave. For example, if interleave``='bip', then ``ext is set to ‘bip’ as well. If ext is an empty string, the image file will have the same name as the .hdr, but without the ‘.hdr’ extension (default: False).

  • interleave (str) – The band interleave format to use for writing the file; interleave should be one of ‘bil’, ‘bip’, or ‘bsq’ (default=``False``).

  • byteorder (int or str) – Specifies the byte order (endian-ness) of the data as written to disk. For little endian, this value should be either 0 or ‘little’. For big endian, it should be either 1 or ‘big’. If not specified, native byte order will be used (default=``False``).

Example

Load and initialize hsio

>>> from hs_process import hsio
>>> io = hsio()  # initialize an instance of the hsio class

Check defaults.envi_write

>>> io.defaults.envi_write
{'dtype': numpy.float32,
 'force': False,
 'ext': '',
 'interleave': 'bip',
 'byteorder': 0}

Modify force parameter and recheck defaults.envi_write

>>> io.set_io_defaults(force=True)
>>> io.defaults.envi_write
{'dtype': numpy.float32,
 'force': True,
 'ext': '',
 'interleave': 'bip',
 'byteorder': 0}
show_img(spyfile=None, band_r=120, band_g=76, band_b=32, vmin=None, vmax=None, cmap='viridis', cbar=True, inline=True)[source]

Displays a datacube as a 3-band RGB image using Matplotlib.

Parameters

  • spyfile (SpyFile object or numpy.ndarray) – The data cube to display; if None, loads from self.spyfile (default: None).

  • band_r (int) – Band to display on the red channel (default: 120)

  • band_g (int) – Band to display on the green channel (default: 76)

  • band_b (int) – Band to display on the blue channel (default: 32)

  • vmin/vmax (scalar, optional) – The data range that the colormap covers. By default, the colormap covers the complete value range of the supplied data (default: None).

  • cmap (str) – The Colormap instance or registered colormap name used to map scalar data to colors. This parameter is ignored for RGB(A) data (default: “viridis”).

  • cbar (bool) – Whether to include a colorbar in the image (default: True).

  • inline (bool) – If True, displays in the IPython console; else displays in a pop-out window (default: True).

Note

The inline parameter points to the hsio.show_img function, and is only expected to work in an IPython console (not intended to be used in a normal Python console).

Example

Load hsio and spatial_mod modules

>>> from hs_process import hsio # load hsio
>>> from hs_process import spatial_mod # load spatial mod

Load the datacube using hsio.read_cube

>>> fname_hdr = r'F:\nigo0024\Documents\hs_process_demo\Wells_rep2_20180628_16h56m_pika_gige_7-Radiance Conversion-Georectify Airborne Datacube-Convert Radiance Cube to Reflectance from Measured Reference Spectrum.bip.hdr'
>>> io = hsio()  # initialize an instance of the hsio class
>>> io.read_cube(fname_hdr)

Perform simple spatial cropping via spatial_mod.crop_single

>>> my_spatial_mod = spatial_mod(io.spyfile)  # initialize spatial_mod instance to crop the image
>>> array_crop, metadata = my_spatial_mod.crop_single(pix_e_ul=250, pix_n_ul=100, crop_e_m=8, crop_n_m=3)

Show an RGB render of the cropped image using hsio.show_img

>>> io.show_img(array_crop)
api/img/utilities/show_img.png
write_cube(fname_hdr, spyfile, metadata=None, dtype=None, force=None, ext=None, interleave=None, byteorder=None)[source]

Wrapper function that accesses the Spectral Python package to save a datacube to file.

Parameters

  • fname_hdr (str) – Output header file path (with the ‘.hdr’ extension).

  • spyfile (SpyFile object or numpy.ndarray) – The hyperspectral datacube to save. If numpy.ndarray, then metadata (dict) should also be passed.

  • metadata (dict) – Metadata to write to the ENVI .hdr file describing the hyperspectral data cube being saved. If SpyFile object is passed to spyfile, metadata will overwrite any existing metadata stored by the SpyFile object (default=None).

  • dtype (numpy.dtype or str) – The data type with which to store the image. For example, to store the image in 16-bit unsigned integer format, the argument could be any of numpy.uint16, ‘u2’, ‘uint16’, or ‘H’ (default=np.float32).

  • force (bool) – If hdr_file or its associated image file exist, force=True will overwrite the files; otherwise, an exception will be raised if either file exists (default=False).

  • ext (None or str) – The extension to use for saving the image file. If not specified or if set to an empty string (e.g., ext=''), a default extension is determined using the same name as fname_hdr, except without the “.hdr” extension. If fname_hdr is provided without the “non-.hdr” extension (e.g., “bip”), then the extension is determined from the interleave parameter. For example, if interleave``='bip', then ``ext is set to ‘bip’ as well. Use of ext is not recommended; instead, just set fname_hdr with the correct extension or use interleave to set the extension (default: None; determined from fname_hdr or interleave).

  • interleave (str) – The band interleave format to use for writing the file; interleave should be one of ‘bil’, ‘bip’, or ‘bsq’ (default=’bip’).

  • byteorder (int or str) – Specifies the byte order (endian-ness) of the data as written to disk. For little endian, this value should be either 0 or ‘little’. For big endian, it should be either 1 or ‘big’. If not specified, native byte order will be used (default=None).

Note

If dtype, force, ext, interleave, and byteorder are not passed, default values will be pulled from hsio.defaults. Thus, hsio.defaults can be modified prior to calling hsio.write_cube() to avoid having to pass each of thes parameters in the hsio.write_cube() function (see the hsio.set_io_defaults() function for support on setting these defaults and for more information on the parameters). Each of these parameters are passed directly to the Spectral Python envi.save_image() function. For more information, please refer to the Spectral Python documentation.

Example

Load hsio and spatial_mod modules

>>> import os
>>> import pathlib
>>> from hs_process import hsio  # load hsio
>>> from hs_process import spatial_mod  # load spatial mod
>>> data_dir = r'F:\nigo0024\Documents\hs_process_demo'
>>> fname_hdr_in = os.path.join(data_dir, 'Wells_rep2_20180628_16h56m_pika_gige_7-Radiance Conversion-Georectify Airborne Datacube-Convert Radiance Cube to Reflectance from Measured Reference Spectrum.bip.hdr')
>>> io = hsio()  # initialize the hsio class
>>> io.read_cube(fname_hdr_in)

Perform simple spatial cropping via spatial_mod.crop_single to generate a new datacube.

>>> my_spatial_mod = spatial_mod(io.spyfile)  # initialize spatial_mod instance to crop the image
>>> array_crop, metadata = my_spatial_mod.crop_single(pix_e_ul=250, pix_n_ul=100, crop_e_m=8, crop_n_m=3)

Save the datacube using hsio.write_cube

>>> fname_hdr = os.path.join(data_dir, 'hsio', 'Wells_rep2_20180628_16h56m_pika_gige_7-hsio-write-cube-cropped.bip.hdr')
>>> pathlib.Path(os.path.dirname(fname_hdr)).mkdir(parents=True, exist_ok=True)
>>> io.write_cube(fname_hdr, array_crop, metadata=metadata, force=True)

Load the datacube into Spectronon for visualization

api/img/utilities/write_cube.png
write_spec(fname_hdr_spec, df_mean, df_std=None, metadata=None, dtype=None, force=None, ext=None, interleave=None, byteorder=None)[source]

Wrapper function that accesses the Spectral Python package to save a single spectra to file.

Parameters

  • fname_hdr_spec (str) – Output header file path (with the ‘.hdr’ extension). If the extension is explicitely specified in fname_hdr_spec and the ext parameter is also specified, fname_hdr_spec will be modified to conform to the extension set using the ext parameter.

  • df_mean (pandas.Series or numpy.ndarray) – Mean spectra, stored as a df row, where columns are the bands.

  • df_std (pandas.Series or numpy.ndarray, optional) – Standard deviation of each spectra, stored as a df row, where columns are the bands. This will be saved to the .hdr file.

  • dtype (numpy.dtype or str) – The data type with which to store the image. For example, to store the image in 16-bit unsigned integer format, the argument could be any of numpy.uint16, ‘u2’, ‘uint16’, or ‘H’ (default=np.float32).

  • force (bool) – If hdr_file or its associated image file exist, force=True will overwrite the files; otherwise, an exception will be raised if either file exists (default=False).

  • ext (None or str) – The extension to use for saving the image file. If not specified or if set to an empty string (e.g., ext=''), a default extension is determined using the same name as fname_hdr_spec, except without the “.hdr” extension. If fname_hdr_spec is provided without the “non-.hdr” extension (e.g., “bip”), then the extension is determined from the interleave parameter. For example, if interleave``='bip', then ``ext is set to ‘bip’ as well. Use of ext is not recommended; instead, just set fname_hdr_spec with the correct extension or use interleave to set the extension (default: None; determined from fname_hdr_spec or interleave).

  • interleave (str) – The band interleave format to use for writing the file; interleave should be one of ‘bil’, ‘bip’, or ‘bsq’ (default=’bip’).

  • byteorder (int or str) – Specifies the byte order (endian-ness) of the data as written to disk. For little endian, this value should be either 0 or ‘little’. For big endian, it should be either 1 or ‘big’. If not specified, native byte order will be used (default=None).

  • metadata (dict) – Metadata to write to the ENVI .hdr file describing the spectra being saved; if None, will try to pull metadata template from hsio.spyfile_spec.metadata or hsio.spyfile.metadata (default=None).

Example

Load and initialize hsio

>>> from hs_process import hsio # load hsio
>>> fname_hdr_in = r'F:\nigo0024\Documents\hs_process_demo\Wells_rep2_20180628_16h56m_pika_gige_7-Radiance Conversion-Georectify Airborne Datacube-Convert Radiance Cube to Reflectance from Measured Reference Spectrum.bip.hdr'
>>> io = hsio()  # initialize the hsio class (note there are no required parameters)
>>> io.read_cube(fname_hdr_in)

Calculate spectral mean via hstools.mean_datacube

>>> spec_mean, spec_std, _ = io.tools.mean_datacube(io.spyfile)
>>> fname_hdr_spec = r'F:\nigo0024\Documents\hs_process_demo\hsio\Wells_rep2_20180628_16h56m_pika_gige_7-mean.spec.hdr'

Save the new spectra to file via hsio.write_spec

>>> io.write_spec(fname_hdr_spec, spec_mean, spec_std)
Saving F:\nigo0024\Documents\hs_process_demo\hsio\Wells_rep2_20180628_16h56m_pika_gige_7-mean.spec

Open Wells_rep2_20180628_16h56m_pika_gige_7-mean.spec in Spectronon for visualization

api/img/utilities/write_spec.png
write_tif(fname_tif, spyfile=None, metadata=None, fname_in=None, projection_out=None, geotransform_out=None, show_img=False)[source]

Wrapper function that accesses the GDAL Python package to save a small datacube subset (i.e., three bands or less) to file.

Parameters

  • fname_tif (str) – Output image file path (with the ‘.tif’ extension).

  • spyfile (SpyFile object or numpy.ndarray, optional) – The data cube to save. If numpy.ndarray, then metadata (dict) should also be passed. If None, uses hsio.spyfile (default: None).

  • metadata (dict) – Metadata information; if geotransform_out is not passed, “map info” is accessed from metadata and geotransform_out is created from that “map info”.

  • fname_in (str, optional) – The filename of the image datacube to be read in initially. This is potentially useful if projection_out and/or geotransform_out are not passed and a numpy.ndarray is passed as the spyfile - in this case, write_tif() uses fname_in to load the fname_in datacube via GDAL, which can in turn be used to load the projection or geotransform information for the output geotiff (default: None).

  • projection_out (str) – The GDAL projection to use while writing the geotiff. Applied using gdal.driver.dataset.SetProjection() (default: None; hsio.projection_out)

  • geotransform_out (str) – The GDAL geotransform to use while writing the geotiff. Applied using gdal.driver.dataset.SetGeoTransform() (default: None; hsio.geotransform_out)

  • show_img (bool or str) – Whether to display a render of the image being saved as a geotiff. Must be False (does not display the image), “inline” (displays the image inline using the IPython console), or “popout” (displays the image in a pop-out window; default: “inline”).

Example

Load and initialize hsio

>>> from hs_process import hsio  # load hsio
>>> fname_hdr_in = r'F:\nigo0024\Documents\hs_process_demo\Wells_rep2_20180628_16h56m_pika_gige_7-Radiance Conversion-Georectify Airborne Datacube-Convert Radiance Cube to Reflectance from Measured Reference Spectrum.bip.hdr'
>>> io = hsio()  # initialize the hsio class
>>> io.read_cube(fname_hdr_in)

Save an RGB render of the datacube to file via hsio.write_tif

>>> fname_tif = r'F:\nigo0024\Documents\hs_process_demo\hsio\Wells_rep2_20180628_16h56m_pika_gige_7.tif'
>>> io.write_tif(fname_tif, spyfile=io.spyfile, fname_in=fname_hdr_in)
Either `projection_out` is `None` or `geotransform_out` is `None` (or both are). Retrieving projection and geotransform information by loading `hsio.fname_in` via GDAL. Be sure this is appropriate for the data you are trying to write.
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
api/img/utilities/write_tif.png

Open Wells_rep2_20180628_16h56m_pika_gige_7.tif in QGIS with the plot boundaries overlaid

api/img/utilities/write_tif_qgis.png