isofit.core.fileio

Attributes

Logger
typemap
max_frames_size

Classes

SpectrumFile	A buffered file object that contains configuration information about formatting, etc.
InputData
IO

Functions

write_bil_chunk(→ None)	Write a chunk of data to a binary, BIL formatted data cube.
initialize_output(output_metadata, outpath, out_shape, ...)	Initialize output file by updating metadata and creating object.

Module Contents

Logger

typemap

max_frames_size = 100

class SpectrumFile(fname, write=False, n_rows=None, n_cols=None, n_bands=None, interleave=None, dtype=np.float32, wavelengths=None, fwhm=None, band_names=None, bad_bands='[]', zrange='{0.0, 1.0}', flag=-9999.0, ztitles='{Wavelength (nm), Magnitude}', map_info='{}', engine_name=None, isofit_version=None)

A buffered file object that contains configuration information about formatting, etc.

frames

write = False

fname = b'.'

wl = None

band_names = None

fwhm = None

flag = -9999.0

n_rows = None

n_cols = None

n_bands = None

open_map_with_retries()

Try to open a memory map, handling Beowulf I/O issues.

get_frame(row)

The frame is a 2D array, essentially a list of spectra. The self.frames list acts as a hash table to avoid storing the entire cube in memory. So we read them or create them on an as-needed basis. When the buffer flushes via a call to flush_buffers, they will be deleted.

write_spectrum(row, col, x)

We write a spectrum. If a binary format file, we simply change the data cached in self.frames and defer file I/O until flush_buffers is called.

read_spectrum(row, col)

Get a spectrum from the frame list or ASCII file. Note that if we are an ASCII file, we have already read the single spectrum and return it as-is (ignoring the row/column indices).

flush_buffers()

Write to file, and refresh the memory map object.

class InputData

meas = None

geom = None

reference_reflectance = None

clear()

class IO(config: Config, forward: ForwardModel, full_statevec: list = [])

…

config

radiance_correction = None

meas_wl

meas_fwhm

writes = 0

reads = 0

n_rows = 1

n_cols = 1

bbl = '{}'

engine_name

n_sv

n_chan

flush_rate

simulation_mode

total_time = 0

current_input_data

esd

get_components_at_index(row: int, col: int) → InputData

Load data from input files at the specified (row, col) index.

Parameters:

• row – row to retrieve data from

• col – column to retrieve data from

Returns:

object containing all current data reads

Return type:

InputData

flush_buffers()

Write all buffered output data to disk, and erase read buffers.

write_datasets(row: int, col: int, output: dict, states: List, flush_immediately=False)

Write all valid datasets to disk (possibly buffered).

Parameters:

• row – row to write to

• col – column to write to

• output – dictionary with keys corresponding to config.input file references

• states – results states from inversion. In the MCMC case, these are interpreted as samples from the

• posterior

• converged (otherwise they are a gradient descent trajectory (with the last spectrum being the)

• solution).

• flush_immediately – IO argument telling us to immediately write to disk, ignoring config settings

build_output(states: List, input_data: InputData, fm: ForwardModel, iv: Inversion, fill_value=-9999.0)

Build the output to be written to disk as a dictionary

Parameters:

• states – results states from inversion. In the MCMC case, these are interpreted as samples from the

• posterior

• converged (otherwise they are a gradient descent trajectory (with the last spectrum being the)

• solution).

• input_data – an InputData object

• fm – the forward model used to solve the inversion

• iv – the inversion object

write_spectrum(row: int, col: int, states: List, fm: ForwardModel, iv: Inversion, flush_immediately=False, input_data: InputData = None)

Convenience function to build and write output in one step

Parameters:

• row – data row to write

• col – data column to write

• states – results states from inversion. In the MCMC case, these are interpreted as samples from the

• posterior

• converged (otherwise they are a gradient descent trajectory (with the last spectrum being the)

• solution).

• meas – measurement radiance

• geom – geometry object of the observation

• fm – the forward model used to solve the inversion

• iv – the inversion object

• flush_immediately – IO argument telling us to immediately write to disk, ignoring config settings

• input_data – optionally overwride self.current_input_data

static initialize_output_files(config, n_rows, n_cols, full_statevector)

static load_esd(file=None)

Loads an earth_sun_distance file. Defaults to the [env.data]/earth_sun_distance.txt if not provided

Parameters:

file (str, default=None) – ESD file to load

Returns:

Loaded ESD. If the file fails to load, creates a default

Return type:

np.array

write_bil_chunk(dat: numpy.array, outfile: str, line: int, shape: tuple, dtype: str = 'float32') → None

Write a chunk of data to a binary, BIL formatted data cube. :param dat: data to write :param outfile: output file to write to :param line: line of the output file to write to :param shape: shape of the output file :param dtype: output data type

Returns:

None

initialize_output(output_metadata, outpath, out_shape, **kwargs)

Initialize output file by updating metadata and creating object.

Parameters:

• output_metadata (kwargs - key-argument pairs to add to) – dict - Dictionary with envi header information

• outpath – str - path to output file

• out_shape – tuple - dimensions of initialized file

• keys_to_del – list - keys to remove from output_metadata

• output_metadata