isofit.core.fileio ================== .. py:module:: isofit.core.fileio Attributes ---------- .. autoapisummary:: isofit.core.fileio.Logger isofit.core.fileio.typemap isofit.core.fileio.max_frames_size Classes ------- .. autoapisummary:: isofit.core.fileio.SpectrumFile isofit.core.fileio.InputData isofit.core.fileio.IO Functions --------- .. autoapisummary:: isofit.core.fileio.write_bil_chunk isofit.core.fileio.initialize_output Module Contents --------------- .. py:data:: Logger .. py:data:: typemap .. py:data:: max_frames_size :value: 100 .. py:class:: SpectrumFile(fname, write=False, n_rows=None, n_cols=None, n_bands=None, interleave=None, dtype=np.float32, wavelengths=None, fwhm=None, band_names=None, bad_bands='[]', zrange='{0.0, 1.0}', flag=-9999.0, ztitles='{Wavelength (nm), Magnitude}', map_info='{}', engine_name=None, isofit_version=None) A buffered file object that contains configuration information about formatting, etc. .. py:attribute:: frames .. py:attribute:: write :value: False .. py:attribute:: fname :value: b'.' .. py:attribute:: wl :value: None .. py:attribute:: band_names :value: None .. py:attribute:: fwhm :value: None .. py:attribute:: flag :value: -9999.0 .. py:attribute:: n_rows :value: None .. py:attribute:: n_cols :value: None .. py:attribute:: n_bands :value: None .. py:method:: open_map_with_retries() Try to open a memory map, handling Beowulf I/O issues. .. py:method:: get_frame(row) The frame is a 2D array, essentially a list of spectra. The self.frames list acts as a hash table to avoid storing the entire cube in memory. So we read them or create them on an as-needed basis. When the buffer flushes via a call to flush_buffers, they will be deleted. .. py:method:: write_spectrum(row, col, x) We write a spectrum. If a binary format file, we simply change the data cached in self.frames and defer file I/O until flush_buffers is called. .. py:method:: read_spectrum(row, col) Get a spectrum from the frame list or ASCII file. Note that if we are an ASCII file, we have already read the single spectrum and return it as-is (ignoring the row/column indices). .. py:method:: flush_buffers() Write to file, and refresh the memory map object. .. py:class:: InputData .. py:attribute:: meas :value: None .. py:attribute:: geom :value: None .. py:attribute:: reference_reflectance :value: None .. py:method:: clear() .. py:class:: IO(config: Config, forward: ForwardModel, full_statevec: list = []) ... .. py:attribute:: config .. py:attribute:: radiance_correction :value: None .. py:attribute:: meas_wl .. py:attribute:: meas_fwhm .. py:attribute:: writes :value: 0 .. py:attribute:: reads :value: 0 .. py:attribute:: n_rows :value: 1 .. py:attribute:: n_cols :value: 1 .. py:attribute:: bbl :value: '{}' .. py:attribute:: engine_name .. py:attribute:: n_sv .. py:attribute:: n_chan .. py:attribute:: flush_rate .. py:attribute:: simulation_mode .. py:attribute:: total_time :value: 0 .. py:attribute:: current_input_data .. py:attribute:: esd .. py:method:: get_components_at_index(row: int, col: int) -> InputData Load data from input files at the specified (row, col) index. :param row: row to retrieve data from :param col: column to retrieve data from :returns: object containing all current data reads :rtype: InputData .. py:method:: flush_buffers() Write all buffered output data to disk, and erase read buffers. .. py:method:: write_datasets(row: int, col: int, output: dict, states: List, flush_immediately=False) Write all valid datasets to disk (possibly buffered). :param row: row to write to :param col: column to write to :param output: dictionary with keys corresponding to config.input file references :param states: results states from inversion. In the MCMC case, these are interpreted as samples from the :param posterior: :param otherwise they are a gradient descent trajectory (with the last spectrum being the converged: :param solution).: :param flush_immediately: IO argument telling us to immediately write to disk, ignoring config settings .. py:method:: build_output(states: List, input_data: InputData, fm: ForwardModel, iv: Inversion, fill_value=-9999.0) Build the output to be written to disk as a dictionary :param states: results states from inversion. In the MCMC case, these are interpreted as samples from the :param posterior: :param otherwise they are a gradient descent trajectory (with the last spectrum being the converged: :param solution).: :param input_data: an InputData object :param fm: the forward model used to solve the inversion :param iv: the inversion object .. py:method:: write_spectrum(row: int, col: int, states: List, fm: ForwardModel, iv: Inversion, flush_immediately=False, input_data: InputData = None) Convenience function to build and write output in one step :param row: data row to write :param col: data column to write :param states: results states from inversion. In the MCMC case, these are interpreted as samples from the :param posterior: :param otherwise they are a gradient descent trajectory (with the last spectrum being the converged: :param solution).: :param meas: measurement radiance :param geom: geometry object of the observation :param fm: the forward model used to solve the inversion :param iv: the inversion object :param flush_immediately: IO argument telling us to immediately write to disk, ignoring config settings :param input_data: optionally overwride self.current_input_data .. py:method:: initialize_output_files(config, n_rows, n_cols, full_statevector) :staticmethod: .. py:method:: load_esd(file=None) :staticmethod: Loads an earth_sun_distance file. Defaults to the [env.data]/earth_sun_distance.txt if not provided :param file: ESD file to load :type file: str, default=None :returns: Loaded ESD. If the file fails to load, creates a default :rtype: np.array .. py:function:: write_bil_chunk(dat: numpy.array, outfile: str, line: int, shape: tuple, dtype: str = 'float32') -> None Write a chunk of data to a binary, BIL formatted data cube. :param dat: data to write :param outfile: output file to write to :param line: line of the output file to write to :param shape: shape of the output file :param dtype: output data type :returns: None .. py:function:: initialize_output(output_metadata, outpath, out_shape, **kwargs) Initialize output file by updating metadata and creating object. :param output_metadata: dict - Dictionary with envi header information :param outpath: str - path to output file :param out_shape: tuple - dimensions of initialized file :param keys_to_del: list - keys to remove from output_metadata :param kwargs - key-argument pairs to add to output_metadata: