padre_meddea.io.fits_tools#

Provides utilities to read and write fits files

Functions

add_process_info_to_header(header[, n])

Add processing info metadata to a fits header.

concatenate_files(files_to_combine[, ...])

Concatenate multiple FITS files into a single daily FITS file, properly combining headers and data.

filter_files_by_provenance(files_to_combine)

Remove files that are already in the provenance table of the existing file.

get_bunit(data_level, data_type)

Get the bunit and comment for a given data level and data type.

get_comment(keyword)

Given a keyword, return the standard comment for a header card.

get_hdu_data_times(hdul_dict, hdu_name)

Extract time information from the data within a FITS file.

get_obs_header(data_level, data_type)

Create a standard FITS header for the observation.

get_prhsh([n, a])

Get the processing hash and standard comment for FITS header.

get_primary_header(file_path, data_level, ...)

Create a standard FITS primary header following SOLARNET conventions.

get_prlib([n, a])

Get the processing library description and standard comment for FITS header.

get_provenance_filenames(fits_path)

Read the PROVENANCE table from an existing FITS file and return the set of filenames.

get_prproc([n])

Get the processing procedure description and standard comment for FITS header.

get_prpver([n])

Get the processing version and standard comment for FITS header.

get_prstep([n])

Get the processing step description and standard comment for FITS header.

get_prver([n, a])

Get the processing version and standard comment for FITS header.

hdu_to_dict(hdu)

Given an hdu, convert it to an hdu dict

split_hdul_by_day(hdul_dict)

Split a FITS HDU dictionary into multiple dictionaries based on day boundaries.

split_provenance_tables_by_day(files_to_combine)

Splits provenance entries into daily Astropy Tables.

update_hdul_date_metadata(hdul_dict)

Function to update the date metadata in the HDU dictionary.

update_hdul_filename_metadata(hdul_dict, ...)

Function to update the filename metadata in the HDU dictionary.
padre_meddea.io.fits_tools._concatenate_input_files(input_files: list[Path], hdu_dict: dict) → dict[source]#

Process additional files and combine their data with the initial structure.

Parameters:

  • input_files (list[Path]) – List of files being combined

  • hdu_dict (dict) – Initial HDU dictionary structure

Returns:

dict – Updated HDU dictionary with combined data

padre_meddea.io.fits_tools._filter_hdul_time_ranges(hdul_dict: dict[int, dict], start_time: Time, end_time: Time) → dict[int, dict][source]#

Filter the HDU dictionary to only include data within the specified time range.

Parameters:

hdul_dict (dict[int, dict]) –

Dictionary representation of a FITS HDU list, where each key is an index into the HDUL Each value is a dict: {

”header”: fits.Header, “data”: Table or np.ndarray, “type”: “bintable” or “image”, “name”: str (optional)

}

Returns:

dict[int, dict] – Filtered HDU dictionary containing only data within the specified time range.

padre_meddea.io.fits_tools._get_combined_list(files_to_combine: list[Path], existing_file: Path = None) → list[Path][source]#

Prepare the list of files to be combined.

Parameters:

  • files_to_combine (list of Path) – List of FITS files to combine

  • existing_file (Path, optional) – Existing daily FITS file to append to

Returns:

list of Path – Sorted list of all files to process

padre_meddea.io.fits_tools._get_output_path(first_file: Path, date_beg: Time) → Path[source]#

Determine the output file path if not provided.

Parameters:

  • first_file (Path) – First file to extract metadata from

  • date_beg (Time) – Beginning date for filename generation

Returns:

Path – Determined output file path

padre_meddea.io.fits_tools._init_hdul_structure(template_file: Path) → dict[source]#

Initialize the HDUL dictionary structure from the first file.

Parameters:

  • template_file (Path) – First FITS file to use as template

  • date_beg (Time) – Calculated time values

  • date_end (Time) – Calculated time values

  • date_avg (Time) – Calculated time values

  • all_parent_files (list[str]) – Combined list of parent files

Returns:

dict – HDU dictionary structure formatted as: {

0: {

“header”: fits.Header, “data”: None, “type”: “primary”

}, 1: {

”header”: fits.Header, “data”: Table or np.ndarray, “type”: “bintable” or “image”, “name”: str (optional)

}

padre_meddea.io.fits_tools._sort_hdul_template(hdul_dict: dict)[source]#
padre_meddea.io.fits_tools._write_output_file(hdu_dict: dict, outfile: Path, retries: int = 5, delay: float = 1.0) → None[source]#

Construct and write the output FITS file with retry mechanism.

Parameters:

  • hdu_dict (dict) – Dictionary containing HDU information

  • outfile (Path) – Output file path

  • retries (int, optional) – Number of retry attempts, default is 3

  • delay (float, optional) – Delay between retries in seconds, default is 1.0

padre_meddea.io.fits_tools.add_process_info_to_header(header: Header, n: int = 1) → Header[source]#

Add processing info metadata to a fits header.

It adds the following SOLARNET compatible FITS cards; PRSTEPn, PRPROCn, PRPVERn, PRLIBnA, PRVERnA, PRLIBnA, PRHSHnA, PRVERnB

Parameters:

  • header (fits.Header) – The fits header to add the new cards to

  • n (int, default 1) – The processing step number. Must be >= 1 and <= 9.

Returns:

header (fits.Header)

padre_meddea.io.fits_tools.concatenate_files(files_to_combine: list[Path], existing_file: Path = None) → list[Path][source]#

Concatenate multiple FITS files into a single daily FITS file, properly combining headers and data.

The function also tracks all parent files in the PARENTXT header keyword.

Note: This function assumes that there is no data stored in the PrimaryHDU of the FITS files.

Parameters:

  • files_to_combine (list of Path) – List of FITS files to combine. Assumed to have the same structure.

  • existing_file (Path, optional) – Existing daily FITS file to append to.

Returns:

output_file (list of Path) – List containing the output file paths of the concatenated daily FITS files.

padre_meddea.io.fits_tools.filter_files_by_provenance(files_to_combine: list[Path], existing_file: Path = None) → list[Path][source]#

Remove files that are already in the provenance table of the existing file. Logs a warning for each file skipped.

padre_meddea.io.fits_tools.get_bunit(data_level: str, data_type: str) → Tuple[str, str][source]#

Get the bunit and comment for a given data level and data type.

Parameters:

  • data_level (str) – Data Processing step (e.g., ‘L0’, ‘L1’)

  • data_type (str) – Type of data being processed

Returns:

tuple – A tuple of (bunit, comment)

padre_meddea.io.fits_tools.get_comment(keyword: str) → str[source]#

Given a keyword, return the standard comment for a header card.

padre_meddea.io.fits_tools.get_hdu_data_times(hdul_dict: dict[int, dict], hdu_name: str) → Time[source]#

Extract time information from the data within a FITS file.

This function parses times differently based on the file descriptor (eventlist, hk, spec) extracted from the filename. It accesses the appropriate HDU and data columns for each file type to calculate accurate time values.

Parameters:

  • hdul_dict (dict[int, dict]) –

    Dictionary representation of a FITS HDU list, where each key is an index into the HDUL Each value is a dict: {

    ”header”: fits.Header, “data”: Table or np.ndarray, “type”: “bintable” or “image”, “name”: str (optional)

    }

  • hdu_name (str) – Name of the HDU to extract time information from

Returns:

Time – Astropy Time object containing the time values extracted from the file data

Raises:

ValueError – If the file descriptor is not recognized or supported

padre_meddea.io.fits_tools.get_obs_header(data_level: str, data_type: str)[source]#

Create a standard FITS header for the observation.

This function creates a new FITS header with standard metadata including the current date, default PADRE attributes, processing information, data level, data type, original APID, and original filename.

Parameters:

  • data_level (str) – Data Processing step (e.g., ‘L0’, ‘L1’)

  • data_type (str) – Type of data being processed

Returns:

fits.Header – A FITS header populated with standard metadata

padre_meddea.io.fits_tools.get_prhsh(n: int = 1, a: str = 'A') → Tuple[str, str][source]#

Get the processing hash and standard comment for FITS header. This function returns a tuple containing the processing hash and the corresponding standard comment based on the Processing step.

Parameters:

  • n (int, optional) – Processing step number, default is 1.

  • a (str, optional) – Library version, default is A.

Returns:

tuple – A tuple of (processing_hash, standard_comment)

padre_meddea.io.fits_tools.get_primary_header(file_path: Path, data_level: str, data_type: str, procesing_step: int = 1) → Header[source]#

Create a standard FITS primary header following SOLARNET conventions.

This function creates a new FITS header with standard metadata including the current date, default PADRE attributes, processing information, data level, data type, original APID, and original filename.

Parameters:

  • file_path (Path) – Path to the original file, used to extract the filename

  • data_level (str) – Data Processing step (e.g., ‘L0’, ‘L1’)

  • data_type (str) – Type of data being processed

  • procesing_step (int, default 1) – Processing step number to be added to header metadata

Returns:

fits.Header – A FITS header populated with standard metadata

padre_meddea.io.fits_tools.get_prlib(n: int = 1, a: str = 'A') → Tuple[str, str][source]#

Get the processing library description and standard comment for FITS header. This function returns a tuple containing the processing library description and the corresponding standard comment based on the Processing step.

Parameters:

  • n (int, optional) – Processing step number, default is 1.

  • a (str, optional) – Library version, default is A.

Returns:

tuple – A tuple of (processing_library_description, standard_comment)

padre_meddea.io.fits_tools.get_provenance_filenames(fits_path: Path) → set[str][source]#

Read the PROVENANCE table from an existing FITS file and return the set of filenames.

padre_meddea.io.fits_tools.get_prproc(n: int = 1) → Tuple[str, str][source]#

Get the processing procedure description and standard comment for FITS header. This function returns a tuple containing the processing procedure description and the corresponding standard comment based on the Processing step.

Parameters:

n (int, optional) – Processing step number, default is 1.

Returns:

tuple – A tuple of (processing_procedure_description, standard_comment)

padre_meddea.io.fits_tools.get_prpver(n: int = 1) → Tuple[str, str][source]#

Get the processing version and standard comment for FITS header. This function returns a tuple containing the processing version and the corresponding standard comment based on the Processing step.

Parameters:

n (int, optional) – Processing step number, default is 1.

Returns:

tuple – A tuple of (processing_version, standard_comment)

padre_meddea.io.fits_tools.get_prstep(n: int = 1) → Tuple[str, str][source]#

Get the processing step description and standard comment for FITS header. This function returns a tuple containing the processing step description and the corresponding standard comment based on the Processing step.

Parameters:

n (int, optional) – Processing step number, default is 1. 1: Raw to L1 2: L1 to L2 3: L2 to L3 4: L3 to L4

Returns:

tuple – A tuple of (processing_step_description, standard_comment)

Raises:

ValueError – If the Processing step number is not in the range 1-4.

padre_meddea.io.fits_tools.get_prver(n: int = 1, a: str = 'A') → Tuple[str, str][source]#

Get the processing version and standard comment for FITS header. This function returns a tuple containing the processing version and the corresponding standard comment based on the Processing step.

Parameters:

  • n (int, optional) – Processing step number, default is 1.

  • a (str, optional) – Library version, default is A.

Returns:

tuple – A tuple of (processing_version, standard_comment)

padre_meddea.io.fits_tools.hdu_to_dict(hdu: <module 'astropy.io.fits.hdu' from '/home/docs/checkouts/readthedocs.org/user_builds/padre-meddea/envs/latest/lib/python3.13/site-packages/astropy/io/fits/hdu/__init__.py'>) → dict[source]#

Given an hdu, convert it to an hdu dict

padre_meddea.io.fits_tools.split_hdul_by_day(hdul_dict: dict) → dict[source]#

Split a FITS HDU dictionary into multiple dictionaries based on day boundaries.

Parameters:#

hdul_dictdict

Dictionary representation of a FITS HDU list

Returns:#

dict

Dictionary where keys are days (as strings in ‘YYYY-MM-DD’ format) and values are HDU dictionaries containing only data from that day

padre_meddea.io.fits_tools.split_provenance_tables_by_day(files_to_combine, existing_file=None)[source]#

Splits provenance entries into daily Astropy Tables. If a file spans multiple days, it is duplicated across those days with start/end times clipped to each day. Take note if the file has no DATE-BEG or DATE-END in the header, it will utilize the file DATEREF as a fallback.

Parameters:

  • files_to_combine (list[Path or object with .name]) – List of FITS file paths or similar objects.

  • existing_file (Path or None) – Optional existing provenance FITS file to pull previous provenance from.

Returns:

dict[str, astropy.table.Table] – Dictionary mapping ‘YYYY-MM-DD’ -> Table with columns FILENAME, DATE_BEG, DATE_END

padre_meddea.io.fits_tools.update_hdul_date_metadata(hdul_dict: dict) → dict[source]#

Function to update the date metadata in the HDU dictionary. This function will update the DATE-BEG, DATE-END, DATE-AVG, and DATEREF keywords in the headers of the HDUs based on the data contained within them.

Parameters:

hdul_dict (dict) – Dictionary representation of a FITS HDU list, where each key is an index and the value is a dictionary with keys “header”, “data”, “type”, and optionally “name”.

Returns:

dict – Updated dictionary representation of the FITS HDU list with updated date metadata.

padre_meddea.io.fits_tools.update_hdul_filename_metadata(hdul_dict: dict, output_file: Path, provenance_table: Table = None) → dict[source]#

Function to update the filename metadata in the HDU dictionary. This function will update the FILENAME keyword in the headers of the HDUs based on the output file name.

Parameters:

  • hdul_dict (dict) – Dictionary representation of a FITS HDU list, where each key is an index and the value is a dictionary with keys “header”, “data”, “type”, and optionally “name”.

  • output_file (Path) – path the hdul_dict is being written to, used to extract filename metadata.

Returns:

dict – Updated dictionary representation of the FITS HDU list with updated filename metadata.