setup_pipeline

class setup_pipeline.PostProcessingPipeline(path_to_inifile=None)

Bases: object

A class to handle the post-processing pipeline.

Initialize a pipeline

Parameters:

path_to_inifile (path) – The location of an ini file to read the parameters for the pipeline (default: None)

create_csv_and_slurm_job_files()

Creates all files the pipeline needs.

create_export_dirs()

Create directories for the finally exported data.

create_log_dirs()

Create directories to store the log files.

static parse_setup_params(path=None)

Parse inifile for running post-processing pipelines.

Parameters:

path (path) – The location of an ini file to read the parameters for the pipeline (default: None)

Return type:

dictionary

setup_pipeline.create_csv(GRID_TYPES=[], METALLICITIES=[], GRID_SLICES=[], COMPRESSIONS=[], step_name='', VERSION='', GRIDS_COMBINED=[], INTERPOLATION_METHODS=[], CONTROL_GRIDS=[], STOP_BEFORE_CARBON_DEPLETION=0, RERUN_TYPE='', DROP_MISSING_FILES=False, CREATE_PLOTS=[], DO_CHECKS=[], PATH_TO_GRIDS='', PATH='', previously_created_files=[], **kwargs)

Create csv file with a list containing all data to process a step.

setup_pipeline.slurm_job(job_name, step_name, PATH_TO_GRIDS=None, PATH_TO_POSYDON=None, PATH='.', ACCOUNT=None, PARTITION=None, WALLTIME=None, MAILTYPE=None, EMAIL=None, CREATE_GRID_SLICES=False, COMBINE_GRID_SLICES=False, CALCULATE_EXTRA_VALUES=False, TRAIN_INTERPOLATORS=False, EXPORT_DATASET=False, RERUN=False, VERBOSE=False, **kwargs)

Create slurm file.