matflow.ElementActionRun#

class matflow.ElementActionRun(id_, is_pending, element_action, index, data_idx, commands_idx, start_time, end_time, snapshot_start, snapshot_end, submission_idx, success, skip, exit_code, metadata, run_hostname)#

Bases: ElementActionRun

The Element Action Run (EAR) is an atomic unit of an enacted workflow, representing one unit of work (e.g., particular submitted job to run a program) within that overall workflow. With looping over, say, parameter spaces, there may be many EARs per element.

Parameters:

id (int) – The ID of the EAR.
is_pending (bool) – Whether this EAR is pending.
element_action – The particular element action that this is a run of.
index (int:) – The index of the run within the collection of runs.
data_idx (dict) – Used for looking up input data to the EAR.
commands_idx (list[int]) – Indices of commands to apply.
start_time (datetime) – Time of start of run, if the run has ever been started.
end_time (datetime) – Time of end of run, if the run has ever ended.
snapshot_start (dict) – Parameters for taking a snapshot of the data directory before the run. If unspecified, no snapshot will be taken.
snapshot_end (dict) – Parameters for taking a snapshot of the data directory after the run. If unspecified, no snapshot will be taken.
submission_idx (int) – What submission was this (if it has been submitted)?
success (bool) – Whether this EAR succeeded (if it has run).
skip (bool) – Whether this EAR was skipped.
exit_code (int) – The exit code, if known.
metadata (dict) – Metadata about the EAR.
run_hostname (str) – Where to run the EAR (if not locally).
id_ (int) –

Methods

`compose_commands`	Write the EAR's enactment to disk in preparation for submission.
`get`	Get a value (parameter, input, output, etc.) from the most recent iteration.
`get_EAR_dependencies`	Get EARs that this EAR depends on.
`get_IFG_input_values`	Get a dict of input values that are to be passed via an input file generator.
`get_OFP_inputs`	Get a dict of input values that are to be passed to output file parsers.
`get_OFP_output_files`	Get a dict of output files that are going to be parsed to generate one or more outputs.
`get_OFP_outputs`	Get the outputs obtained by parsing an output file.
`get_all_previous_iteration_runs`	Get a list of run over all iterations that correspond to this run, optionally including this run.
`get_data_idx`	Get the data index of a value in the most recent iteration.
`get_dependent_EARs`	Get downstream EARs that depend on this EAR.
`get_environment`	What environment to run in?
`get_environment_spec`	What environment to run in?
`get_input_dependencies`	Get information about locally defined input, sequence, and schema-default values that this EAR depends on.
`get_input_values`	Get a dict of (optionally a subset of) inputs values for this run.
`get_input_values_direct`	Get a dict of input values that are to be passed directly to a Python script function.
`get_parameter_names`	Get parameter types associated with a given prefix.
`get_parameter_sources`	Get the source or sources of a parameter in the most recent iteration.
`get_resources`	Resolve specific resources for this EAR, considering all applicable scopes and template-level resources.
`write_source`	Write values to files in standard formats.

Attributes

`action`	The action this is a run of.
`app`
`commands_idx`	Indices of commands to apply.
`data_idx`	Used for looking up input data to the EAR.
`dir_diff`	The changes to the EAR working directory due to the execution of this EAR.
`element`	The element this is a run of.
`element_action`	The particular element action that this is a run of.
`element_iteration`	The iteration information of this run.
`end_time`	When the EAR finished.
`env_spec`	Environment details.
`exit_code`	The exit code of the underlying program run by the EAR, if known.
`id_`	The ID of the EAR.
`index`	Run index.
`input_files`	The input files to the controlled program.
`inputs`	The inputs to this EAR.
`is_pending`	Whether this EAR is pending.
`metadata`	Metadata about the EAR.
`output_files`	The output files from the controlled program.
`outputs`	The outputs from this EAR.
`resources`	The resources to use with (or used by) this EAR.
`run_hostname`	Where to run the EAR, if known/specified.
`skip`	Was the EAR skipped?
`snapshot_end`	The snapshot of the data directory at the end of the run.
`snapshot_start`	The snapshot of the data directory at the start of the run.
`start_time`	When the EAR started.
`status`	The state of this EAR.
`submission_idx`	What actual submission index was this?
`success`	Did the EAR succeed?
`task`	The task that this EAR is part of the implementation of.
`workflow`	The workflow this is a run of.

property action#: The action this is a run of.

app = App(name='MatFlow', version='0.3.0a131')#

property commands_idx#: Indices of commands to apply.

compose_commands(jobscript, JS_action_idx)#

Write the EAR’s enactment to disk in preparation for submission.

Returns:

commands – List of argument words for the command that enacts the EAR.
shell_vars – Dict whose keys are command indices, and whose values are lists of tuples, where each tuple contains: (parameter name, shell variable name, “stdout”/”stderr”).

Parameters:

jobscript (Jobscript) –
JS_action_idx (int) –

Return type:

Tuple[str, List[str], List[int]]

property data_idx#: Used for looking up input data to the EAR.

property dir_diff: DirectorySnapshotDiff#: The changes to the EAR working directory due to the execution of this EAR.

property element#: The element this is a run of.

property element_action#: The particular element action that this is a run of.

property element_iteration#: The iteration information of this run.

property end_time#: When the EAR finished.

property env_spec: Dict[str, Any]#: Environment details.

property exit_code#: The exit code of the underlying program run by the EAR, if known.

get(path=None, default=None, raise_on_missing=False, raise_on_unset=False)#

Get a value (parameter, input, output, etc.) from the most recent iteration.

Parameters:

path (str) – Path to the value.
default (Any) – Default value to provide if value absent.
raise_on_missing (bool) – Whether to raise an exception on an absent value. If not, the default is returned.
raise_on_unset (bool) – Whether to raise an exception on an explicitly unset value. If not, the default is returned.

get_EAR_dependencies(as_objects=False)#: Get EARs that this EAR depends on.

get_IFG_input_values()#

Get a dict of input values that are to be passed via an input file generator.

Return type:: Dict[str, Any]

get_OFP_inputs()#

Get a dict of input values that are to be passed to output file parsers.

Return type:: Dict[str, str | List[str]]

get_OFP_output_files()#

Get a dict of output files that are going to be parsed to generate one or more outputs.

Return type:: Dict[str, str | List[str]]

get_OFP_outputs()#

Get the outputs obtained by parsing an output file.

Return type:: Dict[str, str | List[str]]

get_all_previous_iteration_runs(include_self=True)#

Get a list of run over all iterations that correspond to this run, optionally including this run.

Parameters:: include_self (bool) –

get_data_idx(path=None)#

Get the data index of a value in the most recent iteration.

Parameters:: path (str) – Path to the parameter.

get_dependent_EARs(as_objects=False)#

Get downstream EARs that depend on this EAR.

Return type:: List[int | ElementActionRun]

get_environment()#

What environment to run in?

Return type:: Environment

get_environment_spec()#

What environment to run in?

Return type:: str

get_input_dependencies()#: Get information about locally defined input, sequence, and schema-default values that this EAR depends on. Note this does not get values from this EAR’s task/schema, because the aim of this method is to help determine which upstream tasks this EAR depends on.

get_input_values(inputs=None, label_dict=True)#

Get a dict of (optionally a subset of) inputs values for this run.

Parameters:

inputs (List[str] | Dict[str, Dict] | None) – If specified, a list of input parameter types to include, or a dict whose keys are input parameter types to include. For schema inputs that have multiple=True, the input type should be labelled. If a dict is passed, and the key “all_iterations` is present and True, the return for that input will be structured to include values for all previous iterations.
label_dict (bool) – If True, arrange the values of schema inputs with multiple=True as a dict whose keys are the labels. If False, labels will be included in the top level keys.

Return type:

Dict[str, Any]

get_input_values_direct(label_dict=True)#

Get a dict of input values that are to be passed directly to a Python script function.

Parameters:: label_dict (bool) –

get_parameter_names(prefix)#

Get parameter types associated with a given prefix.

For inputs, labels are ignored. See Action.get_parameter_names for more information.

Parameters:: prefix (str) – One of “inputs”, “outputs”, “input_files”, “output_files”.
Return type:: List[str]

get_parameter_sources(path=None, typ=None, as_strings=False, use_task_index=False)#

Get the source or sources of a parameter in the most recent iteration.

Parameters:

path (str) – Path to the parameter.
typ (str) – The parameter type.
as_strings (bool) – Whether to return the result as human-readable strings.
use_task_index (bool) – Whether to use the task index.

get_resources()#: Resolve specific resources for this EAR, considering all applicable scopes and template-level resources.

property id_: int#: The ID of the EAR.

property index#: Run index.

property input_files#: The input files to the controlled program.

property inputs#: The inputs to this EAR.

property is_pending: bool#: Whether this EAR is pending.

property metadata#: Metadata about the EAR.

property output_files#: The output files from the controlled program.

property outputs#: The outputs from this EAR.

property resources#: The resources to use with (or used by) this EAR.

property run_hostname#: Where to run the EAR, if known/specified.

property skip#: Was the EAR skipped?

property snapshot_end#: The snapshot of the data directory at the end of the run.

property snapshot_start#: The snapshot of the data directory at the start of the run.

property start_time#: When the EAR started.

property status#: The state of this EAR.

property submission_idx#: What actual submission index was this?

property success#: Did the EAR succeed?

property task#: The task that this EAR is part of the implementation of.

property workflow#: The workflow this is a run of.

write_source(js_idx, js_act_idx)#

Write values to files in standard formats.

Parameters:

js_idx (int) –
js_act_idx (int) –