matflow.ElementActionRun#

class matflow.ElementActionRun(id_, is_pending, element_action, index, data_idx, commands_idx, start_time, end_time, snapshot_start, snapshot_end, submission_idx, success, skip, exit_code, metadata, run_hostname)#

Bases: ElementActionRun

The Element Action Run (EAR) is an atomic unit of an enacted workflow, representing one unit of work (e.g., particular submitted job to run a program) within that overall workflow. With looping over, say, parameter spaces, there may be many EARs per element.

Parameters:
  • id (int) – The ID of the EAR.

  • is_pending (bool) – Whether this EAR is pending.

  • element_action – The particular element action that this is a run of.

  • index (int:) – The index of the run within the collection of runs.

  • data_idx (dict) – Used for looking up input data to the EAR.

  • commands_idx (list[int]) – Indices of commands to apply.

  • start_time (datetime) – Time of start of run, if the run has ever been started.

  • end_time (datetime) – Time of end of run, if the run has ever ended.

  • snapshot_start (dict) – Parameters for taking a snapshot of the data directory before the run. If unspecified, no snapshot will be taken.

  • snapshot_end (dict) – Parameters for taking a snapshot of the data directory after the run. If unspecified, no snapshot will be taken.

  • submission_idx (int) – What submission was this (if it has been submitted)?

  • success (bool) – Whether this EAR succeeded (if it has run).

  • skip (bool) – Whether this EAR was skipped.

  • exit_code (int) – The exit code, if known.

  • metadata (dict) – Metadata about the EAR.

  • run_hostname (str) – Where to run the EAR (if not locally).

  • id_ (int) –

Methods

compose_commands

Write the EAR's enactment to disk in preparation for submission.

get

Get a value (parameter, input, output, etc.) from the most recent iteration.

get_EAR_dependencies

Get EARs that this EAR depends on.

get_IFG_input_values

Get a dict of input values that are to be passed via an input file generator.

get_OFP_inputs

Get a dict of input values that are to be passed to output file parsers.

get_OFP_output_files

Get a dict of output files that are going to be parsed to generate one or more outputs.

get_OFP_outputs

Get the outputs obtained by parsing an output file.

get_all_previous_iteration_runs

Get a list of run over all iterations that correspond to this run, optionally including this run.

get_data_idx

Get the data index of a value in the most recent iteration.

get_dependent_EARs

Get downstream EARs that depend on this EAR.

get_environment

What environment to run in?

get_environment_spec

What environment to run in?

get_input_dependencies

Get information about locally defined input, sequence, and schema-default values that this EAR depends on.

get_input_values

Get a dict of (optionally a subset of) inputs values for this run.

get_input_values_direct

Get a dict of input values that are to be passed directly to a Python script function.

get_parameter_names

Get parameter types associated with a given prefix.

get_parameter_sources

Get the source or sources of a parameter in the most recent iteration.

get_resources

Resolve specific resources for this EAR, considering all applicable scopes and template-level resources.

write_source

Write values to files in standard formats.

Attributes

action

The action this is a run of.

app

commands_idx

Indices of commands to apply.

data_idx

Used for looking up input data to the EAR.

dir_diff

The changes to the EAR working directory due to the execution of this EAR.

element

The element this is a run of.

element_action

The particular element action that this is a run of.

element_iteration

The iteration information of this run.

end_time

When the EAR finished.

env_spec

Environment details.

exit_code

The exit code of the underlying program run by the EAR, if known.

id_

The ID of the EAR.

index

Run index.

input_files

The input files to the controlled program.

inputs

The inputs to this EAR.

is_pending

Whether this EAR is pending.

metadata

Metadata about the EAR.

output_files

The output files from the controlled program.

outputs

The outputs from this EAR.

resources

The resources to use with (or used by) this EAR.

run_hostname

Where to run the EAR, if known/specified.

skip

Was the EAR skipped?

snapshot_end

The snapshot of the data directory at the end of the run.

snapshot_start

The snapshot of the data directory at the start of the run.

start_time

When the EAR started.

status

The state of this EAR.

submission_idx

What actual submission index was this?

success

Did the EAR succeed?

task

The task that this EAR is part of the implementation of.

workflow

The workflow this is a run of.

property action#

The action this is a run of.

app = App(name='MatFlow', version='0.3.0a132')#
property commands_idx#

Indices of commands to apply.

compose_commands(jobscript, JS_action_idx)#

Write the EAR’s enactment to disk in preparation for submission.

Returns:

  • commands – List of argument words for the command that enacts the EAR.

  • shell_vars – Dict whose keys are command indices, and whose values are lists of tuples, where each tuple contains: (parameter name, shell variable name, “stdout”/”stderr”).

Parameters:
Return type:

Tuple[str, List[str], List[int]]

property data_idx#

Used for looking up input data to the EAR.

property dir_diff: DirectorySnapshotDiff#

The changes to the EAR working directory due to the execution of this EAR.

property element#

The element this is a run of.

property element_action#

The particular element action that this is a run of.

property element_iteration#

The iteration information of this run.

property end_time#

When the EAR finished.

property env_spec: Dict[str, Any]#

Environment details.

property exit_code#

The exit code of the underlying program run by the EAR, if known.

get(path=None, default=None, raise_on_missing=False, raise_on_unset=False)#

Get a value (parameter, input, output, etc.) from the most recent iteration.

Parameters:
  • path (str) – Path to the value.

  • default (Any) – Default value to provide if value absent.

  • raise_on_missing (bool) – Whether to raise an exception on an absent value. If not, the default is returned.

  • raise_on_unset (bool) – Whether to raise an exception on an explicitly unset value. If not, the default is returned.

get_EAR_dependencies(as_objects=False)#

Get EARs that this EAR depends on.

get_IFG_input_values()#

Get a dict of input values that are to be passed via an input file generator.

Return type:

Dict[str, Any]

get_OFP_inputs()#

Get a dict of input values that are to be passed to output file parsers.

Return type:

Dict[str, str | List[str]]

get_OFP_output_files()#

Get a dict of output files that are going to be parsed to generate one or more outputs.

Return type:

Dict[str, str | List[str]]

get_OFP_outputs()#

Get the outputs obtained by parsing an output file.

Return type:

Dict[str, str | List[str]]

get_all_previous_iteration_runs(include_self=True)#

Get a list of run over all iterations that correspond to this run, optionally including this run.

Parameters:

include_self (bool) –

get_data_idx(path=None)#

Get the data index of a value in the most recent iteration.

Parameters:

path (str) – Path to the parameter.

get_dependent_EARs(as_objects=False)#

Get downstream EARs that depend on this EAR.

Return type:

List[int | ElementActionRun]

get_environment()#

What environment to run in?

Return type:

Environment

get_environment_spec()#

What environment to run in?

Return type:

str

get_input_dependencies()#

Get information about locally defined input, sequence, and schema-default values that this EAR depends on. Note this does not get values from this EAR’s task/schema, because the aim of this method is to help determine which upstream tasks this EAR depends on.

get_input_values(inputs=None, label_dict=True)#

Get a dict of (optionally a subset of) inputs values for this run.

Parameters:
  • inputs (List[str] | Dict[str, Dict] | None) – If specified, a list of input parameter types to include, or a dict whose keys are input parameter types to include. For schema inputs that have multiple=True, the input type should be labelled. If a dict is passed, and the key “all_iterations` is present and True, the return for that input will be structured to include values for all previous iterations.

  • label_dict (bool) – If True, arrange the values of schema inputs with multiple=True as a dict whose keys are the labels. If False, labels will be included in the top level keys.

Return type:

Dict[str, Any]

get_input_values_direct(label_dict=True)#

Get a dict of input values that are to be passed directly to a Python script function.

Parameters:

label_dict (bool) –

get_parameter_names(prefix)#

Get parameter types associated with a given prefix.

For inputs, labels are ignored. See Action.get_parameter_names for more information.

Parameters:

prefix (str) – One of “inputs”, “outputs”, “input_files”, “output_files”.

Return type:

List[str]

get_parameter_sources(path=None, typ=None, as_strings=False, use_task_index=False)#

Get the source or sources of a parameter in the most recent iteration.

Parameters:
  • path (str) – Path to the parameter.

  • typ (str) – The parameter type.

  • as_strings (bool) – Whether to return the result as human-readable strings.

  • use_task_index (bool) – Whether to use the task index.

get_resources()#

Resolve specific resources for this EAR, considering all applicable scopes and template-level resources.

property id_: int#

The ID of the EAR.

property index#

Run index.

property input_files#

The input files to the controlled program.

property inputs#

The inputs to this EAR.

property is_pending: bool#

Whether this EAR is pending.

property metadata#

Metadata about the EAR.

property output_files#

The output files from the controlled program.

property outputs#

The outputs from this EAR.

property resources#

The resources to use with (or used by) this EAR.

property run_hostname#

Where to run the EAR, if known/specified.

property skip#

Was the EAR skipped?

property snapshot_end#

The snapshot of the data directory at the end of the run.

property snapshot_start#

The snapshot of the data directory at the start of the run.

property start_time#

When the EAR started.

property status#

The state of this EAR.

property submission_idx#

What actual submission index was this?

property success#

Did the EAR succeed?

property task#

The task that this EAR is part of the implementation of.

property workflow#

The workflow this is a run of.

write_source(js_idx, js_act_idx)#

Write values to files in standard formats.

Parameters:
  • js_idx (int) –

  • js_act_idx (int) –