matflow.Action#

class matflow.Action(environments=None, commands=None, script=None, script_data_in=None, script_data_out=None, script_data_files_use_opt=False, script_exe=None, script_pass_env_spec=False, abortable=False, input_file_generators=None, output_file_parsers=None, input_files=None, output_files=None, rules=None, save_files=None, clean_up=None)#

Bases: Action

An atomic component of a workflow that will be enacted within an iteration structure.

Parameters:

environments (list[ActionEnvironment]) – The environments in which this action can run.
commands (list[Command]) – The commands to be run by this action.
script (str) – The name of the Python script to run.
script_data_in (str) – Information about data input to the script.
script_data_out (str) – Information about data output from the script.
script_data_files_use_opt (bool) – If True, script data input and output file paths will be passed to the script execution command line with an option like --input-json or --output-hdf5 etc. If False, the file paths will be passed on their own. For Python scripts, options are always passed, and this parameter is overwritten to be True, regardless of its initial value.
script_exe (str) – The executable to use to run the script.
script_pass_env_spec (bool) – Whether to pass the environment details to the script.
abortable (bool) – Whether this action can be aborted.
input_file_generators (list[InputFileGenerator]) – Any applicable input file generators.
output_file_parsers (list[OutputFileParser]) – Any applicable output file parsers.
input_files (list[FileSpec]) – The input files to the action’s commands.
output_files (list[FileSpec]) – The output files from the action’s commands.
rules (list[ActionRule]) – How to determine whether to run the action.
save_files (list[str]) – The names of files to be explicitly saved after each step.
clean_up (list[str]) – The names of files to be deleted after each step.

Methods

`compose_source`	Generate the file contents of this source.
`expand`	Expand this action into a list of actions if necessary.
`from_json_like`	Make an instance of this class from JSON (or YAML) data.
`generate_data_index`	Generate the data index for this action of an element iteration whose overall data index is passed.
`get_command_input_file_labels`	Get input files types from commands.
`get_command_input_types`	Get parameter types from commands.
`get_command_output_types`	Get parameter types from command stdout and stderr arguments.
`get_commands_action_env`	Get the actual environment to use for the action commands.
`get_environment`	Get the primary environment.
`get_environment_name`	Get the name of the primary environment.
`get_environment_spec`	Get the specification for the primary envionment, assuming it has been expanded.
`get_input_file_generator_action_env`	Get the actual environment to use for an input file generator.
`get_input_file_labels`	Get the labels from the input files.
`get_input_types`	Get the input types that are consumed by commands and input file generators of this action.
`get_output_file_labels`	Get the labels from the output files.
`get_output_file_parser_action_env`	Get the actual environment to use for an output file parser.
`get_output_types`	Get the output types that are produced by command standard outputs and errors, and by output file parsers of this action.
`get_param_dump_file_path_HDF5`	Get the path of the HDF56 dump file.
`get_param_dump_file_path_JSON`	Get the path of the JSON dump file.
`get_param_load_file_path_HDF5`	Get the path of the HDF5 load file.
`get_param_load_file_path_JSON`	Get the path of the JSON load file.
`get_parameter_dependence`	Find if/where a given parameter is used by the action.
`get_parameter_names`	Get parameter types associated with a given prefix.
`get_possible_scopes`	Get the action scopes that are inclusive of this action, ordered by decreasing specificity.
`get_precise_scope`	Get the exact scope of this action.
`get_required_executables`	Return executable labels required by this action.
`get_script_name`	Return the script name.
`get_snippet_script_path`	Get the substituted script snippet path, or False if there is no snippet.
`get_snippet_script_str`	Get the substituted script snippet path as a string.
`is_input_type_required`	Determine if the given input type is required by this action.
`is_snippet_script`	Returns True if the provided script string represents a script snippets that is to be modified before execution (e.g.
`process_script_data_formats`	Convert script data information into standard form.
`test_rules`	Test all rules against the specified element iteration.
`to_dict`	Serialize this object as a dictionary.
`to_json_like`	Serialize this object as an object structure that can be trivially converted to JSON.

Attributes

`app`
`script_data_in_grouped`	Get input parameter types by script data-in format.
`script_data_in_has_direct`	Return True if the script requires some inputs to be passed directly from the app.
`script_data_in_has_files`	Return True if the script requires some inputs to be passed via an intermediate file format.
`script_data_out_grouped`	Get output parameter types by script data-out format.
`script_data_out_has_direct`	Return True if the script produces some outputs to be passed directly to the app.
`script_data_out_has_files`	Return True if the script produces some outputs via an intermediate file format.
`script_is_python`	Return True if the script is a Python script (determined by the file extension)
`task_schema`	The task schema that this action came from.
`commands`	The commands to be run by this action.
`script`	The name of the Python script to run.
`script_data_in`	Information about data input to the script.
`script_data_out`	Information about data output from the script.
`script_data_files_use_opt`	If True, script data input and output file paths will be passed to the script execution command line with an option like --input-json or --output-hdf5 etc.
`script_exe`	The executable to use to run the script.
`script_pass_env_spec`	Whether to pass the environment details to the script.
`environments`	The environments in which this action can run.
`abortable`	Whether this action can be aborted.
`input_file_generators`	Any applicable input file generators.
`output_file_parsers`	Any applicable output file parsers.
`input_files`	The input files to the action's commands.
`output_files`	The output files from the action's commands.
`rules`	How to determine whether to run the action.
`save_files`	The names of files to be explicitly saved after each step.
`clean_up`	The names of files to be deleted after each step.

abortable#: Whether this action can be aborted.

app = App(name='MatFlow', version='0.3.0a132')#

clean_up#: The names of files to be deleted after each step.

commands#: The commands to be run by this action.

compose_source(snip_path)#

Generate the file contents of this source.

Parameters:: snip_path (Path) –
Return type:: str

environments#: The environments in which this action can run.

expand()#: Expand this action into a list of actions if necessary. This converts input file generators and output file parsers into their own actions.

classmethod from_json_like(json_like, shared_data=None)#

Make an instance of this class from JSON (or YAML) data.

Parameters:

json_like (Union[Dict, List]) – The data to deserialise.
shared_data (Optional[Dict[str, ObjectList]]) – Shared context data.

Return type:

The deserialised object.

generate_data_index(act_idx, EAR_ID, schema_data_idx, all_data_idx, workflow, param_source)#

Generate the data index for this action of an element iteration whose overall data index is passed.

This mutates all_data_idx.

Return type:: List[int]

get_command_input_file_labels()#

Get input files types from commands.

Return type:: Tuple[str]

get_command_input_types(sub_parameters=False)#

Get parameter types from commands.

Parameters:: sub_parameters (bool) – If True, sub-parameters (i.e. dot-delimited parameter types) will be returned untouched. If False (default), only return the root parameter type and disregard the sub-parameter part.
Return type:: Tuple[str]

get_command_output_types()#

Get parameter types from command stdout and stderr arguments.

Return type:: Tuple[str]

get_commands_action_env()#: Get the actual environment to use for the action commands.

get_environment()#

Get the primary environment.

Return type:: Environment

get_environment_name()#

Get the name of the primary environment.

Return type:: str

get_environment_spec()#

Get the specification for the primary envionment, assuming it has been expanded.

Return type:: Dict[str, Any]

get_input_file_generator_action_env(input_file_generator)#

Get the actual environment to use for an input file generator.

Parameters:: input_file_generator (InputFileGenerator) –

get_input_file_labels()#: Get the labels from the input files.

get_input_types(sub_parameters=False)#

Get the input types that are consumed by commands and input file generators of this action.

Parameters:: sub_parameters (bool) – If True, sub-parameters (i.e. dot-delimited parameter types) in command line inputs will be returned untouched. If False (default), only return the root parameter type and disregard the sub-parameter part.
Return type:: Tuple[str]

get_output_file_labels()#: Get the labels from the output files.

get_output_file_parser_action_env(output_file_parser)#

Get the actual environment to use for an output file parser.

Parameters:: output_file_parser (OutputFileParser) –

get_output_types()#

Get the output types that are produced by command standard outputs and errors, and by output file parsers of this action.

Return type:: Tuple[str]

get_param_dump_file_path_HDF5(js_idx, js_act_idx)#

Get the path of the HDF56 dump file.

Parameters:

js_idx (int) –
js_act_idx (int) –

get_param_dump_file_path_JSON(js_idx, js_act_idx)#

Get the path of the JSON dump file.

Parameters:

js_idx (int) –
js_act_idx (int) –

get_param_load_file_path_HDF5(js_idx, js_act_idx)#

Get the path of the HDF5 load file.

Parameters:

js_idx (int) –
js_act_idx (int) –

get_param_load_file_path_JSON(js_idx, js_act_idx)#

Get the path of the JSON load file.

Parameters:

js_idx (int) –
js_act_idx (int) –

get_parameter_dependence(parameter)#

Find if/where a given parameter is used by the action.

Parameters:: parameter (SchemaParameter) –

get_parameter_names(prefix)#

Get parameter types associated with a given prefix.

For example, with the prefix “inputs”, this would return [‘p1’, ‘p2’] for an action that has input types p1 and p2. For inputs, labels are ignored. For example, for an action that accepts two inputs of the same type p1, with labels one and two, this method would return (for the “inputs” prefix): [‘p1[one]’, ‘p1[two]’].

This method is distinct from TaskSchema.get_parameter_names in that it returns action-level input/output/file types/labels, whereas TaskSchema.get_parameter_names returns schema-level inputs/outputs.

Parameters:: prefix (str) – One of “inputs”, “outputs”, “input_files”, “output_files”.
Return type:: List[str]

get_possible_scopes()#

Get the action scopes that are inclusive of this action, ordered by decreasing specificity.

Return type:: Tuple[ActionScope]

get_precise_scope()#

Get the exact scope of this action. The action must have been expanded prior to calling this.

Return type:: ActionScope

get_required_executables()#

Return executable labels required by this action.

Return type:: Tuple[str]

classmethod get_script_name(script)#

Return the script name.

Parameters:: script (str) –
Return type:: str

classmethod get_snippet_script_path(script_path, env_spec=None)#

Get the substituted script snippet path, or False if there is no snippet.

Parameters:: env_spec (Dict[str, Any] | None) –
Return type:: Path

classmethod get_snippet_script_str(script, env_spec=None)#

Get the substituted script snippet path as a string.

Parameters:: env_spec (Dict[str, Any] | None) –
Return type:: str

input_file_generators#: Any applicable input file generators.

input_files#: The input files to the action’s commands.

is_input_type_required(typ, provided_files)#

Determine if the given input type is required by this action.

Parameters:

typ (str) –
provided_files (List[FileSpec]) –

Return type:

bool

static is_snippet_script(script)#

Returns True if the provided script string represents a script snippets that is to be modified before execution (e.g. to receive and provide parameter data).

Parameters:: script (str) –
Return type:: bool

output_file_parsers#: Any applicable output file parsers.

output_files#: The output files from the action’s commands.

process_script_data_formats()#: Convert script data information into standard form.

rules#: How to determine whether to run the action.

save_files#: The names of files to be explicitly saved after each step.

script#: The name of the Python script to run.

script_data_files_use_opt#: If True, script data input and output file paths will be passed to the script execution command line with an option like –input-json or –output-hdf5 etc. If False, the file paths will be passed on their own. For Python scripts, options are always passed, and this parameter is overwritten to be True, regardless of its initial value.

script_data_in#: Information about data input to the script.

property script_data_in_grouped: Dict[str, List[str]]#: Get input parameter types by script data-in format.

property script_data_in_has_direct: bool#: Return True if the script requires some inputs to be passed directly from the app.

property script_data_in_has_files: bool#: Return True if the script requires some inputs to be passed via an intermediate file format.

script_data_out#: Information about data output from the script.

property script_data_out_grouped: Dict[str, List[str]]#: Get output parameter types by script data-out format.

property script_data_out_has_direct: bool#: Return True if the script produces some outputs to be passed directly to the app.

property script_data_out_has_files: bool#: Return True if the script produces some outputs via an intermediate file format.

script_exe#: The executable to use to run the script.

property script_is_python: bool#: Return True if the script is a Python script (determined by the file extension)

script_pass_env_spec#: Whether to pass the environment details to the script.

property task_schema#: The task schema that this action came from.

test_rules(element_iter)#

Test all rules against the specified element iteration.

Return type:: Tuple[bool, List[int]]

to_dict()#: Serialize this object as a dictionary.

to_json_like(dct=None, shared_data=None, exclude=None, path=None)#: Serialize this object as an object structure that can be trivially converted to JSON. Note that YAML can also be produced from the result of this method; it just requires a different final serialization step.