Command-line interface#

CLI reference documentation.

matflow#

matflow [OPTIONS] COMMAND [ARGS]...

Options

--version#: Show the version of MatFlow and exit.

--hpcflow-version#: Show the version of hpcflow and exit.

--help#: Show this message and exit.

--run-time-info#: Print run-time information!

--config-dir <config_dir>#: Set the configuration directory.

--config-key <config_key>#: Set the configuration invocation key.

--with-config <with_config>#: Override a config item in the config file

--timeit#: Time function pathways as the code executes and write out a summary at the end. Only functions decorated by TimeIt.decorator are included.

--timeit-file <timeit_file>#: Time function pathways as the code executes and write out a summary at the end to a text file given by this file path. Only functions decorated by TimeIt.decorator are included.

cancel#

Stop all running jobscripts of the specified workflow.

WORKFLOW_REF is the local ID (that provided by the show command}) or the workflow path.

matflow cancel [OPTIONS] WORKFLOW_REF

Options

-r, --ref-type <ref_type>#

How to interpret a reference, as an ID, a path, or to guess.

Options:: assume-id | id | path

Arguments

WORKFLOW_REF#: Required argument

config#

Configuration sub-command for getting and setting data in the configuration file(s).

matflow config [OPTIONS] COMMAND [ARGS]...

Options

--no-callback <no_callback>#: Exclude a named get/set callback function during execution of the command.

add-scheduler#

matflow config add-scheduler [OPTIONS] NAME

Options

--defaults <defaults>#

Arguments

NAME#: Required argument

add-shell#

matflow config add-shell [OPTIONS] NAME

Options

--defaults <defaults>#

Arguments

NAME#: Required argument

add-shell-wsl#

matflow config add-shell-wsl [OPTIONS]

Options

--defaults <defaults>#

append#

Append a new value to the specified configuration item.

NAME is the dot-delimited path to the list to be appended to.

matflow config append [OPTIONS] NAME VALUE

Options

--json#: Interpret VALUE as a JSON string.

Arguments

NAME#: Required argument

VALUE#: Required argument

get#

Show the value of the specified configuration item.

matflow config get [OPTIONS] NAME

Options

--all#: Show all configuration items.

--metadata#: Show all metadata items.

--file#: Show the contents of the configuration file.

Arguments

NAME#: Required argument

import#

Update the config file with keys from a YAML file.

matflow config import [OPTIONS] FILE_PATH

Options

--rename, --no-rename#: Rename the currently loaded config file according to the name of the file that is being imported (default is to rename). Ignored if –new is specified.

--new#: If True, generate a new default config, and import the file into this config. If False, modify the currently loaded config.

Arguments

FILE_PATH#: Required argument

init#

matflow config init [OPTIONS] KNOWN_NAME

Options

--path <path>#: An fsspec-compatible path in which to look for configuration-import files.

Arguments

KNOWN_NAME#: Required argument

list#

Show a list of all configurable keys.

matflow config list [OPTIONS]

load-data-files#

Check we can load the data files (e.g. task schema files) as specified in the configuration.

matflow config load-data-files [OPTIONS]

open#

Alias for matflow open config: open the configuration file, or retrieve it’s path.

matflow config open [OPTIONS]

Options

--path#

pop#

Remove a value from a list-like configuration item.

NAME is the dot-delimited path to the list to be modified.

matflow config pop [OPTIONS] NAME INDEX

Arguments

NAME#: Required argument

INDEX#: Required argument

prepend#

Prepend a new value to the specified configuration item.

NAME is the dot-delimited path to the list to be prepended to.

matflow config prepend [OPTIONS] NAME VALUE

Options

--json#: Interpret VALUE as a JSON string.

Arguments

NAME#: Required argument

VALUE#: Required argument

set#

Set and save the value of the specified configuration item.

matflow config set [OPTIONS] NAME VALUE

Options

--json#: Interpret VALUE as a JSON string.

Arguments

NAME#: Required argument

VALUE#: Required argument

set-github-demo-data-dir#

matflow config set-github-demo-data-dir [OPTIONS] SHA

Arguments

SHA#: Required argument

unset#

Unset and save the value of the specified configuration item.

matflow config unset [OPTIONS] NAME

Arguments

NAME#: Required argument

update#

Update a map-like value in the configuration.

NAME is the dot-delimited path to the map to be updated.

matflow config update [OPTIONS] NAME VALUE

Options

--json#: Interpret VALUE as a JSON string.

Arguments

NAME#: Required argument

VALUE#: Required argument

configure-env#

Configure an app environment, using, for example, the currently activated Python environment.

matflow configure-env [OPTIONS] NAME

Options

--use-current-env#

--setup <setup>#

--env-source-file <env_source_file>#

Arguments

NAME#: Required argument

demo-data#

Interact with builtin demo data files.

matflow demo-data [OPTIONS] COMMAND [ARGS]...

Options

-l, --list#: Print available example data files.

cache#

Ensure a demo data file is in the demo data cache.

matflow demo-data cache [OPTIONS] FILE_NAME

Options

--all#: Cache all demo data files.

Arguments

FILE_NAME#: Required argument

copy#

Copy a demo data file to the specified location.

matflow demo-data copy [OPTIONS] FILE_NAME DESTINATION

Arguments

FILE_NAME#: Required argument

DESTINATION#: Required argument

demo-software#

matflow demo-software [OPTIONS] COMMAND [ARGS]...

doSomething#

matflow demo-software doSomething [OPTIONS]

Options

-i1, --infile1 <infile1>#: Required

-i2, --infile2 <infile2>#: Required

-v, --value <value>#

-o, --out <out>#

demo-workflow#

Interact with builtin demo workflows.

matflow demo-workflow [OPTIONS] COMMAND [ARGS]...

Options

-l, --list#: Print available builtin demo workflows.

copy#

matflow demo-workflow copy [OPTIONS] WORKFLOW_NAME DESTINATION

Options

--doc, --no-doc#

Arguments

WORKFLOW_NAME#: Required argument

DESTINATION#: Required argument

go#

matflow demo-workflow go [OPTIONS] WORKFLOW_NAME

Options

--format <format>#

If specified, one of “json” or “yaml”. This forces parsing from a particular format.

Options:: yaml | json

--path <path>#: The directory path into which the new workflow will be generated.

--name <name>#: The name of the workflow. If specified, the workflow directory will be path joined with name. If not specified the workflow template name will be used, in combination with a date-timestamp.

--overwrite#: If True and the workflow directory (path + name) already exists, the existing directory will be overwritten.

--store <store>#

The persistent store type to use.

Options:: zarr | zip | json

--ts-fmt <ts_fmt>#: The datetime format to use for storing datetimes. Datetimes are always stored in UTC (because Numpy does not store time zone info), so this should not include a time zone name.

--ts-name-fmt <ts_name_fmt>#: The datetime format to use when generating the workflow name, where it includes a timestamp.

-v, --var <variables>#: Workflow template variable value to be substituted in to the template file or string. Multiple variable values can be specified.

--js-parallelism <js_parallelism>#: If True, allow multiple jobscripts to execute simultaneously. Raises if set to True but the store type does not support the jobscript_parallelism feature. If not set, jobscript parallelism will be used if the store type supports it.

--wait#: If True, this command will block until the workflow execution is complete.

--add-to-known, --no-add-to-known#: If True, add this submission to the known-submissions file.

--print-idx#: If True, print the submitted jobscript indices for each submission index.

--tasks <tasks>#: List of comma-separated task indices to include in this submission. By default all tasks are included.

--cancel#: Immediately cancel the submission. Useful for testing and benchmarking.

--status, --no-status#: If True, display a live status to track submission progress.

Arguments

WORKFLOW_NAME#: Required argument

make#

matflow demo-workflow make [OPTIONS] WORKFLOW_NAME

Options

--format <format>#

If specified, one of “json” or “yaml”. This forces parsing from a particular format.

Options:: yaml | json

--path <path>#: The directory path into which the new workflow will be generated.

--name <name>#: The name of the workflow. If specified, the workflow directory will be path joined with name. If not specified the workflow template name will be used, in combination with a date-timestamp.

--overwrite#: If True and the workflow directory (path + name) already exists, the existing directory will be overwritten.

--store <store>#

The persistent store type to use.

Options:: zarr | zip | json

--ts-fmt <ts_fmt>#: The datetime format to use for storing datetimes. Datetimes are always stored in UTC (because Numpy does not store time zone info), so this should not include a time zone name.

--ts-name-fmt <ts_name_fmt>#: The datetime format to use when generating the workflow name, where it includes a timestamp.

-v, --var <variables>#: Workflow template variable value to be substituted in to the template file or string. Multiple variable values can be specified.

--status, --no-status#: If True, display a live status to track workflow creation progress.

Arguments

WORKFLOW_NAME#: Required argument

show#

matflow demo-workflow show [OPTIONS] WORKFLOW_NAME

Options

--syntax, --no-syntax#

--doc, --no-doc#

Arguments

WORKFLOW_NAME#: Required argument

go#

Generate and submit a new MatFlow workflow.

TEMPLATE_FILE_OR_STR is either a path to a template file in YAML or JSON format, or a YAML/JSON string.

matflow go [OPTIONS] TEMPLATE_FILE_OR_STR

Options

--string#: Determines if passing a file path or a string.

--format <format>#

If specified, one of “json” or “yaml”. This forces parsing from a particular format.

Options:: yaml | json

--path <path>#: The directory path into which the new workflow will be generated.

--name <name>#: The name of the workflow. If specified, the workflow directory will be path joined with name. If not specified the workflow template name will be used, in combination with a date-timestamp.

--overwrite#: If True and the workflow directory (path + name) already exists, the existing directory will be overwritten.

--store <store>#

The persistent store type to use.

Options:: zarr | zip | json

--ts-fmt <ts_fmt>#: The datetime format to use for storing datetimes. Datetimes are always stored in UTC (because Numpy does not store time zone info), so this should not include a time zone name.

--ts-name-fmt <ts_name_fmt>#: The datetime format to use when generating the workflow name, where it includes a timestamp.

-v, --var <variables>#: Workflow template variable value to be substituted in to the template file or string. Multiple variable values can be specified.

--js-parallelism <js_parallelism>#: If True, allow multiple jobscripts to execute simultaneously. Raises if set to True but the store type does not support the jobscript_parallelism feature. If not set, jobscript parallelism will be used if the store type supports it.

--wait#: If True, this command will block until the workflow execution is complete.

--add-to-known, --no-add-to-known#: If True, add this submission to the known-submissions file.

--print-idx#: If True, print the submitted jobscript indices for each submission index.

--tasks <tasks>#: List of comma-separated task indices to include in this submission. By default all tasks are included.

--cancel#: Immediately cancel the submission. Useful for testing and benchmarking.

--status, --no-status#: If True, display a live status to track submission progress.

Arguments

TEMPLATE_FILE_OR_STR#: Required argument

helper#

matflow helper [OPTIONS] COMMAND [ARGS]...

clear#

Remove the PID file (and kill the helper process if it exists). This should not normally be needed.

matflow helper clear [OPTIONS]

log-path#

Get the path to the helper log file (may not exist).

matflow helper log-path [OPTIONS]

pid#

Get the process ID of the running helper, if running.

matflow helper pid [OPTIONS]

Options

-f, --file#

restart#

Restart (or start) the helper process.

matflow helper restart [OPTIONS]

Options

--timeout <timeout>#

Helper timeout in seconds.

Default:: 3600

--timeout-check-interval <timeout_check_interval>#

Interval between testing if the timeout has been exceeded in seconds.

Default:: 60

--watch-interval <watch_interval>#

Polling interval for watching workflows (and the workflow watch list) in seconds.

Default:: 10

run#

Run the helper functionality.

matflow helper run [OPTIONS]

Options

--timeout <timeout>#

Helper timeout in seconds.

Default:: 3600

--timeout-check-interval <timeout_check_interval>#

Interval between testing if the timeout has been exceeded in seconds.

Default:: 60

--watch-interval <watch_interval>#

Polling interval for watching workflows (and the workflow watch list) in seconds.

Default:: 10

start#

Start the helper process.

matflow helper start [OPTIONS]

Options

--timeout <timeout>#

Helper timeout in seconds.

Default:: 3600

--timeout-check-interval <timeout_check_interval>#

Interval between testing if the timeout has been exceeded in seconds.

Default:: 60

--watch-interval <watch_interval>#

Polling interval for watching workflows (and the workflow watch list) in seconds.

Default:: 10

stop#

Stop the helper process, if it is running.

matflow helper stop [OPTIONS]

uptime#

Get the uptime of the helper process, if it is running.

matflow helper uptime [OPTIONS]

watch-list#

Get the list of workflows currently being watched.

matflow helper watch-list [OPTIONS]

watch-list-path#

Get the path to the workflow watch list file (may not exist).

matflow helper watch-list-path [OPTIONS]

internal#

Internal CLI to be invoked by scripts generated by the app.

matflow internal [OPTIONS] COMMAND [ARGS]...

get-invoc-cmd#

Get the invocation command for this app instance.

matflow internal get-invoc-cmd [OPTIONS]

workflow#

matflow internal workflow [OPTIONS] PATH COMMAND [ARGS]...

Arguments

PATH#: Required argument

check-loop#

Check if an iteration has met its loop’s termination condition.

matflow internal workflow PATH check-loop [OPTIONS] LOOP_NAME EAR_ID

Arguments

LOOP_NAME#: Required argument

EAR_ID#: Required argument

get-ear-skipped#

Return 1 if the given EAR is to be skipped, else return 0.

matflow internal workflow PATH get-ear-skipped [OPTIONS] EAR_ID

Arguments

EAR_ID#: Required argument

save-parameter#

matflow internal workflow PATH save-parameter [OPTIONS] NAME VALUE EAR_ID
                                              CMD_IDX

Options

--stderr#

Arguments

NAME#: Required argument

VALUE#: Required argument

EAR_ID#: Required argument

CMD_IDX#: Required argument

set-ear-end#

matflow internal workflow PATH set-ear-end [OPTIONS] JS_IDX JS_ACT_IDX EAR_ID
                                           EXIT_CODE

Arguments

JS_IDX#: Required argument

JS_ACT_IDX#: Required argument

EAR_ID#: Required argument

EXIT_CODE#: Required argument

set-ear-skip#

matflow internal workflow PATH set-ear-skip [OPTIONS] EAR_ID

Arguments

EAR_ID#: Required argument

set-ear-start#

matflow internal workflow PATH set-ear-start [OPTIONS] EAR_ID

Arguments

EAR_ID#: Required argument

write-commands#

matflow internal workflow PATH write-commands [OPTIONS] SUBMISSION_IDX
                                              JOBSCRIPT_IDX JS_ACTION_IDX
                                              EAR_ID

Arguments

SUBMISSION_IDX#: Required argument

JOBSCRIPT_IDX#: Required argument

JS_ACTION_IDX#: Required argument

EAR_ID#: Required argument

make#

Generate a new MatFlow workflow.

TEMPLATE_FILE_OR_STR is either a path to a template file in YAML or JSON format, or a YAML/JSON string.

matflow make [OPTIONS] TEMPLATE_FILE_OR_STR

Options

--string#: Determines if passing a file path or a string.

--format <format>#

If specified, one of “json” or “yaml”. This forces parsing from a particular format.

Options:: yaml | json

--path <path>#: The directory path into which the new workflow will be generated.

--name <name>#: The name of the workflow. If specified, the workflow directory will be path joined with name. If not specified the workflow template name will be used, in combination with a date-timestamp.

--overwrite#: If True and the workflow directory (path + name) already exists, the existing directory will be overwritten.

--store <store>#

The persistent store type to use.

Options:: zarr | zip | json

--ts-fmt <ts_fmt>#: The datetime format to use for storing datetimes. Datetimes are always stored in UTC (because Numpy does not store time zone info), so this should not include a time zone name.

--ts-name-fmt <ts_name_fmt>#: The datetime format to use when generating the workflow name, where it includes a timestamp.

-v, --var <variables>#: Workflow template variable value to be substituted in to the template file or string. Multiple variable values can be specified.

--status, --no-status#: If True, display a live status to track workflow creation progress.

Arguments

TEMPLATE_FILE_OR_STR#: Required argument

manage#

Infrequent app management tasks.

App config is not loaded.

matflow manage [OPTIONS] COMMAND [ARGS]...

clear-cache#

Delete the app cache directory.

matflow manage clear-cache [OPTIONS]

Options

--hostname#

clear-demo-data-cache#

Delete the app demo data cache directory.

matflow manage clear-demo-data-cache [OPTIONS]

clear-known-subs#

Delete the contents of the known-submissions file.

matflow manage clear-known-subs [OPTIONS]

clear-temp-dir#

Delete all files in the user runtime directory.

matflow manage clear-temp-dir [OPTIONS]

get-config-path#

Print the config file path without loading the config.

This can be used instead of {app_name} open config –path if the config file is invalid, because this command does not load the config.

matflow manage get-config-path [OPTIONS]

Options

--config-dir <config_dir>#: The directory containing the config file whose path is to be returned.

reset-config#

Reset the configuration file to defaults.

This can be used if the current configuration file is invalid.

matflow manage reset-config [OPTIONS]

Options

--config-dir <config_dir>#: The directory containing the config file to be reset.

open#

Open a file (for example MatFlow’s log file) using the default application.

matflow open [OPTIONS] COMMAND [ARGS]...

config#

Open the MatFlow config file, or retrieve it’s path.

matflow open config [OPTIONS]

Options

--path#

demo-data-cache-dir#

matflow open demo-data-cache-dir [OPTIONS]

Options

--path#

env-source#

Open a named environment sources file, or the first one.

matflow open env-source [OPTIONS]

Options

--name <name>#

--path#

known-subs#

Open the known-submissions text file.

matflow open known-subs [OPTIONS]

Options

--path#

log#

Open the MatFlow log file.

matflow open log [OPTIONS]

Options

--path#

user-cache-dir#

matflow open user-cache-dir [OPTIONS]

Options

--path#

user-cache-hostname-dir#

matflow open user-cache-hostname-dir [OPTIONS]

Options

--path#

user-data-dir#

matflow open user-data-dir [OPTIONS]

Options

--path#

user-data-hostname-dir#

matflow open user-data-hostname-dir [OPTIONS]

Options

--path#

user-runtime-dir#

matflow open user-runtime-dir [OPTIONS]

Options

--path#

workflow#

Open a workflow directory using, for example, File Explorer on Windows.

matflow open workflow [OPTIONS] WORKFLOW_REF

Options

--path#

-r, --ref-type <ref_type>#

How to interpret a reference, as an ID, a path, or to guess.

Options:: assume-id | id | path

Arguments

WORKFLOW_REF#: Required argument

rechunk#

Rechunk metadata/runs and parameters/base arrays.

WORKFLOW_REF is the local ID (that provided by the show command}) or the workflow path.

matflow rechunk [OPTIONS] WORKFLOW_REF

Options

-r, --ref-type <ref_type>#

How to interpret a reference, as an ID, a path, or to guess.

Options:: assume-id | id | path

--backup, --no-backup#: First copy a backup of the array to a directory ending in .bak.

--chunk-size <chunk_size>#: New chunk size (array items per chunk). If unset (as by default), the array will be rechunked to a single chunk array (i.e with a chunk size equal to the array’s shape).

--status, --no-status#: If True, display a live status to track rechunking progress.

Arguments

WORKFLOW_REF#: Required argument

show#

Show information about running and recently active workflows.

matflow show [OPTIONS]

Options

-r, --max-recent <max_recent>#: The maximum number of inactive submissions to show.

--no-update#: If True, do not update the known-submissions file to remove workflows that are no longer running.

-f, --full#: Allow multiple lines per workflow submission.

--legend#: Display the legend for the show command output.

submission#

Submission-related queries.

matflow submission [OPTIONS] COMMAND [ARGS]...

Options

--os-info#: Print information about the operating system.

get-known#

Print known-submissions information as a formatted Python object.

matflow submission get-known [OPTIONS]

Options

--json#: Do not format and only show JSON-compatible information.

scheduler#

matflow submission scheduler [OPTIONS] SCHEDULER_NAME COMMAND [ARGS]...

Arguments

SCHEDULER_NAME#: Required argument

shell-info#

Show information about the specified shell, such as the version.

matflow submission shell-info [OPTIONS] {bash|powershell|wsl+bash|wsl}

Options

--exclude-os#

Arguments

SHELL_NAME#: Required argument

tc#

For showing template component data.

matflow tc [OPTIONS]

test#

Run MatFlow test suite.

PY_TEST_ARGS are arguments passed on to Pytest.

matflow test [OPTIONS] [PY_TEST_ARGS]...

Arguments

PY_TEST_ARGS#: Optional argument(s)

test-hpcflow#

Run hpcFlow test suite.

PY_TEST_ARGS are arguments passed on to Pytest.

matflow test-hpcflow [OPTIONS] [PY_TEST_ARGS]...

Arguments

PY_TEST_ARGS#: Optional argument(s)

unzip#

Generate a copy of the specified zipped workflow in the submittable Zarr format in the current working directory.

WORKFLOW_PATH is path of the zip file to unzip.

matflow unzip [OPTIONS] WORKFLOW_PATH

Options

--path <path>#: Path at which to create the new unzipped workflow. If this is an existing directory, the new workflow directory will be created within this directory. Otherwise, this path will represent the new workflow directory path.

--log <log>#: Path to a log file to use during unzipping.

Arguments

WORKFLOW_PATH#: Required argument

workflow#

Interact with existing MatFlow workflows.

WORKFLOW_REF is the path to, or local ID of, an existing workflow.

matflow workflow [OPTIONS] WORKFLOW_REF COMMAND [ARGS]...

Options

-r, --ref-type <ref_type>#

How to interpret a reference, as an ID, a path, or to guess.

Options:: assume-id | id | path

Arguments

WORKFLOW_REF#: Required argument

abort-run#

Abort the specified run.

matflow workflow WORKFLOW_REF abort-run [OPTIONS]

Options

--submission <submission>#

--task <task>#

--element <element>#

get-all-params#

Get all parameter values.

matflow workflow WORKFLOW_REF get-all-params [OPTIONS]

get-param#

Get a parameter value by data index.

matflow workflow WORKFLOW_REF get-param [OPTIONS] INDEX

Arguments

INDEX#: Required argument

get-param-source#

Get a parameter source by data index.

matflow workflow WORKFLOW_REF get-param-source [OPTIONS] INDEX

Arguments

INDEX#: Required argument

is-param-set#

Check if a parameter specified by data index is set.

matflow workflow WORKFLOW_REF is-param-set [OPTIONS] INDEX

Arguments

INDEX#: Required argument

rechunk#

Rechunk metadata/runs and parameters/base arrays.

matflow workflow WORKFLOW_REF rechunk [OPTIONS]

Options

--backup, --no-backup#: First copy a backup of the array to a directory ending in .bak.

--chunk-size <chunk_size>#: New chunk size (array items per chunk). If unset (as by default), the array will be rechunked to a single chunk array (i.e with a chunk size equal to the array’s shape).

--status, --no-status#: If True, display a live status to track rechunking progress.

rechunk-parameter-base#

Rechunk the parameters/base array.

matflow workflow WORKFLOW_REF rechunk-parameter-base [OPTIONS]

Options

--backup, --no-backup#: First copy a backup of the array to a directory ending in .bak.

--chunk-size <chunk_size>#: New chunk size (array items per chunk). If unset (as by default), the array will be rechunked to a single chunk array (i.e with a chunk size equal to the array’s shape).

--status, --no-status#: If True, display a live status to track rechunking progress.

rechunk-runs#

Rechunk the metadata/runs array.

matflow workflow WORKFLOW_REF rechunk-runs [OPTIONS]

Options

--backup, --no-backup#: First copy a backup of the array to a directory ending in .bak.

--chunk-size <chunk_size>#: New chunk size (array items per chunk). If unset (as by default), the array will be rechunked to a single chunk array (i.e with a chunk size equal to the array’s shape).

--status, --no-status#: If True, display a live status to track rechunking progress.

show-all-status#

Show the submission status of all workflow EARs.

matflow workflow WORKFLOW_REF show-all-status [OPTIONS]

sub#

Interact with existing MatFlow workflow submissions.

SUB_IDX is the submission index.

matflow workflow WORKFLOW_REF sub [OPTIONS] SUB_IDX COMMAND [ARGS]...

Arguments

SUB_IDX#: Required argument

get-active-jobscripts#

Show active jobscripts and their jobscript-element states.

matflow workflow WORKFLOW_REF sub SUB_IDX get-active-jobscripts 
    [OPTIONS]

js#

Interact with existing MatFlow workflow submission jobscripts.

JS_IDX is the jobscript index within the submission object.

matflow workflow WORKFLOW_REF sub SUB_IDX js [OPTIONS] JS_IDX COMMAND
                                             [ARGS]...

Arguments

JS_IDX#: Required argument

deps#

Get jobscript dependencies.

matflow workflow WORKFLOW_REF sub SUB_IDX js JS_IDX deps [OPTIONS]

path#

Get the file path to the jobscript.

matflow workflow WORKFLOW_REF sub SUB_IDX js JS_IDX path [OPTIONS]

res#

Get resources associated with this jobscript.

matflow workflow WORKFLOW_REF sub SUB_IDX js JS_IDX res [OPTIONS]

show#

Show the jobscript file.

matflow workflow WORKFLOW_REF sub SUB_IDX js JS_IDX show [OPTIONS]

needs-submit#

Check if this submission needs submitting.

matflow workflow WORKFLOW_REF sub SUB_IDX needs-submit [OPTIONS]

outstanding-js#

Get a list of jobscript indices that have not yet been submitted.

matflow workflow WORKFLOW_REF sub SUB_IDX outstanding-js [OPTIONS]

status#

Get the submission status.

matflow workflow WORKFLOW_REF sub SUB_IDX status [OPTIONS]

submitted-js#

Get a list of jobscript indices that have been submitted.

matflow workflow WORKFLOW_REF sub SUB_IDX submitted-js [OPTIONS]

submit#

Submit the workflow.

matflow workflow WORKFLOW_REF submit [OPTIONS]

Options

--js-parallelism <js_parallelism>#: If True, allow multiple jobscripts to execute simultaneously. Raises if set to True but the store type does not support the jobscript_parallelism feature. If not set, jobscript parallelism will be used if the store type supports it.

--wait#: If True, this command will block until the workflow execution is complete.

--add-to-known, --no-add-to-known#: If True, add this submission to the known-submissions file.

--print-idx#: If True, print the submitted jobscript indices for each submission index.

--tasks <tasks>#: List of comma-separated task indices to include in this submission. By default all tasks are included.

--cancel#: Immediately cancel the submission. Useful for testing and benchmarking.

--status, --no-status#: If True, display a live status to track submission progress.

unzip#

Generate a copy of the zipped workflow in the submittable Zarr format in the current working directory.

matflow workflow WORKFLOW_REF unzip [OPTIONS]

Options

--path <path>#: Path at which to create the new unzipped workflow. If this is an existing directory, the new workflow directory will be created within this directory. Otherwise, this path will represent the new workflow directory path.

--log <log>#: Path to a log file to use during unzipping.

wait#

matflow workflow WORKFLOW_REF wait [OPTIONS]

Options

-j, --jobscripts <jobscripts>#: Wait for only these jobscripts to finish. Jobscripts should be specified by their submission index, followed by a colon, followed by a comma-separated list of jobscript indices within that submission (no spaces are allowed). To specify jobscripts across multiple submissions, use a semicolon to separate patterns like these.

zip#

Generate a copy of the workflow in the zip file format in the current working directory.

matflow workflow WORKFLOW_REF zip [OPTIONS]

Options

--path <path>#: Path at which to create the new zipped workflow. If this is an existing directory, the zip file will be created within this directory. Otherwise, this path is assumed to be the full file path to the new zip file.

--overwrite#: If set, any existing file will be overwritten.

--log <log>#: Path to a log file to use during zipping.

--include-execute#

--include-rechunk-backups#

zip#

Generate a copy of the specified workflow in the zip file format in the current working directory.

WORKFLOW_REF is the local ID (that provided by the show command}) or the workflow path.

matflow zip [OPTIONS] WORKFLOW_REF

Options

--path <path>#: Path at which to create the new zipped workflow. If this is an existing directory, the zip file will be created within this directory. Otherwise, this path is assumed to be the full file path to the new zip file.

--overwrite#: If set, any existing file will be overwritten.

--log <log>#: Path to a log file to use during zipping.

--include-execute#

--include-rechunk-backups#

-r, --ref-type <ref_type>#

How to interpret a reference, as an ID, a path, or to guess.

Options:: assume-id | id | path

Arguments

WORKFLOW_REF#: Required argument

Command-line interface#

matflow#

cancel#

config#

add-scheduler#

add-shell#

add-shell-wsl#

append#

get#

import#

init#

list#

load-data-files#

open#

pop#

prepend#

set#

set-github-demo-data-dir#

unset#

update#

configure-env#

demo-data#

cache#

copy#

demo-software#

doSomething#

demo-workflow#

copy#

go#

make#

show#

go#

helper#

clear#

log-path#

pid#

restart#

run#

start#

stop#

uptime#

watch-list#

watch-list-path#

internal#

get-invoc-cmd#

workflow#

check-loop#

get-ear-skipped#

save-parameter#

set-ear-end#

set-ear-skip#

set-ear-start#

write-commands#

make#

manage#

clear-cache#

clear-demo-data-cache#

clear-known-subs#

clear-temp-dir#

get-config-path#

reset-config#

open#

config#

demo-data-cache-dir#

env-source#

known-subs#

log#

user-cache-dir#

user-cache-hostname-dir#

user-data-dir#

user-data-hostname-dir#

user-runtime-dir#

workflow#

rechunk#

show#

submission#

get-known#

scheduler#

get-login-nodes#

shell-info#