matflow.make_and_submit_workflow#

matflow.make_and_submit_workflow(template_file_or_str, is_string=False, template_format=None, path=None, name=None, overwrite=False, store='zarr', ts_fmt=None, ts_name_fmt=None, store_kwargs=None, variables=None, JS_parallelism=None, wait=False, add_to_known=True, return_idx=False, tasks=None, cancel=False, status=True)#

Generate and submit a new MatFlow workflow from a file or string containing a workflow template parametrisation.

Parameters:

  • template_path_or_str – Either a path to a template file in YAML or JSON format, or a YAML/JSON string.

  • is_string (bool | None) – Determines whether template_path_or_str is a string or a file.

  • template_format (str | None) – If specified, one of “json” or “yaml”. This forces parsing from a particular format.

  • path (PathLike | None) – The directory in which the workflow will be generated. The current directory if not specified.

  • name (str | None) – The name of the workflow. If specified, the workflow directory will be path joined with name. If not specified the WorkflowTemplate name will be used, in combination with a date-timestamp.

  • overwrite (bool | None) – If True and the workflow directory (path + name) already exists, the existing directory will be overwritten.

  • store (str | None) – The persistent store to use for this workflow.

  • ts_fmt (str | None) – The datetime format to use for storing datetimes. Datetimes are always stored in UTC (because Numpy does not store time zone info), so this should not include a time zone name.

  • ts_name_fmt (str | None) – The datetime format to use when generating the workflow name, where it includes a timestamp.

  • store_kwargs (Dict | None) – Keyword arguments to pass to the store’s write_empty_workflow method.

  • variables (Dict[str, str] | None) – String variables to substitute in template_file_or_str.

  • JS_parallelism (bool | None) – If True, allow multiple jobscripts to execute simultaneously. Raises if set to True but the store type does not support the jobscript_parallelism feature. If not set, jobscript parallelism will be used if the store type supports it.

  • wait (bool | None) – If True, this command will block until the workflow execution is complete.

  • add_to_known (bool | None) – If True, add the new submission to the known-submissions file, which is used by the show command to monitor current and recent submissions.

  • return_idx (bool | None) – If True, return a dict representing the jobscript indices submitted for each submission.

  • tasks (List[int] | None) – List of task indices to include in this submission. By default all tasks are included.

  • cancel (bool | None) – Immediately cancel the submission. Useful for testing and benchmarking.

  • status (bool | None) – If True, display a live status to track workflow creation and submission progress.

  • template_file_or_str (PathLike | str) –

Return type:

Dict[int, int]