matflow.SlurmPosix#

class matflow.SlurmPosix(*args, **kwargs)#

Bases: SlurmPosix

Notes

runs in current working directory by default [2]

# TODO: consider getting memory usage like: https://stackoverflow.com/a/44143229/5042280

References

[1] https://manpages.org/sbatch [2] https://ri.itservices.manchester.ac.uk/csf4/batch/sge-to-slurm/

Methods

`cancel_jobs`
`format_array_request`
`format_core_request_lines`
`format_options`
`format_std_stream_file_option_lines`
`format_switch`
`get_job_state_info`	Query the scheduler to get the states of all of this user's jobs, optionally filtering by specified job IDs.
`get_submit_command`
`get_version_info`
`is_jobscript_active`	Query if a jobscript is running/pending.
`is_num_cores_supported`
`parse_submission_output`	Extract scheduler reference for a newly submitted jobscript
`process_resources`	Perform scheduler-specific processing to the element resources.
`wait_for_jobscripts`

Attributes

`DEFAULT_ARRAY_ITEM_VAR`
`DEFAULT_ARRAY_SWITCH`
`DEFAULT_DEL_CMD`
`DEFAULT_JS_CMD`
`DEFAULT_LOGIN_NODES_CMD`
`DEFAULT_LOGIN_NODE_MATCH`
`DEFAULT_SHEBANG_ARGS`
`DEFAULT_SHELL_ARGS`
`DEFAULT_SHELL_EXECUTABLE`
`DEFAULT_SHOW_CMD`
`DEFAULT_SUBMIT_CMD`
`app`
`state_lookup`
`unique_properties`

DEFAULT_ARRAY_ITEM_VAR = 'SLURM_ARRAY_TASK_ID'#

DEFAULT_ARRAY_SWITCH = '--array'#

DEFAULT_DEL_CMD = 'scancel'#

DEFAULT_JS_CMD = '#SBATCH'#

DEFAULT_LOGIN_NODES_CMD = None#

DEFAULT_LOGIN_NODE_MATCH = '*login*'#

DEFAULT_SHEBANG_ARGS = ''#

DEFAULT_SHELL_ARGS = ''#

DEFAULT_SHELL_EXECUTABLE = '/bin/bash'#

DEFAULT_SHOW_CMD = ['squeue', '--me']#

DEFAULT_SUBMIT_CMD = 'sbatch'#

app = App(name='MatFlow', version='0.3.0a129')#

cancel_jobs(js_refs, jobscripts=None)#

Parameters:

js_refs (List[str]) –
jobscripts (List) –

format_array_request(num_elements, resources)#

format_core_request_lines(resources)#

format_options(resources, num_elements, is_array, sub_idx)#

format_std_stream_file_option_lines(is_array, sub_idx)#

format_switch(switch)#

get_job_state_info(js_refs=None)#

Query the scheduler to get the states of all of this user’s jobs, optionally filtering by specified job IDs.

Jobs that are not in the scheduler’s status output will not appear in the output of this method.

Parameters:: js_refs (List[str]) –
Return type:: Dict[str, Dict[int, JobscriptElementState]]

get_submit_command(shell, js_path, deps)#

Parameters:

shell (Shell) –
js_path (str) –
deps (List[Tuple]) –

Return type:

List[str]

get_version_info()#

is_jobscript_active(job_ID)#

Query if a jobscript is running/pending.

Parameters:: job_ID (str) –

static is_num_cores_supported(num_cores, core_range)#

Parameters:: core_range (List[int]) –

parse_submission_output(stdout)#

Extract scheduler reference for a newly submitted jobscript

Parameters:: stdout (str) –
Return type:: str

classmethod process_resources(resources, scheduler_config)#

Perform scheduler-specific processing to the element resources.

Note: this mutates resources.

Parameters:: scheduler_config (Dict) –
Return type:: None

state_lookup = {'CANCELLED': JobscriptElementState.cancelled, 'COMPLETED': JobscriptElementState.finished, 'COMPLETING': JobscriptElementState.running, 'FAILED': JobscriptElementState.errored, 'OUT_OF_MEMORY': JobscriptElementState.errored, 'PENDING': JobscriptElementState.pending, 'RUNNING': JobscriptElementState.running, 'TIMEOUT': JobscriptElementState.errored}#

property unique_properties#

wait_for_jobscripts(js_refs)#

Parameters:: js_refs (List[Any]) –
Return type:: None