matflow.SGEPosix

class matflow.SGEPosix(cwd_switch=None, *args, **kwargs)

Bases: SGEPosix

A scheduler that uses SGE.

Keyword Arguments:

• cwd_switch (str) – Override of default switch to use to set the current working directory.

• shell_args (str) – Arguments to pass to the shell. Pre-quoted.

• shebang_args (str) – Arguments to set on the shebang line. Pre-quoted.

• options (dict) – Options to the jobscript command.

Notes

• runs in serial by default

References

[1] https://gridscheduler.sourceforge.net/htmlman/htmlman1/qsub.html [2] https://softpanorama.org/HPC/Grid_engine/Queues/queue_states.shtml

Methods

cancel_jobs	Cancel submitted jobs.
format_options	Format the options to the jobscript command.
format_switch	Format a particular switch to use the JS command.
get_job_state_info	Query the scheduler to get the states of all of this user's jobs, optionally filtering by specified job IDs.
get_job_statuses	Get information about all of this user's jobscripts that currently listed by the scheduler.
get_login_nodes	Return a list of hostnames of login/administrative nodes as reported by the scheduler.
get_submit_command	Get the command to use to submit a job to the scheduler.
get_version_info	Get the version of the scheduler.
is_jobscript_active	Query if a jobscript is running/pending.
is_num_cores_supported	Test whether particular number of cores is supported in given range of cores.
parse_submission_output	Extract scheduler reference for a newly submitted jobscript
process_resources	Perform scheduler-specific processing to the element resources.
wait_for_jobscripts	Wait for jobscripts to update their state.

Attributes

DEFAULT_ARRAY_ITEM_VAR	Default shell variable with array ID.
DEFAULT_ARRAY_SWITCH	Default prefix to enable array processing.
DEFAULT_CWD_SWITCH	Default switch to control CWD.
DEFAULT_DEL_CMD	Default cancel command.
DEFAULT_JS_CMD	Default job control directive prefix.
DEFAULT_LOGIN_NODES_CMD	Default command to get the login nodes.
DEFAULT_LOGIN_NODE_MATCH	Default pattern for matching the names of login nodes.
DEFAULT_SHEBANG_ARGS	Default args for shebang line.
DEFAULT_SHELL_ARGS	Default value for arguments to the shell.
DEFAULT_SHOW_CMD	Default command to show the queue state.
DEFAULT_SUBMIT_CMD	Default submission command.
app
state_lookup	Maps scheduler state codes to JobscriptElementState values.
unique_properties	Unique properties, for hashing.

DEFAULT_ARRAY_ITEM_VAR = 'SGE_TASK_ID'

Default shell variable with array ID.

DEFAULT_ARRAY_SWITCH = '-t'

Default prefix to enable array processing.

DEFAULT_CWD_SWITCH = '-cwd'

Default switch to control CWD.

DEFAULT_DEL_CMD = 'qdel'

Default cancel command.

DEFAULT_JS_CMD = '#$'

Default job control directive prefix.

DEFAULT_LOGIN_NODES_CMD = ['qconf', '-sh']

Default command to get the login nodes.

DEFAULT_LOGIN_NODE_MATCH = '*login*'

Default pattern for matching the names of login nodes.

DEFAULT_SHEBANG_ARGS = ''

Default args for shebang line.

DEFAULT_SHELL_ARGS = ''

Default value for arguments to the shell.

DEFAULT_SHOW_CMD = ['qstat']

Default command to show the queue state.

DEFAULT_SUBMIT_CMD = 'qsub'

Default submission command.

app = App(name='MatFlow', version='0.3.0a132')

cancel_jobs(js_refs, jobscripts=None)

Cancel submitted jobs.

Parameters:

• js_refs (List[str]) –

• jobscripts (List) –

format_options(resources, num_elements, is_array, sub_idx)

Format the options to the jobscript command.

format_switch(switch)

Format a particular switch to use the JS command.

get_job_state_info(js_refs=None)

Query the scheduler to get the states of all of this user’s jobs, optionally filtering by specified job IDs.

Jobs that are not in the scheduler’s status output will not appear in the output of this method.

Parameters:

js_refs (List[str]) –

Return type:

Dict[str, Dict[int, JobscriptElementState]]

get_job_statuses()

Get information about all of this user’s jobscripts that currently listed by the scheduler.

get_login_nodes()

Return a list of hostnames of login/administrative nodes as reported by the scheduler.

get_submit_command(shell, js_path, deps)

Get the command to use to submit a job to the scheduler.

Return type:

List of argument words.

Parameters:

• shell (Shell) –

• js_path (str) –

• deps (List[Tuple]) –

get_version_info()

Get the version of the scheduler.

is_jobscript_active(job_ID)

Query if a jobscript is running/pending.

Parameters:

job_ID (str) –

static is_num_cores_supported(num_cores, core_range)

Test whether particular number of cores is supported in given range of cores.

Parameters:

core_range (List[int]) –

parse_submission_output(stdout)

Extract scheduler reference for a newly submitted jobscript

Parameters:

stdout (str) –

Return type:

str

classmethod process_resources(resources, scheduler_config)

Perform scheduler-specific processing to the element resources.

Note: this mutates resources.

Parameters:

scheduler_config (Dict) –

Return type:

None

state_lookup = {'Eh': JobscriptElementState.errored, 'Eq': JobscriptElementState.errored, 'RS': JobscriptElementState.errored, 'RT': JobscriptElementState.errored, 'Rr': JobscriptElementState.running, 'Rs': JobscriptElementState.errored, 'Rt': JobscriptElementState.errored, 'S': JobscriptElementState.errored, 'T': JobscriptElementState.errored, 'dR': JobscriptElementState.cancelled, 'dS': JobscriptElementState.cancelled, 'dT': JobscriptElementState.cancelled, 'dr': JobscriptElementState.cancelled, 'ds': JobscriptElementState.cancelled, 'dt': JobscriptElementState.cancelled, 'hR': JobscriptElementState.waiting, 'hq': JobscriptElementState.waiting, 'qw': JobscriptElementState.pending, 'r': JobscriptElementState.running, 's': JobscriptElementState.errored, 't': JobscriptElementState.running, 'tS': JobscriptElementState.errored, 'tT': JobscriptElementState.errored, 'ts': JobscriptElementState.errored}

Maps scheduler state codes to JobscriptElementState values.

property unique_properties

Unique properties, for hashing.

wait_for_jobscripts(js_refs)

Wait for jobscripts to update their state.

Parameters:

js_refs (List[Any]) –

Return type:

None