Scheduling Jobs on Fox

Jobs are scheduled based on their priority. In addition, lower priority jobs can be back-filled if there are idle resources and the jobs can start without delaying higher-priority jobs. Also, some projects have financed compute nodes, and get priority access to their own nodes.

The command pending lists all of pending jobs, including their priority.

The priority of a job depends on what type of job it is, how long it has waited in the queue, and fair share usage of the project (account). In the current setup, job size does not affect priority.

Job Priorities

The total priority of a job is the sum of

• Wait time priority: a job's priority increases 1 point per minute the job is waiting in the queue,^[1] starting at 0 and maxing out at 10080 after 7 days.
• Job type priority: each job runs with a quality of service (QoS), which determins its QoS priority (see below).
• Fairshare priority: A number between 0 and 10000. This gives less priority to projects that have run a lot recently, and more to projects that have run little recently. Details about fairshare in Slurm can be found here.

Currently, these QoS priorities are used:

• devel jobs (qos devel): 30000
• All other jobs (qos normal): 10000

Footnotes

1.

Currently, only the priority of 10 jobs for each user within each project increase with time. As jobs start, more priorities start to increase. This is done in order to avoid problems if a user submits a large amount of jobs over a short time. Note that the limit is per user and project, so if a user has jobs in several projects, 10 of the user's jobs from each project will increase in priority at the same time. This limit might change in the future.

^{CC Attribution: This page is maintained by the University of Oslo IT FFU-BT group. It has either been modified from, or is a derivative of, "Scheduling" by NRIS under CC-BY-4.0. Changes: Significant rephrasing and excision of texts.}