Run job with WorkloadPriority

Usually, in Kueue, workload’s priority is calculated using for pod’s priority for queuing and preemption.
By using a WorkloadPriorityClass, you can independently manage the priority of workloads for queuing and preemption, separate from pod’s priority.

This page contains instructions on how to run a job with workload priority.

Before you begin

Make sure the following conditions are met:

• A Kubernetes cluster is running.
• The kubectl command-line tool has communication with your cluster.
• Kueue is installed.

0. Create WorkloadPriorityClass

The WorkloadPriorityClass should be created first.

apiVersion: kueue.x-k8s.io/v1beta2
kind: WorkloadPriorityClass
metadata:
  name: sample-priority
value: 10000
description: "Sample priority"

1. Create Job with kueue.x-k8s.io/priority-class label

You can specify the WorkloadPriorityClass by setting the label kueue.x-k8s.io/priority-class. This is same for other CRDs like RayJob.

apiVersion: batch/v1
kind: Job
metadata:
  name: sample-job
  labels:
    kueue.x-k8s.io/queue-name: user-queue
    kueue.x-k8s.io/priority-class: sample-priority
spec:
  parallelism: 3
  completions: 3
  suspend: true
  template:
    spec:
      containers:
      - name: dummy-job
        image: registry.k8s.io/e2e-test-images/agnhost:latest
        args: ["pause"]
      restartPolicy: Never

Kueue generates the following Workload for the Job above. The priority of workloads is utilized in queuing, preemption, and other scheduling processes in Kueue. This priority doesn’t affect pod’s priority.
The kueue.x-k8s.io/priority-class label on the Job can be changed while the Job is suspended. When updated, Kueue reconciles the Workload’s priority fields accordingly. On the Workload, Priority is always mutable. priorityClassRef (and its group/kind) is mutable while the Workload is pending, and becomes immutable once the QuotaReserved condition is True. priorityClassRef.name follows the same rule, except when priorityClassRef.kind is WorkloadPriorityClass. In that case, .priorityClassRef.name can still be updated after the QuotaReserved condition is True.

apiVersion: kueue.x-k8s.io/v1beta2
kind: Workload
metadata:
  name: job-sample-job-7f173
spec:
  priorityClassSource: kueue.x-k8s.io/workloadpriorityclass
  priorityClassName: sample-priority
  priority: 10000
  queueName: user-queue
  podSets:
  - count: 3
    name: dummy-job
    template:
      spec:
        containers:
        - name: dummy-job
          image: registry.k8s.io/e2e-test-images/agnhost:latest
          args: ["pause"]