I'm trying to replace my legacy job scheduler with Kubernetes job and wondering how to write sequential jobs as a Kubernetes job.
First, I wrote the following script to execute job1
and job2
in the written order but it didn't work as I expected.
apiVersion: batch/v1
kind: Job
metadata:
name: sequential
spec:
activeDeadlineSeconds: 100
template:
metadata:
name: sequential_jobs
spec:
containers:
- name: job1
image: image1
- name: job2
image: image2
restartPolicy: Never
The job described above seems to run job1
and job2
in parallel.
Is there any good way to run job1
and job2
in the written order?
Appended.
I recently found https://github.com/argoproj/argo very good for my usecase.
After a few attempts, I did this and that solved the basic problem (similar to what the OP has posted). This configuration ensures that job-1
completes before job-2
begins. If job-1
fails, job-2
container is not started. I still need to work on the retries and failure handling, but the basics works. Hopefully, this will help others:
apiVersion: v1
kind: Pod
metadata:
name: sequential-job
spec:
initContainers:
- name: job-1
image: busybox
# runs for 15 seconds; echoes job name and timestamp
command: ['sh', '-c', 'for i in 1 2 3; do echo "job-1 `date`" && sleep 5s; done;']
- name: job-2
image: busybox
# runs for 15 seconds; echoes job name and timestamp
command: ['sh', '-c', 'for i in 1 2 3; do echo "job-2 `date`" && sleep 5s; done;']
# I don't really need the 'containers', but syntax requires
# it so, I'm using it as a place where I can report the
# completion status
containers:
- name: job-done
image: busybox
command: ['sh', '-c', 'echo "job-1 and job-2 completed"']
restartPolicy: Never
Update
The same configuration as above also works inside a Job spec:
apiVersion: batch/v1
kind: Job
metadata:
name: sequential-jobs
spec:
template:
metadata:
name: sequential-job
spec:
initContainers:
- name: job-1
image: busybox
command: ['sh', '-c', 'for i in 1 2 3; do echo "job-1 `date`" && sleep 5s; done;']
- name: job-2
image: busybox
command: ['sh', '-c', 'for i in 1 2 3; do echo "job-2 `date`" && sleep 5s; done;']
containers:
- name: job-done
image: busybox
command: ['sh', '-c', 'echo "job-1 and job-2 completed"']
restartPolicy: Never