Azure Pipelines - Parallel steps (YAML)

Question

I'm setting up my Azure Devops Pipelines, and have a build that requires some fairly lengthy setup steps to run. These need to run before other tasks, which can be run in parallel.

However, I can only see this being done by specifying jobs, which would require to do these lengthy steps each time. Ie:

jobs:
  - job: Run1
     steps:
       - task: Long running setup task
       - task: Run taskA
  - job: Run2
       - task: Long running setup task
       - task: Run taskB  

Is there a way to have this long running task run, and have task A/B depend on that environment without running them sequentially? Ideally it'd be something like:

-job
  steps:
    -task: Long running setup
    -task: Parallel: taskA
    -task: Parallel: taskB

Or have the previous jobs take a container/image snapshot and reuse if that's possible?

Answer 1

Short answer, you can't.

Tasks within a job cannot run in parallel as they run on the same agent and the environments can't be "snapshotted" by Azure Devops to be re-utilized by other jobs in parallel later. But jobs can run in parallel as they can be scheduled on different agents, so setup will run twice but in parallel. So there is a trade-off between time and resource usage which you will need to decide on based on your requirements.

There is another solution though, based on how much you would like to invest in this:

If your "setup" doesn't change that often and you are willing to host your own agents. Then you could run a separate "setup + agent" build which creates a docker image of your agents, pushes it out to your azure container registry and then deploys this image to your self-hosted agents (Azure Kubernetes Service) cluster. Then Task A and Task B can easily run in parallel with the assumption that the environment they are running in (agent + setup docker image) is always ready. This is exactly my setup.

See : Azure DevOps Docker