How to migrate your Jenkins pipelines to Buildkite
How to migrate your Jenkins pipelines to Buildkite
So you discovered Buildkite and fell in love with its hybrid architecture, blazing speed, security, and flexibility, but you have a problem. All your CI/CD pipelines currently run in Jenkins. If that's the case, then this hands-on tutorial is for you, as we'll cover how to migrate your pipelines from Jenkins to Buildkite.
Setting up Buildkite
Unlike Jenkins, getting started with Buildkite is straightforward. You just need to install your first agent by following the instructions in the documentation. You can install the Buildkite agent, either locally or in the cloud, on all major operating systems, including Ubuntu, Debian, Red Hat, FreeBSD, macOS, Windows, Linux, Docker, AWS, and Google Cloud.
Given its hybrid nature, once the agent is installed, you can use the intuitive Buildkite UI to orchestrate your pipelines. However, you will need to migrate your existing pipelines from Jenkins to Buildkite first.
Migrating Jenkins pipelines to Buildkite
Now that your Buildkite agent is installed, the fun begins: migrating your Jenkinsfiles.
To do this, consider the following Jenkins pipeline:
As you can see, there is no logic applied. This is intentional since the point of this pipeline is to show you how to migrate each block of code to Buildkite's pipeline.yml. Let's start with the two top-level elements: pipeline and agent.
pipeline: In the Jenkins declarative syntax, it is essential to explicitly declare the pipeline using this argument.
agent: Similar to pipeline, Jenkins’ declarative syntax requires specifying which agent will execute the pipeline or stage. In the example above, agent any means that any agent can process the pipeline.
On the other hand, in Buildkite, the arguments explained above are not necessary. In other words, the following it’s a valid pipeline in Buildkite:
It’s not necessary to declare the pipeline. As for the agents, if you want any of them to take care of the job (agent any) then you can skip its declaration. But what if you want a specific agent to run a pipeline?
In Jenkins, the syntax would be similar to the following:
For its part, Buildkite uses tags to create pools of agents. So the equivalent pipeline could be like the following:
Note that to use this feature, you must specify the tag when starting the agent you want to use. For example:
For more information, we suggest you read Buildkite Agent Job Queues.
Let's now move on to environmental variables. In Jenkins, its structure is similar to the following:
In Buildkite, declaring your environmental variables is as easy as it is in Jenkins:
So far, so good. The migration process is relatively easy. However, everything changes when declaring the stages and steps of the pipeline. :-)
Declaring stages and steps in Buildkite
When it comes to stages and steps, it’s essential to understand the differences between how Jenkins and Buildkite work.
stages: this attribute is required in Jenkins to indicate the start of a group of stages such as dev, staging, prod, etc.
stage: defines a specific stage of the pipeline. The example above declares "Build" using the stage attribute.
steps: Jenkins uses the steps attribute to group commands, actions, and more.
Buildkite's approach to defining stages and steps is different from Jenkins's. To begin with, declaring each stage separately is unnecessary since the steps attribute does just that. This is possible because Buildkite uses six types of steps:
command: as its name suggests, the command attribute is used to execute shell commands.
wait: the wait step pauses the pipeline until the previous steps complete successfully.
block: the block step is similar to the wait step, with the notable difference that the pipeline requires the user to unlock it manually.
input: like the block step, the input step requires user intervention to resume the pipeline; however, it does not have dependencies with any other step.
trigger: thanks to the trigger step, you can start another pipeline which can be helpful in complex use cases where you want to separate jobs into different pipelines.
group: as its name suggests, the group step packs several steps to show them as one in the Buildkite UI.
Based on the above, we could migrate the example Jenkinsfile to the following Buildkite pipeline.yml file:
Note that the syntax is simpler and more flexible than in Jenkins. That opens the door to migrate your current pipelines and optimize them by moving them to Buildkite.
Final thoughts
There's a lot more to talk about Tot, like the fact that you can build pipelines programmatically using YAML files or from Buildkite's UI. Moreover, there are other topics to discuss, like migrating conditionals, skipping builds on errors, and more.
