Concourse CI/CD is an open source platform which runs on two main Docker containers. Once Concourse is installed, all you have to do is to use the docker-compose up command to bring up the Concourse server. Concourse uses Postgres as its database.
Let’s Install Concourse CI
Let’s install the CI first and then then we can talk about the components.
Concourse is distributed as a single
concourse binary, making it easy to run just about anywhere, especially with Docker.
$ curl -O https://concourse-ci.org/docker-compose.yml $ docker-compose up -d Creating docs_concourse-db_1 ... Creating docs_concourse-db_1 ... done Creating docs_concourse_1 ... Creating docs_concourse_1 ... done
Concourse will be running at localhost:8080 on your machine. You can log in with the username/password as
Next, install the
fly CLI by downloading it from the web UI and login to your local Concourse as the
# MacOS $ curl 'http://localhost:8080/api/v1/cli?arch=amd64&platform=darwin' -o fly \ && chmod +x ./fly && mv ./fly /usr/local/bin/ # Linux $ curl 'http://localhost:8080/api/v1/cli?arch=amd64&platform=linux' -o fly \ && chmod +x ./fly && mv ./fly /usr/local/bin/ $ fly -t tutorial login -c http://localhost:8080 -u test -p test logging in to team 'main' target saved
You’ll notice that every
fly command in this tutorial has to have the target (
-t tutorial) specified. This is annoying when you only have one Concourse to target, but it helps ensure you don’t trigger a job on the wrong Concourse instance. It will save you from hurting yourself!
Once you’ve confirmed everything is up and running by logging in through fly and the web UI, you can move onto the next section. If you refresh this page you should see the web UI in the frame below.
What is Fly ?
fly CLI is a command line tool that you need to use to set up a pipeline on Concourse. fly can be downloaded once the Concourse server instance is running and when you are logged in for the first time. The initial user ID and password by default are “test” and “test,” but you can change them later.
fly CLI will be used to set up, un-pause, pause, and destroy pipelines.
The Concourse pipeline set up needs a set of files that are associated with each repository. Each repo represents a microservice. It is best practice to associate one set of files related to Concourse to one repo.
Learn by Examples
Configuring self-contained Concourse pipelines is a great way to try things out before diving into the deeper content.
Each example contains a pipeline YAML snippet which can be copy-pasted to a local file and configured on your instance via
fly set-pipeline. From there you may want to poke around and try changing parts of the configuration to learn how things work. All the available knobs to turn are covered in the Docs.
Hello World pipeline
Let’s start where all tutorials start, with a
Hello World! example!
In this section you’re going to create a pipeline that simply prints
Hello world! to the console. While building up the pipeline we will pause to explain the core pieces of the pipeline.
Let’s first answer: what is a pipeline made up from?
The simplest Concourse pipeline is made of two objects:
If you’ve used other pipeline building tools in the past, then what you think of as a pipeline is probably most similar to a job in Concourse.
Hello World! pipeline we will need one job with one step. This is the smallest pipeline you can make in Concourse. The single step is what will print
Hello World! to the console.
Create and open a new file called
hello-world.yml. Inside that file let’s add the first top-level key,
jobs key is where we define the list of jobs that should make up our pipeline. The order of the jobs does not matter. The order of jobs does not define the structure of the pipeline. We’ll get into pipeline structure and job ordering later when we talk about Resources and passed constraints.
We only need one job right now so let’s add a single job named
jobs: - name: hello-world-job
jobs: - name: hello-world-job plan:
jobs: - name: hello-world-job plan: - task: hello-world-task
Fantastic! Now we need to tell Concourse how to run our task step. We do that by providing a task config.
jobs: - name: hello-world-job plan: - task: hello-world-task config:
At this point we are going to pause to explain steps a bit more.
A step is a single container running on a Concourse worker. Each step in a job plan runs in its own container. You can run anything you want inside the container (i.e. run my tests, run this bash script, build this image, etc.).
So if you have a job with five steps Concourse will create five containers, one for each step. Therefore, we need to tell Concourse the following about each step:
- What type of worker to run the task on (linux/windows/darwin)
- What container image to use (Linux only)
- What command to run inside the container
Concourse currently only supports Linux containers. Concourse does not yet run Windows containers. Darwin does not have native containers.
Let’s answer the previous three questions for our
- What type of worker to run the task on (linux/windows/darwin)Linux, because our docker-composed Concourse only has one linux worker. You can verify this by running
fly -t tutorial workers
- What container image to use (Linux only)We’ll use the super small busybox image
- What command to run inside the container
echo "Hello world!"
jobs: - name: hello-world-job plan: - task: hello-world-task config: # Tells Concourse which type of worker this task should run on platform: linux # This is one way of telling Concourse which container image to use for a # task. We'll explain this more when talking about resources image_resource: type: registry-image source: repository: busybox # images are pulled from docker hub by default # The command Concourse will run inside the container # echo "Hello world!" run: path: echo args: ["Hello world!"]
$ fly -t tutorial set-pipeline -p hello-world -c hello-world.yml # pipelines are paused when first created $ fly -t tutorial unpause-pipeline -p hello-world # trigger the job and watch it run to completion $ fly -t tutorial trigger-job --job hello-world/hello-world-job --watch
You’ll see extra output than what we’re showing below (the busybox image being downloaded) but the last four lines will be the task executing.
selected worker: 701785fa43a1 running echo Hello world! Hello world! succeeded
The following is an iframe to http://localhost:8080/teams/main/pipelines/hello-world