Molecule CI in GitHub Actions

Intro

---

In the last article we took a look at a fully automated CI for an Ansible role using Molecule. Running this locally works fine during development but to test contentiously against multiple distributions, we need to scale this up a bit. A great and free option are GitHub Actions that run on every new push. This example will stay with testing for AlmaLinux9 and Ubuntu22.04 for the demonstration.

Any files used for this demonstration can be found here @@TODO ensure nice preview

Requirements

---

GitHub

You will need a GitHub account, a repository and a basic understanding of the ways GitHub and git itself work. I will give a brief intro to GitHub Actions and link to the documentation for further details.

VSCode-Extension

While not an actual requirement I can recommend using VSCode and the official GitHub Actions extension to see running actions and their results right in the editor interface. It also provides some level of linting while writing workflows. This way you can spot certain errors before committing, which is always nice to have. Keep in mind that the linting is quite limited and doesn't seem check content like bash code.

Overview

---

Tools

We will keep using VirtualBox, Vagrant and Molecule - just like before. But this time they don't install or run on your local machine, they run inside a GitHub-Hosted-Runner. And since all of that belongs to Microsoft, in the Azure-Cloud, see the documentation for further details. We will write a YAML (.yml) file called Workflow which will describe the installation of all required tools and python packages. In this case a python virtual environment isn't necessary since the GitHub Actions run inside virtual machines are ephemeral anyway. Meaning they get deleted right after the run and just report back the results.

Side node on Virtual-Box

While it has the great benefit of working cross-platform (Linux, Windows, Mac) it also introduces a bottleneck and overhead. For now I'll stick with it since the pipeline is working well enough to develop roles but to optimize it in the future I plan on giving the molecule-lxd driver and the molecule podman driver a try. Will write about it this as soon as I got something working.

Role Adjustments

---

Per-Distribution Scenarios

First we need to add dedicated molecule scenarios per distribution we want to test. This also ensures that all results will be easily to distinguish in GitHub Actions later on. This can be achieved by adding additional sub-directories in the molecule/ directory. So in this case a almalinux9 and a ubuntu2204 sub-directory shall do it. Now we could just copy over all the .yaml files form the default scenario but this would be very inefficient. Instead we just copy the molecule.yml files, the one we actually need to adjust per scenario and symlink the others. This way we can just point to the .yml files placed inside the default/ scenario for any files except the molecule.yml file

From the repositories root run this neat one-liner, which I split here over multiple lines or run the commands step by step.

mkdir -p molecule/{almalinux9,ubuntu2204} && \
cp molecule/default/molecule.yml molecule/almalinux9/ && \
cp molecule/default/molecule.yml molecule/ubuntu2204/ && \
(cd molecule/almalinux9 && ln -s ../default/{create,converge,verify,destroy}.yml .) && \
(cd molecule/ubuntu2204 && ln -s ../default/{create,converge,verify,destroy}.yml .)

After that just change the name, box and the box_version in molecule/ubuntu2204/molecule.yml to something like this, based on vagrant cloud box.

  - name: Ubuntu2204
    box: ubuntu/jammy64
    box_version: "20241002.0.0"

The result should look something like this.

ls -go /molecule/ubuntu2204
total 4
lrwxrwxrwx. 1  23 Aug 16 11:29 converge.yml -> ../default/converge.yml
lrwxrwxrwx. 1  21 Aug 16 11:29 create.yml -> ../default/create.yml
lrwxrwxrwx. 1  22 Aug 16 11:29 destroy.yml -> ../default/destroy.yml
-rw-r--r--. 1 555 Aug 16 11:35 molecule.yml
lrwxrwxrwx. 1  21 Aug 16 11:29 verify.yml -> ../default/verify.yml

Role Adjustments

To get this Ansible role working for AlmaLinux9 and Ubuntu2204 a like I needed to implement a couple of adjustments, which mainly focus on improving the re-usability of the logic. For example the package installation now looks like this.

- name: Install Apache web server
  become: true
  ansible.builtin.package:
    name: "{{ webserver_package[ansible_os_family] }}"
    state: present

---

- name: Gather facts
  ansible.builtin.gather_facts:

Turns out: Depending on the OS family (AlmaLinux9 is based on RedHat, while Ubuntu is based on Debian) the package has a different name and writing one task per option is pretty redundant, so I added the package names into a small dictionary. This change also needed to be applied to the service task and a bunch of other places.

- name: Install Apache web server
  become: true
  when: ansible_os_family == "Debian"
  ansible.builtin.apt:
    cache_valid_time: 3600
    update_cache: true

Other than that the Ubuntu vagrant box comes with an outdated or empty apt cache so an update task was required. Those were the critical changes, see this commit for details.

GitHub Actions

---

This is GitHub's integrated CI/CD and Automation platform, see the docs for more detailed information. As operating system version we gonna use Ubuntu 24.04 since this is the latest Linux runner image provided by GitHub at this point. See also this overview of currently available runners and their specs as well as the repository for their images.

Setup

---

Create a Workflow

After creating and cloning the repository (assuming it's a new one), we need to create a .yml file inside the repository at this location.

<repo-root>/.github/workflows/<workflow-name>.yml

You might need to create some of the directories first. GitHub expects workflow files in that exact location and will call them only this way.

Workflow Structure

We will create one workflow per molecule test scenario since this allows for easy workflow badge creation later on for the README.md file. Depending on your use-case you might want a different structure tho. To keep the start simple we will walk through the first workflow in a quite hard-coded way before moving the logic into its own file to create a reusable workflow in the next part.

Workflow Implementation

I'll provide the workflow .yml file here in snippets and explain briefly below them what they do. Fortunately the human readable and simple structure of YAML should make understanding them pretty straight forward. Any steps using the run statement execute bash commands. When using ubuntu-latest runner here, these commands get executed like they would in a terminal.

---

name: Ubuntu2204-CI

on:
  push:
    branches: ["main"]
    paths-ignore:
        - 'README.md'
        - '**/*.md'
        - .gitignore
        - ./meta/main.yml

• Line 1 - Any YAML starts with --- at the first line
• Line 3 - Workflow name, used as reference in several overviews
• Line 5 ff. - Workflow trigger event, "on push to main" is used here, see other options
• Line 8 ff. - Ignore paths used here for .md and other files with config/meta information

jobs:
  molecule-ci:
    runs-on: ubuntu-latest

• Line 14 - The only job we need gets defined (molecule-ci) will show up in summary
• Line 16 - GitHub runner type ubuntu-latest, other runner images can be found here

    steps:
      - name: Checkout repo
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

• Line 18 - List of steps starts, these execute the workflow logic
• Line 19 ff. - Repo gets cloned onto the runner, by default, the repo it was started from

      - name: Set up Python 3.12
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install Python dependencies
        run: |
          python -m pip install --upgrade pip
          if [ -f requirements.txt ]; then pip install -r requirements.txt; fi

      - name: Install Ansible dependencies
        run: |
          if [ -f requirements.yml ]; then ansible-galaxy install -r requirements.yml; fi

• Line 24 ff. - Run setup-python to let the workflow install the requested python version
• Line 29 ff. - Install Python requirements in case a requirements.txt file exists
• Line 34 ff. - Install Ansible requirements in case a requirements.yml file exists

echo "LOGNAME=$USER" >> $GITHUB_ENV
echo "USER=$(whoami)" >> $GITHUB_ENV

• Line 38 ff. - Environment variables used by VirtualBox @@TODO what are they used for?

echo "LOGNAME=$USER" >> $GITHUB_ENV
echo "USER=$(whoami)" >> $GITHUB_ENV

• Line 43 ff. - Setup VirtualBox repository and install VirtualBox 7.1.X
• Line 50 ff. - Verify VirtualBox version after installation

Note on reliability

Workflow might fail when VirtualBox servers are down or vagrant boxes are temporarly unavailable, this already happened a few times sicne I started using it. Also sometimes the AlmaLinux9 box just fails to create for no particular reason - so just restart the job

echo "LOGNAME=$USER" >> $GITHUB_ENV
echo "USER=$(whoami)" >> $GITHUB_ENV

• Line 53 ff. - Setup Hashicorp repository and install latest version of Vagrant
• Line 61 ff. - Verify Vagrant version after installation

echo "LOGNAME=$USER" >> $GITHUB_ENV
echo "USER=$(whoami)" >> $GITHUB_ENV

• Line 64 - Run molecule test for chosen scenario

These commands should be similar or even identical to the ones you ran to install Molecule and VirtualBox on your local system. Implemented in a GitHub Actions Workflow like this they will be executed each time the workflows gets triggered (e.g. on push to main) keeping the setup consistent and repeatable to run in a fully automated way. And due to the fact that the environment gets constructed from the ground up every time you don't need to worry about any kind of cleanups here.

In case a workflow run fails due to hick-ups that are not related to the current code you can simply rerun it from the workflow from the workflows summary page found under the Actions tab of your repository.

Wrap up

This should now work on any new push to the main branch and run your roles for you on different distributions automatically.

Now imagine you implement a bunch of Ansible roles using this CI setup - wouldn't it make sense to have the testing environment separated and just link it to the role repositories. So updating the testing setup can be done in one place and all roles receive it. That's exactly what we gonna do next time using Git Sub modules.

Thanks for reading and stay tuned.

Share on Socials

Share on Share on