Testing and CI

Every pull request and every push to the default branch runs automated checks with GitHub Actions. A PR is only merged when they pass.

diffbot

Workflow	File	What it checks
CI	.github/workflows/diffbot_ci_action.yml	Builds and tests all catkin packages with industrial_ci, against ROS Noetic packages from the ROS testing and main repositories (one job each)
Firmware	.github/workflows/firmware.yml	Builds the Teensy firmware in diffbot_firmware with PlatformIO, once for each board in its platformio.ini: teensy40 (Teensy 4.0) and teensy31 (Teensy 3.1/3.2)
Dev container	.github/workflows/devcontainer.yml	Builds the dev container image, creates the container (which builds the workspace), then builds and runs the tests in it
PR description	.github/workflows/pr_description.yml	Checks that the pull request's description links its documentation: the docs pull request, a page on this site, or None (how)
Robot container	.github/workflows/robot_container.yml	Builds the robot image when it or its packages change, natively on GitHub's arm64 and amd64 runners. It checks the settings template and runs the image's smoke test without hardware, in the image and through Docker Compose

industrial_ci

industrial_ci starts a ROS Docker image, installs the packages' dependencies with rosdep, builds the workspace with catkin and runs the tests. The workflow runs it twice: with ROS packages from the main repository, which users install, and from testing, where new package versions appear first (ROS testing repository).

The simulation test also drives Remo, whose model remo_description isn't a released ROS package, so rosdep can't install it. UPSTREAM_WORKSPACE: diffbot_dev.repos in the workflow makes industrial_ci clone the repositories in this vcstool file, the same source dependencies that the dev container imports. industrial_ci builds them first, in an upstream workspace, then builds the diffbot packages on top of it and runs only their tests (workspace management).

ccache speeds up the C++ builds. The workflow stores its cache with actions/cache. The cache key contains the run ID, so each successful run saves a new cache, and restore-keys loads the newest one at the start of the next run.

Dev container workflow

devcontainers/ci creates the container exactly as VS Code or the Dev Container CLI would, including setup.sh, which builds the workspace. Then it runs:

catkin build --catkin-make-args run_tests
catkin_test_results build

So a broken Dockerfile, a missing dependency or a failing build shows up in the PR, not on a developer's machine weeks later.

PR description

diffbot's pull request template has a placeholder under ## Documentation for the docs pull request or the page on this site that explains the change. A template only fills in the text when a pull request is opened, so pr_description.yml checks that the section was filled in.

What it accepts

• a docs pull request, as ros-mobile-robots/ros-mobile-robots.github.io#N or its URL on GitHub
• a link to a page on ros-mobile-robots.com that already explains the change
• None or None needed, alone or with a reason after a dash or colon, when nothing user-visible changes

The template's comments don't count. The check looks at the format only. Whether the linked page really explains the change is for the review.

When it runs

The excerpts show the workflow at the diffbot commit that this page pins, with its line numbers. lines 7–20:

on:
  pull_request:
    # edited: an edit of the description runs the check again with the new text. "Re-run jobs"
    # on an old run doesn't, because a re-run keeps the old event and its old description.
    # synchronize: each new commit gets the check too, as it does for the other workflows.
    types: [opened, edited, reopened, synchronize]

# Runs for the same pull request share a group, and a new one cancels the one still running
concurrency:
  group: pr-description-${{ github.event.pull_request.number }}
  cancel-in-progress: true

# The script reads only the event, so it needs no token permissions
permissions: {}

• types: the workflow runs when a pull request is opened, reopened or edited, and when it gets new commits. So after fixing the description, the new run shows the result, without a new commit. "Re-run jobs" on the old, failed run doesn't help: a re-run repeats the old event, with the old description.
• concurrency: runs for the same pull request share a group. When a new run starts, for example after another edit, GitHub cancels the one still running.
• permissions: {}: the check only reads the event, so it turns off the GITHUB_TOKEN permissions that a workflow can set (permissions). The token can't be used to write to the repository.

How the script works

The job has one step, a Python script, lines 30–34:

- name: Check the Documentation section
  env:
    BODY: ${{ github.event.pull_request.body }}
  run: |
    python3 - <<'EOF'

env puts the description into the environment variable BODY. python3 - reads Python code from its standard input: the shell's <<'EOF' passes it the lines that follow, up to a line with only EOF. The quotes around EOF keep the shell from changing anything in those lines. The script removes the template's comments and finds the section from ## Documentation to the next ## heading or the end of the description, lines 40–44:

# The template's comments are hints, not content
body = re.sub(r"<!--.*?-->", "", os.environ.get("BODY") or "", flags=re.S)
section = re.search(r"^##\s+Documentation\s*$(.*?)(?=^##\s|\Z)", body,
                    flags=re.M | re.S | re.I)
text = section.group(1) if section else ""

Then it looks for the three accepted forms, lines 46–67:

# The docs repository's full reference, not part of a longer name or path
docs_pr = re.search(r"(?<![\w./-])ros-mobile-robots/ros-mobile-robots\.github\.io#\d+\b", text)

def linked(url):
    """A docs pull request on GitHub, or a page on the documentation site."""
    # Punctuation after a bare link ends the sentence, as in GitHub's autolinks
    url = url.rstrip(".,:;!?*_~")
    try:
        parts = urlsplit(url)
    except ValueError:
        return False
    host = (parts.hostname or "").lower()
    if parts.scheme != "https" or parts.username is not None or parts.password is not None:
        return False
    if host == "github.com":
        return re.fullmatch(r"/ros-mobile-robots/ros-mobile-robots\.github\.io/pull/\d+(/.*)?",
                            parts.path) is not None
    return host in ("ros-mobile-robots.com", "www.ros-mobile-robots.com")

urls = re.findall(r"https?://[^\s<>()\[\]`\"']+", text)
# "None" or "None needed", alone or followed by a reason after a dash or colon
none = re.search(r"^\s*None(?: needed)?(?:\.?\s*$|\s*[-–—:]\s*\S)", text, flags=re.M)

• docs_pr: the docs repository's full reference with a number. The (?<![\w./-]) before it makes sure that it isn't the end of a longer name or path.
• linked(): a link counts only with https and the right host. urlsplit() takes the link apart, so https://ros-mobile-robots.com.example.org (host ros-mobile-robots.com.example.org) and https://ros-mobile-robots.com@example.org (host example.org) don't count. On github.com, only the docs repository's pull requests count. Punctuation after a bare link, such as the period at the end of a sentence, isn't part of it.
• none: None at the start of a line, alone or with a reason after a dash or colon. "None yet, a docs PR will follow" doesn't count.

Last, it reports the result, lines 69–82:

if section is None:
    problem = 'The description has no "## Documentation" section.'
elif docs_pr or none or any(linked(url) for url in urls):
    print("The Documentation section is filled in.")
    sys.exit(0)
else:
    problem = ('The "## Documentation" section names neither a docs pull request nor '
               'a page on ros-mobile-robots.com, nor says None.')
print("::error title=PR description::" + problem + " Link the docs pull request as "
      "ros-mobile-robots/ros-mobile-robots.github.io#N, or a page on "
      "https://ros-mobile-robots.com, or write None if nothing user-visible changes. "
      "Editing the description runs this check again.")
sys.exit(1)
EOF

The printed ::error title=…::… line is a workflow command: the runner, the machine that runs the job, turns it into an error annotation on the pull request. The exit code 1 fails the check.

Why the description is an environment variable

GitHub replaces each ${{ … }} expression in a run: script with its value before the shell runs the script. Anyone who opens a pull request writes its description. If the description were inserted between triple double quotes in a Python string, a crafted description could close the string with """ and add Python statements of its own, which would run on the runner. That's a script injection.

With env, the description stays data. GitHub puts it into the variable BODY, and the script reads it with os.environ.get("BODY") as a string, which is never run. GitHub recommends this for untrusted input such as a pull request's title or description (good practices).

Why the script prints no description

The runner reads every line that the script prints, and runs the workflow commands among them, such as the ::error line above. If the script printed the description, a description with a line like ::error::… would create an annotation of its own. So the script prints only fixed text. The runner lists the step's environment, the description included, at the top of the step's log, but it doesn't run commands from that list. In a test run on diffbot#163, a description with ##[error] and ::error:: lines showed them in that list and created no annotation.

Tests

Only diffbot_gazebo registers catkin tests so far: the simulation test and the geometry test below. The robot image has a smoke test of its own. So for most changes the checks prove that everything builds, and a pull request describes how it was tested, for example in the simulation or on the robot. Unit tests for the firmware on the PC (diffbot#158) and tests for diffbot_base without the robot (diffbot#114) are planned.

Simulation test

diffbot_gazebo drives DiffBot and Remo in Gazebo and checks where they went, with the robot's odometry and its true pose. Both workflows run it on every pull request. Simulation Test explains what it checks, how Gazebo and ros_control work together in it, and how to run it and add checks.

Geometry test

test_geometry_consistency.py checks that each robot's URDF and its controller configuration agree on the wheel radius and the wheel separation, as the simulation's and the real robot's launch files load them. It also checks the wheel joints that the controller drives by name. It runs without Gazebo, for both robots, and catches differences that the simulation test would miss. Geometry Test explains what it compares, how it works and how to run it.

Robot container smoke test

The robot image is checked without hardware: its packages and programs, the robot models, the launch files, and Remo's bringup with both controllers running. Whether the robot drives with it needs a test on the robot (diffbot#109).

GitHub Actions versions

The workflows use the latest major versions of the GitHub actions, which run on Node 24. GitHub retires old versions: in 2026 the CI workflow failed before building anything because actions/cache@v2 had been switched off (GitHub notice), and Node 20 actions were retired in September 2026 (GitHub changelog). When a check fails during "Set up job", an outdated action is the likely cause.

Writing tests and debugging

• Tests: ROS 1 packages use gtest for C++ unit tests and rostest for tests that start ROS nodes. Ros-Test-Example shows both in a catkin workspace (slides). catkin-tools explains building and running tests.

• Debugging: a debugger can only stop at breakpoints in a workspace built with debug symbols:

  catkin build --save-config --cmake-args -DCMAKE_BUILD_TYPE=Debug
  

  See the catkin-tools cheat sheet for more.

Running the checks locally

• Workspace build and tests: in the dev container, in ~/catkin_ws, run the two commands from the dev container workflow above.

• Firmware: build it on the host, not in the ROS container: the current PlatformIO Teensy tools need a newer C library (glibc 2.34 or later) than Ubuntu 20.04 has. In our test, the Teensy tool teensy_size stopped with a missing GLIBC_2.34 in the ROS container. CI builds on Ubuntu 24.04, and the steps below were tested on Ubuntu 24.04 (WSL 2) with Python 3.12 and PlatformIO 6.2. From the diffbot folder:

  sudo apt install python3-venv
  python3 -m venv ~/.venvs/platformio
  ~/.venvs/platformio/bin/pip install platformio
  cd diffbot_firmware
  ~/.venvs/platformio/bin/pio run
  

  pio run builds the Teensy 4.0, the default environment; pio run -e teensy31 builds the Teensy 3.1/3.2.

This documentation site

Workflow	File	What it does
Documentation CI	.github/workflows/ci.yml	Builds the site with mkdocs build --strict, which fails on any warning (strict mode), such as a broken link or anchor (link validation). On a push to main it also publishes the site to the gh-pages branch, which GitHub Pages serves at ros-mobile-robots.com.
Lint	.github/workflows/lint.yml	Checks the spelling with codespell; its settings are in .codespellrc.
PR preview	.github/workflows/preview.yml	Builds each pull request from a branch in this repository and publishes it at https://ros-mobile-robots.com/pr-preview/pr-<number>/, so changes can be checked on the real site before merging. The preview is removed when the PR is closed. Pull requests from forks get no preview, only the build and spelling checks.

To build the site locally:

pip install -r requirements.txt
mkdocs serve

Comments

Comments are GitHub Discussions, shown with giscus. Loading them connects to giscus.app (hosted by Vercel, USA) and GitHub (USA); you need a GitHub account to comment.

Show comments