Local CI

pnpm ci:local runs the jobs of .github/workflows/ci.yml on your machine with act and Docker. When the run ends, the command sets the CI (local) commit status on the commit. A green CI (local) status is a sign-off: the person who ran the command says that the workflow passed on that commit.

Use it when GitHub Actions cannot run CI, for example when the account has no Actions credits. A GitHub ruleset can then require the sign-off before a merge.

Also use it for draft pull requests and stacked pull requests, whose base is not main. GitHub runs CI only for pull requests into main that are ready for review, so that these pull requests do not spend Actions minutes. A stacked pull request runs CI on GitHub after its base changes to main and its branch is up to date.

Set up your machine

Do these steps one time on each machine. Only macOS with OrbStack is tested.

1. Install Homebrew and the workspace tools in Getting started.

2. Install OrbStack, act, and the GitHub CLI:

   brew install --cask orbstack
   brew install act gh

3. Open OrbStack one time, and let it finish its setup. In Settings > System, give it all CPUs and at least 8 GB of memory. The tested machine is a Mac with 8 CPUs, and OrbStack had all 8 and 12 GB. With less, the jobs are slower and the browser tests can time out. orb config show prints the current values. orb config set cpu <count> and orb config set memory_mib <MiB> change them; restart OrbStack after the change, which stops the running containers.

4. Sign in to GitHub. Use the account that has write access to conveyal/concourse:

   gh auth login

   The default scopes include repo, which the command needs to read the commit and set the status.

5. Run pnpm install in the repository root, as in Getting started. The command reads the Playwright version from the installed packages.

6. Check the setup from the repository root:

   gh auth status
   docker info --format '{{.OperatingSystem}}'
   act --version

   docker info must print OrbStack. If it prints an error, start OrbStack. If another Docker context is active, run docker context use orbstack.

The command needs about 9 GB of free disk space:

• The images use about 3.5 GB: the job image, which shares most of its layers with catthehacker/ubuntu:act-latest, and MySQL.
• The concourse-ci-local Docker volume keeps the caches between runs, about 5 GB. Each of the two jobs has a pnpm store of about 2.1 GB, the Playwright browsers use about 700 MB, and the Turbo caches use about 150 MB. The command removes Turbo cache entries that are older than 7 days.
• ~/.cache/concourse-ci-local keeps the clone and the action downloads, about 300 MB.

docker system df --verbose shows the size of the volume. You can delete the volume at any time with docker volume rm concourse-ci-local; the next run fills it again.

GitHub Codespaces cannot run the command, because the devcontainer has no Docker.

Sign off a pull request

1. Commit your change and push the branch. The command tests a clean clone of the commit, and it sets the status on the commit on GitHub. Uncommitted changes are not tested.

2. From the repository root, run:

   pnpm ci:local

3. Wait for the command to finish. On the tested machine:

   • A run that cannot use the Turbo caches takes about 6 minutes. This is the case when the change affects every package, for example a change to pnpm-lock.yaml.
   • A run after a small change takes less, because Turbo uses the earlier results of the packages that did not change. A run on a commit that already ran takes about 1 minute.
   • The first run on a machine takes 1 to 2 minutes longer, because it also downloads the images, builds the job image, and fills the caches.

The command prints the act output of all jobs. Each line starts with the job ID, such as [checks] or [e2e]. At the end it prints one of these results:

• The CI workflow passed on <commit>. The status is success.
• CI jobs failed on <commit>: <jobs>. The status is failure. Search the output for ❌ to find the failed step.

On the pull request, the CI (local) status shows the result, the run time, which packages ran, and the act version, image, and Docker host that ran it. affected since <commit> means the packages that the changes since that commit on main affect. all packages means a commit that is already on main. GitHub also shows who set the status.

Run the command again after each push. A new commit has no CI (local) status, and the ruleset requires the status on the latest commit of the pull request. The ruleset also requires the branch to be up to date with main. So after you rebase onto main, push and run the command again.

A stacked pull request needs the status only when its base is main. When the pull request below it merges, rebase onto main, push, and run the command.

To stop a run, press Ctrl+C. The command sets the status to error and removes the containers.

Set up GitHub

An administrator does these steps on GitHub. Before you start, run pnpm ci:local one time on any pull request. GitHub suggests only the status names that it saw in the last 7 days.

Require the sign-off

The main ruleset requires the verify and Preview health checks. GitHub Actions makes both, so no pull request can merge while Actions cannot run. To require the sign-off instead:

1. Open Settings > Rules > Rulesets in conveyal/concourse, and open the main ruleset.
2. In Require status checks to pass, select Add checks. Type CI (local), and select it.
3. Set the source of CI (local) to Any source. A person sets this status with the GitHub CLI, not a GitHub App, so a source app would never match.
4. Remove verify and Preview health. Write down that you removed them, so that you can add them again.
5. Keep Require branches to be up to date before merging on.
6. Select Save changes.

To check the result, run:

gh api repos/conveyal/concourse/rulesets \
  --jq '.[] | select(.name == "main") | .id' |
  xargs -I{} gh api repos/conveyal/concourse/rulesets/{} \
  --jq '.rules[] | select(.type == "required_status_checks") | .parameters'

The output must list CI (local) and no integration_id for it.

Without Preview health, a pull request can merge without a Concourse Preview. The preview-database.yml workflow makes the Preview database, and it needs GitHub Actions and the repository secrets. Local CI cannot replace it.

When GitHub Actions runs again

1. In the main ruleset, add verify and Preview health again. Set their source to GitHub Actions.
2. Remove CI (local), or keep it. If you keep it, every pull request needs both a GitHub run and a local run.

Production deployments

Local CI does not change the production migration gate. Vercel assigns the production alias only after the CI / verify job and the Concourse production migration status pass on the main commit. GitHub Actions makes both. While Actions cannot run, Vercel builds each main commit, but it does not assign the production alias.

Who can sign off

Each person with write access to the repository can set a commit status. The status is a statement, not proof: GitHub cannot see where the tests ran. Anyone with write access can also set the status with gh api and not run the tests. Use the sign-off only with people that you trust to run the command.

The jobs run the code of the commit on your machine, as pnpm test does. Read a change from somebody else before you run it. The jobs get no GitHub token and no access to the Docker host: only the command uses your GitHub CLI login, to set the status.

How the run differs from GitHub

The jobs and steps are the same as on GitHub, and so is the choice of packages. A commit with changes on top of main runs as a pull request, and Turbo runs only the packages that the changes affect. GitHub compares with the newest main commit, and the command compares with the last main commit in the branch. For a branch that is up to date with main, which the ruleset requires, both are the same commit. A commit that is already on main runs every package, as a push to main does.

These things are different:

• On Apple silicon, the containers are linux/arm64. GitHub runs linux/amd64.
• The job image is catthehacker/ubuntu:act-latest at a fixed digest, with the system packages of the Playwright browsers. It has fewer tools than the GitHub image.
• The two jobs share one Docker host and its CPUs. GitHub gives each job its own machine.
• Turbo uses the task results of earlier local runs on the same machine. On GitHub it uses the Vercel remote cache. In both, Turbo uses a result only when the hash of the task inputs is the same.
• Under act, the workflow skips the remote cache step, because act has no GitHub OIDC token. It skips the report upload, because the act artifact server does not accept it. It copies the clean clone instead of fetching the commit, so the jobs need no GitHub token. pnpm/setup does not cache the pnpm store, because the volume keeps it.
• The command takes the place of the verify job.

How the command works

apps/cli/operations/ci-local.ts owns the run. In order, it:

1. Checks the GitHub CLI login, Docker, and act, so that a missing tool fails before the tests. It asks GitHub to compare the commit with main. The answer gives the base commit for Turbo, and it shows that GitHub has the commit. It stops when the commit has an .actrc file, when the repository is shallow, or when the containers of another act run of a workflow named CI run.

2. Makes a clean clone of the commit in ~/.cache/concourse-ci-local/checkout. For a pull request, it writes the pull_request event with the base commit in ~/.cache/concourse-ci-local.

3. Removes the Docker resources that an earlier run left. It builds the job image concourse-ci-local:runner: the first build takes about 90 seconds, and Docker then reuses it. It creates the concourse-ci-local volume, removes the old Turbo cache entries and the caches of jobs that the workflow no longer has, and creates the act-CI-local network.

4. Sets the CI (local) status to pending.

5. Runs the checks and e2e jobs at the same time, each in its own act process. In checks, the compiler checks, the unit tests, the integration tests, and the builds run one after the other, as on GitHub.

6. Removes the containers, networks, and volumes whose names start with act-CI-, and sets the status to success or failure.

Under act, the workflow uses a checkout step without inputs. act skips that step and copies the clean clone into each job container, with its Git history. So the jobs need no GitHub token, and they get no access to the Docker host. act runs with an empty home folder, so that no .actrc file of yours adds options such as --dryrun.

The jobs cannot both use the host network. On one network they share ports, and the browser tests use the same fixed ports as the Auth and Email unit tests. So e2e runs on the act-CI-local network, where each container has its own ports. The network has IPv6, like the host network: without an IPv6 address, Node resolves localhost to 127.0.0.1 only, but Next binds localhost to ::1. checks stays on the host network, because its integration tests reach the MySQL service on 127.0.0.1.

The volume has a pnpm store and a Turbo cache for each job, so two jobs never write to one cache at the same time. Two installs into one pnpm store can fail: the prepare script of the workspace patches packages in the store. Each job also has its own folder for the action downloads, because parallel downloads collide.

To use a newer catthehacker/ubuntu:act-latest, change the digest in ci-local.ts. The command builds the job image again on the next run.

Run one pnpm ci:local at a time on a machine. Two runs use the same clone, network, and container names.

Troubleshooting