# Docs

https://shipcheck.run/docs/

Your agent runs its tests with `shipcheck run`, and that run gives the GitHub check its PR waits for. shipcheck starts the run itself: it runs your checks locally, natively in an OS sandbox or as your workflows in containers with act, keeps the result for each git tree, and posts one required check per job, such as `shipcheck/test`.

New here? [Install it](https://shipcheck.run/docs/install.md), then follow the [quickstart](https://shipcheck.run/docs/quickstart.md). It takes a few minutes for a repository owner and one command for each teammate. Once it works, [speed up runs](https://shipcheck.run/docs/speed.md).

Using a coding agent? [Agents](https://shipcheck.run/docs/agents.md) says what `init` writes for it, with any agent.

Deciding whether to trust it? [What a green check proves](https://shipcheck.run/docs/proof.md) and [Security](https://shipcheck.run/docs/security.md) say what a check guarantees and what a job can reach.

# Install

https://shipcheck.run/docs/install/

> What shipcheck needs on the machine, and how to install it.

What a repository needs depends on its [mode](https://shipcheck.run/docs/config/#modes):

- Host mode needs no Docker. On a Mac it needs only shipcheck. On Linux it needs shipcheck and bubblewrap.
- act mode, and any `[jobs]` entry with `runner = "act"`, needs act 0.2.86 or later and a running Docker engine: Docker Desktop 4.44.3 or later, OrbStack, or colima as the Docker engine. On Linux, Docker Engine with runc 1.2.8 or 1.3.3 or later. shipcheck refuses older versions.
- A job with `engine = "colima"` needs colima and lima on the Mac, besides act and the running Docker engine above, which shipcheck checks before every act run and which runs the jobs a colima job needs. The job's own steps use the Docker inside its VM.

Every repository also needs the [shipcheck App](#post-the-checks) to post its checks on PRs.

## Set up a Mac

Tested on macOS with Docker Desktop on Apple silicon. Host mode uses macOS's own `sandbox-exec`.

1. Install shipcheck with [Homebrew](https://brew.sh). It also installs act, git and the Docker client:

   ```sh
   brew install gabrielkoerich/tap/shipcheck
   ```

2. For act mode, install and start a Docker engine. Host mode needs none:

   | Engine | Install | What ran on it |
   |---|---|---|
   | Docker Desktop 4.44.3 or later | [docker.com](https://www.docker.com/products/docker-desktop/), then open it once | The act mode trials and benchmarks on these pages. Recommended |
   | OrbStack | [orbstack.dev](https://orbstack.dev) | act jobs. It sometimes showed a job the files of the previous run at the same path, under `/var/folders`. shipcheck fails such a job |
   | colima | `brew install colima && colima start` | act jobs, with the repository and `SHIPCHECK_HOME` under your home directory, the only one colima shares by default. Elsewhere a job sees an empty checkout and fails |

   OrbStack and colima also switch Docker's default context to their own when they start. Here colima is only the Docker engine behind act's containers. `engine = "colima"`, a VM of its own for one job, is a different thing ([engines](https://shipcheck.run/docs/config/#engines)).

   shipcheck refuses Docker Desktop before 4.44.3, which let any container reach the Docker engine.

3. Check it:

   ```sh
   shipcheck doctor
   ```

   It checks git, the OS sandbox or act and the Docker engine, the disk and the repository, marks each line `ok` or `fix`, and prints the command that fixes each `fix`. It exits 0 when a run can work.

macOS disks use APFS, so each job's checkout is a copy-on-write clone and costs no disk. On Apple silicon, jobs run arm64 images. A job whose tools ship no arm64 build can get an amd64 image of its own with `act.jobs` in [`shipcheck.toml`](https://shipcheck.run/docs/config.md). Turn on Rosetta in Docker Desktop's settings so amd64 images run faster.

shipcheck keeps its results, logs and checkouts in `~/.local/state/shipcheck`. An older shipcheck kept them in `~/Library/Application Support/shipcheck`: the first command of a newer one moves that folder and says so. Then run `shipcheck install` once, so the daemon logs to the new folder. `shipcheck doctor` says when it still logs to the old one.

## Set up Linux

Tested on Debian 13 with Docker Engine 29 on x86_64.

For host mode, install bubblewrap, which runs each check in a sandbox: `sudo apt install bubblewrap` or `sudo dnf install bubblewrap`.

Ubuntu 23.10 and later let only programs with an AppArmor profile create user namespaces, and bubblewrap needs them. Allow bwrap alone with the profile apparmor ships, and leave the restriction on for everything else:

```sh
sudo apt install apparmor-profiles
sudo cp /usr/share/apparmor/extra-profiles/bwrap-userns-restrict /etc/apparmor.d/
sudo apparmor_parser -r /etc/apparmor.d/bwrap-userns-restrict
```

`shipcheck doctor` runs bwrap once. When it fails and `/proc/sys/kernel/apparmor_restrict_unprivileged_userns` is 1, it prints these commands, or only the last one when the profile is in `/etc/apparmor.d` but not loaded. Without a working bubblewrap, host mode refuses to run.

1. For act mode, install [Docker Engine](https://docs.docker.com/engine/install/) from Docker's own package repository, which brings a current runc. shipcheck refuses runc before 1.2.8 or 1.3.3, which have known container escapes.

2. Let your user run Docker without sudo, then log out and in:

   ```sh
   sudo usermod -aG docker "$USER"
   ```

   Membership of the `docker` group is as strong as root on that machine. shipcheck uses the Docker socket to start jobs, and never gives the socket to a job.

3. Install shipcheck. The install script needs no other tool:

   ```sh
   curl -fsSL https://shipcheck.run/install.sh | sh
   ```

   Homebrew on Linux works too, and also installs act, git and the Docker client: `brew install gabrielkoerich/tap/shipcheck`. Without Homebrew, install git from your distribution and act from its [releases](https://github.com/nektos/act/releases).

4. Check it:

   ```sh
   shipcheck doctor
   ```

   It prints the command that fixes each line it marks `fix`, as on a Mac.

On ext4 there is no copy-on-write, so each job's checkout is a plain git worktree: a full copy of the files and a slower start. btrfs, or XFS with reflink, gives copy-on-write clones as on a Mac. Jobs run as `runner`, uid 1001, so shipcheck gives the checkout to that uid before each job and back to your user after it.

## Other ways to install

With cargo, from crates.io. This builds it with Rust 1.89 or later:

```sh
cargo install shipcheck
```

Or with [cargo-binstall](https://github.com/cargo-bins/cargo-binstall), which downloads the binary of the latest release, signed and notarized on macOS, with no Rust needed. [mise](https://mise.jdx.dev) uses it too:

```sh
cargo binstall shipcheck
mise use -g cargo:shipcheck
```

mise waits a while before it offers a new release, so right after one it can install the one before.

With the install script, on macOS or Linux, arm64 or x86_64:

```sh
curl -fsSL https://shipcheck.run/install.sh | sh
```

It downloads the binary from the latest [GitHub release](https://github.com/gabrielkoerich/shipcheck/releases), checks it against the release's `SHA256SUMS`, and installs it to `~/.local/bin`. Set `SHIPCHECK_INSTALL_DIR` to install it somewhere else.

Or build it from source, from the repository or from a clone, with Rust 1.89 or later:

```sh
cargo install --git https://github.com/gabrielkoerich/shipcheck shipcheck
git clone https://github.com/gabrielkoerich/shipcheck && cargo install --path shipcheck
```

A build with cargo, from crates.io or from source, gets a new signature each time, so the keychain asks again after it ([where shipcheck keeps its keys](#where-shipcheck-keeps-its-keys)).

## Post the checks

The [shipcheck App](https://github.com/apps/shipcheck-run) puts the checks on your PRs. Only a GitHub App can create checks, and a ruleset can require each check to come from that App, so an agent's own token cannot post a passing one ([require the checks](https://shipcheck.run/docs/branch-protection.md)). Your machine runs the checks, and shipcheck's service posts the result with the App. The App's key stays in the service, never on your machine.

The App has three permissions:

| Permission | For |
|---|---|
| Checks: write | Posting the job checks and the `shipcheck` summary |
| Pull requests: write | Listing a repository's open PRs for the daemon, finding a commit's PRs, learning who pushed a PR's head, and the [report comment](https://shipcheck.run/docs/how-it-works/#the-report-comment) |
| Metadata: read | Basic repository data, which every App has |

It has no Contents permission, so it cannot read or change your code.

### Install the App

1. Open [github.com/apps/shipcheck-run/installations/new](https://github.com/apps/shipcheck-run/installations/new). `shipcheck init` prints this link and opens it when you apply.
2. Pick your user or an organization. On an organization, an owner installs it, or approves the request of a member who asks.
3. Choose all repositories, or only the ones that use shipcheck. Public and private repositories work the same.

To add a repository later, open [github.com/settings/installations](https://github.com/settings/installations), then shipcheck-run, Configure. For an organization, open its settings, GitHub Apps, shipcheck-run, Configure.

On a private repository, the checks and their log tails are visible to everyone who can read the repository. On a public one, everyone sees them ([`logs`](https://shipcheck.run/docs/config.md)).

### Sign in each machine

Each machine that posts needs a machine token. You need write, maintain or admin access to the repository to post its checks.

```sh
shipcheck login
```

It prints a one-time code and the page github.com/login/device. In a terminal, Enter opens that page in the browser. Enter the code there and approve. shipcheck then says `signed in as <login>` and keeps the token, in the login keychain on macOS ([where shipcheck keeps its keys](#where-shipcheck-keeps-its-keys)). shipcheck's service revokes the GitHub token from the sign-in once it knows who you are, and keeps only a hash of the machine token.

`shipcheck init` and `shipcheck install` sign the machine in the same way when it cannot post yet and they run in a terminal. Without a terminal they print one line that says to run `shipcheck login` or set `SHIPCHECK_TOKEN`, and go on.

- A machine token stops working after 90 days without a post. Run `shipcheck login` again then.
- `shipcheck logout` revokes this machine's token and deletes it.
- The service receives the check's text and the report comment, never your code. The text holds the end of each job's log ([the checks on the PR](https://shipcheck.run/docs/how-it-works/#the-checks-on-the-pr)).

### Unattended machines

A VM, a CI runner or an agent box has no person to enter the code. Make a token for it on a machine where you can open a browser:

```sh
shipcheck token --name agent-vm
```

It prints a code to enter at github.com/login/device on stderr, as `login` does, then only the token on stdout. It does not touch this machine's own sign-in, and it keeps no copy. Put the token in the other machine's environment as `SHIPCHECK_TOKEN`, or in the agent's secret store. It lasts 90 days after its last post. `shipcheck logout` on that machine revokes it.

If a machine with a token is compromised, run `shipcheck logout --all` on any signed-in machine. It revokes every token of your GitHub user at once, this machine's too, so sign the others in again after. Each GitHub user can hold 50 live tokens, and `logout --all` also clears that limit.

### The daemon

The daemon posts with the same machine token. It runs a PR only when the GitHub login you signed in as, or a login `daemon_authors` lists, opened the PR and pushed its head ([when a PR changes on GitHub](https://shipcheck.run/docs/how-it-works/#when-a-pr-changes-on-github)). The service `init` and `install` start gets only `PATH` from your environment, so it uses the `shipcheck login` token, never `SHIPCHECK_TOKEN`. Until the machine is signed in, the daemon posts nothing and its log says to run `shipcheck login`.

### Check it

`shipcheck doctor` shows two lines about posting:

- `post` says which token posts: the `shipcheck login` token or the `SHIPCHECK_TOKEN` machine token. On a machine with neither, the line stays `ok` and says ``not signed in. Optional: `shipcheck login`, or SHIPCHECK_TOKEN from `shipcheck token`, lets `shipcheck post` put the check on PRs``.
- `app`, on a machine that can post, asks GitHub for the repository's open PRs. It says `installed on <owner>/<repo>, runs can post its checks`, or, when the App is not installed there or you have no access, marks it `fix` with `add <owner>/<repo> to the shipcheck App: https://github.com/settings/installations (shipcheck-run, Configure)`.

A post that fails leaves the same reason and fix in `shipcheck status`, `shipcheck run`, the push's output and the daemon's log.

## What works without the GitHub App

These work on any repository with `shipcheck.toml` on `main`, with no App:

- `shipcheck run` runs the checks locally and gives the verdict as an exit code.
- The result cache answers at once for a tree that already passed.
- The git hooks start runs in the background on commit or push.
- `shipcheck status` and `shipcheck logs` show the stored result and logs of a commit.

## Where shipcheck keeps its keys

shipcheck keeps two secrets on your machine: the machine token from `shipcheck login`, and the secrets act jobs get, one file per repository at `secrets/<owner>/<repo>.env` in `SHIPCHECK_HOME`, in act's `NAME=value` format. `shipcheck token` prints a token for another machine and keeps no copy. The App's private key lives only in shipcheck's service.

On macOS, `shipcheck login` writes its token straight into an item of your login keychain. A secrets file you write moves into an item the first time shipcheck reads it, and the file goes to the Trash. Only shipcheck reads an item without asking, so an agent running as you gets a keychain prompt instead of the secret. To change the secrets of a repository, write the file again. An item an older shipcheck kept under its old home, `~/Library/Application Support/shipcheck`, gets its new name the first time shipcheck reads it.

The keychain knows shipcheck by its code signature. Release builds, from Homebrew or the install script, are signed with one Developer ID and notarized by Apple, so an upgrade keeps access. A build from `cargo install` or from source gets a new signature each time: macOS then asks once per item whether the new shipcheck may read it. Run `shipcheck doctor` at the Mac and choose Always Allow. Choose it only for shipcheck, never for `security` or another program.

On Linux they stay files of mode `0600`, which any program you run can read. The Linux keyring has no per-program access, so it would not keep an agent out.

# Quickstart

https://shipcheck.run/docs/quickstart/

> Set up a repository once, then each clone once, then work as usual.

## Setup

1. Install the binary: `brew install gabrielkoerich/tap/shipcheck`, or `curl -fsSL https://shipcheck.run/install.sh | sh` ([install](https://shipcheck.run/docs/install.md)).
2. Run `shipcheck init` in a clone of the repository, on `main`.
3. Commit the `shipcheck.toml` it writes and merge it to `main`.
4. Install the [shipcheck App](https://github.com/apps/shipcheck-run) on the repository. `init` prints this link.

Each other clone then needs `shipcheck install` once. The sections below say what each step does.

## For the repository owner, once

In a clone of the repository, on `main`:

```sh
shipcheck init
```

It reads your workflows and project files and changes nothing until you apply. Its steps are tabs: Checks, Install, Workflows, GitHub and Review. Move between them with the arrow keys, Tab or 1 to 5, and go back to any of them. Each starts on the recommended choice. On Checks, Space or Enter moves a job to another place it can run: a host check, act on Docker, act on a colima VM where colima could run it, or GitHub. A job becomes a host check only when it ran as one in init's trial. Apply on the Review tab. Quitting, or Ctrl-C at any point before you apply, leaves the repository and the machine as they were.

An agent or a script runs `shipcheck init --yes`, which asks nothing and applies the recommended choices, except the browser and the ruleset. Without a terminal and without `--yes`, init only previews: it tries each check with a temporary `SHIPCHECK_HOME`, prints the plan and the review `--yes` would apply, and writes nothing.

### Checks

`init` turns each job of a workflow that GitHub runs for feature branches into a check named like the job.

- The job's `run` steps are joined by `&&`.
- A job with `needs` runs the jobs it needs first.
- Package installs into the checkout, such as `npm ci`, `bun install --frozen-lockfile` or `uv sync`, stay in the check.
- The workflow's and the job's `env` are exported first, and a step's `env` only in that step.
- Commands that set up the machine, such as `apt`, `sudo`, `curl`, `brew`, `rustup`, `pip install`, `go install`, `git config` or `gh`, are left out. A step that needs root besides other commands, such as one that makes and mounts a volume, is left out whole, and so is a `TMPDIR` or `HOME` value, since the sandbox sets both.
- Steps with a GitHub expression, also in their `env`, `if:` or `continue-on-error` are left out, and so are multi-line steps with control flow.
- A setup action that installs into the checkout keeps its install: `pnpm/setup` (unless `install: false`) and `pnpm/action-setup` with `run_install: true` add `pnpm install`, `ruby/setup-ruby` with `bundler-cache: true` adds `bundle install`. `setup-node` and `setup-uv` caches install nothing. `gitleaks/gitleaks-action` becomes a `gitleaks dir --redact` scan of the commit's tracked files, which leaves out the build output a warm clone holds. A history scan stays on GitHub.
- Deploy, publish and release jobs are left out, and so are jobs that only install, move files or run git, jobs that download artifacts from a job with no check, and jobs that need a Rust target (`rustup target add`, `targets`). A pinned Rust toolchain, such as `toolchain: "1.94.1"` for `dtolnay/rust-toolchain` or `actions-rust-lang/setup-rust-toolchain`, is pinned with `RUSTUP_TOOLCHAIN` when rustup has it here. Otherwise the check runs on the default toolchain, and init lists it under "run on this machine's version, not the workflow's" until you run `rustup toolchain install 1.94.1`. A `rust-toolchain.toml` needs nothing, since rustup reads it.
- A job no host check runs as GitHub does, such as one with a dropped step, a `container:` or a Linux matrix, runs with act as a `[jobs]` entry when Docker is running, and init adds `network = "internet"`. A job stays on GitHub when act cannot run it as GitHub does either: Windows jobs, macOS jobs off a Mac, matrices with macOS or Windows entries, and jobs that read the PR anywhere, report to GitHub, or run Docker. init writes `engine = "colima"` only when you move a job to a colima VM on the Checks tab. A matrix of one combination, such as `python-version: [3.13]`, is a plain job. [Which jobs run where](https://shipcheck.run/docs/how-it-works/#which-jobs-run-where) has the table.

Only when no job gives a check, a `package.json` `test` or `lint` script, a `Cargo.toml`, a `pyproject.toml` or a `go.mod` at the root gives the checks they imply. A repository with no `.github/workflows` gets its checks this way, in host mode, the only mode it can use.

It shows each check with the file it came from, and what it left out and why. The checks run natively in a sandbox, and before the review init runs each one there once, as `shipcheck run` would. A check that fails runs again without forced color, then with color off. A job whose check still fails stays on GitHub, and init prints the end of its log and the fix. init does not try the act jobs, so a failing one shows on its first run. Edit `shipcheck.toml` afterwards to drop one:

```toml
runner = "host"

[host]
versions = ["cargo --version", "rustc --version"]

[host.checks]
test = "cargo fmt --check && cargo clippy --all-targets -- -D warnings && cargo test"
service = "cd service && bun run test"
```

shipcheck has two [modes](https://shipcheck.run/docs/config/#modes). `init` writes host mode when it finds checks, with act entries for the jobs a host check cannot run as GitHub does. It writes act mode for the whole repository when a job runs a Linux matrix, every job runs on Linux and Docker is running, since host mode tests only this machine's toolchain:

- Host mode runs your commands natively in an OS sandbox, each in a warm copy-on-write clone. It is fast, uses the tools on your machine and needs no Docker.
- act mode, `runner = "act"`, runs your GitHub Actions workflows in containers. It is the closest to GitHub's runners and slower to start. It needs act and a running Docker engine, and so does each `[jobs]` act entry in host mode ([install](https://shipcheck.run/docs/install.md)). A `shipcheck.toml` with no `runner` line runs act mode.

### GitHub: the App

It shows the install link of the [shipcheck App](https://github.com/apps/shipcheck-run), which posts the checks, and opens it in the browser when you apply. Install it on the repository.

Pick your user or organization and the repositories there ([install the App](https://shipcheck.run/docs/install/#install-the-app)). At the end, `init` signs this machine in if it cannot post yet ([sign in each machine](https://shipcheck.run/docs/install/#sign-in-each-machine)).

### Workflows: hosted Actions only on `main`

For each workflow that GitHub runs for feature branches, it shows one line, such as `ci.yml: on PRs shipcheck runs shipcheck/clippy, shipcheck/fmt, shipcheck/test, GitHub keeps running ci.yml on pushes to main, and on fork PRs`, and a diff. The workflow keeps its triggers. Each job a shipcheck check covers gets an `if:` that runs it on GitHub only on your default branch, on the other branches the workflow lists for pushes, on PRs into other branches, for Dependabot, and for a fork's PR, which shipcheck never runs. A job no check covers stays on GitHub with no `if:` added ([which jobs run where](https://shipcheck.run/docs/how-it-works/#which-jobs-run-where)).

- Comments and other triggers stay.
- The recommended choice is yes. init compares the commands, not the names, so a check called `fmt` covers a job called `test` that runs `cargo fmt --check`. The line names the jobs GitHub keeps running on PRs.

### GitHub: branch protection

It shows one check name per job, such as `shipcheck/test`, and GitHub's own check names of the jobs that stay on GitHub. The recommended choice protects the default branch with a ruleset, created with your own `gh` login: no direct pushes, only PRs, and a PR merges once its branch is up to date and each of those checks passes, from the App or from GitHub Actions.

For a job on GitHub whose check names init cannot tell, such as a job whose name or matrix comes from an expression, the ruleset keeps the checks the branch requires now. When `gh` cannot read the branch's rules, init makes no ruleset and leaves them as they are. Install the App before the first PR, since a required check that nobody posts blocks it. When your `gh` login has no admin rights on the repository, init says so and gives the link to set the ruleset up by hand, and the rest still applies. Update the ruleset when a job is added, renamed or removed. [Require the checks](https://shipcheck.run/docs/branch-protection.md) has the manual steps.

### Install

It writes the agent instructions ([agents](https://shipcheck.run/docs/agents.md)). Then, as `shipcheck install` does below, it removes the agent hooks an older version wrote, installs the git hooks for this clone, signs the machine in if it cannot post, and starts the daemon.

### Other cases

- When `shipcheck.toml` already exists, `init` leaves it as it is.
- Without a terminal and without `--yes`, as for an agent or a script that forgot the flag, it prints the plan and writes nothing: no `shipcheck.toml`, no `AGENTS.md`, no hooks, no daemon and no sign-in. Run `shipcheck init --yes` to apply it.
- When every job stays on GitHub, or no check passes when init tries it, it writes no `shipcheck.toml`, says why, and stops: no App, workflow or ruleset step, no agent instructions, hooks or daemon.
- When it finds no check in the workflows or the project files, it writes `runner = "act"` with every workflow in `workflows`, run in containers, and `shard_tests = true`. Remove the workflows that must not run on a laptop, such as deploys. [Modes](https://shipcheck.run/docs/config/#modes) compares the two. With no workflows and none of those project files, it stops with an error.

Add `on = ["push", "commit"]` to start runs on each commit too.

Commit it and merge it to `main`. shipcheck reads this file and the workflows only from `main`, so a PR cannot change what it must pass. Until the file is on `main` there is nothing to run: `shipcheck run` exits 2, and the hooks and the daemon start nothing. [Every setting](https://shipcheck.run/docs/config.md).

Your workflows keep running on GitHub on pushes to `main`, as a backstop after each merge.

### Add the badge

To show that the repository's checks come from shipcheck, add this line to its README:

```markdown
[![checks: shipcheck](https://img.shields.io/badge/checks-shipcheck-1f9d55)](https://shipcheck.run)
```

[![checks: shipcheck](https://img.shields.io/badge/checks-shipcheck-1f9d55)](https://shipcheck.run)

## For each teammate, once per clone

```sh
shipcheck install
```

It installs two hooks, pre-push and post-commit. The post-commit hook starts a run only when `on` has `commit`. It also starts the daemon as a background service, a launchd agent on macOS or a systemd user unit on Linux, and adds this repository to it. Set `SHIPCHECK_NO_DAEMON=1` to skip that on one machine. Git never installs hooks on clone, so each clone needs this one step. To make it automatic, run it from the repository's usual setup: the `prepare` script in `package.json`, a `just setup` recipe, or a `mise` task.

If the repository already uses husky, lefthook or another hook manager, through `core.hooksPath` or a `pre-push` hook of its own, `shipcheck install` leaves its hooks alone and stops with an error that gives the line to add. It then does not sign the machine in or start the daemon, so run it again after you add the line. A `post-commit` hook of its own only gets a note with the line.

When the machine cannot post yet, `install` signs it in: in a terminal it prints a code to enter at github.com/login/device. Without a terminal it prints one line and goes on.

## For an agent machine

1. [Install the binary](https://shipcheck.run/docs/install.md).
2. Run `shipcheck install` in each clone. In a terminal, it signs the machine in if it cannot post yet.

A machine with nobody at its terminal, such as a VM, gets a token instead: run `shipcheck token --name <machine>` where you can open a browser and set the output as `SHIPCHECK_TOKEN` there ([unattended machines](https://shipcheck.run/docs/install/#unattended-machines)).

## Every day

Commit and push as usual. Nothing waits on shipcheck:

- Each commit or push starts a run in the background, as `on` says.
- A newer commit on the same branch stops the older run.
- A tree that was already tested needs no run at all.
- The checks go on PRs, so `shipcheck commit` and `shipcheck post` refuse on the default branch. `shipcheck run` works on any branch.
- With the App installed and the machine signed in, a push is enough. The run it starts posts one check per job on the PR, such as `shipcheck/test`, in progress at once and then green or red, and a `shipcheck` summary of the run.
- When a PR changes on GitHub, such as a rebase with the Update branch button, the daemon reruns its checks ([how it works](https://shipcheck.run/docs/how-it-works/#when-a-pr-changes-on-github)).

To run the checks in the foreground and wait for the verdict, for example as an agent's test step before it pushes:

```sh
shipcheck run
```

When a run fails, give the agent the logs:

```sh
shipcheck logs
```

# Require the check

https://shipcheck.run/docs/branch-protection/

> Make main accept a PR only after its shipcheck check passes.

Once the [shipcheck App](https://github.com/apps/shipcheck-run) posts checks on a repository, tell GitHub to require them ([post the checks](https://shipcheck.run/docs/install/#post-the-checks)). shipcheck posts one check per job, as GitHub Actions does, and those job checks are the required ones. Then a PR can merge only after every job passes, whoever opened it.

## With a ruleset (recommended)

In the repository: Settings, Rules, Rulesets, New ruleset, New branch ruleset.

| Setting | Value |
|---|---|
| Enforcement status | Active |
| Target branches | Include default branch |
| Require status checks to pass | On |
| Status checks | Add each job check, such as `shipcheck/test` and `shipcheck/lint`, and set the source of each to the shipcheck.run App |
| Require branches to be up to date before merging | On |

Each job's check is `shipcheck/<job>`, named by the job id or the `[host.checks]` name. A job id that two required workflows share gets the workflow file's name without `.yml` or `.yaml` in front, `shipcheck/<stem>/<job>`, for those workflows only. When those workflows' stems are not all different, as with `ci.yml` and `ci.yaml`, the whole file name goes in front instead, `shipcheck/ci.yml/test`, so no two jobs share a check name. `shipcheck init` shows the names and, by default, creates this ruleset for you, with Require a pull request before merging turned on and 0 required approvals, so nobody pushes to the default branch directly.

A job check is queued until a machine runs the commit, in progress while it runs, then success or failure.

- Each job's check holds the end of its log and how long it took ([the checks on the PR](https://shipcheck.run/docs/how-it-works/#the-checks-on-the-pr)). `logs = "failed"` in `shipcheck.toml` keeps the log only on failed jobs.
- A job allowed to fail with `continue-on-error` is neutral.
- A change outside `paths` runs nothing, and every job check is skipped.

GitHub counts success, skipped and neutral as passing a required check ([About protected branches](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches#require-status-checks-before-merging)).

The PR also shows a `shipcheck` check, a summary of the whole run with the cache key. It fails when any job does. Do not require it, the job checks already say the same.

Set the source of each check. Without it, any token with write access could post a check named `shipcheck/test` and satisfy the rule, an agent's own token included. With it, only checks from the App count, and the App posts only results shipcheck produced.

Require branches to be up to date. An up-to-date branch already contains `main`, so the tree shipcheck tested is the tree a squash merge puts on `main`.

Rulesets also work for an organization, across many repositories at once: Organization settings, Repository, Rulesets.

## With classic branch protection

Settings, Branches, Add branch protection rule for `main`:

1. Turn on Require status checks to pass before merging.
2. Turn on Require branches to be up to date before merging.
3. Search for each job check, such as `shipcheck/test`, add it, and pick the shipcheck.run App as its source.

Rulesets are GitHub's newer form of the same rule, and the one to use for new setups.

## When jobs change

The required names follow the jobs on `main`. When a job is added, renamed or removed, update the ruleset in the same PR that changes the job, or right after it merges: Settings, Rules, Rulesets, `shipcheck`, then add or remove the names under Status checks.

- A required name that no job posts anymore blocks every PR.
- A new job that is not required can fail without blocking.

`shipcheck doctor` compares the ruleset with the current jobs when your `gh` login can read the branch's rules, and names what to add or remove. It does not read classic branch protection.

## Path filters on main

With GitHub Actions alone, a path filter and a required check do not mix. A workflow that `paths` or `paths-ignore` skips posts no check, so a PR that requires it waits forever ([GitHub's workflow syntax](https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax#onpushpull_requestpull_request_targetpathspaths-ignore)). The usual way out is an extra job that diffs the PR and marks the check as passed.

With shipcheck, the required checks are shipcheck's, and it posts every one on every PR head. Your workflows can then run only on `main`, after each merge, and filter paths freely. To skip the tests and the release on a merge that changes only Markdown:

```yaml
on:
  push:
    branches: [main]
    paths-ignore: ["**/*.md"]
```

## Who may change what runs

shipcheck reads `shipcheck.toml` and the workflows from `main`, never from the PR. A PR that rewrites its own workflow to pass, or empties `workflows`, is still checked with `main`'s version and fails. Its change takes effect only after it merges.

So the risk is a weaker CI that gets merged. Require a human owner's review for those files with GitHub's CODEOWNERS:

```text
# .github/CODEOWNERS
/shipcheck.toml       @your-name
/.github/workflows/   @your-name
```

In the same ruleset, turn on Require a pull request before merging, set Required approvals to 0, and turn on Require review from Code Owners. A PR that touches none of those files then merges with no review, and one that touches them needs a code owner's approval.

To help that review, the check's title ends in ", changes CI" when a PR changes those files, and its summary names each job or step the PR lets fail with `continue-on-error` or turns off with `if: false`, and each job or step it removes.

GitHub does not let anyone approve their own PR, so a code owner's own PR that changes CI waits for another code owner. A sole owner adds themselves to the ruleset's bypass list, then merges such a PR or pushes the change to `main` directly. GitHub records every bypass.

Give agents their own GitHub identity: a bot account or a fine-grained token, without admin, and not a code owner. An agent that pushes with the owner's login is the owner as far as GitHub can tell, and CODEOWNERS and the bypass list then protect nothing.

## Before you turn it on

A required check that nobody posts blocks every PR. Turn the rule on after the App is installed and a first PR has shown the job checks. If a PR's job checks stay queued, the commit's result is missing: run `shipcheck run` on that commit, or check that the hooks are installed in the clone it came from.

## Merge queue

Do not turn on GitHub's merge queue for `main`. The queue requires the check on each group it builds, and shipcheck posts its checks only on PR heads, so every group would wait.

# Agents

https://shipcheck.run/docs/agents/

> How a coding agent learns to run shipcheck instead of the tests and git commit, with any agent.

An agent should run `shipcheck run` instead of the project's tests, and `shipcheck commit -m "<subject>"` instead of `git commit`. Then the run it waits for is the PR's check, and every commit is a tree that already passed. When a run fails, the agent reads `shipcheck logs`, reruns the failed job with `shipcheck run --job <id>`, and finishes with one full `shipcheck run` before it commits. shipcheck works the same with any agent, or with a person. It writes no agent config, hooks or permissions, since it does not know which agent you use or how you set it up.

## Instructions

`shipcheck init` writes a short block between `<!-- shipcheck:start v4 -->` and `<!-- shipcheck:end -->` markers into the files agents read. Commit the files it changes. Running it again replaces only the text between the markers.

| File | When |
|---|---|
| `AGENTS.md` | Always. Most agents read it |
| `CLAUDE.md`, `.claude/CLAUDE.md` | When it exists and does not import `AGENTS.md`: an `@AGENTS.md` line in `CLAUDE.md`, or `@../AGENTS.md` in `.claude/CLAUDE.md` |
| `GEMINI.md` | When it exists |
| `.github/copilot-instructions.md` | When it exists |

If your agent reads another file, point it at `AGENTS.md` in that agent's own settings.

[SKILL.md](https://shipcheck.run/SKILL.md) says how to run shipcheck, read a result and fix a failed check. Put it wherever your agent loads skills, if it does.

## Git hooks

`shipcheck install` installs the `pre-push` and `post-commit` git hooks for the clone. They start a run in the background, so an agent that runs `git commit` instead of `shipcheck commit` still gets its check, after one more run. They never block git.

## Your agent's sandbox

shipcheck keeps its results in `SHIPCHECK_HOME`, a folder your user can write. Nothing in shipcheck stops an agent from writing a fake result there. If your agent has a sandbox, keep it on and keep that folder outside its writable paths. See [limits](https://shipcheck.run/docs/limits.md).

## Upgrading from an older version

Older versions wrote hooks, permission rules and sandbox settings into `.claude/settings.local.json`, `.codex/hooks.json`, `.codex/rules/shipcheck.rules`, `.cursor/hooks.json`, `.gemini/settings.json` and `.github/hooks/shipcheck.json`. Run `shipcheck install` in each clone. It removes only the entries shipcheck wrote, keeps your own settings, and deletes a file only when nothing of yours is left in it. A file your team tracks in git is left alone with a note.

## Check it

`shipcheck doctor` says whether each instruction file has the current block.

# shipcheck.toml

https://shipcheck.run/docs/config/

> Every setting of the one file the repository owner writes.

`shipcheck.toml` lives at the repository root. shipcheck reads it only from the default branch, as [what runs](https://shipcheck.run/docs/how-it-works/#what-runs) says, so a branch cannot change what it must pass. Its settings follow the inputs of a workflow.

Without it on `main` there is nothing to run: `shipcheck run` exits 2 and says to run `shipcheck init`, the hooks and the daemon start nothing, and `shipcheck post` refuses. `--base <ref>` reads it from another ref for a trial, which never posts ([commands](https://shipcheck.run/docs/cli.md)).

`shipcheck init` writes host mode with a check for each workflow job that runs on PRs and passes when init tries it, and a `[jobs]` entry with `runner = "act"` for each job no host check runs as GitHub does, when Docker is running ([which jobs run where](https://shipcheck.run/docs/how-it-works/#which-jobs-run-where)). When one of those runs on Docker, it also writes `network = "internet"` once, at the top. When no job gives one, it uses the checks of a `package.json` `test` or `lint` script, a `Cargo.toml`, a `pyproject.toml` or a `go.mod` at the root. With none of these it writes act mode with every workflow, `network = "internet"` and `shard_tests = true`. It writes `engine = "colima"` only when you move a job to a colima VM on the Checks tab, never by default. It leaves an existing file as it is. It writes `runner`, so the file says which mode it uses, unless every job it places runs with act through `[jobs]`. A host mode file:

```toml
runner = "host"
on = ["push", "commit"]
max_parallel = 4

[host]
versions = ["cargo --version", "rustc --version"]

[host.checks]
test = "cargo test"
clippy = "cargo clippy --all-targets -- -D warnings"
```

An act mode file:

```toml
runner = "act"
workflows = ["ci.yml"]
on = ["push", "commit"]
paths = ["src/**", "Cargo.*"]
timeout_minutes = 30
retries = 1
max_parallel = 4
network = "internet"
shard_tests = true

[act]
platforms = { "ubuntu-latest" = "catthehacker/ubuntu@sha256:<digest>" }
jobs = { "ci.yml/build" = "catthehacker/ubuntu@sha256:<amd64 digest>" }
cache_paths = ["/home/runner/.shipcheck", "/home/runner/.npm", "/home/runner/.bun/install/cache", "/home/runner/.cache"]
slot_paths = ["/var/lib/apt/lists"]

[act.job_homes]
RUSTUP_HOME = "rustup"
CARGO_HOME = { dir = "cargo", path = "bin" }
GOMODCACHE = "gomod"
```

| Setting | Default | Meaning |
|---|---|---|
| `workflows` | none, must be set with `runner = "act"` unless `jobs` is set | Workflow files in `.github/workflows` that must pass. Every job in them runs under act, also with `runner = "host"` when `jobs` is set, except a job a host `jobs` entry takes out. Workflows not listed run only on GitHub, with their own triggers. Without `jobs`, empty with `runner = "host"` |
| `on` | `["push"]` | When a run starts on its own. `push`: the pre-push hook starts one for the tip of each pushed branch, never for a push to the default branch. `commit`: the post-commit hook starts one for each new commit, so the result is often ready before the push |
| `paths` | every file | Glob patterns. When a change touches no matching file since it left `main`, the check passes without running and says so |
| `timeout_minutes` | `30` | A workflow that runs longer fails |
| `retries` | `1` | How many times a failed job runs again, in a new checkout, before it counts as failed. A failed attempt's log stays as `<job>.attempt-<n>.log`, and `shipcheck run` names it |
| `max_parallel` | no cap | The most act jobs and host checks one run starts at once, counted together. A job starts once every job in its `needs` has passed. Without a cap every job starts at once, and many jobs share the CPUs and Docker's memory. Set a number to keep part of the machine free |
| `runner` | `"act"` | `host` runs the `[host]` checks natively in a sandbox. `act` runs the workflows in Docker, with settings under `[act]`. `shipcheck init` writes it, see [modes](#modes). `jobs` sets it for one job |
| `jobs` | none | One job's mode, as `[jobs."<file>/<job>"]` with `runner = "act"` or `"host"`, and `engine`, `"docker"` by default or `"colima"`, for act entries only ([engines](#engines)). An act entry runs that job and the jobs it `needs` under act, beside the host checks, in the same run and under the same `shipcheck` check. A host entry takes that job out of a listed workflow, and the `host.checks` entry of the same id runs in its place |
| `network` | `"none"` | What jobs can reach. `none`: nothing, not this machine, the local network or the internet. `internet`: the internet, from a Docker network of the run's own. Jobs also reach the local network, and on a Mac with Docker Desktop services on the Mac's `127.0.0.1` through `host.docker.internal`, see [the limits](https://shipcheck.run/docs/limits.md). Turn it on only for code you trust as much as your own |
| `shard_tests` | `false` | `true` splits the pytest step of each act job, or the pytest command of each host check, into parallel pytest processes: as many as the Docker engine's CPUs and memory allow at 1.5 GiB each in act mode, or the machine's CPUs in host mode, at most 6. A number fixes the count, from 1 to 32. It never edits your command or lockfile and installs nothing. A sharded run that fails runs again whole, and that result counts. The cost: a test that fails only after another test's leftovers can pass sharded. See [speed up runs](https://shipcheck.run/docs/speed.md) |
| `logs` | `"all"` | Which job checks carry the end of their log. `all`: every job, passed or failed. `failed`: only failed jobs. A failed job always carries its log, and no setting hides it. On a public repository every reader of a PR sees this output. See [the checks on the PR](https://shipcheck.run/docs/how-it-works/#the-checks-on-the-pr) |
| `daemon` | `true` | `false` keeps the daemon away from this repository: `init` and `install` do not add it, and a running daemon skips it |
| `daemon_authors` | none | GitHub logins whose PRs and pushes the daemon on any machine may run when a PR's head has no result. Each daemon runs a head when its own user or a listed login opened the PR and pushed the head. Any other head only gets a stored result posted. Forks never run |
| `host.checks` | none | With `runner = "host"`: each check's name and its shell command, or a `[host.checks.<name>]` table with `run` and `sandbox = false` to run it outside the sandbox, [below](#outside-the-sandbox). A name starts with a letter or `_`, then letters, digits, `_` and `-` |
| `host.versions` | none | Commands whose output goes into the cache key, such as `node --version`, so a new toolchain runs again. A failing one stops the run |
| `host.env` | none | Names of variables a check gets from your environment, beyond the base list in [modes](#modes). Names that carry credentials are refused |
| `act.platforms` | `ubuntu-latest`, `ubuntu-24.04`, `ubuntu-24.04-arm` and `ubuntu-slim` on `catthehacker/ubuntu:runner-24.04`, `ubuntu-22.04` and `ubuntu-22.04-arm` on `catthehacker/ubuntu:runner-22.04`, pinned by digest | Maps a `runs-on` label to an image, pinned by digest. The default images run jobs as `runner` with `sudo`, as GitHub does. An image of your own runs as its own user. `-self-hosted`, which act uses to run a job on the machine itself, is refused here. Only `engine = "colima"` uses it, inside the VM shipcheck starts for the job |
| `act.jobs` | none | An image for one job, as `"<workflow>/<job>"`, when jobs that share a `runs-on` label need different images. For example, an `amd64` image for a job whose tools ship no `arm64` build, on an Apple silicon Mac |
| `act.cache_paths` | the four paths above, plus each detected language's | Paths in the job container kept in a Docker volume per repository, so package caches survive between runs. Parallel jobs share these volumes, so list only caches whose lock lives inside the path. `/home/runner/.shipcheck` holds the per-job homes of `act.job_homes` |
| `act.slot_paths` | apt's lists, plus each detected language's | Paths kept in a Docker volume per repository, run slot and job, for caches that break when two jobs use them at once: apt's lists, Maven, Gradle, sbt, yarn v1 and nub. Jobs in one slot run one at a time. Each slot keeps its own copy, so disk grows with the slots a repository has used. Set `slot_paths = []`, or a shorter list, to drop them, and `shipcheck clean` to remove the volumes |
| `act.job_homes` | `RUSTUP_HOME`, `CARGO_HOME`, `RUNNER_TOOL_CACHE` and `AGENT_TOOLSDIRECTORY`, plus each detected language's: pnpm (`npm_config_store_dir`), Composer (`COMPOSER_CACHE_DIR`), Bundler (`BUNDLE_USER_CACHE`), Hex, Mix and rebar3 (`HEX_HOME`, `MIX_HOME`, `REBAR_CACHE_DIR`) | Gives each job its own directory under `/home/runner/.shipcheck/<job>/` for each environment variable, so parallel jobs never install into one toolchain. `~/.rustup` breaks when two jobs install into it at once, cargo keeps its lock at the root of `~/.cargo`, setup actions such as setup-java or setup-zig copy into act's one tool cache for every job, and the package caches lock badly or not at all across containers. `path` adds a subdirectory to the job's `PATH`. A job that sets the variable itself keeps its value. `job_homes = {}` under `[act]` turns it off |

shipcheck adds a language's cache paths, slot paths and homes only when the tested commit has one of its files: a JavaScript lockfile, `go.sum`, `pom.xml`, `build.gradle`, `build.sbt`, `Gemfile.lock`, `composer.lock`, `mix.lock`, `pubspec.lock` or a `.csproj`, among others. A list you set yourself is used exactly as written, with nothing added.

Write a report to `$GITHUB_STEP_SUMMARY` and it appears in one shipcheck comment on the PR ([the report comment](https://shipcheck.run/docs/how-it-works/#the-report-comment)). No setting turns it on.

## Modes

`runner = "host"` runs commands you declare, natively on this machine in an OS sandbox, with no Docker. A pass means your code passed on this machine's toolchain, operating system and system libraries. When the machine where the agent runs is set up like production, with the same tools, dependencies, OS and system libraries (not the same CPU, memory or scale), a host check tests the code where it will run. That can be more faithful than act or GitHub's hosted runner, whose Ubuntu image is not production. It is also the fastest mode, and the one `shipcheck init` writes when it finds a check, unless the whole repository runs with act. Jobs a host check cannot run as GitHub does then run with act through `[jobs]`, and Windows jobs, macOS jobs off a Mac and jobs that read the PR stay on GitHub ([which jobs run where](https://shipcheck.run/docs/how-it-works/#which-jobs-run-where)).

`runner = "act"` runs the GitHub workflows in Linux containers with `act`, isolated from the machine. It is the mode closest to GitHub's CI: a pass means the workflows as written passed in a container close to GitHub's runner. It needs Docker, and it is the default when `shipcheck.toml` sets no `runner`. `shipcheck init` writes it, with `network = "internet"`, when a job runs a matrix of Linux entries, Docker is running, every job runs on Linux and none runs Docker, since host mode would test only this machine's toolchain. To switch a repository, change `runner` and, for act, list the workflows in `workflows`.

```toml
runner = "host"

[host]
checks = { test = "npm test", lint = "npm run lint" }
versions = ["node --version", "npm --version"]
env = ["NODE_OPTIONS"]
```

- Each check runs with `sh -c` in its own copy-on-write clone of the tested tree, never in your worktree, all at once, or up to `max_parallel` when it is set. A check passes on exit 0, and a run passes when every check does.
- The clone starts from a template warmed from your checkout at most once an hour, so gitignored directories such as `node_modules` and `target/` are already there. This needs a volume that clones files, such as APFS, btrfs or XFS. Elsewhere a check gets a plain worktree and starts cold. Hidden directories are shared only when they hold build caches, such as `.next`, `.turbo`, `.gradle` and `.uv-cache`. A Python `.venv` stores absolute paths, so it is not shared: put `uv sync` in the check, and it rebuilds from uv's cache in seconds. `git cow share <dir>` adds a hidden folder to the list and `git config cow.warmInterval 30m` changes how often the template is warmed, for host checks too.
- Each check runs in an OS sandbox: `sandbox-exec` on macOS, bubblewrap on Linux ([install](https://shipcheck.run/docs/install/#set-up-linux)). Without one, host mode refuses to run, and `shipcheck doctor` says so with the fix. A check writes only its clone, a new empty `HOME` and `TMPDIR`, and the repository's tool caches in shipcheck's folder, one for runs of `main`'s tree and one for other trees. It reads the system and your toolchains, such as `~/.cargo/bin`, `~/.rustup/toolchains`, `~/.nvm/versions`, `~/.local/share/uv/python`, avm's `~/.avm/bin` and Solana's `~/.cache/solana`, and nothing else in your home. Toolchains found only through `$HOME`, Docker's CLI plugins and Solana's platform-tools, are linked into the new `HOME`. It also gets its own rustup home of links and its own copy of Solana's SBF SDK, since `anchor build` relinks both on every build. It can serve and connect to Unix sockets in its clone, `TMPDIR` and caches, such as a test server on a socket in `TMPDIR`, and cannot reach Docker's socket or any other outside them, or start a process outside the sandbox ([security](https://shipcheck.run/docs/security.md)).
- shipcheck points `CARGO_HOME`, `npm_config_cache`, `BUN_INSTALL_CACHE_DIR`, `UV_CACHE_DIR`, `XDG_CACHE_HOME`, `GOPATH` and `GOCACHE` into the tool caches, so the first run downloads dependencies once. `RUSTUP_HOME` and `AVM_HOME` point at your installed toolchains, which a check can read but not change. Your `~/.cargo/config.toml`, `~/.npmrc` and other config in your home do not apply.
- A check gets `PATH`, `USER`, `LOGNAME`, `LANG`, `LC_ALL`, `SHELL` and `TERM` from your environment, the names in `host.env`, and `CI=true`. `PATH` loses every entry in a temp folder or inside a git checkout, except Homebrew's, so a tool an agent wrote there never runs in place of the real one. shipcheck itself starts every process, git, act and bubblewrap included, with this `PATH`. Nothing else, so tokens, `SSH_AUTH_SOCK`, `AWS_*` and the like stay out. `host.env` refuses `SSH_AUTH_SOCK`, `GIT_*`, `SHIPCHECK_*`, `AWS_*`, `GOOGLE_*`, `AZURE_*`, any name with `TOKEN`, `SECRET`, `PASSWORD` or `CREDENTIAL`, and names ending in `_KEY`.
- A check has the network this machine has, the internet, the LAN and local TCP services. A timeout kills the check's whole process group.
- The cache key holds the tree, the checks and the output of `versions`, each of which may take 5 s. Pin files in the tree, such as `rust-toolchain.toml` or `.nvmrc`, are in the key through the tree. In both modes the `shipcheck` check names a branch's changes to files that can change what a command does: `package.json` scripts, `Makefile`, `justfile`, `pyproject.toml` tool settings, `Cargo.toml` outside its dependencies, and `.cargo/config.toml`.
- A toolchain in your home outside the sandbox's list cannot be read, so its check fails. Install it under `/opt/homebrew`, `/usr/local` or a listed path, or set its variable in `host.env` to a place outside your home, for example `env = ["RUSTUP_HOME"]`.
- After an hourly warm from your checkout, the next run rebuilds what embeds your checkout's path, such as cargo's workspace crates. Later runs reuse the run's own build.
- Commands come only from `shipcheck.toml` on `main`, never from the CLI or the branch.

### Outside the sandbox

Some checks need a system service the sandbox denies, such as the iOS simulator. A check written as a table with `sandbox = false` on `main` can run without the sandbox:

```toml
[host.checks.ios]
run = "xcodebuild test -scheme App -destination 'platform=iOS Simulator,name=iPhone 17'"
sandbox = false
```

It runs outside the sandbox only on a machine where you ran `shipcheck trust` in the repository. Until then it runs in the sandbox, and every run and `shipcheck doctor` say `this repository asks to run it with sandbox = false`, with the command. `shipcheck trust --remove` takes it back. Trust follows the repository's origin URL, and the daemon and the hooks see it too.

**Warning:** that check runs any branch's code with your full access: your home, your SSH keys, your `gh` token and every other secret your user can read. Use it only on a repository where you trust every author the daemon runs PRs for. The check still gets its own clone, a new `HOME` and `TMPDIR`, the tool caches and the filtered environment, and `shipcheck doctor` and every run name it. A PR cannot turn it on, since `shipcheck.toml` comes from `main`, and a repository you never trusted cannot turn it on for you.

A `jobs` entry runs one job under act beside the host checks, such as tests that need Docker:

```toml
runner = "host"

[host.checks]
test = "cargo test"

[jobs."ci.yml/e2e"]
runner = "act"
```

`network` and `[act]` apply only when some job runs under act, and setting them when none does is an error. `shard_tests` applies in both modes.

Host mode runs the branch's code as you, in the sandbox, with your network. Use it only for your own and your agents' work. When the config has host checks, the daemon never checks a fork's PR, not even with a stored result. `shipcheck init` writes host checks from the workflows' run steps, or else the project files it finds, and prints where each came from. Check them before you commit the file.

To get the digest of one architecture of an image:

```sh
docker manifest inspect catthehacker/ubuntu:runner-24.04
```

Pull an `amd64` image once before the first run (`docker pull --platform linux/amd64 <image@digest>`). Docker runs it under emulation on Apple silicon, which is slower, so keep it to the jobs that need it.

A commit whose message holds `[skip ci]`, `[ci skip]` or `[skip shipcheck]` starts no run, and neither does a commit or push made with `SHIPCHECK_SKIP=1`. A skip never posts a result, so a PR that requires the job checks cannot merge until a later commit runs. `shipcheck run` ignores skips. Use `paths` for changes that should pass without running.

Runs started by `on` never block a commit or a push. They run in the background, and a failure turns the PR's job checks red. A newer commit on the same branch stops the older run first, like `cancel-in-progress` in a workflow.

shipcheck stops with an error when:

- `workflows` is empty under act or set under host, both without `jobs`, or names a file that does not exist or names one twice
- a `jobs` key is not `"<a workflow file>/<job>"` or names no job of that file, a host entry's file is not in `workflows`, a host entry has no `host.checks` entry of its id, host entries take every job out of a workflow, an act job needs a job a host entry moves, or a host entry sets `engine`
- `max_parallel` is 0
- an `act.jobs` key is not `"<a file in workflows>/<job>"`
- a `paths` pattern is invalid
- an image is not pinned by digest
- a cache or slot path is not an absolute path made of letters, digits and `._/-`
- a `job_homes` entry is not a variable name with relative paths
- a `shard_tests` number is outside 1 to 32
- host mode has no checks, a check name breaks the rule above, a command is empty, or a `host.env` name is refused
- `[host]` is set under act without `jobs`
- a setting is unknown, so a typo cannot pass unnoticed

## Engines

`engine` in a `[jobs]` act entry picks what the job runs on:

```toml
[jobs."ci.yml/e2e"]
runner = "act"
engine = "colima"
```

- `docker`, the default: a container on the current Docker context, with no Docker socket. It needs act and a running Docker engine ([install](https://shipcheck.run/docs/install.md)).
- `colima`: a Colima VM of its own, with its own Docker and Linux kernel, for a job that runs Docker or needs a kernel feature a container blocks. It needs colima and lima installed, and also act and a running Docker engine on this machine, since shipcheck checks Docker before every act run and runs the jobs a colima job needs there. The VM runs its own Docker, started with `--runtime docker`, and its own act. Each `SHIPCHECK_HOME` keeps one stopped base VM, `sc-base-<hash>`, set up once with the VM's packages and act. Each job gets a copy-on-write clone of that base, so it boots already set up, and shipcheck deletes the VM, with the data disk colima leaves, once the job's result is back. A new colima or lima version, or a new setup, makes a new base and deletes the old one. shipcheck records each VM it starts. While no colima job runs, a run's cleanup and `shipcheck clean` delete the `sc-` VMs that this home recorded and a dead run left, and `shipcheck clean --all` deletes the base too. A VM another home or tool made is never touched.

How it was tested: shipcheck's own repository runs its `ci.yml/test` job with `engine = "colima"`, so the `shipcheck/test` check of every shipcheck PR comes from a colima VM. Its end-to-end tests run real VMs on a Mac with colima, and passed on 2026-10-30. They check that a job runs in its own VM and leaves no VM behind, that the base VM stays for the next job, that a port the job listens on in the VM does not show up on the Mac's `127.0.0.1`, that a run killed while its VM runs still gets that VM deleted, that other colima profiles are left alone, that a failing job and a job past its timeout both fail and leave no VM, and that a job its `if:` skips boots no VM. [Speed](https://shipcheck.run/docs/benchmarks/#colima) has its times.

A colima job has these limits:

- Use it only for code you trust. The VM reaches this machine's `127.0.0.1` through 192.168.5.2, and the internet, whatever `network` says. That includes the artifact servers of Docker runs going at the same time. act runs beside the job's code in the VM, so the job can change its own result. `shipcheck doctor` says so.
- Secrets reach act once, through a FIFO in the VM that is removed after act reads it, so they never stay on the VM's disk. Steps have `sudo` in the VM, and root can still read act's memory.
- One VM per 16 GB of this machine's memory at a time, since each VM takes 8 GB, and the CPUs are split between them. A colima job past that waits for a VM and holds its job slot while it waits. The wait counts toward its `timeout_minutes`, so at the deadline it fails as timed out, and its log says it waited for another colima job's VM.
- Linux labels only. Labels with an image in `[act] platforms`, plus `self-hosted` and `linux`, run on the VM. Any other label, such as `macos-latest`, fails as it does on Docker.
- A colima job cannot pass artifacts to or from other jobs. shipcheck refuses one that uses `actions/upload-artifact` or `actions/download-artifact`, so it fails before it runs.
- Each job boots a VM, and no package or build cache crosses VMs, so a colima job is slower than a Docker job. shipcheck fetches the job's actions on this machine, with act, and copies them into the VM, so a job never changes what a later one runs. The cache key holds the colima and lima versions and a hash of the VM's setup. It does not hold the VM image's digest, which colima does not report, so a new image under the same colima version reuses old results. `shipcheck doctor` says so.
- A job that a colima job `needs` runs on Docker, so Docker must run too.

# Speed

https://shipcheck.run/docs/benchmarks/

> How long shipcheck took against GitHub's hosted runners on the same commits, the slower runs included.

Each row compares shipcheck on an Apple silicon Mac with GitHub's hosted runners on the same commit, unless the notes say otherwise. GitHub's time runs from the run's creation to its end, so it includes the queue. To make your own runs faster, see [speed up runs](https://shipcheck.run/docs/speed.md).

## What makes a run fast

- A tree that was already tested answers from shipcheck's cache with no run, in 0.13 to 1.05 s in the runs on this page. A rebase or a squash that keeps the tree needs no new run.
- After the first run, the toolchains, the packages and the build output, such as `target/`, are already on the machine, so a warm run builds only what changed.
- GitHub gives every job a machine of its own. shipcheck runs as many jobs at once as the machine allows, so a workflow of many short jobs can take longer here.

A first run is cold. It downloads the images, the toolchains and the packages, and builds every dependency, so it is often slower than GitHub.

## Act mode

Measured on an idle Apple silicon Mac, Docker Desktop with 14 CPUs and 15.6 GiB, act 0.2.89.

| Repository | Run | shipcheck | GitHub Actions | Date |
|---|---|---|---|---|
| A large Rust app, 4 jobs, about 2,100 tests | Cold | 238 s | 116 s | 2026-10-23 |
| A large Rust app, 4 jobs, about 2,100 tests | Warm | 105 s | 124 s | 2026-10-23 |
| A large Rust app, 4 jobs, about 2,100 tests | Warm | 105 s | 115 s | 2026-10-23 |
| pallets/flask, 9 jobs | Cold | 131 s | about 37 s | 2026-10-23 |
| pallets/flask, 9 jobs | Warm | 38 and 39 s | about 37 s | 2026-10-23 |

The Rust app's gitleaks job failed under act in every round, so its times are to that result. A warm run compiles only the app's own crate. The cold run also installs the toolchain and builds every dependency.

## Host mode

On 2026-10-27, `shipcheck init` wrote host checks for some of the jobs of each repository below, and the rest stayed on GitHub. One repository ran at a time, with a load of 4 to 6 on 16 cores before each run and no other shipcheck run or build. Docker was down, so init kept every Docker job on GitHub. The GitHub column is the longest of those jobs, as the median over the last 10 successful PR heads.

| Repository | Jobs shipcheck ran | Cold | Warm | Cached | Those jobs on GitHub |
|---|---|---|---|---|---|
| shipcheck | 2 of 3 | 134 s | 70 s | 0.36 s | 172 s |
| BurntSushi/ripgrep | 4 of 23 | 30 s | 23 s | 0.48 s | 49.5 s |
| pallets/flask | 1 of 14 | failed every run | failed every run | | 18 s |
| cli/cli | 1 of 10 | 16 s | 15 s | 0.36 s | 31.5 s |
| curl/curl | 1 of 178 | 10 s | 9.7 s | 1.05 s | 20 s |

- flask's pre-commit check failed in the sandbox on a path its cache stored from an earlier run. shipcheck PR 122 lets those runs read it.
- shipcheck's `ci.yml` no longer ran on PRs, so its GitHub time is from push runs on main.

Host mode runs on your machine's OS. On the idle Mac of the act mode table, on 2026-10-23, the large Rust app's tests took 105 s on macOS against 16 s in act's Linux container.

## Colima

The job is shipcheck's own `ci.yml/test`, which runs the fmt check, clippy and `cargo test`, with `engine = "colima"`, run alone with `shipcheck run --job test`. Each version ran in its own empty `SHIPCHECK_HOME`, on the same Apple silicon Mac. These colima runs were measured on 2026-10-30 with the Mac's load between 2.6 and 4.1, so they are a first look. The time is the job's runner time.

| Version | Run | shipcheck | Load at start | Date |
|---|---|---|---|---|
| v0.2.1 | Cold | 179 s | 4.0 | 2026-10-30 |
| v0.2.1 | Warm | 190 s | 2.6 | 2026-10-30 |
| v0.2.4 | Cold, builds the base VM once | 134 s | 2.6 | 2026-10-30 |
| v0.2.4 | Warm, clones the base VM | 85 s | 4.1 | 2026-10-30 |

The same job on GitHub Actions, on `ubuntu-latest` with rust-cache, took 86 s of runner time, from 13:25:18 to 13:26:44 UTC (run 40881747310 on main, 2026-10-29). v0.2.4 boots each job's VM from a clone of a set-up base, fetches the actions on the Mac, and deletes the VM after the result, which is where the difference comes from. One more v0.2.1 warm run took 493 s while something else ran, and it is left out.

## First look, measured on a busy machine

Other benchmarks or agents ran on the Mac during these runs, or the load was not recorded. The shipcheck times are rough.

### Host mode

| Repository | Run | shipcheck | GitHub Actions | Date |
|---|---|---|---|---|
| A small Rust crate, fmt, clippy and tests | First run, in the sandbox | 4.9 s | about 20 s | 2026-10-22 |
| A small Rust crate, fmt, clippy and tests | Warm, in the sandbox | 1.9 to 2.3 s | about 20 s | 2026-10-22 |
| expressjs/express, lint and 1,261 tests | Warm, in the sandbox | 3.4 to 6.8 s, one round failed | about 3.5 min | 2026-10-22 |
| Another large Rust app, 5 checks | Warm | 422 s | 413 s | 2026-10-23 |
| Another large Rust app, 5 checks | Warm, after a rebase onto main | 498 s | 413 s | 2026-10-23 |

- The small crate's GitHub time is its Linux leg.
- express's tests were flaky under the load, also outside shipcheck.
- The second Rust app's GitHub time is one run of its CI on main.

### Act mode

| Repository | Run | shipcheck | GitHub Actions | Date |
|---|---|---|---|---|
| A large Python app, 1 job, about 2,000 tests | Cold | 716 s, the retry passed | 367 s | 2026-10-21 |
| A large Python app, 1 job, about 2,000 tests | Warm | 468 s | 316 s | 2026-10-21 |
| A large Python app, 1 job, about 2,000 tests | Warm, after setup fixes | 274 to 341 s | 331 to 411 s | 2026-10-22 |
| A large Python app, 1 job, about 2,000 tests | Warm, tests in 6 shards | 162 and 195 s | 345 and 337 s | 2026-10-22 |
| A small Rust crate, fmt, clippy and tests | Cold | 30 s | 38 s | 2026-10-22 |
| A small Rust crate, fmt, clippy and tests | Warm | 7 s | 21 and 33 s | 2026-10-22 |
| pallets/flask, 9 jobs | Cold | 289 s | 36 s | 2026-10-22 |
| pallets/flask, 9 jobs | Warm | 63 and 64 s | 36 s | 2026-10-22 |
| BurntSushi/ripgrep, 8 jobs | Cold | 284 s | 78 s | 2026-10-22 |
| BurntSushi/ripgrep, 8 jobs | Warm | 73 and 84 s | 78 s | 2026-10-22 |
| junegunn/fzf, 1 job | Cold | 462 s | 378 s | 2026-10-22 |
| junegunn/fzf, 1 job | Warm | 381 and 385 s | 378 s | 2026-10-22 |
| expressjs/express, 10 jobs | Cold | 220 s | 148 s, 96 s of it queued | 2026-10-22 |
| colinhacks/zod, 5 jobs | Cold | 651 s | 177 s | 2026-10-22 |
| colinhacks/zod, 5 jobs | Warm | 588 s, one more round failed | 177 s | 2026-10-22 |
| faker-ruby/faker, 6 jobs | Cold | 106 s | 29 s | 2026-10-22 |
| faker-ruby/faker, 6 jobs | Warm | 67 and 68 s | 29 s | 2026-10-22 |

- The Python app lost on pytest. Four tests call an outside API many times, and those calls take longer from the Mac than from GitHub. Sharding split the tests over 6 processes.
- flask and faker finish on GitHub in 30 to 40 s, and starting containers costs more than their tests. On a later version, flask took 38 and 39 s warm in the idle run above.
- zod's package install took 140 s or more on the macOS bind mount, against 12 s on GitHub.
- express's warm runs failed on a port clash between jobs that shared one network.
- The public repositories' GitHub times come from a recent run on their default branch. shipcheck ran the same commit with only the Linux jobs.

### On a 1-CPU Linux server

A rented server with 1 vCPU and 3.9 GB of memory, serving a small web app at the same time. It ran one job at a time.

| Repository | Cold | Warm | GitHub Actions | Date |
|---|---|---|---|---|
| A small Rust crate, 1 job | 103 s | 19 and 20 s | 23 s | 2026-10-22 |
| gin-gonic/gin, 11 jobs | 2,579 s | 513 and 519 s | 105 s | 2026-10-22 |
| expressjs/express, 10 jobs | 1,235 s | 318 and 321 s | 148 s, 96 s of it queued | 2026-10-22 |

A job took about as long there as on GitHub's runner. GitHub runs 10 or 11 jobs at once, so a warm run there took about as long as the sum of GitHub's job times.

### A tree already tested

| Repository | Mode | shipcheck | GitHub Actions | Date |
|---|---|---|---|---|
| A large Python app | Act | 0.13 s | a full run | 2026-10-22 |
| A small Rust crate and expressjs/express | Host | 0.26 s | a full run | 2026-10-22 |

# Speed up runs

https://shipcheck.run/docs/speed/

> Use host mode, shard pytest, let jobs download, keep caches, and pick a Docker engine.

[Speed](https://shipcheck.run/docs/benchmarks.md) has cold and warm times against GitHub's hosted runners, with the runs where GitHub was faster. The settings below go in [`shipcheck.toml`](https://shipcheck.run/docs/config.md) on `main`.

## Use host mode

`runner = "host"` runs the checks natively, with no container to start.

- Each check's clone starts from a template warmed from your checkout, so a built `target/` or `node_modules` is already there. Keep your checkout built.
- After each hourly warm, the next run rebuilds what embeds your checkout's path, such as cargo's workspace crates. Later runs reuse the run's own build.
- A Python `.venv` is not shared. Put `uv sync` in the check, and it rebuilds from uv's cache in seconds.
- The tool caches start empty, so the first run downloads dependencies once.
- Host mode runs on this machine's OS. A large Rust app's tests took 105 s on macOS against 16 s in act's Linux container, so a suite that is slow on your OS can be faster in act mode.

The sections below are for act mode.

## Shard pytest

```toml
shard_tests = true
```

shipcheck splits the pytest step of each act job into parallel pytest processes in the job's container, as many as the Docker engine's CPUs and memory allow at 1.5 GiB each, at most 6. In host mode it splits the pytest command of each check the same way, as many as the machine's CPUs, at most 6. `shard_tests = 4` fixes the count. Each process collects the whole suite and runs every N-th test. shipcheck checks that the shards together ran every collected test once and all passed. Anything else, a failing test included, runs the job again whole, and that result counts. Your command, lockfile and dependencies stay as they are, and nothing is installed.

The step must be one line in bash, such as `pytest -v`, `uv run pytest`, `poetry run pytest` or `python -m pytest`, with no `;`, `&&`, pipes, redirects, `$` or pytest-xdist flags. A host check can run other commands around it with `&&`, such as `uv sync && uv run pytest -v`, and shipcheck splits only the pytest part. A check with two pytest commands is not split. `shipcheck doctor` names the workflows and host checks that have one. Several agents at once each run their own shards, so on a busy machine a fixed `shard_tests = 3` keeps the total lower.

The cost: a test that fails only after another test's leftovers can pass sharded, since the two may run in different processes. GitHub's serial run on `main` still catches it. `shipcheck init` turns sharding on when it writes act mode. Set `shard_tests = false` if a pass must mean the suite passed in order.

## Give Docker more memory

Sharding runs out of memory before CPUs. shipcheck counts 1.5 GiB for each pytest process, so Docker Desktop's default 8 GB VM fits 5 shards. Parallel jobs share the same memory: every job starts at once unless `max_parallel` caps them, and Rust or JVM builds side by side can swap or be killed for lack of memory. Raise it in Docker Desktop, Settings, Resources, Memory, such as 16 GB on a Mac with 32 GB or more. More memory does not speed up one serial job. `shipcheck doctor` says when Docker has under 8 GB or under a third of the Mac's memory.

## Let jobs download

```toml
network = "internet"
```

With the default `none`, a job reaches nothing, so steps that install packages or toolchains fail. With `internet` they work. A job then reaches the local network, and on a Mac with Docker Desktop also services on the Mac's `127.0.0.1` through `host.docker.internal`, see [the limits](https://shipcheck.run/docs/limits.md). Turn it on only for code you trust as much as your own.

## Keep caches between runs

`act.cache_paths` keeps paths in the job container in a Docker volume per repository: `/home/runner/.npm`, bun's cache, `/home/runner/.cache`, which holds uv and pip, and the per-job tool homes in `/home/runner/.shipcheck`. Add the cache of another tool if its lock lives inside the path. Each job's checkout is also a copy-on-write clone of the last one, build output included, so `target/` or `node_modules` start warm.

## Pick a Docker engine

Use Docker Desktop, the engine the act mode trials and benchmarks ran on. OrbStack and colima also ran act jobs, with limits: OrbStack sometimes showed a job the files of the previous run at the same path, which shipcheck now fails, and colima shares only your home directory, so the repository and `SHIPCHECK_HOME` must be under it. See [install](https://shipcheck.run/docs/install.md). This is colima as the Docker engine, not `engine = "colima"`, whose tests and times are in [engines](https://shipcheck.run/docs/config/#engines) and [speed](https://shipcheck.run/docs/benchmarks/#colima). Give the engine's VM enough CPUs and memory for `max_parallel` jobs at once, and for the shards of each.

# Commands

https://shipcheck.run/docs/cli/

> The commands, their exit codes and their environment variables.

| Command | Does |
|---|---|
| `shipcheck run [--sha <rev> \| --staged] [--no-fetch]` | Runs the checks in `shipcheck.toml`, host checks or workflows, and waits for the verdict. With no `--sha`, it tests the working copy: tracked edits and untracked files that are not gitignored, which it lists. `--staged` tests the index. A working copy with no change is `HEAD`. `--sha` tests that commit only. Fetches the remote's default branch first unless `--no-fetch`. A tree that already passed answers at once. A change outside `paths` runs nothing and prints `nothing ran`, and that skip is never stored. For each failed job it prints the failed step and its exit code, the failing tests by name, the last 60 lines of the step and the path of the full log. A failed host check gets its tests and the last 60 lines of its output. `--json` prints one object instead, see [JSON](#json) |
| `shipcheck run --job <id> \| --workflow <file>` | A partial run, for the loop after a failure: runs only those jobs, or every job of those workflows, and the jobs they `needs`. Both flags repeat. A matrix job's id runs all its legs. `--job <file>/<job>` picks the job of one workflow. In host mode `--job` names a `[host]` check and runs only it, and `--workflow` is an error. When a host check and an act job share a name, a bare name picks the host check. The result is never stored, so it is never the check: `status` ignores it, and `shipcheck commit` refuses after it, even with `--allow-failed`. A `--job` that names no job lists each one it takes, with its file and matrix legs, such as `test (ci.yml/test, legs test__1, test__2)`. Exit codes as usual. Fix, run the failed job as often as needed, then one full `shipcheck run` and `shipcheck commit` |
| `shipcheck commit -m <message> \| -F <file> [--allow-failed] [--no-verify]` | Commits exactly the tree the last `shipcheck run` of this working copy tested, through `git commit`, so hooks and signing work as usual. Refuses when a file changed since the run, when the last run was partial, and when the run failed unless `--allow-failed`. `--no-verify` goes to `git commit`. The commit's check is then that run's result. Refuses with exit 2 on the default branch, since PRs carry the checks: `git switch -c <branch>` first. Plain `git commit` still works there |
| `shipcheck doctor` | Checks git, act, the Docker engine and its versions, or in host mode the OS sandbox, copy-on-write, the count and size of shipcheck's Docker volumes and images, the remote's default branch and its `shipcheck.toml`, the network setting, colima when a job uses it, the hooks, whether `shipcheck post` can post, and where shipcheck's secrets are, reading each so that after an upgrade the keychain asks here. Marks each line `ok` or `fix`, a green check or a red cross in a terminal, with the command that fixes it. Exits 0 when a run can work, 1 otherwise |
| `shipcheck status [<rev> \| --staged] [--json]` | Prints the stored result for the tree `run` would test. With no revision it first prints whether the working copy's tree and the index's tree have a result, so you know before a commit whether it reuses one. With no result it exits 2, and with `--json` it prints `{"tree": "<id>", "result": null}`. Without `--base`, a tree that has no result under `main`'s `shipcheck.toml` shows its last run's, such as one with `--base <branch>`, and says which base it read |
| `shipcheck logs [<rev> \| --staged] [--base <ref>]` | Prints the full logs of that tree, one per check or workflow, then the last partial run's, with control characters escaped. Without `--base` it finds the last run of the tree as `status` does |
| `shipcheck history [--json] [--limit <n>]` | Prints the last runs of this repository on this machine, 20 by default: when, commit, mode, result, time, whether a stored pass answered, and what started the run. `--json` prints the records as one array. See [Run records](#run-records) |
| `shipcheck post [<rev>] [--home-ok]` | Posts the stored result of the commit's tree on GitHub as one completed check per job, `shipcheck/<job>`, and a `shipcheck` summary. A job id two workflows share gets the workflow in its name, as [require the checks](https://shipcheck.run/docs/branch-protection/#with-a-ruleset-recommended) says. The job checks are the ones to require. A job `main` has that the result lacks posts as a failure. Posts only a result read from the daemon's base for this clone or from the default branch the remote reports, so a trial with `--base` never posts. When the remote does not answer, only the daemon's base posts. Refuses when `main` has no `shipcheck.toml`, and when the commit is the tip of `main`, locally or on `origin`, since PRs carry the checks. Each job check carries its log tail as `logs` in `shipcheck.toml` says, every job by default and always a failed one, and anyone who can read the repository sees it. Each check shows how long its job took. Refuses when `SHIPCHECK_HOME`, `HOME` or `XDG_STATE_HOME` moves the results away from the default, since an agent could have written them there. Pass `--home-ok` if you set it yourself. The daemon takes `--home-ok` too |
| `shipcheck login` | Signs this machine in to shipcheck's service with GitHub, so `shipcheck post`, the pushes' runs and the daemon post the checks. Posting needs the shipcheck App installed on the repository and write, maintain or admin access for you. Prints a code to enter at github.com/login/device, and in a terminal Enter opens that page in the browser. Then it prints `signed in as <login>`. The machine token stops working after 90 days without a post, then run it again. On macOS it goes in the login keychain, see [where shipcheck keeps its keys](https://shipcheck.run/docs/install/#where-shipcheck-keeps-its-keys) |
| `shipcheck logout [--all]` | Revokes this machine's token and deletes it. With `SHIPCHECK_TOKEN` set, revokes that token instead and keeps the stored login token. `--all` revokes every token of your GitHub user, on every machine, and prints how many. When this machine's token is no longer valid, `--all` revokes nothing and fails, so sign in again first |
| `shipcheck token [--name <machine>]` | Makes a machine token for another machine to use as `SHIPCHECK_TOKEN`. Prints a code to enter at github.com/login/device on stderr, as `login` does, then only the token on stdout. Leaves this machine's sign-in alone. `--name` labels it, `token` by default. Each user can hold 50 live tokens |
| `shipcheck init [--yes]` | For the repository owner: turns each workflow job it can into a host check and each job act can run as GitHub does into a `[jobs]` act entry, tries each host check in the sandbox, writes `shipcheck.toml` with the ones that pass, writes the agent instructions into `AGENTS.md` and the other files agents read, and runs `install`. On a terminal its steps are tabs, Checks, Install, Workflows, GitHub and Review, which also cover where each job runs, the App, hosted Actions only on `main`, and a ruleset that protects the default branch. It changes nothing until you apply on the Review tab. `--yes` applies the recommended choices without asking, but opens no browser and creates no ruleset. Without a terminal and without `--yes` it only previews: it prints the plan `--yes` would apply and writes nothing, so an agent or a script passes `--yes`. It leaves an existing `shipcheck.toml` as it is. When the remote does not say its default branch, it refuses and says to run it again online. See [the quickstart](https://shipcheck.run/docs/quickstart.md) |
| `shipcheck install` | For each clone: installs the pre-push and post-commit hooks, removes the agent hooks an older version wrote, and starts the daemon service for this repository, with the default branch the remote reports as its base. When the remote does not answer, it adds no repository to the daemon and says to run it again online. Running it again changes nothing, and an error with the service is one line that never fails the install. In a terminal, it signs the machine in when it cannot post yet. Without one it prints how to. See [Agents](https://shipcheck.run/docs/agents.md) |
| `shipcheck daemon [--home-ok \| --uninstall]` | `init` and `install` start it as a background service, a launchd agent on macOS or a systemd user unit on Linux, and add the repository to `daemon.toml` in `SHIPCHECK_HOME`. It checks the open PRs into the configured branch within seconds of each push to a PR, from the App's webhook through shipcheck's service, and every `interval_seconds` (10 by default) while it cannot hear those events or a head is still waiting. A head with no completed `shipcheck` check gets its stored result at once, or, with no result, a run that posts in progress and then the result, as a push does. It runs only PRs you, or a login `daemon_authors` lists, both opened and pushed the head of, never a head that contains one it refused, and never a fork's. It posts with the `shipcheck login` token, and runs no PR until the machine is signed in. `--uninstall` stops and removes the service, but only from the default `SHIPCHECK_HOME`, since the service belongs to it |
| `shipcheck trust [--remove]` | Lets this repository's host checks with `sandbox = false` on `main` run outside the sandbox on this machine, with your full access. Without it they run in the sandbox. `--remove` takes it back. It records the origin URL in `trusted` in `SHIPCHECK_HOME` ([outside the sandbox](https://shipcheck.run/docs/config/#outside-the-sandbox)) |
| `shipcheck clean [--all]` | Removes what shipcheck keeps for this repository: its Docker cache volumes, warm checkouts and run slots, host mode's tool caches in `hostcache/` in `SHIPCHECK_HOME`, and the colima VMs this home recorded and a dead run left. `--all` does it for every repository, also deletes the colima base VM, removes stored results, logs and run records, and removes the daemon service when `SHIPCHECK_HOME` is the default. It never removes a Docker image. A volume that a container uses, running or stopped, stays, and so does a tool cache a run holds. Everything rebuilds on the next run, which is slower once. Refuses while a run is going |

The git hooks call `shipcheck on-push` and `shipcheck on-commit`. Those start a background run when `on` asks for one, never fail, and never make git wait. On a machine that can post, a push's run posts the checks. They read `shipcheck.toml` from the base `install` stored for the clone, else `origin/main`, and never ask the remote, so git does not wait on the network. With no stored base on a repository whose default branch is not `main`, they print one line to stderr that says to run `shipcheck install`. You do not call them yourself.

`--base <ref>` reads `shipcheck.toml` and the workflows from another ref than the default branch, and turns off the fetch. A result from another ref is a trial, which `post` refuses. It is for trying shipcheck on a repository before the file is on `main`.

## JSON

`--json` prints one object on stdout and the progress on stderr. Exit codes stay the same.

```json
{
  "tree": "4b825dc6...", "sha": "9f2c1e0a...", "partial": false, "passed": false, "could_not_run": false, "skipped": null, "ci_changes": [],
  "workflows": [{
    "file": "ci.yml", "passed": false, "exit": 1, "attempts": 1, "log": ".../logs/<key>/ci.yml.log",
    "jobs": [{
      "id": "test", "status": "failure", "attempts": 1, "shards": null,
      "failed_step": "Run pytest", "exit": 1, "note": null,
      "failing_tests": ["tests/test_math.py::test_add"], "failing_tests_total": 1,
      "tail": ["...", "FAILED tests/test_math.py::test_add - assert 3 == 4"],
      "log": ".../logs/<key>/ci.yml/test.log", "attempt_logs": []
    }]
  }]
}
```

Test names come from pytest, cargo test, nextest, go test, jest, vitest, rspec and phpunit output, at most 30. `sha` is `working copy` or `index` when the run tested one of those. `partial` is true for a `--job` or `--workflow` run. In host mode, one workflow named `host` holds a job per check, with no `failed_step` or `exit`, and `tail` is the end of the check's output. `failing_tests_total` counts them all. `tail` holds at most 60 lines, with control characters escaped. `note` explains a failure that is not the code's, such as a stale checkout or a job with no image on this host.

## Run records

Each run adds one line to `records/<owner>/<repo>.jsonl` in `SHIPCHECK_HOME`: when it ran, the commit and tree, the branch, what started it, the machine name and OS, the shipcheck and tool versions, whether a stored pass answered, and for each job its status, attempts, times and failing test names. The daemon also adds the PR number and author, whether it posted the checks and the comment, when a PR it checked gets a new head, and when one is merged and by whom. A record holds no code, file contents, log output, environment values or secrets.

`shipcheck history` reads them from this file.

Records are not deleted with old results and logs. A file past 50 MB moves to `<repo>.<n>.jsonl`. To delete them, remove the `records` folder, or run `shipcheck clean --all`, which also removes results, logs and warm checkouts.

## Exit codes

| Code | Meaning |
|---|---|
| 0 | Every check passed, or nothing under `paths` changed |
| 1 | A check failed, or `doctor` found something to fix. The output names the failed step and tests, `shipcheck logs` prints every log |
| 2 | shipcheck could not run, for example `shipcheck.toml` is missing on `main` or invalid, the OS sandbox is missing in host mode, or act is not installed in act mode. The message says what to fix. For `status`, the tree has no result yet |

## Environment

| Variable | Default | Meaning |
|---|---|---|
| `SHIPCHECK_HOME` | `$XDG_STATE_HOME/shipcheck`, else `~/.local/state/shipcheck` | Where results, logs and warm checkouts live. Only `~/.local/state/shipcheck` counts as the default home: when `SHIPCHECK_HOME` or `XDG_STATE_HOME` moves it, `post` and the daemon refuse without `--home-ok`, and `init` and `install` leave the daemon service out. A home left at `~/Library/Application Support/shipcheck` by an older shipcheck moves here on first use. The default is inside a dot folder, so a test that refuses paths with a hidden folder in them fails there: set `SHIPCHECK_HOME` elsewhere, and pass `--home-ok` to post. Background runs log to `logs/background.log` in it. At the start of each run, results untouched for 14 days, logs untouched for 90 days and warm checkouts no run used for 7 days are deleted. At its end, the Docker volumes of this home that no run used for 14 days are removed, unless a container uses them, and so are the runner images an older shipcheck pinned. shipcheck never removes a volume it did not make, or an image a workflow names |
| `SHIPCHECK_TIMINGS` | unset | Set, `run` prints how long each phase of each job took to stderr, such as the clone, act's start and the cleanup |
| `SHIPCHECK_MAX_RUNS` | a quarter of the CPUs, at least 1 | How many runs the hooks and the daemon start at once on this machine. Others wait for a slot. `shipcheck run` never waits |
| `SHIPCHECK_API` | `https://api.shipcheck.run` | The service `login`, `logout`, `token` and `post` talk to |
| `SHIPCHECK_NO_DAEMON` | unset | `1` makes `init` and `install` leave the daemon service out on this machine |
| `SHIPCHECK_SKIP` | unset | `1` on a commit or push starts no background run, see [skips](https://shipcheck.run/docs/config.md) |
| `SHIPCHECK_TOKEN` | the token from `shipcheck login` | A machine token for `post`, for an unattended machine such as a VM or an agent box. Make one with `shipcheck token`. When set, `post` uses it instead of the login token, and `logout` revokes it. The daemon service gets only `PATH`, so it never sees it ([the daemon](https://shipcheck.run/docs/install/#the-daemon)) |

# How it works

https://shipcheck.run/docs/how-it-works/

> What shipcheck runs, where, and when it trusts a stored result.

## A tool reports the result, never the agent

The agent can start a run. Only shipcheck turns a run into a check, from facts it reads itself: the exit codes of the processes it started, the git tree of the commit, and the config on `main`. An agent that says the tests passed changes nothing.

## What runs

`runner` in `shipcheck.toml` picks the mode of every job, and `[jobs]` moves one job to the other mode in the same run ([modes](https://shipcheck.run/docs/config/#modes)). `shipcheck init` writes host mode when it finds checks in the workflows or a project file it knows, with a `[jobs]` act entry for each job no host check runs as GitHub does. It writes act mode for the whole repository when a job runs a Linux matrix, every job runs on Linux and Docker is running, and when it finds nothing ([which jobs run where](#which-jobs-run-where)).

- Host mode, `runner = "host"`: the commands the owner lists under `[host] checks`, such as `cargo test` or `npm test`, run natively on this machine in an OS sandbox, each in its own clone. A pass means the code passed on this machine's toolchain, operating system and system libraries ([modes](https://shipcheck.run/docs/config/#modes)).
- act mode, `runner = "act"`: the GitHub workflows run in Linux containers with act. This is the mode closest to GitHub's CI: a pass means the workflows as written passed in a container like GitHub's runner. It needs Docker.

shipcheck reads `shipcheck.toml` and `.github/workflows/` from the base branch, found in one order for every command: the base `install` stored for this clone, then the default branch the remote reports (`git ls-remote --symref origin HEAD`), then `origin/main` with a warning. The hooks never ask the remote, so they skip the second step. `init` refuses when the remote does not answer, and nothing posts from the `origin/main` fallback. These docs call that branch `main`. shipcheck never reads them from the branch or from your working copy, so a PR that edits a workflow to `exit 0` or empties `workflows` changes nothing until it merges.

When `paths` is set and a change touches none of those files since it left `main`, nothing runs and every job check is skipped with that reason. shipcheck works this out again on each run, because it depends on where `main` is.

In act mode, before a run, shipcheck edits its own copy of each workflow it runs. The repository's files stay as they are.

- It adds a `workflow_dispatch` trigger, so act can run a workflow that on GitHub runs only on pushes to `main`. A job guarded by `if: github.ref == 'refs/heads/main'`, such as a deploy, is skipped, because the ref is the branch.
- It adds a first and a last step to every job. They print markers with a code that is new for each run, which tell shipcheck that each job started and how it ended.
- It splits each matrix into one job per combination. act does not always pick the same platform for matrix jobs, and separate jobs run the same way every time.

A workflow passes when every job that started reached its end with status `success`, and no job was skipped for an unsupported platform. As on GitHub, a failed job with `continue-on-error: true` does not fail the workflow. act runs a PR's checks as `workflow_dispatch`, so when it skips a job by its `if:`, that skip does not tell whether GitHub would run the job on the PR. So a job act skips by its `if:` fails, with ``failure, act skipped it by its `if:`, which may hold on a PR, and only GitHub can tell``, unless that `if:` is false on every PR:

- `false`
- a push event, such as `github.event_name == 'push'`
- one branch, such as `github.ref == 'refs/heads/main'`
- a tag, such as `startsWith(github.ref, 'refs/tags/')`

A condition counts when it ands one of these terms with the rest. An `||`, or a `!` other than `!=`, could make it hold on a PR, so a condition with one never counts. A job whose `if:` reads the PR, such as `github.event.pull_request.draft == false`, or reads `vars`, fails too. A job that needs a skipped job fails as well when its `if:` uses `always()`, `failure()` or `cancelled()`, since GitHub would run it anyway.

Steps get the same check. A step whose `if:` reads something `workflow_dispatch` and a PR answer differently, the event, `github.ref`, `head_ref`, `base_ref`, the actor or `vars`, gets a `shipcheck if` step before it in shipcheck's copy of the workflow. That also counts when the `if:` reads an `env.` value whose step, job or workflow `env` reads one of them. When act skips the step, the guard fails the job with `act skipped the step '<name>' by its if:, which may hold on a PR, and only GitHub can tell`. A step whose `if:` is false on every PR, or that runs only on `failure()` or `cancelled()`, gets no guard. The guard is part of the workflow as it runs, so the cache key holds it.

The step check has two gaps. It does not see steps inside a composite action. It does not catch a step that act runs but GitHub would skip on the PR.

To fix such a job or step, keep the job on GitHub by leaving its workflow out of `workflows`, or make the `if:` false on every PR with one of the terms above.

## Which jobs run where

`shipcheck init` places each job of a workflow GitHub runs for feature branches:

| Job | For a PR it runs |
|---|---|
| A host check runs it as GitHub does | Here, as a host check |
| No host check runs it as GitHub does, such as a job with a heredoc or `\` step, `sudo` or `apt`, an action no command stands for, a `container:`, a JDK this machine lacks, or a Linux matrix, Docker running | Here, with act, as a `[jobs."<file>/<job>"]` entry with `runner = "act"`. init also writes `network = "internet"`, since such jobs download what their steps set up |
| The same, no Docker | On GitHub Actions |
| A Linux matrix, Docker running, every job on Linux and none running Docker | Here, with act. The whole repository uses act mode, with `network = "internet"` and `shard_tests = true` |
| A matrix with macOS or Windows entries | On GitHub Actions |
| On macOS | Here as a host check on a Mac, on GitHub Actions elsewhere |
| On Windows | On GitHub Actions |
| It reads the PR: through its `if:`, a step's `if:`, an `env`, secrets, `vars` or ref value, a script, Makefile, justfile or `package.json` its command reads, or `HEAD` in a PR workflow | On GitHub Actions |
| It uses a PR-aware action, such as one that lints only the changed lines, or an action that reports to GitHub or publishes | On GitHub Actions |
| Its `runs-on` label has no act image, or it uses a local action | On GitHub Actions |
| It runs Docker or changes the kernel, such as `mount`, `sysctl`, `bwrap` or `modprobe` | On GitHub Actions. colima could run it, and you can move it to a colima VM on init's Checks tab. init never picks that by default, since the VM reaches this machine's loopback ([engines](https://shipcheck.run/docs/config/#engines)) |
| Its host check fails when init tries it | On GitHub Actions, and init prints the end of its log and the fix |

A job goes to act only when every job it needs can run there too. A job a job on GitHub needs stays on GitHub with it. A Rust toolchain or Ruby that this machine lacks does not keep a job off: its check runs on this machine's version, and init lists it under "run on this machine's version, not the workflow's".

A matrix of one combination is a plain job. A matrix exists to test several toolchains or systems, and host mode tests only the one on this machine, so a matrix job runs with act or stays on GitHub. That holds for a matrix such as express's, whose `run` steps never name the matrix, since its setup actions do. When the whole repository uses act mode, a workflow with a job act cannot run as GitHub does stays on GitHub as it is, since act runs a listed workflow whole.

shipcheck does not support Windows yet, so Windows jobs stay on GitHub Actions ([limits](https://shipcheck.run/docs/limits/#windows)).

`init` prints each job that stays on GitHub and why. A workflow with such a job is kept whole: its triggers stay, and each job shipcheck runs gets an `if:` that runs it on GitHub only on the default branch, on the other branches the workflow lists for pushes, on PRs into other branches, for Dependabot and for a fork's PR, and-ed with an `if:` it already has. On a PR into the default branch, GitHub then runs only the jobs that stay, and on the default branch it runs them all. A job that a job on GitHub needs, directly or through others, stays on GitHub too, since GitHub skips a job whose needs it skipped. If you decline the edit, the workflow keeps running fully on GitHub for PRs, beside shipcheck's checks.

The ruleset `init` offers requires `shipcheck/<job>` from the shipcheck App and, from GitHub Actions, the checks of the jobs that stay on GitHub. When their names depend on a matrix `include` or an expression, `init` says so and you add them once a PR shows them. Jobs that stay on GitHub still use Actions minutes.

To move a repository to act anyway, set `runner = "act"` and list the workflows in `workflows` in `shipcheck.toml` ([modes](https://shipcheck.run/docs/config/#modes)). To give a job back to GitHub, remove its check from `[host.checks]` or its `[jobs]` entry, and its `if:` from the workflow.

## When it runs

Runs start from the hooks, as `on` in `shipcheck.toml` says, or from `shipcheck run`.

`shipcheck run` tests the working copy, so the test run an agent does before it commits becomes the check. shipcheck builds a tree from the working copy in a copy of the index: tracked files with their edits and untracked files that are not gitignored. Your branch, index and stash stay as they are.

`--staged` builds it from the index instead. When you commit that same tree, with `shipcheck commit` or with git, the commit hook and `shipcheck post` find the result by tree and nothing runs again. A commit with any other tree, such as one with a file left out, runs again.

- A run from a hook goes to the background. The commit or push carries on at once, and the run's output goes to `logs/background.log` in shipcheck's home.
- There is one background run per repository and branch. A newer commit stops the older run, its containers included, before its own starts, like `cancel-in-progress` in a workflow. A request for the commit already running leaves that run alone, so a push right after a commit does not restart it.
- On a machine signed in with `shipcheck login`, or with `SHIPCHECK_TOKEN` set, a push's run also posts the checks. Once GitHub has the commit, every check shows in progress, and when the run ends it turns green or red. A tree that already passed posts at once. A newer push stops the older run, which posts nothing. A machine that is not signed in writes one line to `logs/background.log` and posts nothing.
- `shipcheck run` runs in the foreground and waits, for when you or an agent want the verdict before going on.

## When a PR changes on GitHub

A rebase or an update on GitHub, such as the Update branch button, makes a new head commit with no local push, so no hook runs. The daemon covers that case. `shipcheck init` and `shipcheck install` start it as a background service.

- It reads the open PRs of each repository it was added to, into the base `install` stored for it: the default branch the remote reported then. With the App installed, shipcheck's service gets GitHub's pull request webhook for each push, rebase, Update branch and merge, and tells the daemon over a socket the daemon keeps open, so the daemon reads the PRs within seconds of the change. While that socket is down, or a head is still waiting, such as for who pushed it, it reads them every 10 seconds by default. Otherwise it reads them every 5 minutes anyway, in case it missed an event.
- When a head has no completed `shipcheck` check and its tree has a stored result, it posts that result at once. A rebase that changes no file has the same tree, so it needs no run.
- When the tree has no result, it runs the checks and posts them, the same way a push does. A newer head stops the older run, which posts nothing.
- It runs a head only when you, or a login `daemon_authors` in `shipcheck.toml` on `main` lists, both opened the PR and pushed that head. It never runs a fork's PR.
- "Update branch" and a rebase on GitHub are pushes by whoever clicked the button. So when you click it on your own PR, your daemon runs the new head. When a teammate clicks it, your daemon refuses the head unless `daemon_authors` lists them. To check it yourself, pull the branch, then run `shipcheck run --sha <head>` and `shipcheck post <head>`.
- The daemon learns who pushed a head from shipcheck's service first, which gets GitHub's pull request webhook for each push to a PR within seconds. Otherwise it reads the PR's timeline, which shows a rebase "Update branch" or a force-push at once. For a merge "Update branch", it reads the merge commit GitHub made, which names the person who clicked. For a PR merged into the branch, such as a stacked PR, it reads who merged it. Last, it reads the repository's events, which GitHub can list hours late. Until it knows, the checks stay queued and it asks again on every poll.
- A head that someone not allowed pushed stays refused for 30 days, and so does every later head that contains it, such as a merge "Update branch" made on top of it. A head whose tree already has a result is never refused: the daemon posts that result.
- With several machines running it, the first to mark a head in progress runs it, and the others wait. Two that start within a few seconds of each other can both run it, and both post the same result.
- When a PR merges into the branch, it checks the branch's new tip too, also when the service cannot say whether a closed PR merged, so `main` has checks of its own, the ones a status badge reads. A squash of a PR whose tree passed posts at once from that result. A commit whose message has `[skip ci]`, such as a release bump, gets none.
- Every 5 minutes it also looks in the folders that hold the repositories it watches, such as `~/Projects`. A clone of another repository is added once its default branch has a `shipcheck.toml` and the App can read the repository, so installing the App on a repository you cloned is enough.

`daemon = false` in `shipcheck.toml` keeps it off a repository, and `shipcheck daemon --uninstall` removes it from the machine.

The daemon's settings live in `~/.local/state/shipcheck/daemon.toml`, next to the repositories it watches. `fetch = "https"` sends shipcheck's own fetches of a GitHub remote over HTTPS with git's credential helper, such as `gh auth git-credential`: the daemon's, a posting run's and the one that finds the default branch. An SSH key that asks for a touch then never signs them. Your remotes, your own git and every push stay as they are.

## Where it runs

In act mode each job gets a new checkout of the tested tree, cloned with copy-on-write from its own warm template, so it starts with the build output of its earlier runs and costs no disk. A working copy is checked out from a commit no ref names, with `HEAD` as parent, as `git stash create` makes. A run never touches your own checkout. Jobs never share a checkout, as on GitHub. A job starts once the jobs in its `needs` pass, and gets their outputs. Gitignored files are not in it.

The checkout sits at a fixed path per repository, slot and job, because builds remember absolute paths. Toolchains and package caches live in Docker volumes per repository, removed after 14 days without a run that uses them. apt's package lists get a volume per slot and job, and `.venv` a new volume each run, off the macOS bind mount. setup-uv steps get `prune-cache: false` in the job's copy, so uv's cache keeps its wheels, and `pnpm/action-setup`'s `run_install` becomes plain `pnpm install` steps after it, since the action prunes the store after an install. Both are fed only by shipcheck's own runs, never by builds on your machine, which would not run in a Linux container.

act runs each job in Docker, never on the machine itself. A job gets its checkout, its cache volumes, no Docker socket, no host token or environment, and by default no network.

A job with `engine = "colima"` runs on a Colima VM that shipcheck starts for that job and deletes after it. The VM shares no folder with this machine. It gets the checkout's tracked files and a `.git` of its own over `colima ssh`, and act runs inside it with `-self-hosted`, so the steps run on the VM, with its own Docker and kernel. The VM reaches this machine's `127.0.0.1` and the internet whatever `network` says, so it is for code you trust ([engines](https://shipcheck.run/docs/config/#engines)).

In host mode each check gets the same kind of clone, but its template is warmed from your checkout at most once an hour, so gitignored `node_modules` and `target/` are there. It runs in `sandbox-exec` on macOS or bubblewrap on Linux: it writes only its clone, a new home and the repository's tool caches, reads the system and your toolchains, and has the network ([config](https://shipcheck.run/docs/config/#modes)).

## When a stored result counts

shipcheck stores each result under a key made of:

- the git tree that ran
- `shipcheck.toml` on `main`
- each workflow as shipcheck runs it, or in host mode each check's command and the output of `versions`
- the act and shipcheck versions
- the pinned images

Two commits with the same tree share a result, so a rebase or a squash that changes nothing needs no new run. A change to the config or the workflows on `main` changes the key, so old results stop counting.

Only a pass is reused. A failure runs again next time, so a flaky failure cannot stick.

Two runs of the same tree at once share one run. The second waits, then reads the result.

## The checks on the PR

shipcheck posts one check per job, `shipcheck/test` for job `test`, as GitHub Actions does, and branch protection requires them ([require the checks](https://shipcheck.run/docs/branch-protection.md)). Each is queued until a machine runs the commit, in progress while it runs, then green or red. A `paths` skip marks every job skipped. A `shipcheck` check beside them sums up the run, with each failed workflow's log tail.

Each job check shows how long the job took and, as `logs` in `shipcheck.toml` says, the end of its log ([config](https://shipcheck.run/docs/config.md)):

- The last 100 lines. A longer log starts with `[...]`, and a host check's log keeps its command line above it, so you still see what ran.
- A failed job lists its failed tests first, then each test's own output, such as its panic, assertion or traceback, even when it is far above the last lines. This works for cargo test, nextest, pytest, jest, vitest and go test.
- When the failures do not fit in GitHub's 65,535 characters, the check keeps the first ones whole and says how many it left out. `shipcheck logs` has the rest.
- Values from the secrets file are replaced with `***`. A log can still hold paths, names or a secret printed in another form.
- A fork's PR never gets a log.

## The report comment

Write a report to `$GITHUB_STEP_SUMMARY` and it appears in one shipcheck comment on the PR. This replaces a step that posts a coverage report with `gh pr comment`, which cannot run under shipcheck: a job has no GitHub token and, by default, no network.

- Each job check gets its own section, failed jobs first. Every step that writes to the file adds to its job's section in step order, as on GitHub.
- The App writes the comment once and then edits it. A later post replaces the sections of the jobs it ran and keeps the others, so a rerun of one job changes only that job's section. A job that ran again and wrote nothing loses its section.
- If someone edited the comment and its hidden markers no longer match, the next post writes the whole comment again from its own result.
- A run where no job wrote a summary posts nothing and deletes nothing.
- Each job keeps at most 20 KB of its summary. The comment stays under GitHub's 65,536 characters, and a section that does not fit says it was left out.
- Values from the secrets file are replaced with `***`.
- A fork's PR never gets the comment, since the fork's own code wrote the Markdown.

It works in both modes. In act mode, act gives each step the file. In host mode, each check gets `GITHUB_STEP_SUMMARY` set to a file in its sandbox. The App needs Pull requests: write for the comment. Until it has it, the checks still post, and `shipcheck post` prints one line saying there is no comment.

# What a green check proves

https://shipcheck.run/docs/proof/

> What a passing shipcheck job check guarantees, and what it does not.

A green `shipcheck/<job>` check on a PR means shipcheck ran that job on the PR's tree, with the config from `main`, and the job passed. The rules below hold for every check.

## What it guarantees

- The result comes only from a run shipcheck started. shipcheck reads the exit codes of the processes it started. Nothing accepts a result, a log or a verdict from an agent or a script, so an agent that says the tests passed changes nothing.
- `shipcheck.toml` and the workflows come from `main`. shipcheck never reads them from the branch or the working copy. A PR that edits a workflow to `exit 0`, or empties `workflows`, is still checked with `main`'s version.
- The job list comes from `main`. A job that `main` has and the result lacks fails, so a PR cannot drop one.
- Each job runs in a new copy-on-write clone of the tested tree, never in the agent's worktree, and never in a checkout another job used. Before the job starts, shipcheck hashes every file of the clone again, and the job fails unless they make exactly the tested tree.
- A job passes only when its process exits 0. In act mode every job that started must also report `success`. A timeout, a missing job or an error is a failure.
- A result belongs to a git tree, never to a commit. A commit gets a stored pass only when its tree is the tree that ran, so a commit with one more file runs again. The key also holds the config on `main`, the workflows as they run, the tool versions and the images, so a change to any of them runs again ([when a stored result counts](https://shipcheck.run/docs/how-it-works/#when-a-stored-result-counts)).
- Only a pass is reused. A failure runs again next time.
- A `paths` skip is worked out again on each run and never stored. Any doubt about which files changed means run.
- With branches required to be up to date, the tested tree is the tree a squash merge puts on `main` ([require the checks](https://shipcheck.run/docs/branch-protection.md)).

## How it knows it tested the right code

A check names the code it tested by hashes, and shipcheck checks each hash itself instead of trusting a file or a caller.

1. The commit names its code. Git names every commit and tree by a hash of its content, so a tree id is a hash of every file in the commit. A result belongs to a tree. After a rebase or a push the PR has a new head commit. The daemon takes that commit's SHA from GitHub, fetches it and reads its tree. A new tree runs again. A tree that was already tested, such as a rebase that changed nothing, reuses its result.
2. Each job's files are that tree. After the clone and before the job starts, shipcheck hashes every tracked file on disk again, into an index with no cached file times. The tree written from that index must be the tested tree. An edited, deleted or relinked file, or a changed mode, fails the job. Build output and untracked files are not part of a tree.
3. The build starts from those files. Every tracked file gets the run's start time, so cargo, make and other tools that compare file times rebuild the code from this tree. They never trust output another tree left.
4. What the check shows is true before it is posted. Before any pass is posted, stored or fresh, shipcheck checks it again. Its tree must be the commit's tree, its base must be on the current `main`, and each log must hash to the digest in the result. A stored result that fails this does not count, and the commit runs again. A fresh one is posted as a failure.

## Check it yourself

The `shipcheck` check on a PR lists the Tree, the Base, the Key and a Log SHA-256 per workflow. From a clone of the repository:

| Field | Command | Holds when |
|---|---|---|
| Tree | `git rev-parse <sha>^{tree}`, with `<sha>` the PR's head | it prints the Tree |
| Base | `git merge-base --is-ancestor <base> origin/<branch>`, with `<branch>` the default branch `git ls-remote --symref origin HEAD` reports | it exits 0 |
| Log SHA-256 | `shasum -a 256 ~/.local/state/shipcheck/logs/<key>/<workflow>.log` on the machine that ran it, where `<workflow>` is the workflow file or `host` for host checks | the first 12 hex digits are the Log SHA-256 |

`shipcheck logs <sha>` prints the same logs. The Key is a hash of the tree, the config on `main`, the workflows as they ran, the tool versions, the shipcheck version and the images. Only a machine with the same tools gets the same Key, so it shows when two results came from the same inputs.

## What it does not prove

- That the tests are good. Tests are part of the PR, so an agent can delete, skip or weaken one and the run passes. The check names changes to CI files and to what a command runs, such as `package.json` scripts, for the reviewer. Review test changes.
- That the code passes on GitHub's runners. Host mode tests this machine's OS and toolchain, and act's images only resemble GitHub's. Your workflows still run on GitHub after each merge to `main`.
- That the suite passed in order, with `shard_tests` on.
- Which binaries ran. Build output is reused between runs to keep them fast. shipcheck makes the build tools see every file of the tree as new, so they rebuild the code, and it does not yet record what they built.
- That no one forged a result. shipcheck runs as your user and keeps its results in a folder your user can write. It stops an agent that claims a pass. It does not stop one that sets out to write a fake result there. Keep that folder outside your agent's sandbox ([agents](https://shipcheck.run/docs/agents/#your-agent-s-sandbox)).

[Limits](https://shipcheck.run/docs/limits.md) has the full list.

# Security

https://shipcheck.run/docs/security/

> What a job can reach on your machine, what the App can do on GitHub, what the service stores, and how to report a vulnerability.

shipcheck runs a branch's code on your machine. Whoever writes that code, a teammate, a dependency or an agent, decides what runs there. So shipcheck treats every job as untrusted code.

## What a job can reach

| | act mode | Host mode |
|---|---|---|
| Where it runs | A Docker container, never on the machine itself. `-self-hosted` runs only inside the VM shipcheck starts for a colima job | Natively, as your user, in `sandbox-exec` on macOS or bubblewrap on Linux |
| Files it writes | Its own checkout and the repository's cache volumes | Its own clone, a new empty `HOME` and `TMPDIR`, and the repository's tool caches |
| Files it reads | Its checkout and volumes | The system and your toolchains, such as `~/.cargo/bin` or `~/.nvm/versions`. Nothing else in your home. Credential files such as `~/.ssh`, `~/.npmrc`, `~/.netrc` and shipcheck's own token and keys stay hidden |
| Docker | No socket | No socket. It cannot connect to a Unix socket outside its own folders, except the DNS resolver's |
| Environment | No host environment and no GitHub token | `PATH`, `USER`, `LANG` and a few more, the names in `host.env`, and `CI=true`. `host.env` refuses names that carry credentials |
| Git | A `.git` of its own with only the tested commit's history. git-lfs and submodules never run on the host | Its own clone |
| Secrets | Only the values the owner put in shipcheck's secrets file for that repository | None |
| Network | None by default. `network = "internet"` gives the internet and the local network. On a Mac with Docker Desktop it also reaches services on the Mac's `127.0.0.1` through `host.docker.internal`. A colima job reaches the internet and the machine's `127.0.0.1` whatever `network` says | The machine's network: the internet, the LAN and local services |

A job with `engine = "colima"` runs on a VM of its own, with no shared folders and no host environment. It can also change act's result inside the VM, so use colima only for code you trust. Its end-to-end tests check that a port the job listens on in the VM does not show up on the Mac's `127.0.0.1` and that every VM is deleted after its job, even when the run is killed ([engines](https://shipcheck.run/docs/config/#engines)).

act mode also refuses Docker options that reach the host, such as `--privileged`, and refuses reusable workflows. It refuses Docker Desktop before 4.44.3 and runc versions with known container escapes.

Host mode gives up the container, so use it only for your own and your agents' work. A host check with `sandbox = false` on `main` also gives up the sandbox and runs with your full access once you run `shipcheck trust` in the repository, so use it only where you trust every author ([outside the sandbox](https://shipcheck.run/docs/config/#outside-the-sandbox)). When the config has host checks, the daemon never checks a fork's PR, not even with a stored result. [Modes](https://shipcheck.run/docs/config/#modes) has the full list of what a host check gets.

Hooks run only your own commits and pushes. The daemon runs a PR only when you, or a login `daemon_authors` on `main` lists, opened it and pushed its head, and never a fork's ([when a PR changes on GitHub](https://shipcheck.run/docs/how-it-works/#when-a-pr-changes-on-github)).

## What the App can do on GitHub

The [shipcheck App](https://github.com/apps/shipcheck-run) asks for three permissions:

| Permission | For |
|---|---|
| Checks: write | Posting the job checks and the `shipcheck` summary |
| Pull requests: write | Listing a repository's open PRs for the daemon, finding the PRs of a commit, learning who pushed a PR's head from its timeline and the pull request webhook, and keeping one [report comment](https://shipcheck.run/docs/how-it-works/#the-report-comment) on a PR |
| Metadata: read | Basic repository data, which every App has, and the repository's events, where the daemon looks last for who pushed a head |

The App has no Contents permission, so it cannot read or change your code. Its private key lives only in shipcheck's service. A job never gets the App's token or any GitHub token.

## What the hosted service stores

`shipcheck post` sends the check text and the report comment to shipcheck's service, which posts them with the App and keeps no copy. The service first checks that your GitHub user has write, maintain or admin access to the repository. Each App token it makes covers one repository and only the permission that request needs.

It stores, for each signed-in machine, a SHA-256 hash of the machine token, your GitHub user id and login, the machine name, and when the token was made and when it expires. It never stores the token itself, and it revokes the GitHub token from the sign-in once it knows who you are. A token stops working after 90 days without a post. It also counts failed sign-ins per IP address and locks an address out for 15 minutes after three. For a push to a PR, it keeps what GitHub's pull request webhook says for 7 days: the repository id, the PR number, the head SHA, the head repository id and ref, and the pusher's login and user id, so the daemon can learn who pushed.

Results and logs stay on your machine. shipcheck sends no telemetry.

## Keys on your machine

On macOS, shipcheck keeps the machine token and the secrets files in your login keychain, where an agent running as you gets a prompt instead of the key. On Linux they are files of mode `0600`, which any program you run can read ([where shipcheck keeps its keys](https://shipcheck.run/docs/install/#where-shipcheck-keeps-its-keys)).

## Report a vulnerability

Use GitHub's [private vulnerability reporting](https://github.com/gabrielkoerich/shipcheck/security/advisories/new). Do not open a public issue. You get an answer within a few days.

# Limits

https://shipcheck.run/docs/limits/

> What shipcheck does not catch, and what act cannot run.

## What shipcheck does not catch

- Weakened tests. Tests are part of the PR. An agent can delete, skip or loosen a test and the run passes. Review test changes.
- Differences from GitHub's runners. Host mode runs your checks on this machine's OS and toolchain, so a test that fails only on Linux can pass on a Mac. Four of orch's tests failed on macOS in host mode. act runs your steps in Docker images that resemble GitHub's runners but are not the same. GitHub still runs your workflows on `main` after each merge, which catches what slips through. Jobs run as `runner`, uid 1001, with `HOME=/home/runner` and `sudo` without a password, as on GitHub, so a step that needs root needs `sudo` here too. The workspace path, the tool cache and some environment variables differ.
- Test pollution, with `shard_tests` on. A test that fails serially only because an earlier test left state behind can pass when the two run in different shards. GitHub's serial run on `main` catches it. Leave `shard_tests` off if a pass must mean the suite passed in order.
- An agent that writes into shipcheck's folder. Results live in `SHIPCHECK_HOME`, which your user can write. shipcheck does not stop an agent from writing a fake result there. Only the agent's own sandbox does, when it is on and the folder is outside its writable paths.
- An agent that skips the instructions. It runs the project's tests and `git commit` as usual. The git hook then runs the commit again on push, or on commit with `on = ["push", "commit"]`, which costs time. The check still comes from that run.
- A determined cheat on the same machine. shipcheck runs as your user. It stops an agent that claims a pass. It is not built to stop someone who sets out to forge one.

## Windows

shipcheck's releases are built for macOS and Linux, on arm64 and x86_64. Host mode needs an OS sandbox (`sandbox-exec` on macOS, bubblewrap on Linux), and shipcheck has none for Windows. Windows jobs stay on GitHub Actions, and `init` leaves them there.

## What act cannot run

- macOS and Windows jobs. Docker runs only Linux jobs, and shipcheck never runs a job on the machine itself, so a workflow with macOS or Windows jobs fails with `no image for its runs-on label on this host`. Leave it to GitHub, as `init` does ([which jobs run where](https://shipcheck.run/docs/how-it-works/#which-jobs-run-where)). Host mode runs macOS jobs on a Mac. The same message for a Linux label, such as `ubuntu-24.04` on act 0.2.89, means the label needs an image in `[act] platforms`.
- The network, by default. With `network = "none"`, the default, a job reaches nothing: no package downloads, no toolchain downloads from setup actions, and no artifacts passed between jobs. With `network = "internet"` all of that works. On a Mac with Docker Desktop, a job with `network = "internet"` also reaches services the Mac binds to `127.0.0.1`, through `host.docker.internal`, and the local network. On Linux the job gets a Docker bridge network of the run's own, not the host's network.
- Reusable workflows. A job with `uses: ./.github/workflows/x.yml` or `uses: owner/repo/.github/workflows/x.yml@ref` fails the run, since act would run the called workflow's jobs without shipcheck's checks, Docker options included. Leave such a workflow out of `workflows` and let GitHub run it.
- Git LFS files and submodules. shipcheck checks out a commit on this machine with git-lfs and submodules turned off, since both run on the host with what the branch's files say. A job sees LFS pointer files and empty submodule directories. Fetch them in a step, which runs in the container.
- The runner user, with an image of your own. The default images run jobs as `runner` with `sudo`, as GitHub does, but an image of your own runs as its own user, often root. Then a test that expects a `chmod 000` file to be unreadable passes on GitHub and fails here. Skip such tests when the effective user is root.
- Jobs that need a full VM. Mounting loop devices, systemd and some service containers do not work in Docker. `engine = "colima"` runs such a job on a VM of its own, for code you trust ([engines](https://shipcheck.run/docs/config/#engines)).
- GitHub's own services. Jobs get no `GITHUB_TOKEN` and no OIDC. `actions/cache` always misses, since the warm checkout per job does its work.
- `actions/upload-artifact` v7. act 0.2.89's artifact server refuses its requests (`unknown field "mime_type"`), so the step fails after five retries. Pin the step to v4 to v6, or leave the upload out of the workflows shipcheck runs.
- A job act skips by its `if:`. act runs a PR's checks as `workflow_dispatch`, so a skip fails the job unless its `if:` is false on every PR: `false`, a push event, one branch or a tag. A step act skips by an `if:` that reads the event, the ref, the actor or `vars` fails its job the same way. Steps inside composite actions are not checked, and neither is a step act runs but GitHub would skip. Keep such a job on GitHub, or make the `if:` false on every PR ([what runs](https://shipcheck.run/docs/how-it-works/#what-runs)).
- The event payload. act starts each workflow with a `workflow_dispatch` event that has no `repository`, so an action that reads the owner from the payload, such as gitleaks-action, fails.
- Docker inside a job. Jobs get no Docker socket, since a job with it controls the whole machine. Steps that run `docker build`, `docker compose` or Testcontainers fail. Docker actions and `services:` still work, since act starts them itself. A colima job gets the VM's own Docker.

`workflows` takes whole workflows. If one job of a workflow cannot run under act, leave that workflow out of `workflows`, move the job to its own workflow, or give it a host check with a `[jobs]` host entry.

## Costs to know

- With `shard_tests` on, a failing run takes the sharded time plus a full serial run, since shipcheck reruns it whole before it reports the failure. A pytest step skipped by its own `if:` also makes the job run twice. Only a one-line pytest step in bash with no `;`, `&&`, pipes, redirects, `$` or pytest-xdist flags is sharded, one per job.
- Warm checkouts and cache volumes are not part of the result key. A broken toolchain in a volume breaks every run of that repository until you remove the volume with `docker volume rm`.
- shipcheck fetches each action once per job and reuses that copy without asking GitHub. An action pinned by tag or branch, such as `@v4`, stays at the commit it had then, even after its owner moves the tag. Pin actions by SHA, or run `shipcheck clean` to fetch them again.
- On macOS, Docker reaches the checkout through the VM's file sharing, which is slower for very large build directories than on Linux.