Error reference — goat and goatbuilder¶
Generated from ops/errors/catalog.yaml by ops/ci/check-error-catalog.py --write; do not edit here. Look a code up with goat explain <code> or goatbuilder explain <code>; paste any error text to goat explain - / goatbuilder explain - to have it interpreted (ADR 1199, error-interpretation.md).
goat¶
| Code | Tool | What it means | Why | What to do | Read | Exit |
|---|---|---|---|---|---|---|
E_PROFILE_MISSING |
goat | no profile for this context — goat was run before goat profile init, or the context named has no profile.toml, certificate or key. |
the bundle was never installed on this machine the bundle was installed into a different context than the one in use |
goat profile init <bundle.cbor>goat --context <name> … (when the bundle was installed under another context) |
docs/design/goat-cli-error-codes.md |
3 |
E_PROFILE_DENIED |
goat | the profile exists but this user cannot read it — the profile directory is there and permission was denied reading it, typically a root-owned 0700 ~/.goat probed without sudo, or bare sudo that reset $HOME to /root. | the profile was created as root bare sudo goat … looked in /root/.goat |
sudo chown -R $USER ~/.goatsudo -E goat … (keeps your $HOME) |
docs/design/goat-cli-error-codes.md |
3 |
E_BUNDLE_INVALID |
goat | the bundle did not verify — at goat profile init the bundle's structure, signature or expiry failed. The help line says which of the three. |
a truncated or wrong download (malformed CBOR) a signing CA this machine does not trust an expired bundle |
goat ca ls (what this machine trusts)goat ca add <root.pem> (when the CA is the problem)ask the operator for a re-issued bundle when it has expired |
docs/design/goat-cli-error-codes.md |
3 |
E_OFF_MESH |
goat | not connected to the mesh — the local mesh client reports no connected tunnel, so nothing on the goatnet is reachable. | the mesh client is stopped or logged out this device was never enrolled the device is on a network that blocks the mesh |
netbird status -dgoat heal && goat doctor |
docs/design/goat-cli-error-codes.md |
5 |
E_CLOCK_SKEW |
goat | the device clock is wrong — the device clock differs from the management server's authenticated HTTPS Date by more than 60 seconds, so certificates cannot be judged valid. | the clock drifted while the device was off or offline | synchronise the clock (chrony/ntp), then: goat doctor |
docs/design/time-hygiene-trusted-time.md |
6 |
E_ROUTER_UNREACHABLE |
goat | the Zenoh router did not answer — the configured router's TCP path or Zenoh session timed out or could not be reached. | the mesh tunnel is up but the route to the router is not the router service is down on its host the endpoint in the profile is stale |
goat doctor --allcompare from another enrolled peer before changing anything local |
docs/design/goat-cli-error-codes.md |
5 |
E_CERT_EXPIRED |
goat | the mTLS certificate is outside its validity — the local certificate is expired, not yet valid, or was rejected by the router. | the certificate's validity interval has passed the device clock is wrong (see E_CLOCK_SKEW) |
goat profile showgoat profile rotate <new-bundle.cbor> (after the operator re-issues it) |
docs/design/internal-tls-ca.md |
6 |
E_DOCTOR_INCONCLUSIVE |
goat | doctor saw a failure it cannot narrow — a local TLS or configuration failure, or a session rejection, without enough evidence for one cause. | router-side admission refused this identity a local TLS setup problem the probe cannot distinguish from the router's |
follow the hint on the failing doctor stepif the local certificate is valid, send the step output to the goatnet operator |
docs/design/goat-cli-error-codes.md |
1 |
E_PORTAL_UNREACHABLE |
goat | the portal API did not answer — the portal was not reachable for network reasons (DNS, TCP or TLS), not an authorization failure. | the portal host is not reachable over the mesh from this device mesh DNS is not resolving the portal name |
goat doctor --allcheck from another peer: if it answers there, the problem is local |
docs/design/goat-cli-error-codes.md |
4 |
E_PORTAL_NOT_FOUND |
goat | the portal answered that what was named is not registered — the portal is up and answered HTTP 404 for the request path shown: the org, topic, schema, machine or person the command named does not exist there. An answer, not a network fault. | a typo in the org prefix, topic key or handle the thing is registered on another goatnet than the active context's it was removed, or never created |
goat vendor ls (the orgs that exist)goat topic ls <org>/ (the topics under one org)goat context show (which goatnet this context talks to) |
docs/design/goat-cli-error-codes.md |
7 |
E_PORTAL_RATE_LIMITED |
goat | the portal is rate-limiting this client — the portal is up and answered HTTP 429: this client sent too many requests. The hint carries the portal's Retry-After when it sent one. | a script calling goat in a tight loop an older goat whose topic ls with no prefix made one request per org |
wait the Retry-After the hint names, then retryspace the calls out, or list one org at a time: goat topic ls <org>/goat update (current goat lists the whole goatnet in one request) |
docs/design/goat-cli-error-codes.md |
8 |
E_PORTAL_SERVER_ERROR |
goat | the portal failed on its own side — the portal is up and answered HTTP 5xx for the request path shown. Nothing on this device's network or credential is at fault. | the portal or a service behind it (its database, the registry) is failing the portal is restarting during a deploy (often a 502 or 503) |
retry after the Retry-After the hint names, or in a minuteif it persists, send the code, status and path to a platform admin; the portal's log on the goatnet's management host names the cause |
docs/design/goat-cli-error-codes.md |
9 |
E_PORTAL_REFUSED |
goat | the portal refused the request — the portal is up and answered a non-2xx status no command curates (a 400, 405, 410, 413 or 422) for the request path shown. | this goat build and the portal are at different API versions a value the command sent is one the portal does not accept |
goat doctor (shows whether goat is behind the served version)goat update |
docs/design/goat-cli-error-codes.md |
4 |
E_PORTAL_BAD_RESPONSE |
goat | the portal answered in a shape this goat build does not understand — the portal returned 2xx but the body did not deserialize. The portal is healthy; the CLI and the portal are at different versions. | goat is behind the portal the portal deployed ahead of the CLI |
goat update (or re-install from get.<site>)goat --version, and report the endpoint if already current |
docs/design/goat-cli-error-codes.md |
4 |
E_TOKEN_INVALID |
goat | the portal rejected the token — the portal accepted the connection and refused the bearer token (HTTP 401): the credential itself is expired or misissued. | the token in ~/.goat/contexts/ the token belongs to another goatnet |
goat profile rotate <new-bundle.cbor> (after re-running onboarding) |
docs/design/goat-cli-error-codes.md |
4 |
E_PORTAL_FORBIDDEN |
goat | the portal refused this identity — HTTP 403 to the probe. The token was accepted; the identity lacks the access. | the active context is a different identity than you meant the grant for this surface was never made |
goat context showask the goatnet operator for the access grant; rotating the token will not grant it |
docs/design/goat-cli-error-codes.md |
4 |
E_NOT_ADMIN |
goat | an operator-only action without operator authority — a goat admin action was refused, client-side (no operator credentials in the profile) or by the portal/NetBird API (HTTP 403, nothing changed). |
the profile is using the vp_ participant token, not the op_ operator key the role is not authorized for this action |
GOAT_PORTAL_TOKEN=op_… goat admin …goat admin operator-key request --reason … (when you hold no op_ key) |
docs/design/platform-admin-lifecycle.md |
4 |
E_TOPIC_CONFLICT |
goat | the topic already exists — goat topic create was asked for a name that is already registered. |
someone created it first a re-run of a create that already succeeded |
goat topic show <topic>pick a new name, or version the topic (docs/design/pubsub-best-practices.md) |
docs/design/goat-cli-error-codes.md |
1 |
E_MACHINE_UNRESOLVED |
goat | the machine named is not exactly one enrolled machine — a goat machine verb was given a handle or slot anchor that matches no enrolled machine this profile can see, or a handle that matches machines under more than one custodian. Nothing was changed. |
a typo in the handle the machine is under another custodian's custody, which a non-operator profile cannot see the machine was revoked two custodians each enrolled a machine with the same handle |
goat machine ls (shows each machine's handle, custodian and slot anchor)pass the 32-hex slot anchor instead of the handle when the handle is ambiguous |
docs/design/goat-cli-error-codes.md |
1 |
E_MESH_PEER_UNRESOLVED |
goat | the enrolled machine has no single mesh peer — the machine is enrolled, but no NetBird peer carries its name, or several do. A machine's peer is named machine-<handle>-<host> by its setup key, or <handle> when renamed by hand. Nothing was changed. |
the machine never joined the mesh with its setup key the peer was renamed off its handle in the NetBird dashboard the machine joined twice, leaving a stale peer beside the live one |
goat admin peer ls (find the machine's peer and its name)goat machine mesh-key <machine> (when it never joined)rename the peer back to, or remove the stale peer with goat admin peer cut <peer-id> |
docs/design/goat-cli-error-codes.md |
1 |
E_NETBIRD_FAILED |
goat | the NetBird management API call failed — a call to the goatnet's NetBird management API did not succeed for a reason other than authorization, so nothing was changed. The help line carries the status or the transport error. | the NetBird admin token was rejected (HTTP 401) the management server is unreachable from this machine the management server answered an unexpected status the peer record lacked a setting the update has to send back unchanged |
check the token: export <CONTEXT>_NB_API_TOKEN (or GOAT_NB_API_TOKEN) with a current admin tokengoat admin peer ls (proves the URL and token work)retry once the management server answers |
docs/design/goat-cli-error-codes.md |
4 |
E_ACL_DECLARATION_MISSING |
goat | no ACL declaration was supplied — goat acl test or goat warrant needs the goatnet's netbird_acl_taxonomy.yml and netbird_acl_matrix.yml and found them nowhere. goat carries no deployment's declaration, so it stops rather than check other rules. |
the command ran outside a checkout and the context holds no declaration the goatnet does not serve its declaration at get. goat context refresh fetched none |
goat acl test --declaration <dir> (a directory holding both files)export GOAT_ACL_DECLARATION_DIR=<dir>set acl_declaration_dir = "<dir>" in ~/.goat/contexts/<name>/profile.tomlgoat context refresh (fills ~/.goat/contexts/<name>/acl/ when the goatnet serves it)run from a checkout that carries ops/ansible/netbird_acl_taxonomy.yml and netbird_acl_matrix.yml |
docs/design/goat-cli-error-codes.md |
1 |
E_RECIPIENT_KEY_UNPUBLISHED |
goat | this anchor's recipient key is not in the recipient-cert store — goat access key asked the portal's recipient-cert store for the anchor's public leaf cert and the portal answered 404, so the key service cannot wrap sealed data to that anchor. The text after the dash is the portal's own reason. |
the anchor's leaf was minted before the recipient-cert store existed, so its public cert was never stored the anchor is not the one meant (a typo, or an anchor from another goatnet) |
goat access key --anchor <anchor> (check the anchor; with no --anchor it checks your own profile's)goat admin rotate <anchor> (an operator re-issues the leaf in place, ADR 1044, which stores its public cert; install the new bundle's cert and key on the peer)goat access key --anchor <anchor> (run again: it reports PUBLISHED) |
docs/design/goat-cli-error-codes.md |
1 |
goatbuilder¶
| Code | Tool | What it means | Why | What to do | Read | Exit |
|---|---|---|---|---|---|---|
GB-001 |
goatbuilder | unknown component — the component named is not declared in ops/pins/fleet-pins.yaml (or, on a fixture, in goatbuilder.yaml). | a typo in the component name the component is declared under its ledger name, not its image name |
grep -n '^ [a-z0-9-]*:' ops/pins/fleet-pins.yaml \| head -60goatbuilder validate |
docs/design/goatbuilder.md |
2 |
GB-002 |
goatbuilder | unknown goatnet — the --env value is not a goatnet key the pins declare. A key is spelled exactly as the pins spell it, underscores included. | the goatnet was typed with a hyphen where its key has an underscore | goatbuilder deploy --env <goatnet> …goatbuilder doctor --all (lists every environment) |
docs/design/goatbuilder.md |
2 |
GB-003 |
goatbuilder | the group is not one the component deploys to — --group named a group that is not a member of the goatnet, or one where the component is declared absent. | the group belongs to another goatnet the pins mark the component absent: true for that group |
goatbuilder deploy --env <env> --component <c> --dry-run (prints the member groups) |
docs/design/goatbuilder.md |
2 |
GB-004 |
goatbuilder | the component declares no image variable — the pins entry has no var: so there is nothing to template into a play. |
an incomplete pins entry | addvar:to the component in ops/pins/fleet-pins.yaml, then: goatbuilder validate |
docs/design/goatbuilder.md |
2 |
GB-005 |
goatbuilder | no deploy playbook resolves for the component — neither |
the component is deployed by another play and does not declare it the playbook was renamed |
declare deploy_playbook: <play>.yml on the component in ops/pins/fleet-pins.yamlgoatbuilder doctor --all (the playbook row names what was looked for) |
docs/design/goatbuilder.md |
2 |
GB-006 |
goatbuilder | the door to the goatnet did not open — the goatnet's management host was not reachable over the mesh, so the SSM door was tried, and it did not open. | no AWS credentials in the keychain for aws-vault (the profile in deploy.aws_profile) the SSM agent on the management host is not connected (TargetNotConnected) the door registry row is stale |
ops/scripts/door-open.sh <goatnet> --statusaws-vault exec <profile> --no-session -- aws sts get-caller-identity |
docs/design/env-doors.md |
1 |
GB-007 |
goatbuilder | a secret the play needs is not in your environment — the play templates one or more variables from the environment and they are not set, so the run was refused before any host was touched. | the refs file for this component was never written the command was run without op run |
put the lines for the named variables from <owner>/goat-credentials refs/goat.refs into .envrc.local.d/<file>.refsOP_ACCOUNT=<account> op run --env-file=.envrc.local.d/<file>.refs -- <the same command> |
docs/design/secret-store-requirements.md |
1 |
GB-008 |
goatbuilder | a mutation needs --yes when there is no terminal — deploy, release, rollback, fastlane, standup, teardown, join-node, federate, apply and grow change state; without a terminal to confirm on, they refuse unless --yes is passed (contract C10). grow asks only when stdin and stderr are both a terminal, so with stderr sent to a file it needs --yes too. | run from a script, CI or a pipe | add --yesadd --dry-run to see the plan without changing anything |
docs/design/goatbuilder.md |
2 |
GB-009 |
goatbuilder | not a current main checkout — releases and deploys run from the invoking checkout, and only a main checkout kept current with origin carries the pin that was merged. A branch or detached checkout can deploy a stale pin and verify green. | running from a feature-branch worktree running from a detached HEAD main is behind origin/main |
git checkout main && git pull --ff-only origin maingoatbuilder doctor --all (the checkout row) |
docs/design/release-candidate-validation.md |
1 |
GB-010 |
goatbuilder | a Python dependency is missing — the pins and intents render through PyYAML and jsonschema, and python3 cannot import one of them. | a fresh machine a venv that is not on PATH |
pip install pyyaml jsonschema (or pipx/venv), then re-run; re-running is safe |
docs/design/goatbuilder.md |
2 |
GB-011 |
goatbuilder | only a repository admin may nominate a release candidate — the GitHub login in use does not hold the admin role on the repository; the rc-validate workflow deletes a tag pushed by anyone else (ADR 1173). | gh is logged in as a different account the account is a maintainer, not an admin |
gh auth statusask a repository admin to push the tag, or to grant the role |
docs/design/release-candidate-validation.md |
1 |
GB-077 |
goatbuilder | this checkout's origin is not where release candidates are cut — the repository that origin names does not run rc-validate.yml, so a tag pushed there is never validated or built where the ladder reads images; the run refuses before the tag (ADR 1173). | the checkout is a clone of a mirror or a fork, not the primary repository | git remote get-url originrun from a checkout whose origin is the primary repository |
docs/design/release-candidate-validation.md |
1 |
GB-078 |
goatbuilder | no release signer is known, so the update refuses — goatbuilder update checks SHA256SUMS against the repository whose release workflow signed it. It found none: the working directory is in no checkout whose ops/goatbuilder/goatbuilder.yaml names repo.release_source or repo.source, and this binary was built outside a release workflow, so it names no publisher itself. Nothing was installed. |
goatbuilder update ran outside a deployment checkout with a locally built binary | goatbuilder -C <checkout> update (a checkout whose goatbuilder.yaml names repo.release_source)install a goatbuilder from a release, which carries its own publisher |
docs/design/goatbuilder-distribution.md |
1 |
GB-265 |
goatbuilder | the run is waiting on the other driver — The peering is declared with two drivers (drivers: two): an edge driver that reaches only the edge host and a controller that reaches the core. This phase needs the edge, so the controller stopped before acting and handed it to the edge driver. The stop prints the exact command to run there, the file it writes, and the two commands to run here after it; with -o json the report carries them as the member handoff. It stops as a by-hand stop does (exit 0). |
the peering's phase needs the edge host, and this machine is the controller | run the command the stop names on the edge driver, bring its result file here, then run the import and the command after it that the stop names |
docs/design/federation-two-drivers.md |
0 |
GB-266 |
goatbuilder | the controller's checkout is not clean, so no hand-off is given — A two-driver peering's hand-off pins the edge driver's checkout to the controller's commit, so the controller gives one only when its own checkout is a git checkout with nothing uncommitted. The run stopped at the phase that needs the edge, before acting, and names each file with changes. | phase 10 placed rendered files, or admission wrote peerings.yml, and they are not committed yet a file was edited by hand and not committed the deployment's directory is not a git checkout |
commit each file the refusal names, then run the same command again |
docs/design/federation-two-drivers.md |
1 |
GB-269 |
goatbuilder | a secret for the node still holds a secret store's reference — A value join-node hands to a node, such as the first-login password phase 2 seeds the SSH key with, still begins with a secret store's reference (op://, bao://, bw:// or the declared ref_prefix) after the secrets injector ran. Sent as it is, the node would take the reference as the secret. Nothing was sent to the node. The message names the variable and the scheme, never the reference. | the store's tool is not signed in, so with-secrets could not resolve the reference the variable was set to a reference under a store that resolves none (env, env-file) |
sign in to the declared store's tool (for 1password: op signin), then re-run: goatbuilder join-node <environment> --site <site> --node <node> --class <class> --address <address> --from 2or export the value itself in the variable the message names |
docs/design/goatbuilder.md |
1 |
GB-267 |
goatbuilder | this part of a two-driver peering is not built yet — A peering declared with two drivers (drivers: two) is handed off between the edge driver and the controller only at phases 1, 2 and 3, where the hand-off is built. These cases are refused before anything runs: the edge driver's own run (goatbuilder federate <partner> --side edge) of a phase other than 1 or 3; the controller's goatbuilder federate import of a phase other than 1 or 3; a two-driver run that would reach a phase after 3 that needs the edge (11, 12, 13 or 14), whose hand-off is designed and not built; and a two-driver run from phase 3 with FED_MEASURE=here, which measures one machine and so has no halves to hand off. That includes every goatbuilder federate deploy, which runs phase 10 then 11: it is refused before its plan is approved or rechecked, so phase 10 never admits the edge on our core. |
goatbuilder federate goatbuilder federate import was run for a phase other than 1 or 3 |
run the peering with one driver for now: remove drivers: two from its peerings: entry, from a machine that reaches both the edge host and the core hostfor phase 3: unset FED_MEASURE (or set it to ssh), so phase 3 measures in halves over SSH |
docs/design/federation-two-drivers.md |
1 |
GB-271 |
goatbuilder | the peering's WireGuard interface does not hold the agreed peer and address — Phase 11 writes the agreement's link values (the counterparty's public key, its allowed address and our link address) to the link's environment file, restarts goat-peering-wg@wg show <interface> allowed-ips and ip -4 addr show dev <interface>. The read-back did not find them, so the link the agreement describes is not up on this edge. The end of the message lists what the interface holds: each peer's public key and allowed addresses, then its addresses. No peer and no address is the interface as it is before phase 11. |
the link unit is still running with the environment it had before phase 11: the agreement's peer and address were written to the link's environment file, but goat-peering-wg@ the unit was restarted and goat-peering-wg-link up stopped partway: its journal says where something other than this role changed the interface after the unit set it up (another peering's unit, or wg set by hand) |
on the edge: sudo systemctl restart goat-peering-wg@<interface>.service, then sudo wg show <interface> and ip -4 addr show dev <interface>: the agreed peer and link address are thereif they are not: sudo journalctl -u goat-peering-wg@<interface>.service -n 50goatbuilder federate <peering> --from 11 |
docs/operations/sovereign-federation-runbook.md |
1 |
GB-272 |
goatbuilder | the partner's acceptance result does not verify — In a bilateral acceptance session each side signs its phase 14 result and the other verifies it under the key bound to that side at phase 5, the same key that signs its federation package. The signature document beside the partner's result did not verify: it is malformed, it names another key, the result read is not the one it signs, or the signature does not verify under the bound key. Nothing from it is combined (docs/design/federation-acceptance.md §6.1). | the result or its signature document was altered, or the result was read before the partner finished writing it the partner signed with a key other than the one bound to them at phase 5 (their federation key was rotated, or the wrong key file was used) the signature document is not goat-acceptance-signature/v1, or a member is missing or malformed |
check with the partner, on the call, the key fingerprint confirmed at phase 5 and the one their signature document namesask the partner to run phase 14 again in the same session, so a new result and signature are published |
docs/design/federation-acceptance.md |
2 |
GB-273 |
goatbuilder | the partner's acceptance result is not from this session — The partner's signed acceptance result is for another agreement, session or attempt, is signed by or for another party, or does not echo the challenge issued to us, so it cannot be from this session's samples. It is refused before its signature is checked, and nothing from it is combined (docs/design/federation-acceptance.md §6.1). | an earlier session's or attempt's result was read again (a replay, or a result left from before) the two sides hold different agreements: the digests read aloud at phase 6 differ our own result was read back as if it were the partner's |
compare the agreement digest and the session id with the partner on the callrun phase 14 again on both sides in one session |
docs/design/federation-acceptance.md |
2 |
GB-274 |
goatbuilder | the two sides derived different acceptance sessions — Both sides of a peering work out the same acceptance session from the agreement each holds and an attempt number, with no file between them (docs/design/federation-acceptance.md §6.2). The partner's session is not the same as ours: the refusal names the members that differ. Nothing is combined with a partner on another session. | the two sides hold different agreements: the digests read aloud at phase 6 differ, or one side composed again since the two sides are on different attempts a session file was edited, so its digest no longer covers its members |
compare the agreement digest and the attempt number with the partner on the callrun phase 14 again on both sides with the same attempt |
docs/design/federation-acceptance.md |
2 |
GB-275 |
goatbuilder | phase 12 would deploy an edge that has none of the files it mounts — Phase 8 mints the peering leaves, and places our CA beside them, in |
goatbuilder federate runs on an operator's workspace, not on the edge host, and the edge group's install of the peering certificates is off the inventory names this machine by an address of its own, not as ansible_connection=local |
set goat_zenoh_edge_install_certs: true for the edge group (the role then installs the leaves, the partner's CA and our CA's certificate from this machine, never a CA key); run goatbuilder federate allocate again when the deployment is rendered; then goatbuilder federate <peering> --from 12if this machine is the edge host, mark it ansible_connection=local in the inventory |
docs/design/federation-certificate-lifecycle.md |
1 |
GB-285 |
goatbuilder | the directory binds our previous signing key, and the seed that hands it over is missing — Phase 5's directory step publishes our pointer signed by the key that signed our package. The directory binds our sovereign_id to the first key that wrote it, and takes a new key only through the entry, when the request is signed by the key it replaces. The directory binds another key than ours: our signing key was rotated, and FED_PREVIOUS_SIGNING_KEY is not set, is not readable or holds the seed of a third key, or the pointer names another of our peerings' packages. Nothing was published; the directory still serves what it held. The step is advisory, so phase 5 goes on. | our signing key was rotated (FED_SIGNING_KEY names the new seed) and FED_PREVIOUS_SIGNING_KEY was not set FED_PREVIOUS_SIGNING_KEY names a seed other than the one the directory binds (the message names both keys) with one package per counterparty, the pointer names another peering's package, and the key is handed over only through the pointer |
FED_PREVIOUS_SIGNING_KEY=<file of the seed that signed our last package> goatbuilder federate <partner> --from 5with several peerings, run phase 5 of the peering whose package the pointer names first; the others then publish under the new keyif the previous seed is lost, the directory's operator clears the entry (saltlick clear), and the next write claims it again |
docs/design/federation-key-rotation.md |
0 |
GB-261 |
goatbuilder | the release's signature does not verify, so the update refuses — goatbuilder update verifies the keyless signature over the release's SHA256SUMS (SHA256SUMS.cosign-bundle) itself, offline, against the Sigstore trusted root: the certificate chains to a Fulcio CA, Rekor's signed entry timestamp verifies, the entry names this file, and the certificate is the release signer's goatbuilder-release.yml at the served version's goatbuilder/v |
the surface serves a SHA256SUMS or a binary that the release workflow did not sign the signature is another repository's, another workflow's, or another version's Sigstore rotated its Fulcio CA or Rekor key after this goatbuilder was built the surface serves no SHA256SUMS.cosign-bundle |
check that the surface serves the release unchanged: the goat-get role's goat_get_goatbuilder_release_tag, and the release page's SHA256SUMSafter a Sigstore key rotation: GOATBUILDER_UPDATE_TRUSTED_ROOT=<a trusted_root.json holding the new key> goatbuilder updategoatbuilder -C <checkout> update, so that repo.release_source names the repository that publishes goatbuilder |
docs/design/goatbuilder-distribution.md |
1 |
GB-262 |
goatbuilder | the verb has no such phase — goatbuilder explain <verb> <phase> answers from the phase catalog (ops/phases/catalog.yaml), by the phase's number or its id. The verb has no phase by that number or id. The message names the phases it has. |
a phase number outside the verb's range, or a mistyped id | goatbuilder explain <verb> (lists the verb's phases, by number and id) |
docs/design/goatbuilder-legibility.md |
2 |
GB-012 |
goatbuilder | the candidate's image build did not succeed — after the tag was pushed, the image workflow did not appear, or ran and failed, so there is no image to validate. | the workflow is not triggered by this tag shape a build failure on the tagged commit |
gh run list --workflow <component>-image.yml --limit 5gh run view <id> --log-failed |
docs/design/release-candidate-validation.md |
1 |
GB-013 |
goatbuilder | the image is not in the registry — the tag's image was not found under repo.image_namespace, so no digest can be pinned. | crane is missing or unauthenticated — rc.sh resolves the digest with crane and discards its error, so a missing crane reads as the image not existing the build published under another name the package is private and the token cannot read it |
brew install crane (then re-run; goatbuilder doctor --all: the tool:crane row says when it is present)crane digest <repo.image_namespace>/<image>:<tag>gh api /orgs/<owner>/packages/container/<image>/versions \| head |
docs/design/release-candidate-validation.md |
1 |
GB-014 |
goatbuilder | a live validation row failed — the candidate deployed and a live row on the goatnet failed. On dev the candidate is dead; on preprod after a dev pass it stays open for investigation. | a row exited 2 — it was never run: its toolchain is missing on this laptop (Playwright for the intake rows, a goat context for the invite rows); the candidate is untested, not dead a real regression in the candidate the row's runner settings are stale for this goatnet |
if the row exited 2: install what it names (goatbuilder doctor --browser-tests fails on a missing browser toolchain, tool:live-rows; tool:goat-contexts names a missing goat context), then resume with --from rows — do not delete the tagops/validation/flow-map.yaml names the rows; read the failed row's evidence under ops/validation/evidence/fix on main and cut the next rc (dev), or investigate before graduating (preprod) |
docs/design/release-candidate-validation.md |
1 |
GB-015 |
goatbuilder | the checkout could not fast-forward after the pin landed — the pin PR merged, and the local main has diverged from origin/main, so the deploy could not proceed from a current tree. The PR is in; re-running resumes. | local commits on main | git -C <repo> status; move local commits to a branch; git pull --ff-only origin main; re-run the same command |
docs/design/release-candidate-validation.md |
1 |
GB-016 |
goatbuilder | no runner settings for the goatnet — ops/validation/goatnet-runners.yaml has no entry for the goatnet the live rows should run on. | a new goatnet without its runner row | add the goatnet's runner block to ops/validation/goatnet-runners.yaml |
docs/design/release-candidate-validation.md |
1 |
GB-017 |
goatbuilder | not converged on every host — the play ran, and at least one host's ledger does not name the declared digest with action=deploy. Done is read from the ledger, never assumed. | the play failed part-way on that host the host is single-homed on another router role and was skipped by design (ADR 1084) the host was unreachable |
goatbuilder deploy --env <env> --component <c> --verify-onlyread the log named in the deploy row; grep -n 'fatal:' <log> |
docs/design/universal-deploy-rollback.md |
3 |
GB-018 |
goatbuilder | a required flag is missing or wrong — release needs --ack and --version or --tag; --from takes tag|dev|rows|preprod|prod; --from prod needs --tag. | the command was typed from memory | goatbuilder release --help |
docs/design/goatbuilder.md |
2 |
GB-019 |
goatbuilder | the goatnet's CA key is not on this machine — re-signing the intents needs |
this laptop never held that goatnet's CA key | run from the machine that holds it, or obtain it through the credentials runbook in <owner>/goat-credentials |
docs/design/consistency-foundation.md |
1 |
GB-020 |
goatbuilder | graduate.py did not produce the expected record — graduation ran, and wrote no record, or the record names a different candidate_ref than the tag resolves to. | an open graduation set blocking the move (ADR 1125) a candidate record edited by hand |
python3 ops/release/graduate.py statusread the record under ops/release/records/ |
docs/design/release-graduation.md |
1 |
GB-021 |
goatbuilder | fastlane refused — fastlane is the deliberate exception to the ladder and refuses an unpinned digest, an open graduation set, or an artifact whose rollback is not a pin revert. | the ref has no digest under repo.image_namespace a graduation set is open for the component |
goatbuilder fastlane <component> --ref <ref> --dry-runpython3 ops/release/graduate.py status |
docs/design/release-graduation.md |
1 |
GB-022 |
goatbuilder | intent-sign failed — re-signing a goatnet's intents after a pin move failed; the signed index is unchanged. | the CA key passphrase was wrong or age could not decrypt it the enrollment Go module is not at ops/enrollment |
cd ops/enrollment && go run ./cmd/intent-sign … (the fix line names the goatnet) |
docs/design/consistency-foundation.md |
1 |
GB-023 |
goatbuilder | rollback refused or guided by artifact type — the component's artifact type is data, cert, forward-only or reversible:false (refused, exit 3), or config/acl (guided, exit 4): its rollback is not a pin flip. | the component carries a migration or a certificate | goatbuilder rollback <component> --statusfollow the guided steps the row prints (config/acl) |
docs/design/universal-deploy-rollback.md |
3 |
GB-024 |
goatbuilder | the fixture is not a shape the mock converger knows — on --mock, converger.yaml names a behaviour the scripted converger does not know, a record is unreadable, or nothing is declared for the environment. | a hand-edited fixture | compare with ops/goatbuilder/fixtures/basic/ |
docs/design/goatbuilder-conformance.md |
2 |
GB-025 |
goatbuilder | the config does not validate — ops/goatbuilder/goatbuilder.yaml failed the versioned schema or a cross-field rule. The config is generated; fix the source and regenerate. | a hand edit to the generated file a pins or ladder change the generator was not run for |
python3 ops/goatbuilder/gen-config.py && goatbuilder validate |
docs/design/goatbuilder-conformance.md |
1 |
GB-026 |
goatbuilder | status reads real hosts only through a converger — without --mock, status has no converger to read hosts through yet. | the push converger's status path is not built | goatbuilder deploy --env <env> --component <c> --verify-onlygoatbuilder status --mock ops/goatbuilder/fixtures/basic --env dev |
docs/design/goatbuilder-conformance.md |
2 |
GB-027 |
goatbuilder | not inside a repository — the repository root could not be resolved from the working directory and -C was not given. | run outside a checkout | goatbuilder -C /path/to/goat … |
docs/design/goatbuilder.md |
2 |
GB-028 |
goatbuilder | a command the verb runs failed — git, gh, crane, python or another tool exited non-zero; its own words follow and are interpreted below when the catalog knows them. | see the tool's message | re-run the quoted command by hand to see its full output |
docs/design/goatbuilder.md |
1 |
GB-029 |
goatbuilder | the play failed on a group — ansible-playbook exited non-zero for the group; the log path is in the row. | a fatal task on a host (see the ansible entries below) a host unreachable over the chosen transport |
grep -n 'fatal:' <log> \| headgoatbuilder doctor --env <env> |
docs/design/goatbuilder.md |
1 |
GB-030 |
goatbuilder | no host of the group is in the loaded inventories — the group is declared in the pins but the inventories loaded for this goatnet carry no host for it, so nothing was deployed (declared, not built). | a site declared ahead of its hardware the wrong inventory pair for this goatnet |
grep -n '^\[<group>\]' ops/ansible/inventory/*.ini |
docs/design/env-doors.md |
1 |
GB-031 |
goatbuilder | an op:// reference in your shell is incomplete — op run validates every environment variable whose value starts with op://, not only the env-file, and refuses the whole run when one has fewer than vault/item/field. |
a variable exported from your shell profile for another project holds a two-part op:// reference | env \| grep 'op://' (find the variable)env -u <THAT_VAR> op run --env-file=… -- <command>or complete the reference to op://<vault>/<item>/<field> |
docs/design/secret-store-requirements.md |
1 |
GB-032 |
goatbuilder | the candidate record is open but the goatnet runs a different ref — graduate.py's C6 check: a record for this component is still open while the goatnet now runs another ref — the record the preprod step superseded was never abandoned. | a release run on a checkout older than #2039, which abandons the superseded record in the preprod change a candidate re-cut while the previous record stayed open |
re-run the same command from a current main: rc.sh andgoatbuilder releaseabandon the superseded record in the preprod change (git pull --ff-only origin main first on an older checkout)python3 ops/release/graduate.py status (what is open) |
docs/design/release-candidate-validation.md |
1 |
GB-033 |
goatbuilder | promote refused a graduation step — goatbuilder promote (the release ladder, ADR 1120/1125) refused: the record, the set, the ladder or the ref does not meet the rule the step names. Nothing graduates by itself; a refusal changes nothing. |
the component is not ready (soak, live rows, convergence evidence) — the lines under the refusal name each blocker the component is a member of an open graduation set and must graduate with it a ref that is not digest-pinned (…@sha256:…), or a hotfix without --ack no open record for the component, or the record named does not exist or already graduated |
goatbuilder promote status (what is open, soaking, ready)goatbuilder promote readiness <component> (each blocker by name)goatbuilder promote graduate-set <set> --ack '<name>' (when it is a set member) |
docs/design/release-graduation.md |
2 |
GB-034 |
goatbuilder | the generated pins or intents differ from fleet-pins.yaml — goatbuilder pins --check regenerated every member's pins.yml and every goatnet's intent from ops/pins/fleet-pins.yaml and they differ from what is committed, or an intent is unsigned or tampered, or a governed pin is defined outside pins.yml. |
fleet-pins.yaml changed without regenerating a hand edit to a generated pins.yml or intent |
goatbuilder pins (regenerate), then re-sign the intents it names, then commit |
docs/design/consistency-foundation.md |
1 |
GB-035 |
goatbuilder | the goatnet's intent names no usable CA — an intent has no ca_id, or one from which the CA's directory under |
a new goatnet whose intent was written without ca_id | add ca_id to the goatnet in ops/pins/fleet-pins.yaml, then: goatbuilder pins |
docs/design/consistency-foundation.md |
1 |
GB-036 |
goatbuilder | the component declares no image workflow — release finds a candidate's image build by the component's image_workflow in fleet-pins.yaml, and there is none. | a component added without image_workflow | add image_workflow: <name>-image.yml to the component in ops/pins/fleet-pins.yaml |
docs/design/release-candidate-validation.md |
2 |
GB-037 |
goatbuilder | the ladder is missing a live rung — ops/release/ladder.yaml has no live rung with a goatnet where the step needs one, or does not read. | a rung marked not live while a goatnet is rebuilt a malformed ladder.yaml |
goatbuilder promote status (the rungs as read)python3 -c 'import yaml; yaml.safe_load(open("ops/release/ladder.yaml"))' |
docs/design/release-graduation.md |
1 |
GB-038 |
goatbuilder | --from preprod-rows cannot resume this candidate — resuming at the preprod rows needs preprod to run the candidate on main and exactly one open record for it; one of the two is missing. | the preprod pin was never landed for this candidate the record was closed, or a second one opened |
goatbuilder release <component> --tag <tag> --from preprod --ack '<name>' (pins preprod and opens the record) |
docs/design/release-candidate-validation.md |
1 |
GB-039 |
goatbuilder | a rollback on a fixture needs its environment — on a fixture (--mock) a rollback declares the earlier ref in one environment and converges it there, so it needs --env; --list and --status read host ledgers, which a fixture does not have. | --mock given without --env --list or --status given with --mock |
goatbuilder --mock <fixture> rollback <component> 1 --env <env> --ack "<your name>" --dry-rungoatbuilder --mock <fixture> status --env <env> (the state a fixture has) |
docs/design/goatbuilder-conformance.md |
2 |
GB-040 |
goatbuilder | that directory is not a tutor workspace — the tutor creates its workspace in an empty or new directory and deletes only a directory that carries its .goatbuilder-tutor marker; it refused to use or delete one that does not. | --workspace (or $GOATBUILDER_TUTOR) points at a directory with other files in it | goatbuilder tutor --workspace <an empty or new directory>unset GOATBUILDER_TUTOR (the default is ~/.goatbuilder/tutor) |
docs/design/goatbuilder.md |
2 |
GB-041 |
goatbuilder | there is no such lesson — the tutor was asked for a lesson number it does not have. | a number past the last lesson | goatbuilder tutor --list |
docs/design/goatbuilder.md |
2 |
GB-042 |
goatbuilder | the live rows cannot run from this machine — the candidate is UNTESTED, not dead — a release step reached the live acceptance rows, and this machine lacks what they need (the console's Playwright toolchain for a browser row, or a tool or context the runner reported with exit 2). The rows did not run; nothing about the candidate failed. | a browser row is mapped and ops/goat-id-console/web/e2e/node_modules is missing, or node/npm is not installed the acceptance runner exited 2 (a tool or a goat context missing on this machine or on the host it runs goat on) |
(cd ops/goat-id-console/web/e2e && npm ci && npx playwright install chromium), then re-run the same commandread the runner's output above for the missing tool or context, fix it, re-run the same command — never delete the tag |
docs/design/release-candidate-validation.md |
1 |
GB-043 |
goatbuilder | the door to the rung did not open for its live rows — the rows for a rung this laptop is not meshed to run goat on that rung's management host through its door (goat_via door: |
aws-vault cannot reach the declared AWS profile (signed out, a malformed credential, or an expiring session) session-manager-plugin is not installed |
ops/scripts/door-open.sh <env> --status, then ops/scripts/door-open.sh <env>aws-vault exec <profile> --no-session -- aws sts get-caller-identity |
docs/design/env-doors.md |
1 |
GB-044 |
goatbuilder | a release refuses before the tag because a secret is not set — every rung this run will deploy to templates the component's secrets from your environment, and at least one is not set. Nothing was tagged, pinned or deployed. | the run was not wrapped in op run with a .refs file naming those variables goat-credentials refs/goat.refs has no line for a variable (ops/ci/check-secrets-coverage.py names it) |
put the reference lines for the NOT SET names in .envrc.local.d/<file>.refs, then OP_ACCOUNT=<account> op run --env-file=.envrc.local.d/<file>.refs -- <the same command> |
docs/operations/goatbuilder-first-run.md |
1 |
GB-045 |
goatbuilder | the published declarations are out of order or altered — a declaration is applied only from an intact, ordered log (contract C2). A row whose signature does not match its content, or a serial not greater than one already accepted, is refused before anything is published, so a replayed or edited declaration never reaches a machine. A new serial at an earlier digest is accepted; that is a revert. | a row of declared/publications.jsonl was edited by hand after it was published an older copy of the log was restored over a newer one, or a row was replayed |
restore declared/publications.jsonl from the source of recordto go back to an earlier ref, declare it anew: goatbuilder --mock <fixture> rollback <component> 1 --env <env> --ack "<your name>" |
docs/design/goatbuilder-conformance.md |
1 |
GB-046 |
goatbuilder | the local declaration is not the source of record's — a deploy applies only what the source of record declares (contract C3). A local copy behind it is fast-forwarded first, or refused under --frozen; a local copy ahead of it or diverged from it is always refused, because a declaration that is not in the source of record is never applied. | the checkout or the fixture's declared/ was not fast-forwarded and --frozen was passed a declaration was edited locally instead of in the source of record |
fast-forward, then re-run: cp source/pins.yaml declared/pins.yaml (on a repository: git pull --ff-only)make the change in the source of record and re-run |
docs/design/goatbuilder-conformance.md |
1 |
GB-047 |
goatbuilder | a declaration must name who made it — every declaration, and every record it converges, carries the person who made it (contract C4). A deploy, rollback or standup run without --ack is refused before anything is published. | --ack was not passed, or was empty | re-run with --ack "<your name>" |
docs/design/goatbuilder.md |
1 |
GB-048 |
goatbuilder | the goatnet is not proven built — the acceptance assay ran and at least one applicable row failed, or could not answer, or no row passed. Only PASS counts toward done; an UNKNOWN row is a row nobody proved (ADR 1115). | a FAIL row is the goatnet failing that check; its caught_would_have text says what it means an UNKNOWN row could not be probed from here, most often a goat CLI row run where no goat context with a fabric identity exists (E_PROFILE_MISSING) a SKIPPED negative row whose paired positive did not pass |
goatbuilder assay <site> --only <row-id> (re-run one row after a fix)goatbuilder assay <site> --var goat_via=<user@host> (run the goat CLI rows where a goat context is)goatbuilder assay <site> --plan (which rows apply to this site) |
docs/design/goatnet-acceptance-suite.md |
3 |
GB-049 |
goatbuilder | the acceptance assay cannot run here — assay derives the site's values from its group_vars and runs ops/validation/run-acceptance.py with python3; one of those could not start, so nothing was probed. | the site name has no ops/ansible/group_vars/ python3 or PyYAML is not installed the runner stopped with an error rather than a verdict |
ls ops/ansible/group_vars/ (the site names this repository declares)pip3 install pyyamlpython3 ops/validation/run-acceptance.py --self-test |
docs/design/goatnet-acceptance-suite.md |
1 |
GB-050 |
goatbuilder | nothing is declared on that goatnet — diff or adopt found no fleet-pins component whose member groups include the goatnet. | the goatnet key is right but no component is pinned there yet every component is absent on it |
goatbuilder doctor --env <goatnet> --all (lists what the goatnet runs) |
docs/design/goatbuilder.md |
2 |
GB-051 |
goatbuilder | the plan request is incomplete or not planned yet — plan writes deploy and rollback plans; a deploy plan needs --env, and a rollback plan reads a host ledger, so it has no fixture form. | plan deploy without --env a verb other than deploy or rollback plan rollback with --mock |
goatbuilder plan deploy <component> --env <goatnet> --out <file>goatbuilder plan rollback <component> --n 1 --out <file> |
docs/design/goatbuilder-conformance.md |
2 |
GB-052 |
goatbuilder | that file is not a goatbuilder plan — apply runs only a file plan wrote, which carries schema goatbuilder.plan/v1. | the wrong file was passed to apply the plan was edited by hand and its schema line lost |
goatbuilder plan <verb> <component> … --out <file> (write it again) |
docs/design/goatbuilder-conformance.md |
2 |
GB-053 |
goatbuilder | the plan no longer matches the tree — apply recomputed the plan's decisions and at least one moved since it was written; it names each, and runs nothing. | main's declaration or the pins file changed since the plan a newer deploy or rollback moved the rollback target the plan is old |
goatbuilder plan <same arguments> --out <file>, review it, then goatbuilder apply <file> |
docs/design/goatbuilder-conformance.md |
1 |
GB-054 |
goatbuilder | adopt could not create its worktree — adopt --yes writes its declarations into a fresh worktree of origin/main beside the checkout; git refused to create it. | origin/main is not fetched a branch or directory of that name already exists |
git fetch origin maingit worktree list (remove a stale adopt worktree) |
docs/design/goatbuilder.md |
2 |
GB-055 |
goatbuilder | a rollback with no --site would flip every goatnet — the component is deployed on more than one goatnet and no --site was given; with no site the roll narrows to nothing, so it would flip every goatnet at once, prod included. Nothing was touched; the error lists the goatnets to choose from. | rollback (or plan rollback) was run on a component that spans several goatnets without --site the goatnet to roll back was left off the command |
goatbuilder rollback <component> <N> --site <goatnet> (one of the goatnets the error names)ops/scripts/rollback.sh <component> <N> --site <goatnet> |
docs/design/universal-deploy-rollback.md |
1 |
GB-060 |
goatbuilder | a standup input is required and not set — A role or play the site's standup will reach asserts this variable, or reads it through \| mandatory, and nothing the site supplies sets it: not group_vars, the inventory, host_vars, the play's vars, a role default, nor an extra var the plan passes. The scan lists every such input at once, before any host is touched. |
the site's config surface was made from a reference goatnet and this key was not carried over the role gained a required input after the site was configured |
set the variable in ops/ansible/group_vars/<group>/ (or the inventory)goatbuilder doctor --site <site> (re-scan; every gap is listed) |
docs/design/goatbuilder-speed.md |
1 |
GB-061 |
goatbuilder | a secret the standup templates is not in this environment — A value the site supplies is lookup('env', NAME), a play or role the standup reaches uses it, and the declared secrets posture cannot supply NAME (ops/secrets/posture.py, ADR 1204). | the run is not under the site's secrets posture the posture's references file has no line for this name |
run under your secrets posture: ops/scripts/with-secrets goatbuilder standup <env> --site <site>add the reference for NAME to your .refs file |
docs/design/pluggable-dependencies.md |
1 |
GB-062 |
goatbuilder | a standup input may be required — The input is required under a when: condition the scan cannot evaluate from the site's own values (it depends on a registered result, a fact, or a template). This is a warning, never a refusal: the scan does not claim more than it knows. |
the condition reads a value only known when the play runs | read the role at the file named and decide whether the condition will hold for this siteset the input anyway if it will |
docs/design/goatbuilder-speed.md |
0 |
GB-063 |
goatbuilder | a standup step's host limit selects no host — The plan runs this play with --limit |
the limit assumes the mgmt- the step installs a service this joining site does not run yet, which its end state names as a gap with the gate row that closes it |
goatbuilder endstate <site> names each gap with its gate row (I1, I4 for an admin surface); a gap is work still to build, not a reason to skip the stepif it should, fix the limit in goat-standup.sh and goatbuilder standup together |
docs/design/goatbuilder-speed.md |
0 |
GB-064 |
goatbuilder | the checkout is behind the trunk in what a deploy reads — A real run reads ops/ansible (roles, plays, group_vars, inventory, the rendered pins) and ops/pins from the checkout it runs in. The trunk carries commits to those paths that this checkout lacks, so a file rendered from it would remove on the host what those commits added, and the play would report success (contract C3, F-410). Refused before any host is touched, in both avenues (ops/scripts/fresh-source.sh). | the checkout was not brought onto the trunk after a merge a feature branch was cut before a change to a role, group_vars or the pins landed |
on main: git pull --ff-only <trunk remote> mainon a branch: git fetch <trunk remote> main && git rebase <trunk remote>/main |
docs/design/goatbuilder-conformance.md |
1 |
GB-065 |
goatbuilder | the trunk could not be fetched, so the checkout cannot be shown current — A real run proves its checkout is current with the trunk before any host is touched (contract C3). The trunk remote did not answer a fetch, so that proof is missing and the run is refused. An islanded site, cut off by design, runs with --offline: it is judged against the cached trunk and the ledger row it writes says source: offline. | no route to the git remote (the laptop is offline, the forge is down) the remote's credentials are not loaded (an ssh key not in the agent) |
restore the connection to the trunk remote and re-runon an islanded site: re-run with --offline |
docs/design/goatbuilder-conformance.md |
1 |
GB-066 |
goatbuilder | the site's declared end state cannot be derived or read — endstate derives what a site should run from its own declarations (group_vars, the fleet pin table, the inventory) and the service catalog, then reads the site's hosts once over Ansible. One of those could not start, so nothing was graded. | the site name has no ops/ansible/group_vars/ python3 or PyYAML is not installed on this machine the catalog file ops/goatbuilder/endstate/catalog.yaml does not parse |
ls ops/ansible/group_vars/ (the site names this repository declares)python3 ops/goatbuilder/endstate/endstate.py model <site> (the derived end state, without reading any host)python3 ops/goatbuilder/endstate/endstate.py --self-test |
docs/design/goatnet-end-state.md |
1 |
GB-067 |
goatbuilder | a site's service presence is undeclared or contradicted — Every service the catalog lists must be declared present or absent for every governed site, by exactly one answer: a flag in the site's group_vars, the fleet pin table (an absent deviation), or the founder rule. The baseline ops/goatbuilder/endstate/baseline.txt lists the gaps that exist today and may only shrink: a new gap, or a baseline row that is no longer true, fails. | a new service in the catalog with no flag declared for a site the fleet pin table renders a component the site's flag says it does not run (or the reverse) a gap was closed and its baseline row was left behind |
set the service's flag (its catalogvar:) in ops/ansible/group_vars/<site>_hosts/vars.ymladdabsent: truefor the member in ops/pins/fleet-pins.yaml, then regenerate and re-sign the pinsdelete the stale row from ops/goatbuilder/endstate/baseline.txt |
docs/design/goatnet-end-state.md |
1 |
GB-068 |
goatbuilder | the goatnet is not in its declared end state — endstate graded every service the site declares: present or absent, deployed (its ledger names the declared ref), active (every container running), healthy (its healthcheck or isitup probe), functional (the acceptance rows that exercise it), and its zero-trust edges (its served name resolves and its certificate verifies, its OIDC client is registered in Hydra, its SPIRE entry exists). At least one grade failed or could not be read, or something runs that the end state does not declare (an orphan). | a declared service is not running, or its ledger is pending, rolled back or at another ref an edge is broken, such as an OIDC client never registered or a name that does not resolve an orphan, such as a container left behind by a retired service, or a Hydra client or SPIRE entry no service declares a grade could not be read, most often because a host was unreachable or the assay did not run |
goatbuilder endstate <site> (each failing grade with its reason)goatbuilder deploy --component <c> --env <goatnet> (restore a declared service)goatbuilder endstate <site> --json (the grades as data) |
docs/design/goatnet-end-state.md |
3 |
GB-069 |
goatbuilder | the rollout preflight cannot run — preflight reads the component's declared rollout facts from ops/pins/fleet-pins.yaml (owns_state, rollback, restore_from, coupled_with) and the open graduation sets. The component is not in the pins, or the pins could not be read, so there is nothing to grade. | the component name is misspelled or not a fleet-pins component key ops/pins/fleet-pins.yaml is missing, unreadable, or did not parse on a fixture (--mock), declared/artifacts.yaml did not parse |
grep -n '^ [a-z].*:' ops/pins/fleet-pins.yaml (the component keys this repository declares)goatbuilder preflight <component> --env <goatnet> (name the component as its fleet-pins key) |
docs/design/goatbuilder-rollout-preflight.md |
1 |
GB-070 |
goatbuilder | survey could read no goatnet at all — survey reads every goatnet it was asked for in parallel and reports what it could not read as unknown. When not one goatnet answered, there is nothing to compare, and the run stops instead of printing an empty survey that would look consistent. | this laptop is on no goatnet's mesh profile and no door opened (no AWS credentials for the SSM door) every goatnet named with --env is misspelled or not in ops/pins/fleet-pins.yaml ansible is not installed, or ops/scripts/with-aws cannot run |
goatbuilder doctor --env <goatnet> --all names the transport each goatnet would use and what is missingname goatnets by their fleet-pins key: --env <goatnet> (repeatable) |
docs/design/goatbuilder.md |
1 |
GB-071 |
goatbuilder | a site's leaf certificate is not on this controller — A play that installs a site's leaf (the zenoh router, the LAN mgmt identity, the dashboard pair, the names collector) found the cert or key nowhere it looks: the site manifest's out_dir ( |
the leaf was never minted for this site, or was minted on another operator's machine the manifest's out_dir points somewhere this machine does not have GOAT_LEAF_DIR is set to a directory that does not hold it |
mint it into the out_dir: ops/scripts/mint-site-certs.sh <site> --only <leaf> (idempotent; an existing leaf is skipped)copy an existing pair into <deploy.state_dir>/<site>-leaves/ (key mode 0600), never into the repository |
docs/operations/goatnet-standup.md |
1 |
GB-072 |
goatbuilder | goatbuilder cannot name the download surface to update from — update (and the behind-version notice) reads the release a goatnet serves at get. |
the first update on this machine was run without --site, and GOATBUILDER_UPDATE_URL is not set the site named with --site has no ops/ansible/group_vars/ the site does not serve goatbuilder (goat_get_goatbuilder_enabled is not true in its group_vars) no goatbuilder release is built for this operating system and architecture |
goatbuilder update --site <site> (a goatnet that serves goatbuilder)run it from a goat checkout, or pass -C <checkout>GOATBUILDER_UPDATE_URL=https://get.<site>.<zone>/goatbuilder goatbuilder update |
docs/design/goatbuilder-distribution.md |
1 |
GB-073 |
goatbuilder | the goatbuilder download surface could not be read — update reads version.json, the platform binary and its .sha256 from get. |
this machine is not on the goatnet's mesh (get. the surface is mid-deploy, or does not publish this platform's binary the link stalled; a slow link is fine, a silent one for 30 seconds is not |
curl -fsS https://get.<site>.<zone>/goatbuilder/version.json (is the surface reachable from here?)retry the update; it is idempotentGOATBUILDER_UPDATE_STALL_SECS=120 goatbuilder update (a slower link) |
docs/design/goatbuilder-distribution.md |
1 |
GB-074 |
goatbuilder | the downloaded goatbuilder is not vouched for by its release — Before touching anything, update checks that the |
the surface is mid-deploy and the binary, its checksum and SHA256SUMS are from different releases the download was altered or truncated on the way the signature over SHA256SUMS is not from the release workflow of the repository goatbuilder.yaml names |
retry in a few minutes; the surface's files are written togethergoatbuilder update --check (what the surface serves now)cosign verify-blob --bundle SHA256SUMS.cosign-bundle --certificate-identity-regexp '^https://github.com/<owner>/<repo>/' --certificate-oidc-issuer https://token.actions.githubusercontent.com SHA256SUMS (the check by hand) |
docs/design/goatbuilder-distribution.md |
1 |
GB-075 |
goatbuilder | goatbuilder could not replace its own binary — The download was verified, but writing it next to the running binary or renaming it into place failed. The rename is atomic, so the binary in place is either the old one, whole, or the new one, whole. | the install directory is not writable by this user (a binary under /usr/local/bin installed with sudo) the disk is full |
run the update as the user that installed goatbuilder (sudo goatbuilder update --site <site> for a system-wide install)install by hand from get.<site>.<zone>/goatbuilder/, checking the .sha256 first |
docs/design/goatbuilder-distribution.md |
1 |
GB-076 |
goatbuilder | the rollout is not cleanly reversible or not ready — preflight graded the three rollout facts (State, Rollback, Scope) and the readiness half, and at least one row is not a clean, declared, validated pass. An undeclared fact fails closed (stateful, rollback none) and reads not-clean on purpose, so the gap is visible rather than guessed green; an UNKNOWN is not a pass either. The facts are still printed — a stateful, restore-required rollout is a valid state to deploy knowingly, with --ack. | a fact is undeclared, so it failed closed (declare owns_state / rollback / restore_from / coupled_with in ops/pins/fleet-pins.yaml) a declared rollback contradicts the artifact type (pin-revert over a data/cert/forward-only component, or one flagged reversible false) a readiness check failed — the ref is not pullable, pins --check found drift, the secrets gate is unsatisfied, the checkout is stale, or the end state is not declared a validation could not answer (UNKNOWN) — no registry probe, or no reachable ledger for the rollback --dry-run |
goatbuilder preflight <component> --env <goatnet> (each not-ok row with its reason)declare the component's rollout facts in ops/pins/fleet-pins.yaml (owns_state, rollback, restore_from, coupled_with)goatbuilder deploy --component <c> --env <goatnet> --ack "<you>" (deploy knowingly; the accepted risk is recorded) |
docs/design/goatbuilder-rollout-preflight.md |
3 |
GB-100 |
goatbuilder | the federation verbs cannot find this party's context directory — goatbuilder federate init (and verify or apply without --as) reads this party's declaration, federation.toml, and init also reads the mTLS certificate and CA roots beside it. goatbuilder has no profile of its own, so it takes the directory from --context-dir, else $GOAT_CONTEXT_DIR, else the goat profile ($GOAT_PROFILE_DIR, else ~/.goat), and none of those was set. | none of --context-dir, GOAT_CONTEXT_DIR, GOAT_PROFILE_DIR or HOME is set, as in a bare container or a service unit | goatbuilder federate init --context-dir <dir> (the directory holding federation.toml, mtls.cert.pem and ca-roots.pem)export GOAT_PROFILE_DIR=<goat profile directory> (the verbs then read its active context, as goat does)goatbuilder federate verify <agreement> --as <sovereign_id> (verify and apply need no context when --as is given) |
docs/design/goatbuilder.md |
2 |
GB-110 |
goatbuilder | the inventory does not name a federation host phase 3 can reach — federate phase 3 measures the edge and the core from this machine, reaching each host the way the peering plays do: the host in the edge group and the host in the core group of inventory/peering-edges.ini, with its address, user, port, key and SSH options. The inventory could not be read, the group has no host or several, or the host uses a connection phase 3 does not use. | ansible-inventory is not installed here, or the inventory does not parse the edge group ( the group has several hosts and none was named the host uses an ansible_connection other than ssh or local |
cd ops/ansible && ansible-inventory -i inventory/peering-edges.ini --playbook-dir . --graph (what the plays see)add the host to the group, or name it: goatbuilder federate init --edge-host <host> --core-host <host> |
docs/design/federation-router-discovery.md |
1 |
GB-111 |
goatbuilder | a federation host did not answer over SSH — Before measuring anything, phase 3 runs true on the edge and on the core over SSH, with the inventory's connection settings and the plays' privilege escalation. That command failed, so nothing was measured. The message carries the ssh command it ran. |
this machine is not on a network that reaches the host (mesh profile, door or jump host) the host refused the key (see X-SSH-001) the host key changed (see X-SSH-003) |
run the ssh command the message prints, without its last argument, and read ssh's own errorgoatbuilder doctor --env <env> (the transport row) |
docs/design/federation-router-discovery.md |
1 |
GB-112 |
goatbuilder | sudo on a federation host wants a password — The host answered over SSH, but sudo -n true failed: phase 3 reads the federation interface and the container runtime as root, the way the plays become root, and it never types a password. |
the host's operator user is not NOPASSWD in sudoers (the baseline play sets it) ansible_become_user names a user this account cannot become without a password |
run ops/ansible/mgmt-host-baseline.yml against the host onceset ansible_become: false for the host if its user can already read wg and the container runtime |
docs/design/federation-router-discovery.md |
1 |
GB-113 |
goatbuilder | no core router to measure on the core host — Phase 3 measures the core router's image and version on the core host. It takes the router the operator names with --router-container, else the name the deployment's variables give (zenoh_stack, or site_id with the goat-zenoh role's default, as |
the declared router is not running on the core host, or the variables give a different name than the one deployed nothing declares the router and no running container carries org.goat.component=zenoh-core or a standard name several core routers run on the host |
goatbuilder federate init --router-container <name> (name it by hand)FED_ROUTER_CONTAINER=<name> goatbuilder federate <partner> (the one command passes it to phase 3 as --router-container)set zenoh_stack or site_id for the core host in the deployment's group_vars or host_varsssh <user>@<core> sudo docker ps --format '{{.Names}} {{.Label "org.goat.component"}}' (what is running) |
docs/design/federation-router-discovery.md |
1 |
GB-114 |
goatbuilder | an edge half of phase 3 cannot be joined — With two drivers, the edge driver measures phase 3's edge half with goatbuilder federate init --over-ssh <partner> --side edge --half-out <file> (or, inside the one command, goatbuilder federate <partner> --side edge --from 3 --to 3) and prints its sha256 and a confirm code. The controller joins it with --side core --edge-half <file> --edge-half-sha256 <code> (or imports it with goatbuilder federate import <partner> 3 <file> --sha256 <code>, which runs the same file checks, and --from 3 joins it) before it builds and signs the package. The half carries the edge's WireGuard public key, which the package is signed with, so it is joined only when it is the file the edge driver wrote. The file given could not be read; its sha256 is not the one confirmed; it is not an edge half; it was measured for another peering, on another host than this checkout's inventory names as the edge, or for another federation interface than the package asks for; or it was measured more than 24 hours ago, or later than now. |
the file was changed or not copied whole on its way from the edge driver, or another file was given the confirm code was mistyped, or read from another run of the edge half the edge half was measured for another partner or on another edge host the declaration or the init flags differ between the two machines, so the edge was measured for another interface (or none) the edge half is older than a day, or one of the two machines' clocks is wrong |
goatbuilder federate init --over-ssh <partner> --side edge --half-out <file> (on the edge driver, with the same declaration and init flags; it prints the confirm code)sha256sum <file> (on both machines: the two must agree) |
docs/design/federation-router-discovery.md |
1 |
GB-120 |
goatbuilder | a peering names a partner with a reserved word — A word of goatbuilder federate (init lint compose diff verify apply help status plan deploy test allocate backout rotate) can never name a partner: goatbuilder federate <word> runs that step, so the peering could never be run by name. |
a peerings entry's with: is one of the reserved words |
name the partner by its sovereign id in goatbuilder.yaml peerings: with: |
docs/design/goatbuilder.md |
2 |
GB-121 |
goatbuilder | two peerings would share a value — A per-peering value (an interface, a UDP or TCP port, a link subnet, a forwarder port, a certificate directory, an inventory name) is pinned on one peering and already held by another, by a pin or by the allocation record, or a partner is declared twice on one environment. Two peerings on one edge or core host cannot share one. | two peerings pin the same value under values: a pin takes a value the allocation record holds for another peering the same partner is declared twice on one environment |
pin a different value on one of the two peerings, or remove the pin and run goatbuilder federate allocate to have one allocatedgoatbuilder validate (every collision, each naming both peerings) |
docs/design/goatbuilder.md |
2 |
GB-122 |
goatbuilder | a pinned port is one the edge host already holds — A pinned WireGuard or carrier listen port is a UDP port the environment's edge hosts already hold (environments[].peering_edge_reserved_udp; absent, NetBird's WireGuard on 51820). A second listener there fails or steals its traffic. | values.wg_listen_port or values.carrier_listen_port is pinned to a reserved port a peering running on the role default 51820, on edges with no mesh client, is being adopted |
remove the pin and run goatbuilder federate allocate (it allocates odd ports from 51823 up and never takes a reserved port)when the environment's edges run no mesh client: set peering_edge_reserved_udp: [] on that environment in goatbuilder.yaml |
docs/design/goatbuilder.md |
2 |
GB-123 |
goatbuilder | a peering's pin is not usable — A pin under a peering's values: is malformed (an interface name over 15 characters, a subnet that is not a /31 or /30 on its boundary, a relative certificate directory, a name with spaces), or pins a forwarder value on a peering no forwarder relays. | a typing mistake in a pin public_port or a carrier_* value pinned without forwarder: |
correct the pin, or remove it to have the value allocateddeclare forwarder: endpoint: on the peering, or remove the forwarder pins |
docs/design/goatbuilder.md |
2 |
GB-124 |
goatbuilder | the peering allocation record cannot be used — ops/goatbuilder/peering-values.yaml, the record of the per-peering values goatbuilder federate allocate chose, is unreadable, has another schema, holds a malformed value, gives two peerings one value, or records a port the edge host already holds. Allocation reads it so a re-run never moves a value; it will not guess around a broken one. | a hand edit or a merge conflict in the record a record from a newer goatbuilder |
git log -p ops/goatbuilder/peering-values.yaml (restore it from its history)remove the broken entry and run goatbuilder federate allocate to allocate it afresh (that peering's values may change) |
docs/design/goatbuilder.md |
2 |
GB-125 |
goatbuilder | a peering file exists that goatbuilder did not generate — goatbuilder federate allocate writes the peering inventory and each peering's peering-values.yml. It never overwrites a file it did not generate (its first line says who did), and never renders a variable a hand-written file already sets for the same hosts (the group, its parent peering group, or the host_vars of its inventory names), because Ansible's file order would then pick the value. Nothing was written. | ops/ansible/inventory/peering-edges.ini or a peering-values.yml was written by hand before the values were allocated a hand-written group_vars or host_vars file sets a variable allocate renders (a listen port, a certificate path) |
move its values into goatbuilder.yaml (pins under the peering's values:, addresses under hosts:), remove the file, and run goatbuilder federate allocate again |
docs/design/goatbuilder.md |
2 |
GB-126 |
goatbuilder | a peering's inventory group is defined twice — Not every peering declares hosts:, so ops/ansible/inventory/peering-edges.ini stays hand-written and each peering that declares hosts: gets its own rendered inventory file (inventory/peering- |
hosts: added to a peering whose group is still in the hand-written peering-edges.ini | remove that peering's groups from ops/ansible/inventory/peering-edges.inior remove hosts: from the peering and keep its groups in the hand-written inventory |
docs/design/goatbuilder.md |
2 |
GB-127 |
goatbuilder | no free per-peering value is left — Every candidate in a per-peering value's sequence is held by another peering of the same environment (128 interfaces, ports or /31 subnets), or by a pin. | many peerings on one environment, or pins scattered over the sequence | pin the value under the peering's values: |
docs/design/goatbuilder.md |
2 |
GB-128 |
goatbuilder | a federate run disagrees with the peering's allocated values — The allocation record holds this peering, and the run's working directory, edge group or core group is not the allocated one. Phase 8 would mint certificates where the edge does not read them, or the plays would run on another peering's hosts. Nothing ran. | --work or FED_WORK names a directory whose pki is not the allocated cert_dir --edge-group names another group than the allocated edge_group a partner declared on a second environment, or a core_group pin: the driver runs the republisher on |
goatbuilder federate allocate --dry-run (the allocated values)pass the --work and --edge-group the refusal names, or pin cert_dir and edge_group under the peering's values: to what you pass |
docs/design/goatbuilder.md |
2 |
GB-129 |
goatbuilder | a peering's offer would widen what federation.toml declares — federation.toml owns the release sets this party offers ([data] release). A peering's offer: may only narrow that list for one partner. It names a set federation.toml does not offer, is empty or malformed, or federation.toml cannot be read. Or the peering's federation.toml is hand-written (federation_toml: hand), whose [data] release phase 3 offers whole, and offer: is narrower than it; a peering whose federation.toml goatbuilder federate allocate generates gets the narrowed list. Nothing ran. | offer: names a set that is not in federation.toml's [data] release offer: is narrower than the [data] release of a hand-written federation.toml (federation_toml: hand), which phase 3 offers whole GOAT_PROFILE_DIR or GOAT_CONTEXT points at another goat context than the one phase 3 runs in |
add the set to [data] release in federation.toml, the one owner of the offered sets, or remove it from offer:declare federation_toml: rendered on the peering so goatbuilder federate allocate generates its federation.toml, or narrow [data] release in the hand-written file to what this partner receives, or remove offer:goatbuilder validate (the offer's shape) |
docs/design/goatbuilder.md |
2 |
GB-210 |
goatbuilder | the host firewall baseline is missing a value it needs — The host firewall baseline (roles/goat-host-firewall, rendered by its goat-host-firewall-render.py) could not build a ruleset, because a value it derives the rules from is missing or malformed. Nothing was applied. | goat_host_firewall_management_sources is empty, so SSH would be admitted from nowhere an edge run reached the role without its peering values (the play was not peering-wg.yml or peering-edge.yml) a port, address or interface name in the input is not one (a partner endpoint given as a name, an interface name over 15 characters) a management source is /0, the mesh interface is the host's uplink, or a br-* bridge that is not docker's (a VXLAN segment bridge) exists while container-bridges is kept twice goat_host_firewall_confirm_timeout plus 15 s is not below goat_host_firewall_rollback_seconds the switch is false but the host still carries the baseline (/etc/goat-host-firewall/profile exists) |
set goat_host_firewall_management_sources in the host's group_vars to the address or CIDR the deployment manages it frompython3 ops/ansible/roles/goat-host-firewall/files/goat-host-firewall-render.py --input <input.json> --list-rules (what the baseline derives) |
docs/design/peering-host-firewall.md |
2 |
GB-211 |
goatbuilder | the host firewall override names a rule the baseline does not have, or is malformed — The deployment's goat_host_firewall_override names a rule that is not in the baseline, names a fixed rule (loopback, established, invalid), uses a key other than drop, sources, ports, interface and reason, asks a fixed-expression rule for more than drop, or gives a reason holding a line break or another control character (the reason is written into the ruleset file). The whole override was refused and nothing was applied. | a misspelled rule name, or a peering rule named for an interface this host does not carry an attempt to add a new allow rule; only named baseline rules can be changed, and other-tcp / other-udp are the named rules for a service the host also runs |
python3 ops/ansible/roles/goat-host-firewall/files/goat-host-firewall-render.py --input <input.json> --list-rules (the names an override may use) |
docs/design/peering-host-firewall.md |
3 |
GB-212 |
goatbuilder | the host firewall is loosened without a recorded reason — An override entry widens a baseline rule (more sources, other ports, another interface), or a role input is wider than the profile (goat_host_firewall_mesh_udp_ports not empty, a mesh DNS port other than 5353, a management source broader than goat_host_firewall_management_prefix_floor, or that floor set broader than /24 or /64), and no reason is recorded for that rule. Tightening is free; loosening a named rule needs a reason recorded in the deployment's configuration. Nothing was applied. | the entry has no reason: key, or a blank one an entry meant to narrow widens instead (an empty sources list means any source) |
add reason: "<why this host needs it>" to the entry in goat_host_firewall_overridenarrow the rule instead, which needs no reason |
docs/design/peering-host-firewall.md |
3 |
GB-213 |
goatbuilder | the host firewall would cut the session managing the host — The rendered ruleset admits no SSH from the source of the session running the play (read from SSH_CONNECTION on the host), so loading it would lock the deployment out. Nothing was applied. | goat_host_firewall_management_sources does not hold the address this play connects from the override drops ssh-management, narrows its sources past this address, or binds it to another interface SSH listens on another port than goat_host_firewall_ssh_port the session's source could not be read, so the check could not run |
add the controller's address to goat_host_firewall_management_sourcesrun the play from a declared management source |
docs/design/peering-host-firewall.md |
4 |
GB-214 |
goatbuilder | the host firewall ruleset was not loaded or not confirmed — Loading the rendered ruleset failed, or a fresh connection to the host did not succeed after it was loaded. A ruleset that was loaded is undone by the rollback timer the role armed before loading it. | the kernel refused the ruleset (nft -c failed), for example an nftables too old for a rule the new ruleset blocks the connection the controller makes, through a path the lockout check could not see (a jump host, a NAT between controller and host) systemd-run is not available on the host, so no rollback could be armed and nothing was loaded |
wait goat_host_firewall_rollback_seconds, then: nft list table inet goat_host_firewall (the previous table is back)read /etc/goat-host-firewall/ruleset.nft.new on the host and the play output |
docs/design/peering-host-firewall.md |
1 |
GB-215 |
goatbuilder | the host firewall would close a port this host listens on — Before loading, the host firewall baseline reads the sockets the host listens on (ss) and found one, on a non-loopback address, that no rule admits and that the deployment has not named as closed on purpose. Loading would cut that service without anyone deciding it. Nothing was applied. | the host runs another service beside the peering (a management server, a portal, an exporter) a port the deployment means to close is not listed in goat_host_firewall_closed_listeners |
admit it: goat_host_firewall_override: {other-tcp: {ports: [<port>], reason: "<why>"}}close it on purpose: goat_host_firewall_closed_listeners: ["tcp/<port>"] (or the process name) |
docs/design/peering-host-firewall.md |
5 |
GB-180 |
goatbuilder | another run of this peering is in progress — A real run of goatbuilder federate holds |
another terminal or a scheduled job is running goatbuilder federate for the same peeringtwo peerings were given the same working folder (--work or FED_WORK) |
wait for the other run to finish; the lock file names its process idgive each peering its own working folder: --work /var/lib/goat-federation/<partner> |
docs/design/goatbuilder.md |
1 |
GB-181 |
goatbuilder | a resource this peering needs is held by another peering on the same host — Before a peering takes a resource on a shared host (a tunnel interface, a port, a link subnet, a certificate folder), goatbuilder reads every other peering's run journal in the same work root under |
two peerings were given the same per-peering value (a pinned value in peerings: or a hand-written group_vars file)a journal in the work root is damaged, so the check cannot tell what it holds |
give this peering a different value, or let allocation choose one, then make a new planread the other peering's journal named in the message: <root>/<peering>/state.json |
docs/design/goatbuilder.md |
1 |
GB-182 |
goatbuilder | the run journal cannot be read — |
the file was edited by hand or cut short a later goatbuilder wrote it with a later schema (the message names the schema and the goatbuilder that wrote it) two peerings share one working folder |
use the goatbuilder that wrote it, or move the file aside and run again (the run then starts with no journal, and --resume skips nothing)give each peering its own working folder: --work /var/lib/goat-federation/<partner> |
docs/design/goatbuilder.md |
1 |
GB-183 |
goatbuilder | a federate phase failed — The word the run journal records for a phase that exited non-zero, with its exit code. The run stopped there; no later phase ran. | the phase's own error, printed above it (a goat federate exit 20 to 24, a play's failed task, a refused gate) |
fix the cause, then run the same command again; with --resume, the phases found unchanged are not run again |
docs/operations/sovereign-federation-runbook.md |
1 |
GB-184 |
goatbuilder | the working folder or its lock cannot be made, taken or written — A real run of goatbuilder federate keeps its lock and its run journal in the peering's working folder, and the allocation lock in the folder above it. One of them could not be created, locked or written, so the run did not start (or, mid-run, its journal is behind; the run says so in a failed journal row). |
the working folder is not writable by this user (the default /var/lib/goat-federation needs root or a folder made for you) the disk is full or the folder is on a read-only mount |
sudo install -d -o $USER /var/lib/goat-federation (or pass --work <a folder you own>)check free space where the working folder is |
docs/design/goatbuilder.md |
1 |
GB-130 |
goatbuilder | a federation input is missing — federate phase 1 (preflight) found inputs the later phases need that nothing declared and nothing could derive. Every missing input is listed with where to declare it. Nothing was measured, signed or deployed. | the deployment does not declare it (the edge group's group_vars, the peering's entry in goatbuilder.yaml peerings:, the inventory) the script avenue (peering-standup.sh) was run without exporting it; goatbuilder derives what it can, the script does not |
declare each listed value where the line names it, then: goatbuilder federate <partner> --from 1 --to 1export FED_<NAME>=... (one command only; an exported value always overrides what goatbuilder derives) |
docs/design/federation-preflight.md |
1 |
GB-131 |
goatbuilder | a port this peering needs is already held on the edge — the tunnel listen port (UDP) or the edge router listen port (TCP) is held on the edge host by something other than this peering's own interface or container. NetBird holds UDP 51820 on a mesh host, which is also the WireGuard default. | the tunnel listen port is the WireGuard default 51820 and NetBird's interface holds it another peering's edge router or interface already uses the port on a shared edge host |
pin another port in the peering declaration: goatbuilder.yaml peerings: values: wg_listen_port / edge_listen_portor set peering_wg_listen_port / goat_zenoh_edge_listen_port in group_vars/<edge group>/, then re-run phase 1 |
docs/design/federation-preflight.md |
1 |
GB-132 |
goatbuilder | the republisher metrics port is already held on the core — the outbound republisher's loopback metrics port is held on the core host by something other than this peering's republisher. One port per peering on a shared core host. | another peering's republisher uses the same metrics port on the core host another service listens on that loopback port |
pin republisher_metrics_port in the peering declaration (goatbuilder.yaml peerings: values:)or set goat_fed_republisher_metrics_port in group_vars/<peering>_core/ |
docs/design/federation-preflight.md |
1 |
GB-133 |
goatbuilder | the edge or core host could not be probed for its ports — the read-only probe (ops/ansible/peering-preflight.yml) returned no answer for a host, or ss could not list its sockets, so the ports were not checked. Not a pass. |
the host is unreachable with the inventory the peering plays use (mesh down, wrong ansible_host, no SSH key) iproute2 ( ss) is not installed on the host |
cd ops/ansible && ansible -i inventory/peering-edges.ini <group> -m pinginstall iproute2 on the host, then re-run phase 1 |
docs/design/federation-preflight.md |
1 |
GB-134 |
goatbuilder | the committed slot list differs from the slot registry — ops/scripts/sync-slot-anchors.py --check found live slots with no declared store or declared stores for slots that are not live. A core router restarted on this list drops or misses receive stores. Slots minted between the fix and the restart drift it again, so the restart step runs this check once more. | slots were minted or revoked since the list was last regenerated and committed the list was regenerated, and a slot was minted before the core router restarted |
python3 ops/scripts/sync-slot-anchors.py --env <goatnet> --db <registry.db> (writes the list; commit it)python3 ops/federation/standup/preflight.py <partner> --work <dir> --only slot-anchors (re-check just before the restart) |
docs/design/federation-preflight.md |
1 |
GB-135 |
goatbuilder | the slot registry could not be read — sync-slot-anchors.py exited 2: the registry was unreadable, or it reported zero live slots. That is not a verdict on drift, and the check is not passed. | FED_SLOT_DB names a file that is not the portal registry, or it cannot be opened read-only the management host named by FED_SLOT_PG_SSH is unreachable, or its portal database container differs |
check FED_SLOT_DB, or FED_SLOT_REGISTRY_STORE=postgres with FED_SLOT_PG_SSH (portal_store in group_vars/<goatnet>_hosts says which) |
docs/design/federation-preflight.md |
1 |
GB-136 |
goatbuilder | the core has no receive store matching what the edge grants — the core router's storage_manager has no store whose key expression equals an ingress scope the edge admits (from/ |
the core keeps only a historical from/** store the core's per-partner store names a different partner or lane than the edge grants |
add a store on the core whose key_expr equals each admitted scope (the goat-zenoh role renders one per partner); keep a historical from/** store beside it if its data is needed |
docs/design/federation-ingress-storage-preflight.md |
1 |
GB-137 |
goatbuilder | the core router's configuration could not be read — preflight found the core router but could not read its configuration, so the receive stores were not checked. Reported SKIP, never PASS; FED_PREFLIGHT_STRICT=1 refuses on it. | the core router reads its configuration from another path than /etc/zenoh/config.json5 the account behind FED_ROUTER_DOCKER_HOST may list containers but not exec in them |
export FED_CORE_CONFIG_PATH=<path inside the core container>inspect the core's storages through its own approved workflow |
docs/design/federation-ingress-storage-preflight.md |
1 |
GB-138 |
goatbuilder | no core router container was found — the selection phase 3 uses (goat federate init) found no running core router on the daemon preflight read: FED_ROUTER_DOCKER_HOST, else the local runtime. Discovery accepts the org.goat.component=zenoh-core label, *-zenoh-router names and legacy goat-zenoh. | the core router runs on another host and FED_ROUTER_DOCKER_HOST does not point at it the core router is stopped, or runs under a custom name without the role label |
export FED_ROUTER_DOCKER_HOST=ssh://<user>@<core> (goatbuilder derives it from the peering's hosts.core, else the one host of inventory group <partner>_core)export FED_ROUTER_CONTAINER=<name or ID> |
docs/design/federation-router-discovery.md |
1 |
GB-139 |
goatbuilder | several core router containers were found — more than one running container qualifies as the core router, and preflight, like phase 3, will not choose among them. | an old and a new core router run side by side two stacks on one host both carry a *-zenoh-router name |
export FED_ROUTER_CONTAINER=<name or ID of the core router> |
docs/design/federation-router-discovery.md |
1 |
GB-140 |
goatbuilder | our CA certificate is missing or is not a CA — the certificate phase 8 signs the peering leaves with does not exist, is not a PEM certificate, or is not a CA (basicConstraints CA:TRUE). | FED_CA_CERT, goat_federation_ca_cert or the derived ops/ansible/enrollment/ca/ a leaf certificate was named instead of the CA |
set goat_federation_ca_cert in group_vars/<edge group>/ to our own CA certificate (never the partner's) |
docs/design/federation-preflight.md |
1 |
GB-141 |
goatbuilder | our CA signing key cannot be resolved — the CA private key phase 8 needs is missing, does not decrypt with the age identity, is not a private key, or does not match the CA certificate. The deployment keeps it at |
the ca.key.age file is not on this machine the age identity (~/.config/age/keys.txt, or FED_AGE_IDENTITY) is not the one the key was encrypted to the key belongs to another CA than the certificate |
restore <deploy.state_dir>/<site>-ca/ca.key.age, or set goat_federation_ca_key_age / FED_CA_KEY_AGEexport FED_AGE_IDENTITY=<the identity file the key was encrypted to> |
docs/design/federation-preflight.md |
1 |
GB-142 |
goatbuilder | our federation signing key cannot be read — FED_SIGNING_KEY names a file that does not exist, or whose content is neither 64 hex characters (the RFC 8032 seed) nor 44 base64. | goat_federation_signing_key or FED_SIGNING_KEY names the wrong path the file holds a PEM or a public key rather than the seed |
set goat_federation_signing_key in group_vars/<edge group>/ to the file holding our Ed25519 seed |
docs/design/federation-preflight.md |
1 |
GB-143 |
goatbuilder | the peering inventory does not reach the edge or core — inventory/peering-edges.ini is missing, has no group for this peering's edge or core, lists no host in it, or does not list the group under [peering_edges:children] / [peering_cores:children]. A play limited to such a group runs against nothing and reports success. | the edge or core group was never added to the deployment's inventory the committed inventory ships the host commented out |
add [<partner>_edge] and [<partner>_core] with their hosts, and list each group under its parent's :children |
docs/design/federation-preflight.md |
1 |
GB-144 |
goatbuilder | the federation declaration or the profile is missing or wrong — the goat context has no federation.toml, it has no valid sovereign_id or it declares a measured capability, or the profile certificate or CA bundle is missing. | goat profile init was not run for this context, or ran as another user (sudo -E keeps HOME)federation.toml was not written, or declares transports, fabric_version, fabric_fips_image or pqc_capable |
goat profile init <bundle.cbor>goat federate init (prints the declaration template when it is absent) |
docs/design/federation-preflight.md |
1 |
GB-145 |
goatbuilder | the core certificate would not name every address the edge dials — FED_CORE_SAN does not cover an address the edge's deployment.yml dials. A core certificate without that SAN fails name verification, and nothing is logged at INFO when it does. | an endpoint was added to group_vars/ |
unset FED_CORE_SAN and let goatbuilder derive it from deployment.yml, or add the missing address |
docs/design/federation-preflight.md |
1 |
GB-146 |
goatbuilder | docker compose is not available for phase 12 — the edge role deploys with docker compose, and this machine has no docker, no compose plugin, or a compose that reports no version. |
docker-compose-plugin is not installed | install docker-compose-plugin (or drop the plugin binary at /usr/libexec/docker/cli-plugins/docker-compose) |
docs/design/federation-preflight.md |
1 |
GB-147 |
goatbuilder | a peering directory value is not usable — FED_DIRECTORY_URL is not https, or it, FED_PACKAGE_URL, FED_COUNTERPARTY_ID or FED_CA_CERT holds characters outside a plain URL. Those values reach a command line, so the run refuses them. | an http:// directory address, or a value with spaces or quotes | set goat_federation_directory_url to the https:// address of the directory |
docs/design/federation-preflight.md |
1 |
GB-148 |
goatbuilder | the federation working directory is not writable — the run's working directory (--work, FED_WORK, else /var/lib/goat-federation/ |
the default under /var/lib needs root | goatbuilder federate <partner> --work <a directory you can write> |
docs/design/federation-preflight.md |
1 |
GB-149 |
goatbuilder | the peering is not declared — the partner is not in goatbuilder.yaml peerings:. A peering nobody declared is one nobody reviewed (contract C1). | the peering was never added to the deployment's goatbuilder.yaml | add the peering to ops/goatbuilder/goatbuilder.yaml peerings: (with, environment, site, edge_class) |
docs/design/federation-preflight.md |
1 |
GB-268 |
goatbuilder | a peering host's clock is not synchronized, is too far from UTC, or could not be read — federate phase 1 (preflight) read each edge and core host's clock from its own time service (chronyc -c tracking, else timedatectl). A Zenoh router refuses a timestamp more than 500 ms ahead of its own clock, and the partner's fabric is synchronized separately, so each side must hold its clock within half that, 250 ms by default (FED_CLOCK_BUDGET_MS), of UTC. A host that is not synchronized to an outside reference, or is over the budget, is a FAIL; a host whose clock could not be read is SKIP, never PASS. | the host's time service follows no reachable time source, or only the host's own local clock the clock is synchronized but still slewing a large step, or the time source itself is wrong neither chronyd nor systemd answered the probe, so the clock was not read |
on the named host: chronyc tracking (or: timedatectl show); repair its time synchronization, then run phase 1 againgoatbuilder federate <partner> --from 1 --to 1 |
docs/design/federation-preflight.md |
1 |
GB-230 |
goatbuilder | allocate would move a peering that already runs — A peering the allocation record does not hold already runs from hand-written configuration (its edge group has hand group_vars, or the hand inventory names it), and the values allocation chose differ from the ones it runs on (its hand group_vars, else the role defaults). Rendering them would move its tunnel or ports on the next phase 2. Nothing was written. | goatbuilder federate allocate run for the first time in a deployment with a peering set up by hand | adopt it as it runs: pin the values the refusal names under the peering's values: in goatbuilder.yaml (a pin on 51820 needs that environment's peering_edge_reserved_udp to allow it)move it on purpose: FED_ALLOCATE_MOVE=<with> goatbuilder federate allocate --yes |
docs/design/goatbuilder.md |
2 |
GB-231 |
goatbuilder | the hand-written peering inventory puts one name in two peering groups — No peering declares hosts:, so ops/ansible/inventory/peering-edges.ini is hand-written, and it puts one inventory name in two peering groups. Ansible merges the group_vars of every group a name is in, so each peering would read the other's rendered values. Nothing was written. | one machine listed under the same name for two peerings | give each peering its own inventory name for the machine, with the same ansible_hostdeclare hosts: on every peering and let goatbuilder federate allocate render the inventory |
docs/design/goatbuilder.md |
2 |
GB-234 |
goatbuilder | the site inventory does not name one edge and one core for a peering without hosts — A peering declares no hosts:, so goatbuilder federate allocate takes its machines from the site inventory (the environment's inventory:, else ops/ansible/inventory/.ini but the peering-.ini files): the group of the peering's edge_class and of the site's other class, each named as the class, |
two inventory files define the site's class groups a class group lists several machines, or the edge class group lists none the edge and the core have different ansible_user values |
declare hosts: {edge: <address>, core: <address>, user: <user>} on the peeringdeclare the environment's inventory: so only that file is readmake the site inventory name one machine per class |
docs/design/goatbuilder.md |
2 |
GB-235 |
goatbuilder | a named allocate would change another peering's inventory lines — Every peering declares hosts:, so ops/ansible/inventory/peering-edges.ini is the whole generated peering inventory. goatbuilder federate allocate <peering> renders the named peering's groups into it and leaves every other line as it was; this run would change or drop a section of another peering (its hosts were changed in goatbuilder.yaml since the last allocate, or it is declared but not yet recorded). Nothing was written. |
another peering's hosts: changed in goatbuilder.yaml since the inventory was last rendered a peering in the generated inventory is declared but has no entry in the allocation record |
goatbuilder federate allocate (every declared peering; review and commit what it changes) |
docs/design/goatbuilder.md |
2 |
GB-236 |
goatbuilder | our core router's goat-zenoh role declarations do not resolve for admission — Phase 10's admission names no route of its own (FED_CORE_PLAY and FED_CORE_RESTART are unset), and an inventory group of the site inventory (ops/ansible/inventory/.ini, not peering-.ini) has group_vars that configure the goat-zenoh role, so admission restarts the core through the role's own play on that group. It could not resolve that play: several groups configure the role and the peering core's address does not pick one, nothing gives the group's site_id, or no single fleet pin whose image variable the group sets names a deploy play that exists. The message names what was looked for and where. Nothing on the core changed. | two inventory groups both configure the goat-zenoh role the core router group's group_vars declare no site_id and the group is not named no component in ops/pins/fleet-pins.yaml has the group's image variable as its var with a deploy play |
declare site_id in group_vars/<core router group>/keep one inventory group per core router, or put the peering's core in the peering inventory with the same addressset FED_CORE_PLAY or FED_CORE_RESTART to name the route by hand |
docs/design/goatbuilder.md |
2 |
GB-237 |
goatbuilder | admission cannot write the partner into the role core's zenoh_sovereign_peerings — On a goat-zenoh role core, phase 10's admission writes the partner's zenoh_sovereign_peerings entry (sovereign, and the CN of the agreement's access-control fragment subject) into ops/ansible/group_vars/ |
a comment inside the zenoh_sovereign_peerings list of peerings.yml, a flow-style top level, or an anchor in the list that another variable aliases zenoh_sovereign_peerings is set by hand in another file of the core router group's group_vars an entry of zenoh_sovereign_peerings writes its sovereign as a Jinja template, or is not a mapping with a literal sovereign zenoh_sovereign_peerings itself is a template, a mapping or a scalar, not a literal list: read as empty, it would be written back holding this partner alone the fragment phase 9 rendered names no subject, or several |
move a comment out of the zenoh_sovereign_peerings list (above it, or elsewhere in the file), write the file onevariable: valueblock per variable, and keep anchors out of the listmove the zenoh_sovereign_peerings entries into group_vars/<core router group>/peerings.yml and remove the variable from the other filewrite zenoh_sovereign_peerings as a literal list, and each entry's sovereign literally, as the partner's sovereign idrun phase 9 again, then make a new plan and deploy |
docs/design/goatbuilder.md |
2 |
GB-238 |
goatbuilder | a federation verb's site flags are given where they do not belong — goatbuilder federate init|lint|compose|diff|verify|apply read this party's context from --context-dir, --cert, --ca-roots, --context and --profile-dir. Those flags belong to the six verbs and come after the verb: given before it they are read as the name of a partner, and given to a peering run or to federate plan they would be ignored. |
the flags were typed after federate and before the verbthe flags were given to goatbuilder federate <partner> or federate plan, which read the deployment's own declarations |
goatbuilder federate lint --context-dir <dir> <package> (the verb first, then its flags)drop the flags from a peering run or a plan |
docs/design/goatbuilder.md |
2 |
GB-239 |
goatbuilder | a flag that does not apply to the federation verb it was given to — The six federation verbs (init, lint, compose, diff, verify, apply) read only the files they are named and the party's context, and run for real. --mock, -C/--repo (except for init --over-ssh) and -o/--output choose things they do not read; --over-ssh and its host flags belong to init alone. A run with one where it does not apply is refused rather than the flag ignored. A flag that belongs to the verb but needs another with it (a host flag on init without --over-ssh) is GB-325. | --mock, -C/--repo or -o/--output given to a federation verb --over-ssh or its host flags given to a verb other than init, to a plan or to a peering run |
drop the flag; the verbs take --json for a JSON reportgoatbuilder -C <repo> federate init --over-ssh <peering> (the one verb that reads a checkout)FED_MEASURE=ssh goatbuilder federate <partner> (a peering run measures phase 3 over SSH) |
docs/design/goatbuilder.md |
2 |
GB-240 |
goatbuilder | a federation switch names no choice it has — Two variables choose how a peering run works: FED_VERBS who runs the federation verbs (goatbuilder, in process, the default; or goat), and FED_MEASURE where phase 3 measures (ssh, the default; or here). Any other value is refused, as peering-standup.sh refuses it, so a misspelling cannot choose a different path on each avenue. | FED_VERBS is set to something other than goat or goatbuilder FED_MEASURE is set to something other than ssh or here |
unset FED_VERBS and FED_MEASURE (the defaults)FED_VERBS=goat goatbuilder federate <partner> (the verbs run in goat)FED_MEASURE=here goatbuilder federate <partner> (phase 3 measures the machine the run is on) |
docs/design/goatbuilder.md |
2 |
GB-241 |
goatbuilder | the environment named is not declared — A verb was given an environment that goatbuilder.yaml (or goatlite.yaml) does not declare in environments:. Nothing runs against an environment nobody declared. | a misspelt environment name the environment is declared in another checkout's goatbuilder.yaml |
the names declared are listed in the message; pass one of themadd the environment to ops/goatbuilder/goatbuilder.yaml environments: |
docs/design/goatbuilder.md |
2 |
GB-242 |
goatbuilder | the command line does not parse — The arguments do not make a command goatbuilder knows: a required argument is missing, a value is not one the option takes, two options that exclude each other were both given, or the verb is unknown. Nothing ran. | a required argument or subcommand was left out an option was given a value it does not take two options that exclude each other were given together a misspelt verb |
goatbuilder <verb> --help (the arguments the verb takes)goatbuilder --help (the verbs) |
docs/design/goatbuilder.md |
2 |
GB-243 |
goatbuilder | a standup names no site — standup stands up one site and needs --site. It may be left out only on a goatlite run, where goatlite.yaml (preset lite) names the one site the environment has. | --site was left out on an environment that is not a goatlite | goatbuilder standup <environment> --site <site> |
docs/operations/goatnet-standup.md |
2 |
GB-244 |
goatbuilder | the config file cannot be read — goatbuilder reads the deployment's goatbuilder.yaml (or the file a verb was named) before anything else. It is not there, or it cannot be read. | the path is wrong, or -C names another checkout the checkout has no ops/goatbuilder/goatbuilder.yaml |
goatbuilder -C <the deployment's checkout> <verb>goatbuilder validate <path> (the file you meant) |
docs/design/goatbuilder.md |
2 |
GB-245 |
goatbuilder | the config file is not YAML — The config file could be read, but it does not parse as YAML, so nothing in it can be checked. The message names the line and column. | a bracket, quote or indentation error near the line named | fix the line the message names, then goatbuilder validate <path> |
docs/design/goatbuilder.md |
2 |
GB-247 |
goatbuilder | the site or class named is not declared — standup and join-node act on a site goatbuilder.yaml declares under the environment, and join-node on a class that site declares and the verb converges (core, obs, breakglass, network-jump). The one named is not there. | a misspelt site or class the site or class is not declared yet |
the names declared are listed in the message; pass one of themdeclare the site under environments[].sites, or the class under its classes:, in ops/goatbuilder/goatbuilder.yaml |
docs/design/goatbuilder.md |
2 |
GB-248 |
goatbuilder | the observability leg cannot run — A site that declares has_obs_host: true stands up its observability host in a leg beside the core. --obs-only asked for that leg alone and there is none to run, or the leg's play (ops/ansible/utility-host-bootstrap.yml) is not in this tree. The message says which. | has_obs_host is not true, its group_vars or inventory host is missing (--obs-only) the checkout has no ops/ansible/utility-host-bootstrap.yml |
goatbuilder standup <environment> --site <site> --skip-obs (stand up the site without its obs host)declare has_obs_host: true, the <site>_obs_hosts group_vars and its inventory host, then run --obs-only |
docs/operations/goatnet-standup.md |
2 |
GB-249 |
goatbuilder | the fixture's standup does not fit what was asked — standup --mock rehearses the standup a fixture's converger.yaml declares for one environment and site, against that fixture's goatbuilder.yaml. The environment or site asked for is another, or a phase of the rehearsal names a site or class the declaration does not allow (a joiner may touch only the founder it attaches to; a class needs machines). | --mock names a fixture for another environment or site the fixture's converger.yaml and goatbuilder.yaml disagree |
goatbuilder standup <the fixture's environment> --site <its site> --mock <fixture>correct the fixture's converger.yaml standup: or its goatbuilder.yaml |
docs/design/goatbuilder.md |
2 |
GB-250 |
goatbuilder | a joiner site names no founder — A site whose vars.yml says goatnet_founder: false joins another site's goatnet, and seeds from it, so it must name that founder in join_founder_site. | join_founder_site is missing from the joiner's group_vars/ |
set join_founder_site: <founder site> in group_vars/<site>_hosts/vars.yml |
docs/tutorials/02-first-federated-node.md |
2 |
GB-251 |
goatbuilder | no SSH user is declared for a new machine — join-node reaches a new machine as the project's declared SSH user before anything is installed on it. The checkout declares none, and no name is right for every project's machines, so a real run refuses (a dry run plans with a placeholder). | deploy.ssh_user (or lite.ssh_user) is not set in goatbuilder.yaml | set deploy.ssh_user in ops/goatbuilder/goatbuilder.yamlgoatbuilder join-node ... --dry-run (plans without it) |
docs/design/goatbuilder.md |
2 |
GB-252 |
goatbuilder | the partner is declared on several environments — The same partner is declared as a peering on more than one environment in goatbuilder.yaml peerings:, so the command cannot tell which peering it is about. | one partner is peered from two environments, and --env was not given | pass --env <environment> (the environments are named in the message) |
docs/design/goatbuilder.md |
2 |
GB-253 |
goatbuilder | a host's back-out survey answered for another peering — Back-out surveys each host of the peering read-only before it plans what to remove, and every answer names the peering it read. One host's answer names another peering, so nothing is planned from it: removing that peering's resources would be removing someone else's. | the inventory groups of two peerings share a host name the survey ran with another peering's working directory |
goatbuilder federate backout <partner> --plan (survey again with this peering's own work directory)give each peering its own inventory name for the machine (allocate does) |
docs/design/goatbuilder.md |
2 |
GB-254 |
goatbuilder | a file goatbuilder reads or writes cannot be read or written — A verb reads or writes a file of the checkout or its state (a pin table, a schema, a record, an inventory, a play) and the operating system refused: the file is missing, unreadable, or its directory is not writable. The message names the file and the system's reason. | the file is not in this checkout, or -C names another one the file or its directory is not readable or writable by this user |
ls -l <the file the message names>goatbuilder -C <the right checkout> <verb> |
docs/design/goatbuilder.md |
2 |
GB-255 |
goatbuilder | nothing is declared for the environment — The fixture's (or checkout's) declarations name no component for the environment asked for, or none by the name given, so there is nothing to read, plan or roll back. | a misspelt environment or component the environment declares no component yet |
goatbuilder pins --check (what fleet-pins.yaml declares)pass an environment and component the declarations hold |
docs/design/goatbuilder.md |
2 |
GB-256 |
goatbuilder | a deploy play's hosts cannot be resolved — To target a deploy, goatbuilder reads the component's play for its hosts: line, following at most three import_playbook hops. The play has neither, or the chain is longer. | the play has no hosts: line and no import_playbook more than three import_playbook hops |
give the play a hosts: line ("{{ target }}" for a play targeted per group)declare deploy_playbook in fleet-pins.yaml for the play that does |
docs/design/goatbuilder.md |
2 |
GB-257 |
goatbuilder | fleet-pins.yaml is not in the shape goatbuilder edits — goatbuilder reads fleet-pins.yaml's components and edits a component's block in place. The file has no components:, or the component's block or its fleet: line is not where the writer looks. | fleet-pins.yaml has no components: mapping the component's block was edited by hand into another shape |
goatbuilder pins --check (the table as goatbuilder reads it)restore the component's block towith afleet:line, as the others are |
docs/design/goatbuilder.md |
2 |
GB-258 |
goatbuilder | a command a verb runs failed — A verb runs git, gh, ansible or one of the repository's own steps and stops when it fails. The message is that command's own error output; it names what went wrong. | the command's own error, quoted in the message (a network failure, a missing credential, a refused push) | read the quoted output: its tool's own catalog entry may name it (goatbuilder explain '<the line>')run the command by hand with the same arguments (goatbuilder -v shows it) |
docs/design/goatbuilder.md |
1 |
GB-259 |
goatbuilder | the assembled tree's marker is incomplete — An assembled checkout (a goatcore commit plus an overlay) carries .goat-assembled, naming the goatcore_ref and overlay_commit it was built from; fresh-source checks against them. A line is missing. | the tree was assembled by hand, or the marker was edited | assemble the tree again with the overlay's assembly stepcat .goat-assembled (goatcore_ref= and overlay_commit= lines) |
docs/design/goatbuilder.md |
2 |
GB-260 |
goatbuilder | a rollback has no history to go back to, or its target is refused — rollback goes back through the host's deploy ledger and the in-repo history (deploy-history.yml). There is no history for the component, not enough of it, or the ref it would roll to is flagged do_not_roll_to, or a host runs a ref neither history knows. | the component has not been adopted into deploy-history.yml the history holds fewer entries than the rollback asks for the target ref is flagged do_not_roll_to in the in-repo ledger a host runs a ref no history records (deployed by hand?) |
goatbuilder rollback <component> --statusinvestigate the host the message names before rolling back |
docs/design/universal-deploy-rollback.md |
2 |
GB-263 |
goatbuilder | a site id cannot name the variables its phases read — Phase 12 reads the NetBird token from |
join_founder_site in the site's group_vars holds a space, a quote or another character outside letters, digits, - and _ the site was named with such a character |
set join_founder_site to the founder's site id (letters, digits, - and _)goatbuilder validate (the site ids goatbuilder.yaml declares) |
docs/design/goatbuilder.md |
2 |
GB-264 |
goatbuilder | no recorded run matches — goatbuilder show and runs diff name a run by its id, a prefix of one, or last, from the run records under |
no verb that drives hosts has run with this state directory (a --mock run writes no record) a mistyped id, or a prefix too short to name one run |
goatbuilder runs (the recorded runs, newest first)give more of the run's id |
docs/design/goatbuilder-legibility.md |
2 |
GB-246 |
goatbuilder | a rollback depth outside the ledger — rollback N goes back N deploys in a host's ledger, which keeps the last seven. N outside 1 to 7 names no deploy the ledger holds. | N is 0, or more than 7 | goatbuilder rollback <component> --env <goatnet> <N from 1 to 7> |
docs/design/goatbuilder.md |
2 |
GB-185 |
goatbuilder | the partner named to federate status is not a declared peering — goatbuilder federate status <partner> reads a peering declared in goatbuilder.yaml peerings: (on --env, when given). The name given is not one of them, so nothing was read. |
the name is misspelled, or the peering is declared under another environment's config the peering has not been declared yet |
goatbuilder federate status (every declared peering)declare it under peerings: in goatbuilder.yaml first (C1) |
docs/design/goatbuilder.md |
1 |
GB-186 |
goatbuilder | the plan was not approved — goatbuilder federate deploy executes a plan only when approval names that plan's hash: --approve-plan <hash>, or a person at a terminal typing yes after reading the plan and its hash. --yes never approves a plan. No approval was given, or it named another hash, so nothing changed. |
there is no terminal and --approve-plan was not passed --approve-plan names the hash of an older plan (the plan was made again since) the person answered anything but yes |
read the plan, then: goatbuilder federate deploy <partner> --approve-plan <the hash it prints>goatbuilder federate plan <partner> (a new plan, when the old one is out of date) |
docs/design/goatbuilder.md |
1 |
GB-187 |
goatbuilder | the plan is not the one approved — The plan deploy was given is for another peering, or its hash changed between approval and execution (it is read again under the peering's lock, just before anything changes). Nothing changed. A plan that does not match its own hash is GB-201. | no plan has been made yet ( the plan was made again, or edited, after it was approved --plan names another peering's plan |
goatbuilder federate plan <partner> (then read it and approve its hash) |
docs/design/goatbuilder.md |
1 |
GB-188 |
goatbuilder | the plan no longer holds — Immediately before anything changes, deploy runs the planner's live recheck: every live prerequisite is checked again and every input section is compared with a plan built now. Something did not hold (a section changed, a check was not PASS or NOT APPLICABLE or was not answered, or the plan was not READY), so the approved plan no longer describes the change, and a saved plan is not proof the environment stayed the same. Nothing changed. | the agreement was composed again (a new package from either side) apply rendered the files again, or one was edited by hand the plan recorded a SKIP of a check deploy does not run again (host-allocation): the inventory names no host for this peering, or a value of another peering on a machine this peering shares does not resolve. The message says which, and what clears it a SKIP the plan accepted (plan --accept) reads differently now: the message names what it reads now and what was accepted |
goatbuilder federate plan <partner> (a new plan from what is there now; approve its hash)for a host-allocation SKIP, do what its line says first (goatbuilder federate allocate <partner>, or name the hosts in ops/ansible/inventory/peering-edges.ini, or set the named value in group_vars), then make a new plan: a new plan alone records the same SKIPgoatbuilder federate plan <partner> --accept host-allocation (when you have judged that SKIP: it is recorded with its exact detail, and deploy goes ahead only while it reads the same) |
docs/design/goatbuilder.md |
1 |
GB-189 |
goatbuilder | a stage of goatbuilder federate was asked for the wrong way — The stages status, deploy and test take the partner after them and run their own phases; --plan and --approve-plan belong to deploy, whose start is phase 10 or, once the admission is done by hand, 11. Nothing ran. |
the partner was left out, or given after a peering name instead of a stage word --to, --resume or a --from other than 10 or 11 was given to a stage --plan or --approve-plan was given outside deploy--bilateral was given outside testrotate <partner> <key> named a key whose rotation is designed and not built in this goatbuilder (transport or signing; this goatbuilder rotates the leaves and the carrier keys), or a key it does not know; the message says what is missing (for transport, that the partner cannot yet receive our new package) |
goatbuilder federate deploy <partner> --approve-plan <hash> (or --from 11 after the admission)goatbuilder federate test <partner>goatbuilder federate rotate <partner> leaves (or carrier: the keys this goatbuilder rotates; the message lists them) |
docs/design/goatbuilder.md |
1 |
GB-282 |
goatbuilder | rotating our CA needs a decision before it can be built — goatbuilder federate rotate <partner> ca would replace the CA that signs our peering leaves. In package v1 every check accepts exactly one CA per party, the one fingerprint our package carries (federation-package-spec.md §5.1), so the partner could not trust the new CA beside the old one while our leaves change over, and data would stop. The overlap of both CAs is decided (ADR 1242: our next CA is announced in package v2) and designed in the key-rotation design §4, but this goatbuilder does not build it yet. Nothing ran. |
rotate <partner> ca was asked for, and this goatbuilder does not yet build the CA overlap ADR 1242 decides |
rotate the leaves instead, under the same CA: goatbuilder federate rotate <partner> leavesread ADR 1242 and the key-rotation design §4 for the CA overlap a later goatbuilder builds |
docs/design/federation-key-rotation.md |
1 |
GB-190 |
goatbuilder | the admission on our core is not recorded as done for what is rendered now — deploy --from 11 skips phase 10, the by-hand admission of the partner's edge on our core, so it runs only when the journal records phase 10 as done for exactly the files rendered now. It does not, so nothing changed: either the admission was never declared, or the rendered files changed after it. |
deploy has not stopped at phase 10 yet, so the admission was never done for these files the files were rendered again after the admission was declared |
goatbuilder federate deploy <partner> --approve-plan <hash> (it stops at phase 10; do the admission, then run it again with --from 11) |
docs/design/goatbuilder.md |
1 |
GB-191 |
goatbuilder | a value for confirmation 1 does not match — At phase 5 or 6 a value given by flag or typed is not the one the files hold: their CA fingerprint is not their package's, their CA certificate does not carry that fingerprint, their verifying key is not 32 bytes of base64 or does not verify their half, a sovereign id is not a plain id or is not the declared counterparty, or the agreement digest differs. The run stopped; at phase 5 no keyring was written, and at phase 6 the composed agreement is left for inspection. | a value was misheard or mistyped on the call the partner published a new package or CA after reading theirs aloud the two sides composed different agreements (their halves differ) |
read the values back group by group with the partner, then run the same command againgoat federate diff <ours> <theirs> (names the field where two agreements differ) |
docs/design/goatbuilder.md |
1 |
GB-192 |
goatbuilder | confirmation 1 is waiting — Phase 5 or 6 was asked to confirm (by --confirm or a value) and something is not there yet: their half or CA certificate has not arrived, a value read aloud was not given and there is no terminal to ask at, or the person did not type yes. The run stopped there, as a by-hand stop does (exit 0); a stop at phase 5 writes no keyring. | the partner has not published yet one of --partner-key, --partner-ca-fingerprint, --agreement-digest is missing |
run the same command again with the values read aloud, or with --confirm at a terminal |
docs/design/goatbuilder.md |
0 |
GB-193 |
goatbuilder | our verifying key cannot be read, or does not verify our half — Phase 5 shows our verifying key to read aloud, derived from the Ed25519 seed that signs our package (FED_SIGNING_KEY: 64 hex characters or 32 bytes base64). It is unset, unreadable or not a seed, or the key it gives does not verify the signature on our half, so the partner would bind a key that cannot verify our package. | FED_SIGNING_KEY is not set in this environment it names a file that is not the package-signing seed, or a different seed than phase 3 signed with |
set FED_SIGNING_KEY to the seed that signed our package (the same one phase 3 uses) |
docs/design/goatbuilder.md |
1 |
GB-194 |
goatbuilder | the edge cannot be admitted on our core — Phase 10 by the command (confirmation 2) did not change our core: the agreement digest has not been confirmed with the partner for the agreement in place, --approve-admission names another digest (something changed since it was read), the deployment does not say where our core's access control lives or how it restarts, their CA does not match the agreement, or the fragment could not be merged, assembled or placed (what was placed is put back). | phase 6 ran without --agreement-digest (or --confirm), so the digest was never compared the files were rendered again after the summary was read FED_CORE_ACL_ORG, FED_CORE_PLAY or FED_CORE_RESTART, or FED_SLOT_* is not set their CA file holds more than the one certificate the agreement names |
compare the digest with the partner: goatbuilder federate <partner> --from 6 --to 6 --agreement-digest <digest>read the summary again and approve the digest it shows nowset the FED_CORE_* settings in the deployment's environment, or run without the admission flags so phase 10 stops, admit by hand, and run --from 11 |
docs/design/goatbuilder.md |
1 |
GB-195 |
goatbuilder | the core's dry run or the slot check refused the admission — After placing the new access control (nothing loads it before a restart), the core router's dry run failed, or the committed slot list differs from the registry or the registry could not be read. Restarting now could drop or add stores, so the files placed were put back and the core was not restarted. | the assembled configuration does not pass the core's play in check mode the slot list in the repository is behind the registry |
run the dry run by hand to read why (the message prints it)regenerate the slot list with ops/scripts/sync-slot-anchors.py, commit it, deploy the core, then run phase 10 again |
docs/design/goatbuilder.md |
1 |
GB-196 |
goatbuilder | the core restart failed after the admission was placed — The new access control is in place on our core, and the restart through the deployment's play or command failed, so it may not be loaded. | the play or restart command failed (its output is above) | fix the cause, then run the same stage again with --approve-admission <digest> |
docs/design/goatbuilder.md |
1 |
GB-197 |
goatbuilder | the admission is waiting for approval — Phase 10 was asked to admit the edge (by --confirm, --approve-admission or --review-in-pr) and stopped without changing our core: the summary was written for review in a pull request, the person did not type yes, or there is no terminal and no --approve-admission. Exit 0, as a by-hand stop. | --review-in-pr, or pr at the terminal no approval was given |
read the summary (or the review in <work>/admission/review.md), then run the same stage with --approve-admission <digest> |
docs/design/goatbuilder.md |
0 |
GB-200 |
goatbuilder | the federation plan's inputs changed — an input the saved plan rests on (versions, the agreement, a rendered file, a certificate, a resource or the acceptance policy) differs from what the plan recorded, so the approval given to that plan does not cover this change. | a phase rendered, minted or composed again after the plan was made the checkout, the goat binary or a declared value changed after the plan was made |
goatbuilder federate plan <partner>, then review and approve the new plan |
docs/design/federation-plan.md |
1 |
GB-201 |
goatbuilder | the federation plan is missing or altered — |
no plan was made for this working directory the plan file was edited by hand after it was written |
goatbuilder federate plan <partner> --work <dir> |
docs/design/federation-plan.md |
1 |
GB-202 |
goatbuilder | a live prerequisite of the federation plan no longer holds — rerun immediately before a change, a live phase-1 check (ports, slot list, core router, receive stores) did not pass, so nothing may change. SKIP is not a pass here. | another service took a port the plan expected free a slot was minted after the slot list was fixed the core router was replaced or stopped |
fix the cause the check names (its own code is shown beside GB-202), then goatbuilder federate plan <partner> --recheck |
docs/design/federation-plan.md |
1 |
GB-232 |
goatbuilder | a rendered deployment half needs a site value no declaration yields — The peering declares deployment: rendered, so goatbuilder federate allocate renders the edge's and the core's deployment halves (peering-deployment.yml). Each required site value is taken from the group_vars of the environment's groups (environments[].groups) or all when one sets it, and is otherwise derived from the declaration that owns it: the core's address from peerings[].hosts.core or its line in the peering inventory, the core router's port from goat_zenoh_listen_port (group_vars, else the goat-zenoh role default), the images from the role defaults and ops/pins/fleet-pins.yaml, the core's certificate name from its name in the site inventory (ops/ansible/inventory/*.ini). One of them could be found in none of these places; the message names each place it read. Nothing is invented and nothing was written. | the peering declares no hosts: and the peering inventory has no single host for its core group no role default and no fleet-pins component pins the image the message names the site values live in a group the environment does not name in environments[].groups |
declare what the message names where it looked (peerings[].hosts, the core group's inventory line, the image pin)or set the named variable once in the group_vars of one of the environment's groupsor declare deployment: hand on the peering and keep writing its deployment halves by hand |
docs/design/goatbuilder.md |
2 |
GB-233 |
goatbuilder | a peering's federation.toml cannot be generated from the environment's — The peering declares federation_toml: rendered, so goatbuilder federate allocate generates its federation.toml from the environment's own ops/federation/ |
the environment has no ops/federation/ sovereign_id, term_days or cut_contact is missing from it |
write ops/federation/<environment>/federation.toml once for the environment (the template goat federate init prints is the shape)or declare federation_toml: hand on the peering and keep writing its federation.toml by hand |
docs/design/goatbuilder.md |
2 |
GB-160 |
goatbuilder | two peerings on one forwarder would share a value — The peerings one forwarder relays must share no carrier interface, public port, carrier port or carrier address on it: the forwarder sends each packet by its port alone. Allocation keeps them apart across every environment the forwarder relays (by its endpoint); this refusal is the backstop for values that collide anyway. | a hand edit of peering-values.yaml gives two relayed peerings one forwarder value a pin under values: (public_port, carrier_port, carrier_interface, carrier_subnet) repeats another peering's value |
pin different public_port, carrier_port, carrier_interface and carrier_subnet values under one peering's values:or give each environment its own forwarder (forwarder: group:) |
docs/design/peering-forwarder.md |
2 |
GB-161 |
goatbuilder | a peering's partner id cannot name a forwarder entry — The forwarder names each peering it relays by the partner id, and its role accepts 2-16 characters of a-z 0-9 and hyphen. This partner id is outside that, so the forwarder would refuse the entry. | the peering's with: holds an underscore, or is one character or longer than 16 | declare the partner by its sovereign id (lower-case letters, digits, hyphens) |
docs/design/peering-forwarder.md |
2 |
GB-162 |
goatbuilder | one forwarder group names two forwarders — The forwarder's peerings are rendered into its inventory group's group_vars, so every host in the group would relay every peering. Two peerings name the same group with different forwarder endpoints. | two forwarders declared without forwarder: group:, so both took the default peering_forwarders | give each forwarder its own group under forwarder: group: |
docs/design/peering-forwarder.md |
2 |
GB-163 |
goatbuilder | a forwarded peering has no allocated values yet — The peering declares forwarder:, so phase 2 runs the forwarder play and copies the carrier keys, and both read the group_vars goatbuilder federate allocate renders from the allocation record. The record does not hold this peering, so those files do not exist. Nothing ran. | forwarder: was added to the peering and goatbuilder federate allocate was not run since | goatbuilder federate allocate (then review and commit what it writes) |
docs/design/peering-forwarder.md |
2 |
GB-203 |
goatbuilder | two peerings would bind the same value on one machine — this peering and another declared peering would bind the same WireGuard interface, tunnel UDP port or republisher metrics port, or overlapping link subnets, on the same machine (by inventory address), so one would break the other. | two environments, or two hand-named inventory groups, place their edges or cores on one machine a value was pinned or declared by hand without looking at the other peerings on that machine |
give this peering another value in its edge or core group's vars (or its allocation pin), then goatbuilder federate plan <partner> |
docs/design/federation-plan.md |
1 |
GB-220 |
goatbuilder | federate backout was not given a partner it can act on — goatbuilder federate backout takes one partner: a sovereign id (2-16 characters of a-z0-9-) that is not a federate subcommand. The working directory and edge group must be plain paths and names, because they go into the play's command line. Nothing ran. | the partner was left out, or is not a sovereign id --work or --edge-group holds a quote, a space or a shell character |
goatbuilder federate backout <partner> --dry-run |
docs/design/goatbuilder.md |
1 |
GB-221 |
goatbuilder | the back-out found a resource it cannot show is this peering's alone — The read-only survey found a resource under this peering's names that another peering holds (a WireGuard interface whose state file or peer names another partner, a carrier another peering's state names), or a core configuration it cannot prove changes by this peering alone (the live file is not seed + org fragment, partner ids overlap, a policy mixes partners). Nothing was changed on any host. | two peerings were given the same interface, carrier or stack name in their group values the core's live configuration was edited by hand after the last assemble the core keeps no seed/org split (slot-pool mode), so the admission cannot be located |
read the REFUSED lines of goatbuilder federate backout <partner> --dry-run; give each peering its own names, or reconcile the core's live file with its seed and org fragment |
docs/design/goatbuilder.md |
1 |
GB-222 |
goatbuilder | the back-out survey did not complete — ops/ansible/peering-backout.yml failed before every host of the peering answered, or no host answered while the working folder holds the peering's agreement, so there is no plan to show and nothing was changed. | a host of neither ops/ansible/inventory/peering-edges.ini nor a per-peering peering- |
read the play output above; fix reachability or the inventory, then goatbuilder federate backout <partner> --dry-run |
docs/design/goatbuilder.md |
1 |
GB-223 |
goatbuilder | the slot recheck did not allow the core router restart — Immediately before the core router restarts without this peering's admission, the committed slot list is checked against the registry, as the admission path does. It drifted, the registry could not be read, or no registry was named, so the core was not restarted. The edge side of the peering is already removed and the partner has no path to the core. | slots were minted or revoked since the list was committed FED_SLOT_DB (or FED_SLOT_REGISTRY_STORE=postgres with FED_SLOT_PG_SSH) is not set for a deployment with lazy slot storage |
regenerate and commit the slot list (ops/scripts/sync-slot-anchors.py --env <env> ...), then goatbuilder federate backout <partner> |
docs/design/goatbuilder.md |
3 |
GB-224 |
goatbuilder | the core router was not restarted on the reviewed candidate — The core step installs only the candidate configuration whose digest the survey printed. The core's files changed since the survey (so the candidate's digest differs), more than one core router was found, or the restart failed. The org fragment kept beside it as |
another change reached the core between the survey and the restart several containers qualify as the core router |
goatbuilder federate backout <partner> --dry-run, review the new candidate, then run it againname the router with peering_backout_core_container or FED_ROUTER_CONTAINER |
docs/design/goatbuilder.md |
3 |
GB-225 |
goatbuilder | resources of the peering are still present after the back-out — The survey run after the removal still finds resources of this peering, or the removal play stopped partway. Every step is idempotent: running the back-out again removes what is left and reports the rest as absent. | a host became unreachable during the removal a unit or container would not stop |
goatbuilder federate backout <partner> |
docs/design/goatbuilder.md |
3 |
GB-226 |
goatbuilder | our package for the counterparty could not be withdrawn from the directory — The back-out asks the peering directory to delete /v1/parties/ |
FED_SIGNING_KEY is not set, or is not the key the directory binds to us the directory is unreachable |
python3 ops/federation/standup/directory_withdraw.py --url <directory> --us <us> --them <partner> --key <signing key> |
docs/design/peering-directory-service.md |
0 |
GB-227 |
goatbuilder | the core still admits the partner through its own configuration — The core router was configured by roles/goat-zenoh from zenoh_sovereign_peerings, which declares the partner's subject and grants as sovereign- |
the core admits the peering from a zenoh_sovereign_peerings entry the back-out did not write, or could not take out the role's play failed after the entry was taken out, and the entry was put back |
remove the partner's entry from zenoh_sovereign_peerings by review where it is declared and run the core play (a router restart); the role then reports sovereign_from_<partner> in .undeclared-storesafter a failed play: read its output, then goatbuilder federate backout <partner> again |
docs/operations/sovereign-federation-runbook.md |
3 |
GB-270 |
goatbuilder | a backed-out partner's store was kept because the router does not yet run without it — On a goat-zenoh role core, the back-out took the partner out of zenoh_sovereign_peerings, ran the role's play to restart the router without the partner's store, then ran ops/ansible/zenoh-drop-partner-store.yml, which deletes |
the role's play failed before or during the router's restart the router did not restart after the configuration without the store was written the partner is declared in another of the core router group's files |
read the role play's output above, then goatbuilder federate backout <partner> |
docs/design/federation-backout-partner-data.md |
3 |
GB-276 |
goatbuilder | --mock was given to a verb that reads the checkout — --mock runs a verb against a fixture directory instead of hosts. Only the verbs that drive or read hosts take one (the message lists them). This verb reads the checkout itself, so a fixture it was named would have been ignored, and the run would have reported on files the person never pointed it at. Nothing ran. | --mock given to a verb that reads the checkout, such as pins, validate or doctor | drop --mock: the verb reads the checkout (-C <dir> names another one)goatbuilder --mock <fixture> <verb> with one of the verbs the message lists |
docs/design/goatbuilder-conformance.md |
2 |
GB-277 |
goatbuilder | an edge result cannot be imported — On a two-driver peering (drivers: two), the controller imports the file the edge driver wrote for a phase's edge half with goatbuilder federate import <peering> <phase> <file> --sha256 <code>, the confirm code the edge driver printed. The import refused it: the file cannot be read; the code is not hex, gives fewer than the 32 hex digits the edge driver prints, or is not the start of the file's sha256; the file is not that phase's edge result for this peering (another schema, another peering, another side); or this peering's journal here is not waiting on the edge driver at that phase. Nothing was placed or recorded. |
the file was changed, or not copied whole, on its way from the edge driver the confirm code was mistyped, or read from another run of the edge driver the file is another peering's, another phase's, or the core side's result the controller's run has not reached the phase's hand-off in this working directory |
copy the file the edge driver wrote again, and give the confirm code it printed: goatbuilder federate import <peering> <phase> <file> --sha256 <code>goatbuilder federate <peering> --from <phase> (on the controller, to reach the hand-off first) |
docs/design/federation-two-drivers.md |
1 |
GB-286 |
goatbuilder | a joining site has no leaf-cert manifest join-node can read — join-node mints a joining site's router and service leaves from the site's manifest (ops/enrollment/manifests/ |
the site's manifest was never written the manifest has a YAML error |
cp ops/enrollment/manifests/TEMPLATE-certs.yaml ops/enrollment/manifests/<site>-certs.yaml, then re-thread every value for the site: its URI SANs name this goatnetops/scripts/mint-site-certs.sh <site> --dry-run prints what the manifest mints, and names a YAML error |
docs/design/goatbuilder.md |
1 |
GB-287 |
goatbuilder | a joining site's leaf-cert manifest has no router leaf — join-node deploys a joining site's data router (zenoh-deploy.yml), whose play reads the router leaf that mint-site-certs.sh mints from the site's manifest. The manifest names no leaf called router, so the plan was refused before any phase ran. | the manifest was written without the router entry ops/enrollment/manifests/TEMPLATE-certs.yaml carries | add a leaf named router to ops/enrollment/manifests/<site>-certs.yaml, as the template shows, with the site's own SANs |
docs/design/goatbuilder.md |
1 |
GB-288 |
goatbuilder | a health publisher a joining site declares has no play in the checkout — join-node deploys each health publisher the end-state model marks present at a joining site (endstate.py model |
the checkout is an older or partial tree that does not carry the publisher's play the pin table declares a publisher at the site that the site does not mean to run |
run join-node from a checkout that carries ops/ansible/<publisher>-deploy.ymldeclare the publisher absent at the site in ops/pins/fleet-pins.yaml, with its reason: components: <publisher>: deviations: <site>_hosts: {absent: true}goatbuilder endstate <env> shows what the model marks present at each site |
docs/design/goatbuilder.md |
1 |
GB-296 |
goatbuilder | a site turned on the audit spool, which is not built — A site set audit_subscriber_spool_enabled: true. The spooling audit subscriber a phone site needs (docs/design/island-mode-audit-spool.md) is designed but not built, so there is nothing to deploy for it. The audit-subscriber role stopped before it changed anything. | a phone site's group_vars set audit_subscriber_spool_enabled: true before the spool exists | set audit_subscriber_spool_enabled: false in that site's group_vars: the end state then reports the site's own statement that its fabric has no audit |
docs/design/island-mode-audit-spool.md |
2 |
GB-293 |
goatbuilder | the back-out cannot tell what our core router's leaf held before this peering — Our core router's leaf (its certificate and key) holds the leaf this peering's admission placed, and no admission run in the peering's working folder recorded what was there before (both peerings admitted by hand, say), so the back-out can neither put it back nor remove it. No other peering is set up from this machine (no working folder beside this one holds an agreement); when one is, the leaf is kept for it and the back-out goes on. Nothing was changed on any host. | the leaf was placed by hand, not by goatbuilder federate admission the admission folder of the peering's working folder was removed or moved |
read the unknown core leaf lines of goatbuilder federate backout <partner> --dry-run; put back what each path held before this peering, or remove it when nothing did, then run the back-out again |
docs/design/federation-backout-partner-data.md |
1 |
GB-280 |
goatbuilder | a joining site has no mesh DNS zone to serve — join-node deploys a mesh DNS replica on a joining site's core node, so the site still resolves the goatnet's names when it is cut off from its founder (ADR 1235). The replica serves the goatnet's generated zone, an ops/dns/ |
the goatnet's zone was never generated or committed (ops/dns/generate-mesh-hosts.py) neither the joining site's nor the founder's group_vars declare site_dns_zone |
python3 ops/dns/generate-mesh-hosts.py, or --init <site_id> for a zone that has no file yet, then commit ops/dns/<dir>/mesh-hostsdeclare site_dns_zone in group_vars/<site>_hosts/vars.yml |
docs/adr/1235-per-site-mesh-dns-nameserver-groups.md |
1 |
GB-281 |
goatbuilder | the founder's management API token for mesh DNS is unset or unresolved — join-node's mesh DNS step registers the joining site's nameserver group through the founder's management API, with the token in |
the variable is not stored under the declared secrets posture the store's tool is not signed in, so with-secrets could not resolve the reference |
python3 ops/secrets/posture.py hint <FOUNDER>_NB_API_TOKEN says where to store itsign in to the declared store's tool, then re-run: goatbuilder join-node <environment> --site <site> --node <node> --class core --address <address> --from 13 |
docs/adr/1235-per-site-mesh-dns-nameserver-groups.md |
1 |
GB-299 |
goatbuilder | no phone, or no Linux VM on it, answers over the Android debug bridge — join-node --via usb reaches a phone node through the Android debug bridge (adb) on a USB cable: SSH is carried by adb shell nc <the VM's address> 22, and the address is looked up on every connection (ops/scripts/phone-node-ssh.sh). adb is not installed, lists no phone as device, the phone is locked, or the phone's Linux VM has no address. Nothing reached the node. |
the phone is not attached, USB debugging is not allowed on it, or this machine's key was not accepted on its screen the phone is locked the Terminal app is closed, so the VM is not running more than one device is attached and ANDROID_SERIAL does not name the phone adb (Android platform tools) is not installed on this machine |
adb devices (the phone must be listed as device)unlock the phone and open its Terminal app, then resume: goatbuilder join-node <environment> --site <site> --node <node> --class phone --via usb --from <phase>ops/scripts/phone-node-ssh.sh --vm-address (prints the VM's address, or says which of the above it is) |
docs/design/phone-node.md |
1 |
GB-300 |
goatbuilder | the phone's Linux VM is not a host a node can be built on — For a phone node, phase 2 of join-node prepares the phone's Linux VM (Debian's kernel with its boot settings, Docker Engine, the deploy user) and then runs the integrity checks in the VM (ops/scripts/phone-node-provision.sh, ops/scripts/phone-node-integrity.sh). The VM could not be prepared, did not come back on Debian's kernel, or failed a check; the line before this one names which. No later phase ran, so nothing is built on the VM. | the VM started again on the kernel Android supplies, or without the boot settings file reads differ between cache drops, a write does not read back the same, or an installed file fails dpkg --verify: the VM returns wrong file data the kernel log holds a filesystem or I/O error the WireGuard module does not load, or a container does not run the phone does not support a VM, or has no Linux Terminal app |
read theintegrity:line the phase printed; it names the checkops/scripts/phone-node-provision.sh --node <node> --check (the checks alone)a VM that returns wrong file data is reset and prepared again: ops/scripts/phone-node-provision.sh --node <node> --resetthen resume: goatbuilder join-node <environment> --site <site> --node <node> --class phone --via usb --from 2 |
docs/design/phone-node.md |
1 |
GB-301 |
goatbuilder | a site does not hold what the machine class phone declares — The machine class phone declares once, in ops/ansible/group_vars/class_phone.yml, what a phone node differs in: no LAN address, no public ingress, no disk encryption inside the VM, memory limits for 4 GB, no observability host. The config preflight (standup-config-preflight.py |
the site's group_vars carry a value copied from a site of another class the node's line was not added under [class_phone] in ops/ansible/inventory/mgmt-hosts.ini the node's line in ops/ansible/inventory/mgmt-hosts.ini carries an ansible_host the site is a founder: a phone node joins a goatnet |
python3 ops/scripts/standup-config-preflight.py <site>_hosts --class phone (each [class phone] line names the key or the inventory line)remove the key the line names from ops/ansible/group_vars/<site>_hosts/list the node under [class_phone] in ops/ansible/inventory/mgmt-hosts.ini, as the name alone |
docs/design/phone-node.md |
1 |
GB-297 |
goatbuilder | a joining node's clock is not synchronized or is too far from UTC — join-node phase 3 read the joining node's clock from its own time service (chronyc -c tracking, else timedatectl), before anything on the node changed. A Zenoh router refuses a timestamp more than 500 ms ahead of its own clock, and nothing tells the machine's owner, so the node must hold its clock within half that, 250 ms by default (JOIN_NODE_CLOCK_BUDGET_MS), of UTC. A node that is not synchronized to an outside reference, or is over the budget, is refused; a node whose clock could not be read is reported as not verified, never as passed. The rule is the federation preflight's check clock (GB-268). | the node's time service follows no reachable time source, or only the node's own local clock the clock is synchronized but still slewing a large step, or the time source itself is wrong |
on the named node: chronyc tracking (or: timedatectl show); repair its time synchronizationgoatbuilder join-node <env> --site <site> --node <node> --class <class> --address <address> --from 3 |
docs/design/federation-preflight.md |
1 |
GB-304 |
goatbuilder | grow was given a file that is not a goatlite — goatbuilder grow hands a goatlite over to the full preset, so it reads goatlite.yaml at the checkout root and needs it to declare preset: lite with a lite: block (ADR 1213). This file declares another preset, or has no lite: block, so there is nothing to grow. A checkout already on the full preset needs no grow. grow changes files in the checkout only, never a host: a finished grow is undone with git before it is committed (restore goatlite.yaml and the files whose generated header it took off, and delete ops/goatbuilder/goatbuilder.yaml), and goatbuilder grow --dry-run shows every write first. | goatlite.yaml declares preset: full, or no preset the lite: block is missing or is not a mapping |
goatbuilder validate goatlite.yaml (it says what the file declares)a full configuration is ops/goatbuilder/goatbuilder.yaml: goatbuilder validate |
docs/design/goatlite-growth.md |
2 |
GB-305 |
goatbuilder | grow would replace a configuration that already exists — goatbuilder grow writes the full configuration to ops/goatbuilder/goatbuilder.yaml. That file already exists and holds another configuration (not the one this goatlite grows into, which a run that stopped part way would have left), so grow refused before changing anything: it never replaces a configuration someone wrote. When the file carries grow's own header, the message says it is grow's file from an earlier run, since edited (or goatlite.yaml changed). grow changes files in the checkout only, never a host: a finished grow is undone with git before it is committed (restore goatlite.yaml and the files whose generated header it took off, and delete ops/goatbuilder/goatbuilder.yaml), and goatbuilder grow --dry-run shows every write first. | the checkout already declares a full configuration of its own the file was edited after an earlier grow wrote it, and goatlite.yaml is still there |
move ops/goatbuilder/goatbuilder.yaml aside, then goatbuilder grow --dry-runor merge what it declares into goatlite.yaml, then grow againgrow's own file from an earlier run, edited: merge the edits into goatlite.yaml and delete ops/goatbuilder/goatbuilder.yaml, or delete it to grow afresh |
docs/design/goatlite-growth.md |
1 |
GB-306 |
goatbuilder | the full configuration grown from a goatlite is incomplete — goatbuilder grow rewrote goatlite.yaml as a full configuration, and the result fails the schema or a cross-field rule, so it was not written. Most often the goatlite is the four-fact file of the tutorial: the lite preset took repo and components from the checkout, and a full configuration declares its own. The message lists each problem and names the fields to add. grow changes files in the checkout only, never a host: a finished grow is undone with git before it is committed (restore goatlite.yaml and the files whose generated header it took off, and delete ops/goatbuilder/goatbuilder.yaml), and goatbuilder grow --dry-run shows every write first. | goatlite.yaml declares no repo or components, and the checkout has no configuration to take them from a value in goatlite.yaml is valid for the lite preset but not in the full shape (the message names it) |
add repo: and components: to goatlite.yaml (the fixture ops/goatbuilder/fixtures/goatnet-from-scratch/00-goatlite shows both), then goatbuilder grow --dry-run |
docs/design/goatlite-growth.md |
1 |
GB-307 |
goatbuilder | an environment declares converger pull-agent, and the pull converger is not built — goatbuilder.yaml declares converger: pull-agent for this environment, one of its sites or one of their classes. No pull converger is built: no agent on a machine fetches and applies its class's release. The only converger on real hosts is push, and pushing to machines declared to pull would change what the declaration means once pull is built. So doctor refuses the environment, and deploy (its dry run too), release's deploy rungs, fastlane's deploy and rollback refuse it before any step. deploy --verify-only reads and is not refused; validate accepts the value, and --mock converges it on a fixture. | the environment, a site or a class was declared pull-agent ahead of the pull converger | declare converger: push-ansible for these machines in ops/goatbuilder/goatbuilder.yaml until the pull converger is builtgoatbuilder doctor --env <environment> (the env:<name>:converger row names each place pull-agent is declared) |
docs/design/goatbuilder-pull-converger-options.md |
2 |
GB-320 |
goatbuilder | a program the run starts could not be started — A goatbuilder verb runs its phases and helpers as programs: bash for a phase's command, python3 for a step written in Python (the preflight, the slot recheck), and the helpers a step names (the core's candidate planner). The operating system refused to start one: it is not installed, not on PATH, or not executable. Nothing of that step ran. | bash or python3 is not installed on the machine goatbuilder runs on, or not on PATH a helper the step names is missing from the checkout, or has lost its executable bit |
command -v bash python3 (install what is missing, then run the same command again)git -C <checkout> status (restore a helper the checkout is missing) |
docs/design/goatbuilder.md |
1 |
GB-321 |
goatbuilder | a file a step of this run wrote is missing or is not what it wrote — A step of a federate run leaves its result in the working folder for the next one to read: phase 1's preflight result and the slot recheck's, the agreement, a back-out host record, the rotation id, an edge result on a two-driver peering. The file the next step read was not there, was not JSON, or did not hold what that step writes (another peering's name, a malformed id). Nothing after it ran. | the step that writes it stopped before writing it (its own failure is above, or in its log) the file was edited, replaced or removed by hand between the steps a different peering's working folder was named (--work, FED_WORK) |
run the step that writes it again (the message names the file; its phase is the one before)remove a file the message says is malformed, then run the same command again |
docs/design/goatbuilder.md |
1 |
GB-322 |
goatbuilder | goatbuilder reached a state its own code rules out — goatbuilder checked one of its own assumptions and found it broken: a plan whose shape the code that builds it no longer matches, a verb dispatched to a place that never takes it, a rotation marked built with no run. No input of yours can cause this; it is a defect in this build. Nothing after the check ran. | a defect in this goatbuilder build | report it with the run record the run printed (goatbuilder runs show <id>) and the exact command |
docs/design/goatbuilder.md |
1 |
GB-323 |
goatbuilder | a file the run writes outside the working folder cannot be written — Some steps write outside the working folder: the back-out deletes the partner record it generated from the checkout, and init --side edge writes the edge half to the file --half-out names. That file could not be written or removed. | the checkout or the folder --half-out names is not writable by this user the disk is full, or the folder is on a read-only mount |
check the folder the message names is writable, then run the same command again |
docs/design/goatbuilder.md |
1 |
GB-324 |
goatbuilder | a setting in the environment has a value goatbuilder does not take — A goatbuilder setting read from the environment (a FED_* variable) has a value outside the ones it takes; the message names the variable and the values it takes. Nothing ran. | the variable was set by hand, or by an older script, to a value this build does not know | unset the variable, or set it to one of the values the message names |
docs/design/goatbuilder.md |
2 |
GB-325 |
goatbuilder | flags given to a federate stage or verb do not go together, or one they need is missing — A federate stage was given flags it cannot take together (--accept with --recheck, --recheck with --mock or --no-live, --side without the files that side reads or writes), a flag without the one it needs (--work for status without the partner, init --over-ssh without -C, a host flag such as --edge-host or --side on init without --over-ssh), or a flag naming something the saved plan does not hold (--accept |
a command line put together from two different runs' flags | run the stage with the flags the message names, apart where it says to run them apartgoatbuilder federate <stage or verb> --help |
docs/design/goatbuilder.md |
2 |
GB-315 |
goatbuilder | a program a step ran gave output or an exit goatbuilder cannot use — A step of a federate run started a program and reads its answer: the core's candidate planner prints a plan on stdout, and phase 1's preflight exits and writes a result whose ready verdict must agree with its exit. The program ran, but what it gave cannot be used: no plan on stdout, or an exit and a result that disagree or cannot be read. Its own words are in the message. Nothing after it ran. | the program failed before it gave its answer (its stderr is in the message) the program is a different version from the one this goatbuilder expects |
read the program's own words in the message, fix what it names, then run the same command againgit -C <checkout> status (a helper the checkout changed or is missing) |
docs/design/goatbuilder.md |
1 |
GB-316 |
goatbuilder | a confirmation could not be read from the terminal — A federate stage asked a person to type a confirmation (the back-out asks for the partner id) and reading the answer from the terminal failed. Nothing was changed. | the terminal closed, or stdin is not readable | run the same command again at a terminalpass --yes where the stage takes it, for a run without a terminal |
docs/design/goatbuilder.md |
1 |
GB-278 |
goatbuilder | the edge driver's checkout or carried files are not the controller's — On a two-driver peering (drivers: two), the edge driver runs a phase's edge half with the command the controller's hand-off names: goatbuilder federate <peering> --side edge --from <N> --to <N> --expect-commit <commit>, with one --expect <file>=<sha256> per file the controller carried. The edge driver refused to run: --expect-commit was not given; its checkout is not a git checkout, has no commit, has changes, or is at another commit than the one named; or a carried file is not under its |
the edge driver's checkout was not updated to the controller's commit, or has local changes the command was typed without the hand-off's --expect-commit a carried file was not copied from the controller's |
git -C <checkout> fetch && git -C <checkout> checkout <commit> (the commit the hand-off names), and commit or remove every changed filecopy each carried file from the controller's <work> to the same path under the edge driver's <work>, then run the hand-off's command again |
docs/design/federation-two-drivers.md |
1 |
GB-289 |
goatbuilder | the IPsec rotation's staged material is missing or unreadable — A credential rotation of an ipsec-fips peering (goat-peering-edge with peering_edge_rotation set) switches the edge to a new leaf that the offline ceremony issued and an operator staged beside the live files: |
the new leaf, its key, or (for ipsec-ca) our new CA was not staged on this edge peering_edge_local_ca_path is not set, or names no file on the edge the staged key was copied without mode 0600 |
stage what the ceremony issued at the paths the message names, with the key at mode 0600, then run the rotation againset peering_edge_local_ca_path in the edge's group_vars to our current per-peering IPsec CA certificate |
docs/design/federation-key-rotation.md |
2 |
GB-290 |
goatbuilder | the staged IPsec certificate is not right for this link — Before an IPsec credential rotation moves anything, goat-peering-edge checks the staged certificate: its key is the staged key, it chains to the CA it must chain to (our current CA for ipsec-leaf, the staged new CA for ipsec-ca), its subject CN is peering_edge_local_id, its key is ECDSA P-384 (secp384r1) as the proposals require, it is valid now, and it stays valid for peering_edge_min_valid_days more days (default 30). The message names each check that failed. Nothing was moved. | the leaf was issued for another peering or another identity the leaf was issued by another CA, or the staged key belongs to another certificate the leaf uses another key type, or expires too soon |
have the ceremony issue the leaf again with Subject CN = peering_edge_local_id and an ECDSA P-384 key, from the right CA, then stage it and run the rotation again |
docs/design/federation-key-rotation.md |
2 |
GB-302 |
goatbuilder | the IPsec link did not come up on the new leaf; the old pair was put back — After switching to the staged leaf, goat-peering-edge loaded the credentials, re-authenticated the IKE SA and waited up to peering_edge_rotation_confirm_seconds (the DPD timeout by default) for an established IKE SA with an installed CHILD SA. None came up, so it put the set-aside pair back from .previous- |
the partner does not trust the CA that issued the new leaf the partner's edge expects another identity, or is not reachable this rotation id was rolled back by an earlier run |
read swanctl --list-sas and the charon log on both edges, then stage the new pair again and start a rotation with a new id |
docs/design/federation-key-rotation.md |
2 |
GB-303 |
goatbuilder | the partner has not read back our new IPsec CA — An ipsec-ca rotation switches the edge to a leaf issued by our new per-peering CA only once the partner trusts that CA. Nothing on our side can read the partner's trust store, so the switch needs the sha256 fingerprint the partner read back after placing our new CA (peering_edge_rotation_ca_readback, or goatbuilder federate rotate |
the partner has not placed our new CA yet, or has not read its fingerprint back on the call the fingerprint read back belongs to another file |
have the partner place our new CA beside their current one and read back its sha256 fingerprint, then run the rotation again with that fingerprint |
docs/design/federation-key-rotation.md |
2 |
GB-298 |
goatbuilder | an IPsec rotation under way, or not yet retired, is not the one asked for — goatbuilder federate rotate |
an earlier rotation stopped before phase 11 passed, and a rotation of the other kind was asked for the partner's read-back changed between the run that started the rotation and this one an earlier rotation switched and its retire step has not run the record was emptied or edited by hand |
finish the rotation the record names with the command that started it (the message names its id and kind)retire the earlier rotation once phase 14 passed here after the switch (and the partner says theirs did): goatbuilder federate rotate <partner> ipsec-leaf\|ipsec-ca --retire <id>removing <work>/.rotation-ipsec starts a new rotation id; the edge keeps /etc/swanctl/.previous-<id>/ as it is (after a GB-302 rollback, with the old pair live), and nothing removes it yet |
docs/design/federation-key-rotation.md |
1 |
GB-309 |
goatbuilder | the retire step finds no confirmed rotation to retire — The retire step of an IPsec credential rotation (goat-peering-edge, peering_edge_rotation_retire) removes the set-aside key in .previous- |
the id is mistyped, or names a rotation of another edge the rotation stopped before its switch, so the live pair is still the set-aside one the rotation was not confirmed: it stopped after the switch and before its confirm wrote .previous- the confirm's record is unreadable: it lacks the id= or at= line the confirm writes neither the IKE SA the confirm recorded nor one established since is up, or the live key is not the live certificate's goatbuilder federate rotate --retire: this side's ledger ( |
check the id the rotation printed (ls /etc/swanctl/.previous-*, or <work>/ipsec-rotations.json)run the rotation again with the same id: it re-authenticates, confirms and writes .previous-<id>/confirmed; retire once phase 14 passed here after the switch, and the partner says theirs did |
docs/design/federation-key-rotation.md |
2 |
GB-311 |
goatbuilder | an IPsec rotation stopped half way through its switch, and what is left does not finish it — The switch of an IPsec credential rotation (goat-peering-edge) moves the staged key, then the staged certificate, over the live names. A run killed between the two moves leaves one moved in and the other still staged. The next run finishes the remaining move when what is left is the moved one's match: the moved file is not the set-aside one, and the two halves are one key pair (each parsed by openssl; a file that does not parse matches nothing). Here they are not, so the live pair would be mixed: nothing was moved. | the staged file left behind was replaced after the run stopped someone moved or edited the live or set-aside files by hand |
restore the staged file the ceremony issued for this rotation, then run the rotation again with the same idor put the set-aside pair in /etc/swanctl/.previous-<id>/ back by hand, check it with openssl, and stage the new pair again |
docs/design/federation-key-rotation.md |
2 |
GB-328 |
goatbuilder | a rotation's values reach no play in the phases this run built — A key rotation of goatbuilder federate rotate passes its values (the rotation id, and for an IPsec rotation its kind and read-back) to the plays of one phase: phase 11 for ipsec-leaf and ipsec-ca, phase 2 for the peering key. It puts them before each play's -e fed_work=. The phase this run built has no such play, so the values would reach nothing and the phase would run as a plain one and report the rotation done. Nothing ran. The phases are built from the agreement each run, and the rotation already checked that the agreement chose its transport, so this is goatbuilder's own fault, not the peering's. |
the phase's command in this goatbuilder no longer runs its play with -e fed_work= |
goatbuilder federate rotate <partner> <key> --dry-run (it prints the phase's command)report the command and this goatbuilder's version (goatbuilder --version); do not run the phase by hand in its place |
docs/design/federation-key-rotation.md |
1 |
GB-351 |
goatbuilder | a Spock subscription has forward_origins set, which in a mesh replicates in a loop — A Spock subscription, or the setting that creates them, has forward_origins other than empty or 'none'. In a mesh of three or more sites such a subscription passes each change on to the next site, which passes it back, without end, while every subscription still reads replicating and nothing is logged. The spock-forward-origins-check role stopped the run before spock-mesh-peering or ory-narrow-repset changed anything. The message names the setting, or each subscription as |
a site's group_vars set spock_mesh_forward_origins or ory_spock_forward_origins a subscription was created by hand with forward_origins := '{all}' |
remove spock_mesh_forward_origins and ory_spock_forward_origins from the site's group_vars (both default to [])on the host and database named, run SELECT spock.sub_drop('<subscription>'); then run the playbook again, which creates it with forward_origins empty |
docs/design/spock-mesh-peering.md |
2 |
tools goat and goatbuilder call¶
| Code | Tool | What it means | Why | What to do | Read | Exit |
|---|---|---|---|---|---|---|
X-ANSIBLE-001 |
ansible | a host was unreachable — ansible could not open a connection to the host over the transport it was given (ssh over the mesh, or the door). | this laptop is not on the goatnet's mesh profile and no door was opened the host is down the SSH key is not accepted (see the ssh entries) |
goatbuilder doctor --env <env> --all (the transport row)ssh -o ConnectTimeout=5 <ssh user>@<mesh ip> true |
docs/design/env-doors.md |
|
X-ANSIBLE-002 |
ansible | sudo on the host wants a password — a task needed privilege escalation and the host's operator user is not NOPASSWD. | the host was not baselined (mgmt-host-baseline.yml sets NOPASSWD sudo) the wrong remote user |
run ops/ansible/mgmt-host-baseline.yml against the host once |
docs/operations/goatnet-standup.md |
|
X-ANSIBLE-003 |
ansible | a play assertion failed on the host — a task asserted a precondition, usually that a templated secret is non-empty, and it was false on the host. | a secret the play templates was empty in the deployer's environment (see GB-007) a group_vars fact missing for this goatnet |
goatbuilder deploy --env <env> --component <c> --dry-run (names every variable the play needs) |
docs/operations/goatnet-standup.md |
|
X-ANSIBLE-004 |
ansible | a task failed on a host — ansible marks the task fatal; its msg field says what the module could not do. | see the msg in the fatal line | grep -n -A3 'fatal:' <log> |
docs/operations/goatnet-standup.md |
|
X-ANSIBLE-005 |
ansible-preflight | the preflight refused a stale or unsafe checkout — ops/scripts/ansible-preflight.sh refuses to run a play from a tree that differs from origin/main in a safety-critical path, or from a stale trunk; ansible reads role defaults from the working tree, so a stale tree templates stale pins. | local main behind origin a feature branch that touches group_vars or roles |
git pull --ff-only origin main (the message prints the exact command) |
docs/operations/goatnet-standup.md |
|
X-SSH-001 |
ssh | the host refused the key — the connection reached the host and SSH rejected every key offered. | your public key is not on the host's operator user (ssh-key-self-service) the wrong user (hosts take the deploy.ssh_user account) the key is not loaded in the agent |
ssh -v <ssh user>@<host> true 2>&1 \| grep -i 'offering\\|denied'goat admin ssh-key … (register the key through the goatnet) |
docs/design/ssh-key-self-service.md |
|
X-SSH-002 |
ssh | no route to the host — the connection never reached the host: timed out, no route, or refused on port 22. | the address is on a network this laptop is not on (a mesh IP without the mesh, a Tailscale IP without the tailnet) the host is down |
netbird status \| head -5goatbuilder doctor --env <env> |
docs/design/env-doors.md |
|
X-SSH-003 |
ssh | the host key changed — the host presented a different key than ~/.ssh/known_hosts holds for that address. | the host was rebuilt (expected after a re-image) an address was reused by another machine |
confirm the rebuild with the operator, then: ssh-keygen -R <address> |
docs/operations/site-node-troubleshooting.md |
|
X-OP-001 |
op | 1Password CLI is not signed in — op needs a session for the account before it can resolve references. | no op signin in this shellOP_ACCOUNT names a different account than the one signed in |
eval $(op signin) (OP_ACCOUNT=<account> for the team store)goatbuilder doctor --credentials --all (the tool:op-session row) |
docs/design/secret-store-requirements.md |
|
X-OP-002 |
op | an op:// reference is malformed — op run validates every op:// value in the environment and refuses the whole run when one has fewer than three parts (vault/item/field). The variable is often one from your shell profile, not the refs file. | a variable in your shell holds a two-part reference (op://Vault/Item) | env \| grep 'op://' (find the variable)env -u <THAT_VAR> op run --env-file=… -- <command>or complete it to op://<vault>/<item>/<field> |
docs/design/secret-store-requirements.md |
|
X-OP-003 |
op | a reference does not resolve to an item — the vault, item or field named in an op:// reference does not exist, or this account cannot see it. | the item was renamed or moved to another vault the account lacks access to the vault |
op item get '<item>' --vault <vault>check refs/goat.refs in <owner>/goat-credentials for the current line |
docs/design/secret-store-requirements.md |
|
X-AWS-001 |
aws | AWS rejected the credentials — the access key is unknown to AWS, or a session token was sent where none is accepted. | aws-vault run without --no-session (STS tokens are rejected by this account) a rotated key still in the keychain |
aws-vault exec <profile> --no-session -- aws sts get-caller-identityaws-vault add <profile> (re-enter the current key) |
docs/design/cloud-infrastructure-discipline.md |
|
X-AWS-002 |
aws-vault | aws-vault has no credentials for the profile — the keychain holds no credentials under the profile name, or the keychain could not be opened (headless). | a fresh machine a headless session cannot prompt the keychain |
aws-vault add <profile>aws-vault list |
docs/design/cloud-infrastructure-discipline.md |
|
X-AWS-003 |
aws | the SSM target is not connected — Session Manager has no live agent for the instance, so the door cannot open. | the host is down or has no route to the SSM endpoint the instance id in doors.yaml is stale |
aws-vault exec <profile> --no-session -- aws ssm describe-instance-information --region <region> |
docs/design/env-doors.md |
|
X-AWS-004 |
aws | the AWS CLI fell back to an expired aws login session — the aws-vault credential for the profile is empty or malformed, so the CLI used an expired aws login session instead and asked to reauthenticate. |
the aws-vault credential is empty or malformed, so the CLI fell back to an expired aws login session AWS_VAULT_KEYCHAIN_NAME in the environment (a direnv export) points aws-vault at the wrong keychain |
aws login (when the error names it)env -u AWS_VAULT_KEYCHAIN_NAME aws-vault exec <profile> --no-session -- aws sts get-caller-identity (when the keychain name is set)aws-vault add <profile> (when the key really is bad; it replaces the stored key without deleting it first) |
docs/design/cloud-infrastructure-discipline.md |
|
X-GIT-001 |
git | not a git repository — the working directory is not inside a checkout. | run from the wrong directory | cd /path/to/goat (or pass -C) |
CONTRIBUTING.md |
|
X-GIT-002 |
git | the branch cannot fast-forward — local and remote histories diverged, so a --ff-only pull refuses. | local commits on main | git log --oneline origin/main..HEAD (what is local); move it to a branch, then git pull --ff-only |
CONTRIBUTING.md |
|
X-GIT-003 |
git | the remote refused access — the push or fetch was refused: the SSH key is not authorised for the repository, or the remote does not exist. | the wrong GitHub account for this remote the key is not loaded |
ssh -T git@github.comgh auth status |
CONTRIBUTING.md |
|
X-GH-001 |
gh | GitHub returned 404 — the repository, run, release or API path does not exist, or this account cannot see it. | a typo in the slug or id a private resource the account is not a member of |
gh auth statusgh repo view <owner>/<repo> |
CONTRIBUTING.md |
|
X-GH-002 |
gh | the GitHub CLI is not logged in — gh has no token for github.com in this environment. | a fresh machine GH_TOKEN set to an expired token |
gh auth login |
CONTRIBUTING.md |
|
X-GH-003 |
gh | GitHub rate-limited or forbade the call — HTTP 403 from the API: a rate limit, or a token without the scope. | many unauthenticated calls in a short time a token missing read:packages or repo |
gh api rate_limitgh auth refresh -s read:packages |
CONTRIBUTING.md |
|
X-COSIGN-001 |
cosign | the signature did not verify — cosign found no signature for the artifact under the expected identity, or the signature does not match. | the artifact was rebuilt without signing the wrong public key or identity |
cosign verify --key <org key> <image@digest> |
docs/reference/goatbuilder/02-install.md |
|
X-DOCKER-001 |
docker | no permission on the Docker socket — the user is not in the docker group on that host, so every docker call is refused. | running docker without sudo on a host where the operator user is not in the docker group | sudo docker … (hosts run docker as root by baseline) |
docs/operations/site-node-troubleshooting.md |
|
X-DOCKER-002 |
docker | the image could not be pulled — the registry answered that the manifest is unknown or the pull is unauthorised. | a tag that was never pushed, or a digest from the other GHCR org the host's registry login has expired |
crane digest <repo.image_namespace>/<image>:<tag>on the host: sudo docker pull <ref> |
docs/design/base-images.md |
|
X-DOCKER-003 |
docker | the Docker daemon is not running — the socket exists but nothing answers, or does not exist at all. | docker.service stopped on the host a local Docker Desktop that is not launched (never launch it for goat work) |
on the host: sudo systemctl status docker |
docs/operations/site-node-troubleshooting.md |
|
X-NETBIRD-001 |
netbird | no such mesh profile — the profile named does not exist on this laptop; profiles are per goatnet. | the profile was never added the hyphen/underscore form differs from the profile name |
netbird profile list |
docs/design/env-doors.md |
|
X-NETBIRD-002 |
netbird | the mesh daemon is not running — the netbird client could not reach its own daemon. | the service is stopped | sudo netbird service start (or restart the desktop client) |
docs/operations/battle-drills/02-mesh-client.md |
|
X-HTTP-001 |
http | the service rejected the credential (401) — the request reached the service and the token or cookie was not accepted. | an expired or wrong token | check which token the command is using; for goat, E_TOKEN_INVALID says how to rotate it |
docs/design/goat-cli-error-codes.md |
|
X-HTTP-002 |
http | the service refused the identity (403) — the credential was accepted and the identity is not allowed to do this. | the wrong one of two credentials (participant vs operator) no grant for this surface |
for goat admin verbs: GOAT_PORTAL_TOKEN=op_… goat … |
docs/design/identity-access-and-onboarding-reference.md |
|
X-HTTP-003 |
http | the service is up but its backend is not (5xx) — the front door answered with a gateway error: Traefik or Caddy is up and the service behind it is not. | the container is restarting or down | on the host: sudo docker ps --format '{{.Names}} {{.Status}}' \| grep -v healthy |
docs/operations/site-node-troubleshooting.md |
|
X-HTTP-004 |
http | an onboarding call went to the portal — goat sent an onboarding call (invite, approve) to the portal instead of the onboarding endpoint: the portal answers 405 Method Not Allowed, or the call names a host that is only portal. |
the context's profile has no onboard_endpoint, so onboarding calls went to the portal | GOAT_ONBOARD_ENDPOINT=<onboard url> goat … (for this run)goat --context <name> profile init <bundle.cbor> from a current bundle (the profile then carries onboard_endpoint); goatbuilder doctor's tool:goat-contexts row names the contexts |
docs/design/three-stage-candidate-onboarding.md |
|
X-DIRENV-001 |
direnv | direnv exec was given an assignment, not a command — direnv exec <dir> VAR=value cmd looks VAR=value up on PATH as the command and fails; direnv exec takes a command. |
an environment assignment placed where direnv exec expects the command | direnv exec . env VAR=value cmd |
docs/operations/admin-operator-guide.md |
|
X-DNS-001 |
dns | the name did not resolve — the resolver has no answer for the name; mesh names resolve only while the mesh is up. | off the mesh (mesh names are served by mesh CoreDNS) a typo in the name |
netbird status \| head -5dig +short <name> (then the same against the mesh resolver the profile names: goat names) |
docs/design/mesh-dns-redundancy.md |
|
X-TLS-001 |
tls | the TLS handshake failed — the certificate presented was not trusted, did not match, or the handshake ended early. | a CA this machine does not trust (goat ca add) a clock far enough off that the certificate reads as invalid |
goat ca lsdate -u (compare with a known-good clock) |
docs/design/internal-tls-ca.md |
|
X-SMOKE-001 |
onboard-intake-smoke | the live rows' browser toolchain is not on this machine — the live acceptance rows (intake-form-browser-submit, review-gate-accepts-open-field-invite) run ops/scripts/onboard-intake-smoke.sh from the deployer's laptop, and it exits 2 before touching the goatnet when ops/goat-id-console/web/e2e has no node_modules. The release did not fail on the goatnet; it failed here. | a laptop that never installed the console's Playwright suite | (cd ops/goat-id-console/web/e2e && npm ci && npx playwright install chromium)goatbuilder doctor --browser-tests (the tool:live-rows row fails until this machine is ready) |
docs/design/release-candidate-validation.md |
|
X-PY-001 |
python | a Python module is missing — python3 could not import a module a script needs. | a fresh machine or a venv not on PATH | pip install <module> (pyyaml, jsonschema are the usual two)a script Ansible runs on the control node uses ansible_playbook_python, which carries PyYAML; a task that still calls bare python3 should too |
docs/design/goatbuilder.md |
|
X-SECRETS-001 |
with-secrets | the declared secrets store is unknown — GOAT_SECRET_STORE or secrets: store: in goatbuilder.yaml names a store goat does not have, so no command can get its secrets. |
a typo in GOAT_SECRET_STORE or in goatbuilder.yaml a custom store whose ops/secrets/ |
goatbuilder secrets store (prints the store in force and where it came from)use one of: 1password, openbao, custom, sops, git-crypt, env-file, env |
docs/design/pluggable-dependencies.md |
|
X-SECRETS-002 |
with-secrets | the env-file store has no file — the declared store is env-file, and its file of values does not exist on this laptop. | the quickstart's env file was never generated a goatlite whose phase 1 has not run yet (phase 1 renders the goatnet, then creates this file) GOAT_SECRET_ENV_FILE points somewhere else |
on a goatlite: goatbuilder standup <name> --to 1 (renders the goatnet, then creates the file)on any other goatnet: goatbuilder secrets init --env <goatnet> (strong random values, 0600, gitignored) |
docs/design/pluggable-dependencies.md |
|
X-SECRETS-003 |
goatbuilder | refusing to write secret values to a file git would commit — secrets init writes plaintext values, so it refuses a path that is not gitignored. |
--file names a path inside the repository that .gitignore does not cover | add the path to .gitignore, or use the default .envrc.local.d/secrets.env |
docs/design/pluggable-dependencies.md |
|
X-SECRETS-004 |
with-secrets | bao:// references need the OpenBao injector — the environment holds bao:// references, and ops/secrets/openbao/run (the OpenBao store's injector, ADR 1203) is not on this checkout. | a .refs file holds bao:// lines on a checkout that predates the OpenBao store | git pull --ff-only origin main (the injector lands with the OpenBao store)or remove the bao:// lines and use op:// references meanwhile |
docs/design/pluggable-dependencies.md |
|
X-SECRETS-005 |
with-secrets | a git-crypt file is still encrypted — the declared store is git-crypt and its file has not been unlocked on this laptop, so its values cannot be read. | this clone was never unlocked | git-crypt unlock |
docs/design/pluggable-dependencies.md |
|
X-ACCEPT-001 |
acceptance.sh | acceptance could not run a required check, so the peering is not accepted — ops/federation/standup/acceptance.sh (phase 14) found no failure, but a required check was BLOCKED (its input or evidence was unavailable) or SKIPPED (not exercised). Such a check proves nothing, so the verdict in |
a surface the check reads is not configured (for example the partner's REST endpoint or the SPIRE servers) the agreement offers nothing in one direction, so that direction's checks had nothing to send a probe session the check needs was not started |
read the BLOCKED and skipped lines above the acceptance-counts line, or the why field of result.json; set what each names, then run phase 14 againgoatbuilder federate <peering> --from 14 |
docs/design/federation-acceptance.md |
|
X-DIGEST-001 |
image-digest | no tool could read the image's digest — ops/scripts/image-digest tried crane, skopeo and docker in turn; none is installed, or each refused, and each tool's own error is printed beneath. | none of crane, skopeo or docker is installed the registry refused this laptop's credentials (each tool's error says which) the tag does not exist yet (its image build has not finished) |
brew install cranegh auth token \| crane auth login ghcr.io -u <user> --password-stdin |
docs/design/pluggable-dependencies.md |