Runmoor Operations
Version note: Runmoor 0.2.0 introduced automatic setup and managed runner updates. Version 0.1.3 uses the explicit pinned configuration and manual image preparation also documented below. Check
runmoor versionbefore using 0.2.0 commands.
These commands manage a launchd or systemd user service with the same drain semantics. Install the binary at a persistent location first. Installation does not overwrite an existing service definition. Uninstall preserves data/configuration and refuses to abandon known live executions when the manager cannot be contacted. Run the service in a functioning user session; availability after logout/reboot depends on that OS session, and Runmoor does not change system login policy.
Service start, stop, and uninstall must use the same --config path that was
recorded when the service was installed. If Runmoor reports
CONFIG_INVALID because the requested configuration does not match the
installed service, retry with the installed configuration path. To replace a
service configuration, drain and uninstall the service with its current path,
then install and start it with the new path. On systemd, inspect ExecStart
and ExecStop with systemctl --user cat runmoor.service; on macOS, inspect
the Runmoor launch agent's ProgramArguments. Preserve an invalid service
definition and resolve its problem before retrying service commands.
Paths containing .. that resolve through a symlink to a different file are
also rejected; use the installed absolute configuration path directly.
If a Linux service action or a macOS service stop/uninstall reports that the
active manager does not match the installed service, gracefully stop it with
runmoor stop --config ORIGINAL_CONFIG_PATH, then retry the service action
with the installed configuration. This lets the active manager drain using
the configuration it was started with before systemd operates on the
replacement definition.
On macOS, stop and uninstall preserve the plist when launchd reports a loaded job without a running process, because Runmoor cannot verify the arguments cached by launchd in that state. Inspect and reconcile the loaded job in the logged-in GUI session before retrying. If the manager is running, stop it with its original configuration path first.
Reload an installed service after upgrading the CLI
Unreleased service reload: After you install a newer Runmoor CLI,
runmoor reloadalso checks the running launchd/systemd user service. If its manager is older, reload updates the service to that installed CLI and restarts only the manager. Active jobs retain their execution environment and timeout. Equal or newer managers reload settings without a version change. Foreground managers keep their existing reload behavior. Reload does not download releases, downgrade a manager or start a stopped service. Releases through 0.2.7 reload configuration only.
Install the new CLI with your existing package manager or verified archive, then run reload in the service user's login session with its installed configuration:
Reload verifies the service's configuration and running manager before replacing
its executable reference. It preserves pause decisions and a committed stop
request. Use runmoor service start explicitly for a stopped service. A directly
started runmoor run process requires your existing manual restart workflow to
change its version.
If reload fails or is interrupted, inspect runmoor status and the OS user
service. The service executable may already have changed while the last accepted
configuration remains in effect. Correct the reported problem and retry reload
with the same installed CLI and configuration. A successful reload confirms the
new running version and configuration acceptance. Keep state and managed data;
never use an older binary with a newer database. The drained backup and rollback
procedure below remains available.
If another tool or a manual edit conflicts with publication of the updated service definition, Runmoor preserves that edit and reports a conflict before replacing the running service. Resolve the conflicting definitions before retrying.
In this unreleased workflow, if Stop completed after an interrupted reload,
wait for the reload command to finish and run runmoor service start with the
same installed CLI and configuration. One Start resumes the verified inactive
service after cleanup completes and preserves pause decisions. If Runmoor cannot
verify ownership or cleanup is unfinished, Start reports a recovery error.
Preserve the service definition, recovery files, state and managed data while
you resolve that error; do not delete recovery files to force a start.
Recover an Ubuntu user service
Run these checks as the Runmoor user in a working login session, without
sudo. service install does not replace an existing definition. If it
reports that a definition already exists or cannot be created securely,
inspect the existing unit before retrying. A previous install can leave the
unit file in place when systemd's reload fails after file creation.
An inactive (dead) and disabled unit is present but not running. In the
unit shown by cat, check that ExecStart names the current Runmoor binary
and the intended --config file, especially after replacing the binary or
configuration. If systemctl --user cannot reach the user manager, log in as
the actual service user rather than running Runmoor through sudo.
When the unit paths are correct and the service is inactive, try
runmoor service start first. Runmoor checks that the unit is a regular,
owner-only file belonging to the current user and does not follow a symlink.
If that command reports exactly DEPENDENCY_UNAVAILABLE: User service command failed, its unit check succeeded and the systemd command failed. Only then
run the same systemd command directly to see its original error:
runmoor service uninstall asks Runmoor to drain and stop owned work, then
disables and stops the user unit. A failure during disable/stop leaves the unit
file in place, but a later systemd reload failure can be reported after Runmoor
has removed it. Both return DEPENDENCY_UNAVAILABLE: User service command failed. Check whether the unit file still exists in the user's systemd
configuration directory and inspect the direct systemctl --user output and
journal before retrying. After runmoor status confirms no active executions
or pending cleanup, the corresponding disable/stop command is
systemctl --user disable --now runmoor.service. Do not manually remove the
unit, state, or managed data while executions or cleanup may still be active.
Use the normal drain/stop path before replacing a service definition.
GitHub PAT for an Ubuntu user service
Create a GitHub PAT for the configured repository or organization. Run the following commands as the Ubuntu user who will run the service. They prompt without displaying the token or placing its value in shell history:
In ~/.config/runmoor/config.toml, replace the connection's environment
reference with a file reference using the absolute path printed by realpath:
mktemp creates a mode 0600 file and mv replaces the credential file without
making its contents public during rotation. TOML does not expand $HOME. Keep
the PAT value out of TOML and the service definition. The credential file must
be a regular file owned by the service user with mode 0600. Then validate and
start the user service:
Run service install only for a new service definition. For an already running
pool, replacing the PAT in the same file does not automatically refresh its
GitHub connection. Revalidate the credential and clear any authentication
suspension after rotation:
If you change the credential reference in TOML, accept that configuration before resuming the affected pool:
A terminal's export RUNMOOR_PAT does not automatically reach systemd's user
manager. If you keep credential = { env = "RUNMOOR_PAT" }, import it before
starting the service:
If the service was already running without the PAT, service start leaves that
process running with its old environment. After importing the variable, use
runmoor service stop, runmoor service start, and runmoor resume --pool linux
to restart the manager and clear the pool's authentication suspension. Service
stop drains active work before exiting. Supply the variable again when the user
manager restarts. A file reference remains available across those restarts. The
user service starts after logout or reboot only if your Ubuntu user-session
policy keeps its systemd user manager running.
Manager-only restart reconciles SQLite with actual Docker/Tart and GitHub state, resumes verified live work and retries incomplete cleanup. Ambiguous resources are quarantined rather than deleted. Confirmed termination releases resources; unresolved cleanup/ownership records remain durable. Runmoor never automatically reruns a failed GitHub job.
With the unreleased Docker cleanup fix, a container that replaces a
stopped runner or daemon remains untouched, even if its Runmoor labels
were copied. Cleanup reports OWNERSHIP_AMBIGUOUS and remains incomplete.
The original execution remains confirmed stopped and its capacity stays
released. Preserve the replacement and Runmoor state while investigating;
copied labels do not prove ownership.
For Tart, OWNERSHIP_AMBIGUOUS keeps the VM, its Runmoor records and its
capacity reservation intact, including after stop --force. Do not remove or
rename the VM to clear the warning. Follow Tart ownership recovery
to restore a matching backup or reimport a known source under a new identity.
A recorded job completion continues through cleanup even if GitHub has already removed its ephemeral runner registration. Capacity becomes available once the owned execution is confirmed stopped, while any remaining cleanup is retried. An upgrade does not automatically recover existing quarantines. For a previously affected completed job, confirm completion in GitHub and verify the exact ownership and stopped state of its local resources before recovering the affected pool with runmoor stop --pool NAME --force.
Back up only after drain and stop. Preserve the complete state and managed-data directories; protect referenced credential files separately. Install the new binary manually and start again. Roll back using a compatible binary and its matching drained state/data backup. Version 0.2.0 upgrades existing state automatically; back up before upgrading and use the matching backup to return to 0.1.3. Unsupported database versions fail without destructive migration; never reuse an older backup while resources created after that backup are still active.
If startup reports Possible legacy SQLite state exists at an ambiguous location, stop every Runmoor manager that may use either location. Preserve complete backups of both locations before recovery. Do not move or delete either database; the original state location may be ambiguous and requires explicit review.
Jobs retain timeout accounting across restart/sleep. Active work requests OS sleep inhibition; warm idle capacity does not keep the machine awake indefinitely. Failure is a warning and does not change system power settings. Forced sleep, lid closure, shutdown and power loss can still interrupt work.
Managed runner updates
While the manager runs, automatic pools check for a new stable runner release at startup and every hour. Request an immediate check with:
Download and validation finish before the replacement activates. Existing jobs finish with their original environment, and replacements using the same scale set wait for the previous generation to drain. An update gets the next available preparation slot without terminating busy jobs. Pause, drain and stop remain in effect during updates.
If GitHub, the image registry or validation is temporarily unavailable, Runmoor keeps the last verified environment and retries with backoff. Status reports the current version, candidate, last check, next retry and failure or capacity wait. A failed first installation waits because no fallback exists. If the known GitHub update deadline passes, new work waits until a supported image is ready; active jobs still finish. Unknown freshness is shown explicitly. GitHub can require urgent security updates earlier than its usual 30-day window.
Current, previous and actively referenced generated images are retained. Unreferenced generated artifacts are cleaned up without deleting user source images or force-removing externally referenced Docker images. Official pulled base layers remain available in Docker. Runmoor itself, Docker, Tart, Guest Agent, macOS and Xcode still use their existing manual/package-manager update workflows.
Troubleshooting and privacy
AUTHENTICATION_FAILED: correct the referenced credential/permissions. Reload can resume the pool when a changed credential reference or runner group passes validation; if the secret changed at the same reference, useresume.IMAGE_INVALIDorRUNNER_VERSION_UNSUPPORTED: for Docker, pre-pull a correct digest; for Tart, prepare a macOS guest whose metadata reportsdarwin, then seal it and update the pool. Linux, missing, or unrecognized Tart guest OS metadata is rejected. An older unsupported sealed image remains listed for diagnosis and can be removed after references are cleared. Managed pools retry runner preparation automatically; inspect status or requestrunner update. Exact pins require explicit replacement. GitHub generally requires replacement within 30 days and may require security updates sooner.CAPACITY_EXHAUSTEDorDISK_LOW: adjust explicit budgets/free disk, or remove an unused sealed image yourself. Do not delete active execution storage.OWNERSHIP_AMBIGUOUS: preserve local state and investigate the exact resource. Use a different scale-set name when another installation owns it; restoring ownership requires the original matching backup.CLEANUP_PENDING: restore Docker/Tart/GitHub connectivity and let reconciliation retry. A stopped manager reports pending cleanup until the next run.SLEEP_INHIBITION_UNAVAILABLE: check OS utility/session permissions; work continues without a sleep guarantee.- Repeated preparation failures suspend the affected pool after three attempts. Unrelated healthy pools continue.
- A suspended pool should show its reason in
runmoor statusandrunmoor doctor. Older state may lack the original reason; Runmoor reports that gap without guessing or automatically resuming the pool. Rundoctor, correct any reported dependency failure, then userunmoor resume --pool NAME.
Diagnostics are local, sanitized structured metadata bounded by seven days and 256 MiB. Completed execution history expires after seven days; unresolved ownership/cleanup remains until reconciliation. Credentials, JIT configuration, workflow secrets and raw job output are excluded. User-created sealed images remain until explicit deletion; generated runner revisions follow managed retention. There is no telemetry or Prometheus endpoint.
Report reproducible issues through GitHub Issues. Include version, platform, safe error code and relevant sanitized status. Do not include credentials, JIT data, raw workflow logs, or private VM contents. The Runmoor documentation site covers the same supported workflows.
See installation and signature verification before any manual update or rollback.
macOS host operations
The unreleased host backend uses the same launchd and pool control operations. Run the manager and trusted jobs under a dedicated CI account. PATH and DEVELOPER_DIR are inherited when set; execution HOME and temporary paths are disposable. Other manager variables and management credentials are excluded from job launch configuration. Separate directories do not prevent account-wide file or Keychain access. Operators own tools, shared caches, signing, Simulators, GUI sessions, global tool settings and any daemon that escapes the managed group.
A manager-only restart reconciles verified surviving work. Completion, cancellation and timeout terminate only the verified execution group and children. Missing supervisors, changed process identities or directories and uncertain termination retain capacity, state and resources. Preserve those resources and inspect the safe status/doctor error; restore a matching paired state/data backup when available. Never repair markers or delete by name. Drain/stop before upgrades or rollback and use a compatible binary with its paired backup. Older binaries must not open newer state. Raw workflow output and runner logs are not retained in manager diagnostics.
Actual host execution, unsigned Xcode builds and live GitHub jobs have not been validated. Host documentation does not expand Docker/Tart or package acceptance.