Development & Deployment Guide¶

This document describes the setup of the development environment as well as build management and deployment for the Docker Instance Manager.

Development Environment¶

Two ways to run the stack from a checkout: inside Docker containers (compose.dev.yaml), which keeps the host free of a Node toolchain, or directly with Node.js.

Prerequisites¶

Docker and Docker Compose (or Docker Desktop) — for the containers; or Node.js 22+ and npm 10+ — for the local setup.
NPM_TOKEN: @stefgo/react-ui-components and @stefgo/js-template-engine come from the GitHub Package Registry, and .npmrc reads the token from this variable. A classic token with read:packages does; gh auth token works when the GitHub CLI is signed in. For the containers, put it into a .env file in the root directory (excluded from git):

NPM_TOKEN=<your-token>

Without Docker¶

git clone https://github.com/stefgo/docker-instance-manager
cd docker-instance-manager
NPM_TOKEN=$(gh auth token) npm install
npm run build -w shared      # the other workspaces import its output

npm run dev:server           # backend on :3000, restarts on change
npm run dev:frontend         # Vite on :5173 with hot reload, proxies /api and /ws to :3000
npm run dev:client           # agent, web UI on :3001

The backend serves the frontend from server/dist/public once it has been built (npm run build); for work on the frontend use the Vite server instead. LOG_LEVEL=debug and LOG_FORMAT=json work as in production.

With Docker¶

The development environment is configured via compose.dev.yaml:

docker compose -f compose.dev.yaml up --build

This starts two services:

Service	Port	Dockerfile	Description
`server-dev`	`3000`	`docker/Dockerfile.server.dev`	Backend in watch mode (`npm run dev -w server/backend`), restarting on code changes. It serves the frontend from `server/dist/public` once that has been built; for hot reload run `npm run dev:frontend` on the host, whose Vite server proxies `/api` and `/ws` to port 3000.
`client-dev`	`3001`	`docker/Dockerfile.client.dev`	Client agent in watch mode (`npm run dev -w client`). Depends on `server-dev`.

Volume mounts: - server/, shared/ → mounted into server-dev for live code editing. - client/, shared/ → mounted into client-dev. - node_modules is isolated as a Docker volume per service to prevent conflicts between host OS (macOS/Windows) and Linux container dependencies. - ../react-ui-components (a sibling checkout of @stefgo/react-ui-components) is mounted into both containers at /app/react-ui-components, and server-dev sets VITE_USE_LOCAL_UI=true to build against it. The mount is unconditional: without the checkout Docker mounts an empty directory, and a frontend build inside the container fails.

UI library outside the dev containers: a plain npm run build or npm run dev:frontend uses the installed library version. Building against a sibling checkout is opt-in with VITE_USE_LOCAL_UI=true, type-checked with npm run typecheck:local-ui -w server/frontend. See frontend.md.

Host filesystem access: - client-dev mounts the host root at /mnt to allow management operations during development.

Documentation Site¶

The pages in docs/ are served twice: GitHub renders them as plain Markdown, and MkDocs Material publishes them to https://stefgo.github.io/docker-instance-manager/. A page has to work in both. Three things follow from that:

Links out of docs/ have to be absolute. A relative ../compose.yaml resolves on GitHub and nowhere else. Use the full https://github.com/stefgo/docker-instance-manager/blob/main/... URL instead.
The hand-written table of contents in api.md uses GitHub's anchors — the emoji is dropped and the leading space becomes a dash, hence #-authentication. mkdocs.yml configures pymdownx.slugs.slugify(case=lower) precisely so that MkDocs produces the same ids. Do not swap the slugify function without checking those links.
A new page needs an entry in nav in mkdocs.yml. index.md is the landing page of the site and not a copy of the README.

To preview locally:

python3 -m venv .venv-docs && source .venv-docs/bin/activate
pip install -r requirements-docs.txt
mkdocs serve          # http://localhost:8000, live reload

requirements-docs.txt pins the version, so the preview and the published site render identically.

.github/workflows/docs.yml runs mkdocs build --strict on every branch and pull request that touches the documentation — a dead internal link, a nav entry without a file or a page outside the nav fails it. Only main deploys the result to GitHub Pages (Settings → Pages → Source: GitHub Actions). The workflow is deliberately separate from ci.yml and build.yml: a typo in a page must not be able to block a release, and a documentation-only commit builds no image.

Screenshots¶

The images in docs/assets/screenshots/ are generated, not captured by hand:

npm run screenshots              # rebuild the frontend, then capture
npm run screenshots -- --no-build
npx playwright install chromium  # once, before the first run

No backend, database, agent or Docker daemon is involved. Playwright serves the built bundle from a local static server and answers every /api/** call and the /ws/dashboard socket from scripts/screenshots/fixtures.mjs. The socket matters here: it is how the server pushes each host's Docker state and which hosts are online, so a capture against a real database would document a dead system.

Two consecutive runs produce byte-identical PNGs: the clock is frozen, the timezone is pinned to UTC, and the version in the header comes from the root package.json instead of from git. Without those three, every run would rewrite every file.

Nothing type-checks the fixtures — they are serialised straight to JSON — so an API shape change has to be followed there by hand. An endpoint with no fixture logs ! unmocked GET /api/v1/…, and a clean run prints no warnings. scripts/screenshots/README.md has the details.

index.md carries the light/dark pairs. Each is a <figure> holding two images, their src ending in Material's #only-light and #only-dark markers. Material hides the wrong one with [data-md-color-scheme=slate] img[src$="#only-light"] and its counterpart, so the pair follows the palette toggle of the site. A <picture> with a prefers-color-scheme source would be wrong here: a media query sees only the operating system setting, so a reader who switched the site to dark on a light desktop would get light screenshots on a dark page. GitHub ignores the fragment and shows both images of a pair stacked; the published site is the primary artifact, so it wins.

Raw HTML is safe only on index.md. MkDocs rewrites paths in Markdown links but not in HTML attributes, and every other page is published a directory deep (install/index.html), where assets resolve as ../assets/… while GitHub still wants assets/…. Every other page embeds a single dark image with ordinary Markdown syntax; shots used only there carry themes: ["dark"] in capture.mjs. The agent's pages are single images for a different reason: client/src/web/public/styles.css defines one dark palette and no light one.

Build Management¶

Production images use multi-stage Docker builds:

Component	Dockerfile
Server	`docker/Dockerfile.server`
Client	`docker/Dockerfile.client`

Build stages: 1. builder: Installs all dependencies, builds all TypeScript workspaces (shared, client, server/frontend, server/backend). 2. runner: Copies only compiled output and production dependencies into a slim base image (node:22-bookworm-slim, for the server and the agent alike). The builder removes the dev dependencies with npm prune --omit=dev, which works on the installed tree — no second registry round trip and no rebuild of better-sqlite3.

.dockerignore keeps node_modules, build output, databases, config.yaml, .env, .git and the docs out of the build context; everything an image needs is copied explicitly.

Version Injection¶

The version string is derived in the same order everywhere — by scripts/generate-version.sh for the agent's dist/VERSION file, and by getVersion() in server/frontend/vite.config.ts for the dashboard:

APP_VERSION / VITE_APP_VERSION. CI passes the released version (1.2.0, without the v of the tag) or <branch>-<short-sha> for a branch build.
The version in the root package.json, which semantic-release maintains. On a commit that carries a release tag it is used as it is; otherwise the commit is appended (1.2.0+abc1234[-dirty]), so a build between releases never looks like the release.
Fallback without a readable manifest: {branch}-{short-hash}[-dirty].

The server ships no VERSION file; nothing on the backend reads one.

Multi-Architecture Support¶

Both the server and client images are built for multiple platforms:

Component	Supported Platforms
Server	`linux/amd64`, `linux/arm64`
Client	`linux/amd64`, `linux/arm64`

Each architecture is built on a native GitHub runner (ubuntu-latest and ubuntu-24.04-arm), without QEMU, and pushed by digest only. Once the smoke test has started those digests, a publish job assembles one manifest list per image and attaches the tags, so docker pull picks the right variant on either platform.

Continuous Integration¶

Type checking, linting and the tests of the pure logic are the quality gates; nothing renders a component or starts the backend. Every job declares its own permissions, and every workflow has a concurrency group; a tag build is never cancelled.

Workflow	Trigger	What it does
Check Code (`ci.yml`)	Push to any branch except `main`, every pull request, and `workflow_call`	Job `verify`: checks that the registry cleanup names every image `build.yml` publishes, then `npm ci`, `npm run build` (type-checks `shared`, `client` and `server/backend`, builds the frontend), `npm test` (Vitest over every workspace; reads `shared` from source, so it does not depend on the build before it), `npm run typecheck` for `shared`, `client` and `server/backend` (their build leaves the tests out of `dist`, and Vitest does not check types), `npm run typecheck -w server/frontend` (the Vite build does not type-check), `npm run lint -w server/frontend`, `npm run lint` (the root ESLint config, covering `shared`, `client` and `server/backend`).
Build Images (`build.yml`)	Push to `main` or `dev` (except documentation-only commits), `v..*` tags, manual dispatch — which is how a release reaches it	See the job graph below.
Create Release (`release.yml`)	Manual, `main` (release) or `dev` (beta)	See Release.
Prune Registry (`cleanup-packages.yml`)	Nightly, manual	See Registry Cleanup.
Merge Dependency Updates (`dependabot-auto-merge.yml`)	Pull requests by Dependabot	See Action Updates.
Publish Docs (`docs.yml`)	Push and pull request touching `docs/`, `mkdocs.yml` or `requirements-docs.txt`, manual	See Documentation Site.

build.yml runs these jobs:

verify ──► prepare ──► build (server, client × amd64, arm64) ──► smoke (amd64, arm64) ──► publish
(ci.yml)                 native runners, pushed by digest,          starts each digest        manifest lists
                         no tag yet                                                           and every tag

verify is ci.yml, reused rather than restated. Nothing is built before it is green.
prepare is the single source of the version string and of the two image names.
build runs four native jobs that push by digest, with a GHA layer cache per image and architecture. Each hands its digest on as a workflow artefact.
smoke starts the digests of its architecture, see below.
publish assembles one manifest list per image from both digests and attaches main or dev and sha-<short>, or — for a release — the version tags and latest. It is the first job that makes anything pullable.

A build that fails the smoke test leaves its digests in the registry untagged; the nightly cleanup removes them. paths-ignore (docs/**, mkdocs.yml, requirements-docs.txt, .github/workflows/docs.yml, **.md) applies to branch pushes only — a tag or a manual run always builds.

npm ci authenticates against GitHub Packages for @stefgo/react-ui-components and @stefgo/js-template-engine with the workflow's GITHUB_TOKEN (packages: read). That works because the packages are public; if one ever becomes private, the step needs a personal access token with read:packages instead. The image builds pass the same GITHUB_TOKEN as the npm_token BuildKit secret. The former NPM_TOKEN repository secret, a personal access token with an expiry date, is no longer read.

Smoke test. The smoke job is the only place where the images are executed: everything before it proves that the code compiles, not that the result starts. It is a gate, not a report — nothing is tagged until it has passed, so latest cannot move to an image that never started. It runs on both architectures, each on its native runner, and addresses the images as <image>@sha256:… from the artefacts of this run: there is no tag yet, and a digest leaves nothing for Docker to choose. It starts the server and the agent (with the runner's Docker socket, since the agent refuses to start without the Docker API) and waits up to 60 s each for {"status":"ok"} from /api/health — the agent's through docker exec, since its route answers loopback only. A second step checks that the same request from the runner does not answer. On failure it prints the container logs.

To reproduce the gate locally, run the same commands without VITE_USE_LOCAL_UI set:

npm run build
npm test
npm run typecheck -w shared
npm run typecheck -w client
npm run typecheck -w server/backend
npm run typecheck -w server/frontend
npm run lint -w server/frontend
npm run lint

ESLint is configured twice, once per kind of code: eslint.config.mjs at the root lints shared, client and server/backend as Node TypeScript and ignores server/frontend, which has its own config with the React plugins. Two configs matching the same file would be two truths about it; a new Node workspace, on the other hand, is covered by the root config without another file.

Tests¶

npm test             # once, as CI runs it
npm run test:watch   # re-runs what a change touches

Vitest, configured in vitest.config.mts at the root, with one project per workspace: shared, frontend, client and backend.

A test lives next to its module — projectQuery.ts and projectQuery.test.ts.
Logic only. Every project runs in the node environment. There is no DOM, so logic that sits inside a hook or a component is first moved into a module of its own and tested there. The same holds for a class with side effects in client or server/backend: what it decides goes into a pure module, and the test is written against that.
The tests read shared from source. @dim/shared exports src under the development condition, which every other project sets; without it they would read shared/dist, which needs a build first and is stale the moment shared/src changes.
shared, client and server/backend build without their tests (tsconfig.build.json), so they end up neither in dist nor in an image. Vitest strips types without checking them, so npm run typecheck -w <workspace> is what checks those files; the frontend's are covered by its own typecheck.
A backend test never opens the installation's files. core/Database.ts opens server/data/server.db and config/AppConfig.ts reads -- and may write -- config.yaml as soon as they are imported. A test of a module that imports either replaces it with vi.mock. server/backend/src/testing/memoryDatabase.ts returns an in-memory database that the real migrations brought to the current schema; the directory is left out of the build.
A special case that so far only a comment guarded gets a test named after it — a host that is offline, an image without a known platform, the bare image id a container reports after a pull.

Registry Cleanup¶

cleanup-packages.yml prunes GHCR every night at 02:00. It uses dataaxiom/ghcr-cleanup-action rather than the more obvious actions/delete-package-versions, and the reason is worth keeping: a multi-arch build pushes its per-architecture images and its attestations untagged — only the manifest list carries the tag.

dim-server:main  ─┬─► sha256:6612…  linux/amd64      ┐
                  ├─► sha256:3695…  linux/arm64      │ each one an untagged
                  ├─► sha256:c2c3…  attestation      │ version of the package
                  └─► sha256:77e4…  attestation      ┘

An action that deletes "untagged versions" therefore hollows out the tagged images from underneath. That is not hypothetical: it is how dim-server:latest, dim-client:latest and the 0.0.3–0.0.5 release tags came to be tags whose children all return 404 — docker pull fails on them. The previous ignore-versions regex could not prevent it, because it is matched against the version name, which for a container package is the digest.

The cleanup in use knows which children belong to a kept tag, and delete-partial-images removes the manifests that already lost theirs. latest, main, dev and anything shaped like a version are excluded from every rule — a deleted 1.2.0 breaks whoever pinned it, so release images accumulate. The exclusion also keeps the already broken tags above in place; latest becomes pullable again with the next release.

validate: true re-checks the result, but only as a warning. A step of the workflow's own therefore resolves every child digest of latest, main and dev individually and fails on a missing one — a hollowed-out image still lists its platforms in the index, only fetching the child shows that it is gone. A failing scheduled run is what GitHub sends a notification about. Until the next release replaces the broken latest, that step is expected to fail.

A manual run defaults to a dry run:

gh workflow run cleanup-packages.yml               # logs only
gh workflow run cleanup-packages.yml -f dry_run=false

The images are listed by name in the workflow; a new image has to be added there by hand. ci.yml fails when build.yml publishes an image that list does not name. Discovering the packages by wildcard would need a classic personal access token with delete:packages — an unattended delete right over every container of the account, and a credential that expires.

Action Updates¶

.github/dependabot.yml watches the GitHub Actions, and only those: one grouped pull request a month, prefixed ci:, aimed at dev. npm is left out on purpose — four workspaces produce a stream of version bumps that nobody can assess without a test suite. Dependabot security updates for npm arrive regardless, they need no configuration file.

This repository is maintained without pull requests, so a monthly one would simply wait. dependabot-auto-merge.yml puts a Dependabot pull request into auto-merge, and GitHub merges it once its checks are green; a red run leaves it open. It targets dev because an action bump mostly concerns actions only build.yml exercises, and that runs after a merge — on dev, a bad bump breaks the developer image, and its smoke test says so.

It needs "Allow auto-merge" enabled under Settings ▸ General ▸ Pull Requests. Without it the workflow fails loudly instead of merging unchecked.

SECURITY.md describes how to report a vulnerability privately.

Commit Messages¶

Commits follow Conventional Commits and are written in English. The commit message is the only input the version number comes from, so it is checked like code: .githooks/commit-msg runs commitlint against commitlint.config.mjs. ci.yml lints the commits of a pull request as well, but this repository is maintained without pull requests, so the hook is the check that actually runs. npm install activates the hooks through the root prepare script:

git config core.hooksPath .githooks   # runs automatically via `npm install`

Type	Effect on the version
`feat`	minor — 1.2.0 → 1.3.0
`fix`, `perf`, `revert`	patch — 1.2.0 → 1.2.1
`build`, `chore`, `ci`, `docs`, `refactor`, `style`, `test`	none

A breaking change is declared with a BREAKING CHANGE: footer. It raises the minor position (breaking: minor in release.yml) and still gets its own section in the changelog. A major version comes only from the release workflow's bump: major.
The feat!: spelling is rejected: the Angular preset semantic-release reads commits with has no ! in its header pattern, so such a commit would be read as typeless and release nothing.
A body line that begins with a single word and a colon (happened: …) is parsed as the start of the footer. Rephrase it.
Release commits (chore(release): x.y.z) are exempt from commitlint: their body is the release notes.
[skip release] anywhere in a message removes that commit from the version calculation.
.githooks/pre-push allows pushing main and dev only; topic branches stay local.
core.hooksPath makes git ignore .git/hooks. A hook of your own belongs in .githooks.

Release¶

semantic-release owns the version number; nobody tags by hand. A release is started from Actions ▸ Create Release ▸ Run workflow — on main for a release, on dev for a beta (1.3.0-beta.1, published as a prerelease). release.yml offers the inputs and names the checks and the build; what a release is and how its notes are put together is the same for all stefgo projects and lives in release-workflows.

Actions ▸ Create Release ▸ Run workflow   (main or dev)
  └─► preflight ─► ci.yml ─► semantic-release
        ├─ commits CHANGELOG.md + package.json   [skip ci]
        ├─ pushes tag v1.2.0, creates the GitHub release
        ├─ fast-forwards dev to the release commit   (from main only)
        └─ gh workflow run build.yml --ref v1.2.0
              └─► build.yml → images 1.2.0, 1.2, latest
                    └─ gh run watch --exit-status   (the release job waits)

dry_run (default on): the next version and the complete notes appear in the run's summary, nothing is written.
bump (auto | patch | minor | major): auto reads the commit types. Any other value is the step that is taken whatever the commits say — lower than they call for, or from a state that holds only docs: commits, which is what the dry run is there to catch. major is the only way a major version is created.
Every release is described by hand in .release/next.md: what is new, and what an upgrade needs. The text is placed above the generated list of commits. A release without it is refused by the preflight job, before the checks run. A beta keeps the text; the release from main empties the file again and drops the entries of its betas from CHANGELOG.md. .release/footer.md is appended to every release page.
dev is merged into main with its history — no squash, no rebase — and main is merged back before the next beta. semantic-release finds a version through the tags a branch contains, so the workflow refuses a release whose beta is not part of main, and a beta while the last release is not part of dev. After a release it fast-forwards dev itself when dev has nothing main lacks.
A beta builds the images as 1.3.0-beta.1 and moves neither 1.3 nor latest.
A run that was asked for and produces no release fails instead of going green without a result.
The tag is pushed over GITHUB_TOKEN, and GitHub starts no workflow for such a push. The release workflow therefore dispatches build.yml on the tag ref and follows it. If that build fails, the last step says that the version exists without images and that re-running Build Images on the tag is the fix — not a second release.
The root package.json carries the released version. It started at 0.0.5, the last tag from before semantic-release; the workspace manifests keep 1.0.0.

To build an image from another branch, dispatch the build manually. It is tagged with the branch name and the short SHA, never with latest:

gh workflow run build.yml --ref my-branch   # -> :my-branch, :sha-abc1234

Indentation and `git blame`¶

Four spaces everywhere, no formatter — match the surrounding file. The switch of the frontend and the JSON files from two to four spaces is a single commit listed in .git-blame-ignore-revs. GitHub skips it in its blame view by itself; locally, tell git once:

git config blame.ignoreRevsFile .git-blame-ignore-revs

A later commit that only reformats belongs in that file too, with its full hash.

Deployment¶

Running a build of your own¶

Operators use the published images — see the Quick Start. The repository's own compose.yaml builds both images from the checkout instead, which is what you want to try a change in production shape:

docker compose up -d --build

Service	Port	Volumes	Description
`dim-server`	`3000`	`server-data` (SQLite DB), `./server-config.yaml`	API + web dashboard.
`dim-client`	`3001`	`client-data`, `./client-config.yaml`	Client agent.

Create both config files (touch) before the first start, or Docker mounts directories in their place. Both services declare a healthcheck against GET /api/health (see Operations).

Which tag moves when:

Trigger	Tags	Moves `latest`
Push to `main`	`main`, `sha-<short>`	no
Push to `dev`	`dev`, `sha-<short>`	no
Release `v1.2.0`	`1.2.0`, `1.2`, `latest`	yes
Beta `v1.3.0-beta.1`	`1.3.0-beta.1`	no
Manual dispatch on a branch	`<branch>`, `sha-<short>`	no

npm Scripts Reference¶

All scripts are defined in the root package.json and target individual workspaces via -w.

Script	Description
`dev:server`	Start backend in watch/dev mode.
`dev:frontend`	Start frontend Vite dev server with HMR.
`dev:client`	Start client agent in watch/dev mode.
`start:server`	Start backend in production mode.
`start:client`	Start client agent in production mode.
`build`	Build `shared` first, then `client`, `server/backend` and `server/frontend`.
`clean`	Remove compiled output from `shared`, `client`, and `server`.
`screenshots`	Regenerate the documentation screenshots (see Screenshots).

There is no start:frontend. The frontend builds into server/dist/public and is served by the backend, so start:server covers it. To serve a production bundle on its own, use npm run preview -w server/frontend.

build lists its workspaces explicitly rather than using --workspaces, which would build shared twice and rely on the order of the workspaces array. A new workspace has to be added to that list.