Shipping pipeline

How a code change becomes a live build that the user gets a banner / sheet about, end to end.

All nine components ship independently. There is no atomic "release"; each subproject bumps its own components.<X>.build on its own cadence. The convergence point is version.json at the repo root — a manifest read by /api/version (dashboard) and ios:build (Realtime) and consumed by every client that cares. (The component list is version.json-driven — when a component is added there, add a row here too.)

The components

version.json currently defines nine components:

version.json's top-level build field is the max(components.*.build). It exists for the Telegram notifier headline + legacy single-counter consumers; nothing on the hot path reads it.

Two-layer build identity (post-build-124)

There are two different "build" numbers in version.json. They mean different things and they're allowed to disagree.

1. Top-level build — monorepo deploy counter. The max of all component builds.
2. components.<X>.build — per-component "what is actually deployed for X". Authoritative for /api/version consumers (iOS BuildUpdateChecker, dashboard BuildUpdateWatcher).

The split exists because of two prior incidents:

• Build 114→115 (TF auto-increment) — local archive said 114, ASC processed as 1.0 - 115 (TF had reserved 114 for an earlier failed upload). We bumped the monorepo to 115 to reconcile.
• Build 117 (web-only ship) — top-level was bumped to 117 for a dashboard fix, but iOS hadn't shipped. The iOS app's update checker read the top-level value and prompted users to install build 117 from TestFlight — which didn't exist.

The new field surgically fixes #2: components.ios.build only moves when fastlane beta succeeds and ASC reports a build number.

The "inc before deploy" rule

Every ship MUST start with a build bump. Concretely:

Mismatches between these locations are bugs. The notifier diffs against notes/.last-ship-state.json and flags every component on every fire.

Parallel sessions — ship.mjs + the pre-push gate

When several Claude Code sessions run in parallel (one per component), conflicts only ever arise in the shared files — version.json, README.md, notes/next-ship.md, notes/.last-ship-state.json. Component subtrees and deploy targets are disjoint and never collide. Three tools keep everyone in sync:

• scripts/sync-version.mjs — version.json is the one source of truth for every component's name/label/host/description/version/build. Edit it, run this, and the package.json mirrors (+ the iOS pbxproj display name) match. --check for CI.
• scripts/gen-readme.mjs — regenerates the README banner + status table from version.json (between <!-- VERSION:banner --> / <!-- VERSION:table --> markers). README is no longer hand-edited, so it can't collide. --check for CI.
• .githooks/pre-push — blocks any push that would drift (sync-version --check + gen-readme --check + sync-audit). Activate per clone with scripts/setup-hooks.sh (git config core.hooksPath .githooks). Bypass only with git push --no-verify.

The one command — scripts/ship.mjs <component> serializes a release behind a repo-local lock (.git/ship.lock) so two sessions can't race the shared files:

node scripts/ship.mjs dashboard            # bump → sync → gen-readme → audit → commit → rebase → push → deploy → notify
node scripts/ship.mjs docs --no-deploy     # everything except the deploy
node scripts/ship.mjs proxy-server -m "fix: ..."

It bumps only that component's build, propagates, regenerates the README, hard- gates on sync-audit, stages only that component's subtree + the shared files (guarding against png/env), commits, git pull --rebasees onto latest main (retrying the push once if the remote moved), deploys that component, and fires notify-telegram --component=<x> (so the iOS APNs push stays deferred). iOS is not a ship.mjs target — its build is ASC-authoritative; ship it with fastlane beta (which now runs sync-version.mjs pre-archive to source the display name from version.json).

Working rules for parallel sessions: stay on main, one session owns one component subtree, write notes/next-ship.md before shipping, and let ship.mjs do the bump/push/deploy — never hand-edit version.json top build or the README banner/table.

Recovery from a runaway session / OOM. Long Claude Code sessions grow the terminal's memory; when it balloons you have to quit and restart. scripts/gen-warp-config.mjs writes a Warp Launch Configuration (~/.warp/launch_configurations/busymate-devtools.yaml) with one tab per component (sourced from version.json components.<x>.path) — each tab opens in the component's directory and runs claude --continue, resuming that directory's most recent session. After a restart: quit Warp → ⌘P → "Open a Launch Configuration" → busymate-devtools, and every component session comes back in place. Re-run the generator whenever a component is added/renamed/moved (--no-resume to cd-only, --cmd "…" for a custom per-tab command). It doesn't fix the memory growth — it makes recovery one launch.

Prune oversized sessions with scripts/sessions.mjs — lists every session for the repo's project(s) by size (with its customTitle name + an ⏱active flag for anything touched in the last 15 min), and safely archives the big ones (--prune --over 50M --yes → moves the .jsonl + subagent dir to ~/.claude/.session-archive/, never the active session, never a hard delete). A monorepo-root session can reach 300M+; archiving those is the direct fix for the terminal OOM. Going per-component (one small session per dir, via the launch config) keeps them small in the first place.

Path A — iOS ship (the hard path)

Step-by-step

1. Bump pbxproj. Six CURRENT_PROJECT_VERSION lines need updating in lockstep — main app + tunnel ext + LA ext × Debug + Release configs. sed -i '' 's/CURRENT_PROJECT_VERSION = X;/CURRENT_PROJECT_VERSION = X+1;/g' ios/BusymateHelper.xcodeproj/project.pbxproj. Verify with grep -c.

2. Update changelog notes. Two files are read by the ship hooks:

• notes/next-ship-tf.md — iOS-only "What to test" changelog. Surfaced in TestFlight's tester email. Keep it user-facing — no implementation prose, no cross-component scope.
• notes/next-ship.md — cross-component summary read by the Telegram notifier when the channel falls back from the per-row diff to a freeform body. Optional but recommended.

3. Run fastlane beta. From ios/. Requires ~/.config/appstoreconnect/{env, key.p8} (ES256-signed API JWT).

Output you're looking for: Successfully finished processing the build 1.0 - N for IOS. That N is what ASC accepted. If your local pbxproj said N-1, ASC auto-incremented — reconcile both locations to N.

4. Update version.json. Set components.ios.build = N (ASC reality). Recompute top-level build = max(components.*.build).

5. Redeploy the dashboard. Even on iOS-only ships. The dashboard reads version.json from its on-disk deployed copy and exposes it at /api/version. Without this redeploy, the iOS update checker (and the Realtime broadcast) still see the old build, so the prompt won't fire on devices.

cd web/dashboard && npm run deploy

6. Fire the notifier. node scripts/notify-telegram.mjs ship does four things:

• Diffs version.json.components against notes/.last-ship-state.json.
• If nothing changed → silent skip (idempotent — safe to call after every deploy regardless of what shipped).
• Renders the Telegram HTML table and posts.
• If the iOS row diffed, calls notify-ios-build over HTTPS — broadcasts on Realtime AND fans APNs alert pushes.
• Saves the new state file (sha + sentAt + components snapshot).

7. The fan-out. notify-ios-build (supabase/functions/notify-ios-build/index.ts) runs in Deno on the Supabase Edge Runtime:

• Validates Authorization: Bearer <NOTIFY_IOS_BUILD_SECRET>.
• POSTs to ${SUPABASE_URL}/realtime/v1/api/broadcast signed with the service role key — payload lands on Realtime channel ios:build as event build-available.
• Queries device_push_tokens for every active row with kind='device', signs an APNs ES256 JWT, fans apns-push-type: alert requests to api.push.apple.com (production tokens) or api.sandbox.push.apple.com (debug tokens) with apns-collapse-id: ios-build-available so a re-fire doesn't queue duplicate banners.
• Returns { broadcast: {ok, status}, apns: {sent, total, results: [...] } }.

8. iOS receives through one of three paths:

• Foreground: BuildUpdateChecker.startRealtime() subscribed to ios:build at app init. The onBroadcast(event: "build-available") handler decodes payload.build, calls applyServerBuild(), flips showSheet if N > localBuild. Sheet pops within ~100 ms of the broadcast landing.
• Background: System receives the APNs alert and renders the banner — no app code runs (unless the alert had content-available: 1, which ours doesn't). User sees "Build N is on TestFlight. Tap to install." Tap → returns to the app where the next .task cycle calls checkNow() and the sheet pops.
• Terminated: Same APNs path. The system delivers the banner without the app process; tap launches the app cold, BuildUpdateChecker.init() runs, .task modifier fires checkNow(), and the inline banner renders on first frame once the server build lands.

The convergence point in all three paths is BuildUpdateChecker.applyServerBuild(N) setting serverIOSBuild = N, from which isOutdated = N > localBuild derives. When outdated, the home screen shows an inline update banner above the readiness card (SetupGuideView.updateBanner) — there is no modal sheet. Tapping Update opens https://ios.busymate.net, which 302-redirects to the current /v1/app/<APP_ID>?build=N deep-link.

Path B — Web component ship (the easy path)

Dashboard / docs / proxy-server / supabase / cdp-connector all follow the same shape (cdp-connector has no service of its own — it rides a dashboard deploy, which rebuilds and serves its installer + tarball):

The notifier hook is iOS-specific. Web-only ships fire Telegram but never the broadcast / push path — that's correct: nobody's iOS app needs to update because the iOS build didn't move.

Per-component deploy commands

# Dashboard
cd web/dashboard && npm run deploy
# → ssh ubuntu@busymate.net 'cd ~/busymate-devtools && git pull && cd web/dashboard && npm install && npm run build && sudo systemctl restart busymate-dashboard'
 
# Docs
cd web/docs && npm run deploy
# → same shape; systemd unit kept as `busymate-md-server` for backward compat
 
# Proxy server
ssh ubuntu@busymate.net 'cd ~/busymate-devtools && git pull && cd web/proxy-server && npm install && sudo systemctl restart busymate-proxy'
 
# Supabase
supabase db push                                                           # migrations
supabase functions deploy <fn> --project-ref xfjplaganjqowkcnznbr           # edge functions
# (or via the Supabase MCP `deploy_edge_function` tool)
 
# cdp-connector (bmc) — no systemd unit of its own
cd web/dashboard && npm run deploy
# the dashboard build rebuilds /install.sh + /bmc.tar.gz from cli/cdp-connector/,
# so deploying the dashboard publishes the new connector. Users then self-update:
#   bmc upgrade        (or re-run the install one-liner from dash.busymate.net/install.sh)
 
# Status board (systemd busymate-status)
cd web/status && npm run deploy
 
# Marketing site (systemd busymate-marketing, apex busymate.net)
cd web/marketing && npm run deploy
 
# BusyBro — code lives under supabase/functions/; busybro/ is the home + build mirror
cd busybro && npm run deploy
# → deploys the busybro-bot, ask, and busybro-threads Edge functions

The post-ship table (three surfaces)

After every ship, the same component table renders in three places — by design:

1. Telegram — emitted by scripts/notify-telegram.mjs. Monospaced cells inside <code> for column alignment, hostname per row as the clickable <a href>.
2. README.md top — first thing in the file. Bumped in the same commit that bumps version.json.
3. Chat reply — when the developer reports a successful ship, the same table closes the message.

Same row order everywhere, fixed in notify-telegram.mjs's COMPONENT_ORDER so the table reads identically each fire — by label: web, api, ios, cdp, bro, dash, docs, proxy, status (manifest keys marketing, supabase, ios, cdp-connector, busybro, dashboard, docs, proxy-server, status). Any future component falls to the end in natural order. ● for components that bumped this fire; ○ for unchanged. Build cell shows (N) or (prev → curr).

The state file

notes/.last-ship-state.json holds the previous-ship snapshot:

{
  "sha": "<HEAD SHA at time of last fire>",
  "sentAt": "<ISO timestamp>",
  "components": { ... full components map ... }
}

The notifier reads this on every ship invocation, computes diffComponents(manifest, prevState), and exits silently if nothing differs. After a successful fire, it writes the new state.

This file is committed to git — checked in alongside the version bump so the next ship's diff is deterministic regardless of which machine fires the notifier.

Endpoints under the hood

Recovery / edge cases

ASC auto-incremented past your pbxproj

Happens when a prior failed upload reserved a build number. The fix:

1. Set components.ios.build = N (what ASC reported).
2. sed -i '' 's/CURRENT_PROJECT_VERSION = <pbx>;/CURRENT_PROJECT_VERSION = N;/g' ios/BusymateHelper.xcodeproj/project.pbxproj so the next archive is N + 1.
3. Redeploy dashboard.
4. Fire the notifier — it'll show (prev → N) correctly.

Notifier fires twice for the same ship

Idempotent by design. The state file holds prev_build = N. The second ship call diffs N against N → nothing changed → silent skip.

To force a re-fire (e.g., testing): node scripts/notify-telegram.mjs ship --force=1. Telegram + push both go out again; the APNs collapse-id ensures the banner doesn't duplicate on a user's lock screen.

Realtime is down

The APNs path is independent — devices still get banners. When Realtime comes back, the next bump's broadcast goes out normally. There's no "replay" of missed broadcasts; FG devices that missed the broadcast still see the sheet via the .active polling on the next foreground tick.

A device is force-quit and has notifications disabled

The user never gets a banner. They'll see the sheet on the next app launch via .task { await buildUpdate.checkNow() }. This is intentional — BuildUpdateChecker is the floor, not the ceiling.

The dashboard never deploys after a fastlane succeeds

/api/version continues to advertise the old components.ios.build. The iOS app's polling sees server == local and stays silent. The TF tester emails fire normally, but no in-app banner. Always redeploy the dashboard after an iOS ship.

What lives where