Web proxy internals

Charles-style HTTP/HTTPS MITM proxy for every client that isn't an iOS BusymateHelper install — desktop browsers, Android devices, debug-build iOS apps, your own backends. Runs on the VPS at proxy.busymate.net as systemd unit busymate-proxy (Node 20 + tsx); a local dev instance can run on localhost:8888.

Captured entries land in the same public.entries table the iOS app writes to, decrypted through the same Supabase pipeline. From the dashboard's point of view a proxy-server-captured row looks identical to a tunnel-captured row.

Wire shape

TrafficLogEntry (defined in web/proxy-server/src/types.ts, mirrored in ios/Shared/TrafficLogEntry.swift and web/dashboard/types.ts):

interface TrafficLogEntry {
  timestamp: string;        // ISO-8601
  method: string;
  url: string;
  statusCode: number;
  duration: number;         // ms
  requestSize: number;
  responseSize: number;
  isSecure: boolean;
  host?: string | null;
  requestHeaders?: Record<string, string> | null;
  responseHeaders?: Record<string, string> | null;
  os?: string | null;       // normalized client OS for THIS request
  /* ...body bytes, error info, breakpoint linkage, etc. */
}

There is no wire-protocol envelope. The streamer writes rows directly into Postgres as INSERT INTO entries (request_id, device_uuid, ts, kind:"event", payload) VALUES …. request_id UNIQUE makes retries idempotent.

OS is captured at two levels — exactly like country. The normalized platform name (iOS / iPadOS / Android / macOS / Windows / Linux / ChromeOS) lives both on the device (devices.os, a stable property set when the device reports its metadata) and per request (payload.os, derived from that request's User-Agent at the same emit seam as deviceCountry). The request grain handles a device whose traffic spans more than one client/OS. MCP entry-detail tools surface payload.os: get_entry returns the raw payload, inspect_requests exposes a per-request os, and export_har adds an _os extension field.

Topology

The proxy-server speaks two things to Supabase: PostgREST INSERTs (pgStreamer.ts) and a Realtime client (pgRealtime.ts, pgSettings.ts, perDevicePresence.ts). It does NOT speak any custom WebSocket protocol.

Listeners

web/proxy-server/src/proxy.ts is the main HTTP listener. The same http.Server accepts connections from three sources:

httpsListener.ts uses a single static cert/key (the wildcard leaf), not per-SNI dynamic minting — the *.busymate.net cert matches every reachable subdomain.

File map

Four more modules in web/proxy-server/src/ carry the traffic-mutation subsystems that ride the same per-flow MITM session:

Boot sequence

web/proxy-server/src/index.ts's main():

1. Load BootConfig from ~/.busymate-proxy/config.json (or env). Carries deviceName, listenPort: 8888, httpsListenPort: 8443, baseDomain: "busymate.net", perDevicePortRange: {start:9000, end:9099}, certBundleAuth creds.
2. Open-relay guard — refuses to bind a non-loopback listenAddress unless certBundleAuth is configured. Without this, any port-forward / public IP would expose an open relay.
3. Load CA — CertificateAuthority reads ~/.busymate-proxy/{ca.pem, ca.key}.
4. Whitelist — new IpWhitelist({ttlMs: 24 h}); start the sweep cleaner.
5. PortMap — new PortMap(persistPath) loads/seeds {uuid: {port, deviceName}}.
6. startProxy — bind http.Server on listenPort; attach connect handler.
7. startHttpsListener — bind tls.Server on httpsListenPort; pipe accepted sockets back onto the main http.Server with __sniDeviceName set.
8. PortPool.bootstrap(existing entries) — call ensurePortListener(port) for every UUID already in the port map.
9. PostgresStreamer — start flush interval; seedIdentity(name, uuid) pre-populates the device-resolve cache with this proxy's own row.
10. PgRealtime — subscribe devices:all (presence-track this proxy with source: "proxy-server") + proxy-control (resend / breakpoint / restart).
11. PgSettings — fetch the proxy's own settings + subscribe to settings_global / settings_device changes for every UUID it's currently serving via the port pool.
12. PerDevicePresence — bootstrap one Realtime client per active port-pool UUID, tracking presence on devices:all with source: "pac".
13. Process signal handlers — drain pgStreamer and write a final device_status row with online: false before process.exit(0).

Capture path

handleConnect (in proxy.ts) handles every CONNECT host:port HTTP/1.1 from a client:

1. Check the IP/SNI/port-pool attribution chain (see Per-device attribution) to pin deviceUuid.
2. Look up the device's effectiveSettings from PerDeviceSettingsCache. If the host matches effectiveSettings.ssl_proxy_domains (or the global mitmAllHosts toggle is on), dispatch to startMitmSession (mitm.ts). Otherwise, raw tunnel via openUpstream.
3. startMitmSession mints (or fetches from cache) a leaf cert for the host via ca.leafFor(host), terminates TLS locally, spins an inner http.Server against the decrypted stream, and dials the real server out (possibly through the user's external CONNECT proxy). Each request that lands on the inner server:
   • Captures bodies (bodies.ts, 1 MB cap, gzip/br/deflate decompressed)
   • Evaluates block rules (blockRules.ts) — a matching block / drop / mock rule short-circuits the flow; a script rule runs inline through the script engine
   • Runs the onRequest script hooks (scriptEngine.ts / scriptRunner.ts) for any matching scripts — a hook may mutate the request or synthesize a response
   • Matches the URL against effectiveSettings.breakpoint_patterns; on match, awaits BreakpointResolution (60 s timeout) from proxy-control Broadcast
   • Forwards the (possibly edited) request upstream
   • Same onResponse script-hook + breakpoint cycle for the response
   • Emits a fully-populated TrafficLogEntry to PostgresStreamer.enqueue (carrying scriptRan / scriptError / scriptTrace when a script touched the flow)

Direct HTTP GET / POST (non-CONNECT) lands in handlePlainHttp. Management URLs are short-circuited before the IP whitelist check; everything else forwards via per-host upstream dial.

Ingest (pgStreamer.ts)

PostgresStreamer.enqueue(entry, deviceName, uuid?):

• FIFO buffer cap 5000 entries, batch size 100, flush interval 1 s.
• Each flush() resolves deviceName → uuid via resolveOrCreateDeviceUuid() (UUIDv5 derivation, cached). The proxy seeds its own identity at boot so renames don't double-create.
• INSERT shape: (request_id: randomUUID(), device_uuid, ts: entry.timestamp, kind: "event", payload: entry).
• FK violation (23503) — at least one row references a device_uuid that doesn't exist in public.devices. Falls back to row-by-row insert, drops the offender, evicts its name from the resolver cache, logs.
• Transient errors → re-queue at the front of the FIFO.

Control plane (pgRealtime.ts)

Two Realtime channels:

• devices:all — presence-track this proxy with {device_uuid, source: "proxy-server"} on every SUBSCRIBED transition. onBroadcast UPDATE re-routes to onDeviceRenamed for portMap propagation (when the dashboard renames a device, the port-map's deviceName field syncs).
• proxy-control (private) — three event handlers:
  • breakpoint-continue → breakpoints.resolve(requestId, {rawHttp, abort}) + markBreakpointResumed UPDATE on the row in breakpoint_events.
  • resend-request → executeServerSideResend() re-issues the captured request from the supplied envelope; the response is INSERTed as a tagged entry on the same device.
  • restart → process.exit(1) so systemd respawns.

Settings sync (pgSettings.ts + perDeviceSettings.ts)

PerDeviceSettingsCache is a UUID-keyed map of EffectiveSettings. Each entry is lazy-fetched via the effective_settings_for_device(target_uuid) RPC (server-side merge of settings_global + settings_device plus pre-expanded SSL-proxying domains from applied service groups). Subscriptions to settings_global and settings_device:all invalidate cached entries; the next getEffectiveByUuid round-trip refetches.

There is no periodic poll — the cache is event-driven through Realtime.

Management endpoints

All gated by HTTP Basic auth against certBundleAuth.{username, password} from boot config:

Deployment

Single VPS, single systemd unit:

# /etc/systemd/system/busymate-proxy.service
[Service]
WorkingDirectory=/home/ubuntu/busymate-devtools/web/proxy-server
ExecStart=/usr/bin/npm start
Restart=on-failure
StandardOutput=append:/var/log/busymate-proxy.log
 
# /etc/systemd/system/busymate-proxy.service.d/env.conf
[Service]
Environment=SUPABASE_URL=https://xfjplaganjqowkcnznbr.supabase.co
Environment=SUPABASE_SERVICE_ROLE_KEY=<service-role-jwt>

nginx (/etc/nginx/sites-enabled/proxy.busymate.net.conf) terminates TLS and reverse-proxies to 127.0.0.1:8888 for the management paths.

Deploy command:

ssh ubuntu@busymate.net 'cd ~/busymate-devtools && git pull && cd web/proxy-server && npm install && sudo systemctl restart busymate-proxy'

State that survives a deploy lives under /home/ubuntu/.busymate-proxy/: config.json, ca.pem + ca.key, port-map.json, identity.json. Nothing else needs to be preserved — entries, device_status, settings, and breakpoint history all live in Supabase.

Scripting engine

The proxy-server hosts the canonical scripting engine — the reference implementation that cdp-connector and iOS mirror byte-for-byte. It runs the frozen onRequest / onResponse contract over all decrypted-HTTPS hosts (anything in the device's SSL-proxy domains, or under the global MITM-all toggle) and plain HTTP, with the fullest reach of any source (1 MB bodies, both phases).

• Sandbox. scriptEngine.ts runs each script inside an isolated-vm V8 isolate — a genuinely separate realm with no require / fetch / fs / timers. A script cannot read the proxy's service-role key (it isn't in the sandbox) or dial the network from author-visible code.
• Runner + caps. scriptRunner.ts enforces 50 ms per hook, 16 MB per isolate, a 1 MB body cap, and a 32-slot global concurrent-hook semaphore that sheds to fail-open under saturation — so a flood of scripted connections can never queue work on the event loop. Any throw / timeout / OOM / malformed return fails open to the original bytes, recorded as scriptError.
• Config. scriptConfig.ts folds settings_{global,device}.data.scripts (global ++ device, via effective_settings_for_device) into the effective script set, and owns the internal SSRF-guarded egressAllowlist for cross-host retargets (not surfaced to script authors).
• Inline action. A block rule with action: "script" runs its code through the same engine inside the block-rule match, ahead of standalone scripts.

The full author-facing contract, security model, and per-source coverage matrix live in Scripts.

Where to look next

• Per-device attribution — port pool, SNI, IP whitelist, pinned-name fallback.
• PAC presence — how perDevicePresence.ts keeps iOS devices Online while the main app is backgrounded.
• BusymateHelper iOS — the on-device capture path (same public.entries sink).
• Shipping pipeline — busymate-proxy deploy step + proxy-server row in the post-ship table.