2026-04-30

End-of-session handoff. The "External Proxy" feature now actually works for domains in the SSL Proxying list — previously every MITM-via-proxy flow deadlocked silently and apps like Dasher hung forever. Verified end-to-end on the iPad against a real upstream proxy (107.166.124.52:29842) with *.doordash.com in the SSL Proxying list. DoorDash app navigates and submits requests; in-app Network tab shows decrypted POST iguazu.doordash.com/iguazu-edge/v1 → 200, POST otel-mobile.doordash.com/v1/traces → 200, and the rest of the traffic.

Commits landed (local, not pushed)

  • 57606d2 — External proxy: actually deliver MITM-via-proxy traffic (the four real bugs)
  • 649cf6d — PacketTunnel: dev-side diagnostics plumbing

What was actually broken

The previous commit b005e92 ("External proxy: cover MITM upstream via NWProtocolFramer") looked like it added MITM-via-proxy support but the framer never actually ran. Four bugs in the upstream-leg path conspired:

1. NWParameters composition — applicationProtocols.insert(framerOptions, at: 0) was a no-op

swift
let params = NWParameters(tls: tlsOptions)
params.defaultProtocolStack.applicationProtocols.insert(framerOptions, at: 0) // ← ignored

On iOS 17/18, post-init mutation of applicationProtocols does not register the framer in the protocol stack used by NWConnection. The framer's init(framer:) was never called, so the connection sat in .preparing forever — neither .ready nor .failed fired, and the calling site (MITMSession.startRelayIfBothReady) waited indefinitely for serverReady = true.

Fix: build the parameter stack from scratch and assign the array in one go.

swift
let params = NWParameters()
params.defaultProtocolStack.transportProtocol = NWProtocolTCP.Options()
params.defaultProtocolStack.applicationProtocols = [tlsOptions, framerOptions]

2. Array order was reversed

Apple's docs: applicationProtocols[0] is "the topmost (most external) layer." For wire flow TCP → CONNECTframer → TLS → user:

  • [0] = TLS (user-facing top)
  • [1] = framer (just above TCP)

Initially I tried [framer, TLS] (intuitively "framer wraps TCP, TLS wraps framer"). Framer init still didn't fire. Reversing to [TLS, framer] made it work. The convention is counter-intuitive — "topmost" in the array means topmost in the user's view, not topmost in the transmission sense.

3. HTTPConnectFramer never sent CONNECT bytes

swift
func start(framer: NWProtocolFramer.Instance) -> NWProtocolFramer.StartResult {
    return .willMarkReady          // ← waits forever
}
func wakeup(framer: ...) {
    framer.writeOutput(...)        // ← never called
}

wakeup(framer:) only fires after an explicit framer.scheduleWakeup(milliseconds:) — which we never called. Apple DTS engineer Quinn's STARTTLS sample writes the handshake bytes from start(framer:) directly:

swift
func start(framer: NWProtocolFramer.Instance) -> NWProtocolFramer.StartResult {
    sendConnectRequest(framer: framer)   // writeOutput here, framework queues
    return .willMarkReady
}

The framework queues writeOutput bytes before the lower TCP transport is .ready and flushes them once it connects.

4. Static-config race produced empty :0 CONNECTs

HTTPConnectFramer.pendingConfig was a single static slot:

swift
HTTPConnectFramer.pendingConfig = (host, port, auth)
let conn = NWConnection(using: params)
conn.start(queue: queue)       // init(framer:) runs *async*, on framework's queue

When the mitm serial queue dispatched two MITMSessions back-to-back, the second SET clobbered the first before either framer's async init(framer:) read it. Result: CONNECTs going out as CONNECT :0 HTTP/1.1 which the proxy rejects.

Fix: thread-safe FIFO queue. Push once per NWConnection(using:), pop once per init(framer:). The framework instantiates framers in start() order so FIFO matches push order.

Bonus orthogonal fix — RST on proxy-fail

TCPForwarder.fallbackToDirectConnection's onFailed was deleting the flow without sending RST. But the synthetic SYN-ACK had already gone out at SYN time (deferForProxyHostname), so the client app sat on a half-open socket for ~2 minutes (TCP retransmit timeout) before giving up. Now we send TCP.RST | TCP.ACK on proxy CONNECT failure so the app fails fast.

Dev workflow established this session

These are what made the bugs above actually findable. Codified in memory under feedback memories feedback_self_diagnose_logs.md and feedback_auto_monitor_logs.md.

Diagnostic pipeline

TrafficLog.diagnostic(_:) now does both:

  1. Appends to Library/Caches/diagnostics.log in the App Group container.
  2. Emits via os_log (Logger.notice) with a stable BMHDIAG: prefix.

Two retrieval paths:

bash
# Pull the on-disk file (preferred — full session history):
xcrun devicectl device copy from --device <CoreDeviceID> \
  --domain-type appGroupDataContainer \
  --domain-identifier group.com.busymatehelper \
  --source Library/Caches/diagnostics.log \
  --destination ~/Desktop/bmh_diag.log
bash
# Stream live (for active debugging):
idevicesyslog -u <libimobiledevice-UDID> | grep --line-buffered "BMHDIAG:"

Two gotchas baked into the workflow

  • devicectl rejects root-level files in appGroupDataContainer with a spurious "File paths cannot contain '..'" error. The fix was moving diagnostics.log from container root into Library/Caches/. Standard subdirs work; root does not.
  • idevicesyslog needs the libimobiledevice UDID (e.g. 00008132-001968C21EF9001C), not the CoreDevice ID that xcrun devicectl uses (e.g. B193DBAA-3334-5381-B514-5E18BFC11682). They are different identifiers for the same device. Test devices reference memory has both.
  • iOS 17+ does not surface user-app Logger.info / Logger.debug to syslog. Only .notice and .error persist and reach idevicesyslog. Use .notice for diagnostic streaming.
  • Toggling the VPN inside the BusymateHelper app does NOT reload the rebuilt extension binary. iOS only re-spawns the provider when the user toggles via iOS Settings → General → VPN, DNS & Device Management → VPN → BusymateHelper → Status. Always tell the user to do that step after devicectl install.

UDP/443 (QUIC) drop counter

PacketTunnelProvider now counts UDP/443 packets the tunnel doesn't handle and emits a periodic top-5 summary every 5 s:

UDP/443 dropped: 240 total, top: 104.18.20.56=34 8.8.8.8=14 8.8.4.4=14 ... — likely QUIC, would need UDP forwarder

This was originally added to test the hypothesis that Dasher uses QUIC and was being silently dropped. Confirmed: Dasher uses TCP/443, not QUIC, so this turned out to not be the cause. Counter kept in place anyway — silent drops are expensive to bisect after the fact, and this answers "is it QUIC?" instantly next time something hangs.

Status: working as expected

  • External Proxy + MITM end-to-end through *.doordash.com (Dasher app loads, navigates, submits requests, all decrypted in Network tab).
  • External Proxy + non-MITM passthrough (e.g. https://example.com).
  • Multiple parallel MITM sessions for the same hostname (HTTP/1.1 connection pooling — Safari/whatismyip.com opens 2 connections, each gets its own listener port and own framer instance via the FIFO config queue).
  • BMHDIAG: os_log stream is live and grep-friendly.
  • Library/Caches/diagnostics.log is pullable via devicectl.

Known remaining issues / open questions

  • QUIC traffic still silently dropped. Apps that prefer QUIC (YouTube, Cloudflare-fronted sites) will currently hang on UDP/443 and only fall back to TCP after a Happy Eyeballs timeout. Next step if this becomes a real problem: implement a UDP forwarder, or send ICMP "destination port unreachable" to force the app's QUIC stack to fall back faster. Not blocking Dasher.
  • HTTPConnectProxy.swift comment is now stale. It says "MITM upstream currently bypasses the external proxy — that's a future improvement." This is no longer true (commit 57606d2 made MITM upstream go through the framer). Worth a one-line cleanup if the file is touched again.
  • Architecture doc may need an update. docs/architecture/busymate-helper.md likely still describes the framer as a recent addition; now it actually works for the MITM upstream leg, not just nominally. Worth re-reading and tightening.
  • .claude/ and ios/build/ are untracked but not in .gitignore. Per-user state that shouldn't be committed; small follow-up.
  • Push to origin not done. Both commits are local on main. User has to authorize pushing; next session, ask before pushing.

Configuration verified this session

  • Test device: Serebano's iPad (iPad Pro 11" M4)
    • CoreDevice ID (for xcrun devicectl): B193DBAA-3334-5381-B514-5E18BFC11682
    • libimobiledevice UDID (for idevicesyslog): 00008132-001968C21EF9001C
  • App Group container UUID on this device: C0C929C0-F716-43EA-BE47-2141417D1EE3 (path: /private/var/mobile/Containers/Shared/AppGroup/<UUID>/)
  • External Proxy: 107.166.124.52:29842 with auth (user-managed, not committed)
  • MITM domains active during testing: *.doordash.com, whatismyip.com, plus the existing busymate.io / busydrivers.com / portal.busydrivers.app from the prior session.

Where to start next session

If the user comes back to this:

  1. Probably want to push the two commits — ask first.
  2. If working on something else but anything proxy-adjacent breaks, the first thing to do is pull Library/Caches/diagnostics.log via the devicectl invocation above and grep BMHDIAG: HTTPConnect[framer]: to see the live state.
  3. If they ask "is QUIC still dropped?" the answer is yes — adding a UDP forwarder is the natural next chunk of work.
2026-05-012026-04-29