Initial import of streamline

This commit is contained in:
2026-11-16 17:26:44 -08:00
commit 8e4aec1855
1221 changed files with 274256 additions and 0 deletions
+291
View File
@@ -0,0 +1,291 @@
---
name: streamline-deploy
description: "把 Streamline 部署到生产:拉代码、构建前端、重启控制面、验证生效,重建卫星 bundle,并自动推到 3 台卫星节点(忙节点走空档部署,不打断任务)。Use when the user asks 部署 / 上线 / deploy / 发布一下 / 把改动发到生产 / restart streamline / 重启服务, or after merging a change that needs to go live. Also use when the satellite bundle / 卫星热部署面板 shows an older SHA than HEAD (--satellite-only)."
---
# 部署 Streamline
一条命令,从 devbox 一路到卫星节点:
```bash
bash .claude/skills/streamline-deploy/deploy.sh # 控制面 + 重建 bundle + 推 3 台卫星
bash .claude/skills/streamline-deploy/deploy.sh --check # 只做预检,不写任何东西
bash .claude/skills/streamline-deploy/deploy.sh --satellite-only # 只重建 bundle 并推送,不碰控制面
bash .claude/skills/streamline-deploy/deploy.sh --skip-satellite # 只发控制面,bundle 和节点都不动
bash .claude/skills/streamline-deploy/deploy.sh --no-push # 重建 bundle 但不推节点(留给人点)
bash .claude/skills/streamline-deploy/deploy.sh --force-push # 立即换装,会中断节点上正在跑的任务
```
默认那条走完**三个产物**:控制面(pull → install → build → restart → verify)、
`satellite/dist/satellite.js`(卫星 bundle)、以及把 bundle 推到各执行节点。
推送对**忙节点默认用「空档部署」**——不打断任何在跑的任务,见下面「卫星」。
## 先确认,再执行
**部署是生产写操作。** 按 CLAUDE.md 的规矩,除非用户这一轮明确要求部署/上线,否则不要自己跑——"顺手帮你发一下"不算授权。用户说了就直接做,不用二次确认。
不确定要不要部署时,先跑 `--check`:它只读,会把 HEAD、分支、工作区是否干净都报出来。
## 目标环境
| | |
|---|---|
| host | `fdbd:dc01:ff:31a:6973:d9b4:988a:23c0`(IPv6,公司内网) |
| 仓库 | `/data00/home/jiaopengwen/streamline` |
| 服务 | `streamline-server.service`(`User=jiaopengwen`) |
| 端口 | `:5173` — API + WS + **前端静态资源**,同一个进程 |
| 对外 | 前端 `https://streamline.bytedance.net`:Caddy `file_server` **直接读 `control-web/dist`**,不经过 :5173;API `https://streamline-internal-api.bytedance.net`:Caddy → :5173(只放 `/api`、`/ws`)。配置在 devbox `/etc/caddy/Caddyfile`,前端的缓存头 / `/assets` 404 规则要和 `control-server/src/index.ts` 保持一致 |
覆盖用环境变量:`STREAMLINE_DEPLOY_HOST` / `_USER` / `_OWNER` / `_REPO` / `_UNIT` / `_BUN_BIN` / `_BRANCH` / `_PRINCIPAL`(默认 `<_USER>@BYTEDANCE.COM`)/ `_ENV_FILE`(默认仓库根 `.env`)。
## 七个反直觉的点(都是踩过的)
手写这套命令基本一定会错在这几处,脚本已经处理,但改脚本前先读:
1. **路径不是 `~/…`。** 是 `/data00/home/jiaopengwen/streamline`。写 `~/home/jiaopengwen/streamline` 会展开成 `/home/<你>/home/jiaopengwen/streamline`,不存在。
2. **必须以 `jiaopengwen` 身份跑,不能用自己的账号。** 仓库属于 `jiaopengwen`,你是别人 → git 直接拒绝:`fatal: detected dubious ownership`。
**不要**用 `git config --global --add safe.directory` 绕过——那样 `git pull` 和 `bun run build` 会以你的身份写文件,`dist/` 和 `.git/` 里混进你拥有的文件,之后以 `jiaopengwen` 运行的服务就管不动它们了。正确做法是 `sudo -n -u jiaopengwen`(该机已配免密 sudo)。
3. **`bun` 不在 `jiaopengwen` 的非交互 PATH 里。** 只有 `npm`/`git` 能找到。bun 在 `/home/jiaopengwen/.bun/bin/bun`,必须显式加进 PATH(systemd unit 里也是这么硬写的)。`bun run build` 内部会 shell out 到 `bun`,所以 `npm run build` 也救不了。
4. **没有 `streamline-web` 这个服务,也不需要。** 系统里只有 `streamline-server` 一个 unit。前端**不是**独立进程:`control-server/src/index.ts:45` 解析 `WEB_DIST` 并由 control-server 自己把 `control-web/dist` 静态托管在 :5173(但对外前端域名由 Caddy 直接读 dist,不走这里,见上表;`:8080` 上没有任何监听,README 里"Vite 常驻"那段对这台机器已经过时)。
→ `bun run build` + 重启 `streamline-server` 就是完整部署。`systemctl restart streamline-web` 会直接报 unit 不存在。
5. **SSH 要 `-6` + 裸 IPv6,不能加方括号。** `ssh jingyuan.yi@[fdbd:…]` 会把整串当 hostname 解析失败;`ssh -6 jingyuan.yi@fdbd:…` 才对。
6. **devbox 只认 Kerberos,ticket 每天过期。** sshd 只开 `gssapi-keyex` / `gssapi-with-mic`,不收公钥;
ticket 约 10 小时,隔夜必失效,症状是 `Permission denied (gssapi-…)`,长得像公钥 / 网络问题。
预检第一步因此先 `klist -s`:没有有效 ticket 就从本机仓库根 `.env` 读 `KINIT_PASSWORD`,
经 stdin 喂给 `kinit --password-file=STDIN`(不进 argv、不落临时文件、不回显;只用 sed 取这一行,
不 `source` 整个 `.env`)。没配这个键就停下并提示手动 `kinit`;kinit 失败多半是 SSO 密码轮换了、
`.env` 里的旧了。它只写本机凭据缓存,所以 `--check` 也会做。
**在 git worktree 里部署**(如 `~/orca/workspaces/streamline/*`):`.env` 被 gitignore、不跟着 worktree 走,
脚本找不到仓库根 `.env` 时会自动退到主检出(worktree 共用的 `.git` 所在仓库,如
`/Users/bytedance/Documents/streamline/.env`)读。要用别的文件就显式设 `STREAMLINE_DEPLOY_ENV_FILE`。
`git push` 到 code.byted.org 不受影响——「代码推得上去、部署连不上」正是 ticket 过期的组合。
7. **卫星 bundle 是另一个产物,`bun run build` 不产它。** 面板顶部那个 SHA 读的是
`satellite/dist/VERSION`,由 `bun run build:satellite` 写;控制面部署完全不碰它。
历史上这导致:控制面已经在新 commit 上,面板还提供几小时前的 bundle,而已经推过的
节点显示「最新」—— 一个**相对旧 bundle** 的最新。现在脚本每次部署都重建它,`--check`
也会把 `bundle SHA vs HEAD` 的漂移报出来。
## 顺序为什么是 pull → install → schema check → build → restart
### install:锁文件变了不等于生产依赖已经变了
`git pull` 只更新 `package.json` 和 `bun.lock`,不会刷新已经存在的 `node_modules`。
部署脚本因此在每次 pull 后运行 `bun install --frozen-lockfile`;锁文件未变化时 Bun 会快速
走完无操作路径,新增运行时依赖时则会在重启前补齐。安装失败会立即停止部署,此时旧进程
仍在运行,避免出现代码已经更新、服务也重启了,但首次执行新路径才报 module not found。
### schema check:谁部署,谁替所有人的未落库迁移买单
这个仓库**没有迁移执行器**——`deploy.sh` 从不碰 schema,迁移靠人手工跑
`scripts/apply-migration.ts`。于是有一个不写下来就一定会再犯的坑:你这次改动不需要
迁移,但 `git pull` 会把别人**已经合进 main、迁移却还没跑**的提交一起带上生产,
重启之后相关查询全线 500。
2026-08-22 就这么炸过:一次知识库改动的部署顺带把 `tasks.method_coverage` 的代码拉了
上去,那一列从没建过,`/api/tasks` 全线 500,面板打不开约四分钟。验证步骤拦住了,
但那时服务已经重启完——**拦得太晚**。
所以 `bun scripts/check-migrations.ts` 卡在 **pull 之后、restart 之前**:此刻新代码在
盘上、老进程还在跑,缺列只是一个可以从容处理的告警。它查表、索引、**以及列**
(真正炸掉的那次就是 `ADD COLUMN`,而原来的复核只看表和索引)。
红了怎么办:按它打印的两行提示,先预演再 `--commit` 跑那份迁移,然后重跑部署。
脚本**不会**替你执行 DDL——那是对生产的写操作,按 CLAUDE.md 归人决定。
故意不落库的对象(例如 `spec_delta_events`,设计已被 `run_conditions` 取代、全仓库无
引用)写在 `check-migrations.ts` 的 `ACKNOWLEDGED` 里,每条都要写明理由:一个天天变红
的检查等于没有检查,人会学会忽略它,然后连真的那次也一起忽略。
### build 为什么在 restart 之前
`vite build` 会先清空 `dist/`。所以构建失败时前端是**空的**。脚本把 build 放在 restart 之前并在失败时立刻退出——宁可服务还跑着旧代码(前端已经没了但 API 正常),也不要把服务重启到一棵半更新的树上。构建失败时按提示修好再重跑,别先重启。
预检还会在**工作区有未提交的 tracked 改动**时拒绝执行:那说明有人直接在生产上改过代码,`git pull` 要么失败要么把他的改动冲掉,得让人先处理。
## 验证(脚本自动做)
- 重启后**轮询到接口真的应答**才继续(`is-active` + `curl :5173/api/tasks`,最多 30 秒),
否则打 journal 最后 30 行。等的是「能服务」而不是「进程起来了」:unit 已经 active、
但还没绑上 :5173 的那个窗口里,紧接着的验证会撞上连接被拒。
这里以前是无条件 `sleep 6` —— 快的时候白等,慢的时候(机器忙 / DB 连接慢)不够,
于是把一次正常启动误判成失败,而假阴性会让人去动一个其实好好的服务。
- `curl :5173/` 里的 asset hash **必须**等于刚构建出来的那两个文件名——不等就说明在服务旧 index.html。
- `GET /api/tasks` → 200。
- 对外域名的 hash 与内部一致(不一致会警告,通常是 Caddy 缓存)。
### 为什么脚本里有一坨 ControlMaster 选项
到 devbox 的一次 SSH **冷握手实测 3.8 秒**,复用后 0.6 秒;而一次完整部署要发 ~30 次
远程命令。没有连接复用时,脚本 80% 的墙钟时间花在握手上而不是干活上——`--check`
(11 次调用)实测 44 秒,加上复用之后 6–10 秒。
所以 `CM_OPTS` 不是可有可无的调优,删掉它等于把部署时间乘以四。`ControlPath` 用 `%C`
(连接元组的哈希)而不是 `%h`:host 是 39 字符的 IPv6 字面量,macOS 的 unix socket
路径上限约 104 字节,把地址拼进路径正是它在别人机器上失效的方式。
一个好用的额外信号:**构建是可复现的**。本地 `bun run build` 和 devbox 上的 asset hash 应该一模一样。不一样就说明两边源码/依赖不同,值得先查清楚。
## 卫星:脚本构建 + 自动推送(忙节点走空档部署)
`shared/` 和 `satellite/` 会被**打进卫星的自包含构建**,卫星是**另外的机器**(`fei-android-01`、`fei-ios-01`、`fei-ios-02`;这台 devbox 上没有卫星,`:5180` 无监听)。
机制是控制面把 bundle 的字节 POST 给节点 `/sat/admin/deploy` → 节点校验 sha256 → 原子替换 →
进程退出 → `scripts/run-satellite.sh` 守护脚本用新代码重启 → 节点带新 VERSION 重新注册,
面板上变成「最新」。不需要 SSH 到卫星,也不需要在节点上放 git 凭据。
### bundle 装了什么,没装什么
热部署只替换 `satellite/dist/satellite.js`(和它旁边的 `VERSION`)。**进 bundle 的**除了
`shared/` + `satellite/`,还有三个 wrapper——`bin/record-evidence.ts`、`bin/conditions.ts`、
`bin/claude-pty.ts`——`satellite/src/entry.ts` 用子命令分发把它们折了进来,正是为了消灭
下面这类漂移。
**没进 bundle 的**只剩两个:`bin/claude-cursor.ts`(生产不跑 cursor 引擎)和
`scripts/run-satellite.sh`(守护循环本身,没法把自己装进自己)。
脚本对节点侧那一半是**减法推导**的(扫 `bin/` 再滤掉 `entry.ts` 折进 bundle 的那几个,
见 `deploy.sh` 的 `BUNDLED_BIN`),不是写死清单——这样往 `bin/` 里新加一个 wrapper 会
**默认触发告警**,而不是被静默漏掉;漏掉正是这一节存在的原因。
#### 为什么这一节存在(`claude-pty` 的教训)
`bin/claude-pty.ts` 是 pty 引擎,生产上**每一次执行**都从它拉起。它曾经只存在于
`<repo>/bin/`,由 `claude_launch.ts` 从**节点自己的检出**里解析——而热部署一次都不碰
那份检出。失败是静默的:节点上报的 `version` 只来自 `dist/VERSION`,对 `bin/` 的状态
一个字都不报,于是面板显示「最新」,而 agent 引擎可能落后好几周。
旧判据(只看 `shared/ satellite/`)会以两种方式漏掉它,**后一种更常见也更误导**:
- **区间里只有 `bin/` 改动** → `SAT_DIFF` 为空 → 打印「No satellite-side changes」。
可构造:`git diff --stat 313a802^..b55ea2a -- shared/ satellite/` 为空,而那两个提交
改的正是 `bin/claude-pty.ts`(104 行)。
- **区间里混着 `shared/`/`satellite/` 改动** → `SAT_DIFF` 非空 → 打印
「Satellite code CHANGED — **that is what the push above was for**」。这是**肯定式的错误保证**:
推送确实发生了,但它装不下 `bin/`。实测部署 `c863538..c80bbea`:`SAT_DIFF` = 105 个文件
(于是打印上面那句),而同区间里 `bin/claude-pty.ts` + `scripts/run-satellite.sh`
一共改了 131 行,**一行都没随 bundle 走**。
实测后果:2026-08-21 探过三台节点最近一次执行的会话日志,终态 `result` 帧里都没有
`reason` 字段(那是 `343f322` 加的)——三台的检出当时都落后于它。
所以 `NODE_FILES` 是**无条件**计算的——它必须在这两种情况下都触发。
`bin/claude-pty.ts` 已于 2026-08-21 折进 bundle(`CLAUDE_PTY_SUBCOMMAND`)。**这意味着
历史上积累的 pty 引擎漂移不需要人工补**:任何节点只要收到一次这之后构建的 bundle,
引擎就跟着一起换了。
#### 剩下那两个还得人工
⚠️ **告警是边沿触发的**:它报的是「**这一次**部署的区间里动了这些文件」。
忽略一次之后,后续区间不再碰这些文件的部署就什么都不打印,漂移的节点会一直漂着、
而且看起来是干净的。**安静的部署不等于节点已同步。**
要把它变成可持续观测的状态量,得让节点上报自己的检出版本(尚未做)。
同步节点检出仍是人工动作,**具体怎么做取决于这台节点是怎么装起来的**:
- 有 git 历史的检出:在各卫星进程自己的目录里 fast-forward 到 `${STREAMLINE_DEPLOY_BRANCH:-main}`
(一台机器可能跑多个卫星,检出也是多份)。`satellite/dist/` 和 `.env` 都在 `.gitignore` 里,
所以不会和热部署产物冲突。
- 用 `scripts/executor-snapshot.sh` 恢复出来的节点:它默认带 `--exclude=.git`(第 121 行),
**没有 `.git`,fast-forward 和 `git rev-parse` 都跑不了**,只能把文件拷进去。
不变量只有一个:**每个节点的这些文件必须处在本次部署的 SHA 上**。
`scripts/run-satellite.sh` 是守护循环本身,**必须杀掉重启守护进程**才生效;
`bin/claude-cursor.ts` 只在 `AGENT_MODE=cursor` 下才被读到,生产不跑它。
**推送以前是面板上的人工点击,现在脚本自己做**(`scripts/push-satellites.ts`)。
改变前提的是「空档部署」:以前唯一的推送方式会立刻让进程退出、打断在跑的任务,所以必须
有人看着哪台空闲;空档部署让节点先把 bundle 存下来、停止接新活、跑完当前任务再在空档里
换装——**没有任务会被打断,也就没有需要人来做的判断了**。所以:
| 节点状态 | 脚本的动作 |
|---|---|
| 空闲 | 立即换装重启 |
| 忙 | **空档部署**:存盘 + 停止接单,当前任务结束后自己换装重启 |
| 已是该版本 / 已排队该版本 | 跳过(版本比对,脚本可重复跑) |
| 离线 | 跳过并明确报出来(回来后**不会**自动补装,需重跑) |
| 忙且卫星太老不认 `x-bundle-mode` | 报错并说明选项,**绝不**擅自强制 |
只有 `--force-push` 会走 强制部署(立刻中断任务),它是显式 opt-in。
**推送范围默认只有 `#default` 池**——也就是 `fei-android-01` / `fei-ios-01` / `fei-ios-02`
这三台。fleet 里还挂着别的团队的池子(`#im-sg` 的 `messaging-sg`、`messaging-sg-ios`,
`#pns` 的 `pns-android-01`),它们按自己的节奏跑、落后好几十个 commit,被一次日常部署
顺手拉到 HEAD 是个比这次发布本身大得多的变更,没人要求过。真要全推:
`bun scripts/push-satellites.ts --all-clusters`(或 `--cluster <tag>`)。
脚本每次都会把「谁不在范围内」打出来,免得局部发布被读成全量。
脚本为什么走 HTTP 而不是调 `/api/deploy`:那条路由是 SSO + admin 门禁(浏览器凭据),
脚本没有;为了让脚本进去而放宽门禁,比多写这个文件糟得多。节点的 `pendingDeploy` 与
`supportsQueuedDeploy` 又只活在控制面**进程内存**里(心跳学来的,见 `cluster.ts`),所以
它从只读的 `GET /api/nodes` 读状态,再用 cluster secret 直接把字节推给节点。
**两个陷阱脚本已经处理**:
1. 控制面刚重启的那十几秒里,它还没从心跳重新学到各节点的 `supportsQueuedDeploy`,
此时推送会把每台忙节点都判成「不支持空档部署」。所以脚本会先等到每个节点都有一次
**重启之后**的心跳(默认最多 45 秒)再动手。
2. **bundle 的 VERSION 是 git SHA,不是内容哈希。** 只改了控制面/前端/脚本的部署,
产出的 bundle 与上一版**逐字节相同**、只是版本号变了——照直推,三台节点就会为了
一个字符串停止接单、排空队列、重启一轮,每次部署都来一遍。所以 `deploy.sh` 会比对
新旧 bundle 的 sha256,相同时给推送脚本传 `--equivalent-version <旧SHA>`,跑着那个
版本的节点直接跳过。**只跳过「字节相同」的那一版**:落后更多版本的节点照推不误。
单独用:
```bash
bun scripts/push-satellites.ts --dry-run # 打印计划,不动任何东西
bun scripts/push-satellites.ts --node fei-ios-01.shiro0.icu
```
(必须在控制面那台机器上跑:要 `.env` 里的 `CLUSTER_SECRET`、要能连到节点。)
面板 **管理 → 卫星热部署** 仍在,用于手动补推、取消排队、看状态。
### 空档部署是怎么实现的
(脚本默认就用它,这里是它凭什么可靠。)
空档部署的实现在 `satellite/src/pending_deploy.ts`:bundle 落盘成 `satellite.js.pending` +
`pending-deploy.json`(**持久化**,中途因别的原因重启也不会丢,新进程会在下一个空档补上),
两个 claim 循环(`evaluator.ts` / `runner.ts`)看到有排队就**不再领新活**——这个「排空闸门」是
它能收敛的关键,否则队列一满,节点上一个任务刚结束下一个就claim走了,永远没有空档。
换装前会**重新校验一次 sha256**(字节在盘上躺了一段时间,截断的 bundle 会让节点重启后起不来)。
排队中的节点在面板上显示「已排队」(不是「待更新」——它不需要你再做什么),可以「取消排队」
让它保持当前版本并立刻恢复接单。
**注意一个过渡期陷阱**:旧卫星不认 `x-bundle-mode` 头,收到就会立刻换装——正好是空档部署要避免的
后果。所以卫星在心跳里用 `queuedDeploy: true` 声明能力,控制面对**没声明**的节点拒绝排队,并提示
「等它空闲,或强制部署」。这意味着本功能自身上线时,还在跑旧 bundle 的节点只能用老办法推一次。
面板上「最新 / 待更新」比的是**节点 VERSION vs bundle VERSION**,不是 vs 控制面 HEAD。
所以 bundle 不重建时,所有节点都可能显示「最新」而其实都在跑旧代码——这就是第 6 点。
**本次部署改没改卫星侧**:脚本会算 `git diff --stat <bundle 的 SHA>..HEAD -- <进 bundle 的路径>`
并在结尾报出「Bundled satellite code CHANGED」。**注意它只涵盖进 bundle 的那部分**,
`bin/` 里没进 bundle 的文件另有一条 `NODE-CHECKOUT files changed` 告警,见上方
「但 bundle 不是全部」。现在这只是**信息**,不再是待办——推送已经做过了。典型场景是改了 prompt
(`shared/src/readable_prompt.ts`、`satellite/src/runner.ts`、`feature_runner.ts`)。
用 `--no-push` / 节点离线 / 节点排队中时的行为:控制面会正常发新字段,**老卫星忽略未知字段**,
退化成改动前的行为——不会崩,但新能力不生效。**排队中的节点也属于这一档**:它要等当前任务
结束才换装,在那之前仍跑旧代码。汇报"已上线"时这点要说清楚。
回滚:上一份 bundle 会留成 `satellite/dist/satellite.js.bak-<sha>`(保留最近 3 份)。
需要回退时把它拷回 `satellite.js`(连带 `VERSION.bak-<sha>` → `VERSION`)再重推
(`bun scripts/push-satellites.ts` 或面板)。
要 SSH 上卫星排查时注意 `fei-ios-01` 的用户名是 `fei`,不是 `bytedance`。
## 部署完不等于生效
改 prompt / 报告生成逻辑时,**已有数据不会被追溯重写**——历史任务仍带着旧的报告 HTML,要用
`scripts/backfill_readable_reports.ts --batch --force` 回填,而**回填是对线上记录的写操作,必须先征得用户同意**,且它依赖卫星已经发布到位。
+489
View File
@@ -0,0 +1,489 @@
#!/usr/bin/env bash
# Deploy Streamline end to end: control plane (pull → install → build → restart → verify),
# then rebuild the satellite bundle, then push it to the executor nodes. One
# command takes a merged commit all the way to running code on every machine.
#
# Safe by construction:
# - every step runs as the repo owner (jiaopengwen), never as the SSH user
# - preflight refuses to proceed on a dirty tree or a wrong-looking host
# - dependencies are reconciled from bun.lock after every pull, so a newly
# added runtime package cannot be missing from an otherwise healthy deploy
# - build happens BEFORE the restart, so a broken build can't leave the
# service restarted onto a half-updated tree
# - the schema check runs between the pull and the restart: an unapplied
# migration (anyone's, not just this deploy's) stops the deploy while the old
# code is still serving, instead of 500ing after the restart
# - the satellite bundle is rebuilt AFTER the control plane is verified live,
# so a bundle failure can never hold the control plane hostage
# - the push to satellites uses 「空档部署」 for any BUSY node: the node parks
# the bundle, stops taking new work, and swaps in the gap after its current
# task. No running task is ever interrupted, which is what made this step
# safe to automate at all (it used to be a human click). 强制部署 — the one
# that does kill tasks — stays opt-in behind --force-push
# - the push covers the `default` node pool only (fei-android-01 / fei-ios-01 /
# fei-ios-02). Other teams' pools (#im-sg, #pns) run their own cadence and
# are not swept along by a routine deploy — see scripts/push-satellites.ts
# - --check does preflight only and writes nothing
# - --satellite-only rebuilds+pushes the bundle (no pull/restart); --skip-satellite
# deploys the control plane and leaves the bundle and nodes alone;
# --no-push builds the bundle but leaves the nodes for a human
#
# See SKILL.md for why each of these is not the obvious command.
set -uo pipefail
HOST="${STREAMLINE_DEPLOY_HOST:-fdbd:dc01:ff:31a:6973:d9b4:988a:23c0}"
# The SSH account is the CORP username, which on a Mac is usually NOT $(id -un)
# (that's often "bytedance"). git user.name is the corp id by convention here, so
# prefer it and fall back to the local user; override with STREAMLINE_DEPLOY_USER.
SSH_USER="${STREAMLINE_DEPLOY_USER:-$(git config user.name 2>/dev/null || true)}"
SSH_USER="${SSH_USER:-$(id -un)}"
# The deploy tree is owned by this account and the systemd unit runs as it.
OWNER="${STREAMLINE_DEPLOY_OWNER:-jiaopengwen}"
REPO="${STREAMLINE_DEPLOY_REPO:-/data00/home/jiaopengwen/streamline}"
UNIT="${STREAMLINE_DEPLOY_UNIT:-streamline-server}"
# bun is NOT on the owner's non-interactive PATH; the systemd unit hardcodes the
# same directory in its Environment=PATH.
BUN_BIN="${STREAMLINE_DEPLOY_BUN_BIN:-/home/jiaopengwen/.bun/bin}"
BRANCH="${STREAMLINE_DEPLOY_BRANCH:-main}"
CHECK_ONLY=0
SKIP_SATELLITE=0
SAT_ONLY=0
PUSH_SATELLITES=1
FORCE_PUSH=0
for arg in "$@"; do
case "$arg" in
--check|--dry-run) CHECK_ONLY=1 ;;
--skip-satellite) SKIP_SATELLITE=1 ;;
--satellite-only) SAT_ONLY=1 ;;
--no-push) PUSH_SATELLITES=0 ;;
--force-push) FORCE_PUSH=1 ;;
-h|--help)
# Print the header comment block, stopping at the first non-comment line.
awk 'NR==1{next} /^#/{sub(/^# ?/,""); print; next} {exit}' "$0"
exit 0 ;;
*) echo "unknown argument: $arg" >&2; exit 2 ;;
esac
done
if [ "$SAT_ONLY" = 1 ] && [ "$SKIP_SATELLITE" = 1 ]; then
echo "--satellite-only and --skip-satellite are contradictory" >&2; exit 2
fi
if [ "$FORCE_PUSH" = 1 ] && [ "$PUSH_SATELLITES" = 0 ]; then
echo "--force-push and --no-push are contradictory" >&2; exit 2
fi
# Connection multiplexing. Measured on this link: a cold handshake to the devbox
# costs ~3.8s, a reused one ~0.6s — and a full deploy makes ~30 remote calls, so
# without this the script spends ~115s doing nothing but handshakes (a --check,
# which is 11 calls, took 44s). The first call opens a master; the rest ride it.
#
# ControlPath uses %C (a hash of the connection tuple) rather than %h: the host is
# a 39-character IPv6 literal and macOS caps unix socket paths at ~104 bytes, so
# spelling the address into the path is how this breaks on someone else's machine.
# ControlPersist keeps the master briefly after the last call — the common
# "--check, then deploy for real" pair then reuses one connection.
CM_OPTS=(-o ControlMaster=auto -o ControlPath=/tmp/streamline-deploy-%C -o ControlPersist=60)
# -6 with the RAW IPv6 address. Bracketing it ("user@[fdbd:...]") makes ssh treat
# the whole thing as a hostname and fail to resolve.
ssh_raw() { ssh -6 -o ConnectTimeout=15 -o BatchMode=yes "${CM_OPTS[@]}" "${SSH_USER}@${HOST}" "$@"; }
# Run a command as the repo owner with bun on PATH.
# `set -o pipefail` must be set in the REMOTE shell: callers pipe through `tail`
# to trim output, and without it the remote pipeline returns tail's status (0),
# so a failed `git pull` / `bun run build` would sail past the caller's `|| fail`
# and we'd restart onto a half-updated tree or an empty dist/.
as_owner() {
ssh_raw "sudo -n -u ${OWNER} bash -lc 'set -o pipefail; export PATH=${BUN_BIN}:\$PATH; cd ${REPO} || exit 1; $1'"
}
say() { printf '\n\033[1m▶ %s\033[0m\n' "$1"; }
fail() { printf '\033[31m✗ %s\033[0m\n' "$1" >&2; exit 1; }
ok() { printf '\033[32m✓\033[0m %s\n' "$1"; }
warn() { printf '\033[33m!\033[0m %s\n' "$1"; }
# The git SHA stamped into the built satellite bundle (empty when never built).
# This is what the hot-deploy panel shows as "latest" and compares node versions
# against — NOT the control plane's HEAD, which is why the two can silently drift.
bundle_version() { as_owner "cat satellite/dist/VERSION 2>/dev/null || true"; }
# ── Preflight ────────────────────────────────────────────────────────────────
say "Preflight"
# Kerberos. The devbox sshd offers ONLY gssapi-keyex / gssapi-with-mic — no
# publickey — and a ticket lives ~10h, so every morning the SSH check below used
# to fail with an error that reads like a key or network problem. Renew it here:
# if there is no valid ticket, kinit with the corp password from the repo's local
# .env (KINIT_PASSWORD; .env is gitignored). The value is parsed out with sed, not
# `source`d, so nothing else in .env leaks into this shell. It reaches kinit over
# stdin (Heimdal's --password-file=STDIN): never on argv, never in a temp file,
# never echoed. This writes only to the local credential cache, so --check does it too.
ENV_FILE="${STREAMLINE_DEPLOY_ENV_FILE:-$(cd "$(dirname "$0")/../../.." && pwd)/.env}"
# git worktree(如 ~/orca/workspaces/streamline/*)里没有 .env —— 它被 gitignore,不跟着 worktree 走。
# 退到主检出的仓库根(worktree 共用的 .git 的上一级)去找,省得每次手动指 STREAMLINE_DEPLOY_ENV_FILE。
if [ -z "${STREAMLINE_DEPLOY_ENV_FILE:-}" ] && [ ! -f "$ENV_FILE" ]; then
MAIN_ROOT=$(cd "$(dirname "$0")" && git rev-parse --path-format=absolute --git-common-dir 2>/dev/null | sed 's#/\.git$##')
[ -n "$MAIN_ROOT" ] && [ -f "$MAIN_ROOT/.env" ] && ENV_FILE="$MAIN_ROOT/.env"
fi
KRB_PRINCIPAL="${STREAMLINE_DEPLOY_PRINCIPAL:-${SSH_USER}@BYTEDANCE.COM}"
if klist -s 2>/dev/null; then
ok "kerberos ticket valid"
else
KINIT_PW=$(sed -n 's/^KINIT_PASSWORD=//p' "$ENV_FILE" 2>/dev/null | tail -1)
KINIT_PW=${KINIT_PW%$'\r'}
# Tolerate one layer of matching quotes, as dotenv does.
if [ "${#KINIT_PW}" -ge 2 ]; then
case "$KINIT_PW" in \"*\"|\'*\') KINIT_PW=${KINIT_PW:1:${#KINIT_PW}-2} ;; esac
fi
[ -n "$KINIT_PW" ] || fail "kerberos ticket missing/expired and no KINIT_PASSWORD in ${ENV_FILE} — add it there, or run: kinit ${KRB_PRINCIPAL}"
printf '%s\n' "$KINIT_PW" | kinit --password-file=STDIN "$KRB_PRINCIPAL" >/dev/null 2>&1
KINIT_RC=$?
unset KINIT_PW
[ "$KINIT_RC" = 0 ] && klist -s 2>/dev/null \
|| fail "kinit ${KRB_PRINCIPAL} failed — KINIT_PASSWORD in ${ENV_FILE} is probably stale (password changed?)"
ok "kerberos ticket renewed for ${KRB_PRINCIPAL}"
fi
ssh_raw 'echo ok' >/dev/null 2>&1 || fail "cannot SSH to ${SSH_USER}@${HOST} — need corp network + a Kerberos ticket (klist); if the username is wrong set STREAMLINE_DEPLOY_USER=<corp id>"
ok "ssh ${SSH_USER}@${HOST}"
ssh_raw "test -d ${REPO}/.git" || fail "no git repo at ${REPO}"
ok "repo ${REPO}"
ssh_raw 'sudo -n true' 2>/dev/null || fail "passwordless sudo unavailable (needed to act as ${OWNER})"
ok "sudo -n"
ssh_raw "test -x ${BUN_BIN}/bun" || fail "bun not found at ${BUN_BIN}/bun"
ok "bun at ${BUN_BIN}"
ssh_raw "systemctl cat ${UNIT}.service >/dev/null 2>&1" || fail "no systemd unit ${UNIT}.service"
ok "unit ${UNIT}.service"
# A dirty tree means someone edited production in place. Pulling over it either
# fails or silently discards their work — stop and let a human decide.
DIRTY=$(as_owner 'git status --porcelain 2>/dev/null | grep -v "^?? " | head -20')
if [ -n "$DIRTY" ]; then
printf '\033[31m✗ working tree at %s has uncommitted tracked changes:\033[0m\n%s\n' "$REPO" "$DIRTY" >&2
fail "refusing to pull over local edits — resolve on the box first"
fi
ok "working tree clean (tracked files)"
# 一次往返取两个值:分支和 SHA 总是一起用,拆成两次调用只是多付一次握手。
HEAD_INFO=$(as_owner "git rev-parse --short HEAD; git rev-parse --abbrev-ref HEAD")
BEFORE=$(printf '%s\n' "$HEAD_INFO" | sed -n 1p)
CUR_BRANCH=$(printf '%s\n' "$HEAD_INFO" | sed -n 2p)
ok "at ${BEFORE} on ${CUR_BRANCH}"
[ "$CUR_BRANCH" = "$BRANCH" ] || fail "on branch '${CUR_BRANCH}', expected '${BRANCH}'"
# Report the satellite bundle's stamped SHA + age. It is a SEPARATE artifact from
# the control plane and nothing before this script rebuilt it, so it routinely sat
# several commits behind while the panel confidently labelled nodes 「最新」.
BUNDLE_BEFORE=$(bundle_version)
if [ -z "$BUNDLE_BEFORE" ]; then
warn "satellite bundle: never built"
else
# 年龄在远端一次算完:本地做减法要 stat 和 date 两次往返,而它们只为了得出一个分钟数。
MINS=$(as_owner "echo \$(( ( \$(date +%s) - \$(stat -c %Y satellite/dist/satellite.js 2>/dev/null || echo 0) ) / 60 ))")
if [ "$BUNDLE_BEFORE" = "$BEFORE" ]; then
ok "satellite bundle at ${BUNDLE_BEFORE} (built ${MINS}m ago, matches HEAD)"
else
warn "satellite bundle at ${BUNDLE_BEFORE} (built ${MINS}m ago) — HEAD is ${BEFORE}"
fi
fi
if [ "$CHECK_ONLY" = 1 ]; then
# Read-only, against the tree that is deployed right now. A real deploy runs
# the same check AFTER the pull, where it is blocking.
say "Schema check (current tree)"
as_owner "bun scripts/check-migrations.ts 2>&1 | tail -20" || warn "生产库缺少迁移声明过的对象(见上)"
say "Preflight only (--check) — nothing was changed"
exit 0
fi
# ── Deploy ───────────────────────────────────────────────────────────────────
# --satellite-only rebuilds just the bundle from the tree that is already
# deployed: no pull, no frontend build, no restart. For the common case of "the
# bundle drifted behind the control plane" there is no reason to bounce a healthy
# service to fix an artifact it never reads.
if [ "$SAT_ONLY" = 1 ]; then
AFTER="$BEFORE"
say "Satellite bundle only (--satellite-only) — control plane left untouched at ${BEFORE}"
else
say "git pull origin ${BRANCH}"
as_owner "git pull origin ${BRANCH} 2>&1 | tail -6" || fail "git pull failed"
AFTER=$(as_owner "git rev-parse --short HEAD")
ok "${BEFORE} → ${AFTER}"
[ "$BEFORE" = "$AFTER" ] && printf ' (already up to date — rebuilding anyway)\n'
say "bun install --frozen-lockfile"
# Pulling package.json/bun.lock does not update an existing node_modules tree.
# Reconcile it on every deploy: Bun's no-op path is cheap, while skipping this
# can leave the freshly restarted server unable to resolve a new runtime package.
# Keep this before schema/build/restart so an install failure leaves the old
# process serving and never restarts onto an incomplete dependency tree.
as_owner "bun install --frozen-lockfile 2>&1 | tail -20" \
|| fail "dependency install failed — old service is still running; fix registry/lockfile and re-run before restarting"
ok "dependencies match bun.lock"
# The schema check has to sit HERE: after the pull (so it sees the migrations the
# incoming code needs) and before the restart (so a missing column is a warning
# on a still-healthy service instead of a 500 on a restarted one).
#
# Why this step exists at all: this repo has no migration runner, so whoever
# deploys pays for EVERYONE's unapplied migrations — `git pull` brings along
# commits that were merged to main without their DDL ever being run. On
# 2026-08-22 a knowledge-base deploy carried `tasks.method_coverage` to
# production, that column had never been created, and /api/tasks returned 500
# for four minutes. The verify step caught it — one restart too late.
say "Schema check"
as_owner "bun scripts/check-migrations.ts 2>&1 | tail -20" \
|| fail "生产库缺少迁移声明过的对象(见上)。先按提示跑 apply-migration.ts,再重跑部署 —— 此刻服务仍在跑旧代码,是安全的"
ok "schema matches drizzle/"
say "bun run build"
# Vite empties dist/ before writing, so a failure here leaves NO frontend. Bail
# before the restart and report it loudly rather than restarting onto rubble.
as_owner "bun run build 2>&1 | tail -8" || fail "build failed — dist/ may be empty; fix and re-run BEFORE restarting"
ASSETS=$(as_owner "ls control-web/dist/assets/")
ok "built: $(echo "$ASSETS" | tr '\n' ' ')"
say "systemctl restart ${UNIT}"
ssh_raw "sudo -n systemctl restart ${UNIT}" || fail "restart failed"
# 固定 `sleep 6` 换成轮询,而且轮询的是**接口真的能应答**,不只是 unit active:
# - 快的时候不白等;
# - 慢的时候(机器忙 / DB 连接慢)不会因为固定秒数不够而把一次正常启动误判成失败 ——
# 旧写法在那一档是**假阴性**,而假阴性会让人以为部署炸了,去动一个其实好好的服务;
# - active 与「能服务」不是一回事:进程起来了但还没绑上 :5173 时,紧接着的验证步骤
# 会撞上连接被拒。等到 /api/tasks 应答才算就绪,后面的断言就不会踩到这个窗口。
# 循环跑在远端 —— 一次往返,不是每轮一次握手。unit 直接 failed 就立刻退出,不等满 30 秒。
WAIT_RC=0
ssh_raw "for i in \$(seq 1 60); do
st=\$(systemctl is-active ${UNIT} 2>/dev/null)
if [ \"\$st\" = failed ]; then exit 2; fi
if [ \"\$st\" = active ] && curl -sf -m 3 -o /dev/null http://127.0.0.1:5173/api/tasks; then exit 0; fi
sleep 0.5
done; exit 1" || WAIT_RC=$?
if [ "$WAIT_RC" != 0 ]; then
ssh_raw "sudo -n journalctl -u ${UNIT} -n 30 --no-pager" >&2
[ "$WAIT_RC" = 2 ] && fail "${UNIT} entered failed state after restart"
fail "${UNIT} did not answer on :5173 within 30s after restart"
fi
ok "${UNIT} active and answering"
# ── Verify ───────────────────────────────────────────────────────────────────
# The frontend is served BY the control-server from control-web/dist, so these
# two checks together prove both tiers picked up the new build.
say "Verify"
# LC_ALL=C on BOTH sides of this comparison, not for tidiness: the devbox sorts
# in C locale and a developer Mac sorts in en_US.UTF-8, which disagree about `_`
# versus a digit (`index-D_k….css` sorts before `index-D7Z….js` on the Mac and
# after it on Linux). The public-vs-served check below compares these two lists
# as STRINGS, so the same two assets in a different order looked like a stale
# Caddy cache — a warning that fires on a healthy deploy, which is the fastest
# way to teach everyone to ignore the one that matters.
SERVED=$(ssh_raw "curl -s -m 15 http://127.0.0.1:5173/ | grep -oE 'index-[A-Za-z0-9_-]+\.(js|css)' | LC_ALL=C sort -u")
[ -n "$SERVED" ] || fail "served index.html references no assets — frontend is broken"
ok "served assets: $(echo "$SERVED" | tr '\n' ' ')"
# The hashes in index.html must be the ones we just built, else a stale index is
# being served (or the build wrote somewhere else entirely).
for a in $SERVED; do
echo "$ASSETS" | grep -qx "$a" || fail "served asset ${a} is not in the fresh build — stale index.html"
done
ok "served assets match the fresh build"
CODE=$(ssh_raw "curl -s -m 20 -o /dev/null -w '%{http_code}' http://127.0.0.1:5173/api/tasks")
[ "$CODE" = "200" ] || fail "GET /api/tasks returned ${CODE}"
ok "GET /api/tasks 200"
PUB=$(curl -s -m 30 https://streamline.bytedance.net/ 2>/dev/null | grep -oE 'index-[A-Za-z0-9_-]+\.(js|css)' | LC_ALL=C sort -u)
if [ -n "$PUB" ]; then
if [ "$PUB" = "$SERVED" ]; then ok "public entrypoint serves the new build"
else printf '\033[33m!\033[0m public entrypoint serves %s (expected %s) — check Caddy cache\n' "$(echo "$PUB" | tr '\n' ' ')" "$(echo "$SERVED" | tr '\n' ' ')"; fi
else
printf '\033[33m!\033[0m could not reach https://streamline.bytedance.net from here (need corp network)\n'
fi
ok "control plane deployed: ${BEFORE} → ${AFTER}"
fi
# ── Satellite bundle ─────────────────────────────────────────────────────────
# `shared/` and `satellite/` are compiled INTO satellite/dist/satellite.js, which
# the hot-deploy panel pushes to the executor nodes. Deploying the control plane
# leaves that artifact untouched, so the panel kept offering an older SHA and
# every node looked up to date. Rebuild it unconditionally: it costs seconds,
# touches no running service, and makes the panel's "latest" mean "what is
# actually deployed" instead of "whatever someone last built by hand".
#
# Pushing is deliberately NOT done here — see the header.
#
# Two questions, not one — two mechanisms carry code to a node:
# bundled shared/ + satellite/, plus the bin/ wrappers that
# satellite/src/entry.ts imports (bun build folds those in).
# NOT bundled the rest of bin/ + scripts/run-satellite.sh. claude_launch.ts
# resolves those from the node's OWN checkout, which hot-deploy
# never touches — the push CANNOT carry them, and the node still
# reports itself current because dist/VERSION is all it reports.
# bin/claude-pty.ts moved into the bundled column once entry.ts started importing
# it. That was the one that actually hurt: the PTY engine runs every production
# job, and a measured deploy shipped 131 unbundled lines of it under an
# affirmative "the push covered it". What is left on the node side is
# bin/claude-cursor.ts (not used in production) and run-satellite.sh (the daemon
# loop itself, which cannot bundle itself).
# NODE_FILES is therefore computed UNCONDITIONALLY: the old `shared/ satellite/`
# pathspec missed bin/ both when it was the only change (silence) and when it
# rode along with real bundle changes (an affirmative "the push covered it").
# Worked example + sync procedure: SKILL.md 「但 bundle 不是全部」.
#
# The node-side list is derived by SUBTRACTION rather than enumerated: scan all
# of bin/, filter out what entry.ts bundles. A new wrapper dropped into bin/ then
# warns by default instead of being silently missed — which is the exact failure
# this block exists to prevent. Promote a file into BUNDLED_BIN when entry.ts
# starts importing it.
# Derived from entry.ts rather than typed out, because the hand-maintained list
# drifted the moment anyone added a wrapper: bin/claude-codex.ts and bin/kb.ts
# were both bundled and both absent from it, so a deploy that touched either one
# printed "NOT carried by the push" about code that WAS carried. A warning that
# cries wolf is worse than no warning — it is the one that gets ignored on the
# day it is right (which is exactly the claude-pty story this block was written
# for). Falls back to the old literal if the grep ever comes up empty, so a
# refactor of entry.ts degrades into over-warning rather than under-warning.
BUNDLED_BIN="$(grep -o 'import("\.\./\.\./bin/[a-z-]*")' satellite/src/entry.ts \
| sed -E 's|import\("\.\./\.\./(bin/[a-z-]*)"\)|\1.ts|' | sort -u | tr '\n' ' ')"
BUNDLED_BIN="${BUNDLED_BIN:-bin/record-evidence.ts bin/conditions.ts bin/claude-pty.ts}"
SAT_BUNDLED_PATHS="shared/ satellite/ ${BUNDLED_BIN}"
NODE_SCAN_PATHS="bin/ scripts/run-satellite.sh"
# Compare against the SHA the bundle was built from, not BEFORE: a skipped
# earlier deploy may already have left satellite changes unbundled.
SAT_BASE="${BUNDLE_BEFORE:-$BEFORE}"
SAT_NEEDED=""
EQUIV_ARGS=""
# One round trip; the bundled entries are filtered off locally, same reason the
# sha256sum below is trimmed locally (as_owner embeds its argument in quotes).
# grep exits 1 when it filters everything out, hence the `|| true`.
NODE_FILES=$(as_owner "git diff --name-only ${SAT_BASE}..${AFTER} -- ${NODE_SCAN_PATHS} 2>/dev/null" \
| grep -vxF "$(printf '%s\n' ${BUNDLED_BIN})" || true)
if [ "$SKIP_SATELLITE" = 1 ]; then
say "Satellite bundle (skipped via --skip-satellite)"
warn "bundle still at ${BUNDLE_BEFORE:-none} — the panel will offer that, not ${AFTER}"
elif [ "${BUNDLE_BEFORE:-}" = "$AFTER" ]; then
# 用预检读到的值,不再问一次:从预检到这里只有 pull / 前端构建 / 重启,没有任何一步
# 会重写 satellite/dist,所以再读一次必然得到同一个答案。
say "Satellite bundle"
ok "already built at ${AFTER} — nothing to do"
else
say "bun run build:satellite"
# Does this deploy actually change BUNDLED satellite behaviour? Answering it
# here is the judgement call the operator used to have to make by hand (and
# miss). Node-checkout files are a separate question — see NODE_FILES above.
SAT_DIFF=$(as_owner "git diff --stat ${SAT_BASE}..${AFTER} -- ${SAT_BUNDLED_PATHS} 2>/dev/null | tail -1")
[ -n "$SAT_DIFF" ] && SAT_NEEDED=1
# Keep the outgoing bundle as .bak-<sha> so a bad rollout can be reverted by
# copying it back and re-pushing; keep the 3 newest so this can't grow forever.
if [ -n "${BUNDLE_BEFORE:-}" ]; then
as_owner "cp -p satellite/dist/satellite.js satellite/dist/satellite.js.bak-${BUNDLE_BEFORE} 2>/dev/null; cp -p satellite/dist/VERSION satellite/dist/VERSION.bak-${BUNDLE_BEFORE} 2>/dev/null; true" >/dev/null
PRUNED=$(as_owner "ls -t satellite/dist/satellite.js.bak-* 2>/dev/null | tail -n +4")
if [ -n "$PRUNED" ]; then
as_owner "ls -t satellite/dist/satellite.js.bak-* 2>/dev/null | tail -n +4 | xargs -r rm -f; ls -t satellite/dist/VERSION.bak-* 2>/dev/null | tail -n +4 | xargs -r rm -f; true" >/dev/null
printf ' (pruned old backups: %s)\n' "$(echo "$PRUNED" | xargs -n99 echo | sed 's#satellite/dist/satellite.js.bak-##g')"
fi
ok "previous bundle kept as satellite.js.bak-${BUNDLE_BEFORE}"
fi
as_owner "bun run build:satellite 2>&1 | tail -4" || fail "satellite bundle build failed — control plane IS deployed and live; the panel still offers ${BUNDLE_BEFORE:-none}"
BUNDLE_AFTER=$(bundle_version)
BUNDLE_SIZE=$(as_owner "stat -c %s satellite/dist/satellite.js 2>/dev/null || echo 0")
# A truncated bundle would pass the node's sha256 check (that only proves the
# bytes arrived intact) and then fail to boot, so sanity-check it right here.
[ "$BUNDLE_AFTER" = "$AFTER" ] \
|| fail "bundle stamped ${BUNDLE_AFTER} but HEAD is ${AFTER} — check for a dirty tree on the box"
[ "${BUNDLE_SIZE:-0}" -ge 100000 ] \
|| fail "bundle is only ${BUNDLE_SIZE} bytes — looks truncated, do NOT push it"
ok "bundle built at ${BUNDLE_AFTER} ($(( BUNDLE_SIZE / 1024 )) KB)"
# A bundle's VERSION is the git SHA, so a deploy that touched only the control
# plane still produces a "new" bundle — byte-for-byte identical, stamped
# differently. Pushing that drains and restarts every node to change a string.
# Compare against the bundle we just backed up: when the bytes match, tell the
# push step that nodes on the OLD version are already up to date.
if [ -n "${BUNDLE_BEFORE:-}" ]; then
# 一次往返算两个文件的哈希,拆开只是多付一次握手。
# 拆解在**本地**做:`as_owner` 把参数塞进单引号包住的 `bash -lc '...'`,远端写
# `cut -d' '` 会提前闭合那个引号,静默得到空哈希(于是永远不跳过那次无谓的换装)。
# 按文件名取而不是按行序取:bak 不存在时 sha256sum 只输出一行,靠行序会把新 bundle
# 的哈希当成旧的,两边一比"相同",于是真该推的一次被跳过。
SHAS=$(as_owner "sha256sum satellite/dist/satellite.js satellite/dist/satellite.js.bak-${BUNDLE_BEFORE} 2>/dev/null")
SHA_NEW=$(printf '%s\n' "$SHAS" | grep " satellite/dist/satellite.js$" | cut -d' ' -f1)
SHA_OLD=$(printf '%s\n' "$SHAS" | grep "\.bak-${BUNDLE_BEFORE}$" | cut -d' ' -f1)
if [ -n "$SHA_OLD" ] && [ "$SHA_NEW" = "$SHA_OLD" ]; then
EQUIV_ARGS="--equivalent-version ${BUNDLE_BEFORE}"
ok "bundle is byte-identical to ${BUNDLE_BEFORE} — nodes on it will NOT be churned"
fi
fi
fi
# ── Push to the satellites ───────────────────────────────────────────────────
# The last mile: get the bundle onto the executor nodes. Runs unconditionally
# (not just when satellite/ changed) because the script is idempotent — a node
# already on this version is skipped by version compare — and because "did
# anything satellite-side change" is exactly the judgement humans got wrong.
#
# It runs ON the box, as the owner: it needs CLUSTER_SECRET from the repo's .env
# (bun loads it from cwd) and a route to the nodes. See scripts/push-satellites.ts
# for why it goes over HTTP rather than through /api/deploy (SSO-gated) — and for
# why it waits for a fresh heartbeat first (the control plane we just restarted
# re-learns each node's queued-deploy capability from heartbeats; pushing into
# that blind window would treat every busy node as unable to park a rollout).
PUSH_RC=0
if [ "$SKIP_SATELLITE" = 1 ] || [ "$PUSH_SATELLITES" = 0 ]; then
say "Satellite push (skipped)"
warn "nodes keep their current bundle — push from 管理 → 卫星热部署 when you want it live"
else
say "Push bundle to satellites"
PUSH_ARGS="${EQUIV_ARGS}"
[ "$FORCE_PUSH" = 1 ] && PUSH_ARGS="${PUSH_ARGS} --force"
as_owner "bun scripts/push-satellites.ts ${PUSH_ARGS} 2>&1" || PUSH_RC=$?
if [ "$PUSH_RC" != 0 ]; then
warn "some nodes did not take the bundle (see above) — the control plane IS deployed and live"
fi
fi
if [ "$SAT_ONLY" = 1 ]; then say "Satellite bundle refreshed (control plane unchanged at ${BEFORE})"
else say "Deployed: ${BEFORE} → ${AFTER}"; fi
if [ "$SKIP_SATELLITE" != 1 ] && [ -n "$SAT_NEEDED" ]; then
printf 'Bundled satellite code CHANGED in this deploy (%s) — that is what the push above was for.\n' "${SAT_DIFF# }"
elif [ "$SKIP_SATELLITE" != 1 ] && [ -n "$EQUIV_ARGS" ]; then
printf 'No bundled satellite changes: the new bundle is byte-identical to %s, so nodes already on it were left alone.\n' "${BUNDLE_BEFORE}"
elif [ "$SKIP_SATELLITE" != 1 ]; then
printf 'No bundled satellite changes in this deploy — any node that still took the bundle was behind for other reasons.\n'
fi
# The bundle push above cannot ship these, no matter how it exits. Say so loudly:
# the failure mode is silent by construction — the node keeps reporting
# dist/VERSION (which IS current), so nothing downstream ever looks wrong.
if [ -n "${NODE_FILES}" ]; then
warn "NODE-CHECKOUT files changed — the bundle push does NOT carry them: $(echo "$NODE_FILES" | tr '\n' ' ')"
# State the invariant, not a recipe: how these files reach a node depends on how
# that node was provisioned, and executor-snapshot.sh strips .git by default, so
# a snapshot-restored node cannot fast-forward anything.
cat <<NODEWARN
They are read from each executor's own checkout of this repo; hot-deploy replaces
only satellite/dist/satellite.js. Until every executor has them at ${AFTER} the
change is NOT live — and the panel still shows each node as up to date, because a
node reports satellite/dist/VERSION and nothing else. A checkout with git history
can fast-forward to ${BRANCH}; one restored from executor-snapshot.sh has no .git,
so the files have to be copied in.
NODEWARN
fi
cat <<NOTE
Still not covered by this script:
* Report backfill. Prompt/report changes only affect NEWLY generated reports;
existing rows keep their old HTML until backfilled (a production write).
* Files read from each node's own checkout (${NODE_SCAN_PATHS}, minus what
entry.ts bundles). Synced by hand; this script only WARNS, and only on the
deploy whose range touches them.
NOTE
exit "$PUSH_RC"