Initial import of streamline

This commit is contained in:
2026-11-16 17:26:44 -08:00
commit 8e4aec1855
1221 changed files with 274256 additions and 0 deletions
+99
View File
@@ -0,0 +1,99 @@
---
name: streamline-retro
description: "回溯调试 Streamline 平台上的某个 Feature / Bugfix / Security 任务:给一个 task id(或粘贴的面板 URL),把该任务的全部产物(记录、TestSpec、ImplReport、session 日志时间线、证据、复核历史、可读性报告)拉到本地并定位原因。Use when the user pastes a streamline/bbbbbbug task uuid, or asks 为什么这个任务失败 / 这个 feature 为什么没实现 / 看下这个任务的日志·证据·报告·复核意见 / 回溯 debug / retro debug a task."
---
# Streamline 任务回溯调试
用户给一个 id 就够了。目标是在**一次抓取**之后,用本地文件回答"当时发生了什么、为什么是这个结果"。
## 前提
- 数据源是控制面的只读 API:`https://streamline-internal-api.bytedance.net`(可用 `STREAMLINE_API` 覆盖)。
需要公司内网;不需要 token、不需要 DB 账号密码、不需要 SSH 到卫星。
- **全程只读**。本 skill 只发 GET。任何写操作(重跑、abort、改 plan、发 MR)都不属于回溯,要做请显式征得用户同意后走面板或另行确认。
## 流程
### 1. 抓取
```bash
bash .claude/skills/streamline-retro/fetch.sh <id> [--out DIR] [--evidence]
```
- `<id>` 接受裸 uuid,也接受粘贴的 URL(`…?task_id=<uuid>`、`…/features/<uuid>`)——脚本自己抠 uuid。
- 脚本自动判定这个 id 是 **feature / task(bug) / security / verification(验证) / prototype(原型)** 中的哪一种(依次探测,404 就换下一个),并按类型选对报告端点(验证另存 `audit-report.json` 结构化报告数据)。
- 每一批的 session / timing / runner 日志都落到 `logs/`。接口把单个文件截在 **8 MB**(尾部),长任务(4 小时的走查有 30–60 MB)会被截断 —— 脚本发现接口给的比节点上的文件短,就用 `scp <节点别名>:<工作区>/logs/<文件>` 只读拷完整的一份(要能 `ssh <节点>`;连不上就保留截断版并在摘要里标出,`--no-ssh` 关掉)。
- `--out` 缺省是 `$TMPDIR/streamline-retro/<id>`。**在 Claude Code 里请显式传 scratchpad 目录**。
- `--evidence` 额外把证据里的**图片**下载到 `evidence/`(最多 40 张),之后可以用 Read 直接看图。视频不下载,用 `evidence.json` 里的 URL。
- stdout 末尾会打印 `OUT=<目录>` 和 `KIND=<features|tasks|security>`,以及每个产物的 HTTP 码(**404 表示该任务本来就没有这个产物,不是故障**)。
### 2. 压缩 session 日志
`session.jsonl` 是 claude stream-json,动辄 3–5 MB,其中 95% 是 thinking 签名和重复的工具输出,**不要直接 Read**。先压成时间线:
```bash
bun .claude/skills/streamline-retro/digest.ts <OUT>/session.jsonl # → session-timeline.md
bun .claude/skills/streamline-retro/digest.ts <OUT>/eval.jsonl # 若存在(评估期日志)
```
产出 `*-timeline.md`(实测 3.5 MB → 59 KB):头部是统计 + **把所有报错的工具调用提到最前面**,然后是按序的 `🔧 工具调用 / ↩︎ 结果摘要 / 💬 助手正文 / ❌ 报错 / 💭 思考`,每条带 `[#N]` 事件序号。
需要看某处完整上下文时,用 `--full` 重跑,或直接 `grep -n` 原始 `session.jsonl` 的第 N 行。
### 2b. 成本与效率画像
```bash
python3 .claude/skills/streamline-retro/stats.py <OUT>/logs/session-rN.log <OUT>/logs/session-rN.timing.jsonl
```
模型、每段会话的上下文起止 / 峰值、token 用量与按标价的估算花费、工具结果按类别的体量占比(截图占多少)、工具执行 vs 思考时长、写死的 sleep、撞 120 秒超时次数。问「这一单花了多少 / 慢在哪 / 上下文为什么这么大」时先跑它。
### 3. 按问题类型选读
抓下来的目录布局:
| 文件 | 内容 | 什么时候读 |
|---|---|---|
| `fetch-summary.txt` | 状态 / 轮次 / 执行节点 / 分支 / MR / failureReason | **永远先读这个** |
| `meta.json` | 完整记录(已剔除 readableReportHtml) | 要看时间戳、watchers、lang、portSourceId 等字段 |
| `OriginalIntro.md` | 原始需求(Meego/PRD 正文已内联) | 判断"是不是理解错了需求" |
| `TestSpec.md` | 人工确认过的实现方案(权威) | 判断"是不是方案本身就有问题" |
| `ImplReport.md` / `FixReport.md` / `SecurityReport.md` | Agent 自述:实现摘要、验收对照、**方案权衡**、**验证局限** | 查"它自己承认哪里没做到"——找原因优先看这两节 |
| `session-timeline.md` | 执行过程 | 查"它到底试了什么、卡在哪一步" |
| `review-history.json` | 历次复核意见(含人工 reviewFeedback) | 查"为什么被打回、打回了几轮" |
| `asking.json` | 执行期人↔Agent 的问答 | 查"中途有没有人给过额外指令" |
| `evidence.json` | 证据文件清单 + TOS 下载 URL | 挑图看;配合 ImplReport 的「测试图清单」对照 |
| `readable-report.html` | 给 PM/复核看的图文报告 | 想快速拿结论而非过程 |
### 4. 常见问法 → 直接去哪儿
- **"为什么没实现 / 为什么失败"** → `fetch-summary.txt` 的 `failure` + `ImplReport.md` 的「验证局限」「方案权衡」+ `session-timeline.md` 头部的报错区。三者交叉。
- **"为什么被打回"** → `review-history.json` + `meta.json` 的 `reviewFeedback`。
- **"它改了什么"** → `ImplReport.md` 的「实现摘要」+ `meta.json` 的 `branchName` / `mrUrl`(要看 diff 就去 MR)。
- **"证据够不够真"** → `ImplReport.md` 的「验收对照」逐条看是否标注了机制级;再对 `evidence.json` 里的 `mech-*` 文件核对。
- **"在哪台机器跑的"** → `fetch-summary.txt` 的 `node`(排查环境类问题时必看:iOS 走 `fei-ios-*`)。
## 已知限制(回答时要如实说明,不要脑补)
- `session.jsonl` 走的是控制面的 **tail** 接口,默认拿到的是**最新一批**。
- **2026-08-13 之后开跑的批次**按批分文件(`session-r{N}.log`),可以逐批取:
`GET /api/{tasks|features|security}/<id>/logs` 列出 `logs/` 下所有文件(含每批的 `critic-N.log`、`task-context-N.json`),
`GET …/logs/<name>?lines=2000` 取其中一份,`GET …/log-file?round=N` 直接取第 N 批。
- **在那之前跑的多轮任务**仍然只剩最后一批(旧代码每批截断重写同一个 `session.log`),早期批次的过程**已经不存在了**,不要脑补,也不要说"在别处"。
- 不依赖节点的那一层是 `GET …/round-digests`:每批的会话摘要(失败过的命令、报错、动过的文件、终止哨兵),存在 DB 里,节点离线/工作区被清也还在。同样只覆盖归档能力上线之后跑的批次。
- `eval.jsonl`(评估期 / TestSpec 生成过程)对**已完成**的任务基本都是 404——`evalLogPath` 跑完就被清空了。只有还在评估中或刚评估完的任务能拿到。新任务的每次评估 / 计划重生成各自落 `eval-r{N}.log`,可以在 `logs` 清单里看到有几次。
- 工作区被清理过 / 任务被删过的老任务,`fix-report`、`log-file`、`evidence` 都会 404,但 `meta.json` 和 `readable-report` 仍在(它们存 DB)。
- 任务硬绑定 home node;节点离线时所有走代理的产物端点都会失败——此时 `fetch-summary.txt` 里 DB 字段仍然正常,别误判成"任务没跑"。
- 跨任务的聚合查询("最近所有 iOS 失败的 feature")REST 给不了,需要直连 PG,见下。
## 需要跨任务聚合时
`/api/features`、`/api/tasks`、`/api/security` 返回全量列表,绝大多数聚合用 `jq` 在本地过滤即可,**优先走这条路**。
注意三者的包装键不同:`{features:[…]}`、`{tasks:[…], eval, currentId}`、`{security:[…]}`。
```bash
curl -sS -m 60 https://streamline-internal-api.bytedance.net/api/features \
| jq '[.features[] | select(.platform=="ios" and .status=="unfixable") | {id,title,failureReason}]'
```
只有列表接口确实不够时才考虑直连生产 PG(`DATABASE_URL` 在仓库 `.env`)。直连的规矩:**只允许 SELECT**,禁止任何 INSERT/UPDATE/DELETE/DDL,且必须先跟用户确认。
+157
View File
@@ -0,0 +1,157 @@
#!/usr/bin/env bun
// ---------------------------------------------------------------------------
// streamline-retro / digest.ts — collapse a claude stream-json session log into
// a readable timeline.
//
// bun digest.ts <session.jsonl> [--out timeline.md] [--full]
//
// Purely local file processing: no network, no DB. A 3-5 MB session.jsonl is
// ~95% base64 thinking signatures and repeated tool output; the timeline is the
// 1% you actually read when asking "what did the agent do, and where did it go
// wrong". Errors are hoisted to the top so the failure point is the first thing
// visible. Use --full to keep untruncated tool inputs/outputs.
// ---------------------------------------------------------------------------
import { basename } from "node:path";
const args = process.argv.slice(2);
const file = args.find((a) => !a.startsWith("--"));
if (!file) {
console.error("usage: bun digest.ts <session.jsonl> [--out timeline.md] [--full]");
process.exit(2);
}
const full = args.includes("--full");
const outIdx = args.indexOf("--out");
const outFile = outIdx >= 0 ? args[outIdx + 1] : file.replace(/\.jsonl$/, "") + "-timeline.md";
const CAP_IN = full ? 100_000 : 300; // tool_use input
const CAP_OUT = full ? 100_000 : 220; // tool_result output
const CAP_ERR = full ? 100_000 : 1_500; // errored tool_result
const CAP_TEXT = full ? 100_000 : 1_200; // assistant prose
const oneLine = (s: string, cap: number) => {
const t = String(s).replace(/\s+/g, " ").trim();
return t.length > cap ? t.slice(0, cap) + ` …(+${t.length - cap} chars)` : t;
};
const block = (s: string, cap: number) => {
const t = String(s).replace(/\r/g, "").trim();
return t.length > cap ? t.slice(0, cap) + `\n…(+${t.length - cap} chars, rerun with --full)` : t;
};
/** One-line gist of a tool call — the arg that identifies *which* call it was. */
function toolGist(name: string, input: any): string {
if (!input || typeof input !== "object") return "";
const pick = (...keys: string[]) => {
for (const k of keys) if (typeof input[k] === "string" && input[k]) return input[k];
return "";
};
switch (name) {
case "Bash": return oneLine(pick("command"), CAP_IN);
case "Read": case "Write": case "NotebookEdit": return pick("file_path", "notebook_path");
case "Edit": return `${pick("file_path")} ⟨${oneLine(pick("old_string"), 60)}⟩`;
case "Grep": return `${oneLine(pick("pattern"), 80)} in ${pick("path", "glob") || "."}`;
case "Glob": return pick("pattern");
case "Task": case "Agent": return oneLine(pick("description", "prompt"), CAP_IN);
case "TodoWrite": return oneLine(JSON.stringify(input.todos ?? input), 160);
default: return oneLine(JSON.stringify(input), CAP_IN);
}
}
const lines = (await Bun.file(file).text()).split("\n").filter((l) => l.trim());
const out: string[] = [];
const errors: string[] = [];
const statuses: string[] = [];
const toolCounts = new Map<string, number>();
const pendingTool = new Map<string, string>(); // tool_use_id -> "Name gist"
let sessions = 0;
let events = 0;
// The init event carries an empty `model` in stream mode; the first assistant
// message is where the real model id shows up, so patch the header line later.
let modelLineIdx = -1;
let model = "";
for (const raw of lines) {
events++;
let ev: any;
try { ev = JSON.parse(raw); } catch { out.push(`[#${events}] ⚠️ unparseable line`); continue; }
const tag = `[#${events}]`;
if (ev.type === "system" && ev.subtype === "init") {
sessions++;
model = ev.model || "";
out.push("", `## ── session ${sessions} · model=\`${model || "?"}\` · cwd=\`${ev.cwd || "?"}\` · sid=\`${ev.session_id || "?"}\``, "");
modelLineIdx = out.length - 2;
continue;
}
if (ev.type === "assistant" && !model && ev.message?.model && modelLineIdx >= 0) {
model = ev.message.model;
out[modelLineIdx] = out[modelLineIdx].replace("model=`?`", `model=\`${model}\``);
}
if (ev.type === "result") {
const dur = ev.duration_ms ? `${(ev.duration_ms / 60000).toFixed(1)}min` : "?";
const cost = ev.total_cost_usd != null ? `$${Number(ev.total_cost_usd).toFixed(2)}` : "?";
out.push("", `## ── result · subtype=\`${ev.subtype || "?"}\` · turns=${ev.num_turns ?? "?"} · ${dur} · ${cost}${ev.is_error ? " · **is_error**" : ""}`);
if (ev.result) out.push("", "```", block(ev.result, full ? 100_000 : 3_000), "```");
continue;
}
const content = ev.message?.content;
if (!Array.isArray(content)) continue;
for (const c of content) {
if (c.type === "thinking") {
// Signatures are noise; only the (usually empty in stream mode) text matters.
const t = String(c.thinking || "").trim();
if (t) out.push(`${tag} 💭 ${block(t, CAP_TEXT)}`);
continue;
}
if (c.type === "text") {
const t = String(c.text || "").trim();
if (!t) continue;
out.push(`${tag} 💬 ${block(t, CAP_TEXT)}`);
for (const m of t.matchAll(/STATUS:\s*([A-Z_]+)/g)) statuses.push(`${tag} STATUS: ${m[1]}`);
continue;
}
if (c.type === "tool_use") {
const gist = toolGist(c.name, c.input);
toolCounts.set(c.name, (toolCounts.get(c.name) ?? 0) + 1);
pendingTool.set(c.id, `${c.name}${gist ? " " + oneLine(gist, 120) : ""}`);
out.push(`${tag} 🔧 **${c.name}** \`${gist.replace(/`/g, "'")}\``);
continue;
}
if (c.type === "tool_result") {
const body = typeof c.content === "string"
? c.content
: Array.isArray(c.content) ? c.content.map((x: any) => x?.text ?? `[${x?.type}]`).join("\n") : "";
const who = pendingTool.get(c.tool_use_id) ?? "tool";
if (c.is_error) {
const rendered = `${tag} ❌ ${who}\n\`\`\`\n${block(body, CAP_ERR)}\n\`\`\``;
out.push(rendered);
errors.push(rendered);
} else {
out.push(`${tag} ↩︎ ${oneLine(body, CAP_OUT) || "(empty)"}`);
}
continue;
}
}
}
const tools = [...toolCounts.entries()].sort((a, b) => b[1] - a[1]).map(([n, c]) => `${n}×${c}`).join(", ");
const header = [
`# session timeline — ${basename(file)}`,
"",
`- events: **${events}** · sessions(rounds): **${sessions}** · errored tool calls: **${errors.length}**`,
`- tools: ${tools || "(none)"}`,
statuses.length ? `- status sentinels: ${statuses.join(" · ")}` : `- status sentinels: (none found)`,
"",
];
if (errors.length) {
header.push(`## ❌ errored tool calls (${errors.length}) — read these first`, "", ...errors.slice(0, 40), "");
if (errors.length > 40) header.push(`…(+${errors.length - 40} more, see timeline below)`, "");
}
header.push("---", "", "# timeline");
await Bun.write(outFile, header.concat(out).join("\n") + "\n");
console.log(`wrote ${outFile} (${events} events, ${sessions} rounds, ${errors.length} errors, tools: ${tools || "none"})`);
+169
View File
@@ -0,0 +1,169 @@
#!/usr/bin/env bash
# ---------------------------------------------------------------------------
# streamline-retro / fetch.sh — snapshot everything about one task id to disk.
#
# bash fetch.sh <id> [--out <dir>] [--evidence] [--lines N]
#
# Read-only: every call is a GET against the control plane's public read API
# (STREAMLINE_API, default https://streamline-internal-api.bytedance.net).
# No DB credentials, no writes. The control plane already proxies per-node
# artifacts (session log / ImplReport / evidence) for us, so one host is enough
# regardless of which satellite ran the task — EXCEPT that its log endpoints cap
# a file at 8 MB (tail). Long runs blow through that (a 4-hour UI walkthrough
# writes 30–60 MB), so per-round logs whose API copy came back short are pulled
# read-only from the executing node with `scp` (needs the `ssh <node>` alias;
# skipped with a note when unreachable). --no-ssh turns that off.
#
# Prints a machine-readable KIND=… OUT=… footer so the caller knows where the
# snapshot landed and what kind of record the id turned out to be.
# ---------------------------------------------------------------------------
set -uo pipefail
BASE="${STREAMLINE_API:-https://streamline-internal-api.bytedance.net}"
ID=""
OUT_ROOT=""
WITH_EVIDENCE=0
LINES=200000
NO_SSH=0
while [ $# -gt 0 ]; do
case "$1" in
--out) OUT_ROOT="$2"; shift 2 ;;
--evidence) WITH_EVIDENCE=1; shift ;;
--lines) LINES="$2"; shift 2 ;;
--no-ssh) NO_SSH=1; shift ;;
-*) echo "unknown flag: $1" >&2; exit 2 ;;
*) ID="$1"; shift ;;
esac
done
[ -n "$ID" ] || { echo "usage: fetch.sh <task-id> [--out DIR] [--evidence] [--lines N]" >&2; exit 2; }
# Accept a pasted URL too (…?task_id=<uuid> / …/features/<uuid>) — pull the uuid out.
if [[ "$ID" =~ ([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}) ]]; then
ID="${BASH_REMATCH[1]}"
else
echo "not a uuid: $ID" >&2; exit 2
fi
OUT_ROOT="${OUT_ROOT:-${TMPDIR:-/tmp}/streamline-retro}"
OUT="$OUT_ROOT/$ID"
mkdir -p "$OUT"
get() { curl -sS -m 60 -w '\n%{http_code}' "$BASE$1" 2>/dev/null; }
# Fetch to a file; keep the file only on HTTP 200. Echoes the status code.
fetch_to() { # <path> <outfile>
local code
code=$(curl -sS -m 120 -o "$2.tmp" -w '%{http_code}' "$BASE$1" 2>/dev/null)
if [ "$code" = "200" ] && [ -s "$2.tmp" ]; then mv "$2.tmp" "$2"; else rm -f "$2.tmp"; fi
echo "$code"
}
# --- 1. Which kind of record is this id? -----------------------------------
KIND=""
for k in features tasks security verifications prototypes; do
code=$(curl -sS -m 20 -o "$OUT/meta.json.tmp" -w '%{http_code}' "$BASE/api/$k/$ID" 2>/dev/null)
if [ "$code" = "200" ]; then KIND="$k"; mv "$OUT/meta.json.tmp" "$OUT/meta.json"; break; fi
done
rm -f "$OUT/meta.json.tmp"
if [ -z "$KIND" ]; then
echo "id not found as feature / task / security / verification / prototype on $BASE" >&2
echo "KIND=none OUT=$OUT"
exit 1
fi
# The report endpoint is the one asymmetry between kinds.
REPORT_PATH="/api/$KIND/$ID/fix-report"
REPORT_FILE="ImplReport.md"
case "$KIND" in
tasks) REPORT_FILE="FixReport.md" ;;
security) REPORT_PATH="/api/security/$ID/security-report"; REPORT_FILE="SecurityReport.md" ;;
# 验证:报告原文(UI 走查 / 用例验证的 .md)+ 面板用的结构化数据;原型:接手包。
verifications) REPORT_PATH="/api/verifications/$ID/report"; REPORT_FILE="Verify-Report.md" ;;
prototypes) REPORT_PATH="/api/prototypes/$ID/report"; REPORT_FILE="ProtoReport.md" ;;
esac
# --- 2. Split the record: big blobs become their own readable files ---------
jq 'del(.readableReportHtml)' "$OUT/meta.json" > "$OUT/meta.slim.json" 2>/dev/null \
&& mv "$OUT/meta.slim.json" "$OUT/meta.json"
jq -r '.executionPlan // empty' "$OUT/meta.json" > "$OUT/TestSpec.md"
jq -r '.description // empty' "$OUT/meta.json" > "$OUT/OriginalIntro.md"
jq -r '.reviewHistory // empty' "$OUT/meta.json" > "$OUT/review-history.json"
[ -s "$OUT/TestSpec.md" ] || rm -f "$OUT/TestSpec.md"
[ -s "$OUT/OriginalIntro.md" ] || rm -f "$OUT/OriginalIntro.md"
[ -s "$OUT/review-history.json" ] || rm -f "$OUT/review-history.json"
# --- 3. Artifacts ----------------------------------------------------------
c_report=$(fetch_to "$REPORT_PATH" "$OUT/$REPORT_FILE")
c_sess=$( fetch_to "/api/$KIND/$ID/log-file?lines=$LINES" "$OUT/session.jsonl")
c_eval=$( fetch_to "/api/$KIND/$ID/eval-log-file?lines=$LINES" "$OUT/eval.jsonl")
c_evid=$( fetch_to "/api/$KIND/$ID/evidence" "$OUT/evidence.json")
c_ask=$( fetch_to "/api/$KIND/$ID/asking/history" "$OUT/asking.json")
c_read=$( fetch_to "/api/$KIND/$ID/readable-report" "$OUT/readable-report.json")
c_audit="-"
[ "$KIND" = "verifications" ] && c_audit=$(fetch_to "/api/verifications/$ID/audit-report" "$OUT/audit-report.json")
# --- 3b. Every round's session log + step timing, full-size ---------------
# log-file above is the latest round only, and capped at 8 MB by the API. Long
# runs need every round and the whole file, so: list logs/, fetch each, and when
# the API copy is shorter than the node says the file is, scp the original.
LOGS_N=0; LOGS_SSH=0; LOGS_SHORT=""
NODE=$(jq -r '.executorNodeId // empty' "$OUT/meta.json"); NODE_ALIAS="${NODE%%.*}"
WS=$(jq -r '.workspacePath // empty' "$OUT/meta.json")
if fetch_to "/api/$KIND/$ID/logs" "$OUT/logs.json" >/dev/null && [ -s "$OUT/logs.json" ]; then
mkdir -p "$OUT/logs"
while IFS=$'\t' read -r name size; do
[ -n "$name" ] || continue
fetch_to "/api/$KIND/$ID/logs/$name?lines=$LINES" "$OUT/logs/$name" >/dev/null
LOGS_N=$((LOGS_N + 1))
got=$(wc -c < "$OUT/logs/$name" 2>/dev/null || echo 0)
if [ "${size%.*}" -gt $((got + 1024)) ]; then
if [ "$NO_SSH" = "0" ] && [ -n "$NODE_ALIAS" ] && [ -n "$WS" ] \
&& scp -q -o BatchMode=yes -o ConnectTimeout=8 "$NODE_ALIAS:$WS/logs/$name" "$OUT/logs/$name" 2>/dev/null; then
LOGS_SSH=$((LOGS_SSH + 1))
else
LOGS_SHORT="$LOGS_SHORT $name"
fi
fi
done < <(jq -r '.files[]? | select(.kind == "session" or .kind == "timing" or .kind == "runner" or .kind == "critic") | [.name, (.size|tostring)] | @tsv' "$OUT/logs.json" 2>/dev/null)
fi
# readable report is JSON-wrapped HTML — unwrap so it can be read/opened directly.
if [ -s "$OUT/readable-report.json" ]; then
jq -r '.html // empty' "$OUT/readable-report.json" > "$OUT/readable-report.html" 2>/dev/null
[ -s "$OUT/readable-report.html" ] || rm -f "$OUT/readable-report.html"
fi
# --- 4. Optional: pull evidence images so they can be looked at ------------
EVID_N=0
if [ "$WITH_EVIDENCE" = "1" ] && [ -s "$OUT/evidence.json" ]; then
mkdir -p "$OUT/evidence"
while IFS=$'\t' read -r name url; do
[ -n "$url" ] || continue
curl -sS -m 120 -o "$OUT/evidence/$name" "$url" 2>/dev/null && EVID_N=$((EVID_N + 1))
done < <(jq -r '.files[]? | select(.isImage == true) | [.name, .url] | @tsv' "$OUT/evidence.json" 2>/dev/null | head -40)
fi
# --- 5. Report what we got -------------------------------------------------
{
echo "kind : $KIND"
echo "id : $ID"
echo "title : $(jq -r '.title // "?"' "$OUT/meta.json")"
echo "status : $(jq -r '.status // "?"' "$OUT/meta.json") attempts=$(jq -r '.attempts // 0' "$OUT/meta.json") platform=$(jq -r '.platform // "?"' "$OUT/meta.json")"
echo "node : $(jq -r '.executorNodeId // "-"' "$OUT/meta.json")"
echo "branch/MR : $(jq -r '.branchName // "-"' "$OUT/meta.json") | $(jq -r '.mrUrl // "-"' "$OUT/meta.json")"
echo "failure : $(jq -r '(.failureReason // "-") | gsub("\n"; " ⏎ ") | .[0:400]' "$OUT/meta.json")"
echo
echo "artifacts (http codes; 404 = that artifact does not exist for this id):"
echo " $REPORT_FILE=$c_report session.jsonl=$c_sess eval.jsonl=$c_eval evidence.json=$c_evid asking.json=$c_ask readable=$c_read audit-report.json=$c_audit"
echo " logs/: $LOGS_N files (per round)$( [ "$LOGS_SSH" -gt 0 ] && echo ", $LOGS_SSH pulled full-size from $NODE_ALIAS via scp" )"
[ -n "$LOGS_SHORT" ] && echo " ⚠ still truncated at the API's 8 MB (node unreachable or --no-ssh):$LOGS_SHORT"
[ "$WITH_EVIDENCE" = "1" ] && echo " evidence images downloaded: $EVID_N"
echo
echo "OUT=$OUT"
echo "KIND=$KIND"
} | tee "$OUT/fetch-summary.txt"
@@ -0,0 +1,19 @@
{
"_comment": "把下面两块合并进 .claude/settings.local.json。Claude 无法自己写这个文件——改权限配置是分类器的硬边界,必须你手动粘。permissions.allow 免掉每次的确认弹窗;autoMode.allow 是 auto 模式分类器的规则(它独立于 allow 列表,之前 `bun run scripts/xxx.ts` 明明匹配 `Bash(bun run *)` 却仍被拦,就是它拦的),$defaults 表示继承内置规则。",
"permissions": {
"allow": [
"Bash(bash .claude/skills/streamline-retro/fetch.sh:*)",
"Bash(bun .claude/skills/streamline-retro/digest.ts:*)",
"Bash(curl -sS -m 60 https://streamline-internal-api.bytedance.net/api/:*)"
]
},
"autoMode": {
"allow": [
"$defaults",
"Read-only retrospection of Streamline tasks: GET requests to https://streamline-internal-api.bytedance.net/api/** (no auth header, no mutation), and the two scripts under .claude/skills/streamline-retro/ (fetch.sh downloads task artifacts over HTTP; digest.ts reformats a local log file). Both write only inside the directory passed as --out.",
"Read-only SELECT queries against the Streamline Postgres through scripts under scripts/ that import control-server/src/db_client. INSERT/UPDATE/DELETE/DDL against that database is NOT covered by this rule and must still be blocked."
]
}
}
+91
View File
@@ -0,0 +1,91 @@
#!/usr/bin/env python3
"""
streamline-retro / stats.py — 一轮(或几段接力)会话的成本与效率画像。
python3 stats.py <session-rN.log> [<session-rN.timing.jsonl>]
输出:模型、每段会话的 API 调用数与上下文起止 / 中位 / 峰值、token 用量(新输入 / 写缓存 / 读缓存 / 输出)
与按标价的估算花费、工具结果体量按类别的占比(截图占多少)、工具执行 vs 思考时长、命令里写死的 sleep。
为什么有它:2026-10 分析 UI 走查和 bug / Feature 的成本时,同一套口径手算了四遍。几个结论都是从这些数里来的:
读缓存(= 每一步重读整段上下文)是成本大头;上下文 1M 窗口下从不压缩;截图常占工具输出一半;
节点 CLI 版本决定默认模型。会话日志要完整的那份(fetch.sh 会从节点 scp,API 那份截在 8 MB)。
价格是 2026-10 的标价(USD / 百万 token),只用于估算;内部结算可能不同。Opus 5 的读缓存价未单列,
按常规的输入价 1/10 假定。
"""
import json, re, sys, collections
PRICE = { # in, out, cache_read, cache_write(5m)
"claude-opus-5-5": (4, 20, 0.20, 5.0),
"claude-opus-5": (5, 25, 0.50, 6.25),
"claude-opus-4-8": (5, 25, 0.50, 6.25),
"claude-sonnet-5-5": (2, 10, 0.20, 2.5),
"claude-fable-5-1": (10, 50, 0.25, 12.5),
}
def main(log, timing=None):
seg = 0; seen = set(); calls = collections.defaultdict(list); models = collections.Counter()
by_id = {}; res = collections.Counter(); usage = collections.Counter(); n_img = 0; compactions = 0
for line in open(log, errors="ignore"):
try: d = json.loads(line)
except Exception: continue
t = d.get("type")
if t == "system" and d.get("subtype") == "init": seg += 1
if t == "system" and d.get("subtype") == "compact_boundary": compactions += 1
if t == "result":
for k, v in (d.get("usage") or {}).items():
if isinstance(v, int): usage[k] += v
if t == "assistant":
m = d.get("message", {}); models[m.get("model")] += 1
for c in m.get("content", []):
if c.get("type") == "tool_use":
a = json.dumps(c.get("input", {}))[:400]; k = c["name"]
if k == "Bash":
k = ("Bash:build/wait" if re.search(r"gradle|xcodebuild|airbuild|wait-for|hotswap|deploy\.sh", a)
else "Bash:git" if re.search(r"\bgit ", a) else "Bash:frida" if "frida" in a
else "Bash:device" if re.search(r"adb |catuse|record-evidence|uiautomator", a) else "Bash:other")
elif k == "Read" and re.search(r"\.(png|jpe?g)", a): k = "Read:image"
elif k.startswith("mcp__"): k = "mcp:" + k.split("__")[-1]
by_id[c["id"]] = k
if m.get("id") in seen or d.get("parent_tool_use_id"): continue
seen.add(m.get("id")); u = m.get("usage") or {}
calls[seg].append((u.get("input_tokens") or 0) + (u.get("cache_read_input_tokens") or 0) + (u.get("cache_creation_input_tokens") or 0))
if t == "user" and isinstance(d.get("message", {}).get("content"), list):
for c in d["message"]["content"]:
if c.get("type") != "tool_result": continue
k = by_id.get(c.get("tool_use_id"), "?"); cont = c.get("content"); n = 0
if isinstance(cont, list):
for x in cont:
if x.get("type") == "image": n += 1600; n_img += 1
else: n += len(x.get("text", "")) // 3
else: n = len(str(cont)) // 3
res[k] += n
model = models.most_common(1)[0][0] if models else "?"
print(f"模型: {dict(models.most_common(3))}")
for s, cs in sorted(calls.items()):
srt = sorted(cs)
print(f"段 {s}: API 调用 {len(cs)},上下文 起 {cs[0]//1000}k → 中位 {srt[len(srt)//2]//1000}k → 末 {cs[-1]//1000}k(峰值 {max(cs)//1000}k),读入合计 {sum(cs)/1e6:.0f}M")
print(f"自动压缩: {compactions} 次")
if usage:
i, o, cr, cw = usage["input_tokens"], usage["output_tokens"], usage["cache_read_input_tokens"], usage["cache_creation_input_tokens"]
print(f"token: 新输入 {i:,} · 写缓存 {cw:,} · 读缓存 {cr:,} · 输出 {o:,}")
p = PRICE.get(model)
if p:
cost = i * p[0] / 1e6 + o * p[1] / 1e6 + cr * p[2] / 1e6 + cw * p[3] / 1e6
print(f"按 {model} 标价估算: ${cost:.1f}(读缓存 ${cr*p[2]/1e6:.1f} · 输出 ${o*p[1]/1e6:.1f} · 写缓存 ${cw*p[3]/1e6:.1f};写缓存按 5 分钟档)")
else:
print(f"(没有 {model} 的价格,不估算)")
tot = sum(res.values()) or 1
print(f"工具结果体量(估 token,共 {tot//1000}k;截图 {n_img} 张):")
for k, v in res.most_common(8): print(f" {k:<18} {v//1000:>6}k {100*v/tot:>3.0f}%")
if timing:
rows = [json.loads(l) for l in open(timing) if l.strip()]
tool = sum(r.get("ms") or 0 for r in rows) / 6e4; gap = sum(r.get("gap") or 0 for r in rows) / 6e4
sl = sum(float(m) for r in rows for m in re.findall(r"sleep (\d+(?:\.\d+)?)", r.get("a") or ""))
to = sum(1 for r in rows if (r.get("ms") or 0) >= 119_000)
print(f"时长: 工具执行 {tool:.0f} 分钟 · 思考 {gap:.0f} 分钟 · 命令里写死的 sleep {sl/60:.1f} 分钟 · 撞 120s 超时 {to} 次")
if __name__ == "__main__":
if len(sys.argv) < 2: sys.exit(__doc__)
main(sys.argv[1], sys.argv[2] if len(sys.argv) > 2 else None)