Initial import of streamline
This commit is contained in:
@@ -0,0 +1,55 @@
|
||||
# Eddy client SSO
|
||||
|
||||
Ordinary employees can connect Eddy from `/eddy/connect` after signing in to Streamline.
|
||||
The browser forwards its existing InspireCloud session to `/api/eddy/auth/authorize`.
|
||||
The server verifies `getUserInfo`; neither the client email header nor admin membership is used to authorize knowledge contribution.
|
||||
|
||||
The CLI owns a temporary IPv4 loopback callback, random state, and PKCE S256 verifier.
|
||||
The server returns a 2-minute, one-use authorization code bound to the callback URI and challenge.
|
||||
`POST /auth/token` atomically consumes the matching grant and issues a random, revocable 30-day session.
|
||||
Only hashes of grants and sessions are stored. Expired entries are pruned during authorization.
|
||||
`GET /auth/session` checks the session; `POST /auth/logout` revokes it.
|
||||
|
||||
The historical session scope is `eddy:knowledge:write` (kept for existing clients).
|
||||
It also identifies the person for private conversation reads/writes under `/api/eddy/conversations`,
|
||||
using the full SSO email with no admin/API-key bypass; see [conversation sync](conversations.md).
|
||||
For existing contributions it is accepted by `POST /api/eddy/knowledge`
|
||||
and session status/logout. Product usage submission `POST /api/eddy/usage` also uses this session;
|
||||
aggregated monitoring is browser-admin-only. Even a session issued to an administrator does not carry admin rights.
|
||||
Direct browser SSO and legacy API keys retain their existing behavior. Release tokens and TOS publishing are unchanged.
|
||||
Knowledge author attribution matches direct browser SSO (normalized email prefix), so either path can revise the same record.
|
||||
Historical records attributed to `api-key:opk_…` retain that owner; an SSO login does not silently claim them.
|
||||
|
||||
The client stores sessions per API base URL in a 0700 directory with 0600 files and atomic, locked writes.
|
||||
Only status/email/expiry reach Portal and CLI output. Browser cookies and SSO session tokens never pass to the CLI.
|
||||
The browser callback carries only the code and state, suppresses logging/referrers and removes the query from browser history.
|
||||
The client does not follow API redirects while exchanging a code or using its contributor session.
|
||||
|
||||
## Deployment
|
||||
|
||||
1. Apply `drizzle/0032_eddy_sso.sql` through the normal approved migration flow before deploying the server.
|
||||
It only adds two tables and their expiry indexes; there is no migration of existing knowledge or permissions.
|
||||
2. Deploy both control-server and control-web from this change. `/eddy/connect` is available to every SSO user, outside admin UI.
|
||||
No new environment secret or SSO provider callback registration is needed: the existing Streamline web login handles SSO.
|
||||
3. Publish an Eddy release containing the CLI, network client, Portal source and rebuilt Portal assets from the matching change.
|
||||
4. On the corporate network, test `eddy auth login`, ordinary-user contribution, logout and subsequent rejection.
|
||||
Local fixture tests verify the entire callback/HTTP/database flow but do not certify the live SSO provider or gateways.
|
||||
|
||||
For a custom deployment, configure Eddy's `streamline.web_url` alongside `streamline.base_url` so the browser and CLI use the same database-backed control plane.
|
||||
The official mapping is `https://streamline-internal-api.bytedance.net` → `https://streamline.bytedance.net`.
|
||||
|
||||
Rollback: keep the additive tables and restore the previous server/client releases; legacy API key contribution remains available.
|
||||
Existing SSO-only clients need the new control plane; rolling the server back makes their sessions unusable until it is restored.
|
||||
|
||||
## Validation
|
||||
|
||||
Use only an isolated local database through `TEST_DATABASE_URL`. Run each integration file separately because they close the pool and mock modules:
|
||||
|
||||
```sh
|
||||
bun test control-server/test/eddy_sso.integration.test.ts
|
||||
bun test control-server/test/eddy_catalog.integration.test.ts
|
||||
bun test control-server/test/eddy_handoff.integration.test.ts
|
||||
```
|
||||
|
||||
Tests cover employee identity, wrong verifier/callback, one-use concurrent exchange, expiry, revocation,
|
||||
knowledge revisions, absence of admin/release/telemetry privileges, and hash-only persistence.
|
||||
@@ -0,0 +1,47 @@
|
||||
# 兼容知识流程:分发与客户端应用
|
||||
|
||||
问题归属与症状分开记录:`source-bug`(源码,只记录)、`devicesim`(通用运行层,可启动独立源码修复任务)、`app-compat`(特定 App 构建的兼容知识)。`unknown` 保留待分类材料,`confidence=suspected` 明确假设,不能用 Catalyst 单端结果证明真机无问题。
|
||||
|
||||
Agent 蒸馏读取本任务明确关联的聊天、观察、真实脚本与回执。`eddy task chat <id> --agent codex --session-id <id>` 或 `--file <JSONL>` 建立关联;不搜集其他聊天。提交 JSON `knowledge` 包含 category/confidence/summary/cause/solution/validation/rollback;可附 `advisory` 文字和 Frida JS / launch-config。源码和 DeviceSim 类型只提交文字。
|
||||
|
||||
```sh
|
||||
./eddy compat propose 'hang:layout' --category app-compat \
|
||||
--knowledge-file knowledge.json --tier frida-guard --payload guard.js --title '修复限定场景的布局递归'
|
||||
```
|
||||
|
||||
基线下载发布与知识提交解耦,不要求 App 在官方目录上架。补丁目标包含 baseline、bundle/version/build、arm64、Catalyst platform=6、完整主程序 UUID、主程序 SHA-256,以及所 hook 的嵌入库 hash(自动收集 Core.framework,其他库缺少身份时先提交文字)。不能仅按市场版本号匹配。版本记录不可变,相同内容重交幂等,修改产生新修订。
|
||||
|
||||
## 认证和检查
|
||||
|
||||
提交共享知识与反馈需要 SSO 或已有 Streamline Open Platform API key。客户端配置 `EDDY_STREAMLINE_TOKEN` 或 installation.json 的 streamline.token;用户名 header 仅署名,不能认证。失败时本地候选保留,401 不冒充提交成功。不要把 token 写进聊天、报告或仓库。
|
||||
|
||||
Streamline 接收经过认证的上报,验证基础格式、载体文件名与声明 hash。新提交的 `app-compat` 文字建议、Frida JS 和 launch-config 保存后即分发,不等待服务端 Agent/模型检查或卫星复验。源码、DeviceSim 和待分类材料仅记录。基线下载是否上架不影响知识分发。native-dylib 暂不进入此自动应用协议,保留为待处理材料。
|
||||
|
||||
管理员可在问题行修改分类,并对旧记录点击「确认分发」,或对已分发的材料停用/恢复;操作记录 actor、时间、auto/admin 来源以及原 manifest/payload hash。修改分类不改动不可变的载体和清单;旧记录不会因升级或补分类而自动发布。改为非基线兼容分类会停用关联的已分发材料,后续上报不能覆盖管理员分类。重复提交同一载体不会重新启用已停用修订,迟到复验也不能复活它;确认分发不伪造或覆盖安全检查结果。
|
||||
|
||||
`distribution` 是目录分发决定,独立于历史 `safety`。新客户端声明 `X-Eddy-Compat-Protocol: 3`,收到绑定当前 manifest/payload 的 `eddy-distribution-v1` 回执。旧客户端收到新分发记录的 candidate 状态,避免把分发误作执行许可。较旧可用修订不会因新修订停用而回退启用。
|
||||
|
||||
客户端核对精确二进制身份、manifest 和 payload;有本机 Agent CLI 时使用用户选择的 CLI 快速安全检查(60 秒上限,按 server/manifest/payload/CLI/policy 缓存)。Codex 在临时目录只读运行,关闭 shell/apps/plugins/多 Agent/浏览器和搜索,忽略用户配置;Claude 使用空工具集与严格空 MCP。检查阻止、未知或不可用时保留为未应用并提示。未安装 CLI 时不做模型检查,仍执行身份、完整性和运行合约校验,不声称服务端已审核。知识文本不是执行工具调用的授权。
|
||||
|
||||
分发不要求旧记录已经补全可执行身份:它可以先被同步、阅读,但缺少主程序 SHA-256 等字段的旧补丁不能自动应用。控制面显示这一边界。历史模型检查仅供参考,不再阻止管理员分发;真正的客户端应用状态由安装回执反馈。
|
||||
|
||||
## 应用时机与恢复
|
||||
|
||||
安装基线后和启动前召回;Portal/启动器运行时每 5 分钟同步已安装基线。同步只准备下次启动的 active.json,不重启、重新附加或改写当前 demo。必须在窗口前安装的 guard 使用第一行 `// eddy-compat-install: bootstrap`。
|
||||
|
||||
guard 安装异常先调用 dispose 尝试清理,隔离失败的 fix@rev 并回传结果;后续启动跳过失败修订,不循环拉起 App。当前任务、产品 registry 和容器数据保留。无法可靠判定哪个补丁导致进程崩溃时不随意归因;通用物理内存上限仍独立生效。服务端停用会在下次同步影响后续启动,已经装入的补丁不会在演示中途被强制重启卸载。
|
||||
|
||||
## 本地验证
|
||||
|
||||
Python 回归使用 `PYTHONPATH=vendor/site-packages ./vendor/python/bin/python3 -m unittest discover -s tests`;JS 合约使用 `node --test tests/test_engine_compat.mjs`。`stub/control_plane.py` 保留旧协议回归用途,不代表正式服务的安全检查;完整 v2 流程在 Streamline 的隔离 PostgreSQL 集成测试验证。
|
||||
|
||||
## Streamline 运维接入
|
||||
|
||||
部署按正常迁移流程应用 `drizzle/0029_eddy_knowledge.sql` 和 `drizzle/0030_eddy_distribution.sql`(开发空库可使用 drizzle-kit push)。需要更新 control-server、control-web 和 Eddy 客户端。分发不要求在线 satellite;DeviceSim 源码修复仍需支持的节点。
|
||||
|
||||
- 服务端不再启动安全审核队列,保留 satellite 安全检查能力供独立诊断使用;它不修改目录分发状态。
|
||||
- DeviceSim 修复节点:设置 `EDDY_DEVICESIM_REPO` 为 `ugc-android/DeviceSim` 本地 Git 仓库,必须由维护者同步到要修的源码;用 `EDDY_DEVICESIM_AGENT=codex|claude` 选择 CLI,可配置 `EDDY_DEVICESIM_AGENT_BIN` 和 `EDDY_DEVICESIM_JOBS`。节点报告 `eddy-devicesim-v1`。管理台点击后,从该 checkout 的 HEAD 创建独立 worktree/branch;Codex 使用 workspace-write 沙盒,Claude 使用限定工具,保存日志、报告和 git diff。不自动注入 App、提交代码、推 MR 或部署;维护者可审查差异后按正常 DeviceSim 流程提交。节点重启显示 interrupted,保留工作区。
|
||||
- 管理台分别显示可编辑分类、真实分发状态、具体建议与载体、验证/撤销说明。历史服务端检查折叠显示,避免将模型判断混为分发状态。
|
||||
- 写 API 复用 Open Platform key;并非发布流水线的 `EDDY_RELEASE_TOKEN`,也不信任 X-Eddy-User。管理员继续用原 SSO 管理。
|
||||
|
||||
集成测试必须显式设置 `TEST_DATABASE_URL` 为隔离本地 PostgreSQL,运行 `bun test control-server/test/eddy_flow.integration.test.ts`;没有该变量时跳过。该测试明确不提供安全扫描节点,验证无模型依赖的自动分发、管理员分类/确认、只读旧载体、幂等、停止保护和客户端协议兼容。
|
||||
@@ -0,0 +1,110 @@
|
||||
# Eddy main-Agent conversation sync
|
||||
|
||||
The Eddy setting is opt-out, enabled by default; there is no separate sync confirmation.
|
||||
The client first requires the person's Streamline SSO session and a task-bound source.
|
||||
It uploads that entire source session's available visible history and new records,
|
||||
including user/assistant messages and tool calls/results, without the summary reader's text budget.
|
||||
Hidden reasoning, system/developer instructions, Claude sidechain records and binary attachments
|
||||
are excluded. Recognizable credentials are masked on the client on a best-effort basis.
|
||||
This is not a complete multimedia replay or a guarantee of arbitrary-secret detection.
|
||||
|
||||
`POST /api/eddy/conversations`: schema 1, batchId (sha256), taskId, taskTitle, sourceId (sha256),
|
||||
provider (codex/claude/file), machine (16 hex), events (1–64). Body is limited to 512 KiB.
|
||||
Each event contains id, streamId (optional hash of the log basename), position (byte offset), block,
|
||||
part (zero-based), parts, kind (message/tool_call/tool_result/attachment), role, text, redacted,
|
||||
and optional timestamp/toolName/callId. Unknown properties are discarded.
|
||||
Long text parts are reconstructed in streamId/position/block/part order; identical repeated messages
|
||||
retain different event ids. The server acknowledges `{batchId, accepted}` after its atomic insert.
|
||||
Duplicate event delivery is accepted without another row. Ownership is part of the deduplication key.
|
||||
|
||||
`GET /api/eddy/conversations?task=<taskId>&machine=<machine>&source=<sourceId>&cursor=<sequence>&limit=100`:
|
||||
returns `{events, nextCursor}`, 1–500 records per page in insertion-sequence order.
|
||||
Each envelope carries taskId/taskTitle, sourceId/provider, machine, receivedAt, sequence and event.
|
||||
Use nextCursor until null. Same main session bound to several Eddy tasks is intentionally associated
|
||||
with all those tasks; these records are not a claim of per-message task attribution.
|
||||
|
||||
## Viewing history
|
||||
|
||||
The SSO-only `/eddy/tasks` page is linked from the account menu (“我的 Eddy 任务”) and
|
||||
Management → Eddy (“我的任务与会话”). It is available to ordinary signed-in users,
|
||||
independent of the administrator-only usage/knowledge panels. Each user sees their own
|
||||
records there. Browser-SSO Streamline administrators can also read every account's history
|
||||
(internal employee data): `/eddy/tasks?scope=all` (linked from Management → Eddy as
|
||||
「全部用户任务与会话」) adds a per-account summary (tasks, sessions, devices, records, first/last
|
||||
sync), an account filter and the owner on each task; an Eddy session never grants this.
|
||||
The standalone page does not boot the task board or its background fetches.
|
||||
|
||||
- Search tasks by latest reported title or task id; same task id on two installations
|
||||
is displayed separately. Task titles reflect the latest received event, not a live
|
||||
task metadata mirror. Tasks with no uploaded conversation do not appear.
|
||||
- Select a Codex/Claude/imported source; the task, machine and source ride in the URL
|
||||
for refresh/back/forward. Opening the link still requires the same SSO owner.
|
||||
- Read available visible messages in source-file position order. Text split over upload
|
||||
or read pages is reconstructed; incomplete parts are labelled until loaded. Repeated
|
||||
messages are preserved. Tool calls/results are collapsed and all content renders as
|
||||
inert text. No remote images, HTML or attachment binaries are loaded by the viewer.
|
||||
- “任务信息与产物” shows task identifiers and the first available user message as a
|
||||
requirement reference, clearly labelled. It is not an independently maintained task
|
||||
description. Screenshots, patches and snapshots are not uploaded by conversation sync;
|
||||
handoff bundles remain accessible through the corresponding Streamline feature.
|
||||
- Read more via bounded pages or refresh explicitly. Empty data, old server (404), expired
|
||||
login (401) and retryable failures have distinct states. Private reads are `no-store`
|
||||
and are not shared by the Web API's cross-component in-flight GET cache.
|
||||
|
||||
Metadata list APIs (same SSO/owner checks and no-store):
|
||||
|
||||
`GET /api/eddy/conversations/tasks?q=<title-or-id>&before=<sequence>&limit=30`
|
||||
returns `{items, nextCursor}`. Each item contains `machine, taskId, taskTitle,
|
||||
lastReceivedAt, lastSequence, recordCount, sourceCount`; there are no transcript bodies.
|
||||
Search is literal/case-insensitive (including `%`, `_` and backslashes), at most 200 characters.
|
||||
|
||||
`GET /api/eddy/conversations/sources?task=<taskId>&machine=<machine>&before=<sequence>&limit=30`
|
||||
requires both selectors and returns `{items, nextCursor}` with `sourceId, provider,
|
||||
firstReceivedAt, lastReceivedAt, lastSequence, recordCount`. A missing or non-owned task
|
||||
returns an empty list. Both lists allow 1–100 rows per page, newest received sequence first;
|
||||
pass `nextCursor` as `before`. New uploads can move an item to the front during browsing;
|
||||
refresh the list to see these updates. Counts are stored fragments, not message counts.
|
||||
|
||||
Both methods require verified browser SSO or an active Eddy SSO session, scoped by the **full email**.
|
||||
No API-key, release-token, X-Eddy-User or administrator bypass. Responses are `Cache-Control: no-store`.
|
||||
The historical scope string stays `eddy:knowledge:write` for existing client compatibility, as for handoffs;
|
||||
these routes use the SSO identity, and do not grant admin/release rights. Nothing enters the public knowledge feed.
|
||||
|
||||
Switching off stops subsequent client reads and uploads after an in-flight short request finishes;
|
||||
it does not delete already stored data. Switching on catches up. Logged-out clients never read transcripts.
|
||||
Only acknowledged offsets/parts are stored locally. 401/404/503/offline preserve progress for retry.
|
||||
An old control plane returns 404; this does not block local Eddy use.
|
||||
|
||||
## Rollout and validation
|
||||
|
||||
Apply additive `drizzle/0065_eddy_conversations.sql` using the normal deployment flow **before** the server release,
|
||||
then release the client. There is no production schema/data mutation in local development.
|
||||
Rolling back the client stops the worker; the additive table can remain. Stop the route to stop receiving.
|
||||
|
||||
```sh
|
||||
# TEST_DATABASE_URL must be an explicitly isolated database.
|
||||
bun test control-server/test/eddy_conversations.test.ts
|
||||
TEST_DATABASE_URL=postgres://user@127.0.0.1:55483/test bun test control-server/test/eddy_conversations.integration.test.ts
|
||||
# Optional actual Eddy Python → HTTP → Postgres probe, using synthetic temporary sessions only:
|
||||
EDDY_TEST_SOURCE=/path/to/eddy-source TEST_DATABASE_URL=postgres://user@127.0.0.1:55483/test bun test control-server/test/eddy_conversations.integration.test.ts
|
||||
```
|
||||
|
||||
The integration database needs `0032_eddy_sso.sql` and `0065_eddy_conversations.sql`.
|
||||
Tests cover authentication, expiry, owner isolation, no admin bypass, retries, pagination, limits,
|
||||
Unicode/long messages, tools, reasoning exclusion, masking and default-on/off/on catch-up.
|
||||
|
||||
The optional browser integration test `control-server/test/eddy_history.browser.test.ts` seeds only
|
||||
synthetic records, starts a loopback HTTP fixture and cleans them up. Build the real page fixture with
|
||||
`bun build control-web/test/fixtures/eddy-history.tsx --target browser --outdir /tmp/eddy-history-bundle
|
||||
--define 'import.meta.env={"VITE_DEBUG":"1","DEV":true}'`, then set `TEST_DATABASE_URL` (isolated PG),
|
||||
`EDDY_HISTORY_BUNDLE=/tmp/eddy-history-bundle` and run the test. The committed driver is
|
||||
`control-web/test/eddyHistory.browser.mjs`; it requires Playwright/Chrome in the test environment.
|
||||
`EDDY_HISTORY_BROWSER_NODE` selects Node, `EDDY_HISTORY_PLAYWRIGHT_PACKAGE` can point at a dependency
|
||||
package.json from which Playwright resolves, and `EDDY_HISTORY_OUTPUT_DIR` selects the screenshots/report
|
||||
directory. The driver rejects a non-loopback fixture URL; it never needs production SSO or data.
|
||||
|
||||
Administrator APIs (browser SSO administrator only, no-store; same pagination and validation):
|
||||
`GET /api/eddy/admin/conversations/users` → `{items:[{owner, taskCount, sourceCount, machineCount,
|
||||
recordCount, firstReceivedAt, lastReceivedAt}]}`; `GET /api/eddy/admin/conversations/tasks?owner=&q=&before=&limit=`
|
||||
(owner optional; items carry `owner`); `GET /api/eddy/admin/conversations/sources?owner=&task=&machine=…` and
|
||||
`GET /api/eddy/admin/conversations?owner=&task=&machine=&source=&cursor=&limit=` (owner required).
|
||||
@@ -0,0 +1,72 @@
|
||||
# Eddy 崩溃报告(POST /api/eddy/crashes)
|
||||
|
||||
Eddy 用户的 App 在任务里崩溃时,Eddy 本机 supervisor 保存这次启动的完整现场并自动重启(重启后没有任何用户动作、任务也没变又崩,才视为循环,停下),
|
||||
按崩溃栈规则判断归属,四类里三类上报到这里,供维护者复现和接力修。**报告里是事实和规则判断,不是结论**:读的时候以证据文件为准。
|
||||
|
||||
**什么时候到达**:Eddy 当前版本(`2026.10.23-eddy-home.19` 起)在崩溃当场按规则上报,不论任务有没有改动(看 `task.edits`),
|
||||
报告固定 `classSource: "rule"`、`confidence: "suspected"`,不带用户本机 Agent 的判断(它只留在用户本机,避免较弱的本地模型推测成为修复的起点)。
|
||||
更早的 Eddy:任务没有改动时当场按初判上报(`suspected`),用户的 Agent 确认后再发同一份报告(`confirmed`、带 `note`),按 id 更新;
|
||||
这类 `note` 是用户本机 Agent 写的,只当线索。
|
||||
|
||||
| class | 含义 | Streamline 侧问题分类 |
|
||||
|---|---|---|
|
||||
| `devicesim` | DeviceSim 的问题(栈落在 `devicesim.dylib`,或与它模拟的摄像头/运动/窗口等能力有关) | `devicesim`(可直接派 DeviceSim 修复 Agent) |
|
||||
| `eddy` | Eddy 自己的问题(`runtime/app_compat`、adhoc 补丁、补丁引擎、启动配置) | `eddy`(记录,交 Eddy 维护者) |
|
||||
| `app` | App 源码问题(例如 unrecognized selector);用户 Agent 另行找绕过办法,经用户同意后用在任务里 | `source-bug` |
|
||||
|
||||
第四类 `task-edit`(用户在任务里要求的改动没改好)由 Agent 在本机继续修,不上报;服务端也拒收。
|
||||
客户端契约与 `crash.json` 字段见 `streamline_playground/API.md`「崩溃报告」,Agent 规则见该仓 `skills/eddy-frida/SKILL.md`「App 崩溃时」。
|
||||
|
||||
## 存储(不加表、无迁移)
|
||||
|
||||
- `eddy_observations`:一份报告一行,`id` = 客户端崩溃 id(`crash-YYYYmmddTHHMMSS-<8hex>`),
|
||||
`payload` = `{kind:"crash-report", report}`;`user` 为会话/SSO 身份,`machine` 为客户端硬件 UUID 的 sha256 前 16 位。
|
||||
同一 id 再次上报:同一上报人(且 baseline、signature 不变)更新 `payload`,返回 `{duplicate: true, updated: true}`;别人的重发忽略。
|
||||
- `eddy_problems`:和其他观察一样按 `(signature, baseline)` 归并(含 alias)。问题**未分类**或分类**来自崩溃报告**
|
||||
(`knowledge.source = "crash-report"`)时,写入报告的分类与可信度;suspected 不会覆盖 confirmed;
|
||||
管理员分类或修法提交带来的分类(没有这个 source)不会被覆盖。
|
||||
- `eddy_knowledge_assets`:证据文件按 sha256 存(base64)。只经下面的管理员路由下载,且必须是该报告列出的文件。
|
||||
- DeviceSim 修复(`POST /admin/problems/devicesim`)派发时,问题数据里附上该问题最近 3 份报告(文本,无附件)。
|
||||
|
||||
## 上传内容与隐私
|
||||
|
||||
报告本身:规则判断的 class 与依据(`reasons`)、signature、异常名与原因、崩溃栈前 12 帧、App bundle/版本/build/UUID、Eddy/DeviceSim/macOS 版本、
|
||||
启动环境**变量名**(只有 `DEVICESIM_*` 带值)、启动参数**个数**、任务改动数与 key 列表(不含脚本内容)、重启决定;
|
||||
`localEvidence` 列出留在用户本机、没有上传的文件(名字、大小、sha256),需要时找上报人要。
|
||||
证据文件(单文件 ≤ 4 MB、合计 ≤ 8 MB):
|
||||
|
||||
| 文件 | 内容 |
|
||||
|---|---|
|
||||
| `report.ips` | macOS 崩溃报告(Apple 已把用户目录写成 `/Users/USER`) |
|
||||
| `exception.txt` | 统一日志里该 PID 的 uncaught exception 行(.ips 常常只有 abort()) |
|
||||
| `devicesim.log` | 这次启动期间 `/tmp/devicesim.log` 写入的全部内容;本机所有 DeviceSim App 共用,可能混有其他实例的行(按 `identity: registered as` 区分)。更早的 Eddy 是尾部 200 行 |
|
||||
| `events.jsonl` | 崩溃那次启动的 supervisor 事件 |
|
||||
| `unified.log.gz` | 这次启动 App 的统一日志(`log show --info`,最多回溯 1 小时),gzip |
|
||||
| `app.log.gz` | App 自己这次启动的输出(stdout/stderr),gzip |
|
||||
| `supervisor.log.gz` | Eddy supervisor 这次启动的日志,gzip |
|
||||
|
||||
`.gz` 文件项带 `of`(原文件名);单个日志压缩后仍超过 4 MB 时只保留结尾,带 `omittedBytes`,解压后第一行也注明。
|
||||
三份日志按上表顺序放进 8 MB 总量,放不下的进 `localEvidence`。文本中的 home 路径已替换为 `~`。
|
||||
App 日志可能含账号等信息,用户可在 Eddy 里关闭自动上报。任务脚本(`registry.json`)、启动参数值和其他环境变量值(`launch.json`)、截图、聊天不上传。
|
||||
|
||||
## 鉴权
|
||||
|
||||
`POST /crashes` 与知识贡献相同:Eddy 会话(`./eddy auth login`)、浏览器 SSO 或 Open Platform API key;只有 `X-Eddy-User` 时 401,
|
||||
客户端保留 pending,登录后 `./eddy crash submit <id>` 重试。读取全部走管理员(SSO + admin):
|
||||
|
||||
- `GET /api/eddy/admin/crashes?class=&baseline=&signature=&limit=` — 列表(最新在前)
|
||||
- `GET /api/eddy/admin/crashes/:id` — 完整报告 + 每个文件的下载地址
|
||||
- `GET /api/eddy/admin/crashes/:id/files/:sha256` — 证据文件
|
||||
|
||||
## 维护者复现
|
||||
|
||||
1. 按 class 拉列表,打开报告,看 `exception`、`stack`、`reasons`(规则依据),再读证据文件;旧报告的 `note` 是用户本机 Agent 写的,只当线索。
|
||||
2. 基线:`report.baseline` 对应 `eddy_releases` 的官方包(`./eddy baseline add <id>`);`app.executableUUID` 核对是同一份构建。
|
||||
3. DeviceSim:`versions.deviceSim`(mode/version/sha256)选同一版本;用 `launch.devicesim` 里的 `DEVICESIM_*` 启动。
|
||||
4. Eddy:`versions.eddy` 是 runtime 版本(`dev` = 源码检出);`task.appCompatibility` / `task.adhocPatches` / `task.engine` 说明当时注入了什么。
|
||||
5. 下载 `report.ips` 用 Console 或 `atos` 看完整栈;`unified.log.gz` 里常有 .ips 没有的原因(异常、断言、系统框架报错);
|
||||
`events.jsonl` 是那次启动的时间线(attach、重放、compat、hang);`devicesim.log` 可能混有同机其他实例的行。
|
||||
|
||||
## 测试
|
||||
|
||||
`bun test control-server/test/eddy_crashes.test.ts`(校验);`eddy_crashes.integration.test.ts` 需要 `TEST_DATABASE_URL`(上传、上报人更新、他人重发忽略、分类随报告更新但不覆盖管理员分类、管理员读取)。
|
||||
@@ -0,0 +1,99 @@
|
||||
# Feature 导出到 Eddy
|
||||
|
||||
在任务详情的「接力到本地」中选择 Claude Code / Cursor 的 Markdown 文档,或 iOS feature 专属的「准备 Eddy 接力包」。Eddy 由任务卫星机(离线时由同集群的其他 iOS 卫星机)生成临时 App 基线,列表和详情展示进度,成功后飞书通知。用户将 App ZIP 和源码 ZIP 放到同一目录,在 Eddy「基线」安装 App ZIP,再把 RELAY.md 交给本地 Agent。Android 保留普通 Markdown 接力,不显示 Eddy 选项;接口和卫星机解析同样拒绝 Android。
|
||||
|
||||
这是 feature 的独立导出作业,不改变 feature 状态、不合并 MR、不发布到 Eddy 官方基线目录。首版支持 `manual_review`、`completed` 且有已推送 MR 分支、已知 `repoKey` 的 iOS feature。
|
||||
|
||||
## 执行和交付约定
|
||||
|
||||
1. 控制面认证点击者,选定执行导出的卫星机(见下文「哪台卫星机导出」),复用普通接力的报告、反馈历史和执行摘要。共用起点规则:人工复核用任务分支,确认已合入用配置的基线,合入记录未知时先任务分支、明确不存在才退回基线;网络失败不触发回退。Feature 的合入状态通过匹配仓库/源分支的 Bits 只读查询补充。卫星机把所选远端起点解析为完整 SHA,文档记录实际分支和提交。
|
||||
2. 服务端持久化提交、构建说明 hash、任务/MR 和完整接力文档快照。相同 feature、提交、分支、构建说明及上下文的有效作业复用;每个请求者各有通知记录。反馈/报告变化时新建快照,不把旧上下文当作刚生成的接力文档。
|
||||
3. 卫星机在串行 implementation scheduler 内领取作业,先获取与 Asking 共用的 checkout 锁。导出占用节点,不与下一次实现、Asking 代码修改或 Eddy compat 验证并行;控制面热部署等待该作业结束。控制面把构建到上传这几个阶段的导出计入该节点的执行槽(`countSlotsPerNode`),所以节点卡片显示「执行 1/1」和「Eddy 导出 · <任务标题>」(`GET /api/eddy-exports/running`,点开进入该 feature),调度台和导出改派也不会把它当成空闲节点。
|
||||
4. 在独立目录 clone 产品仓,在分支名上检出固定 SHA(`git checkout -B <分支> <提交>`,源码包和接力文档从这里出),再把节点的共享检出(`$AI_WORKSPACE/TikTok`)固定到同一提交(先用验证任务同一个 `checkoutChangeCommit` fetch 分支、`checkout -f --detach`、`clean -fd`,再 `checkout -B <分支> <提交>` 放到分支名上——`airb sync` 按分支名取提交,detached 的需求分支提交远端取不到(`Commit not exist in codebase`);本地分支若有会被丢掉的提交,先存到 `refs/eddy-export/backup/`),并从官方 `eddy/LATEST` 获取 runtime(验证发布 ZIP hash 和分发 MANIFEST;只复制分发清单文件,不复制卫星机 Eddy 的账号、任务、已安装 App)。
|
||||
5. **构建由智能体按 DeviceSim 的 `Skill(airbuild)` 完成**——和该节点修复/验证任务出包是同一套做法,构建细节只维护在 skill 里(「工程专用」的 TikTok 与 TikTokNotes / Whee 两节)。智能体在共享检出上用它已绑定的 airspace 构建(和修复/验证任务一样;**不 `airb init`、不 `airb release`**——每人只有 2 个空间,原先每次导出新开一个、失败的还保留,配额满后导出全部在 init 失败),把产物 `airb pull` 到作业的 `build/`,并写回执 `airbuild-receipt.json`。随后 worker 只认:独立 clone 与共享检出的 HEAD 都不变且无改动/新文件、`build/` 里恰好是声明的那个 `.app`、回执的产品与提交与作业一致。回执是构建声明,作为 `BUILD.json` 进包;源码包始终按固定提交切。
|
||||
6. 打包直接调用 Eddy 的 `portable/build_app_baseline.build`(runtime `2026.10.15-eddy-home.12` 起支持 `extend`):先在作业目录的 `DEVICESIM_ROOT` 里装好 DeviceSim(runtime `2026.10.23-eddy-home.18` 及以前用包里随附的;之后的 runtime 不带 DeviceSim,由 Eddy 从 DeviceSim 官方渠道下载最新 stable,所以卫星机要能访问办公网下载入口),用它经 cat-supervisor 完成转换、注入和签名,切源码包、写基线;导出只通过 `extend` 加入任务来源、`BUILD.json`、`RELAY.md`、`INSTALL.md`,产物在作业目录的 `package/artifacts/`。冒烟后由 Eddy 的 `devicesim_install.stop_capture_daemon()` 停掉从作业 DeviceSim 拉起的截图 daemon。`DEVICESIM_ROOT`、`EDDY_STATE_DIR` 和 cat-supervisor 的实例目录都落在作业目录里,不切换、不触碰节点自己的 `/Users/Shared/DeviceSim` 与 `~/iosapp`。
|
||||
7. 在作业自己的 runtime / state 下安装 ZIP、DeviceSim 选 `installed`(即上一步在作业目录里启用的那份),启动 App 并运行 Eddy smoke。要求原生连接 active、无产品编辑、无 Frida engine attach、产出有效 PNG。此检查证明基线可启动、可截图,不等于 feature 的业务验收。
|
||||
8. 两份 ZIP 经 SHA-256 和长度检查后,由卫星机直传 TOS。服务端仅签发该作业对象的 PUT 请求,不给卫星机 AK/SK:签的是 **TOS 临时签名**,地址是桶的**内网子域名** `https://<bucket>.tos-cn-north.byted.org/<key>`,2 小时内有效,每次重试重新签发。不能用图片上传那套 S3 签名:卫星机在办公网,办公网 DNS 只解析内网子域名,`tos-s3-cn-north.byted.org`(任何形式)和裸 `tos-cn-north.byted.org` 都解析不了。临时签名是 TOS 文档(「TOS 临时签名」「不同环境如何访问 TOS」)给无 Consul 环境的上传方式,HMAC 的 key 是桶的 access key,签的是方法、路径、签名名称和过期时间,不含内容摘要,所以字节的正确性由卫星机上传前的预检和控制面随后的回读保证。控制面从用户下载地址流式回读两份文件并校验;完成请求立即返回 202,校验在后台运行,卫星机继续心跳及查询。
|
||||
9. 只有回读成功才标记 ready、开始 7 天有效期并发送飞书卡片。卡片链接回任务详情,避免保存一个即将过期的直接下载地址。
|
||||
|
||||
产物:
|
||||
|
||||
- `baseline-<baselineId>.zip`:App、DeviceSim、baseline.json、MANIFEST.json、安装说明及 feature 上下文。
|
||||
- `baseline-<baselineId>-source.zip`:Eddy 标准源码快照,严格对应同一提交。包含受支持的 tracked 文本文件,不是完整 Git 仓或可直接重编译的全量依赖副本。
|
||||
- 卫星机保留 `build.log`、`result.json`、`verification.json`、`smoke.png` 等诊断材料。作业结束清理源码检出、runtime 副本及大型中间产物;若原生停止失败则保留 runtime/state 供排查。
|
||||
|
||||
## 配置与部署
|
||||
|
||||
默认 `EDDY_EXPORT_ENABLED=0`。开启前须部署控制面、shared 和卫星机代码,按仓库正常发布流程应用 `drizzle/0033_eddy_feature_exports.sql`。该迁移仅新增两张表和索引,可重复执行;回滚代码时关闭开关即可保留历史记录。
|
||||
|
||||
控制面配置:
|
||||
|
||||
| 配置 | 用途 |
|
||||
| --- | --- |
|
||||
| `EDDY_EXPORT_ENABLED=1` | 启用创建、下载、节点执行及维护 |
|
||||
| `CLUSTER_SECRET` | 可选,与集群其余接口一致:配置了就校验,控制面和卫星机须相同;留空则不校验 |
|
||||
| `TOS_ACCESS_KEY`、`TOS_SECRET_KEY`、`TOS_REGION`、`TOS_S3_ENDPOINT`、`TOS_BUCKET` | 导出与图片上传共用同一套配置和 bucket;导出上传用 `TOS_ACCESS_KEY` 签临时签名 |
|
||||
| `EDDY_EXPORT_TOS_DOMAIN` | 可选,导出上传用的内网子域名根,默认 `tos-cn-north.byted.org`(桶在其他区域时改成对应的内网子域名) |
|
||||
| `APP_BASE_URL` | 飞书卡片返回任务详情的地址 |
|
||||
| 现有 Lark 机器人凭据 | 就绪通知;未配置时产物仍可下载,通知保持待发 |
|
||||
|
||||
对象固定落在 `eddy/task-exports/<exportId>/<sha256>/<filename>`,下载地址由 `shared/src/protocol.ts` 的 `TOS_DOWNLOAD_BASE` 拼出,与图片上传同源。在 TOS 为该前缀配置 **8 天生命周期**,覆盖上传至校验完成的时间和随后的 7 天下载窗口。API 的过期检查不等于物理删除,已经拿到的直接 URL 在对象删除前仍可能有效。失败产生的对象也由同一生命周期清理。首版单文件上限 4 GiB,使用单次 PUT,不做分片上传。
|
||||
|
||||
卫星机需要已解锁的 Apple Silicon macOS 15+ 会话、Xcode 工具、产品仓读取权限、到控制面的网络、能解析并访问 TOS 内网子域名 `<bucket>.tos-cn-north.byted.org`(办公网默认可以),并有足够容纳源码和数份 App 副本的磁盘空间。
|
||||
|
||||
**卫星机不需要任何导出专用配置。** 只要具备修复/验证任务出包本来就用的两样——PATH 上的 `airb`(且已登录 Airbuild)和 DeviceSim 的 airbuild skill(`<AI_WORKSPACE>/.claude/skills/airbuild` 或 `~/.claude/skills/airbuild`)——就会领取导出。缺任意一样时,「准备 Eddy 接力包」会直接报出缺的是什么。
|
||||
|
||||
接力对话框的导出记录会写明由哪台卫星机执行;排队时说明在等什么(卫星机正忙、已离线、刚等待领取),改派过的会注明接力文档里没有修复报告。
|
||||
|
||||
**哪台卫星机导出。** 任务卫星机在线就由它导出,它报的错(缺 airb、查不到远端分支等)原样返回。离线时改派:导出只用远端已推送的提交、独立 clone、Airbuild 远端构建和官方 runtime,不读执行节点磁盘上的任何东西,唯一拿不到的是修复报告(接力文档里会标注未能获取)。候选是在线、iOS、集群标签与该 feature 相交、不是实验机(`manual-only`)的节点,空闲的排前面;控制面同时请它们解析起点,选排名最靠前且解析成功的那台,作业从此只派给它。都不行时报错里列出各节点的原因。改派占用的是那台卫星机的时间和它共享检出已绑定的 Airbuild 空间。
|
||||
|
||||
**构建什么由任务决定。** `product` 取自 feature 的 product profile id(无 profile 时回落成 `tiktok`),在 `shared/src/eddy_export.ts` 的 `EDDY_EXPORT_PRODUCTS` 里查到 Eddy 目标名、产物名和启动参数;**怎么构建**只写在 DeviceSim airbuild skill 的「工程专用」里,Streamline 不再维护任何构建脚本或节点配方。目前支持:
|
||||
|
||||
| product | skill 一节 | 产物 | Eddy key | 启动参数 |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| `tiktok` | TikTok | `TikTokM.app` | `tiktokm` | `-skipForceLogin` |
|
||||
| `notes` | TikTokNotes | `TikTokNotes.app` | `notes` | 无 |
|
||||
| `whee` | Whee | `Whee.app` | `whee` | `-skipForceLogin` |
|
||||
|
||||
`lemon8` 等表里没有的 product 会干净报错「该产品还不支持导出到 Eddy」,不会误编成别的 App。新增产品 = 先在 skill 里写清它的构建,再在表里加一行。
|
||||
|
||||
- `EDDY_EXPORT_ROOT` 默认 `~/.streamline/eddy-exports`;`EDDY_EXPORT_PYTHON` 默认系统 Python。官方 runtime 包缓存固定在 `~/.cache/streamline/eddy`。
|
||||
- Eddy 更新不会覆盖节点的现有安装。每次作业从最新官方分发生成隔离 runtime,并将实际版本留在回执中。
|
||||
|
||||
## 重试与排查
|
||||
|
||||
状态为 `queued → building → packaging → verifying → uploading → ready`,中途可失败。执行租约为 120 秒,卫星机每 20 秒续租;失联或进程重启后作业失败,旧 lease 的回报不能覆盖新状态。排队中的作业只派给选定的那台卫星机:它超过 10 分钟没有心跳,作业标为失败,「导出当前提交」会重新选卫星机(短暂离线如部署重启不受影响);排队超过 24 小时同样失败。失败或过期时点「导出当前提交」,会重新解析当时的分支 SHA 并新建作业。
|
||||
|
||||
断线后的完成确认可以重复请求;已经 ready 的作业直接返回既有结果。上传清单固定后不能换文件内容。飞书通知单独重试,常规重复点击不重复通知;极端情况下发出成功、落库前进程崩溃,通知可能重发一次。
|
||||
|
||||
日志在 `<EDDY_EXPORT_ROOT>/<exportId>/`:`build.log`(worker 各阶段)、`build-agent.log`(构建智能体的会话流)、`airbuild-receipt.json`(构建回执,失败时带 `error`)。UI 错误包含 exportId,构建失败时带回执里的原因。构建、签名、原生 smoke 或校验失败时不发布下载入口,原 feature 的实现结果保持原样。
|
||||
|
||||
导出不开也不释放 Airspace:构建用共享检出已绑定的那个,失败后它照常留给下一个修复/验证任务;共享检出停在导出的提交上(分离 HEAD),下一个任务会检出自己的分支。
|
||||
|
||||
## 验证
|
||||
|
||||
```bash
|
||||
bun run typecheck
|
||||
bun run build
|
||||
bun run build:satellite
|
||||
bun test control-server/test/eddy_export.test.ts
|
||||
python3 -m unittest discover -s satellite/eddy-export -v
|
||||
```
|
||||
|
||||
数据库集成测试只接受显式提供的本机 `*_test` 数据库;先在可丢弃数据库加载 schema,再运行:
|
||||
|
||||
```bash
|
||||
TEST_DATABASE_URL=postgres://test@127.0.0.1:5432/streamline_test \
|
||||
bun test control-server/test/eddy_export.integration.test.ts
|
||||
```
|
||||
|
||||
集成测试使用真实 PostgreSQL,拦截全部 SSO、卫星机和对象下载网络请求,覆盖鉴权、并发幂等、任务变化、固定提交、阶段顺序、过期租约、错误产物、异步完成确认、通知预约、过期重建;不发真实飞书消息。
|
||||
|
||||
可选真实工具链测试:编译一个最小扁平 Catalyst App 代替构建智能体,经该 runtime 自带的 cat-supervisor 打包,并从本地 ZIP 安装 App 与源码;不启动原生 App。`EDDY_EXPORT_TEST_RUNTIME` 指向解压后的官方 runtime(含 `MANIFEST.json` 的目录):
|
||||
|
||||
```bash
|
||||
EDDY_EXPORT_TEST_RUNTIME=/path/to/official/eddy-runtime \
|
||||
python3 -m unittest discover -s satellite/eddy-export -v
|
||||
```
|
||||
|
||||
UI 的本地示例在 `/test/fixtures/eddy-export.html`,可检查初始、构建中、可下载、失败、过期和未启用状态。上线前还需在配置好的卫星机上对真实 feature 跑一次完整构建、原生启动、TOS 上传/回读和客户端下载安装。
|
||||
@@ -0,0 +1,55 @@
|
||||
# Eddy 摄取容量、限速和正文去重
|
||||
|
||||
## 范围与默认值
|
||||
|
||||
单次请求的512 KiB/64事件(对话)、128 KiB/100事件(使用行为)限制继续保留。
|
||||
新增服务端配额,不接受客户端传入配额或身份。SSO完整邮箱派生SHA256预算标识,同一用户所有设备共用额度;该标识只用于预算,不进入产品指标响应。
|
||||
使用行为 schema 2 允许不带凭据上传(只收正式环境、一批只能来自一台设备)。**所有未登录上传共用一份额度**(预算标识固定为 `anonymous` 的 SHA256),不按客户端声明的设备标识分:设备标识可以随便填,按它分额度会让每个请求都换一份新额度、新增一行预算记录,进而把全局额度耗尽。这份共用额度单独可配(`EDDY_USAGE_ANONYMOUS_BYTES/_ROWS/_DAILY_BYTES/_DAILY_ROWS/_RPM/_BURST`,默认 1 GiB / 400 万行 / 每天 64 MiB、40 万行 / 每分钟 300 批、突发 16 批),累计和速率都不超过全局的一半,登录用户的上传始终有余量。设备标识只用于统计;未登录的数据是客户端自报的,看板把它单独标为「未登录」。
|
||||
|
||||
| 项目 | 对话默认 | 使用行为默认 |
|
||||
| --- | --- | --- |
|
||||
| 账号每日新增逻辑字节 | 256 MiB | 16 MiB |
|
||||
| 账号每日新增行数 | 250,000 | 100,000 |
|
||||
| 账号累计逻辑字节 | 2 GiB | 256 MiB |
|
||||
| 账号累计行数 | 2,000,000 | 1,000,000 |
|
||||
| 全局累计行数 | 10,000,000 | 10,000,000 |
|
||||
| 账号速率 / 突发 | 60批/分钟 / 8批 | 同左 |
|
||||
| 全局速率 / 突发 | 600批/分钟 / 32批 | 同左 |
|
||||
| 原始记录保留 | 接收后90天 | 接收后90天 |
|
||||
|
||||
`EDDY_CONVERSATION_GLOBAL_BYTES` 和 `EDDY_USAGE_GLOBAL_BYTES` **必须显式配置正整数**。未配置、为0或非法时,摄取503拒绝,绝不默认为无限。`.env.example` 中的全局数字只是语法示例,须先由部署人员核对DB可用容量、索引/WAL/副本余量并填写预算。代码未查询生产磁盘余量,未实现物理磁盘水位自动熔断,不能把逻辑字节预算当数据库物理磁盘上限。
|
||||
|
||||
配置前缀为 `EDDY_CONVERSATION_` / `EDDY_USAGE_`;其余后缀见 `.env.example`。额度按UTC天计算。修改配置、重启生效,不需改schema;降低至已有用量之下只会暂停新增,不删除旧数据。保留期可由 `EDDY_CONVERSATION_RETENTION_DAYS` 调整。
|
||||
|
||||
## 去重和兼容
|
||||
|
||||
`eddy_conversation_records` 只保存一次正文,键为SHA256(完整SSO用户、设备、来源、事件id)。不跨用户合并,不按文本值去重;同样文字出现在两个不同事件里仍完整保留。
|
||||
原 `eddy_conversation_events` 保留每个任务的轻量索引、sequence、标题和来源,正文改为record_id引用。维持现有逐任务分页、不同设备隔离及任务/来源列表;**任务索引仍按事件存在,去重降低正文空间,不承诺降低总行数**。配额同时计入规范化UTF-8正文和任务元数据、两张表的新增行数。PG行头、索引、WAL、副本、物理膨胀不计入逻辑字节。
|
||||
|
||||
新客户端可在schema1请求里附带至多32个`tasks`,只发一份事件。服务端事务生成所有任务索引,返回`taskIds`;客户端核对全部关联后才推进游标。旧客户端单任务请求仍可用并在服务端去重;旧服务器无多任务回执时,新客户端回退逐任务请求,不推进未确认的任务游标;每小时重新探测能力。来源/文件/游标不同的任务保持分别上传。
|
||||
|
||||
## 并发、拒绝和清理
|
||||
|
||||
PG共享令牌桶在独立短事务中扣请求额度,失败的写请求也消耗请求次数。每个通道使用跨进程try advisory lock,最多一个写事务,繁忙返回429;有500ms锁等待和SQL超时,避免大量摄取请求长时间占满连接池。
|
||||
存储事务先查询已有事件/任务索引,在同一事务中检查并占用全局/账号累计与每日额度,再批量写入正文及索引。拒绝在大正文写入前发生,整批无部分成功;重试仅确认已有记录,不重复扣存储额度。成功响应的`accepted`包含幂等重复记录。
|
||||
|
||||
拒绝提供安全错误码和`Retry-After`:`eddy_rate_limited`、`eddy_daily_quota`、`eddy_storage_quota`、`eddy_ingestion_unconfigured`。客户端不推进游标,遵循提示并指数退避/随机延迟;设置显示额度暂停,原始日志不删除。产品使用本地队列仍最多7天/1000条,长时间暂停后超期数据可能被淘汰。
|
||||
|
||||
每小时按最多5000条清理过期任务索引/无引用的旧正文,使用同一通道锁,删除与累计额度回收同事务;不退还每日新增额度。清理繁忙或失败延后,不阻塞正常本地使用。计时按接收时间,客户端从头重传的旧日志可能重新进入保留窗口,仍受所有配额约束。预算账本每个用户仅一行,没有无限增长的每日明细表。
|
||||
|
||||
## 部署、观察和回滚
|
||||
|
||||
本改动包含`0067_eddy_ingestion_budgets.sql`,**迁移、服务更新必须安排一次摄取维护窗口**:停止旧控制面写入,使用正常部署迁移流程原子执行迁移,再启动新控制面。不要让旧版写入器跨迁移继续写,以免旧写入不计预算。迁移将既有正文去重、保留原任务索引和分页sequence,并初始化累计用量;每日用量从新版本接收时开始累计。既有使用事件没有SSO预算归属,迁移只计入全局预算,不猜用户归属。
|
||||
|
||||
全局字节预算未配置时,新版仍能读历史,所有摄取等待配置;建议先配置小范围灰度预算核对监控,再恢复客户端开关。只通过迁移和部署流程发布,开发机不得执行生产DDL。
|
||||
|
||||
管理员可用`GET /api/eddy/admin/ingestion`查看两个通道的全局用量、行数、配置和是否就绪;普通用户/API key/客户端会话不授予管理员访问权。接口不返回账号标识或聊天正文。
|
||||
|
||||
回滚:先停止摄取和新服务,通过部署迁移工具恢复旧列正文:
|
||||
|
||||
```sql
|
||||
UPDATE eddy_conversation_events e SET event = r.event
|
||||
FROM eddy_conversation_records r WHERE r.id = e.record_id AND e.event IS NULL;
|
||||
```
|
||||
|
||||
恢复后才启动旧二进制,暂不删新表/索引;旧版本没有预算保护,保持客户端上传关闭。再次升级需要重新核算预算,不直接复用回滚期间的账本。
|
||||
@@ -0,0 +1,47 @@
|
||||
# Eddy knowledge catalog v1
|
||||
|
||||
Reference materials, not executable distribution. All complete categories (source-bug,
|
||||
devicesim, app-compat, unknown) are accepted and initially Enabled. Classification is
|
||||
diagnostic metadata. It does not select replay behavior or require administrator/model approval.
|
||||
The old compat API remains unchanged for old clients; new clients do not execute that feed.
|
||||
|
||||
Authenticated POST /api/eddy/knowledge accepts {document, files}, where files maps SHA-256
|
||||
to base64. Document schema is eddy-knowledge-v1: id, title, category, confidence,
|
||||
problem {description, steps, expected, actual, cause, signature?},
|
||||
solution {description, kind, validation, rollback, contract?}, observedEnvironment,
|
||||
origin {task?, derivedFrom?, legacy?}, attachments [{name, role, mediaType, sha256, bytes}].
|
||||
Kinds: frida-js, devicesim-diff, launch-config, advice. JS needs a code attachment;
|
||||
DeviceSim diff needs a diff attachment. Screenshots are optional and should contain only relevant evidence.
|
||||
24 attachments maximum, 8 MiB each, 32 MiB total; text fields bounded and assets hash-checked.
|
||||
An author can revise their record; an SSO administrator can also revise.
|
||||
Eddy clients sign in through browser SSO (`eddy auth login`); ordinary employees receive a knowledge-only session without requesting an API key.
|
||||
See [client SSO](client-sso.md) for the grant flow, session lifetime and deployment order.
|
||||
Repeated content returns its immutable revision receipt; a new revision never undoes Disable.
|
||||
|
||||
GET /api/eddy/knowledge/feed?cursor=...&limit=50 delivers all enabled materials, independent
|
||||
of installed baseline. Stable watermark pagination and serialized change sequence allocation
|
||||
prevent skipped concurrent writes. A subsequent incremental pass catches writes after the
|
||||
watermark. Disable is a tombstone; clients remove it from default retrieval without deleting
|
||||
task edits already created from it. A concurrent disable can make an attachment unavailable;
|
||||
clients keep the prior cursor and restart a full visibility scan on 404.
|
||||
GET /knowledge/:id/:revision and /knowledge/assets/:hash read content; disabled content
|
||||
is restricted to administrators. Baseline identity remains metadata for relevance and
|
||||
local execution checks.
|
||||
|
||||
Administrator APIs under /api/eddy/admin:
|
||||
- GET /knowledge: paged full catalog including Disabled, with latest action actor/time.
|
||||
- POST /knowledge/:id/state with {enabled: boolean}: audited Enable / Disable.
|
||||
- GET /knowledge/:id/export: ZIP containing knowledge.json, receipt.json, problem.md,
|
||||
solution.md and every attachment. The extracted package is compatible with Eddy knowledge save.
|
||||
|
||||
Deployment: apply drizzle/0031_eddy_knowledge_catalog.sql before serving these routes.
|
||||
Then explicitly run scripts/migrate-eddy-knowledge.ts against the intended database.
|
||||
It preserves old material, author/ID/revision provenance and stopped states, never modifies
|
||||
old-client compat states and skips imported revisions on repeat. Invalid/missing payloads
|
||||
are reported for follow-up rather than silently discarded or fabricated.
|
||||
Unsolved observations remain in the historical audit store; they are not invented solutions.
|
||||
Review migration output before switching users to the new knowledge view.
|
||||
|
||||
Local validation: TEST_DATABASE_URL must point to a dedicated local PostgreSQL database.
|
||||
Run bun test control-server/test/eddy_catalog.integration.test.ts and the existing
|
||||
eddy_flow.integration.test.ts separately (both close their connection pool).
|
||||
@@ -0,0 +1,20 @@
|
||||
# Eddy 维护任务的全局租约(locks)
|
||||
|
||||
有些维护任务全世界只能同时跑一个,比如 Eddy 的定时自动发布(Eddy 仓 `maintainer/autorelease`,每天两次检查 DeviceSim main、TikTok M / Notes / Whee 的 alpha 版本,比线上新就打包发布)。它们在开始前向控制面登记一个具名租约,运行中心跳,结束时注销。代码 `control-server/src/eddy_locks.ts`,表 `eddy_locks`(迁移 `drizzle/0074_eddy_locks.sql`)。
|
||||
|
||||
## 接口
|
||||
|
||||
都在 `/api/eddy/admin/locks`,权限和 `/admin/releases/*` 相同:SSO 管理员,或 `Authorization: Bearer $EDDY_RELEASE_TOKEN`(`X-Eddy-User` 只做归因,记为 `actor`)。名字是 1–64 位小写字母、数字、`.`、`_`、`-`。
|
||||
|
||||
| 请求 | 体 | 成功 | 失败 |
|
||||
|---|---|---|---|
|
||||
| `POST /:name/register` | `{holder, ttlSeconds?, info?}` | `{token, lock}` | 409 `eddy_lock_held`,`lock` 里是当前持有者 |
|
||||
| `POST /:name/heartbeat` | `{token, ttlSeconds?, status?}` | `{lock}`,过期时间顺延 | 409 `eddy_lock_lost`:已被别人接管或已注销 |
|
||||
| `POST /:name/unregister` | `{token, result?}` | `{lock}`,`lock.last` 记下这次运行 | 409 `eddy_lock_lost` |
|
||||
| `GET /:name`、`GET /` | — | 当前状态(不含 token) | — |
|
||||
|
||||
- `holder` 1–128 个可打印 ASCII 字符(如 `mac-a pid 123`);`ttlSeconds` 30–3600,默认 600;`info` / `status` 不超过 4 KB,`result` 不超过 8 KB,都必须是 JSON 对象。
|
||||
- token 只在 register 的响应里出现一次,表里只存它的 SHA-256。
|
||||
- 空闲或已过期的租约才能 register,同一条 SQL 完成判断和写入,并发登记只有一个成功。
|
||||
- 过期但还没人接管时,原持有者的心跳仍然有效(token 没变)。被接管后原 token 失效,`last` 记录被接管的那次运行(`end: expired`、它最后的 `status`)。持有者应在每个不可逆步骤前确认自己还持有租约。
|
||||
- 正常注销后 `last` 是 `{holder, info, actor, acquiredAt, endedAt, end: "released", status, result}`。
|
||||
@@ -0,0 +1,88 @@
|
||||
# Eddy product usage monitor
|
||||
|
||||
Client default-on **产品使用数据** switch is independent of conversation sync and compatibility telemetry.
|
||||
Off stops collection/uploads and clears pending events; it does not retroactively collect the disabled interval.
|
||||
Only fixed enums and small numeric fields are accepted: random installation/session/operation/event ids,
|
||||
time, event/action/source/outcome, duration/queue time, Eddy release and macOS major.
|
||||
No message, user input, task title/id, filesystem path, screenshot, URL, raw exception, script or App content.
|
||||
|
||||
`POST /api/eddy/usage`: schema 1 needs browser/Eddy SSO; schema 2 (Eddy runtime level 6 / Mac app 0.9.13) may also come
|
||||
without any credential. batchId (64 hex), events (1–100).
|
||||
Body max 128 KiB. Events older than 8 days or more than 5 minutes in the future are rejected.
|
||||
Each event: id/installation (32 hex), at (ms), name/action/source (closed enums), release, macosMajor;
|
||||
optional session/operation (32 hex), outcome, durationMs/queueMs (0–7 days).
|
||||
Portal views/sessions/engagement require a portal source and session; engagement max 60 seconds.
|
||||
Runtime events require runtime source; jobs require job source. Unknown event fields are rejected.
|
||||
PK = SHA256(installation + ':' + id). Atomic insert ignores duplicates and returns `{batchId, accepted}`.
|
||||
Each new event records the uploading account (`owner`, lower-case SSO email) for the administrator per-user view.
|
||||
|
||||
Schema 2 (`drizzle/0073_eddy_usage_devices.sql`) adds per event: `device` (16 hex: sha256(IOPlatformUUID)[:16], the value
|
||||
every Eddy request already sends as `X-Eddy-Machine`; the same Mac keeps it across reinstalls and folders), `deviceSource`
|
||||
(hardware | hostname), `env` (clients upload only `production`; anything else is rejected as a client bug), optional
|
||||
`appVersion` (`0.9.13+15`) and `props` — a closed set per name/action of enums, flags, bounded integers and version strings
|
||||
(`eddy_usage_rules.ts` PROPS, mirrored by `eddyctl/usage.py`). The Mac app's own events use source `shell`: sessions,
|
||||
engagement (≤5 min per event), `view.opened` of its screens, `runtime.prepare`, `home.shown`, `permission.flow`,
|
||||
`update.flow`, `device.profile`. A schema 2 batch without a credential must come from one device; it is stored with
|
||||
`owner` NULL and `anonymous` true and charged to one shared `anonymous` ingestion budget (its own limits, at most half the global ones — see ingestion-limits.md;
|
||||
the device id is reporting data, never the quota key, so changing it buys nothing). A credential that does not verify
|
||||
is still 401, so a lapsed sign-in is noticed.
|
||||
The client deletes only acknowledged files, caps the queue at 1,000/7 days, uploads every 30 seconds,
|
||||
and binds a queue entry to its original API endpoint. Collection does not wait on upload requests.
|
||||
|
||||
Names: session.started, view.opened, engagement, operation.started, operation.completed,
|
||||
app.launch, app.session, agent_job.completed; schema 2 adds runtime.prepare, home.shown, permission.flow, update.flow,
|
||||
device.profile, and (P1) desk.action, preview.change, dock.attach, agent.entry, notice.shown, notice.action, app.crash
|
||||
(runtime) and the operations storage.clean (props what / freedMB) and storage.tidy. Actions, views (Portal and Mac app)
|
||||
and props are in `eddy_usage_rules.ts`. The metrics add `details` — those P1 events by action and their one closed
|
||||
detail (kind / what / class / agent / value) with counts, devices, success rate, median duration and MB freed — crashes
|
||||
per App hour (summary.appCrashes over appRuntimeMs) and more adoption (desk captures, previews, docking, Agent entry).
|
||||
Portal workers emit operation starts/ends; their nested CLI skips counting. Manual/Agent CLI uses
|
||||
exit status plus top-level ok:false/isError. Actual launches measure spawn→ready including failed spawn;
|
||||
jobs use their actual exit and artifact validation. App runtime is ready→supervisor-exit duration,
|
||||
not user-interaction time. Session ids restart per Portal document load.
|
||||
|
||||
`GET /api/eddy/admin/usage?days=28&release=...&user=<email>`: browser SSO administrator only, no Eddy-session admin bypass.
|
||||
Returns summary (installations, devices, accounts, devices with anonymous uploads, Portal and Mac app sessions and
|
||||
time), daily UTC buckets, per-name/action/source counts and percentiles, release breakdown,
|
||||
`users` (per account, `anonymous` for uploads without a sign-in: devices, installations, sessions, actions, launch/operation
|
||||
outcomes, Portal/Mac app/App time, last event), `devices` (per Mac: first 8 hex digits only, installations, number of
|
||||
accounts but never which, sessions, time, launches, App time, Mac app builds, latest hardware profile; up to 500),
|
||||
`funnel` (devices whose first launch is in the range — a Mac app start on a fresh eddy folder or its first runtime
|
||||
preparation — and how many then had the runtime ready, reached home, installed an App, opened a task, had an Agent edit,
|
||||
shared; median time to each), `retention` (those devices by UTC week of first launch: any event on day 1, days 7–13,
|
||||
days 28–34; a window counts only once it has fully passed), `adoption` (of the active devices, how many opened a task,
|
||||
used the design desk, cold-restarted an App, had an Agent edit, shared, imported, relayed), `appVersions`, and `flows`
|
||||
(update and permission steps);
|
||||
`user` filters every section to one account. No raw rows or installation ids. Activity counts use distinct random
|
||||
installations, **not deduplicated people**; the per-user rows group by uploading account. Events from before
|
||||
`owner` was recorded are matched by `budget_owner` (sha256 of the email) against known Eddy accounts
|
||||
(sign-ins, synced-history owners); unmatched ones appear as `sha256:<prefix>`, events older than that as null.
|
||||
Success denominator includes success/failure/timeout/interrupted, excludes cancellation and missing outcomes.
|
||||
Latency P50/P95 uses successful completed events; queue P95 includes all completions.
|
||||
Portal active time requires visible/focused/recent-input, excludes long sleep gaps. Ongoing App sessions
|
||||
are absent from completed runtime sums. Days are rolling 24-hour periods; edge daily bins can be partial.
|
||||
Abandoned pre-SSO sign-in, offline/opted-out clients, dropped queue entries and missing completions
|
||||
are not a complete product funnel. No native FPS/CPU claim or automatic alerting.
|
||||
|
||||
Management → Eddy → **使用与表现** exposes 7/28/90-day, release and user filters, device/account/install and Mac app
|
||||
KPIs, the activation funnel, weekly retention, feature use, per-user and per-device tables, per-action reliability/latency
|
||||
tables, update and permission steps, Mac app builds, daily trends and page counts. Auto refresh every visible minute.
|
||||
`eddy_usage_events` retains received events for 90 days, pruned at boot, hourly and during ingestion.
|
||||
Data collection is additional to legacy compatibility observations, not an extension of their raw payloads.
|
||||
|
||||
Rollout: apply additive `drizzle/0066_eddy_usage.sql` through deployment, then server, then client. Schema 2 the same
|
||||
way: `drizzle/0073_eddy_usage_devices.sql`, then this server (it keeps accepting schema 1), then the Eddy runtime, then
|
||||
the Mac app.
|
||||
This follows the separate conversation migration 0065 when both features ship together.
|
||||
Client rollback stops new collection; server rollback can leave the additive table in place.
|
||||
Development tests use only explicitly isolated `TEST_DATABASE_URL`, never the production database.
|
||||
|
||||
```sh
|
||||
bun test control-server/test/eddy_usage.test.ts
|
||||
TEST_DATABASE_URL=postgres://user@127.0.0.1:55483/test bun test control-server/test/eddy_usage.integration.test.ts
|
||||
EDDY_TEST_SOURCE=/path/to/eddy-source TEST_DATABASE_URL=postgres://user@127.0.0.1:55483/test bun test control-server/test/eddy_usage.integration.test.ts
|
||||
```
|
||||
|
||||
Integration database needs migrations 0032 (SSO) and 0066. Synthetic cross-repository probe verifies
|
||||
actual Python→HTTP→Hono→Postgres, authentication, no admin bypass, fixed-field rejection, idempotency,
|
||||
exact success/percentile denominators, range/version filters, retention and opt-out.
|
||||
Reference in New Issue
Block a user