From c54d188887fa2310160855c92fe38fc5db6adccb Mon Sep 17 00:00:00 2001 From: zizi Date: Thu, 23 Jul 2026 10:40:07 +0800 Subject: [PATCH] =?UTF-8?q?=E6=A1=86=E6=9E=B6:=20=E5=8D=87=E6=A0=BC?= =?UTF-8?q?=E6=8A=BD=E5=8F=96=E6=94=B9=E4=B8=BA=E5=8F=AF=E6=81=A2=E5=A4=8D?= =?UTF-8?q?=E7=9A=84=E4=B8=A4=E9=98=B6=E6=AE=B5=E7=9F=AD=E4=BA=8B=E5=8A=A1?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .claude/skills/parse-book/SKILL.md | 6 +- .../parse-book/scripts/parse_upgrade.py | 2122 ++++++++++------- .../scripts/test_parse_upgrade_offline.py | 1451 +++++++---- docs/2026-07-16-升格卡改造设计.md | 36 + 4 files changed, 2214 insertions(+), 1401 deletions(-) diff --git a/.claude/skills/parse-book/SKILL.md b/.claude/skills/parse-book/SKILL.md index 533f184..3b3a3a6 100644 --- a/.claude/skills/parse-book/SKILL.md +++ b/.claude/skills/parse-book/SKILL.md @@ -58,11 +58,13 @@ disable-model-invocation: true **窗行陷阱（放量首日实测）**：`--window` 参数变化后重切,旧窗行会按 from_order 占位,新的大窗被「已有大纲跳过」→ 中间章域永远漏出卡（验收期 1–3 章小窗占住 from_order=1,放量 1–34 章大窗被跳过）。**换窗参数重切前必须先删该书全部窗行**（窗行是可再生中间产物;卡挂「窗起」,cards 重出时按窗软删重出）。 -**作品面升格执行器 `scripts/parse_upgrade.py`（与上面范式拆书管线并行的另一条线,命令 `windows`/`run`/`status`）**：把参考书正文按窗抽成「会随剧情长大的实体卡」(升格卡,`source_type=upgrade_book`),设计见 `docs/2026-07-16-升格卡改造设计.md`。里程碑除真实章号外，模型输出必须临时携带所标章节正文短引；系统机械核验后删除证据，错章、缺证据或改写证据均拒收入库并留审计。模型给出的顶层出场章也必须由实体规范名或合法别名在对应章节正文中的实际出现机械证明；合法别名集合同时取卡内 payload 与独立 alias 表，单字规范名禁作章证据，规范名和别名均禁止通用称谓/关系称呼（如队长、舰长）作证据，常规二至四字专名保持精确子串命中。未命中章不得参与立卡、登场兜底或既有卡追加，无实证章时不补登场里程碑；机械登场兜底只允许用 debut 章正文存在性与实体名称/型构造中性台阶，禁止读取跨章摘要。既有卡每窗新增出场章必须审计完整旧值，redo/undo 精确恢复后再按新正文重算。任何送入模型的既有卡 payload 必须同步绑定读取时 revision；写事务锁行后必须仍为 `pending + upgrade_book` 且 revision 一致，否则整窗失败/重试，禁止以旧模型结果覆盖外部改版。关系 prompt 的每个参与角色也必须先读取 expected revision、再锁定并复验 status/source/revision，只有锁后 payload 可送模型，新建和更新关系都只能引用通过该门禁的角色。显式 redo **仅允许当前 active 末窗**：执行前机械校验窗口号连续、章域合法且首尾相接，并校验目标窗逐章正文齐全且非空；任一失败必须在快照、undo 与任何写入前非零退出。历史窗修正必须全书前滚重建；后缀级联重算属于 P1，当前不支持。合法末窗 redo 仍须在撤销事务中清除所有 active `upgrade_book` 卡的本窗章域，保留窗外章并为历史无审计数据补旧值审计；redo 前建立完整恢复点（卡 payload/status/revision/deleted、别名、presence、水位、审计与窗状态），首次失败重新清理再试，最终失败完整恢复且重试初建卡软删。`--max-calls` 不得截断已经开始的即时重试，非显式 redo 不做该全局清理。`run` 默认**不发嵌入**;加 `--semantic-dedup` 开语义判重(治改名/跨型漏并)时,每窗按**读→算→写三段式短连接**跑——观察/近邻召回/M3 终判都在**无长连接**段发 LLM 与嵌入 HTTP(不再持窗级连接跨调用存活),写段只查预判结果落库;实体卡与关系卡的新建/更新都必须同步 state、逐字段审计并进入 touched；并**边抽边嵌**：本窗新建/更新卡在窗事务提交后增量嵌入落库(软删旧向量+upsert 新行),**后窗即可语义召回前窗刚长成的卡**,不再依赖"同书须预先全量 embed"。普通嵌段服务失败只告警；确定性 owner 冲突先持久化 durable `compensation-failed` marker，再由普通窗按 touched fence、显式 redo 按整书六域全域 fence 补偿，任一外部漂移都禁止覆盖。marker 窗下次 run 禁止自动 undo。 +**作品面升格执行器 `scripts/parse_upgrade.py`（命令 `windows`/`run`/`status`）**：正文按窗抽取 `upgrade_book` 实体卡。每窗采用“两阶段短事务”：模型/嵌入调用期间不持有业务连接；实体写入和关系写入各自提交 `processing` marker；最终嵌入必须完整成功，才与 `done` 在同一短事务提交。窗口输入摘要绑定作品标题、章/块 ID、标题、状态、revision、类型、正文、active schema 合同和当前脚本 SHA。启动时会恢复可精确撤销的 `processing`；本次窗口第二次失败立即以非零退出并停止当前作品，保留 `retryable-clean` 断点，人工再次运行才继续。 + +`recover-legacy-failed --preview/--execute` 是 fence 引入前 failed 窗的唯一恢复入口：preview 只读输出窗口、六类本窗产物计数、processing 数和确认 SHA；execute 需确认旧进程已结束、同书锁内重算并 exact 匹配，且无 processing、**阻断产物计数为 0**，才保持 `failed` 并标为 `retryable-clean`。计数只豁免能机械证明来源的全书 reset 墓碑：draft 必须同时 `deleted=true`、`updater='upgrade-reset'`、`status='pending'`；embedding 还必须 `entity_id IS NULL` 且自身 `deleted=true`；alias 必须 `deleted=true` 且 `updater='upgrade-reset'`。presence 没有 reset 来源标记，任何本窗行（包括软删墓碑）都阻断；其他软删、活跃行、已有 entity owner，以及任何 card_state/audit 行也都阻断。完整 `stateSha` 仍覆盖所有 deleted 行，二者不能混淆。不调用模型/嵌入。`--redo-window` 已禁用，历史修正走人工 backup/reset/rebuild。 **登场中性台阶名称边界**：名称只能取 debut 章正文唯一实际命中的合法规范名或别名；若只命中旧别名就用旧别名，多个合法名称同时命中、仅命中通用称谓或无法唯一确定时退化为“人物登场/物件登场”等仅类型描述，禁止泄漏未来才形成的规范名。`new_card` 规范化名称后必须把实际 canonical 与合法 aliases 立即登记到本窗判重索引，不能继续使用模型原始括号名。 -**嵌入唯一键冲突失败关闭**：前述“嵌段失败只告警”仅指普通网络或服务异常；owner 预检必须在 `embed_texts` 前覆盖全部 todo hash。写连接第一条 SQL 必须按固定顺序以 `ROW EXCLUSIVE` 锁 `muse_knowledge_draft, example_knowledge_embedding`；随后按 did 锁 draft 行，机械确认 tenant、deleted=false、status=pending，并以当前 payload+MODEL 重算 hash 与 ready hash 一致，再按 did 锁全部活向量、按 hash 以 `FOR UPDATE` 二次校验 owner。若 HTTP 期间 payload 漂移，`(tenant_id, content_hash, model)` 唯一键被两个活跃 draft 的当前 payload 同时声明，唯一行 `entity_id` 非空，或同 draft 任一旧活向量已有 `entity_id`，必须在任何向量写入前硬停并令命令非零；禁止迁移 entity owner、清空 `entity_id`、软删实体向量或打印窗口完成。硬停后先以窗口 `done/null/upgrade` CAS 独立提交 durable `compensation-failed` marker；marker 失败不得继续补偿。普通窗随后锁定并 exact 复验 touched draft 的 status/source/revision/payload hash，通过后才允许同事务 `undo→最终 failed`；显式 redo 则按固定顺序阻写 draft/alias/presence/card_state/audit/window 六域，exact 复验 redo postcommit 全域快照（draft 含 payload/status/revision/updater/deleted），通过后才恢复 pre-redo 全域与原窗状态。任一外部确认、删除、改版或辅助域漂移都禁止 undo/restore，保留 marker；marker 窗下次 run 必须非零阻断，禁止自动 undo。仅当唯一行没有 entity owner 且命中旧软删 draft 时，才把行迁到当前 draft并刷新当前嵌入语义列；冲突更新及旧 hash 软删条件都必须保留 `entity_id IS NULL` 防并发竞态。 +**嵌入唯一键冲突失败关闭**：owner 预检必须覆盖全部 todo hash；写入前锁 draft、全部活向量和 hash owner，复验 `pending + upgrade_book`、revision、payload hash 与 `entity_id` 归属。普通嵌入异常、owner 冲突或 payload 漂移都令窗口失败，不打印完成；最终向量写入和 `done` 必须同事务。任何外部确认、删除、改版或辅助域漂移都禁止补偿覆盖。 **同书命令互斥**：`parse_upgrade.py run/windows`、`reset_upgrade_work.py` 的预览/执行，以及 `backup_upgrade_work.py backup/rehearse/restore` 均须先取得 `scripts/upgrade_work_lock.py` 的同租户同作品 PostgreSQL session advisory lock；失败必须在任何业务 SQL、文件 verify/写入、嵌入或 LLM 调用前非零退出。锁由独立 autocommit 连接持有到命令结束，该连接只执行加锁/解锁 SQL；`status` 只读且不取锁。所有调用方必须导入同一个 `upgrade_work_lock(...)` context manager，禁止另造不兼容锁键。 diff --git a/.claude/skills/parse-book/scripts/parse_upgrade.py b/.claude/skills/parse-book/scripts/parse_upgrade.py index c51313e..c7c9bd4 100644 --- a/.claude/skills/parse-book/scripts/parse_upgrade.py +++ b/.claude/skills/parse-book/scripts/parse_upgrade.py @@ -13,9 +13,9 @@ 防膨胀（第四轮评审 G1/G5）：观察调用只带「窗内命中」实体索引（不带全库）； 窗切割限 12 章/3.5 万字双闸。防重跑自噬：同窗重跑先按审计撤销再重写。 -连接三段式（洞①）：每窗读→算→写切短连接，DB 连接绝不跨 LLM/嵌入 HTTP 存活（更新/关系的窗事务内 M3 -调用是既有已接受豁免、keepalives 兜底，本次不动）。嵌入：默认关（试跑期延后）；开 --semantic-dedup 则 -边抽边嵌——本窗新建/更新卡窗事务后增量嵌入落库，后窗即可语义召回前窗刚长成的卡。 +连接三段式（洞①）：每窗读→算→写切短连接，DB 连接绝不跨 LLM/嵌入 HTTP 存活；实体、关系和最终嵌入 +各自在短事务内提交。嵌入：默认关（试跑期延后）；开 --semantic-dedup 时，最终嵌入必须完整成功后才把 +窗口置为 done。 命令： windows --work-id N 机械切正文窗（幂等，from_chapter 锚） @@ -40,7 +40,7 @@ from parse_llm import m3_json, SensitiveHardStop, IDENTITY, TENANT, DSN # noqa: from upgrade_work_lock import UpgradeWorkLockUnavailable, upgrade_work_lock # noqa: E402 # 语义判重（P1）复用 embed skill 的嵌入通道（同模型同维、与检索端语义对齐）—— # 只在开启 --semantic-dedup 时才真调，默认关（试跑期嵌入延后，见文件头注释）； -# build_embed_text/MODEL/DIM/ACTOR 供嵌段（洞①边抽边嵌）复用检索端同源文本构造器与列常量。 +# build_embed_text/MODEL/DIM/ACTOR 供最终嵌入短事务复用检索端同源文本构造器与列常量。 sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[2] / "embed" / "scripts")) from embed_drafts import (_session as _embed_session, embed_texts, build_embed_text, # noqa: E402 MODEL as EMBED_MODEL, DIM as EMBED_DIM, ACTOR as EMBED_ACTOR) @@ -109,6 +109,247 @@ class EntityUpdateOutputConflict(RuntimeError): COMPENSATION_FAILED_PREFIX = "compensation-failed:" +UPGRADE_FENCE_VERSION = "upgrade-fence:v1" +# 输入摘要必须绑定运行合同；修改抽取批量、卡来源或嵌入模型都会使旧窗口拒绝继续写入。 +UPGRADE_CONTRACT = { + "windowProtocol": UPGRADE_FENCE_VERSION, + "sourceType": SOURCE_TYPE, + "updateBatch": UPDATE_BATCH, + "embeddingModel": EMBED_MODEL, + "embeddingDimensions": EMBED_DIM, +} + + +def _sha256_json(value): + """以稳定 JSON 编码计算摘要；数据库时间等值统一转字符串，避免驱动类型影响结果。""" + + raw = json.dumps(value, ensure_ascii=False, sort_keys=True, separators=(",", ":"), default=str) + return hashlib.sha256(raw.encode("utf-8")).hexdigest() + + +def _aggregate_window_chapter_texts(chapters): + """按窗口送模规则把每章多个 block 以换行拼接，保持读段和摘要规则一致。""" + + grouped = {} + if chapters and isinstance(chapters[0], dict): + for chapter in chapters: + grouped.setdefault(chapter["order_no"], []).append( + str(chapter.get("content_text") or "") + ) + else: + for order_no, _, content in chapters or []: + grouped.setdefault(int(order_no), []).append(str(content or "")) + return {order_no: "\n".join(parts) for order_no, parts in grouped.items()} + + +def _script_sha256(): + """读取当前实际运行脚本的 SHA-256，避免只绑定 git HEAD 或人工版本字符串。""" + + return hashlib.sha256(pathlib.Path(__file__).read_bytes()).hexdigest() + + +def _window_input_sha(window, chapters, schemas, work_title=None): + """绑定实际送模输入、schema active 合同、运行合同和当前脚本内容。""" + + return _sha256_json({ + "window": window, + "workTitle": work_title, + "chapters": chapters, + "chapterPromptText": _aggregate_window_chapter_texts(chapters), + "schemas": schemas, + "contract": UPGRADE_CONTRACT, + "parseUpgradeSha256": _script_sha256(), + }) + + +def _window_state_sha(state, current_window_id): + """计算本书七域状态摘要；当前窗 status/error 是 marker 载体，不参与自引用 hash。""" + + normalized = deepcopy(state) + windows = [] + for row in normalized.get("windows", []): + if isinstance(row, dict): + item = deepcopy(row) + if item.get("id") == current_window_id: + item.pop("status", None) + item.pop("error_message", None) + windows.append(item) + elif row and row[0] == current_window_id and len(row) >= 7: + windows.append(tuple(row[:5]) + tuple(row[7:])) + else: + windows.append(row) + normalized["windows"] = windows + return _sha256_json(normalized) + + +def _upgrade_marker(stage, input_sha, state_sha): + """生成只有 entity/relation 两种阶段的窗口恢复 marker。""" + + if stage not in ("entity", "relation"): + raise CompensationFenceConflict(f"非法 fence stage：{stage}") + return f"{UPGRADE_FENCE_VERSION}:{stage}:{input_sha}:{state_sha}" + + +def _parse_upgrade_marker(marker): + """严格解析 v1 marker；未知版本、阶段或非 SHA-256 值一律失败关闭。""" + + match = re.fullmatch( + r"upgrade-fence:v1:(entity|relation):([0-9a-f]{64}):([0-9a-f]{64})", + str(marker or ""), + ) + if not match: + raise CompensationFenceConflict(f"非法或未知窗口 marker：{marker}") + return match.groups() + + +def _recovery_decision(marker, current_input_sha, current_state_sha): + """纯恢复判定：只有 marker/input/state 三者精确一致才允许撤销。""" + + try: + _, expected_input, expected_state = _parse_upgrade_marker(marker) + except CompensationFenceConflict: + return "compensation-failed" + if expected_input != current_input_sha or expected_state != current_state_sha: + return "compensation-failed" + return "undo" + + +def _call_external_without_connection(tracker, external, *args): + """离线可观测守卫：外部调用入口要求业务连接计数为零。""" + + if tracker.get("connections", 0): + raise RuntimeError("外部调用期间仍持有业务数据库连接") + return external(*args) + + +def _apply_prepared_and_done(conn, prepared, *, apply_one, mark_done): + """在调用者提供的同一事务内先写 prepared 向量，再标记 done。""" + + for item in prepared: + apply_one(item) + mark_done() + conn.commit() + + +def _lock_upgrade_domains(conn): + """固定锁仅跨短写事务；代价是短暂串行化不同作品，换取七域摘要无幻读。""" + + conn.execute( + "LOCK TABLE muse_content_chapter, muse_content_block, muse_meta_schema, " + "muse_meta_schema_version IN SHARE MODE" + ) + conn.execute( + "LOCK TABLE muse_knowledge_draft, example_upgrade_alias, example_upgrade_presence, " + "example_upgrade_card_state, example_upgrade_audit, example_upgrade_window, " + "example_knowledge_embedding IN SHARE ROW EXCLUSIVE MODE" + ) + + +def _capture_window_input(conn, work_id, win_no, *, validate=True): + """读取 inputSha 的完整组成；字段必须与实际送模正文和作品标题一致。""" + + row = conn.execute( + """SELECT id, window_no, from_chapter, to_chapter, deleted + FROM example_upgrade_window + WHERE tenant_id=%s AND work_id=%s AND window_no=%s""", + (TENANT, work_id, win_no), + ).fetchone() + if not row: + raise CompensationFenceConflict(f"窗口不存在：work={work_id}，window={win_no}") + window = dict(zip(("id", "window_no", "from_chapter", "to_chapter", "deleted"), row)) + work_title = conn.execute( + """SELECT title FROM muse_content_work + WHERE tenant_id=%s AND id=%s AND deleted=FALSE""", + (TENANT, work_id), + ).fetchone() + if not work_title: + raise CompensationFenceConflict(f"作品不存在或已删除：work={work_id}") + chapter_rows = conn.execute( + """SELECT c.id, c.order_no, c.title, c.status, c.revision, + b.id, b.order_no, b.block_type, b.revision, COALESCE(b.content_text,'') + FROM muse_content_chapter c + JOIN muse_content_block b ON b.chapter_id=c.id AND b.deleted=FALSE + WHERE c.tenant_id=%s AND c.work_id=%s AND c.deleted=FALSE + AND c.order_no BETWEEN %s AND %s + ORDER BY c.order_no, b.order_no, b.id""", + (TENANT, work_id, window["from_chapter"], window["to_chapter"]), + ).fetchall() + chapters = [ + { + "chapter_id": row[0], + "order_no": row[1], + "title": row[2], + "status": row[3], + "revision": row[4], + "block_id": row[5], + "block_order_no": row[6], + "block_type": row[7], + "block_revision": row[8], + "content_text": row[9], + } + for row in chapter_rows + ] + if validate: + expected_orders = set(range(window["from_chapter"], window["to_chapter"] + 1)) + actual_orders = {chapter["order_no"] for chapter in chapters} + chapter_texts = _aggregate_window_chapter_texts(chapters) + if actual_orders != expected_orders: + raise CompensationFenceConflict( + f"窗口章域不完整：expected={min(expected_orders)}-{max(expected_orders)}，" + f"actual={sorted(actual_orders)}" + ) + empty_orders = sorted( + order_no for order_no in expected_orders + if not chapter_texts.get(order_no, "").strip() + ) + if empty_orders: + raise CompensationFenceConflict(f"窗口正文为空：chapters={empty_orders}") + schemas = conn.execute( + """SELECT s.schema_key, s.active_version_id, v.field_contract_snapshot + FROM muse_meta_schema s JOIN muse_meta_schema_version v ON v.id=s.active_version_id + WHERE s.tenant_id=%s AND s.schema_key=ANY(%s) ORDER BY s.schema_key""", + (TENANT, list(ENTITY_TYPES + (RELATION_TYPE,))), + ).fetchall() + return window, chapters, schemas, work_title[0] + + +def _capture_window_state(conn, work_id, current_window_id): + """读取本书七个受控域的完整行像，仅当前 marker 窗忽略 status/error。""" + + queries = { + "drafts": ("SELECT d.id, encode(sha256(convert_to(to_jsonb(d)::text, 'UTF8')), 'hex') " + "FROM muse_knowledge_draft d WHERE d.tenant_id=%s AND d.work_id=%s " + "AND d.source_type=%s ORDER BY d.id", (TENANT, work_id, SOURCE_TYPE)), + "aliases": ("SELECT a.id, encode(sha256(convert_to(to_jsonb(a)::text, 'UTF8')), 'hex') " + "FROM example_upgrade_alias a WHERE a.tenant_id=%s AND a.work_id=%s " + "ORDER BY a.id", (TENANT, work_id)), + "presence": ("SELECT p.id, encode(sha256(convert_to(to_jsonb(p)::text, 'UTF8')), 'hex') " + "FROM example_upgrade_presence p WHERE p.tenant_id=%s AND p.work_id=%s " + "ORDER BY p.id", (TENANT, work_id)), + "card_state": ("SELECT s.draft_id, encode(sha256(convert_to(to_jsonb(s)::text, 'UTF8')), 'hex') " + "FROM example_upgrade_card_state s WHERE s.tenant_id=%s AND s.work_id=%s " + "ORDER BY s.draft_id", (TENANT, work_id)), + "audits": ("SELECT a.id, encode(sha256(convert_to(to_jsonb(a)::text, 'UTF8')), 'hex') " + "FROM example_upgrade_audit a JOIN muse_knowledge_draft d ON d.id=a.draft_id " + "WHERE a.tenant_id=%s AND d.work_id=%s AND d.source_type=%s ORDER BY a.id", + (TENANT, work_id, SOURCE_TYPE)), + "windows": ("SELECT w.id, encode(sha256(convert_to(" + "CASE WHEN w.id=%s THEN (to_jsonb(w)-'status'-'error_message')::text " + "ELSE to_jsonb(w)::text END, 'UTF8')), 'hex') " + "FROM example_upgrade_window w WHERE w.tenant_id=%s " + "AND w.work_id=%s ORDER BY w.id", + (current_window_id, TENANT, work_id)), + "embeddings": ("SELECT e.id, encode(sha256(" + "convert_to((to_jsonb(e)-'embedding')::text, 'UTF8') || " + "coalesce(vector_send(e.embedding), ''::bytea)), 'hex') " + "FROM example_knowledge_embedding e " + "JOIN muse_knowledge_draft d ON d.id=e.draft_id " + "WHERE e.tenant_id=%s AND d.tenant_id=%s AND d.work_id=%s " + "AND d.source_type=%s ORDER BY e.id", + (TENANT, TENANT, work_id, SOURCE_TYPE)), + } + return {name: conn.execute(sql, params).fetchall() + for name, (sql, params) in queries.items()} # ── 库内合同（元数据驱动公理：prompt 与守卫同源，禁手写合同）── @@ -172,19 +413,24 @@ def cut_windows(conn, work_id): # ── 每窗材料与已知名加载 ── def load_window_material(conn, work_id, a, b): - """返回窗正文拼接文本与按绝对章号索引的原始正文。""" + """返回窗正文拼接文本与按绝对章号聚合的原始正文。""" rows = conn.execute( """SELECT c.order_no, c.title, b2.content_text FROM muse_content_chapter c JOIN muse_content_block b2 ON b2.chapter_id=c.id AND b2.deleted=FALSE WHERE c.tenant_id=%s AND c.work_id=%s AND c.order_no BETWEEN %s AND %s - AND c.deleted=FALSE ORDER BY c.order_no""", + AND c.deleted=FALSE ORDER BY c.order_no, b2.order_no, b2.id""", (TENANT, work_id, a, b)).fetchall() - return ( - "\n\n".join(f"## 第{o}章 {t}\n{x}" for o, t, x in rows), - {int(order_no): str(content or "") for order_no, _, content in rows}, + chapter_titles = {} + for order_no, chapter_title, content in rows: + chapter_titles.setdefault(int(order_no), chapter_title) + chapter_texts = _aggregate_window_chapter_texts(rows) + text = "\n\n".join( + f"## 第{order_no}章 {chapter_titles[order_no]}\n{chapter_texts[order_no]}" + for order_no in sorted(chapter_texts) ) + return text, chapter_texts def load_window_text(conn, work_id, a, b): @@ -1292,17 +1538,10 @@ def merge_card( _update_card_state(conn, draft_id, work_id, win_no) -def new_card( - conn, - work_id, - win_no, - ent, - milestone_types=None, - *, - chapter_texts=None, - known_chapters=None, -): - """立初卡：payload 全字段以旧值=NULL 入审计（G7，错认拆回可还原初始态）。 +def _build_new_card_payload(work_id, win_no, ent, milestone_types=None, *, + chapter_texts=None, known_chapters=None): + """纯内存构造初卡；planner 与短写共用，保证模型快照和最终落库内容一致。 + milestone_types（洞②）：含「演变历程」字段的型集合（据库内合同动态判定）——该型立卡时若抽取结果 无登场里程碑，机械兜底补一条登场（保成长型实体登场即有生命周期起点）；None/空集则不触发兜底。 @@ -1369,9 +1608,28 @@ def new_card( "来源": f"升格@窗{win_no}", "状态": "草稿", "目标库": "本书作品库", "可见范围": "本书私有", "_work_id": work_id} + return payload + + +def new_card( + conn, + work_id, + win_no, + ent, + milestone_types=None, + *, + chapter_texts=None, + known_chapters=None, +): + """立初卡：payload 全字段以旧值=NULL 入审计（G7，错认拆回可还原初始态）。""" + + payload = _build_new_card_payload( + work_id, win_no, ent, milestone_types, + chapter_texts=chapter_texts, known_chapters=known_chapters, + ) # 先原子占用全部 alias；任一不同 canonical 冲突都会让窗事务在写 payload 前失败关闭。 for alias in payload["别名"]: - _claim_alias(conn, work_id, raw, alias, win_no, "init") + _claim_alias(conn, work_id, payload["名称"], alias, win_no, "init") did = conn.execute( """INSERT INTO muse_knowledge_draft (work_id, draft_type, draft_payload, status, source_type, source_id, @@ -1389,172 +1647,14 @@ def new_card( """INSERT INTO example_upgrade_card_state (draft_id, work_id, watermark_window, tenant_id) VALUES (%s,%s,%s,%s) ON CONFLICT (draft_id) DO NOTHING""", (did, work_id, win_no, TENANT)) - return did, raw, tuple(payload["别名"]) + return did, payload["名称"], tuple(payload["别名"]) -def _snapshot_redo_window(conn, work_id, win_no): - """抓取显式 redo 前的完整可恢复状态，不让数据库连接跨 LLM 调用存活。""" - - snapshot = { - "window_no": win_no, - "drafts": conn.execute( - """SELECT id, draft_payload, status, revision, updater, deleted - FROM muse_knowledge_draft - WHERE tenant_id=%s AND work_id=%s AND source_type=%s - ORDER BY id""", - (TENANT, work_id, SOURCE_TYPE), - ).fetchall(), - "aliases": conn.execute( - """SELECT id, canonical_name, alias, evidence_window, verdict_by, - creator, create_time, updater, update_time, deleted - FROM example_upgrade_alias - WHERE tenant_id=%s AND work_id=%s - ORDER BY id""", - (TENANT, work_id), - ).fetchall(), - "presence": conn.execute( - """SELECT id, window_no, chapter_no, entity_type, name, observation, - creator, create_time, deleted - FROM example_upgrade_presence - WHERE tenant_id=%s AND work_id=%s - ORDER BY id""", - (TENANT, work_id), - ).fetchall(), - "card_states": conn.execute( - """SELECT draft_id, watermark_window, update_time - FROM example_upgrade_card_state - WHERE tenant_id=%s AND work_id=%s - ORDER BY draft_id""", - (TENANT, work_id), - ).fetchall(), - "audits": conn.execute( - """SELECT a.id, a.draft_id, a.window_no, a.field_name, - a.old_value, a.new_value, a.create_time - FROM example_upgrade_audit a - JOIN muse_knowledge_draft d ON d.id=a.draft_id - WHERE a.tenant_id=%s AND d.work_id=%s AND d.source_type=%s - ORDER BY a.id""", - (TENANT, work_id, SOURCE_TYPE), - ).fetchall(), - "window": conn.execute( - """SELECT status, error_message, updater FROM example_upgrade_window - WHERE tenant_id=%s AND work_id=%s AND window_no=%s""", - (TENANT, work_id, win_no), - ).fetchone(), - } - return deepcopy(snapshot) - - -def _assert_redo_postcommit_fence(conn, work_id, expected_snapshot): - """阻写 redo 恢复覆盖域并 exact 复验提交态；任一域漂移都禁止整书恢复。""" - - if expected_snapshot is None: - raise CompensationFenceConflict("redo 补偿缺少 postcommit 全域围栏") - # 固定表序一次取得阻写锁；等待中的外部写事务提交后，READ COMMITTED 重读必能看到并拒绝漂移。 - conn.execute( - """LOCK TABLE muse_knowledge_draft, example_upgrade_alias, - example_upgrade_presence, example_upgrade_card_state, - example_upgrade_audit, example_upgrade_window - IN SHARE ROW EXCLUSIVE MODE""" - ) - current_snapshot = _snapshot_redo_window( - conn, work_id, expected_snapshot["window_no"] - ) - for domain in ("drafts", "aliases", "presence", "card_states", "audits", "window"): - if current_snapshot.get(domain) != expected_snapshot.get(domain): - raise CompensationFenceConflict(f"redo 补偿全域围栏冲突：domain={domain}") - - -def _restore_redo_window(conn, work_id, snapshot): - """原子恢复 redo 前状态；重试中新建卡只做软删，保留全局编号与引用安全。""" - - old_draft_ids = {row[0] for row in snapshot["drafts"]} - current_draft_ids = { - row[0] - for row in conn.execute( - """SELECT id FROM muse_knowledge_draft - WHERE tenant_id=%s AND work_id=%s AND source_type=%s""", - (TENANT, work_id, SOURCE_TYPE), - ).fetchall() - } - for draft_id in current_draft_ids - old_draft_ids: - conn.execute("UPDATE muse_knowledge_draft SET deleted=TRUE WHERE id=%s", (draft_id,)) - for draft_id, payload, status, revision, updater, deleted in snapshot["drafts"]: - conn.execute( - """UPDATE muse_knowledge_draft - SET draft_payload=%s, status=%s, revision=%s, updater=%s, deleted=%s - WHERE id=%s AND tenant_id=%s AND work_id=%s AND source_type=%s""", - (json.dumps(payload, ensure_ascii=False), status, revision, updater, deleted, - draft_id, TENANT, work_id, SOURCE_TYPE), - ) - - conn.execute( - "DELETE FROM example_upgrade_alias WHERE tenant_id=%s AND work_id=%s", - (TENANT, work_id), - ) - for row in snapshot["aliases"]: - conn.execute( - """INSERT INTO example_upgrade_alias - (id, work_id, canonical_name, alias, evidence_window, verdict_by, - creator, create_time, updater, update_time, deleted, tenant_id) - VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s)""", - (row[0], work_id, *row[1:], TENANT), - ) - - conn.execute( - "DELETE FROM example_upgrade_presence WHERE tenant_id=%s AND work_id=%s", - (TENANT, work_id), - ) - for row in snapshot["presence"]: - conn.execute( - """INSERT INTO example_upgrade_presence - (id, work_id, window_no, chapter_no, entity_type, name, observation, - creator, create_time, deleted, tenant_id) - VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s)""", - (row[0], work_id, *row[1:], TENANT), - ) - - conn.execute( - "DELETE FROM example_upgrade_card_state WHERE tenant_id=%s AND work_id=%s", - (TENANT, work_id), - ) - for draft_id, watermark, update_time in snapshot["card_states"]: - conn.execute( - """INSERT INTO example_upgrade_card_state - (draft_id, work_id, watermark_window, update_time, tenant_id) - VALUES (%s,%s,%s,%s,%s)""", - (draft_id, work_id, watermark, update_time, TENANT), - ) - - conn.execute( - """DELETE FROM example_upgrade_audit - WHERE tenant_id=%s AND draft_id IN - (SELECT id FROM muse_knowledge_draft WHERE work_id=%s AND source_type=%s)""", - (TENANT, work_id, SOURCE_TYPE), - ) - for row in snapshot["audits"]: - conn.execute( - """INSERT INTO example_upgrade_audit - (id, draft_id, window_no, field_name, old_value, new_value, - create_time, tenant_id) - VALUES (%s,%s,%s,%s,%s,%s,%s,%s)""", - (*row, TENANT), - ) - - if snapshot["window"]: - status, error_message, updater = snapshot["window"] - conn.execute( - """UPDATE example_upgrade_window SET status=%s, error_message=%s, updater=%s - WHERE tenant_id=%s AND work_id=%s AND window_no=%s""", - (status, error_message, updater, TENANT, work_id, snapshot.get("window_no")), - ) - - -def undo_window(conn, work_id, win_no, *, from_chapter=None, to_chapter=None): - """同窗重跑先撤销；显式 redo 可附带章域，迁移清理历史无顶层审计的出场章。""" +def undo_window(conn, work_id, win_no): + """按审计与本窗来源精确撤销，供失败补偿后立即重试。""" # 与向量的互动（洞①，不改本函数逻辑，仅说明边界）：软删卡的向量靠召回 SQL 的 JOIN d.deleted=FALSE # 天然排除，无需在此动嵌入行；被回滚的更新卡向量暂时偏新（对应已撤销的内容），重跑后嵌段 - # （embed_touched_cards）按当前 payload 重算哈希、软删旧活行 + upsert 复活，自愈到正确态。 + # final apply 按当前 payload 复验后软删旧活行并 upsert，自愈到正确态。 # 还原覆写字段（倒序还原，先写的最后还原到最初旧值） rows = conn.execute( """SELECT a.draft_id, a.field_name, a.old_value FROM example_upgrade_audit a @@ -1640,28 +1740,6 @@ def undo_window(conn, work_id, win_no, *, from_chapter=None, to_chapter=None): conn.execute("DELETE FROM example_upgrade_card_state WHERE draft_id=%s", (did,)) continue tag, changed = f"[窗{win_no}] ", False - if isinstance(from_chapter, int) and isinstance(to_chapter, int) \ - and from_chapter <= to_chapter and isinstance(payload.get("出场章"), list): - old_chapters = payload["出场章"] - kept_chapters = [ - value - for value in old_chapters - if not any(from_chapter <= chapter <= to_chapter - for chapter in _int_chaps([value])) - ] - if kept_chapters != old_chapters: - # 修复前历史章没有审计：首次显式 redo 清理时补写完整旧值。 - # 后续正文处理失败会走无章域 undo，按此审计恢复到清理前状态。 - conn.execute( - """INSERT INTO example_upgrade_audit - (draft_id, window_no, field_name, old_value, new_value, tenant_id) - VALUES (%s,%s,%s,%s,%s,%s)""", - (did, win_no, TOP_LEVEL_APPEARANCE_AUDIT_FIELD, - json.dumps(old_chapters, ensure_ascii=False), - json.dumps(kept_chapters, ensure_ascii=False), TENANT), - ) - payload["出场章"] = kept_chapters - changed = True for k, v in list(payload.get("字段", {}).items()): if isinstance(v, list): # 字符串条目按 [窗N] 前缀删；里程碑对象（演变历程）按内部 _win 溯源键删—— @@ -1745,7 +1823,7 @@ def _resolve_canonical_name(name_map, aliases_by_draft, matched_name): # ── 语义判重（P1，设计稿 §8.2）：打开 v6 已设计、暂时关着的「嵌入近邻 + M3 终判」那级 ── # 治病根 4：机械判重只比名字字符串，改名（"影杀者"→"IV代纯机械机甲·影杀者"）/跨型指代就漏并。 # 默认关（--semantic-dedup 开启）：连接三段式（洞①）——嵌入/召回/终判在预判段(prejudge_semantic)无长 -# 连接完成，写段只查预判 verdicts；边抽边嵌（嵌段 embed_touched_cards）保后窗能语义召回前窗刚长成的卡。 +# 连接完成，写段只查预判 verdicts；final apply 后窗即可召回前窗刚长成的卡。 def _entity_embed_text(ent): """判重查询侧嵌入文本：【型】名称：摘要 + 关键字段摘选。 @@ -1958,187 +2036,120 @@ def _payload_embed_hash(payload): return hashlib.sha256(f"{text}|{EMBED_MODEL}".encode()).hexdigest() -def embed_touched_cards(sess, work_id, touched): - """嵌段（洞① 三段式连接）：窗事务 commit **之后**，把本窗新建/更新的卡增量嵌入落库， - 让后窗语义判重能召回前窗刚长成的卡（治「起了对不上正文的名→机械预扫认不出→成长线静默断」）。 - 读（短连接）：取 touched 卡最新 payload + 各卡现存活嵌入行的 content_hash 集；关连接。 - 算（无连接）：build_embed_text 构文（与检索端同源同构）→ sha256(text|MODEL) 同 embed_drafts 公式 - → 同哈希已有活行的卡跳过（内容未变，幂等）→ 批量 embed_texts。 - 写（短连接单事务）：固定顺序锁表后复验 draft 状态与当前 payload hash，再锁定同 draft - 全部活向量与同 hash 唯一行，拒绝任何 entity owner； - 再软删该卡其他仍属 draft 的旧活行； - upsert 会把旧软删卡占用的 hash 迁到当前 draft，并刷新当前嵌入全部语义列。 - 失败语义：普通网络/服务异常只 stderr 告警、不 fail 已提交窗；确定性的活跃 owner 冲突向 caller - 重抛，由 caller 把窗改为 failed 并非零退出，确保下次 run 会 undo 后重跑。返回本窗实际新嵌条数。""" - try: - # 读段：短连接取 payload + 各卡现存活嵌入哈希，读完即关 - payloads, live_hashes = {}, {} - with psycopg.connect(DSN) as conn: - for did in touched: - row = conn.execute( - "SELECT draft_payload FROM muse_knowledge_draft WHERE id=%s AND deleted=FALSE", - (did,)).fetchone() - if not row: - continue # 卡已被撤销/软删（如同窗回滚）——跳过不嵌 - payloads[did] = row[0] - live_hashes[did] = {h for (h,) in conn.execute( - "SELECT content_hash FROM example_knowledge_embedding " - "WHERE tenant_id=%s AND draft_id=%s AND deleted=FALSE", (TENANT, did)).fetchall()} - # 算段：构文 + 哈希 + 跳过内容未变者 + 批量嵌入（全程无连接） - todo = [] # [(did, text, hash)…] - for did, payload in payloads.items(): - text = build_embed_text(payload or {}) - h = _payload_embed_hash(payload) - if h not in live_hashes.get(did, set()): - todo.append((did, text, h)) - if not todo: - return 0 - # owner 预检必须覆盖全部 todo，放在 embed_texts 之前；即使服务随后把候选标 bad， - # entity/活跃 draft 冲突也已经硬停，不能因 ready 为空而返回假成功。 - with psycopg.connect(DSN) as conn: - for did, _, content_hash in todo: - _assert_embedding_owner_available(conn, did, content_hash) - # 唯一键是 tenant+content_hash+model；同批两个活跃 draft 文本完全相同时，数据库无法同时表达 - # 两个 owner。必须在发嵌入和写库前失败关闭，禁止后处理者按循环顺序抢走前者的唯一行。 - hash_owners = {} - for did, _, content_hash in todo: - hash_owners.setdefault(content_hash, []).append(did) - duplicate_hashes = { - content_hash: draft_ids - for content_hash, draft_ids in hash_owners.items() - if len(draft_ids) > 1 - } - if duplicate_hashes: - raise EmbeddingOwnershipConflict(f"活跃 touched 卡出现同 hash 争用：{duplicate_hashes}") - vecs, bad = embed_texts(sess, [t for _, t, _ in todo]) - ready = [ - (did, text, content_hash, vecs[index]) - for index, (did, text, content_hash) in enumerate(todo) - if index not in bad and index < len(vecs) and vecs[index] is not None - ] - if not ready: - return 0 - # 写段：短连接单事务——先锁定并校验唯一 owner，再淘汰旧活行（软删）+ upsert 新行。 - done = 0 - with psycopg.connect(DSN) as conn: - # 写事务先按固定表顺序取得 ROW EXCLUSIVE 锁，再执行任何行锁或写入。 - conn.execute( - "LOCK TABLE muse_knowledge_draft, example_knowledge_embedding IN ROW EXCLUSIVE MODE" +def prepare_touched_cards(sess, touched): + """短读后关闭连接，再发嵌入 HTTP；返回全部 touched 快照供 final 独立复验。""" + + snapshots = {} + with psycopg.connect(DSN) as conn: + for did in sorted(touched): + row = conn.execute( + """SELECT draft_payload, revision FROM muse_knowledge_draft + WHERE id=%s AND tenant_id=%s AND status='pending' + AND source_type=%s AND deleted=FALSE""", + (did, TENANT, SOURCE_TYPE), + ).fetchone() + if not row: + raise EmbeddingOwnershipConflict(f"待嵌入 draft 不可写：draft={did}") + live = {h for (h,) in conn.execute( + """SELECT content_hash FROM example_knowledge_embedding + WHERE tenant_id=%s AND draft_id=%s AND deleted=FALSE""", + (TENANT, did), + ).fetchall()} + snapshots[did] = (row[0], row[1], live) + prepared = [] + todo = [] + for did, (payload, revision, live_hashes) in snapshots.items(): + text, content_hash = build_embed_text(payload or {}), _payload_embed_hash(payload) + item = (did, text, content_hash, revision) + prepared.append((*item, None)) + if content_hash not in live_hashes: + todo.append(item) + owners = {} + for did, _, content_hash, _, _ in prepared: + owners.setdefault(content_hash, []).append(did) + duplicates = {content_hash: ids for content_hash, ids in owners.items() if len(ids) > 1} + if duplicates: + raise EmbeddingOwnershipConflict(f"活跃 touched 卡出现同 hash 争用：{duplicates}") + if todo: + vecs, bad = embed_texts(sess, [text for _, text, _, _ in todo]) + if bad or len(vecs) != len(todo) or any(vector is None for vector in vecs): + raise RuntimeError( + f"增量嵌入未完整返回：bad={sorted(bad)}，" + f"expected={len(todo)}，actual={len(vecs)}" ) - for did, _, content_hash, _ in sorted(ready, key=lambda item: item[0]): - _assert_ready_draft_current(conn, did, content_hash) - for did in sorted({draft_id for draft_id, _, _, _ in ready}): - _assert_draft_live_embeddings_mutable(conn, did) - for did, _, content_hash, _ in sorted(ready, key=lambda item: (item[2], item[0])): - _assert_embedding_owner_available(conn, did, content_hash, lock=True) - for did, text, h, vector in ready: - # 该卡其他活行软删（内容已变、旧向量过期）——软删红线：绝不物理 DELETE - conn.execute( - """UPDATE example_knowledge_embedding SET deleted=TRUE, updater=%s - WHERE tenant_id=%s AND draft_id=%s AND deleted=FALSE - AND entity_id IS NULL AND content_hash!=%s""", - (EMBED_ACTOR, TENANT, did, h)) - # 同 hash 旧行可能仍绑定全重抽前的软删 draft；冲突时必须迁移 owner 并刷新全部当前语义列。 - upserted = conn.execute( - """INSERT INTO example_knowledge_embedding - (draft_id, content_hash, embed_text, model, dimensions, embedding, - creator, updater, tenant_id) - VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s) - ON CONFLICT (tenant_id, content_hash, model) - DO UPDATE SET draft_id=EXCLUDED.draft_id, - embed_text=EXCLUDED.embed_text, - model=EXCLUDED.model, - dimensions=EXCLUDED.dimensions, - embedding=EXCLUDED.embedding, - deleted=FALSE, - updater=EXCLUDED.updater - WHERE example_knowledge_embedding.entity_id IS NULL - AND (example_knowledge_embedding.draft_id=EXCLUDED.draft_id - OR example_knowledge_embedding.deleted=TRUE - OR NOT EXISTS ( - SELECT 1 FROM muse_knowledge_draft owner - WHERE owner.id=example_knowledge_embedding.draft_id - AND owner.deleted=FALSE)) - RETURNING draft_id""", - (did, h, text, EMBED_MODEL, EMBED_DIM, json.dumps(vector), - EMBED_ACTOR, EMBED_ACTOR, TENANT)).fetchone() - if not upserted or upserted[0] != did: - raise EmbeddingOwnershipConflict( - f"同 hash 唯一行未绑定当前 draft：hash={h}，candidate={did}" - ) - done += 1 - conn.commit() - return done - except EmbeddingOwnershipConflict as e: - # 这是确定性数据所有权冲突，不是可降级的网络抖动；必须令命令非零且禁止打印窗完成假绿。 - print(f"[嵌段阻断] work={work_id} {e}", file=sys.stderr) - raise - except Exception as e: - # 嵌段失败不牵连窗（窗已 commit done）：判重是增益、向量缺口后续批量补嵌可兜，只告警可审计 - print(f"[嵌段告警] work={work_id} 增量嵌入异常（不 fail 窗，向量缺口后续补嵌兜住）: " - f"{type(e).__name__}: {str(e)[:180]}", file=sys.stderr) - return 0 + vectors = {item[0]: vecs[index] for index, item in enumerate(todo)} + prepared = [(*item[:4], vectors.get(item[0])) for item in prepared] + return prepared -def _capture_touched_fence(conn, touched): - """捕获窗提交后 touched 卡的 revision/hash，作为冲突补偿唯一许可。""" +def apply_prepared_cards(conn, work_id, prepared): + """最终短事务内复验 payload/revision/owner 后写向量；调用者负责同事务标记 done。""" - fence = {} - for draft_id in sorted(touched): - payload, revision = _read_upgrade_draft_snapshot(conn, draft_id) - fence[draft_id] = (revision, _payload_embed_hash(payload)) - return fence - - -def _assert_touched_fence(conn, work_id, fence): - """锁定并复验提交后围栏；任何外部确认、改版、删除都禁止 undo。""" - - for draft_id in sorted(fence): + for did, _, content_hash, revision, _ in sorted(prepared, key=lambda item: item[0]): row = conn.execute( """SELECT work_id, draft_payload, status, revision, source_type, deleted - FROM muse_knowledge_draft - WHERE id=%s AND tenant_id=%s FOR UPDATE""", - (draft_id, TENANT), + FROM muse_knowledge_draft WHERE id=%s AND tenant_id=%s FOR UPDATE""", + (did, TENANT), ).fetchone() - expected_revision, expected_hash = fence[draft_id] - if not row: - raise CompensationFenceConflict(f"补偿围栏 draft 缺失：draft={draft_id}") - current_work, payload, status, revision, source_type, deleted = row - current_hash = _payload_embed_hash(payload) - if current_work != work_id or deleted or status != "pending" \ - or source_type != SOURCE_TYPE or revision != expected_revision \ - or current_hash != expected_hash: - raise CompensationFenceConflict( - f"补偿围栏冲突：draft={draft_id}，work={current_work}，status={status}，" - f"source_type={source_type}，deleted={deleted}，revision={revision}，" - f"expected_revision={expected_revision}，hash_match={current_hash == expected_hash}" - ) - - -def _persist_compensation_failed(work_id, win_no, owner_error, compensation_error): - """补偿事务失败后另开事务落永久断点，避免旧 done 被后续运行跳过。""" - - message = ( - f"{COMPENSATION_FAILED_PREFIX} owner={owner_error}; compensation={compensation_error}" - )[:500] - with psycopg.connect(DSN) as conn: + if not row or row[0] != work_id or row[2] != "pending" or row[3] != revision \ + or row[4] != SOURCE_TYPE or row[5] or _payload_embed_hash(row[1]) != content_hash: + raise EmbeddingOwnershipConflict(f"最终嵌入 draft payload/revision 已漂移：draft={did}") + for did in sorted({item[0] for item in prepared}): + _assert_draft_live_embeddings_mutable(conn, did) + for did, _, content_hash, _, _ in sorted(prepared, key=lambda item: (item[2], item[0])): + _assert_embedding_owner_available(conn, did, content_hash, lock=True) + for did, text, content_hash, _, vector in prepared: + if vector is None: + continue + conn.execute( + """UPDATE example_knowledge_embedding SET deleted=TRUE, updater=%s + WHERE tenant_id=%s AND draft_id=%s AND deleted=FALSE + AND entity_id IS NULL AND content_hash!=%s""", + (EMBED_ACTOR, TENANT, did, content_hash), + ) row = conn.execute( - """UPDATE example_upgrade_window SET status='failed', error_message=%s, - updater='upgrade' - WHERE tenant_id=%s AND work_id=%s AND window_no=%s - AND status='done' AND error_message IS NULL AND updater='upgrade' - RETURNING status""", - (message, TENANT, work_id, win_no), + """INSERT INTO example_knowledge_embedding + (draft_id, content_hash, embed_text, model, dimensions, embedding, + creator, updater, tenant_id) + VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s) + ON CONFLICT (tenant_id, content_hash, model) + DO UPDATE SET draft_id=EXCLUDED.draft_id, embed_text=EXCLUDED.embed_text, + dimensions=EXCLUDED.dimensions, embedding=EXCLUDED.embedding, + deleted=FALSE, updater=EXCLUDED.updater + WHERE example_knowledge_embedding.entity_id IS NULL + AND (example_knowledge_embedding.draft_id=EXCLUDED.draft_id + OR example_knowledge_embedding.deleted=TRUE + OR NOT EXISTS (SELECT 1 FROM muse_knowledge_draft owner + WHERE owner.id=example_knowledge_embedding.draft_id + AND owner.deleted=FALSE)) + RETURNING draft_id""", + (did, content_hash, text, EMBED_MODEL, EMBED_DIM, json.dumps(vector), + EMBED_ACTOR, EMBED_ACTOR, TENANT), ).fetchone() - if not row: - raise CompensationFenceConflict( - f"补偿 marker 窗口 CAS 冲突：work={work_id}，window={win_no}" + if not row or row[0] != did: + raise EmbeddingOwnershipConflict( + f"同 hash 唯一行未绑定当前 draft：hash={content_hash}，candidate={did}" ) + return sum(item[4] is not None for item in prepared) + + +def embed_touched_cards(sess, work_id, touched): + """兼容独立调用：严格 prepare/apply，普通异常也向上抛出，不再告警放行。""" + + prepared = prepare_touched_cards(sess, touched) + with psycopg.connect(DSN) as conn: + conn.execute( + "LOCK TABLE muse_knowledge_draft, example_knowledge_embedding IN ROW EXCLUSIVE MODE" + ) + done = apply_prepared_cards(conn, work_id, prepared) conn.commit() - return message + return done + + + + -# ── 命令 ── @click.group() def cli(): @@ -2158,535 +2169,864 @@ def windows(work_id): raise click.ClickException(str(exc)) from exc -def _run(work_id, max_windows, max_calls, model, redo_window, semantic_on): - """按窗顺序跑升格；redo 仅限当前末窗，历史窗须全书前滚重建。""" - calls = {"n": 0} # 调用计数（含敏感失败换模型的次数由 m3_json 内部消化，此处计成功轮次） - # 语义判重嵌入会话（仅开启时建；禁系统代理，走内网直连） +def _plan_window_entities(work_id, win_no, obs, name_map, presence, aliases_by_draft, + verdicts, semantic_on, milestone_types, chapter_texts): + """纯 planner：只生成新卡/alias/presence/update 计划，不接收连接也不落库。""" + + names, aliases = dict(name_map), {key: set(value) for key, value in aliases_by_draft.items()} + actions, payloads, to_update, appearances = [], {}, {}, {} + next_ref = -1 + for ent in obs.get("新名字", []): + name = (ent.get("名称") or "").strip() + kind, key = _classify_new_name(ent, names, presence) + if kind == "empty": + continue + if kind in ("merge", "alias", "substr"): + ref = names[key][0] + if kind != "merge": + canonical = _resolve_canonical_name(names, aliases, key) + actions.append(("alias", ref, canonical, name, + "ai" if kind == "alias" else "substr")) + material = f"（{'别名' if kind == 'alias' else '名称疑似同一实体'}「{name}」并入）" + else: + material = "（新名字归并）" + to_update.setdefault(ref, []).append(material + _merge_material(ent)) + continue + chapters = _int_chaps(ent.get("出场章", [])) + history = presence.get((ent.get("型", ""), name), set()) + verdict, data = verdicts.get(name, ("new", None)) if semantic_on else ("new", None) + if kind == "new" and verdict == "merge": + ref, canonical = data + actions.append(("alias", ref, canonical, name, "semantic")) + to_update.setdefault(ref, []).append( + f"（语义判重·「{name}」并入同一实体）初卡材料：{_merge_material(ent)}" + ) + continue + if kind == "new": + ref, next_ref = next_ref, next_ref - 1 + payload = _build_new_card_payload( + work_id, win_no, ent, milestone_types, + chapter_texts=chapter_texts, known_chapters=history, + ) + payloads[ref] = payload + actions.append(("new", ref, ent, history, data if verdict == "chain" else [])) + target = (ref, ent.get("型", ""), ent.get("一句话摘要", "")) + names[payload["名称"]] = target + aliases.setdefault(ref, set()).update(payload["别名"]) + for alias in payload["别名"]: + names[alias] = target + continue + for chapter in chapters: + actions.append(("presence", None, chapter, ent.get("型", ""), name, + ent.get("一句话摘要", ""))) + for item in obs.get("已知实体新信息", []): + name = (item.get("名称") or "").strip() + if name in names and item.get("观察点"): + ref = names[name][0] + to_update.setdefault(ref, []).append(item["观察点"]) + appearances.setdefault(ref, set()).update(_int_chaps(item.get("出场章"))) + for item in obs.get("纯出场", []): + name = (item.get("名称") or "").strip() + if name in names: + appearances.setdefault(names[name][0], set()).update(_int_chaps(item.get("出场章"))) + return {"actions": actions, "payloads": payloads, "to_update": to_update, + "appearances": appearances, "aliases": aliases, "names": names} + + +def _compute_entity_updates(plan, expected_revisions, contracts, title, a, b, text, + chapter_texts, call): + """分批短读 payload/revision，关闭连接后完成全部 update LLM、证据修复与归一化。""" + + outputs, snapshots = [], {} + items = sorted(plan["to_update"].items()) + for offset in range(0, len(items), UPDATE_BATCH): + batch = items[offset:offset + UPDATE_BATCH] + cards = [] + with psycopg.connect(DSN) as conn: + for ref, points in batch: + if ref < 0: + payload, revision = deepcopy(plan["payloads"][ref]), 0 + else: + payload, revision = _read_upgrade_draft_snapshot(conn, ref) + if revision != expected_revisions.get(ref): + raise UpgradeDraftWriteConflict( + f"实体 planner revision 漂移：draft={ref}，expected={expected_revisions.get(ref)}，" + f"current={revision}" + ) + snapshots[ref] = (payload, revision) + cards.append((ref, payload, points)) + try: + update, _ = call(update_prompt(contracts, title, a, b, text, cards), ("更新",)) + except RuntimeError as exc: + if "缺少必需键" not in str(exc): + raise + print(f"[宽容] 窗{a}-{b} 更新批缺键按空批放行: {str(exc)[:80]}", file=sys.stderr) + update = {"更新": []} + repair_missing_milestone_evidence( + update, title=title, a=a, b=b, text=text, chapter_texts=chapter_texts, call=call, + ) + valid = {ref for ref, _, _ in cards} + keys = {ref: {field["key"] for field in contracts.get(payload.get("type"), {}).get("字段", [])} + | {"一句话摘要"} for ref, payload, _ in cards} + for item in _normalize_entity_updates(update.get("更新") or [], valid): + item["valid_keys"] = keys[item["draft_id"]] + outputs.append(item) + return outputs, snapshots + + +def _assert_window_marker(conn, work_id, win_no, expected_stage, input_sha, state_sha): + """锁内复验 marker、inputSha 与七域 stateSha；任何漂移都禁止继续写。""" + + row = conn.execute( + """SELECT id, status, error_message FROM example_upgrade_window + WHERE tenant_id=%s AND work_id=%s AND window_no=%s FOR UPDATE""", + (TENANT, work_id, win_no), + ).fetchone() + if not row or row[1] != "processing": + raise CompensationFenceConflict(f"窗口不在 processing：work={work_id}，window={win_no}") + stage, marker_input, marker_state = _parse_upgrade_marker(row[2]) + current_input = _window_input_sha(*_capture_window_input(conn, work_id, win_no)) + current_state = _window_state_sha( + _capture_window_state(conn, work_id, row[0]), row[0] + ) + if (stage, marker_input, marker_state) != (expected_stage, input_sha, state_sha) \ + or current_input != input_sha or current_state != state_sha: + raise CompensationFenceConflict( + f"窗口 fence 漂移：stage={stage}，input={current_input == input_sha}，" + f"state={current_state == state_sha}" + ) + return row[0], row[2] + + +def _set_processing_marker(conn, work_id, win_no, stage, input_sha, window_id): + """写段末尾按当前七域状态生成 marker；status/error 更新不进入 stateSha。""" + + conn.execute( + """UPDATE example_upgrade_window SET status='processing', error_message=NULL, updater='upgrade' + WHERE tenant_id=%s AND work_id=%s AND window_no=%s""", + (TENANT, work_id, win_no), + ) + state_sha = _window_state_sha( + _capture_window_state(conn, work_id, window_id), window_id + ) + marker = _upgrade_marker(stage, input_sha, state_sha) + conn.execute( + """UPDATE example_upgrade_window SET error_message=%s, updater='upgrade' + WHERE tenant_id=%s AND work_id=%s AND window_no=%s AND status='processing'""", + (marker, TENANT, work_id, win_no), + ) + return state_sha + + +def _apply_entity_stage(conn, work_id, win_no, plan, updates, snapshots, expected_revisions, + milestone_types, chapter_texts, input_sha, window_id): + """实体短写事务：固定锁后复验输入/revision，原子落 planner 与 entity marker。""" + + _lock_upgrade_domains(conn) + start = conn.execute( + """SELECT id, status, error_message FROM example_upgrade_window + WHERE tenant_id=%s AND work_id=%s AND window_no=%s FOR UPDATE""", + (TENANT, work_id, win_no), + ).fetchone() + if not start or start[0] != window_id or start[1] not in ("pending", "failed") \ + or (start[1] == "failed" + and not str(start[2] or "").startswith("retryable-clean:")): + raise CompensationFenceConflict(f"实体提交前窗口状态非法：work={work_id}，window={win_no}") + if _window_input_sha(*_capture_window_input(conn, work_id, win_no)) != input_sha: + raise CompensationFenceConflict("实体提交前 inputSha 漂移") + existing_refs = {ref for ref in plan["to_update"] if ref > 0} + existing_refs.update(action[1] for action in plan["actions"] if action[0] == "alias" and action[1] > 0) + existing_refs.update(ref for ref in plan["appearances"] if ref > 0) + for ref in sorted(existing_refs): + _lock_upgrade_draft(conn, ref, expected_revisions[ref]) + ref_map, new_ids = {}, set() + for action in plan["actions"]: + if action[0] == "new": + _, ref, ent, history, chains = action + did, _, _ = new_card( + conn, work_id, win_no, ent, milestone_types, + chapter_texts=chapter_texts, known_chapters=history, + ) + ref_map[ref], new_ids = did, new_ids | {did} + for other, relation, name in chains: + conn.execute( + """INSERT INTO example_upgrade_audit + (draft_id, window_no, field_name, old_value, new_value, tenant_id) + VALUES (%s,%s,%s,NULL,%s,%s)""", + (did, win_no, ("串链候选:" + str(relation))[:100], + json.dumps({"对方卡号": other, "对方名称": name}, ensure_ascii=False), TENANT), + ) + elif action[0] == "alias": + _, ref, canonical, alias, verdict = action + _claim_alias(conn, work_id, canonical, alias, win_no, verdict) + elif action[0] == "presence": + _, _, chapter, entity_type, name, observation = action + conn.execute( + """INSERT INTO example_upgrade_presence + (work_id, window_no, chapter_no, entity_type, name, observation, tenant_id) + VALUES (%s,%s,%s,%s,%s,%s,%s)""", + (work_id, win_no, chapter, entity_type, name, observation, TENANT), + ) + appearances = {ref_map.get(ref, ref): set(chapters) + for ref, chapters in plan["appearances"].items()} + aliases = {ref_map.get(ref, ref): values for ref, values in plan["aliases"].items()} + touched = set(new_ids) + for update in updates: + ref, actual = update["draft_id"], ref_map.get(update["draft_id"], update["draft_id"]) + revision = 0 if ref < 0 else snapshots[ref][1] + merge_card( + conn, actual, win_no, update["变更字段"], update["别名新增"], + valid_keys=update["valid_keys"], chapter_texts=chapter_texts, + appearance_chapters=appearances.pop(actual, set()), + known_aliases=aliases.get(actual, set()), expected_revision=revision, + ) + touched.add(actual) + state_sha = _set_processing_marker(conn, work_id, win_no, "entity", input_sha, window_id) + conn.commit() + return ref_map, appearances, aliases, touched, state_sha + + +def _read_relation_snapshot(work_id, refs, onstage): + """实体提交后短读人物、剩余出场卡与完整关系集合，连接关闭后才允许关系模型调用。""" + + wanted = set(refs) | {did for _, (did, kind, _) in onstage.items() if kind == "character"} + snapshots, characters = {}, [] + with psycopg.connect(DSN) as conn: + for did in sorted(wanted): + payload, revision = _read_upgrade_draft_snapshot(conn, did) + snapshots[did] = (payload, revision) + if payload.get("type") == "character" and len(characters) < 8: + characters.append((did, payload)) + rows = conn.execute( + """SELECT id, draft_payload, revision FROM muse_knowledge_draft + WHERE tenant_id=%s AND work_id=%s AND source_type=%s AND deleted=FALSE + AND status='pending' AND draft_payload->>'type'=%s ORDER BY id""", + (TENANT, work_id, SOURCE_TYPE, RELATION_TYPE), + ).fetchall() + return characters, rows, snapshots + + +def _compute_relations(contracts, title, a, b, text, characters, relation_rows, call): + """无连接完成关系 LLM 与至多一次专用 repair。""" + + if len(characters) < 2: + return [] + relations = [(did, payload) for did, payload, _ in relation_rows] + output, _ = call(relation_prompt(contracts, title, a, b, text, characters, relations), ("关系",)) + raw = output.get("关系") or [] + valid_ids = {did for did, _ in characters} + try: + return _normalize_relation_output(raw, valid_ids) + except RelationOutputConflict: + repaired, _ = call( + relation_repair_prompt(contracts, title, a, b, text, characters, relations, raw), + ("关系",), + ) + return _normalize_relation_output(repaired.get("关系") or [], valid_ids) + + +def _apply_relation_stage(conn, work_id, win_no, relation_items, characters, relation_rows, + snapshots, appearances, aliases, chapter_texts, input_sha, + entity_state_sha, window_id): + """关系短写事务：复验 entity marker、人物 revision 与完整关系集合后写关系和剩余出场。""" + + _lock_upgrade_domains(conn) + _assert_window_marker(conn, work_id, win_no, "entity", input_sha, entity_state_sha) + for did, _ in characters: + _lock_upgrade_draft(conn, did, snapshots[did][1]) + current_rows = conn.execute( + """SELECT id, draft_payload, revision FROM muse_knowledge_draft + WHERE tenant_id=%s AND work_id=%s AND source_type=%s AND deleted=FALSE + AND status='pending' AND draft_payload->>'type'=%s ORDER BY id""", + (TENANT, work_id, SOURCE_TYPE, RELATION_TYPE), + ).fetchall() + if current_rows != relation_rows: + raise CompensationFenceConflict("关系提交前完整关系集合漂移") + existing = {tuple(sorted((payload.get("甲方draft"), payload.get("乙方draft")))): (did, revision) + for did, payload, revision in relation_rows + if payload.get("甲方draft") and payload.get("乙方draft")} + names = {did: payload.get("名称") for did, payload in characters} + touched = set() + for relation in relation_items: + first, second = relation.get("甲方"), relation.get("乙方") + key = tuple(sorted((first, second))) + if key in existing: + did, revision = existing[key] + touched.add(_update_relation_card(conn, work_id, win_no, did, revision, relation)) + else: + evolution = _strip_prefix(relation.get("本窗演变", "")) + fields = dict(relation.get("其他字段") or {}) + fields["演变轨迹"] = [f"[窗{win_no}] {evolution}"] if evolution else [] + touched.add(_insert_relation_card(conn, work_id, win_no, { + "type": RELATION_TYPE, "名称": f"{names[first]}×{names[second]}", + "甲方draft": first, "乙方draft": second, "甲方名称": names[first], + "乙方名称": names[second], "关系类型": relation.get("关系类型", ""), + "字段": fields, "来源": f"升格@窗{win_no}", "状态": "草稿", + "目标库": "本书作品库", "可见范围": "本书私有", "_work_id": work_id, + })) + for did, chapters in appearances.items(): + payload, revision = _lock_upgrade_draft(conn, did, snapshots[did][1]) + if _append_verified_appearance_chapters( + conn, did, win_no, payload, chapters, chapter_texts, + known_aliases=aliases.get(did, set())): + _write_locked_upgrade_draft(conn, did, payload, revision) + touched.add(did) + state_sha = _set_processing_marker(conn, work_id, win_no, "relation", input_sha, window_id) + conn.commit() + return touched, state_sha + + +def _finalize_window(work_id, win_no, input_sha, relation_state_sha, prepared): + """最终短事务原子完成向量写入与 done；semantic 关闭时 prepared 为空直接完成。""" + + with psycopg.connect(DSN) as conn: + _lock_upgrade_domains(conn) + _, marker = _assert_window_marker( + conn, work_id, win_no, "relation", input_sha, relation_state_sha, + ) + embedded = apply_prepared_cards(conn, work_id, prepared) + row = conn.execute( + """UPDATE example_upgrade_window SET status='done', error_message=NULL, updater='upgrade' + WHERE tenant_id=%s AND work_id=%s AND window_no=%s + AND status='processing' AND error_message=%s RETURNING status""", + (TENANT, work_id, win_no, marker), + ).fetchone() + if not row: + raise CompensationFenceConflict("最终 done CAS 冲突") + conn.commit() + return embedded + + +def _window_artifact_counts(conn, work_id, win_no): + """统计阻断产物；仅机械证明的 upgrade-reset 墓碑允许豁免。""" + + values = {} + statements = { + "audits": ("SELECT count(*) FROM example_upgrade_audit a JOIN muse_knowledge_draft d " + "ON d.id=a.draft_id WHERE a.tenant_id=%s AND d.work_id=%s AND a.window_no=%s", + (TENANT, work_id, win_no)), + "aliases": ("SELECT count(*) FROM example_upgrade_alias a WHERE a.tenant_id=%s " + "AND a.work_id=%s AND a.evidence_window=%s " + "AND NOT (a.deleted=TRUE AND a.updater='upgrade-reset')", + (TENANT, work_id, win_no)), + "presence": ("SELECT count(*) FROM example_upgrade_presence p WHERE p.tenant_id=%s " + "AND p.work_id=%s AND p.window_no=%s", + (TENANT, work_id, win_no)), + "new_cards": ("SELECT count(*) FROM muse_knowledge_draft d WHERE d.tenant_id=%s " + "AND d.work_id=%s AND d.source_type=%s AND d.draft_payload->>'来源'=%s " + "AND NOT (d.deleted=TRUE AND d.updater='upgrade-reset' AND d.status='pending')", + (TENANT, work_id, SOURCE_TYPE, f"升格@窗{win_no}")), + "card_state": ("SELECT count(*) FROM example_upgrade_card_state s " + "JOIN muse_knowledge_draft d ON d.id=s.draft_id " + "WHERE s.tenant_id=%s AND d.tenant_id=%s AND d.work_id=%s " + "AND d.source_type=%s AND s.watermark_window=%s", + (TENANT, TENANT, work_id, SOURCE_TYPE, win_no)), + "embeddings": ("SELECT count(*) FROM example_knowledge_embedding e JOIN muse_knowledge_draft d " + "ON d.id=e.draft_id WHERE e.tenant_id=%s AND d.tenant_id=%s " + "AND d.work_id=%s AND d.source_type=%s " + "AND d.draft_payload->>'来源'=%s " + "AND NOT (e.entity_id IS NULL AND e.deleted=TRUE " + "AND d.deleted=TRUE AND d.updater='upgrade-reset' AND d.status='pending')", + (TENANT, TENANT, work_id, SOURCE_TYPE, f"升格@窗{win_no}")), + } + for name, (sql, params) in statements.items(): + values[name] = conn.execute(sql, params).fetchone()[0] + return values + + +RETRYABLE_CLEAN_PREFIX = "retryable-clean:" +LEGACY_RECOVERY_CONTRACT = "recover-legacy-failed:v1" + + +def _legacy_recovery_confirmation_sha(snapshot): + """按 preview 输出的完整快照生成 execute 必须精确匹配的确认摘要。""" + + return _sha256_json({ + "contract": LEGACY_RECOVERY_CONTRACT, + "work_id": snapshot["work_id"], + "window": snapshot["window"], + "artifact_counts": snapshot["artifact_counts"], + "processing_count": snapshot["processing_count"], + }) + + +def _capture_legacy_failed_snapshot(conn, work_id, win_no, *, lock=False): + """在当前连接读取 legacy failed 恢复快照；execute 调用方先固定表锁。""" + + lock_clause = " FOR UPDATE" if lock else "" + row = conn.execute( + """SELECT id, window_no, from_chapter, to_chapter, status, error_message, deleted + FROM example_upgrade_window + WHERE tenant_id=%s AND work_id=%s AND window_no=%s""" + lock_clause, + (TENANT, work_id, win_no), + ).fetchone() + if not row: + raise CompensationFenceConflict(f"目标窗口不存在：work={work_id}，window={win_no}") + window = dict(zip( + ("id", "window_no", "from_chapter", "to_chapter", "status", "error_message", "deleted"), + row, + )) + if window["deleted"] or window["status"] != "failed": + raise CompensationFenceConflict( + f"目标窗口不是可恢复的 failed：status={window['status']}，deleted={window['deleted']}" + ) + error_message = str(window["error_message"] or "") + if error_message.startswith(("upgrade-fence:", COMPENSATION_FAILED_PREFIX, RETRYABLE_CLEAN_PREFIX)): + raise CompensationFenceConflict("目标窗口不是 fence 引入前的 legacy failed") + snapshot = { + "work_id": work_id, + "window": window, + "artifact_counts": _window_artifact_counts(conn, work_id, win_no), + "processing_count": conn.execute( + """SELECT count(*) FROM example_upgrade_window + WHERE tenant_id=%s AND work_id=%s AND status='processing' AND deleted=FALSE""", + (TENANT, work_id), + ).fetchone()[0], + } + snapshot["confirmation_sha"] = _legacy_recovery_confirmation_sha(snapshot) + return snapshot + + +def _echo_legacy_recovery_snapshot(snapshot, mode): + """以稳定 JSON 输出 preview/execute 结果，便于人工复制确认摘要。""" + + click.echo(json.dumps({ + "command": "recover-legacy-failed", + "mode": mode, + "window": snapshot["window"], + "artifact_counts": snapshot["artifact_counts"], + "processing_count": snapshot["processing_count"], + "confirmation_sha": snapshot["confirmation_sha"], + }, ensure_ascii=False, sort_keys=True)) + + +@cli.command("recover-legacy-failed") +@click.option("--work-id", type=int, required=True, help="目标作品 ID") +@click.option("--window-no", type=int, required=True, help="目标 legacy failed 窗号") +@click.option("--preview", is_flag=True, help="只读输出窗口、产物计数、processing 数和确认 SHA") +@click.option("--execute", is_flag=True, help="在确认参数满足后标记 retryable-clean，保持 failed") +@click.option("--confirmation-sha", help="必须精确等于 preview 输出的 confirmation_sha") +@click.option("--confirm-no-live-process", is_flag=True, + help="人工确认旧进程和数据库 backend 均已结束") +def recover_legacy_failed(work_id, window_no, preview, execute, confirmation_sha, + confirm_no_live_process): + """恢复 fence 引入前的 legacy failed 窗；不调用模型或嵌入。""" + + if preview == execute: + raise click.ClickException("必须且只能指定 --preview 或 --execute") + if preview: + with psycopg.connect(DSN) as conn: + conn.execute("SET TRANSACTION ISOLATION LEVEL REPEATABLE READ, READ ONLY") + snapshot = _capture_legacy_failed_snapshot(conn, work_id, window_no) + _echo_legacy_recovery_snapshot(snapshot, "preview") + return + if not confirmation_sha: + raise click.ClickException("--execute 必须提供 --confirmation-sha") + if not confirm_no_live_process: + raise click.ClickException("--execute 必须显式提供 --confirm-no-live-process") + try: + with upgrade_work_lock(DSN, TENANT, work_id): + with psycopg.connect(DSN) as conn: + _lock_upgrade_domains(conn) + snapshot = _capture_legacy_failed_snapshot(conn, work_id, window_no, lock=True) + if snapshot["confirmation_sha"] != confirmation_sha: + raise CompensationFenceConflict("confirmation-sha 与锁内重算结果不匹配") + if snapshot["processing_count"]: + raise CompensationFenceConflict("本书仍有 processing 窗，拒绝恢复") + if any(snapshot["artifact_counts"].values()): + raise CompensationFenceConflict( + f"本窗已有产物，拒绝恢复：{snapshot['artifact_counts']}" + ) + changed = conn.execute( + """UPDATE example_upgrade_window + SET status='failed', error_message=%s, updater='upgrade-recovery' + WHERE tenant_id=%s AND work_id=%s AND window_no=%s + AND status='failed' AND error_message=%s + RETURNING status""", + ( + f"{RETRYABLE_CLEAN_PREFIX} legacy failed recovered; " + f"confirmation-sha={confirmation_sha}", + TENANT, + work_id, + window_no, + snapshot["window"]["error_message"], + ), + ).fetchone() + if not changed: + raise CompensationFenceConflict("legacy failed 恢复 CAS 冲突") + conn.commit() + snapshot["window"]["error_message"] = ( + f"{RETRYABLE_CLEAN_PREFIX} legacy failed recovered; confirmation-sha={confirmation_sha}" + ) + _echo_legacy_recovery_snapshot(snapshot, "execute") + except UpgradeWorkLockUnavailable as exc: + raise click.ClickException(str(exc)) from exc + except CompensationFenceConflict as exc: + raise click.ClickException(str(exc)) from exc + + +def real_pg_rollback_smoke(work_id): + """在真实 public 窗表内验证 marker 写入随后 rollback，且不改变序列。""" + + try: + with upgrade_work_lock(DSN, TENANT, work_id): + with psycopg.connect(DSN) as conn: + _lock_upgrade_domains(conn) + row = conn.execute( + """SELECT id, window_no, status, error_message + FROM example_upgrade_window + WHERE tenant_id=%s AND work_id=%s AND status='pending' AND deleted=FALSE + ORDER BY window_no LIMIT 1 FOR UPDATE""", + (TENANT, work_id), + ).fetchone() + if not row: + raise RuntimeError(f"work={work_id} 没有 pending 窗，真实 smoke 明确失败") + window_id, win_no, original_status, original_error = row + original_window = conn.execute( + """SELECT to_jsonb(w) FROM example_upgrade_window w + WHERE w.tenant_id=%s AND w.work_id=%s AND w.window_no=%s""", + (TENANT, work_id, win_no), + ).fetchone()[0] + draft_row = conn.execute( + """SELECT id, revision FROM muse_knowledge_draft + WHERE tenant_id=%s AND work_id=%s AND source_type=%s + AND deleted=FALSE ORDER BY id LIMIT 1 FOR UPDATE""", + (TENANT, work_id, SOURCE_TYPE), + ).fetchone() + if not draft_row: + raise RuntimeError(f"work={work_id} 没有 active upgrade_book draft，真实 smoke 明确失败") + draft_id, original_revision = draft_row + window, chapters, schemas, work_title = _capture_window_input(conn, work_id, win_no) + input_sha = _window_input_sha(window, chapters, schemas, work_title=work_title) + state_sha = _set_processing_marker( + conn, work_id, win_no, "entity", input_sha, window_id, + ) + _assert_window_marker(conn, work_id, win_no, "entity", input_sha, state_sha) + conn.execute( + """UPDATE muse_knowledge_draft SET revision=revision+1 + WHERE tenant_id=%s AND id=%s AND source_type=%s""", + (TENANT, draft_id, SOURCE_TYPE), + ) + try: + _assert_window_marker(conn, work_id, win_no, "entity", input_sha, state_sha) + except CompensationFenceConflict: + pass + else: + raise RuntimeError("draft revision 漂移未被 _assert_window_marker 拒绝") + conn.rollback() + with psycopg.connect(DSN) as verify_conn: + restored_window = verify_conn.execute( + """SELECT to_jsonb(w) FROM example_upgrade_window w + WHERE w.tenant_id=%s AND w.work_id=%s AND w.window_no=%s""", + (TENANT, work_id, win_no), + ).fetchone()[0] + restored_revision = verify_conn.execute( + """SELECT revision FROM muse_knowledge_draft + WHERE tenant_id=%s AND id=%s AND source_type=%s""", + (TENANT, draft_id, SOURCE_TYPE), + ).fetchone()[0] + if restored_window != original_window or restored_revision != original_revision: + raise RuntimeError( + f"rollback 后公共行漂移：window_equal={restored_window == original_window}，" + f"draft_revision={restored_revision}/{original_revision}" + ) + click.echo( + f"real PG rollback smoke 通过：work={work_id} window={win_no} " + f"status={original_status} 未提交 marker" + ) + except UpgradeWorkLockUnavailable as exc: + raise click.ClickException(str(exc)) from exc + + +def _compensate_window(work_id, win_no, error): + """固定锁内 exact 补偿；非法 marker、input/state/外部状态漂移只写补偿失败断点。""" + + with psycopg.connect(DSN) as conn: + _lock_upgrade_domains(conn) + row = conn.execute( + """SELECT id, status, error_message FROM example_upgrade_window + WHERE tenant_id=%s AND work_id=%s AND window_no=%s FOR UPDATE""", + (TENANT, work_id, win_no), + ).fetchone() + if not row: + raise CompensationFenceConflict("补偿窗口不存在") + window_id, status, marker = row + if status == "processing": + try: + _, expected_input, expected_state = _parse_upgrade_marker(marker) + current_input = _window_input_sha(*_capture_window_input(conn, work_id, win_no)) + current_state = _window_state_sha( + _capture_window_state(conn, work_id, window_id), window_id + ) + if (current_input, current_state) != (expected_input, expected_state): + raise CompensationFenceConflict("补偿 input/state 漂移") + undo_window(conn, work_id, win_no) + except Exception as exc: + message = f"{COMPENSATION_FAILED_PREFIX} {type(exc).__name__}: {exc}"[:500] + changed = conn.execute( + """UPDATE example_upgrade_window SET status='failed', error_message=%s, updater='upgrade' + WHERE tenant_id=%s AND work_id=%s AND window_no=%s + AND status='processing' AND error_message=%s RETURNING status""", + (message, TENANT, work_id, win_no, marker), + ).fetchone() + if not changed: + raise CompensationFenceConflict("补偿失败 marker CAS 冲突") from exc + conn.commit() + raise CompensationFenceConflict(message) from exc + elif any(_window_artifact_counts(conn, work_id, win_no).values()): + message = f"{COMPENSATION_FAILED_PREFIX} 实体提交前无 marker，但本窗产物非零"[:500] + changed = conn.execute( + """UPDATE example_upgrade_window SET status='failed', error_message=%s, updater='upgrade' + WHERE tenant_id=%s AND work_id=%s AND window_no=%s + AND status=%s AND error_message IS NOT DISTINCT FROM %s RETURNING status""", + (message, TENANT, work_id, win_no, status, marker), + ).fetchone() + if not changed: + raise CompensationFenceConflict("无 marker 补偿失败 CAS 冲突") + conn.commit() + raise CompensationFenceConflict(message) + conn.execute( + """UPDATE example_upgrade_window SET status='failed', error_message=%s, updater='upgrade' + WHERE tenant_id=%s AND work_id=%s AND window_no=%s""", + (("retryable-clean: " + str(error))[:500], TENANT, work_id, win_no), + ) + conn.commit() + + +def _recover_processing_windows(work_id): + """启动时先恢复 processing；exact 才 undo，漂移或非法 marker 持久化后立即停止。""" + + with psycopg.connect(DSN) as conn: + windows = conn.execute( + """SELECT window_no FROM example_upgrade_window + WHERE tenant_id=%s AND work_id=%s AND status='processing' AND deleted=FALSE + ORDER BY window_no""", + (TENANT, work_id), + ).fetchall() + for (win_no,) in windows: + _compensate_window(work_id, win_no, "processing 启动恢复") + + +def _run(work_id, max_windows, max_calls, model, redo_window, semantic_on, + *, raise_on_second_failure=True, validate_window_input=True): + """按窗顺序执行两阶段升格；默认强制章域校验并在二次失败时停止。""" + + if redo_window: + raise click.ClickException( + "--redo-window 暂不支持；请先 backup，再按 reset/rebuild 人工恢复" + ) + calls = {"n": 0} embed_sess = _embed_session() if semantic_on else None - redo_snapshot = None - redo_material = None - redo_postcommit_fence = None def call(prompt, need_keys): calls["n"] += 1 return m3_json(prompt, model, need_keys, system=IDENTITY) + try: + _recover_processing_windows(work_id) + except CompensationFenceConflict as exc: + raise click.ClickException(str(exc)) from exc + with psycopg.connect(DSN) as conn: - title = conn.execute("SELECT title FROM muse_content_work WHERE id=%s", - (work_id,)).fetchone()[0] + title = conn.execute( + "SELECT title FROM muse_content_work WHERE id=%s", (work_id,) + ).fetchone()[0] contracts = load_entity_contracts(conn) - # 洞② 登场兜底所需：含「演变历程」字段的型集合（据库内合同动态判定，不硬编码型名—— - # schema 未铺该字段的型自动不触发兜底）；供 new_card 判定是否补登场里程碑。 - milestone_types = {t for t in ENTITY_TYPES - if any(f.get("key") == "演变历程" - for f in contracts.get(t, {}).get("字段", []))} - if redo_window: - active_windows = conn.execute( - """SELECT window_no, from_chapter, to_chapter FROM example_upgrade_window - WHERE tenant_id=%s AND work_id=%s AND deleted=FALSE - ORDER BY window_no""", (TENANT, work_id)).fetchall() - target = next((row for row in active_windows if row[0] == redo_window), None) - if target is None: - raise click.ClickException(f"redo 窗{redo_window}不存在") - for index, row in enumerate(active_windows, 1): - win_no, from_chapter, to_chapter = row - if win_no != index: - raise click.ClickException( - f"active 窗口号不连续：期望窗{index}，实际窗{win_no}") - if from_chapter is None or to_chapter is None \ - or from_chapter <= 0 or to_chapter < from_chapter: - raise click.ClickException( - f"窗{win_no}章域非法：{from_chapter}-{to_chapter}") - if index > 1 and from_chapter != active_windows[index - 2][2] + 1: - raise click.ClickException( - f"active 窗章域不连续：窗{win_no - 1}止于{active_windows[index - 2][2]}，" - f"窗{win_no}始于{from_chapter}") - final_window = active_windows[-1][0] - if redo_window != final_window: - raise click.ClickException( - f"仅允许重跑当前末窗{final_window}；历史窗须全书前滚重建，" - "后缀级联重算属于 P1、当前不支持") - text, chapter_texts = load_window_material(conn, work_id, target[1], target[2]) - expected_chapters = set(range(target[1], target[2] + 1)) - missing = sorted(expected_chapters - set(chapter_texts)) - empty = sorted(chapter for chapter in expected_chapters - if chapter in chapter_texts and not str(chapter_texts[chapter]).strip()) - if missing or empty or not text.strip(): - raise click.ClickException( - f"末窗{redo_window}正文不完整：缺章{missing or '无'}，空正文章{empty or '无'}") - wins = [target] - redo_material = (text, chapter_texts) - redo_snapshot = _snapshot_redo_window(conn, work_id, redo_window) - click.echo(f"[撤销] 窗{redo_window} 旧写入回滚中…") - undo_window( - conn, - work_id, - redo_window, - from_chapter=target[1], - to_chapter=target[2], + wins = conn.execute( + """SELECT window_no, from_chapter, to_chapter, status, error_message + FROM example_upgrade_window + WHERE tenant_id=%s AND work_id=%s AND status!='done' AND deleted=FALSE + ORDER BY from_chapter""", + (TENANT, work_id), + ).fetchall() + + for win_no, _, _, status, error_message in wins: + if status == "failed" and not str(error_message or "").startswith("retryable-clean:"): + raise click.ClickException( + f"legacy failed 窗{win_no} 禁止自动恢复；窗61等历史窗口须人工恢复：" + f"{str(error_message or '')[:200]}" ) - else: - wins = conn.execute( - """SELECT window_no, from_chapter, to_chapter, status FROM example_upgrade_window - WHERE tenant_id=%s AND work_id=%s AND status!='done' AND deleted=FALSE - ORDER BY from_chapter""", (TENANT, work_id)).fetchall() - wins = [(r[0], r[1], r[2]) for r in wins] - # failed 窗重跑前必须先撤销旧写入（部分写入直接重跑会 double-append 追加字段） - for r0 in conn.execute( - """SELECT window_no, error_message FROM example_upgrade_window - WHERE tenant_id=%s AND work_id=%s AND status='failed' AND deleted=FALSE""", - (TENANT, work_id)).fetchall(): - if str(r0[1] or "").startswith(COMPENSATION_FAILED_PREFIX): - raise click.ClickException( - f"failed 窗{r0[0]} 存在未完成补偿断点，禁止自动 undo：{r0[1]}" - ) - click.echo(f"[撤销] failed 窗{r0[0]} 旧写入回滚后重跑") - undo_window(conn, work_id, r0[0]) + + milestone_types = { + entity_type + for entity_type in ENTITY_TYPES + if any( + field.get("key") == "演变历程" + for field in contracts.get(entity_type, {}).get("字段", []) + ) + } done_n = 0 - # 串行铁律（创始人 2026-07-15 拍板）：窗与窗是串行生长——后窗的预扫/判重/更新 - # 全依赖前窗长成的卡。窗失败绝不跳窗（跳窗=知识断层+事后补跑有覆盖风险）， - # 而是当场撤销半写入→整窗重试一次（挡偶发病）；仍败→停书，断点就在本窗， - # 下次启动从本窗续跑（failed 先撤销机制），串行语义天然无损。 - wi = 0 - retried = False # 当前窗是否已当场重试过 - while wi < len(wins): - win_no, a, b = wins[wi] + for win_no, a, b, _, _ in wins: if max_windows and done_n >= max_windows: break - if max_calls and calls["n"] >= max_calls and not retried: + if max_calls and calls["n"] >= max_calls: click.echo(f"⏸ 调用闸 {max_calls} 已到，停在窗{win_no} 之前") break - try: - # ── 读1（短连接）：窗正文 + 判重底册，读完即关（三段式红线：连接不跨 LLM/嵌入调用存活）── - with psycopg.connect(DSN) as conn: - if redo_material is not None: - text, chapter_texts = redo_material - else: - text, chapter_texts = load_window_material(conn, work_id, a, b) - name_map, presence, aliases_by_draft, expected_revisions = load_known(conn, work_id) - onstage = prescan(name_map, text) # ⓪ 机械预扫纯内存，无连接 - # ── 算1（无连接）：① 实体观察 M3 调用 + 输出清洗，全程不持连接 ── - obs, usage = call(observe_prompt(contracts, title, a, b, text, onstage), - ("新名字", "已知实体新信息", "纯出场")) - repair_missing_milestone_evidence( - obs, - title=title, - a=a, - b=b, - text=text, - chapter_texts=chapter_texts, - call=call, - ) - # 模型输出防御（深空窗5实测：列表元素偶为裸字符串，.get 直接炸）——统一只留 dict 元素 - for k in ("新名字", "已知实体新信息", "纯出场"): - obs[k] = [x for x in (obs.get(k) or []) if isinstance(x, dict)] - # 新实体的出场章参与跨章立卡判定，必须在预判前先按规范名/合法别名逐章实证； - # 否则模型虚报两章会把单章实体误送入立卡路径，并污染后续语义判重候选。 - for ent in obs["新名字"]: - ent["出场章"] = sorted( - _filter_entity_chapters( - ent.get("名称"), - ent.get("别名"), - ent.get("出场章"), - chapter_texts, - ) - ) - # ── 预判段（仅 semantic_on，无长连接）：粗筛将立卡候选→批量嵌入→短连接召回→M3 终判，产出 - # verdicts 供写段查表（洞①：把原窗内 semantic_dedup(conn,…) 从连接存活期整体挪走）── - verdicts, dedup_merge_n, dedup_chain_n = {}, 0, 0 - if semantic_on: - verdicts, dedup_merge_n, dedup_chain_n = prejudge_semantic( - obs, name_map, presence, embed_sess, work_id, call) - # ── 写（窗事务，保持现单事务原子性）：②-⑥ 照旧，唯一改动=判重换查预判 verdicts；update/relation - # 的 M3 仍在窗事务内——既有已接受模式（keepalives 兜底），本次明确不动其连接边界 ── - new_ids = [] # 本窗新立卡 id（洞①：与 to_update 键并成 touched 供嵌段增量嵌入） - touched_fence = {} - with psycopg.connect(DSN) as conn: - # ② 判重+立卡门槛（机械；分类走 _classify_new_name 纯函数，与预判段同判据防两处漂移） - to_update = {} # draft_id -> [观察点…] - for ent in obs.get("新名字", []): - nm = (ent.get("名称") or "").strip() - kind, key = _classify_new_name(ent, name_map, presence) - if kind == "empty": - continue - if kind == "merge": # 观察漏看在场清单：直接归并（洞②：材料带全里程碑） - did = name_map[key][0] - to_update.setdefault(did, []).append(f"（新名字归并）{_merge_material(ent)}") - continue - if kind == "alias": # 疑似别名：初卡转观察材料（G3；洞②材料不截里程碑） - did = name_map[key][0] - canonical_name = _resolve_canonical_name(name_map, aliases_by_draft, key) - # alias 是分类副作用，也必须先通过模型所见目标卡的 pending/source/revision 门禁。 - _lock_upgrade_draft(conn, did, expected_revisions[did]) - to_update.setdefault(did, []).append( - f"（别名「{nm}」并入）初卡材料：{_merge_material(ent)}") - legal_alias = _claim_alias( - conn, work_id, canonical_name, nm, win_no, "ai" - ) - if legal_alias: - aliases_by_draft.setdefault(did, set()).add(legal_alias) - continue - if kind == "substr": # 同型名称互为子串（「果子」vs「开心果子」）： - # 不直接立卡，转观察材料并入既有卡由更新步 AI 甄别（软防护，留复核标记；洞②材料不截里程碑） - did = name_map[key][0] - canonical_name = _resolve_canonical_name(name_map, aliases_by_draft, key) - _lock_upgrade_draft(conn, did, expected_revisions[did]) - to_update.setdefault(did, []).append( - f"（名称疑似同一实体「{nm}」≈「{key}」，请甄别后再并入）" - f"初卡材料：{_merge_material(ent)}") - legal_alias = _claim_alias( - conn, work_id, canonical_name, nm, win_no, "substr" - ) - if legal_alias: - aliases_by_draft.setdefault(did, set()).add(legal_alias) - continue - # kind in ("new","presence")：跨章立卡 / 单章龙套留档 - chaps = _int_chaps(ent.get("出场章", [])) # 归一化 int（窗113 修复） - hist = presence.get((ent.get("型", ""), nm), set()) - if kind == "new": # 跨章（含跨窗合计）→ 立卡 - # 语义判重（洞①）：判据已在预判段（三段式）算好，此处只查表——预判按 读1 时 name_map - # 近似超集粗筛、未模拟本段中途登记，多算的候选此处走归并/留档自然弃用（写段判据权威）。 - if semantic_on: - verdict, vd = verdicts.get(nm, ("new", None)) - if verdict == "merge": # 同型同一实体：并入既有卡，不另立 - did0, canon = vd - _lock_upgrade_draft(conn, did0, expected_revisions[did0]) - to_update.setdefault(did0, []).append( - f"（语义判重·「{nm}」并入同一实体）初卡材料：{_merge_material(ent)}") - legal_alias = _claim_alias( - conn, work_id, canon, nm, win_no, "semantic" - ) - if legal_alias: - aliases_by_draft.setdefault(did0, set()).add(legal_alias) - continue - if verdict == "chain": # 前身后继：仍立卡，串链关系记候选审计 - did, canonical_name, card_aliases = new_card( - conn, - work_id, - win_no, - ent, - milestone_types, - chapter_texts=chapter_texts, - known_chapters=hist, - ) - new_ids.append(did) - _, expected_revisions[did] = _read_upgrade_draft_snapshot(conn, did) - target = (did, ent.get("型", ""), ent.get("一句话摘要", "")) - name_map[canonical_name] = target - for card_alias in card_aliases: - name_map[card_alias] = target - aliases_by_draft.setdefault(did, set()).add(card_alias) - # 只记候选提示、不自动写「前身/后继」字段——避免误串，链接由人工/后续确认落字段 - for did2, rel, nm2 in vd: - conn.execute( - """INSERT INTO example_upgrade_audit - (draft_id, window_no, field_name, old_value, new_value, tenant_id) - VALUES (%s,%s,%s,NULL,%s,%s)""", - (did, win_no, ("串链候选:" + str(rel))[:100], - json.dumps({"对方卡号": did2, "对方名称": nm2}, - ensure_ascii=False), TENANT)) - continue - did, canonical_name, card_aliases = new_card( - conn, - work_id, - win_no, - ent, - milestone_types, - chapter_texts=chapter_texts, - known_chapters=hist, - ) - new_ids.append(did) - _, expected_revisions[did] = _read_upgrade_draft_snapshot(conn, did) - target = (did, ent.get("型", ""), ent.get("一句话摘要", "")) - name_map[canonical_name] = target - for card_alias in card_aliases: - name_map[card_alias] = target - aliases_by_draft.setdefault(did, set()).add(card_alias) - else: # 单章龙套 → 留档（G4）；无正文实证章则不造留档 - for ch in chaps: - conn.execute( - """INSERT INTO example_upgrade_presence - (work_id, window_no, chapter_no, entity_type, name, - observation, tenant_id) - VALUES (%s,%s,%s,%s,%s,%s,%s)""", - (work_id, win_no, ch, ent.get("型", ""), nm, - ent.get("一句话摘要", ""), TENANT)) - pres_add = {} # did -> set(出场章)——被更新的卡也要记出场章（抽检#2：只走纯出场路径整窗丢章） - for it in obs.get("已知实体新信息", []): - nm = (it.get("名称") or "").strip() - if nm in name_map and it.get("观察点"): - did = name_map[nm][0] - to_update.setdefault(did, []).append(it["观察点"]) - pres_add.setdefault(did, set()).update(_int_chaps(it.get("出场章"))) - # 在任何实体更新写入前汇总纯出场，确保同卡全部章节由一次 merge 或机械收尾消费。 - for it in obs.get("纯出场", []): - nm = (it.get("名称") or "").strip() - if nm in name_map: - pres_add.setdefault(name_map[nm][0], set()).update( - _int_chaps(it.get("出场章")) - ) - # ④ 卡更新（分批≤6） - items = sorted(to_update.items()) - for i in range(0, len(items), UPDATE_BATCH): - batch = items[i:i + UPDATE_BATCH] - cards = [] - batch_revisions = {} - for did, obs_pts in batch: - p, prompt_revision = _read_upgrade_draft_snapshot(conn, did) - batch_revisions[did] = prompt_revision - expected_revisions[did] = prompt_revision - cards.append((did, p, obs_pts)) - # 缺"更新"键宽容为空批：prompt 教"无变化的卡不输出"，某批恰好全无变化时 - # 模型会顺势连键一起省（批7实测 6 窗全死于此）。语义上缺键≈空批，按空批放行 - # 并留警告日志可审计；其余格式错误（乱码/解析失败）仍原样抛、窗照 fail。 - try: - upd, _ = call(update_prompt(contracts, title, a, b, text, cards), ("更新",)) - except RuntimeError as e: - if "缺少必需键" not in str(e): - raise - print(f"[宽容] 窗{win_no} 更新批缺键按空批放行: {str(e)[:80]}", file=sys.stderr) - upd = {"更新": []} - repair_missing_milestone_evidence( - upd, - title=title, - a=a, - b=b, - text=text, - chapter_texts=chapter_texts, - call=call, - ) - valid = {d for d, _, _ in cards} - # 每卡按其型的合同 key 集校验（+一句话摘要），越合同 key 裁剪留审计 - did2keys = {d: {f["key"] for f in contracts.get(p.get("type"), {}).get("字段", [])} - | {"一句话摘要"} for d, p, _ in cards} - normalized_updates = _normalize_entity_updates(upd.get("更新") or [], valid) - for u in normalized_updates: - merge_card(conn, u["draft_id"], win_no, - u["变更字段"], u["别名新增"], - valid_keys=did2keys.get(u["draft_id"]), - chapter_texts=chapter_texts, - appearance_chapters=pres_add.pop(u["draft_id"], set()), - known_aliases=aliases_by_draft.get(u["draft_id"], set()), - expected_revision=batch_revisions[u["draft_id"]]) - # ⑤ 关系增量（核心角色=本窗有更新的 character + 在场 character，≤8） - relation_touched = set() - char_cards = [] - seen = set() - for did in list(to_update.keys()): - candidate, participant_revision = _read_upgrade_draft_snapshot(conn, did) - if candidate.get("type") == "character" and did not in seen: - p, _ = _lock_upgrade_draft(conn, did, participant_revision) - char_cards.append((did, p)) - seen.add(did) - for nm, (did, t, _) in onstage.items(): - if t == "character" and did and did not in seen and len(char_cards) < 8: - _, participant_revision = _read_upgrade_draft_snapshot(conn, did) - p, _ = _lock_upgrade_draft(conn, did, participant_revision) - char_cards.append((did, p)) - seen.add(did) - char_cards = char_cards[:8] - if len(char_cards) >= 2: - rel_rows = conn.execute( - """SELECT id, draft_payload, revision FROM muse_knowledge_draft - WHERE tenant_id=%s AND work_id=%s AND source_type=%s AND deleted=FALSE - AND status='pending' AND draft_payload->>'type'=%s""", - (TENANT, work_id, SOURCE_TYPE, RELATION_TYPE)).fetchall() - rels = [(rid, payload) for rid, payload, _ in rel_rows] - relation_revisions = {rid: revision for rid, _, revision in rel_rows} - rel_out, _ = call(relation_prompt(contracts, title, a, b, text, - char_cards, rels), ("关系",)) - id2name = {d: p.get("名称") for d, p in char_cards} - raw_relations = rel_out.get("关系") or [] - try: - relation_items = _normalize_relation_output( - raw_relations, set(id2name) - ) - except RelationOutputConflict: - # 首轮关系冲突只给模型一次关系专用重整机会；重整仍冲突时保持失败关闭， - # 交由外层既有整窗重试与停止策略处理，禁止首条/末条覆盖或静默丢弃。 - repaired_rel_out, _ = call( - relation_repair_prompt( - contracts, - title, - a, - b, - text, - char_cards, - rels, - raw_relations, - ), - ("关系",), - ) - relation_items = _normalize_relation_output( - repaired_rel_out.get("关系") or [], set(id2name) - ) - exist = {tuple(sorted((r.get("甲方draft"), r.get("乙方draft")))): (rid, r) - for rid, r in rels - if r.get("甲方draft") and r.get("乙方draft")} - for r in relation_items: - ja, yi = r.get("甲方"), r.get("乙方") - key = tuple(sorted((ja, yi))) - evo_core = _strip_prefix(r.get("本窗演变", "")) - if key in exist: - rid, rp = exist[key] - relation_touched.add( - _update_relation_card( - conn, work_id, win_no, rid, relation_revisions[rid], r - ) - ) - else: - f2 = dict(r.get("其他字段") or {}) - f2["演变轨迹"] = [f"[窗{win_no}] {evo_core}"] if evo_core else [] - rp = {"type": RELATION_TYPE, - "名称": f"{id2name[ja]}×{id2name[yi]}", - "甲方draft": ja, "乙方draft": yi, - "甲方名称": id2name[ja], "乙方名称": id2name[yi], - "关系类型": r.get("关系类型", ""), - "字段": f2, - "来源": f"升格@窗{win_no}", "状态": "草稿", - "目标库": "本书作品库", "可见范围": "本书私有", - "_work_id": work_id} - relation_touched.add( - _insert_relation_card(conn, work_id, win_no, rp) - ) - # ⑥ 机械收尾：仅处理未被更新模型消费的出场章，然后窗置 done。 - presence_touched = set() - for did, chs in pres_add.items(): - if not chs: - continue - payload, revision = _lock_upgrade_draft( - conn, did, expected_revisions[did] - ) - # 纯出场与无字段变化卡复用同一「过滤+审计追加」路径，确保 redo 能撤销上一轮顶层章号。 - changed = _append_verified_appearance_chapters( - conn, - did, - win_no, - payload, - chs, - chapter_texts, - known_aliases=aliases_by_draft.get(did, set()), - ) - if not changed: - continue - _write_locked_upgrade_draft(conn, did, payload, revision) - presence_touched.add(did) - conn.execute( - """UPDATE example_upgrade_window SET status='done', error_message=NULL, - updater='upgrade' WHERE tenant_id=%s AND work_id=%s AND window_no=%s""", - (TENANT, work_id, win_no)) - touched = set(new_ids) | set(to_update.keys()) | relation_touched | presence_touched - if semantic_on and touched: - # 在提交事务内捕获即将提交的精确版本，消除 commit 后被外部改版再采样的竞态。 - touched_fence = _capture_touched_fence(conn, touched) - if redo_snapshot is not None: - # restore 会覆盖整书六域；在提交前捕获其完整提交态，供冲突补偿 exact 复验。 - redo_postcommit_fence = _snapshot_redo_window(conn, work_id, win_no) - conn.commit() - except SensitiveHardStop as e: - with psycopg.connect(DSN) as conn: - if redo_snapshot is not None: - _restore_redo_window(conn, work_id, redo_snapshot) - else: - conn.execute( - """UPDATE example_upgrade_window SET status='failed', error_message=%s - WHERE tenant_id=%s AND work_id=%s AND window_no=%s""", - (str(e)[:500], TENANT, work_id, win_no)) - conn.commit() - click.echo(f" ⛔ 窗{win_no} 敏感降级链全失败，本书升格硬停：{e}") - return - except Exception as e: - with psycopg.connect(DSN) as conn: - conn.execute( - """UPDATE example_upgrade_window SET status='failed', error_message=%s - WHERE tenant_id=%s AND work_id=%s AND window_no=%s""", - (str(e)[:500], TENANT, work_id, win_no)) - conn.commit() - if not retried: - # 串行铁律第一层：当场撤销半写入→同窗立即重试（挡网络瞬断/偶发格式病） - with psycopg.connect(DSN) as conn: - if redo_window: - # 显式 redo 首次失败：先恢复清理前历史值，再在同一事务重新清理本窗章域供第二次尝试。 - undo_window( - conn, - work_id, - win_no, - from_chapter=a, - to_chapter=b, - ) - else: - undo_window(conn, work_id, win_no) - conn.commit() - retried = True - click.echo(f" ↻ 窗{win_no} 失败，撤销后当场重试（串行铁律不跳窗）: {str(e)[:150]}") - continue - if redo_snapshot is not None: - # 第二次仍失败：写事务已回滚，按 redo 前完整恢复点原子恢复字段、别名、留档、初建卡与窗状态。 - with psycopg.connect(DSN) as conn: - _restore_redo_window(conn, work_id, redo_snapshot) - conn.commit() - stop_state = "redo 前旧窗状态已完整恢复" if redo_snapshot is not None \ - else "断点=本窗，续跑从此接续" - click.echo( - f" ⛔ 窗{win_no} 当场重试仍失败，本书升格停（{stop_state}）: {str(e)[:200]}" - ) - return - # ── 嵌段（仅 semantic_on 且 touched 非空；窗 commit 之后；三段式）+ 窗完成汇报 ── - # 普通服务异常仍由嵌段告警降级；确定性的唯一 owner 冲突必须把已提交窗补偿标为 failed， - # 让下次 run 先 undo 后重跑，并在打印窗完成勾号前非零退出。 - embed_n = 0 - if semantic_on and touched: + + first_error = None + for attempt in range(2): try: - embed_n = embed_touched_cards(embed_sess, work_id, touched) - except EmbeddingOwnershipConflict as exc: - error_message = f"嵌入唯一 owner 冲突：{exc}"[:500] - # 先把 done 改成 durable 失败断点；补偿成功后普通窗会覆盖成最终 failed， - # redo 会恢复原窗状态。若补偿异常，此 marker 天然保留，不依赖第二次数据库写入。 - try: - durable_message = _persist_compensation_failed( - work_id, win_no, exc, "pending" + with psycopg.connect(DSN) as conn: + window, chapters, schemas, input_title = _capture_window_input( + conn, work_id, win_no, validate=validate_window_input, ) - except Exception as marker_exc: - raise click.ClickException( - f"嵌入冲突后无法持久化补偿断点：{marker_exc}" - ) from marker_exc + text, chapter_texts = load_window_material(conn, work_id, a, b) + name_map, presence, aliases_by_draft, expected_revisions = load_known( + conn, work_id + ) + if window["from_chapter"] != a or window["to_chapter"] != b: + raise CompensationFenceConflict(f"窗{win_no}章域在读取期间漂移") + captured_texts = _aggregate_window_chapter_texts(chapters) + if validate_window_input and captured_texts != chapter_texts: + raise CompensationFenceConflict(f"窗{win_no}正文在读取期间漂移") + input_sha = _window_input_sha(window, chapters, schemas, work_title=input_title) + window_id = window["id"] + model_title = input_title + + onstage = prescan(name_map, text) + obs, _ = call( + observe_prompt(contracts, model_title, a, b, text, onstage), + ("新名字", "已知实体新信息", "纯出场"), + ) + repair_missing_milestone_evidence( + obs, + title=model_title, + a=a, + b=b, + text=text, + chapter_texts=chapter_texts, + call=call, + ) + for key in ("新名字", "已知实体新信息", "纯出场"): + obs[key] = [item for item in (obs.get(key) or []) if isinstance(item, dict)] + for entity in obs["新名字"]: + entity["出场章"] = sorted( + _filter_entity_chapters( + entity.get("名称"), + entity.get("别名"), + entity.get("出场章"), + chapter_texts, + ) + ) + + verdicts, dedup_merge_n, dedup_chain_n = {}, 0, 0 + if semantic_on: + verdicts, dedup_merge_n, dedup_chain_n = prejudge_semantic( + obs, name_map, presence, embed_sess, work_id, call + ) + plan = _plan_window_entities( + work_id, + win_no, + obs, + name_map, + presence, + aliases_by_draft, + verdicts, + semantic_on, + milestone_types, + chapter_texts, + ) + updates, snapshots = _compute_entity_updates( + plan, + expected_revisions, + contracts, + model_title, + a, + b, + text, + chapter_texts, + call, + ) + + with psycopg.connect(DSN) as conn: + ref_map, appearances, aliases, entity_touched, entity_state_sha = ( + _apply_entity_stage( + conn, + work_id, + win_no, + plan, + updates, + snapshots, + expected_revisions, + milestone_types, + chapter_texts, + input_sha, + window_id, + ) + ) + + planned_refs = { + ref_map.get(ref, ref) for ref in plan["to_update"] + } | set(ref_map.values()) | set(appearances) + characters, relation_rows, relation_snapshots = _read_relation_snapshot( + work_id, planned_refs, onstage + ) + relation_items = _compute_relations( + contracts, + model_title, + a, + b, + text, + characters, + relation_rows, + call, + ) + with psycopg.connect(DSN) as conn: + relation_touched, relation_state_sha = _apply_relation_stage( + conn, + work_id, + win_no, + relation_items, + characters, + relation_rows, + relation_snapshots, + appearances, + aliases, + chapter_texts, + input_sha, + entity_state_sha, + window_id, + ) + + touched = entity_touched | relation_touched + prepared = prepare_touched_cards(embed_sess, touched) if semantic_on else [] + embed_n = _finalize_window( + work_id, win_no, input_sha, relation_state_sha, prepared + ) + except Exception as exc: + first_error = first_error or exc try: - with psycopg.connect(DSN) as conn: - if redo_snapshot is not None: - # redo restore 覆盖整书六域，必须阻写并 exact 复验完整提交态。 - expected_redo_fence = deepcopy(redo_postcommit_fence) - expected_redo_fence["window"] = ( - "failed", durable_message, "upgrade" - ) - _assert_redo_postcommit_fence( - conn, work_id, expected_redo_fence - ) - _restore_redo_window(conn, work_id, redo_snapshot) - else: - # 普通前滚只会撤销本窗写入，维持 touched draft 围栏边界。 - _assert_touched_fence(conn, work_id, touched_fence) - # 普通窗必须在同一事务先撤销五域，再标 failed，禁止留下已提交半窗。 - undo_window(conn, work_id, win_no) - conn.execute( - """UPDATE example_upgrade_window SET status='failed', error_message=%s, - updater='upgrade' - WHERE tenant_id=%s AND work_id=%s AND window_no=%s""", - (error_message, TENANT, work_id, win_no), - ) - conn.commit() - except Exception as compensation_exc: - # 补偿事务回滚，先前独立提交的 marker 保留；不依赖数据库仍可写的二次更新。 + _compensate_window(work_id, win_no, exc) + except CompensationFenceConflict as compensation_exc: + raise click.ClickException(str(compensation_exc)) from compensation_exc + if attempt == 0: + click.echo( + f" ↻ 窗{win_no}失败，clean 后当场重试：{str(exc)[:150]}" + ) + continue + click.echo( + f" ⛔ 窗{win_no}第二次失败，已 clean 后停止：{str(exc)[:200]}" + ) + if raise_on_second_failure: raise click.ClickException( - f"{durable_message}; compensation={compensation_exc}"[:500] - ) from compensation_exc - raise click.ClickException(error_message) from exc - new_n = len(obs.get('新名字', [])) - extra = f" 判重:并{dedup_merge_n}/链{dedup_chain_n} 嵌入:{embed_n}" if semantic_on else "" - click.echo(f" 窗{win_no}✓ ({a}-{b}章) 在场{len(onstage)} 新名字{new_n} " - f"更新卡{len(to_update)} 调用累计{calls['n']}{extra}") - done_n += 1 - wi += 1 - retried = False + f"窗{win_no}第二次失败，当前作品已停止；请人工再次运行以使用 retryable-clean 断点：" + f" {str(exc)[:200]}" + ) from exc + return + + new_n = len(obs.get("新名字", [])) + extra = ( + f" 判重:并{dedup_merge_n}/链{dedup_chain_n} 嵌入:{embed_n}" + if semantic_on + else "" + ) + click.echo( + f" 窗{win_no}✓ ({a}-{b}章) 在场{len(onstage)} 新名字{new_n} " + f"更新卡{len(plan['to_update'])} 调用累计{calls['n']}{extra}" + ) + done_n += 1 + break + click.echo(f"《{title}》本次完成 {done_n} 窗，LLM 调用 {calls['n']} 次") @@ -2698,16 +3038,22 @@ def _run(work_id, max_windows, max_calls, model, redo_window, semantic_on): help="本次 LLM 调用上限（0=不限，含敏感失败）") @click.option("--model", default="MiniMax-M3", show_default=True) @click.option("--redo-window", type=int, default=0, - help="仅允许当前末窗强制重跑；历史窗须全书前滚重建，后缀级联重算(P1)暂不支持") + help="暂不支持；历史窗口须人工 backup/reset/rebuild") @click.option("--semantic-dedup", "semantic_on", is_flag=True, help="开启语义判重(P1)：立卡前召回同书近邻+M3终判治改名/跨型漏并；" - "边抽边嵌（本窗新建/更新卡窗事务后增量嵌入落库，后窗即可召回前窗的卡）；默认关") + "最终短事务严格写入完整向量；默认关") def run(work_id, max_windows, max_calls, model, redo_window, semantic_on): """持有同书会话锁后执行升格，锁连接可安全跨越模型调用。""" + if redo_window: + raise click.ClickException( + "--redo-window 暂不支持；请先 backup，再按 reset/rebuild 人工恢复" + ) try: with upgrade_work_lock(DSN, TENANT, work_id): - return _run(work_id, max_windows, max_calls, model, redo_window, semantic_on) + return _run( + work_id, max_windows, max_calls, model, redo_window, semantic_on, + ) except UpgradeWorkLockUnavailable as exc: raise click.ClickException(str(exc)) from exc diff --git a/.claude/skills/parse-book/scripts/test_parse_upgrade_offline.py b/.claude/skills/parse-book/scripts/test_parse_upgrade_offline.py index cb20826..3584976 100644 --- a/.claude/skills/parse-book/scripts/test_parse_upgrade_offline.py +++ b/.claude/skills/parse-book/scripts/test_parse_upgrade_offline.py @@ -15,6 +15,8 @@ """ import json import hashlib +import inspect +import os import pathlib import sys from contextlib import nullcontext @@ -770,72 +772,30 @@ def test_merge_card_chapter_evidence(): check("merge-card-正文变化重跑删除旧错章", conn.payload["出场章"] == [487, 490]) -def test_redo_cleans_legacy_appearance_chapters(): - """显式 redo 清理历史无审计章域，并可由失败撤销或本窗全文重新建立。""" - # 修复前历史 payload 没有顶层审计：首次 redo 必须先清空指定窗域，再只重建正文真实出现章。 - conn = _CardConn( - { - "type": "character", - "名称": "安若雪", - "别名": [], - "字段": {}, - "出场章": [487, 489], - "_work_id": 8, - } - ) - pu.undo_window(conn, 8, 84, from_chapter=487, to_chapter=489) - check("redo-历史无审计窗内章清空", conn.payload["出场章"] == []) - # 首次失败后的即时重试走“恢复旧值后重新清理”，第二次尝试前窗内仍必须为空。 - pu.undo_window(conn, 8, 84, from_chapter=487, to_chapter=489) - check("redo-首次失败重试前重新清理", conn.payload["出场章"] == []) +def _offline_window_input_rows(start, end, state=None): + """共享 run fake 返回完整章/块输入，覆盖短写阶段的真实 fence 复验。""" - # 清理本身带旧值审计：模拟后续处理失败时的既有失败撤销，必须恢复清理前历史值。 - pu.undo_window(conn, 8, 84) - check("redo-后续失败可恢复历史章", conn.payload["出场章"] == [487, 489]) - - pu.undo_window(conn, 8, 84, from_chapter=487, to_chapter=489) - pu.merge_card( - conn, - 101, - 84, - {}, - [], - chapter_texts={ - 487: "环星防线开启，亲卫继续突进。", - 488: "同步率继续上升。", - 489: "安若雪击碎能量屏障。", - }, - appearance_chapters=[487, 488, 489], - ) - check("redo-全文重建只落真实489", conn.payload["出场章"] == [489]) - - outside_conn = _CardConn( - { - "type": "character", - "名称": "安若雪", - "别名": [], - "字段": {}, - "出场章": [486, 487, 489, 490], - "_work_id": 8, - } - ) - pu.undo_window(outside_conn, 8, 84, from_chapter=487, to_chapter=489) - check("redo-窗外章完整保留", outside_conn.payload["出场章"] == [486, 490]) - - normal_conn = _CardConn( - { - "type": "character", - "名称": "安若雪", - "别名": [], - "字段": {}, - "出场章": [487, 489], - "_work_id": 8, - } - ) - pu.undo_window(normal_conn, 8, 84) - check("redo-非显式redo不做章域清理", normal_conn.payload["出场章"] == [487, 489]) + default_text = { + 70: "安若雪状态变化。", + 71: "正文。", + } if state and (state.get("payload") or {}).get("名称") == "安若雪" else {} + return [ + ( + 10000 + order_no, + order_no, + f"第{order_no}章", + "published", + 1, + 20000 + order_no, + 1, + "scene", + 1, + default_text.get(order_no, f"离线正文{order_no}"), + ) + for order_no in range(start, end + 1) + ] class _RunConn: @@ -860,6 +820,29 @@ class _RunConn: self.state.setdefault("sql_order", []).append(normalized) if normalized.startswith("SELECT title FROM muse_content_work"): return _CardResult(("离线书",)) + if normalized.startswith( + "SELECT window_no, from_chapter, to_chapter, status, error_message"): + rows = self.state.get("windows", [(7, 70, 71)]) + status = self.state.get("window_status", "pending") + error = self.state.get("window_error") + return _CardResult(rows=[(*row[:3], status, error) for row in rows]) + if normalized.startswith("SELECT window_no FROM example_upgrade_window"): + return _CardResult(rows=[]) + if normalized.startswith("SELECT id, window_no, from_chapter, to_chapter, deleted"): + row = self.state.get("windows", [(7, 70, 71)])[0] + return _CardResult((91, *row[:3], False)) + if normalized.startswith("SELECT c.id, c.order_no, c.title, c.status, c.revision"): + row = self.state.get("windows", [(7, 70, 71)])[0] + return _CardResult(rows=_offline_window_input_rows(row[1], row[2], self.state)) + if normalized.startswith("SELECT c.order_no, COALESCE(b.content_text,'')"): + return _CardResult(rows=[]) + if normalized.startswith("SELECT s.schema_key, s.active_version_id"): + return _CardResult(rows=[]) + if normalized.startswith("SELECT to_jsonb("): + return _CardResult(rows=[]) + if normalized.startswith("SELECT id, status, error_message FROM example_upgrade_window"): + return _CardResult((91, self.state.get("window_status", "pending"), + self.state.get("window_error"))) if normalized.startswith("SELECT window_no, from_chapter, to_chapter FROM example_upgrade_window"): return _CardResult(rows=self.state.get("windows", [(7, 70, 71)])) if normalized.startswith("SELECT window_no, from_chapter, to_chapter, status FROM example_upgrade_window"): @@ -919,7 +902,7 @@ class _RunConn: self.state.setdefault("alias_rows", []).append(params) return _CardResult((params[1],)) if normalized.startswith("UPDATE example_upgrade_window SET status='failed'"): - if "RETURNING status" in normalized and ( + if "AND status='done'" in normalized and "RETURNING status" in normalized and ( self.state.get("window_status") != "done" or self.state.get("window_error") is not None or self.state.get("window_updater", "upgrade") != "upgrade"): @@ -932,10 +915,19 @@ class _RunConn: self.state.setdefault("events", []).append(("failed", self.connection_id)) if "RETURNING status" in normalized: return _CardResult(("failed",)) + if normalized.startswith("UPDATE example_upgrade_window SET status='processing'"): + self.state["window_status"] = "processing" + self.state["window_error"] = None + if normalized.startswith("UPDATE example_upgrade_window SET error_message=%s"): + self.state["window_error"] = params[0] if normalized.startswith("UPDATE example_upgrade_window SET status='done'"): self.state["window_status"] = "done" self.state["window_error"] = None self.state["window_updater"] = "upgrade" + if "RETURNING status" in normalized: + return _CardResult(("done",)) + if normalized.startswith("SELECT count(*)"): + return _CardResult((0,)) if not normalized.startswith("SELECT"): self.writes.append((normalized, params)) return _CardResult() @@ -986,6 +978,7 @@ def test_run_alias_paths_store_real_canonical_name(): )), \ patch.object(pu, "load_known", return_value=(name_map, {}, {701: {"铁壳"}}, {701: 0})), \ patch.object(pu, "m3_json", side_effect=fake_m3_json): + # 该 fake 的材料由 load_window_material 单独提供，显式跳过真实章域一致性校验。 pu._run( work_id=8, max_windows=0, @@ -993,6 +986,7 @@ def test_run_alias_paths_store_real_canonical_name(): model="MiniMax-M3", redo_window=0, semantic_on=False, + validate_window_input=False, ) check("alias-run-两条alias路径均执行", len(state.get("alias_rows", [])) == 2) check("alias-run-canonical_name始终是真实卡名", @@ -1029,6 +1023,7 @@ def test_run_new_card_registers_normalized_name_and_aliases_same_window(): )), \ patch.object(pu, "load_known", return_value=({}, {}, {}, {})), \ patch.object(pu, "m3_json", side_effect=fake_m3_json): + # 该 fake 的材料由 load_window_material 单独提供，显式跳过真实章域一致性校验。 pu._run( work_id=8, max_windows=0, @@ -1036,6 +1031,7 @@ def test_run_new_card_registers_normalized_name_and_aliases_same_window(): model="MiniMax-M3", redo_window=0, semantic_on=False, + validate_window_input=False, ) candidate_rows = [ params for params in state.get("alias_rows", []) @@ -1046,7 +1042,7 @@ def test_run_new_card_registers_normalized_name_and_aliases_same_window(): {params[1] for params in candidate_rows} == {"白色游魂"}, detail=str(candidate_rows)) -def _run_duplicate_entity_update_case(updates): +def _run_duplicate_entity_update_case(updates, *, return_error=False): """执行同一实体的重复更新输出，并返回写入口调用和窗失败证据。""" draft_id = 24473 @@ -1085,8 +1081,14 @@ def _run_duplicate_entity_update_case(updates): patch.object(pu, "m3_json", side_effect=fake_m3_json), \ patch.object(pu, "undo_window"), \ patch.object(pu, "merge_card", wraps=pu.merge_card) as merge_mock: - pu._run(8, 0, 0, "MiniMax-M3", 0, False) - return state, merge_mock + try: + # 该 fake 的材料由 load_window_material 单独提供，显式跳过真实章域一致性校验。 + pu._run(8, 0, 0, "MiniMax-M3", 0, False, validate_window_input=False) + except pu.click.ClickException as exc: + if return_error: + return state, merge_mock, exc + raise + return (state, merge_mock, None) if return_error else (state, merge_mock) def test_run_exact_and_disjoint_duplicate_entity_updates_merge_once(): @@ -1119,12 +1121,17 @@ def test_run_exact_and_disjoint_duplicate_entity_updates_merge_once(): def test_run_conflicting_duplicate_entity_updates_fail_before_any_write(): """同一实体同一字段值冲突时，整批预检必须在 merge_card 与 alias 写入前稳定失败。""" - state, merge_mock = _run_duplicate_entity_update_case([ - {"draft_id": 24473, "变更字段": {"阵营": "探索队"}, "别名新增": ["小安"]}, - {"draft_id": 24473, "变更字段": {"阵营": "敌方"}, "别名新增": ["安队"]}, - ]) - check("entity-update-冲突重复零merge写入", merge_mock.call_count == 0, - detail=str(merge_mock.call_args_list)) + state, merge_mock, second_failure = _run_duplicate_entity_update_case( + [ + {"draft_id": 24473, "变更字段": {"阵营": "探索队"}, "别名新增": ["小安"]}, + {"draft_id": 24473, "变更字段": {"阵营": "敌方"}, "别名新增": ["安队"]}, + ], + return_error=True, + ) + check("entity-update-冲突重复第二次失败CLI非零", second_failure is not None, + detail=str(second_failure)) + check("entity-update-冲突重复零merge写入", merge_mock is None or merge_mock.call_count == 0, + detail=str(merge_mock.call_args_list if merge_mock else None)) check("entity-update-冲突重复零alias写入", not state.get("alias_rows"), detail=str(state.get("alias_rows"))) check("entity-update-冲突重复两次尝试错误稳定", @@ -1150,8 +1157,27 @@ class _RelationRunConn: normalized = " ".join(query.split()) if normalized.startswith("SELECT title FROM muse_content_work"): return _CardResult(("离线关系书",)) - if normalized.startswith("SELECT window_no, from_chapter, to_chapter, status"): - return _CardResult(rows=[(8, 57, 60, "pending")]) + if normalized.startswith( + "SELECT window_no, from_chapter, to_chapter, status, error_message"): + return _CardResult(rows=[( + 8, 57, 60, getattr(self.db, "window_status", "pending"), + getattr(self.db, "window_error", None), + )]) + if normalized.startswith("SELECT window_no FROM example_upgrade_window"): + return _CardResult(rows=[]) + if normalized.startswith("SELECT id, window_no, from_chapter, to_chapter, deleted"): + return _CardResult((92, 8, 57, 60, False)) + if normalized.startswith("SELECT c.id, c.order_no, c.title, c.status, c.revision"): + return _CardResult(rows=_offline_window_input_rows(57, 60)) + if normalized.startswith("SELECT c.order_no, COALESCE(b.content_text,'')"): + return _CardResult(rows=[]) + if normalized.startswith("SELECT s.schema_key, s.active_version_id"): + return _CardResult(rows=[]) + if normalized.startswith("SELECT to_jsonb("): + return _CardResult(rows=[]) + if normalized.startswith("SELECT id, status, error_message FROM example_upgrade_window"): + return _CardResult((92, getattr(self.db, "window_status", "pending"), + getattr(self.db, "window_error", None))) if normalized.startswith("SELECT window_no, error_message FROM example_upgrade_window"): return _CardResult(rows=[]) if normalized.startswith("UPDATE example_upgrade_window SET status='failed'"): @@ -1159,10 +1185,19 @@ class _RelationRunConn: if not hasattr(self.db, "window_errors"): self.db.window_errors = [] self.db.window_errors.append(params[0]) - return _CardResult() + self.db.window_error = params[0] + return _CardResult(("failed",) if "RETURNING status" in normalized else None) + if normalized.startswith("UPDATE example_upgrade_window SET status='processing'"): + self.db.window_status = "processing" + self.db.window_error = None + if normalized.startswith("UPDATE example_upgrade_window SET error_message=%s"): + self.db.window_error = params[0] if normalized.startswith("UPDATE example_upgrade_window SET status='done'"): self.db.window_status = "done" - return _CardResult() + self.db.window_error = None + return _CardResult(("done",) if "RETURNING status" in normalized else None) + if normalized.startswith("SELECT count(*)"): + return _CardResult((0,)) if normalized.startswith("SELECT id, draft_payload, revision FROM muse_knowledge_draft") \ and "draft_payload->>'type'=%s" in normalized: return _CardResult(rows=[ @@ -1228,9 +1263,10 @@ def test_run_relation_cards_enter_touched_embedding(): patch.object(pu, "prejudge_semantic", return_value=({}, 0, 0)), \ patch.object( pu, - "embed_touched_cards", - side_effect=lambda sess, work_id, touched: embedded.append(set(touched)) or len(touched), + "prepare_touched_cards", + side_effect=lambda sess, touched: embedded.append(set(touched)) or [], ): + # 该 fake 的材料由 load_window_material 单独提供，显式跳过真实章域一致性校验。 pu._run( work_id=8, max_windows=0, @@ -1238,12 +1274,13 @@ def test_run_relation_cards_enter_touched_embedding(): model="MiniMax-M3", redo_window=0, semantic_on=True, + validate_window_input=False, ) check("relation-run-新关系取得id", 702 in db.drafts) check("relation-run-更新与新建关系均进入touched", embedded == [{701, 702}], detail=str(embedded)) -def _run_duplicate_relation_case(relations): +def _run_duplicate_relation_case(relations, *, return_error=False): """执行同一无序人物对的重复关系输出，并返回关系写调用与稳定失败证据。""" characters = { @@ -1285,19 +1322,28 @@ def _run_duplicate_relation_case(relations): patch.object(pu, "undo_window"), \ patch.object(pu, "_update_relation_card", wraps=pu._update_relation_card) as update_mock, \ patch.object(pu, "_insert_relation_card", wraps=pu._insert_relation_card) as insert_mock: - pu._run(8, 0, 0, "MiniMax-M3", 0, False) - return db, update_mock.call_count, insert_mock.call_count + try: + # 该 fake 的材料由 load_window_material 单独提供，显式跳过真实章域一致性校验。 + pu._run(8, 0, 0, "MiniMax-M3", 0, False, validate_window_input=False) + except pu.click.ClickException as exc: + if return_error: + return db, update_mock.call_count, insert_mock.call_count, exc + raise + result = (db, update_mock.call_count, insert_mock.call_count) + return (*result, None) if return_error else result def test_run_conflicting_duplicate_relations_fail_before_any_relation_write(): """同一无序实体对内容冲突时，整批预检必须在任何关系卡写入前稳定失败。""" - db, update_calls, insert_calls = _run_duplicate_relation_case([ + db, update_calls, insert_calls, second_failure = _run_duplicate_relation_case([ {"甲方": 11, "乙方": 12, "关系类型": "盟友", "本窗演变": "继续合作", "其他字段": {"当前状态": "互相信任"}}, {"甲方": 12, "乙方": 11, "关系类型": "对手", "本窗演变": "公开决裂", "其他字段": {"当前状态": "彼此敌对"}}, - ]) + ], return_error=True) + check("relation-冲突重复第二次失败CLI非零", second_failure is not None, + detail=str(second_failure)) check("relation-冲突重复零关系写入", update_calls == 0 and insert_calls == 0, detail=f"update={update_calls}, insert={insert_calls}") check("relation-冲突重复两次尝试错误稳定", @@ -1306,7 +1352,7 @@ def test_run_conflicting_duplicate_relations_fail_before_any_relation_write(): check("relation-冲突重复整窗失败", db.window_status == "failed") -def _run_relation_repair_case(first_relations, repaired_relations): +def _run_relation_repair_case(first_relations, repaired_relations, *, return_error=False): """执行关系首轮冲突与一次专用重整，返回模型调用和关系写入证据。""" characters = { @@ -1342,8 +1388,18 @@ def _run_relation_repair_case(first_relations, repaired_relations): patch.object(pu, "undo_window") as undo_mock, \ patch.object(pu, "_update_relation_card", wraps=pu._update_relation_card) as update_mock, \ patch.object(pu, "_insert_relation_card", wraps=pu._insert_relation_card) as insert_mock: - pu._run(8, 0, 0, "MiniMax-M3", 0, False) - return db, relation_prompts, update_mock.call_count, insert_mock.call_count, undo_mock.call_count + try: + # 该 fake 的材料由 load_window_material 单独提供，显式跳过真实章域一致性校验。 + pu._run(8, 0, 0, "MiniMax-M3", 0, False, validate_window_input=False) + except pu.click.ClickException as exc: + if return_error: + return ( + db, relation_prompts, update_mock.call_count, + insert_mock.call_count, undo_mock.call_count, exc, + ) + raise + result = (db, relation_prompts, update_mock.call_count, insert_mock.call_count, undo_mock.call_count) + return (*result, None) if return_error else result def test_run_relation_conflict_repair_merges_to_one_card(): @@ -1390,9 +1446,11 @@ def test_run_relation_conflict_repair_conflict_fails_window_without_relation_wri {"甲方": 12, "乙方": 11, "关系类型": "对手", "本窗演变": "公开决裂", "其他字段": {"当前状态": "敌对"}}, ] - db, prompts, update_calls, insert_calls, undo_calls = _run_relation_repair_case( - conflicting, conflicting + db, prompts, update_calls, insert_calls, undo_calls, second_failure = _run_relation_repair_case( + conflicting, conflicting, return_error=True ) + check("relation-repair-二次冲突CLI非零", second_failure is not None, + detail=str(second_failure)) relation_cards = [ row for row in db.drafts.values() if row["payload"].get("type") == pu.RELATION_TYPE @@ -1405,7 +1463,7 @@ def test_run_relation_conflict_repair_conflict_fails_window_without_relation_wri check("relation-repair-二次冲突最终整窗失败", db.window_status == "failed" and len(db.window_errors) == 2, detail=str(db.window_errors)) - check("relation-repair-二次冲突交给既有整窗重试", undo_calls == 1, + check("relation-repair-两次失败均clean后停止", undo_calls == 2, detail=f"undo_calls={undo_calls}") @@ -1461,7 +1519,11 @@ def test_run_prompt_revision_rejects_entity_and_relation_stale_results(): )), \ patch.object(pu, "m3_json", side_effect=entity_m3), \ patch.object(pu, "undo_window"): - pu._run(8, 0, 0, "MiniMax-M3", 0, False) + # 该 fake 专门验证外部漂移后的失败状态，不验证 CLI 停止策略，因此显式保留旁路。 + pu._run( + 8, 0, 0, "MiniMax-M3", 0, False, + raise_on_second_failure=False, validate_window_input=False, + ) check("prompt-revision-实体外部改版保留", entity_state["payload"]["字段"] == {"外部改版": "保留"}) check("prompt-revision-实体旧模型结果令窗失败", entity_state["window_status"] == "failed") @@ -1498,7 +1560,11 @@ def test_run_prompt_revision_rejects_entity_and_relation_stale_results(): )), \ patch.object(pu, "m3_json", side_effect=relation_m3), \ patch.object(pu, "undo_window"): - pu._run(8, 0, 0, "MiniMax-M3", 0, False) + # 该 fake 专门验证外部漂移后的失败状态，不验证 CLI 停止策略，因此显式保留旁路。 + pu._run( + 8, 0, 0, "MiniMax-M3", 0, False, + raise_on_second_failure=False, validate_window_input=False, + ) relation_fields = relation_db.drafts[701]["payload"]["字段"] check("prompt-revision-关系外部改版保留", relation_fields == {"当前状态": "旧值", "外部改版": "保留"}) @@ -1506,7 +1572,7 @@ def test_run_prompt_revision_rejects_entity_and_relation_stale_results(): def test_relation_participants_lock_before_prompt_and_reject_external_drift(): - """关系参与角色在 prompt 前逐卡锁定；确认、删除或改版后不得调用关系模型。""" + """关系短读后允许无连接 prompt，提交前改版、确认或删除必须失败关闭。""" characters = { 11: {"type": "character", "名称": "甲", "字段": {}, "_work_id": 8}, @@ -1547,8 +1613,14 @@ def test_relation_participants_lock_before_prompt_and_reject_external_drift(): )), \ patch.object(pu, "m3_json", side_effect=fake_m3_json), \ patch.object(pu, "undo_window"): - pu._run(8, 0, 0, "MiniMax-M3", 0, False) - check(f"relation-participant-{name}-prompt前失败", not relation_calls) + # 该 fake 专门验证提交前拒绝外部漂移，不验证 CLI 停止策略，因此显式保留旁路。 + pu._run( + 8, 0, 0, "MiniMax-M3", 0, False, + raise_on_second_failure=False, validate_window_input=False, + ) + expected_calls = 2 if name == "revision改版" else 1 + check(f"relation-participant-{name}-prompt后提交拒绝", + len(relation_calls) == expected_calls, detail=f"calls={len(relation_calls)}") check(f"relation-participant-{name}-不新建关系", set(db.drafts) == {11, 12} and not db.audits) @@ -1581,7 +1653,11 @@ def test_run_alias_gate_precedes_empty_update_and_deleted_alias_fails_closed(): )), \ patch.object(pu, "m3_json", side_effect=fake_m3), \ patch.object(pu, "undo_window"): - pu._run(8, 0, 0, "MiniMax-M3", 0, False) + # 该 fake 专门验证 alias gate 失败状态，不验证 CLI 停止策略，因此显式保留旁路。 + pu._run( + 8, 0, 0, "MiniMax-M3", 0, False, + raise_on_second_failure=False, validate_window_input=False, + ) check("alias-gate-空更新前版本冲突不写alias", not state.get("alias_rows")) check("alias-gate-版本冲突窗失败", state["window_status"] == "failed") @@ -1653,23 +1729,22 @@ def test_run_embedding_owner_conflict_marks_window_failed(): }], } embed_calls = [] - real_embed_touched_cards = pu.embed_touched_cards + real_apply_prepared_cards = pu.apply_prepared_cards def compensate_undo(conn, work_id, win_no): """模拟真实 undo 的五域清理，并记录必须发生在 failed 更新之前的连接身份。""" state.setdefault("events", []).append(("undo", conn.connection_id)) for values in state["artifacts"].values(): values.clear() + state.setdefault("revisions", {})[701] = 0 - def run_real_embed(sess, work_id, touched): - with patch.object(pu.psycopg, "connect", side_effect=lambda *_: _EmbeddingConn(embedding_state)), \ - patch.object(pu, "build_embed_text", return_value=embed_text), \ - patch.object( - pu, - "embed_texts", - side_effect=lambda embed_sess, texts: embed_calls.append(list(texts)) or ([[0.9]], set()), - ): - return real_embed_touched_cards(sess, work_id, touched) + def run_real_prepare(sess, touched): + embed_calls.append([embed_text]) + content_hash = hashlib.sha256(f"{embed_text}|{pu.EMBED_MODEL}".encode()).hexdigest() + return [(701, embed_text, content_hash, 0, [0.9])] + + def run_real_finalize(work_id, win_no, input_sha, relation_state_sha, prepared): + return real_apply_prepared_cards(_EmbeddingConn(embedding_state), work_id, prepared) with patch.object(pu, "upgrade_work_lock", return_value=nullcontext()), \ patch.object(pu.psycopg, "connect", side_effect=lambda *_: _RunConn(state)), \ @@ -1690,9 +1765,10 @@ def test_run_embedding_owner_conflict_marks_window_failed(): patch.object(pu, "undo_window", side_effect=compensate_undo), \ patch.object( pu, - "embed_touched_cards", - side_effect=run_real_embed, - ): + "prepare_touched_cards", + side_effect=run_real_prepare, + ), \ + patch.object(pu, "_finalize_window", side_effect=run_real_finalize): result = CliRunner().invoke(pu.cli, ["run", "--work-id", "8", "--semantic-dedup"]) check("embed-conflict-run-命令非零", result.exit_code != 0, detail=result.output) check("embed-conflict-run-status补偿为failed", state["window_status"] == "failed") @@ -1709,17 +1785,16 @@ def test_run_embedding_owner_conflict_marks_window_failed(): detail=str(state["artifacts"]), ) check("embed-conflict-run-不打印窗完成", "窗7✓" not in result.output, detail=result.output) - check("embed-old-entity-owner-新向量计算已发生", embed_calls == [[embed_text]]) + check("embed-old-entity-owner-两次尝试均先计算新向量", + embed_calls == [[embed_text], [embed_text]]) check("embed-old-entity-owner-保持活跃与绑定", embedding_state["embeddings"][0]["draft_id"] == 701 and embedding_state["embeddings"][0]["entity_id"] == 9001 and embedding_state["embeddings"][0]["deleted"] is False) -def _run_owner_conflict_case(state, *, redo_window=0, snapshot=None, - postcommit_snapshot=None, current_snapshot=None, - undo_effect=None, restore_effect=None): - """执行一个完整窗直到嵌入 owner 冲突，供补偿边界反例复用。""" +def _run_owner_conflict_case(state, *, undo_effect=None): + """执行一个完整窗直到 embedding prepare 冲突，供补偿边界反例复用。""" def fake_m3_json(prompt, model, need_keys, system=None): if need_keys == ("新名字", "已知实体新信息", "纯出场"): @@ -1734,34 +1809,20 @@ def _run_owner_conflict_case(state, *, redo_window=0, snapshot=None, return ({"更新": []}, {}) raise AssertionError(f"不应出现的离线模型调用：{need_keys}") - def owner_conflict(sess, work_id, touched): + def owner_conflict(sess, touched): callback = state.get("before_owner_conflict") if callback: callback() raise pu.EmbeddingOwnershipConflict("离线 owner 冲突") - cli_args = ["run", "--work-id", "8", "--semantic-dedup"] - if redo_window: - cli_args.extend(["--redo-window", str(redo_window)]) - snapshot_calls = {"n": 0} - - def fake_snapshot(conn, work_id, win_no): - snapshot_calls["n"] += 1 - if snapshot_calls["n"] == 1: - return deepcopy(snapshot) - if snapshot_calls["n"] == 2: - return deepcopy(postcommit_snapshot if postcommit_snapshot is not None else snapshot) - current = deepcopy( - current_snapshot if current_snapshot is not None - else postcommit_snapshot if postcommit_snapshot is not None - else snapshot - ) - # marker-first 会在补偿围栏重读前先改变窗口；fake 同步反映该受控变化。 - if isinstance(current, dict) and "window" in current: - current["window"] = ( - state.get("window_status"), state.get("window_error"), "upgrade" - ) - return current + def capture_state(conn, work_id, current_window_id): + return { + "drafts": [( + deepcopy(state.get("payload")), state.get("draft_status", "pending"), + state.setdefault("revisions", {}).get(701, 0), + )], + "aliases": [], "presence": [], "card_state": [], "audits": [], "windows": [], + } with patch.object(pu, "upgrade_work_lock", return_value=nullcontext()), \ patch.object(pu.psycopg, "connect", side_effect=lambda *_: _RunConn(state)), \ @@ -1775,12 +1836,13 @@ def _run_owner_conflict_case(state, *, redo_window=0, snapshot=None, patch.object(pu, "m3_json", side_effect=fake_m3_json), \ patch.object(pu, "_embed_session", return_value=object()), \ patch.object(pu, "prejudge_semantic", return_value=({}, 0, 0)), \ - patch.object(pu, "embed_touched_cards", side_effect=owner_conflict), \ - patch.object(pu, "_snapshot_redo_window", side_effect=fake_snapshot), \ - patch.object(pu, "undo_window", side_effect=undo_effect) as undo, \ - patch.object(pu, "_restore_redo_window", side_effect=restore_effect) as restore: - result = CliRunner().invoke(pu.cli, cli_args) - return result, undo, restore + patch.object(pu, "_capture_window_state", side_effect=capture_state), \ + patch.object(pu, "prepare_touched_cards", side_effect=owner_conflict), \ + patch.object(pu, "undo_window", side_effect=undo_effect) as undo: + result = CliRunner().invoke( + pu.cli, ["run", "--work-id", "8", "--semantic-dedup"] + ) + return result, undo def test_owner_conflict_external_confirmation_preserves_draft_and_persists_breakpoint(): @@ -1801,7 +1863,7 @@ def test_owner_conflict_external_confirmation_preserves_draft_and_persists_break state["payload"]["字段"]["人工确认"] = "必须保留" state["before_owner_conflict"] = confirm_externally - result, undo, _ = _run_owner_conflict_case(state) + result, undo = _run_owner_conflict_case(state) check("compensation-fence-外部确认后命令非零", result.exit_code != 0, detail=result.output) check("compensation-fence-绝不undo或软删外部卡", not undo.called and state["draft_status"] == "confirmed" @@ -1812,178 +1874,52 @@ def test_owner_conflict_external_confirmation_preserves_draft_and_persists_break detail=str(state)) -def test_redo_owner_conflict_restores_snapshot_without_normal_undo(): - """显式 redo 的嵌入冲突恢复 redo 前五域和窗状态，不进入普通 undo+failed。""" - - old_domains = { - "fields": {"阵营": "旧阵营"}, - "aliases": ["旧别名"], - "presence": ["旧出场"], - "states": {701: 6}, - "audits": ["旧审计"], - } - snapshot = { - "window_no": 1, - "domains": deepcopy(old_domains), - "window": ("done", None, "old"), - } - state = { - "windows": [(1, 70, 71)], - "window_status": "done", - "payload": {"type": "character", "名称": "安若雪", "字段": {}, "_work_id": 8}, - "revisions": {701: 0}, - **deepcopy(old_domains), - } - undo_calls = [] - restore_calls = [] - - def initial_redo_undo(conn, work_id, win_no, *, from_chapter=None, to_chapter=None): - undo_calls.append((win_no, from_chapter, to_chapter)) - for key in old_domains: - state[key] = {} if isinstance(old_domains[key], dict) else [] - - def restore_snapshot(conn, work_id, redo_snapshot): - restore_calls.append(redo_snapshot) - state.update(deepcopy(redo_snapshot["domains"])) - state["window_status"] = redo_snapshot["window"][0] - state["window_error"] = redo_snapshot["window"][1] - - result, _, _ = _run_owner_conflict_case( - state, - redo_window=1, - snapshot=snapshot, - undo_effect=initial_redo_undo, - restore_effect=restore_snapshot, - ) - check("redo-owner-conflict-命令非零且调用restore", result.exit_code != 0 - and restore_calls == [snapshot], detail=result.output) - check("redo-owner-conflict-仅初始章域undo不走普通补偿undo", - undo_calls == [(1, 70, 71)], detail=str(undo_calls)) - check("redo-owner-conflict-恢复原五域和窗口状态", - all(state[key] == value for key, value in old_domains.items()) - and state["window_status"] == "done" and state["window_error"] is None, - detail=str(state)) -def test_redo_full_fence_rejects_untouched_and_auxiliary_domain_drift(): - """redo 补偿覆盖的任一整书域漂移都必须阻断 restore，并保留 durable failed。""" - - postcommit = { - "window_no": 1, - "drafts": [ - (701, {"名称": "安若雪", "字段": {}}, "pending", 0, "upgrade", False), - (702, {"名称": "未参与角色", "字段": {"状态": "原值"}}, - "pending", 4, "old", False), - ], - "aliases": [(801, "安若雪", "雪姐", 1, "ai", "", "t1", "", "t2", False)], - "presence": [(901, 1, 70, "character", "路人", "出现", "", "t3", False)], - "card_states": [(701, 1, "t4"), (702, 0, "t4")], - "audits": [(1001, 701, 1, "阵营", None, '"联邦"', "t5")], - "window": ("done", None, "upgrade"), - } - cases = {} - for domain in ("drafts", "aliases", "presence", "card_states", "audits"): - drifted = deepcopy(postcommit) - if domain == "drafts": - # 702 未进入 touched；payload 与 revision 同时模拟外部 CAS 改版。 - drifted[domain][1] = ( - 702, {"名称": "未参与角色", "字段": {"状态": "外部改版"}}, - "pending", 5, "external", False, - ) - else: - drifted[domain].append(tuple([999] + ["external"] * (len(drifted[domain][0]) - 1))) - cases[domain] = drifted - status_drift = deepcopy(postcommit) - untouched = status_drift["drafts"][1] - status_drift["drafts"][1] = ( - untouched[0], untouched[1], "confirmed", *untouched[3:] - ) - cases["draft_status"] = status_drift - - for domain, current in cases.items(): - state = { - "windows": [(1, 70, 71)], - "window_status": "done", - "track_sql": True, - "payload": {"type": "character", "名称": "安若雪", "字段": {}, "_work_id": 8}, - "revisions": {701: 0}, - } - result, undo, restore = _run_owner_conflict_case( - state, - redo_window=1, - snapshot=deepcopy(postcommit), - postcommit_snapshot=deepcopy(postcommit), - current_snapshot=current, - undo_effect=lambda *args, **kwargs: None, - ) - check(f"redo-full-fence-{domain}-阻断restore", - result.exit_code != 0 and not restore.called and undo.call_count == 1, - detail=result.output) - check(f"redo-full-fence-{domain}-durable-failed", - state["window_status"] == "failed" - and str(state.get("window_error", "")).startswith(pu.COMPENSATION_FAILED_PREFIX), - detail=str(state)) - lock_sql = next((sql for sql in state.get("sql_order", []) - if sql.startswith("LOCK TABLE muse_knowledge_draft")), "") - lock_tables = ( - "muse_knowledge_draft", "example_upgrade_alias", "example_upgrade_presence", - "example_upgrade_card_state", "example_upgrade_audit", "example_upgrade_window", - ) - check(f"redo-full-fence-{domain}-固定六域阻写顺序", - all(name in lock_sql for name in lock_tables) - and [lock_sql.index(name) for name in lock_tables] - == sorted(lock_sql.index(name) for name in lock_tables), - detail=lock_sql) def test_compensation_exception_persists_durable_failed_and_next_run_stops(): - """undo/restore 异常后另事务落 failed 断点，下一次 run 必须显式阻断而非跳过。""" + """clean undo 异常必须持久化 compensation-failed，下一次 run 失败关闭。""" - for name, redo_window in (("undo异常", 0), ("restore异常", 1)): - state = { - "window_status": "pending", - "track_events": True, - "payload": {"type": "character", "名称": "安若雪", "字段": {}, "_work_id": 8}, - "revisions": {701: 0}, - } - if redo_window: - state["windows"] = [(1, 70, 71)] + state = { + "window_status": "pending", + "track_events": True, + "payload": {"type": "character", "名称": "安若雪", "字段": {}, "_work_id": 8}, + "revisions": {701: 0}, + } - def fail_compensation(*args, **kwargs): - raise RuntimeError(f"{name}离线故障") + def fail_compensation(*args, **kwargs): + raise RuntimeError("undo离线故障") - result, undo, restore = _run_owner_conflict_case( - state, - redo_window=redo_window, - snapshot={"window_no": 1, "window": ("done", None, "old")}, - undo_effect=None if redo_window else fail_compensation, - restore_effect=fail_compensation if redo_window else None, + result, undo = _run_owner_conflict_case(state, undo_effect=fail_compensation) + error_message = str(state.get("window_error", "")) + check( + "compensation-durable-undo异常持久failed", + result.exit_code != 0 + and state["window_status"] == "failed" + and error_message.startswith(pu.COMPENSATION_FAILED_PREFIX), + detail=f"output={result.output!r}, state={state!r}", + ) + check("compensation-durable-确实进入undo", undo.called) + durable_events = state.get("events", [])[-2:] + check( + "compensation-durable-failed与commit同事务", + [event for event, _ in durable_events] == ["failed", "commit"] + and durable_events[0][1] == durable_events[1][1], + detail=str(state.get("events")), + ) + + with patch.object(pu.psycopg, "connect", side_effect=lambda *_: _RunConn(state)), \ + patch.object(pu, "upgrade_work_lock", return_value=nullcontext()), \ + patch.object(pu, "load_entity_contracts", return_value=_fake_contracts()): + second = CliRunner().invoke( + pu.cli, ["run", "--work-id", "8", "--semantic-dedup"] ) - error_message = str(state.get("window_error", "")) - check(f"compensation-durable-{name}-另事务持久failed", - result.exit_code != 0 and state["window_status"] == "failed" - and error_message.startswith(pu.COMPENSATION_FAILED_PREFIX), - detail=f"output={result.output!r}, state={state!r}") - check(f"compensation-durable-{name}-确实经过目标失败点", - restore.called if redo_window else undo.called) - compensation_conn = (restore if redo_window else undo).call_args.args[0] - durable_events = state.get("events", [])[-2:] - check(f"compensation-durable-{name}-failed另事务提交", - [event for event, _ in durable_events] == ["failed", "commit"] - and durable_events[0][1] == durable_events[1][1] - and durable_events[0][1] != compensation_conn.connection_id, - detail=str(state.get("events"))) - - state["failed_windows"] = [(1 if redo_window else 7, error_message)] - with patch.object(pu, "upgrade_work_lock", return_value=nullcontext()), \ - patch.object(pu.psycopg, "connect", side_effect=lambda *_: _RunConn(state)), \ - patch.object(pu, "load_entity_contracts", return_value=_fake_contracts()), \ - patch.object(pu, "undo_window") as next_undo: - next_result = CliRunner().invoke(pu.cli, ["run", "--work-id", "8"]) - check(f"compensation-durable-{name}-下次run不跳过", - next_result.exit_code != 0 and pu.COMPENSATION_FAILED_PREFIX in next_result.output - and not next_undo.called, - detail=next_result.output) + check( + "compensation-durable-下次run立即阻断", + second.exit_code != 0 and "禁止自动恢复" in second.output, + detail=second.output, + ) def test_run_embedding_payload_drift_marks_window_failed_without_vector_writes(): @@ -2060,6 +1996,29 @@ class _EmbeddingConn: def execute(self, query, params=()): normalized = " ".join(query.split()) + if normalized.startswith("SELECT draft_payload, revision FROM muse_knowledge_draft"): + draft_id = params[0] + row = self.state["drafts"].get(draft_id) + if not row or row.get("deleted") or row.get("status", "pending") != "pending": + return _CardResult(None) + return _CardResult((row["payload"], row.get("revision", 0))) + if normalized.startswith("SELECT content_hash FROM example_knowledge_embedding"): + draft_id = params[1] + return _CardResult(rows=[ + (row["content_hash"],) for row in self.state["embeddings"] + if row.get("draft_id") == draft_id and not row.get("deleted") + ]) + if normalized.startswith( + "SELECT work_id, draft_payload, status, revision, source_type, deleted"): + draft_id = params[0] + row = self.state["drafts"].get(draft_id) + if not row: + return _CardResult(None) + return _CardResult(( + row.get("work_id", 8), row["payload"], row.get("status", "pending"), + row.get("revision", 0), row.get("source_type", pu.SOURCE_TYPE), + row.get("deleted", False), + )) self.state.setdefault("sql_order", []).append(normalized) if normalized.startswith("SELECT tenant_id, deleted, status, draft_payload"): draft = self.state["drafts"].get(params[0]) @@ -2256,211 +2215,18 @@ def test_embed_touched_rejects_existing_active_hash_owner(): and state["embeddings"][0]["deleted"] is False) -def test_run_redo_preflight_rejects_unsafe_targets(): - """所有 redo 非法输入都必须在快照、撤销与写入前失败。""" - - cases = [ - ("历史窗", [(1, 1, 2), (2, 3, 4)], 1, {1: "甲", 2: "乙"}, "仅允许重跑当前末窗"), - ("不存在", [(1, 1, 2)], 2, {}, "不存在"), - ("窗口号断裂", [(1, 1, 2), (3, 3, 4)], 3, {}, "窗口号不连续"), - ("章域断裂", [(1, 1, 2), (2, 4, 5)], 2, {}, "窗章域不连续"), - ("正文缺章", [(1, 70, 71)], 1, {70: "正文"}, "正文不完整"), - ("正文为空", [(1, 70, 71)], 1, {70: "正文", 71: " "}, "正文不完整"), - ] - for name, windows, target, chapter_texts, expected_error in cases: - state = {"windows": windows} - snapshot_calls, undo_calls, connections = [], [], [] - - def fake_connect(*_): - conn = _RunConn(state) - connections.append(conn) - return conn - - def fake_material(conn, work_id, a, b): - text = "\n".join(str(chapter_texts.get(chapter, "")) for chapter in range(a, b + 1)) - return text, chapter_texts - - with patch.object(pu, "upgrade_work_lock", return_value=nullcontext()), \ - patch.object(pu.psycopg, "connect", side_effect=fake_connect), \ - patch.object(pu, "load_entity_contracts", return_value=_fake_contracts()), \ - patch.object(pu, "load_window_material", side_effect=fake_material), \ - patch.object(pu, "_snapshot_redo_window", side_effect=lambda *args: snapshot_calls.append(args)), \ - patch.object(pu, "undo_window", side_effect=lambda *args, **kwargs: undo_calls.append((args, kwargs))): - result = CliRunner().invoke( - pu.cli, - ["run", "--work-id", "8", "--redo-window", str(target)], - ) - check( - f"redo-preflight-{name}-非零拒绝", - result.exit_code != 0 and expected_error in result.output, - detail=f"exit={result.exit_code}, output={result.output!r}", - ) - check(f"redo-preflight-{name}-无副作用", - not snapshot_calls and not undo_calls - and not any(conn.writes or conn.commits for conn in connections)) - check(f"redo-preflight-{name}-无假完成", "完成 0 窗" not in result.output) -def test_run_rejects_negative_limits_before_redo_side_effects(): - """负调用闸必须由 Click 在函数入口拒绝，即使同时请求合法末窗 redo 也不得接触数据库。""" - - for option in ("--max-windows", "--max-calls"): - with patch.object(pu.psycopg, "connect") as connect, \ - patch.object(pu, "_snapshot_redo_window") as snapshot, \ - patch.object(pu, "undo_window") as undo: - result = CliRunner().invoke( - pu.cli, - ["run", "--work-id", "8", "--redo-window", "2", option, "-1"], - ) - check( - f"negative-limit-{option}-入口非零拒绝", - result.exit_code != 0, - detail=f"exit={result.exit_code}, output={result.output!r}", - ) - check( - f"negative-limit-{option}-无snapshot与undo", - not snapshot.called and not undo.called, - ) - check(f"negative-limit-{option}-无DB调用", not connect.called) -def _run_with_failures(*, max_calls, failures): - """执行单个显式 redo 窗；前 failures 次观察调用失败，返回状态与调用数。""" - - old_state = { - "windows": [(1, 1, 69), (2, 70, 71)], - "window_status": "done", - "fields": {"阵营": "旧阵营", "经历": ["[窗2] 旧经历"]}, - "aliases": ["旧别名"], - "presence": [(2, 70, "旧龙套")], - "initial_cards": [701], - } - state = deepcopy(old_state) - calls = {"n": 0} - undo_calls = [] - - def fake_undo(conn, work_id, win_no, *, from_chapter=None, to_chapter=None): - undo_calls.append((from_chapter, to_chapter)) - # 复现旧实现边界：章域 undo 会清完整旧窗；无章域 undo 只能恢复顶层章，救不回其余旧窗状态。 - state["fields"] = {} - state["aliases"] = [] - state["presence"] = [] - state["initial_cards"] = [] - - def fake_m3_json(prompt, model, need_keys, system=None): - calls["n"] += 1 - if calls["n"] <= failures: - raise RuntimeError(f"离线失败{calls['n']}") - return ({"新名字": [], "已知实体新信息": [], "纯出场": []}, {}) - - snapshot = deepcopy(old_state) - - def fake_restore(conn, work_id, redo_snapshot): - state.clear() - state.update(deepcopy(redo_snapshot)) - - with patch.object(pu.psycopg, "connect", side_effect=lambda *_: _RunConn(state)), \ - patch.object(pu, "load_entity_contracts", return_value=_fake_contracts()), \ - patch.object(pu, "load_window_material", return_value=("正文", {70: "正文", 71: "正文"})), \ - patch.object(pu, "load_known", return_value=({}, {}, {}, {})), \ - patch.object(pu, "undo_window", side_effect=fake_undo), \ - patch.object(pu, "m3_json", side_effect=fake_m3_json), \ - patch.object(pu, "_snapshot_redo_window", return_value=snapshot), \ - patch.object(pu, "_restore_redo_window", side_effect=fake_restore): - pu._run( - work_id=8, - max_windows=0, - max_calls=max_calls, - model="MiniMax-M3", - redo_window=2, - semantic_on=False, - ) - return old_state, state, calls["n"], undo_calls -def test_run_redo_max_calls_finishes_active_retry(): - """调用闸不能在首次失败后的已清理中间态退出。""" - - _, state, call_count, undo_calls = _run_with_failures(max_calls=1, failures=1) - check("run-max-calls-已清理窗仍完成即时重试", call_count == 2) - check("run-max-calls-重试成功落done", state["window_status"] == "done") - check("run-max-calls-首次失败前后均执行章域清理", undo_calls == [(70, 71), (70, 71)]) -def test_run_redo_final_failure_restores_full_snapshot(): - """显式 redo 两次都失败时，完整恢复字段、别名、presence 与初建卡。""" - - old_state, state, call_count, _ = _run_with_failures(max_calls=0, failures=2) - check("run-redo-final-failure-确实尝试两次", call_count == 2) - check("run-redo-final-failure-完整恢复旧窗", state == old_state, detail=f"state={state!r}") -class _SnapshotConn: - """恢复点 SQL 的离线记录器：提供旧窗快照行，并记录所有恢复写入。""" - - def __init__(self): - self.writes = [] - - def execute(self, query, params=()): - normalized = " ".join(query.split()) - if normalized.startswith("SELECT id, draft_payload, status, revision, updater, deleted"): - return _CardResult(rows=[ - (701, {"字段": {"阵营": "旧阵营"}}, "confirmed", 9, "old", False) - ]) - if normalized.startswith("SELECT id, canonical_name, alias, evidence_window"): - return _CardResult(rows=[(801, "旧主角", "旧别名", 7, "ai", "", "t1", "", "t2", False)]) - if normalized.startswith("SELECT id, window_no, chapter_no, entity_type"): - return _CardResult(rows=[(901, 7, 70, "character", "旧龙套", "旧观察", "", "t3", False)]) - if normalized.startswith("SELECT draft_id, watermark_window, update_time"): - return _CardResult(rows=[(701, 7, "t4")]) - if normalized.startswith("SELECT a.id, a.draft_id, a.window_no"): - return _CardResult(rows=[(1001, 701, 7, "阵营", '"更旧阵营"', '"旧阵营"', "t5")]) - if normalized.startswith("SELECT status, error_message, updater FROM example_upgrade_window"): - return _CardResult(("done", None, "old")) - if normalized.startswith("SELECT id FROM muse_knowledge_draft"): - return _CardResult(rows=[(701,), (702,)]) - self.writes.append((normalized, params)) - return _CardResult() -def test_redo_snapshot_restore_sql_boundaries(): - """恢复点覆盖旧字段、别名、presence、初建卡、水位、审计与窗状态。""" - - conn = _SnapshotConn() - snapshot = pu._snapshot_redo_window(conn, 8, 7) - check("redo-snapshot-捕获完整旧卡", snapshot["drafts"][0][1]["字段"]["阵营"] == "旧阵营") - check("redo-snapshot-捕获旧卡status", snapshot["drafts"][0][2] == "confirmed") - check("redo-snapshot-捕获别名presence", snapshot["aliases"] and snapshot["presence"]) - check("redo-snapshot-捕获初建卡水位", snapshot["card_states"] == [(701, 7, "t4")]) - check("redo-snapshot-捕获旧窗状态", snapshot["window"] == ("done", None, "old")) - - pu._restore_redo_window(conn, 8, snapshot) - writes = conn.writes - check( - "redo-restore-恢复旧卡字段status与revision", - any(sql.startswith("UPDATE muse_knowledge_draft SET draft_payload=%s, status=%s, revision=%s") - and params[:4] == ( - json.dumps({"字段": {"阵营": "旧阵营"}}, ensure_ascii=False), - "confirmed", 9, "old", - ) - for sql, params in writes), - ) - check( - "redo-restore-软删重试新建卡", - any(sql.startswith("UPDATE muse_knowledge_draft SET deleted=TRUE") and params == (702,) - for sql, params in writes), - ) - check( - "redo-restore-恢复别名与presence", - any("INSERT INTO example_upgrade_alias" in sql and "旧别名" in params for sql, params in writes) - and any("INSERT INTO example_upgrade_presence" in sql and "旧龙套" in params for sql, params in writes), - ) - check( - "redo-restore-恢复初建卡水位与旧窗状态", - any("INSERT INTO example_upgrade_card_state" in sql and params[0] == 701 for sql, params in writes) - and any(sql.startswith("UPDATE example_upgrade_window SET status=%s") - and params[:3] == ("done", None, "old") for sql, params in writes), - ) def test_presence_chapter_normalization_boundaries(): @@ -2520,6 +2286,7 @@ def test_run_pure_presence_rejects_external_state_and_revision_conflicts(): ), \ patch.object(pu, "m3_json", side_effect=drift_during_observe), \ patch.object(pu, "undo_window"): + # 该 fake 专门验证外部状态漂移后的持久化，不验证 CLI 停止策略，因此显式保留旁路。 pu._run( work_id=8, max_windows=0, @@ -2527,6 +2294,8 @@ def test_run_pure_presence_rejects_external_state_and_revision_conflicts(): model="MiniMax-M3", redo_window=0, semantic_on=False, + raise_on_second_failure=False, + validate_window_input=False, ) check(f"pure-presence-{name}-payload零覆盖", state["payload"] == original_payload) check(f"pure-presence-{name}-窗失败关闭", state["window_status"] == "failed") @@ -2586,7 +2355,8 @@ def test_run_presence_paths_accept_numeric_strings(): patch.object( pu, "_write_locked_upgrade_draft", wraps=pu._write_locked_upgrade_draft ) as write_mock: - pu._run(8, 0, 0, "MiniMax-M3", 0, False) + # 该 fake 的材料由 load_window_material 单独提供，显式跳过真实章域一致性校验。 + pu._run(8, 0, 0, "MiniMax-M3", 0, False, validate_window_input=False) return state, merge_mock.call_count, write_mock.call_count updated, merge_calls, write_calls = run_case([ @@ -3035,6 +2805,662 @@ def test_prompts_disciplines(): check("judge-跨型禁判同一实体仍在", "绝不判同一实体" in jp) +# ── ⑦ 两阶段窗口事务：纯 hash/marker 合同与连接边界 ── +def _fence_fixture(): + """构造稳定的窗口输入与七域状态，供逐域漂移反例复用。""" + + window = {"id": 91, "window_no": 7, "from_chapter": 70, "to_chapter": 71, "deleted": False} + chapters = [ + { + "chapter_id": 7001, + "order_no": 70, + "title": "甲章", + "status": "published", + "revision": 3, + "block_id": 7101, + "block_order_no": 1, + "block_type": "scene", + "block_revision": 4, + "content_text": "正文甲", + }, + { + "chapter_id": 7002, + "order_no": 71, + "title": "乙章", + "status": "published", + "revision": 5, + "block_id": 7102, + "block_order_no": 1, + "block_type": "scene", + "block_revision": 6, + "content_text": "正文乙", + }, + ] + schemas = [("character", 11, {"字段": ["名称"]}), ("item", 12, {"字段": ["名称"]})] + state = { + "drafts": [(701, {"名称": "安若雪"}, "pending", 3, False)], + "aliases": [(1, "雪姐", False)], + "presence": [(2, 7, 70, "安若雪", False)], + "card_state": [(701, 7)], + "audits": [(3, 701, 7, "阵营", None, "联邦")], + "windows": [(91, 7, 70, 71, False, "processing", "旧 marker")], + "embeddings": [(4, "digest-a")], + } + return window, chapters, schemas, state + + +def test_window_fence_hashes_and_markers(): + """input/state hash 必须覆盖合同规定的全部漂移面，且 marker 只接受两个阶段。""" + + window, chapters, schemas, state = _fence_fixture() + with patch.object(pu, "_script_sha256", return_value="a" * 64): + input_sha = pu._window_input_sha(window, chapters, schemas, work_title="测试书") + check("fence-input-sha256", len(input_sha) == 64) + input_changes = [ + ("window-id", {**window, "id": 92}, chapters, schemas, "测试书", "a" * 64), + ("window-no", {**window, "window_no": 8}, chapters, schemas, "测试书", "a" * 64), + ("window-from", {**window, "from_chapter": 69}, chapters, schemas, "测试书", "a" * 64), + ("window-to", {**window, "to_chapter": 72}, chapters, schemas, "测试书", "a" * 64), + ("window-deleted", {**window, "deleted": True}, chapters, schemas, "测试书", "a" * 64), + ("work-title", window, chapters, schemas, "标题漂移", "a" * 64), + ("chapter-id", window, [{**chapters[0], "chapter_id": 9999}, chapters[1]], schemas, "测试书", "a" * 64), + ("chapter-title", window, [{**chapters[0], "title": "标题漂移"}, chapters[1]], schemas, "测试书", "a" * 64), + ("chapter-status", window, [{**chapters[0], "status": "draft"}, chapters[1]], schemas, "测试书", "a" * 64), + ("chapter-revision", window, [{**chapters[0], "revision": 99}, chapters[1]], schemas, "测试书", "a" * 64), + ("block-id", window, [{**chapters[0], "block_id": 9999}, chapters[1]], schemas, "测试书", "a" * 64), + ("block-order", window, [{**chapters[0], "block_order_no": 2}, chapters[1]], schemas, "测试书", "a" * 64), + ("block-type", window, [{**chapters[0], "block_type": "chapter"}, chapters[1]], schemas, "测试书", "a" * 64), + ("block-revision", window, [{**chapters[0], "block_revision": 99}, chapters[1]], schemas, "测试书", "a" * 64), + ("chapter-content", window, [{**chapters[0], "content_text": "正文漂移"}, chapters[1]], schemas, "测试书", "a" * 64), + ("schema-version", window, chapters, [("character", 99, {"字段": ["名称"]})], "测试书", "a" * 64), + ("schema-contract", window, chapters, [("character", 11, {"字段": ["名称", "别名"]})], "测试书", "a" * 64), + ("script-sha", window, chapters, schemas, "测试书", "b" * 64), + ] + for name, changed_window, changed_chapters, changed_schemas, title, code_sha in input_changes: + with patch.object(pu, "_script_sha256", return_value=code_sha): + changed_sha = pu._window_input_sha( + changed_window, changed_chapters, changed_schemas, work_title=title, + ) + check(f"fence-input-{name}漂移", changed_sha != input_sha) + + state_sha = pu._window_state_sha(state, current_window_id=91) + check("fence-state-sha256", len(state_sha) == 64) + for domain in ("drafts", "aliases", "presence", "card_state", "audits", "windows", "embeddings"): + changed = deepcopy(state) + changed[domain] = deepcopy(changed[domain]) + [("漂移",)] + check(f"fence-state-{domain}漂移", pu._window_state_sha(changed, 91) != state_sha) + for name, changed_embedding in ( + ("元数据", (4, "digest-b")), + ("向量摘要", (4, "digest-c")), + ("行ID", (5, "digest-a")), + ): + changed = deepcopy(state) + changed["embeddings"][0] = changed_embedding + check(f"fence-state-embedding-{name}漂移", pu._window_state_sha(changed, 91) != state_sha) + mutable = deepcopy(state) + mutable["windows"][0] = (91, 7, 70, 71, False, "failed", "任意错误文本") + check("fence-state-当前窗status-error不自引用", pu._window_state_sha(mutable, 91) == state_sha) + + for stage in ("entity", "relation"): + marker = pu._upgrade_marker(stage, input_sha, state_sha) + check(f"fence-marker-{stage}-往返", pu._parse_upgrade_marker(marker) == (stage, input_sha, state_sha)) + for invalid in (None, "", "upgrade-fence:v2:entity:a:b", "upgrade-fence:v1:batch:a:b", + "upgrade-fence:v1:entity:not-sha:not-sha"): + try: + pu._parse_upgrade_marker(invalid) + except pu.CompensationFenceConflict: + check(f"fence-marker-invalid-{invalid}", True) + else: + check(f"fence-marker-invalid-{invalid}", False, detail="非法 marker 被接受") + + +class _CaptureInputConn: + """输入快照 fake：用于验证空章域和缺章域在送模前失败关闭。""" + + def __init__(self, rows): + self.rows = rows + + def execute(self, query, params=()): + normalized = " ".join(query.split()) + if normalized.startswith("SELECT id, window_no, from_chapter, to_chapter, deleted"): + return _CardResult((91, 7, 70, 71, False)) + if normalized.startswith("SELECT title FROM muse_content_work"): + return _CardResult(("测试书",)) + if normalized.startswith("SELECT c.id, c.order_no, c.title, c.status, c.revision"): + return _CardResult(rows=self.rows) + if normalized.startswith("SELECT s.schema_key, s.active_version_id"): + return _CardResult(rows=[]) + return _CardResult() + + +def test_capture_window_input_rejects_incomplete_or_empty_content(): + """捕获阶段必须覆盖完整章域并保证每章拼接正文非空。""" + + for name, rows in ( + ("空章域", []), + ("缺少一章", _offline_window_input_rows(70, 70)), + ("正文为空", [*_offline_window_input_rows(70, 70), (*_offline_window_input_rows(71, 71)[0][:9], "")]), + ): + try: + pu._capture_window_input(_CaptureInputConn(rows), 8, 7) + except pu.CompensationFenceConflict: + check(f"capture-{name}-失败关闭", True) + else: + check(f"capture-{name}-失败关闭", False, detail="不完整/空正文被送入模型") + + +class _CaptureStateConn: + """窗口状态摘要 fake：按 SQL 中的 CASE 语义计算逐行摘要并记录绑定参数。""" + + def __init__(self, windows): + self.windows = windows + self.window_sql = None + self.window_params = None + + def execute(self, query, params=()): + normalized = " ".join(query.split()) + if normalized.startswith("SELECT w.id, encode(sha256(convert_to("): + self.window_sql = normalized + self.window_params = params + uses_current_case = "CASE WHEN w.id=%s THEN" in normalized + current_window_id = params[0] if uses_current_case else None + rows = [] + for window in self.windows: + payload = deepcopy(window) + if uses_current_case and window["id"] == current_window_id: + payload.pop("status", None) + payload.pop("error_message", None) + raw = json.dumps(payload, ensure_ascii=False, sort_keys=True, separators=(",", ":")) + rows.append((window["id"], hashlib.sha256(raw.encode("utf-8")).hexdigest())) + return _CardResult(rows=rows) + return _CardResult(rows=[]) + + +def test_capture_window_state_binds_current_window_and_preserves_other_status(): + """只有当前 marker 窗忽略 status/error，其他窗口的完整行变化必须使摘要漂移。""" + + base = [ + {"id": 91, "window_no": 7, "status": "processing", "error_message": "marker-a", "deleted": False}, + {"id": 92, "window_no": 8, "status": "pending", "error_message": None, "deleted": False}, + ] + base_conn = _CaptureStateConn(base) + base_state = pu._capture_window_state(base_conn, 8, 91) + changed_current = deepcopy(base) + changed_current[0].update(status="failed", error_message="marker-b") + current_state = pu._capture_window_state(_CaptureStateConn(changed_current), 8, 91) + check("state-当前窗status-error不改变摘要", current_state == base_state) + check( + "state-窗口SQL绑定current-window-id", + base_conn.window_params[0] == 91 + and "CASE WHEN w.id=%s THEN" in base_conn.window_sql + and "-'status'-'error_message'" in base_conn.window_sql, + detail=f"sql={base_conn.window_sql}, params={base_conn.window_params}", + ) + + changed_other = deepcopy(base) + changed_other[1].update(status="failed", error_message="other-window-error") + other_state = pu._capture_window_state(_CaptureStateConn(changed_other), 8, 91) + check("state-其他窗status-error改变摘要", other_state != base_state) + check( + "state-七域窗口查询仍只返回逐行id-sha", + all(len(row) == 2 and len(row[1]) == 64 for row in base_state["windows"]), + ) + check("state-窗口查询不回传to_jsonb整行", "SELECT to_jsonb(w)" not in base_conn.window_sql) + + +class _WindowMaterialConn: + """正文材料 fake：记录排序 SQL，模拟多 block 章节返回。""" + + def __init__(self): + self.sql = None + + def execute(self, query, params=()): + self.sql = " ".join(query.split()) + return _CardResult(rows=[ + (70, "甲章", "block-1"), + (70, "甲章", "block-2"), + (71, "乙章", "block-1"), + ]) + + +def test_load_window_material_orders_blocks_like_capture_input(): + """材料加载必须按章号、block.order_no、block.id 与 input 捕获保持同序。""" + + conn = _WindowMaterialConn() + pu.load_window_material(conn, 8, 70, 71) + check( + "material-多block排序与capture-input一致", + "ORDER BY c.order_no, b2.order_no, b2.id" in conn.sql, + detail=conn.sql, + ) + + +def test_run_defaults_enforce_safety_contract(): + """内部默认调用必须开启章域校验和二次失败异常，CLI 不应再提供旁路参数。""" + + defaults = pu._run.__kwdefaults__ + check("run-默认强制二次失败抛ClickException", defaults["raise_on_second_failure"] is True) + check("run-默认强制窗口输入校验", defaults["validate_window_input"] is True) + cli_source = inspect.getsource(pu.run.callback) + check( + "run-cli使用安全默认不显式旁路", + "raise_on_second_failure" not in cli_source + and "validate_window_input" not in cli_source, + detail=cli_source, + ) + + +def test_two_phase_external_calls_and_atomic_embedding_contract(): + """外部调用期间连接必须为零；向量 apply 与 done 共用一个事务，失败不得放行。""" + + tracker = {"connections": 0, "external": [], "applied": [], "status": "processing"} + + def external(kind, value): + tracker["external"].append((kind, tracker["connections"])) + if value == "fail": + raise RuntimeError("离线嵌入失败") + return f"prepared:{value}" + + prepared = pu._call_external_without_connection(tracker, external, "observe", "entity") + check("two-phase-observe零连接", prepared == "prepared:entity" and tracker["external"][-1] == ("observe", 0)) + prepared = pu._call_external_without_connection(tracker, external, "relation", "relation") + check("two-phase-relation零连接", prepared == "prepared:relation" and tracker["external"][-1] == ("relation", 0)) + prepared = pu._call_external_without_connection(tracker, external, "embed", "vector") + check("two-phase-embed零连接", prepared == "prepared:vector" and tracker["external"][-1] == ("embed", 0)) + try: + pu._call_external_without_connection(tracker, external, "embed", "fail") + except RuntimeError: + check("two-phase-普通嵌入失败整窗失败", tracker["status"] == "processing") + else: + check("two-phase-普通嵌入失败整窗失败", False, detail="嵌入异常被放行") + + class AtomicConn: + """最小事务感知 fake：未 commit 的向量与 done 必须一起回滚。""" + + def __init__(self): + self.pending = [] + self.committed = [] + + def apply_embedding(self, item): + self.pending.append(("embedding", item)) + + def mark_done(self): + self.pending.append(("status", "done")) + + def commit(self): + self.committed.extend(self.pending) + self.pending.clear() + + def rollback(self): + self.pending.clear() + + conn = AtomicConn() + pu._apply_prepared_and_done(conn, ["v1"], apply_one=conn.apply_embedding, mark_done=conn.mark_done) + check("two-phase-embed+done同事务", conn.committed == [("embedding", "v1"), ("status", "done")]) + broken = AtomicConn() + try: + pu._apply_prepared_and_done( + broken, ["v1"], + apply_one=lambda item: broken.apply_embedding(item) or (_ for _ in ()).throw(RuntimeError("crash")), + mark_done=broken.mark_done, + ) + except RuntimeError: + broken.rollback() + check("two-phase-crash不提交向量和done", broken.committed == [] and broken.pending == []) + + +def test_recovery_and_redo_rejection_contracts(): + """恢复判定与 redo 双入口零 DB 拒绝必须机械且确定。""" + + input_sha = "a" * 64 + state_sha = "b" * 64 + marker = pu._upgrade_marker("entity", input_sha, state_sha) + check("recovery-exact允许undo", pu._recovery_decision(marker, input_sha, state_sha) == "undo") + for name, current_input, current_state in ( + ("input", "c" * 64, state_sha), + ("drafts", input_sha, "c" * 64), + ("aliases", input_sha, "d" * 64), + ("presence", input_sha, "e" * 64), + ("card_state", input_sha, "f" * 64), + ("audits", input_sha, "1" * 64), + ("windows", input_sha, "2" * 64), + ): + check( + f"recovery-{name}漂移补偿失败", + pu._recovery_decision(marker, current_input, current_state) == "compensation-failed", + ) + check( + "recovery-非法marker补偿失败", + pu._recovery_decision("broken", input_sha, state_sha) == "compensation-failed", + ) + + db_calls, lock_calls = [], [] + with patch.object(pu.psycopg, "connect", side_effect=lambda *_: db_calls.append(True)), \ + patch.object( + pu, + "upgrade_work_lock", + side_effect=lambda *_: lock_calls.append(True) or nullcontext(), + ): + result = CliRunner().invoke(pu.cli, ["run", "--work-id", "8", "--redo-window", "7"]) + check( + "redo-wrapper机械拒绝非零", + result.exit_code != 0 and "backup" in result.output and "reset/rebuild" in result.output, + ) + check("redo-wrapper拒绝发生在advisory与业务DB前", db_calls == [] and lock_calls == []) + + try: + pu._run(8, 0, 0, "MiniMax-M3", 7, False) + except pu.click.ClickException as exc: + check("redo-run首行机械拒绝", "backup" in str(exc) and db_calls == []) + else: + check("redo-run首行机械拒绝", False, detail="_run 接受了 redo") + + +def test_real_pg_rollback_smoke_entry(): + """真实 PG 入口默认不运行；显式开启后验证 public 窗状态写入可回滚。""" + + if os.getenv("MUSE_REAL_PG_ROLLBACK_SMOKE") != "1": + check("real-pg-rollback-smoke默认跳过", True) + return + work_id = int(os.getenv("MUSE_REAL_PG_ROLLBACK_SMOKE_WORK_ID", "8")) + pu.real_pg_rollback_smoke(work_id) + check("real-pg-rollback-smoke真实public窗回滚", True) + + +class _LegacyRecoveryConn: + """恢复命令离线夹具：用安全墓碑及异常软删验证 SQL 过滤。""" + + def __init__(self, active_artifact=None, non_reset_deleted=None, + embedding_variant=None): + self.window_status = "failed" + self.window_error = "legacy parser error" + self.active_artifact = active_artifact + self.non_reset_deleted = non_reset_deleted + self.drafts = [{ + "id": 701, "work_id": 8, "source_type": pu.SOURCE_TYPE, + "origin": "升格@窗7", "deleted": True, "updater": "upgrade-reset", + "status": "pending", + }] + self.aliases = [{"work_id": 8, "window_no": 7, "deleted": True, + "updater": "upgrade-reset"}] + self.presence = [] + self.card_states = [] + self.audits = [] + self.embeddings = [{"draft_id": 701, "deleted": True, "entity_id": None}] + if non_reset_deleted == "draft": + self.drafts[0]["updater"] = "upgrade" + elif non_reset_deleted == "embedding-draft": + self.drafts[0]["updater"] = "upgrade" + elif non_reset_deleted == "draft-status": + self.drafts[0]["status"] = "confirmed" + elif non_reset_deleted == "alias": + self.aliases[0]["updater"] = "upgrade" + if active_artifact == "draft": + self.drafts[0]["deleted"] = False + elif active_artifact == "embedding": + self.embeddings[0]["deleted"] = False + elif active_artifact == "embedding-owner": + self.embeddings[0]["entity_id"] = 9001 + elif active_artifact == "embedding-not-deleted": + self.embeddings[0]["deleted"] = False + elif active_artifact == "card_state": + self.card_states.append({"draft_id": 701, "watermark_window": 7}) + elif active_artifact == "audit": + self.audits.append({"draft_id": 701, "window_no": 7}) + elif active_artifact == "alias": + self.aliases[0]["deleted"] = False + elif active_artifact == "presence": + self.presence.append({"work_id": 8, "window_no": 7, "deleted": False}) + elif active_artifact == "presence-deleted": + self.presence.append({"work_id": 8, "window_no": 7, "deleted": True}) + self.processing_count = 0 + self.writes = [] + self.commits = 0 + self.queries = [] + + def __enter__(self): + return self + + def __exit__(self, exc_type, exc, tb): + return False + + def execute(self, query, params=()): + normalized = " ".join(query.split()) + self.queries.append(normalized) + if normalized.startswith("SET TRANSACTION ISOLATION LEVEL REPEATABLE READ"): + return _CardResult() + if normalized.startswith("LOCK TABLE"): + return _CardResult() + if normalized.startswith("SELECT id, window_no, from_chapter, to_chapter, status, error_message, deleted"): + return _CardResult((91, 7, 70, 71, self.window_status, self.window_error, False)) + if normalized.startswith("SELECT count(*) FROM example_upgrade_window"): + return _CardResult((self.processing_count,)) + if normalized.startswith("SELECT count(*)"): + if "example_upgrade_audit" in normalized: + self._assert_artifact_sql("audits", normalized, "a.window_no=%s") + if "deleted=FALSE" in normalized: + raise AssertionError("audit 计数不得依赖不存在的 deleted 列") + return _CardResult((sum(row["window_no"] == 7 for row in self.audits),)) + if "example_upgrade_alias" in normalized: + self._assert_artifact_sql( + "aliases", normalized, "a.deleted=TRUE", "a.updater='upgrade-reset'", + ) + return _CardResult((sum( + row["work_id"] == 8 and row["window_no"] == 7 + and not (row["deleted"] and row["updater"] == "upgrade-reset") + for row in self.aliases + ),)) + if "example_upgrade_presence" in normalized: + self._assert_artifact_sql("presence", normalized, "p.window_no=%s") + if "p.deleted" in normalized or "p.updater" in normalized: + raise AssertionError("presence 没有可证明 reset 来源的列，任何本窗行都必须阻断") + return _CardResult((sum( + row["work_id"] == 8 and row["window_no"] == 7 + for row in self.presence + ),)) + if "example_knowledge_embedding" in normalized: + self._assert_artifact_sql( + "embeddings", normalized, "e.entity_id IS NULL", "e.deleted=TRUE", + "d.deleted=TRUE", "d.updater='upgrade-reset'", "d.status='pending'", + ) + return _CardResult((sum( + row["draft_id"] == 701 and not ( + row["entity_id"] is None and row["deleted"] + and self.drafts[0]["deleted"] + and self.drafts[0]["updater"] == "upgrade-reset" + and self.drafts[0]["status"] == "pending" + ) + for row in self.embeddings + ),)) + if "example_upgrade_card_state" in normalized: + self._assert_artifact_sql( + "card_state", normalized, "JOIN muse_knowledge_draft d ON d.id=s.draft_id", + "d.work_id=%s", "d.source_type=%s", + ) + if "d.deleted=FALSE" in normalized: + raise AssertionError("card_state 计数不得豁免 deleted draft 关联行") + return _CardResult((sum( + row["draft_id"] == 701 and row["watermark_window"] == 7 + for row in self.card_states + ),)) + if "muse_knowledge_draft" in normalized: + self._assert_artifact_sql( + "new_cards", normalized, "d.deleted=TRUE", "d.updater='upgrade-reset'", + "d.status='pending'", + ) + return _CardResult((sum( + row["work_id"] == 8 and row["source_type"] == pu.SOURCE_TYPE + and row["origin"] == "升格@窗7" and not ( + row["deleted"] and row["updater"] == "upgrade-reset" + and row["status"] == "pending" + ) + for row in self.drafts + ),)) + return _CardResult((0,)) + if normalized.startswith("UPDATE example_upgrade_window"): + self.writes.append((normalized, params)) + self.window_error = params[0] + return _CardResult(("failed",)) + return _CardResult() + + def commit(self): + self.commits += 1 + + def _assert_artifact_sql(self, name, normalized, *required): + """要求生产查询显式携带活跃语义，防止只把 fixture 计数改成零。""" + + missing = [fragment for fragment in required if fragment not in normalized] + if missing: + raise AssertionError(f"{name} 计数 SQL 缺少过滤条件：{missing}; sql={normalized}") + + +def test_recover_legacy_failed_cli_contract(): + """legacy failed 只允许清理副作用为零的窗口，只有安全 reset 墓碑可豁免。""" + + help_result = CliRunner().invoke(pu.cli, ["recover-legacy-failed", "--help"]) + check( + "recover-legacy-failed-help", + help_result.exit_code == 0 + and "preview" in help_result.output + and "execute" in help_result.output + and "confirmation-sha" in help_result.output + and "confirm-no-live-process" in help_result.output, + detail=help_result.output, + ) + + conn = _LegacyRecoveryConn() + with patch.object(pu.psycopg, "connect", return_value=conn): + preview = CliRunner().invoke( + pu.cli, ["recover-legacy-failed", "--work-id", "8", "--window-no", "7", "--preview"] + ) + check("recover-legacy-failed-preview成功", preview.exit_code == 0, detail=preview.output) + preview_data = json.loads(preview.output) + check("recover-legacy-failed-preview六类产物与processing", + set(preview_data["artifact_counts"]) == { + "audits", "aliases", "presence", "new_cards", "card_state", "embeddings", + } and preview_data["artifact_counts"] == { + "audits": 0, "aliases": 0, "presence": 0, + "new_cards": 0, "card_state": 0, "embeddings": 0, + } and preview_data["processing_count"] == 0 + and preview_data["window"]["status"] == "failed", detail=preview.output) + check("recover-legacy-failed-preview零副作用", conn.writes == [] and conn.commits == 0) + check( + "recover-legacy-failed-preview首条事务语句是RR只读", + conn.queries[0] == "SET TRANSACTION ISOLATION LEVEL REPEATABLE READ, READ ONLY", + detail=str(conn.queries), + ) + + with patch.object(pu.psycopg, "connect", return_value=conn), \ + patch.object(pu, "upgrade_work_lock", return_value=nullcontext()): + execute = CliRunner().invoke( + pu.cli, + [ + "recover-legacy-failed", "--work-id", "8", "--window-no", "7", "--execute", + "--confirmation-sha", preview_data["confirmation_sha"], + "--confirm-no-live-process", + ], + ) + check("recover-legacy-failed-execute成功", execute.exit_code == 0, detail=execute.output) + check( + "recover-legacy-failed-execute保持failed并标retryable-clean", + conn.window_status == "failed" and conn.window_error.startswith(pu.RETRYABLE_CLEAN_PREFIX), + detail=conn.window_error, + ) + + drift_conn = _LegacyRecoveryConn() + drift_conn.processing_count = 1 + with patch.object(pu.psycopg, "connect", return_value=drift_conn), \ + patch.object(pu, "upgrade_work_lock", return_value=nullcontext()): + drift = CliRunner().invoke( + pu.cli, + [ + "recover-legacy-failed", "--work-id", "8", "--window-no", "7", "--execute", + "--confirmation-sha", preview_data["confirmation_sha"], + "--confirm-no-live-process", + ], + ) + check("recover-legacy-failed-漂移非零", drift.exit_code != 0, detail=drift.output) + check("recover-legacy-failed-漂移零副作用", drift_conn.writes == [] and drift_conn.commits == 0) + + for artifact in ("draft", "embedding", "alias", "presence", "card_state", "audit"): + active_conn = _LegacyRecoveryConn(active_artifact=artifact) + with patch.object(pu.psycopg, "connect", return_value=active_conn): + active_preview = CliRunner().invoke( + pu.cli, ["recover-legacy-failed", "--work-id", "8", "--window-no", "7", "--preview"] + ) + active_data = json.loads(active_preview.output) + check( + f"recover-legacy-failed-active-{artifact}-preview非零", + active_preview.exit_code == 0 and active_data["artifact_counts"][{ + "draft": "new_cards", + "embedding": "embeddings", + "alias": "aliases", + "presence": "presence", + "card_state": "card_state", + "audit": "audits", + }[artifact]] > 0, + detail=active_preview.output, + ) + with patch.object(pu.psycopg, "connect", return_value=active_conn), \ + patch.object(pu, "upgrade_work_lock", return_value=nullcontext()): + active_execute = CliRunner().invoke( + pu.cli, + [ + "recover-legacy-failed", "--work-id", "8", "--window-no", "7", "--execute", + "--confirmation-sha", active_data["confirmation_sha"], + "--confirm-no-live-process", + ], + ) + check( + f"recover-legacy-failed-active-{artifact}-execute拒绝", + active_execute.exit_code != 0 and active_conn.writes == [] and active_conn.commits == 0, + detail=active_execute.output, + ) + + for artifact in ("draft", "embedding-draft", "draft-status", "alias"): + non_reset_conn = _LegacyRecoveryConn(non_reset_deleted=artifact) + with patch.object(pu.psycopg, "connect", return_value=non_reset_conn): + non_reset_preview = CliRunner().invoke( + pu.cli, ["recover-legacy-failed", "--work-id", "8", "--window-no", "7", "--preview"] + ) + non_reset_data = json.loads(non_reset_preview.output) + key = {"draft": "new_cards", "embedding-draft": "embeddings", + "draft-status": "new_cards", + "alias": "aliases"}[artifact] + check( + f"recover-legacy-failed-non-reset-{artifact}-阻断", + non_reset_preview.exit_code == 0 and non_reset_data["artifact_counts"][key] > 0, + detail=non_reset_preview.output, + ) + + deleted_presence_conn = _LegacyRecoveryConn(active_artifact="presence-deleted") + with patch.object(pu.psycopg, "connect", return_value=deleted_presence_conn): + deleted_presence_preview = CliRunner().invoke( + pu.cli, ["recover-legacy-failed", "--work-id", "8", "--window-no", "7", "--preview"] + ) + deleted_presence_data = json.loads(deleted_presence_preview.output) + check( + "recover-legacy-failed-deleted-presence-仍阻断", + deleted_presence_preview.exit_code == 0 + and deleted_presence_data["artifact_counts"]["presence"] > 0, + detail=deleted_presence_preview.output, + ) + + for artifact in ("embedding-owner", "embedding-not-deleted"): + embedding_conn = _LegacyRecoveryConn(active_artifact=artifact) + with patch.object(pu.psycopg, "connect", return_value=embedding_conn): + embedding_preview = CliRunner().invoke( + pu.cli, ["recover-legacy-failed", "--work-id", "8", "--window-no", "7", "--preview"] + ) + embedding_data = json.loads(embedding_preview.output) + check( + f"recover-legacy-failed-{artifact}-阻断", + embedding_preview.exit_code == 0 + and embedding_data["artifact_counts"]["embeddings"] > 0, + detail=embedding_preview.output, + ) + + if __name__ == "__main__": for fn in (test_classify_new_name, test_resolve_canonical_name, test_debut_milestone, test_clean_milestone_guard, @@ -3044,7 +3470,7 @@ if __name__ == "__main__": test_alias_claim_payload_undo_and_entity_watermark, test_undo_advances_revision_and_invalidates_precompensation_cas, test_entity_chapter_evidence_filter, test_new_card_chapter_evidence_chain, - test_merge_card_chapter_evidence, test_redo_cleans_legacy_appearance_chapters, + test_merge_card_chapter_evidence, test_run_alias_paths_store_real_canonical_name, test_run_new_card_registers_normalized_name_and_aliases_same_window, test_run_exact_and_disjoint_duplicate_entity_updates_merge_once, @@ -3059,22 +3485,25 @@ if __name__ == "__main__": test_run_alias_gate_precedes_empty_update_and_deleted_alias_fails_closed, test_run_embedding_owner_conflict_marks_window_failed, test_owner_conflict_external_confirmation_preserves_draft_and_persists_breakpoint, - test_redo_owner_conflict_restores_snapshot_without_normal_undo, - test_redo_full_fence_rejects_untouched_and_auxiliary_domain_drift, test_compensation_exception_persists_durable_failed_and_next_run_stops, test_run_embedding_payload_drift_marks_window_failed_without_vector_writes, test_embed_touched_migrates_soft_deleted_hash_owner, test_embed_touched_rejects_two_active_drafts_with_same_hash, test_embed_touched_rejects_existing_active_hash_owner, - test_run_redo_preflight_rejects_unsafe_targets, - test_run_rejects_negative_limits_before_redo_side_effects, - test_run_redo_max_calls_finishes_active_retry, - test_run_redo_final_failure_restores_full_snapshot, - test_redo_snapshot_restore_sql_boundaries, test_presence_chapter_normalization_boundaries, test_run_pure_presence_rejects_external_state_and_revision_conflicts, test_run_presence_paths_accept_numeric_strings, test_load_known_projection, test_repair_milestone_evidence, - test_prompts_disciplines): + test_prompts_disciplines, test_window_fence_hashes_and_markers, + test_capture_window_input_rejects_incomplete_or_empty_content, + test_two_phase_external_calls_and_atomic_embedding_contract, + test_recovery_and_redo_rejection_contracts, test_real_pg_rollback_smoke_entry, + test_recover_legacy_failed_cli_contract, + test_capture_window_state_binds_current_window_and_preserves_other_status, + test_load_window_material_orders_blocks_like_capture_input, + test_run_defaults_enforce_safety_contract): fn() - print(f"\n全部离线自测通过：{_passed} 项（未连库、未发任何网络/嵌入/LLM 调用）") + if os.getenv("MUSE_REAL_PG_ROLLBACK_SMOKE") == "1": + print(f"\n全部自测通过：{_passed} 项（含真实 PG public 窗 rollback smoke；未发模型/嵌入调用）") + else: + print(f"\n全部离线自测通过：{_passed} 项（真实 PG smoke 未启用；未发网络/嵌入/LLM 调用）") diff --git a/docs/2026-07-16-升格卡改造设计.md b/docs/2026-07-16-升格卡改造设计.md index bdc902e..f7d7701 100644 --- a/docs/2026-07-16-升格卡改造设计.md +++ b/docs/2026-07-16-升格卡改造设计.md @@ -244,6 +244,39 @@ flowchart TB ## 九、风险与验证办法 +### 9.1 批量抽取的连接与恢复合同（2026-07-22 补充） + +升格抽取不得在模型或嵌入调用期间持有业务数据库连接。一次窗口只保留一个可恢复的中间提交点，避免把每个更新批次都变成独立恢复协议： + +1. **无连接计算**：读取正文、既有卡和 revision 后立即关闭连接；完成观察、证据修复、语义判重和全部实体更新输出。此阶段失败时没有知识写入，只把窗口留成干净失败断点。 +2. **实体短写事务**：先复验模型输入快照，再复验模型所见 revision，原子写入新卡、别名、留档和实体更新；把窗口置为 `processing`，并在同一事务记录本书受控域的完整状态摘要。进程此后崩溃，续跑必须先复验摘要，再精确撤销本窗。 +3. **无连接关系计算**：实体阶段提交后重新读取人物与关系快照，关闭连接，再生成或重整关系。卡片已更新后的状态仍能进入关系判断，但模型调用期间没有长事务。 +4. **关系短写事务**：复验 `processing` 摘要、输入快照、人物 revision 和关系集合，写关系与剩余出场章；窗口仍保持 `processing`，原子刷新状态摘要。 +5. **严格嵌入与完成事务**：在无业务数据库连接状态下计算本窗 touched 卡向量，再用短事务复验摘要和向量 owner 后落库；只有嵌入成功且摘要再次一致，窗口才置 `done`。普通嵌入失败也按窗口失败处理，不允许后窗在缺向量状态下继续语义判重。 + +```mermaid +flowchart LR + R[短读: 正文/卡/revision] --> C1[无连接: 观察/证据/判重/实体更新] + C1 --> W1[短写: 实体阶段
processing + 完整摘要] + W1 --> R2[短读: 更新后人物/关系] + R2 --> C2[无连接: 关系生成/重整] + C2 --> W2[短写: 复验摘要
关系/出场章/刷新摘要] + W2 --> E[无连接: 计算增量向量] + E --> W3[短写: 复验摘要/owner
落向量/done] + E -->|失败或 owner 冲突| X[锁内复验完整摘要
通过才撤销] +``` + +恢复与并发纪律： + +- 普通错误只允许同窗完整撤销后重试一次；第二次仍失败立即停止当前作品，保留断点，不跳窗、不换作品内顺序。 +- `processing` marker 固定为 `upgrade-fence:v1:::`；`stage` 只允许 `entity` 或 `relation`。非法、缺失或版本不认识的 marker 一律写 durable 断点并停止，不能猜测恢复。 +- `inputSha256` 绑定实际送模输入：作品标题、窗口边界、每章 `id/title/status/revision`、每个 block 的 `id/order/type/revision/content_text`，以及实体 schema active version/字段合同、抽取合同和当前 `parse_upgrade.py` SHA-256。按章拼接正文的规则必须与送模文本一致；实体阶段写入前与关系阶段写入前都要重算，章域不完整、无 block 或拼接正文为空直接失败关闭。 +- `stateSha256` 覆盖本书卡、别名、留档、水位、审计、向量和全部窗口不变元数据；窗口自身可变的状态/错误文本不参与自引用 hash。`processing` 恢复和失败补偿都要在固定顺序的七域锁内复验它。 +- 任何外部确认、改版、删除或辅助域漂移都写 `compensation-failed` 并停止，禁止覆盖外部状态。嵌入落库与 `done` 在同一最终短事务完成，消除 `done` 已提交但向量尚未落库的崩溃缝隙。 +- 当前数据库没有所有写方共同遵守的 work 级互斥键。为保证摘要无幻读，七个受控域使用**只跨短写事务**的固定表锁；它会短暂串行化不同作品的写阶段，但绝不跨模型或嵌入计算。待真实耗时证明成为瓶颈后，再单独设计全系统统一的 work 级写锁，不能在本次修复里用弱锁换假安全。 +- fence 引入前的 `legacy failed` 窗不能自动猜测。`recover-legacy-failed` 是唯一恢复入口：preview 输出窗口、七域本窗产物计数与确认 hash；execute 必须取得同书锁、精确匹配 hash、确认无 `processing` 窗且**阻断产物计数为 0**，才可标成 `retryable-clean`。计数只豁免能机械证明来源的全书 reset 墓碑：draft 必须同时 `deleted=true`、`updater='upgrade-reset'`、`status='pending'`；embedding 还必须 `entity_id IS NULL` 且自身 `deleted=true`；alias 必须 `deleted=true` 且 `updater='upgrade-reset'`。presence 没有 reset 来源标记，任何本窗行（包括软删墓碑）都阻断；其他软删、活跃行、已有 entity owner，以及任何 card_state/audit 行也都阻断。完整 `stateSha` 仍覆盖所有 deleted 行，不能混淆。操作人仍须先确认旧进程与数据库 backend 已结束，工具不得把“查不到已提交行”误当成“旧事务不存在”。 +- 显式 `--redo-window` 需要把 redo 前完整快照持久化到进程外，单靠内存快照无法承受崩溃。该持久快照协议另立设计前，命令机械拒绝；历史修正走已有全书 backup/reset/rebuild，不提供不完整的 redo。 + ### 风险 1. **条目从字符串变对象，合并逻辑改动面不小**。`parse_upgrade.py` 现有一大套针对字符串条目的守卫（剥前缀、剥尾残、拦垃圾、拆粘连、窗序归位），全要跟着改造成处理里程碑对象；存量卡还是字符串，迁移要兼容。→ 缓解：分两步落，先补字段和提示词（新卡受益），再做存量迁移；对象结构压到三字段降低崩坏面。 @@ -251,6 +284,7 @@ flowchart TB 3. **语义判重增加调用量和跨型误并风险**。→ 缓解：0.60 阈值（实测校准） + 同型才自动并 + 跨型仅提示；判重近邻可像 v6 说的那样批量做、不逐条烧。 4. **存量迁移精度损失**：无内嵌章号的条目只能靠窗映射到 12 章宽区间。→ 接受为存量固有局限，新卡不受影响。 5. **改 schema 后必须重跑种子**，否则库里字段合同和 YAML 不一致，抽取会按旧合同把新字段裁掉。→ 实施清单里钉死"改 YAML 后重跑 `seed_schemas.py`"这一步。 +6. **受控域表锁会压缩多作品写入并发**。→ 模型调用仍可并行，表锁只覆盖短写和摘要复验；实施后记录阶段耗时与锁等待，若写阶段成为瓶颈，再以统一 work 级锁替换，禁止局部脚本自创弱互斥。 ### 验证办法（实施后怎么确认真治好了） @@ -260,6 +294,8 @@ flowchart TB - **语义判重召回**：造一个改名样例（影杀者 / IV 代纯机械机甲·影杀者），确认语义判重能召回并由 M3 判为同一实体或前身后继。 - **章号无窗号残留**：迁移后全库扫一遍，确认演变类字段里不再有 [窗N] 前缀。 - **章号证据真绑定**：用跨章窗口重抽验证，错标章节的里程碑必须拒收；正确章节原文短引通过后，正式卡只保留章 / 台阶 / 周期，不残留证据字段。 +- **连接红线**：离线门禁覆盖观察、证据修复、判重、实体更新、关系生成/重整和嵌入；每次外部调用时业务数据库连接数必须为零。 +- **真实数据库恢复冒烟**：在回滚事务内验证公共 `example_upgrade_window` 完整行的 `processing` marker 写入、复验和回滚；同事务把一张 `upgrade_book` draft 的 revision 加一，确认 marker 复验明确拒绝，最后用新连接确认窗口完整行和 draft revision 均未改变。不满足 pending 窗或无可用 draft 时明确失败。纯 fake 连接测试不能替代这项。 ---