框架: 共享运行时装成可安装包,切断 Skill 之间的 sys.path 互指

被多个 Skill 或看板消费的连接、模型、嵌入、Claude 运行时与声音账入口
各只保留一份实现;Skill 只留 CLI/落库,看板只读 muse-db,门禁锁死跨域注入。

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
zizi 2026-08-20 00:45:20 +08:00
parent 6fe3fd0a10
commit 061010ba1b
118 changed files with 1443 additions and 1457 deletions

View File

@ -49,6 +49,7 @@ Skill 按实现性质分两类,合同要求不同。**系统能力 Skill** 执
## 4. Tool 合同
- Tool 放在所属 Skill 的 `scripts/`,不散落一次性脚本;参照 Skill 的 `scripts/` 只放工作表与清单文本,不含 Tool。
- 被两个以上 Skill 或看板消费的确定性实现升级为共享运行时包(如 `muse_db`、`muse-deai`、`claude_runtime`、`muse_llm`、`muse_embed`);所属 Skill 只保留 CLI 与落库编排。调用方 `import` 已安装的包,不得 `sys.path` 指向其它 Skill 的 `scripts/`。
- 默认从仓库任意工作目录调用,必须自行解析项目根和输入绝对路径,不能依赖调用者先 `cd` 到特定目录。
- 机械事实必须结构化输出稳定状态和错误码;人读日志是补充,不是唯一接口。
- Tool 不调用模型,除非所属 Skill 明确声明该步骤本质需要模型。

View File

@ -9,14 +9,14 @@
- 它只干两件事:对库做**只读查询**,把结果渲染成人能读的页面。
- 它**绝不触发任何写操作**。接受、合并、丢弃、确认这些写,仍由 `decide-candidate` Skill 和主会话走,看板只展示结果。
- 它看到的 = 库里的。看板上空白的地方,就是落库的缺口——所以看板天然是“一切输入产出必须落库”这条纪律的验收面。
- `/ai-flavor` 也是数据库视图:默认读取 AI 味案例卡与重验证账本表,页面明确标注“候选命中,不是确认结论”;只有数据库不可用时才显示离线回退及原因。其余视图同样以数据库为权威。
- `/ai-flavor` 也是数据库视图:默认读取 AI 味案例卡与重验证账本表,页面明确标注“候选命中,不是确认结论”;只有数据库不可用时才显示 `dashboard/fixtures/` 中的离线 JSON 回退及原因。其余视图同样以数据库为权威。
- 它服务本机单用户,不做多用户、权限管理、对外分享。
## 2. 只读硬约束(怎么保证它绝不写)
这是看板的命根子,验收时按机械门查:
- **连接只读**:看板用独立的只读连接,默认事务只读(`SET TRANSACTION READ ONLY` 或库侧只读角色);连接串与写通道(`access-database` Skill)分开。
- **连接只读**:看板用 `muse_db.connect(readonly=True)` 开独立只读会话(会话级 `default_transaction_read_only=on`,写语句被 PostgreSQL 直接拒)。禁止调用 `access-database` 的可写 CLI 或任何会写库的 Skill。
- **代码无写语句**:全模块只允许 `SELECT`,不得出现任何 `INSERT/UPDATE/DELETE` 或 DDL。**真门禁是库级只读连接**(写语句被 PostgreSQL 直接拒);“grep 无写语句”是辅助门,须用 `\bINSERT\b` / `\bUPDATE\b` / `\bDELETE\b` 词边界查(否则 `deleted=false` 里的 DELETE 子串会误报)。
- **不接会写的通道**:看板不调用 `decide-candidate` 或任何会写库的 Skill,只自己读库。
- **挂了不牵连**:看板进程崩了、断网了,库内正式内容和创作链不受任何影响。
@ -73,7 +73,8 @@
## 6. 与现有通道的关系
- 看板**不替代** `access-database` Skill。后者是面向 agent 和主会话的唯一数据库通道(可写可查);看板是面向人的独立只读渲染面。
- 看板**不替代** `access-database` Skill。后者是面向 agent 和主会话的可写 CLI 通道;连接实现由共享运行时包 `muse_db` 提供。看板经 `connect(readonly=True)` 读库,不经会写的 CLI。
- 额度窗上限(MiniMax `$24` / 全模型 `6000` 次)与 `muse_llm` 共用 `muse_db.WINDOW_BUDGET_USD` / `WINDOW_CALL_CAP`,看板只展示账本水位,不 import `muse_llm`。
- 看板只读库,不经过会写库的通道;它的存在和死活都不影响创作链。
## 7. 看板必须正确呈现的库内约定(防误导)

View File

@ -5,7 +5,7 @@ description: 通过唯一受控入口查询或修改 muse-example PostgreSQL,
# 访问 muse-example 数据库
对应 muse API 面:数据访问层。连接事实与凭据见 [`db/连接信息.md`](../../../db/连接信息.md)(DSN 已锁死在脚本内,只连 `muse-example`)。
对应 muse API 面:数据访问层。连接事实与凭据见 [`db/连接信息.md`](../../../db/连接信息.md)。连接实现在共享运行时包 `muse_db`(DSN 锁死 `muse-example`):本 Skill 提供人和主会话用的 CLI,其它 Skill 与只读看板直接 `from muse_db import connect`,不 import 本目录脚本。
## 用法(仓库根目录执行,python 一律用 `.venv/bin/python`)
@ -36,7 +36,7 @@ description: 通过唯一受控入口查询或修改 muse-example PostgreSQL,
## 红线
- **只连 `muse-example`**:DSN 硬编码锁库;严禁改造脚本去碰共享 PG 上的 muse_local / muse_slice_live / *_test。
- **只连 `muse-example`**:DSN 锁死在 `muse_db` 内;严禁另拼连接串去碰共享 PG 上的 muse_local / muse_slice_live / *_test。
- 软删约定照主仓:删除=UPDATE `deleted=TRUE`,不物理删(example_* 表同样遵守)。
- 批量导入/嵌入等专用写路径由 `import-book`/`embed-knowledge` Skill 封装(内部同走 psycopg 直连),本 Skill 承担通用查改与 DDL 应用。
- 建表/改表先落 `db/ddl/` 文件再 `apply`,不敲一次性 DDL——文件即审计。

View File

@ -1,34 +1,22 @@
#!/usr/bin/env python3
"""muse-example 唯一数据库通道(access-database Skill 脚本层)。
- DSN 锁死 muse-example:严禁触碰共享 PG 上其他库(muse_local / muse_slice_live / *_test)。
- query 卡片式打印=审查面;exec 报影响行数;apply 整文件一个事务失败全回滚。
- 失败原样抛错不静默(公约)。
连接本身归共享模块 `muse_db`:本文件只提供 CLI,不再作为其它 Skill 的 import 目标。
"""
import json
import sys
import click
import psycopg
from muse_db import connect
from psycopg import sql
# 连接事实与凭据来源:db/连接信息.md(内网 Tailscale 段,凭据明文入仓为既定政策)
DSN = "postgresql://root:f6710e2d0294eb1c10e26a805a64bc54@100.64.0.8:5433/muse-example"
TRUNC = 160 # 卡片模式长值截断阈值(字符)
def connect(readonly: bool = False):
"""统一连接入口:复用锁死的 muse-example DSN(即开即关,Tailscale 不持长事务)。
readonly=True 时会话级锁死只读(写语句被 PG 直接拒)——query 命令与看板用。
其它 skill 的写路径需要参数化短连接时,`from db import connect` 复用同一 DSN,
不要各自硬编码连接串(仿 call-content-model 的 _bump_window)。
"""
if readonly:
return psycopg.connect(DSN, options="-c default_transaction_read_only=on")
return psycopg.connect(DSN)
def _fmt(value, full: bool) -> str:
"""卡片值格式化:NULL 显示 ∅;长值默认截断并标注总长。"""
if value is None:

View File

@ -13,11 +13,10 @@ import json
import pathlib
import sys
import psycopg
from muse_db import connect
import yaml
from psycopg.types.json import Jsonb
DSN = "postgresql://root:f6710e2d0294eb1c10e26a805a64bc54@100.64.0.8:5433/muse-example"
SCHEMA_DIR = pathlib.Path(__file__).resolve().parents[4] / "meta" / "schemas"
TENANT, ACTOR = 1, "1" # 实验写入约定:系统主账号
@ -112,7 +111,7 @@ def main():
if not files:
sys.exit(f"未找到 schema YAML: {SCHEMA_DIR}")
rows = []
with psycopg.connect(DSN) as conn:
with connect() as conn:
for p in files:
doc = yaml.safe_load(p.read_text())
rows.append(seed_one(conn, doc, p.name))

View File

@ -16,7 +16,7 @@ import json
import re
from pathlib import Path
from db import connect # 复用锁死的 DSN(与 db.py 同目录,脚本目录自动在 sys.path)
from muse_db import connect
ROOT = Path(__file__).resolve().parents[4] # .claude/skills/access-database/scripts → 仓库根
AGENTS_DIR = ROOT / ".claude" / "agents"

View File

@ -10,14 +10,9 @@ from __future__ import annotations
import copy
import hashlib
import json
import sys
from pathlib import Path
from typing import Any, Mapping, Sequence
_HUMANIZATION_SRC = Path(__file__).resolve().parents[4] / "humanization" / "src"
if str(_HUMANIZATION_SRC) not in sys.path:
sys.path.insert(0, str(_HUMANIZATION_SRC))
from deai.schemas import validate as validate_humanization_contract # noqa: E402
from deai.schemas import validate as validate_humanization_contract
from writer_contract import (
ContractError,

View File

@ -12,10 +12,7 @@ import json
import sys
from pathlib import Path
# 复用 access-database Skill 锁死的 DSN
DB_SCRIPTS = Path(__file__).resolve().parents[2] / "access-database" / "scripts"
sys.path.insert(0, str(DB_SCRIPTS))
from db import connect # noqa: E402
from muse_db import connect
CREATOR = "read-context"

View File

@ -7,16 +7,14 @@ description: 通过 New-API 的统一治理入口调用内容模型,执行额
创始人拍板(2026-07-13):清洗与拆书的内容生产 LLM **全部走 New-API 的 MiniMax-M3**;主会话(Fable5)只固化 agent/提示词/skill 与发起调用。本 skill 是唯一出口。
管线内容生产调用的**标准入口是 `chat_governed`**(受 5 小时额度窗 + 全局降级链治理);`chat`/`chat` CLI 是不受治理的直连,仅供调试。治理政策的机械事实源是运行配置、共享额度账本 `example_llm_quota` 和模型运行适配器(`llm.py`),模型链切换必须由该 skill 治理并留下日志。
管线内容生产调用的**标准入口是 `chat_governed`**(受 5 小时额度窗 + 全局降级链治理);`chat`/`chat` CLI 是不受治理的直连,仅供调试。治理政策的机械事实源是运行配置、共享额度账本 `example_llm_quota` 和模型运行适配器——库实现装为共享包 `muse-llm`(`-e ./muse-llm`),调用方 `from muse_llm import chat_governed`,`scripts/llm.py` 只是本 Skill 的 CLI;模型链切换必须由该 skill 治理并留下日志。
## 用法
管线内所有内容生产型 LLM 调用(清洗探测、拆书抽取、知识卡审核等)**必须用 `chat_governed`**:
```python
# 其他 skill 内 import(clean_detect / deconstruct-book / review-knowledge-cards 的标准姿势)
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[2] / "llm" / "scripts"))
from llm import chat_governed, extract_json
from muse_llm import chat_governed, extract_json
content, usage, used_model = chat_governed(prompt, system=IDENTITY)
if used_model is None:

View File

@ -1,389 +1,11 @@
#!/usr/bin/env python3
"""call-content-model Skill:New-API 统一调用入口(默认 MiniMax-M3)。
管线内所有内容生产型 LLM 调用(清洗探测/拆书抽取)必须经此入口:
- trust_env=False(本机代理环境变量会劫持内网直连,教训固化);
- 超时 + 指数退避重试;<think> 剥离;JSON 三级容错提取(json-repair 兜底);
- 每次调用向 stderr 打印 token 用量与耗时(成本审计),stdout 只出内容。
"""
"""call-content-model Skill CLI。库实现安装为 muse_llm。"""
import json
import pathlib
import re
import sys
import time
import click
import psycopg
import requests
BASE = "http://100.64.0.8:3000"
# New-API 普通令牌(仓库政策允许明文;严禁换管理令牌打 /v1)
TOKEN = "sk-DyVqO3lDmEvQZ3PqGpbNaaaHZHhbh0xaHRIiynhYSmVlLHl2"
DEFAULT_MODEL = "MiniMax-M3"
# ── 额度治理常量(B: 把散在各调用方的降级链上收到 chat_governed 统一治理)──
# 额度账本 DSN(内网 Tailscale,凭据明文入仓为既定政策;带 keepalives 防长空转被掐)
QUOTA_DSN = ("postgresql://root:f6710e2d0294eb1c10e26a805a64bc54@100.64.0.8:5433/muse-example"
"?keepalives=1&keepalives_idle=15&keepalives_interval=5&keepalives_count=3")
MINIMAX_MODELS = {"MiniMax-M3", "MiniMax-M2.7"} # 计入每窗预算的模型
BUDGET_CHAIN = ["MiniMax-M3", "MiniMax-M2.7", "glm-5.2", "deepseek-v4-flash"] # 全局统一降级链
WINDOW_BUDGET_USD = 24.0 # 每窗 MiniMax 花费上限(创始人 2026-07-18 提额 $10→$24),超则切 glm-5.2→deepseek
WINDOW_CALL_CAP = 6000 # 每窗全模型调用上限(创始人 2026-07-18 提额 4000→6000),达则自动睡到下一窗续跑
# 上游实测上限:请求前主动裁剪,避免依赖不同渠道含混甚至错误的 HTTP 400 文案再猜测重发。
# M3 / deepseek 未观察到该限制,故不在表内、不主动裁剪。
MODEL_MAX_TOKENS = {
"MiniMax-M2.7": 196608,
"glm-5.2": 12000,
}
# 费率兜底(model_ratio, completion_ratio, cache_ratio),与 New-API /api/pricing 一致(2026-07-16 快照)
PRICING_FALLBACK = {
"MiniMax-M3": (0.15, 4.0, 0.2),
"MiniMax-M2.7": (0.15, 4.0, 0.2),
"glm-5.2": (0.5634, 3.5, 0.25),
"deepseek-v4-flash": (0.07, 2.0, 0.071428571429),
}
class SensitiveError(Exception):
"""上游内容安全拦截(响应体含 sensitive,如 new_sensitive 1026)。
同模型退避重试必再触发(放量实测每敏感章空烧 3 次),故不在此退避,
立即抛给上层走模型降级链(创始人 2026-07-14:M3→MiniMax-M2.7→deepseek-v4-flash)。"""
class PlanQuotaExhausted(Exception):
"""上游模型渠道的 Token Plan 已耗尽。
该错误在同一额度窗内重试不会恢复,必须立即交给治理层熔断当前模型;它与普通限流 429
不同,普通 429 仍保留指数退避重试。"""
def _default_persist_call(event):
"""按需加载运行证据持久化器,避免离线调用被迫连库。"""
evidence_scripts = (
pathlib.Path(__file__).resolve().parents[2]
/ "record-run-evidence"
/ "scripts"
)
if str(evidence_scripts) not in sys.path:
sys.path.insert(0, str(evidence_scripts))
from persist_llm_call import persist_call
return persist_call(event)
def chat(prompt, model=DEFAULT_MODEL, max_tokens=512000, temperature=0.2,
retries=2, timeout=900, system=None, top_p=None, *, run_id=None,
caller=None, requested_model_id=None, persist_call=None):
"""单轮对话,返回 (content, usage)。网络错/5xx/普通 429 指数退避重试。
content 已剥离 <think>…</think>(推理模型可能把思考混进正文)。
system:身份段与任务材料分离(角色遵从更稳、身份段利于上游缓存)。
top_p:随 temperature 分化实验用(M 家族官方推荐 1.0/0.95,eval A/B 后定版)。
max_tokens 默认 512000;仅对有实测硬上限的 M2.7/GLM 请求前主动裁剪。
预扣费机制备忘:New-API 按 max_tokens 预扣(512k 预扣 $0.15375/次,网关已验证接受该值;
结算按实际用量,余额充足时预扣不产生额外成本)——**余额须 ≥ 并发路数 × $0.154**,
否则触发 403「预扣费额度失败」(2026-07-15 余额见底实测坐实此机制)。
"""
if persist_call is None and (run_id or caller):
persist_call = _default_persist_call
if persist_call is not None and not callable(persist_call):
raise TypeError("persist_call 必须是可调用对象")
s = requests.Session()
s.trust_env = False # 本机代理 env 会劫持内网直连
messages = ([{"role": "system", "content": system}] if system else []) \
+ [{"role": "user", "content": prompt}]
model_cap = MODEL_MAX_TOKENS.get(model)
effective_max_tokens = min(max_tokens, model_cap) if model_cap is not None else max_tokens
if effective_max_tokens != max_tokens:
print(f"[llm] {model} max_tokens={max_tokens} 主动裁为模型上限 {effective_max_tokens}",
file=sys.stderr)
payload = {
"model": model,
"messages": messages,
"max_tokens": effective_max_tokens,
"temperature": temperature,
}
if top_p is not None:
payload["top_p"] = top_p
prompt_raw = json.dumps({"messages": messages, **payload},
ensure_ascii=False, sort_keys=True, separators=(",", ":"))
requested_model_id = requested_model_id or model
last_err = None
for attempt in range(retries + 1):
try:
t0 = time.time()
r = s.post(f"{BASE}/v1/chat/completions",
headers={"Authorization": f"Bearer {TOKEN}"},
json=payload, timeout=timeout)
# Token Plan 耗尽不是瞬时限流:同模型退避只会白等 8/16 秒,立即交治理层按窗熔断。
if r.status_code == 429 and "Token Plan 用量上限" in r.text:
raise PlanQuotaExhausted(f"Token Plan 已耗尽 HTTP 429: {r.text[:200]}")
# 内容安全拦截:同模型退避重试必再敏感,立即抛 SensitiveError 交上层
# 降级换模型,不在此浪费退避(否则一敏感章空烧 3 次,实测占放量请求 23%)
if r.status_code >= 500 and "sensitive" in r.text.lower():
raise SensitiveError(f"内容安全拦截 HTTP {r.status_code}: {r.text[:150]}")
# 429/5xx 属于可重试的服务端瞬时问题
if r.status_code in (429,) or r.status_code >= 500:
last_err = f"HTTP {r.status_code}: {r.text[:200]}"
raise requests.RequestException(last_err)
r.raise_for_status()
data = r.json()
content = data["choices"][0]["message"]["content"] or ""
content = re.sub(r"<think>.*?</think>", "", content, flags=re.S).strip()
usage = data.get("usage", {})
# 缓存命中数(OpenAI 式 prompt_tokens_details.cached_tokens)——验证前缀缓存是否生效、省了多少
cached = (usage.get("prompt_tokens_details") or {}).get("cached_tokens", 0)
print(f"[llm] {model} in={usage.get('prompt_tokens', '?')} "
f"cached={cached} out={usage.get('completion_tokens', '?')} "
f"耗时{time.time() - t0:.0f}s finish={data['choices'][0].get('finish_reason')}",
file=sys.stderr)
if persist_call is not None:
persist_call({
"window_key": window_key(_now()),
"run_id": run_id,
"caller": caller or "",
"requested_model_id": requested_model_id,
"actual_model_id": model,
"usage": usage,
"cost_usd": cost_usd(model, usage),
"stop_reason": data["choices"][0].get("finish_reason"),
"duration_ms": max(0, int(round((time.time() - t0) * 1000))),
"prompt": prompt_raw,
"response": json.dumps(data, ensure_ascii=False, sort_keys=True,
separators=(",", ":"), default=str),
"role": caller,
})
return content, usage
except (requests.RequestException, KeyError, json.JSONDecodeError) as e:
last_err = str(e)
if attempt < retries:
wait = 8 * (2 ** attempt)
print(f"[llm] 第{attempt + 1}次失败({last_err[:120]}),{wait}s 后重试",
file=sys.stderr)
time.sleep(wait)
raise RuntimeError(f"LLM 调用重试耗尽: {last_err}")
def extract_json(text):
"""JSON 三级容错提取:直接解析 → 首尾括号截取 → json-repair 兜底。
opus 试拆实测过两类 JSON 病(中文引号、缺逗号)——任何模型都可能犯,统一在此兜住。
"""
try:
return json.loads(text)
except json.JSONDecodeError:
pass
# 剥 markdown 代码围栏后按最外层大括号/中括号截取
t = re.sub(r"^```(?:json)?\s*|\s*```$", "", text.strip(), flags=re.M)
for a, b in (("{", "}"), ("[", "]")):
i, j = t.find(a), t.rfind(b)
if i != -1 and j > i:
frag = t[i:j + 1]
try:
return json.loads(frag)
except json.JSONDecodeError:
import json_repair
return json_repair.loads(frag)
import json_repair
return json_repair.loads(t)
# ══ 额度治理层(chat_governed)══
# WHY 上收:此前每个调用方各写一套模型降级链(parse_llm/parse_outline/review_cards 各一份,
# 链名/顺序还不一致),既无法全局限预算、也无法跨进程共享"这一窗烧了多少/调了多少次"。
# 统一到 chat_governed 后:一本共享账本按 5 小时窗计钱计次,MiniMax 超 $24/窗自动切非 MiniMax 链,
# 全窗调用达 6000 次自动睡到下一窗续跑——降级策略只此一处,调用方只管拿结果。
_PRICING_CACHE = None
# 仅保存当前额度窗内已确认 Token Plan 耗尽的模型。进程重启会自然重探;跨窗也会清空重探。
_PLAN_QUOTA_OPEN = {}
def _plan_quota_open_models(wk):
"""返回当前窗已熔断模型集合,并清除其他窗口的陈旧状态。"""
stale = [key for key in _PLAN_QUOTA_OPEN if key != wk]
for key in stale:
del _PLAN_QUOTA_OPEN[key]
return _PLAN_QUOTA_OPEN.setdefault(wk, set())
def get_pricing():
"""返回 {model: (model_ratio, completion_ratio, cache_ratio)}。
进程内只拉一次 /api/pricing;拉取失败或字段异常时回退硬编码,绝不因定价接口抖动崩管线。"""
global _PRICING_CACHE
if _PRICING_CACHE is not None:
return _PRICING_CACHE
# WHY 先复制兜底再逐字段覆盖:定价接口只是"锦上添花",任何一环出问题都必须能退回硬编码,
# 让成本核算继续跑;只认能解析成正数的字段,脏数据/0/负数一律不覆盖(算废预算比抖动更危险)。
merged = {m: list(r) for m, r in PRICING_FALLBACK.items()}
try:
s = requests.Session()
s.trust_env = False # 与 chat 同源:本机代理 env 会劫持内网直连
r = s.get(f"{BASE}/api/pricing", timeout=10)
r.raise_for_status()
by_name = {row.get("model_name"): row
for row in (r.json().get("data") or []) if isinstance(row, dict)}
for m in merged:
row = by_name.get(m)
if not row:
continue
for idx, key in enumerate(("model_ratio", "completion_ratio", "cache_ratio")):
try:
v = float(row.get(key))
except (TypeError, ValueError):
continue # 字段缺失/非数:保留兜底值
if v > 0:
merged[m][idx] = v
except Exception as e:
# 网络/HTTP/JSON 任何异常:整体回退硬编码(不吃半拉子覆盖的脏账)
print(f"[llm] /api/pricing 拉取失败({type(e).__name__}),用兜底费率", file=sys.stderr)
_PRICING_CACHE = {m: tuple(r) for m, r in PRICING_FALLBACK.items()}
return _PRICING_CACHE
_PRICING_CACHE = {m: tuple(r) for m, r in merged.items()}
return _PRICING_CACHE
def cost_usd(model, usage):
"""按 New-API 口径算单次调用美元成本($1 = 500000 配额单位)。
cost = model_ratio × ((prompt-cached) + cached×cache_ratio + completion×completion_ratio) / 500000"""
pricing = get_pricing()
if model in pricing:
model_ratio, completion_ratio, cache_ratio = pricing[model]
else:
# 未知模型宁高估勿漏计(漏计会让预算穿底),用 M3 费率兜底并告警
model_ratio, completion_ratio, cache_ratio = pricing["MiniMax-M3"]
print(f"[llm] cost_usd 未知模型 {model},用 MiniMax-M3 费率兜底计价", file=sys.stderr)
prompt = usage.get("prompt_tokens", 0) or 0
completion = usage.get("completion_tokens", 0) or 0
cached = (usage.get("prompt_tokens_details") or {}).get("cached_tokens", 0) or 0
billable = (prompt - cached) + cached * cache_ratio + completion * completion_ratio
return model_ratio * billable / 500000
def _now():
from datetime import datetime
return datetime.now() # 单独封装便于单测打桩
def window_key(dt):
"""把时刻归到所属窗口边界键。窗口起点 0/5/10/15/20 点,末窗 20-24=4h。"""
wh = (dt.hour // 5) * 5 # 0..4→0,5..9→5,10..14→10,15..19→15,20..23→20
return f"{dt:%Y-%m-%d}T{wh:02d}"
def seconds_to_next_window(dt):
"""距下一窗边界的秒数(<5→05:00,<10→10:00,<15→15:00,<20→20:00,否则次日00:00)。"""
from datetime import timedelta
h = dt.hour
if h < 5:
boundary = dt.replace(hour=5, minute=0, second=0, microsecond=0)
elif h < 10:
boundary = dt.replace(hour=10, minute=0, second=0, microsecond=0)
elif h < 15:
boundary = dt.replace(hour=15, minute=0, second=0, microsecond=0)
elif h < 20:
boundary = dt.replace(hour=20, minute=0, second=0, microsecond=0)
else:
boundary = (dt + timedelta(days=1)).replace(hour=0, minute=0, second=0, microsecond=0)
# 至少 1 秒:边界精确命中时避免 0/负导致空睡后原地打转
return max(1, int((boundary - dt).total_seconds()))
def _read_window(wk):
"""读某窗账本,返回 (minimax_usd:float, total_calls:int);无行返回 (0.0,0)。短连接即关。"""
# WHY 短连接:LLM/sleep 期间绝不持 DB 连接(Tailscale 长事务空转会被掐断),读完立刻释放
with psycopg.connect(QUOTA_DSN) as c:
row = c.execute(
"SELECT minimax_usd, total_calls FROM example_llm_quota WHERE window_key=%s",
(wk,)).fetchone()
if not row:
return 0.0, 0
return float(row[0]), int(row[1])
def _bump_window(wk, add_usd):
"""原子累加:该窗 minimax_usd += add_usd、total_calls += 1,返回累加后的 (usd,calls)。
单语句 upsert,多分片共用一本账靠 PG 行锁串行化。短连接即关。"""
with psycopg.connect(QUOTA_DSN) as c:
row = c.execute(
"""INSERT INTO example_llm_quota (window_key, minimax_usd, total_calls, updated_at)
VALUES (%s, %s, 1, now())
ON CONFLICT (window_key) DO UPDATE
SET minimax_usd = example_llm_quota.minimax_usd + EXCLUDED.minimax_usd,
total_calls = example_llm_quota.total_calls + 1, updated_at = now()
RETURNING minimax_usd, total_calls""",
(wk, add_usd)).fetchone()
return float(row[0]), int(row[1]) # psycopg 返回 Decimal,转 float
def chat_governed(prompt, model=DEFAULT_MODEL, system=None, max_tokens=512000,
temperature=0.2, top_p=None, *, run_id=None, caller=None,
persist_call=None):
"""全局额度治理下的对话入口,返回 (content, usage, actual_model)。
契约:成功→三元组;全链耗尽(所有模型敏感/不可用)→(None,None,None)。
model 参数仅作兼容保留:实际用哪个模型由全局额度策略决定,不由调用方指定。
策略(每次调用前):
1) 读本窗账本;本窗 total_calls ≥ WINDOW_CALL_CAP → 打日志、睡到下一窗边界(不持DB连接)、重读续跑;
2) 本窗 minimax_usd ≥ WINDOW_BUDGET_USD → 降级链去掉 MiniMax 前缀(只剩 glm-5.2→deepseek),否则用全链;
3) 跳过本窗已确认 Token Plan 耗尽的模型;其余模型沿链调用,敏感/不可用时换下一个;成功即止;
4) 成功后 _bump_window(本窗, MiniMax模型才计成本否则0),返回三元组;全链失败返回 (None,None,None)。"""
from datetime import timedelta
while True:
wk = window_key(_now())
usd, calls = _read_window(wk) # 短连接读完即释放,下面 LLM/sleep 阶段不持连接
# 1) 调用数达上限:睡到下一窗边界再重来(睡眠期间不持任何 DB 连接)
if calls >= WINDOW_CALL_CAP:
now2 = _now()
secs = seconds_to_next_window(now2)
# 目标窗边界:secs 经 int() 截断可能落在边界前 <1s(如 04:59:59),+1s 归整到整分,
# 否则 %H 会把 04:59:59 显示成上一整点"04"、误导成非法边界(窗边界只有 00/05/10/15/20)
wake = (now2 + timedelta(seconds=secs + 1)).replace(second=0, microsecond=0)
print(f"[llm] 本窗 {wk} 已达 {calls} 次调用上限(≥{WINDOW_CALL_CAP}),"
f"睡 {secs // 60} 分钟到下一窗 {wake:%H:%M} 续跑", file=sys.stderr)
time.sleep(secs)
continue # 醒来重读账本:跨过窗边界后是新窗,calls 归 0
# 2) 预算耗尽:本窗改用非 MiniMax 链;否则用全链
if usd >= WINDOW_BUDGET_USD:
chain = [m for m in BUDGET_CHAIN if m not in MINIMAX_MODELS]
print(f"[llm] 本窗 {wk} MiniMax 花费 ${usd:.4f} 已达预算上限 ${WINDOW_BUDGET_USD},"
f"本窗改用非 MiniMax 链 {chain}", file=sys.stderr)
else:
chain = list(BUDGET_CHAIN)
plan_quota_open = _plan_quota_open_models(wk)
skipped = [m for m in chain if m in plan_quota_open]
if skipped:
print(f"[llm] 本窗 {wk} 跳过 Token Plan 已耗尽模型 {skipped}", file=sys.stderr)
chain = [m for m in chain if m not in plan_quota_open]
# 3) 沿链逐个模型调用;撞敏感/不可用换下一个
for m in chain:
try:
content, usage = chat(
prompt,
model=m,
system=system,
max_tokens=max_tokens,
temperature=temperature,
top_p=top_p,
run_id=run_id,
caller=caller,
requested_model_id=model,
persist_call=persist_call,
)
except PlanQuotaExhausted as e:
plan_quota_open.add(m)
print(f"[llm] 治理链 {m} Token Plan 本窗耗尽,立即熔断并降级下一个:{str(e)[:80]}",
file=sys.stderr)
continue
except (SensitiveError, RuntimeError) as e:
print(f"[llm] 治理链 {m} 失败({type(e).__name__}: {str(e)[:80]}),降级下一个",
file=sys.stderr)
continue
# 4) 成功记账:只有 MiniMax 计入 $24/窗 预算,其余模型成本计 0(只占调用数)
add = cost_usd(m, usage) if m in MINIMAX_MODELS else 0.0
_bump_window(wk, add) # 全新短连接原子累加,写完即释放
return content, usage, m
# 全链走完仍无成功:交上层处置(拆书硬停 / 判重保守 keep / 审核标 blocked)
return None, None, None
from muse_llm import DEFAULT_MODEL, chat, extract_json
@click.group()

View File

@ -117,8 +117,8 @@ disable-model-invocation: true
详细字段和失败码见 [`references/case-card-contract.md`](references/case-card-contract.md)。规则候选的完整合同评测见 `humanization/src/deai/evaluation.py`;`project-sample` 产出的四类样例可用 `--samples` 作为评测输入;样例带 `case_card_id` 时,评测还必须提供 `--cards` 与 `--verification`,脚本会复核 canonical、verified 投影、样例正文和规则引用。holdout 必须保留 `sf_hit`、`snf_false_repair`、`boundary_false_repair`、`regression_safe` 等分层计数及派生指标;没有 holdout 和人工审批,候选永远不能写成 active;唯一例外是所有者留痕豁免——激活门代码不放宽,豁免必须在规则 evidence 写明决定、日期与理由(见 humanization/rules 2026-08-16 批量豁免)。
首版回填清单与候选规则种子见 [`references/fixtures/`](references/fixtures/);其中既有作品只保留 hash/位置,不能直接确认。
`backfill-inventory-*.json` 与 `revalidation-*.json` 是可复核的导出/恢复证据;正式内容在 `muse-example` 的
`inventory`/`revalidate` 的 `--output` 导出是可复核的恢复证据;正式内容在 `muse-example` 的
`example_ai_flavor_case`、`example_ai_flavor_revalidation_batch`、`example_ai_flavor_revalidation` 三张表。
`dashboard/server.py` 的 `/ai-flavor` 默认查这三张表,数据库不可用时才明确标注离线回退;页面不会因打开而重新读取原文。
`dashboard/server.py` 的 `/ai-flavor` 默认查这三张表,数据库不可用时读它自己的 `dashboard/fixtures/` 离线回退(导出件由人放入,看板不读本 Skill 目录);页面不会因打开而重新读取原文。
状态语义:案例卡 `shadow` 只供复核,`canonical` 仅表示获授权且完成评审,`rejected`/`archived` 不进入生成上下文;重验证 `verified` 才能确认、投影样例或消费规则,`stale`(全文哈希变化)、`unavailable`(来源不可得)和 `card_mismatch`(锚点变化)都使当前卡在这些动作上失效,但历史回执保留。

View File

@ -3,11 +3,11 @@
这些文件是检测运行的导出/恢复证据;正式内容自动写入 PostgreSQL `muse-example`,不是靠这些文件承载。
- `backfill-hash-only.yaml`:从本地既有作品扫描得到的第一批候选;只保留全文哈希、片段哈希、位置和待复核观察,不含第三方正文。
- `backfill-inventory-2026-08-13.json`:对 `小说清单/` 8 本作品的批量回填导出,共 788 张 hash-only Shadow 卡;同一 `inventory` 命令已经自动写入 `example_ai_flavor_case`。
- `revalidation-2026-08-14.json`:同一检测运行的来源重验证回执;当前 `verified=788`、`stale=0`、`unavailable=0`、`card_mismatch=0`,同时追加到 `example_ai_flavor_revalidation_batch` + `example_ai_flavor_revalidation`。
- `canonical-samples.yaml`:公版/合成短片段的已确认样例,用于验证卡→样例投影和反例门。
- `rule-candidates.yaml`:由两类以上来源、同时含正反证据的候选规则;状态固定为 `candidate`,不能直接加载为生产 `active`。
`inventory`/`revalidate` 的 JSON 导出不留在本目录:看板的离线回退夹具归 [`dashboard/fixtures/`](../../../../../dashboard/fixtures/) 所有,看板不读本 Skill 目录。
页面入口:启动 `dashboard/server.py` 后访问 `/ai-flavor`。页面默认只读展示 PostgreSQL 正式表;数据库不可用时才显示离线回退。确认、样例投影和规则消费必须携带 `verified` 回执。
样例标签含义:`sf` = 当前评审认为应修,`snf` = 表面相似但有功能不应修,`boundary` = 需上下文裁决,`regression` = 错误修复回归。

View File

@ -21,13 +21,6 @@ from typing import Iterable
import yaml
_AGENT_ROOT = Path(__file__).resolve().parents[4]
_HUMANIZATION_SRC = _AGENT_ROOT / "humanization" / "src"
if str(_HUMANIZATION_SRC) not in sys.path:
sys.path.insert(0, str(_HUMANIZATION_SRC))
# 作为 CLI 执行时也注册稳定模块名,自动落库模块复用同一份合同异常类型,
# 避免失败路径被重复 import 变成未捕获 traceback。
if __name__ == "__main__":

View File

@ -15,14 +15,8 @@ from pathlib import Path
import yaml
SCRIPT_DIR = Path(__file__).resolve().parent
AGENT_ROOT = SCRIPT_DIR.parents[3]
for path in (
SCRIPT_DIR,
AGENT_ROOT / "humanization" / "src",
AGENT_ROOT / ".claude" / "skills" / "access-database" / "scripts",
):
if str(path) not in sys.path:
sys.path.insert(0, str(path))
if str(SCRIPT_DIR) not in sys.path:
sys.path.insert(0, str(SCRIPT_DIR))
from capture_cases import ( # noqa: E402
CaseCardError,
@ -32,8 +26,8 @@ from capture_cases import ( # noqa: E402
load_verification,
project_sample,
)
from db import connect # noqa: E402
from deai import evaluation, load # noqa: E402
from muse_db import connect # noqa: E402
class MiningError(ValueError):

View File

@ -1,9 +1,9 @@
#!/usr/bin/env python3
"""把 AI 味案例卡与来源重验证回执写入 agent-example 的 muse-example。
采集脚本保持确定性、可离线回放;本脚本是唯一的持久化边界,复用
``access-database`` 的连接入口。案例卡做幂等当前投影,重验证批次/回执做
append-only 账本。研究限定来源只写 hash、位置和观察,不写第三方正文。
采集脚本保持确定性、可离线回放;本脚本是唯一的持久化边界,连接来自共享的
``muse_db``。案例卡做幂等当前投影,重验证批次/回执做 append-only 账本。
研究限定来源只写 hash、位置和观察,不写第三方正文。
"""
from __future__ import annotations
@ -19,11 +19,8 @@ import yaml
SCRIPT_DIR = Path(__file__).resolve().parent
DB_SCRIPTS = SCRIPT_DIR.parents[1] / "access-database" / "scripts"
if str(SCRIPT_DIR) not in sys.path:
sys.path.insert(0, str(SCRIPT_DIR))
if str(DB_SCRIPTS) not in sys.path:
sys.path.insert(0, str(DB_SCRIPTS))
from capture_cases import ( # noqa: E402
CaseCardError,
@ -32,7 +29,7 @@ from capture_cases import ( # noqa: E402
load_verification,
validate_card,
)
from db import connect # noqa: E402
from muse_db import connect # noqa: E402
TENANT_ID = 1

View File

@ -8,14 +8,9 @@ import hashlib
import json
import pathlib
import re
import sys
from typing import Any, Mapping, Protocol, Sequence, runtime_checkable
SCRIPT_DIR = pathlib.Path(__file__).resolve().parent
EXECUTION_DIR = SCRIPT_DIR.parents[1] / "execute-claude-task" / "scripts"
if str(EXECUTION_DIR) not in sys.path:
sys.path.insert(0, str(EXECUTION_DIR))
try:
from claude_runtime import ExecutionProfile, contains_path_traversal, run_claude, sha256_json # type: ignore[import-not-found] # noqa: E402
except ImportError:

View File

@ -17,8 +17,7 @@ import sys
import click
import psycopg
DSN = ("postgresql://root:f6710e2d0294eb1c10e26a805a64bc54@100.64.0.8:5433/muse-example"
"?keepalives=1&keepalives_idle=15&keepalives_interval=5&keepalives_count=3")
from muse_db import connect
TENANT, ACTOR = 1, "1"
MIN_LEN, MAX_LEN = 4, 500
MAX_CH_RATIO = 0.20 # 单章累计删除上限(占章长比)
@ -96,7 +95,7 @@ def main(work_id, batch, file_, model, dry_run, report_md):
# 一次拉齐所有目标章及其邻章(±1),在内存统一匹配,减少 DB 往返
targets = sorted({n for no in by_ch for n in (no - 1, no, no + 1) if n >= 1})
stats = {"删除段": 0, "删除字数": 0, "拒绝": [], "弹性命中": 0, "邻章命中": 0}
with psycopg.connect(DSN) as conn:
with connect() as conn:
# 书名水印豁免:盗版源每章插孤立书名行——孤行精确等于本书书名不可能是正文叙述
work_title = (conn.execute(
"SELECT title FROM muse_content_work WHERE tenant_id=%s AND id=%s",

View File

@ -13,10 +13,8 @@ import subprocess
import sys
import click
import psycopg
DSN = ("postgresql://root:f6710e2d0294eb1c10e26a805a64bc54@100.64.0.8:5433/muse-example"
"?keepalives=1&keepalives_idle=15&keepalives_interval=5&keepalives_count=3")
from muse_db import connect
TENANT = 1
OUT = pathlib.Path("/tmp/muse-clean")
HERE = pathlib.Path(__file__).resolve().parent
@ -36,7 +34,7 @@ def run(args):
@click.option("--batch", required=True, help="放量批次号(幂等判断依据,重跑请保持一致)")
@click.option("--report-dir", default="docs", show_default=True)
def main(work_ids, batch, report_dir):
with psycopg.connect(DSN) as conn:
with connect() as conn:
titles = dict(conn.execute(
"SELECT id, title FROM muse_content_work WHERE tenant_id=%s AND deleted=FALSE AND id=ANY(%s)",
(TENANT, list(work_ids))).fetchall())
@ -58,7 +56,7 @@ def main(work_ids, batch, report_dir):
click.echo(f" [缺窗] {missing},本书暂不 apply(重跑本命令自动补)")
continue
# 3) 幂等防重删:该批次已落审计则跳过 apply
with psycopg.connect(DSN) as conn:
with connect() as conn:
done = conn.execute(
"SELECT 1 FROM example_clean_log WHERE work_id=%s AND batch=%s LIMIT 1",
(wid, batch)).fetchone()

View File

@ -13,9 +13,7 @@ import time
import click
# 统一走 call-content-model 受治理入口(额度窗/全局降级链/熔断 + trust_env/重试/<think>剥离/JSON 容错都在那边)
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[2] / "call-content-model" / "scripts"))
from llm import chat_governed, extract_json # noqa: E402
from muse_llm import chat_governed, extract_json
OUT = pathlib.Path("/tmp/muse-clean")

View File

@ -8,8 +8,7 @@ import sys
import click
import psycopg
DSN = ("postgresql://root:f6710e2d0294eb1c10e26a805a64bc54@100.64.0.8:5433/muse-example"
"?keepalives=1&keepalives_idle=15&keepalives_interval=5&keepalives_count=3")
from muse_db import connect
TENANT = 1
OUT = pathlib.Path("/tmp/muse-clean")
@ -20,7 +19,7 @@ OUT = pathlib.Path("/tmp/muse-clean")
@click.option("--from", "from_", type=int, default=1, help="起始章 order_no")
@click.option("--to", type=int, default=0, help="结束章 order_no(0=到末章)")
def main(work_id, window, from_, to):
with psycopg.connect(DSN) as conn:
with connect() as conn:
title = conn.execute("SELECT title FROM muse_content_work WHERE id=%s", (work_id,)).fetchone()[0]
sql = """SELECT c.order_no, c.title, b.content_text
FROM muse_content_chapter c JOIN muse_content_block b ON b.chapter_id=c.id AND b.deleted=FALSE

View File

@ -12,8 +12,7 @@ import sys
import click
import psycopg
DSN = ("postgresql://root:f6710e2d0294eb1c10e26a805a64bc54@100.64.0.8:5433/muse-example"
"?keepalives=1&keepalives_idle=15&keepalives_interval=5&keepalives_count=3")
from muse_db import connect
TENANT, ACTOR = 1, "1"
@ -32,7 +31,7 @@ def flex_pattern(exact):
help="手动种子(替代审计自动发现;用于人工确认过的碎水印,如孤行网址)")
@click.option("--dry-run", is_flag=True)
def main(work_id, batch, min_occur, min_len, manual_seeds, dry_run):
with psycopg.connect(DSN) as conn:
with connect() as conn:
# 种子:手动指定(人工确认的碎水印),或该书审计里的重复删除段(已过全部守卫的真垃圾)
seeds = list(manual_seeds) or [r[0] for r in conn.execute(
"""SELECT removed_text FROM example_clean_log

View File

@ -9,16 +9,10 @@ check_writer_acceptance 是无副作用纯函数,不碰库;本模块负责
from __future__ import annotations
import json
import pathlib
import sys
from datetime import datetime, timedelta, timezone
from typing import Any, Mapping
DB_DIR = pathlib.Path(__file__).resolve().parents[2] / "access-database" / "scripts"
if str(DB_DIR) not in sys.path:
sys.path.insert(0, str(DB_DIR))
from db import connect # noqa: E402
from muse_db import connect
PRODUCTION_POLICY = "writer-production-v1"
# 候选接受窗口:生成后 24 小时内必须完成接受,超期 preflight 报 CANDIDATE_EXPIRED。

View File

@ -8,14 +8,10 @@
"""
import argparse
import json
import pathlib
import sys
from copy import deepcopy
DB_SCRIPTS = pathlib.Path(__file__).resolve().parents[2] / "access-database" / "scripts"
sys.path.insert(0, str(DB_SCRIPTS))
from db import connect # noqa: E402
from muse_db import connect
TENANT = 1

View File

@ -22,10 +22,8 @@ from typing import Any, Mapping, Sequence
SCRIPT_DIR = pathlib.Path(__file__).resolve().parent
SKILLS_DIR = SCRIPT_DIR.parents[1]
READ_CONTEXT_DIR = SKILLS_DIR / "assemble-context" / "scripts"
DB_DIR = SKILLS_DIR / "access-database" / "scripts"
for path in (READ_CONTEXT_DIR, DB_DIR):
if str(path) not in sys.path:
sys.path.insert(0, str(path))
if str(READ_CONTEXT_DIR) not in sys.path:
sys.path.insert(0, str(READ_CONTEXT_DIR))
from writer_contract import normalize_text # noqa: E402
@ -270,7 +268,7 @@ def propose_fact_deltas(
) -> dict[str, Any]:
"""抽取侧登记增量**提案**(status=proposed);升格必须另行显式批准。"""
from db import connect # 延迟导入:纯校验路径(测试)不需要库
from muse_db import connect # 延迟导入:纯校验路径(测试)不需要库
normalized = validate_delta_batch(
deltas, candidate_body=candidate_body, target_chapter=target_chapter)

View File

@ -15,16 +15,9 @@
from __future__ import annotations
import hashlib
import pathlib
import sys
from typing import Any, Iterable
SCRIPT_DIR = pathlib.Path(__file__).resolve().parent
DB_DIR = SCRIPT_DIR.parents[1] / "access-database" / "scripts"
if str(DB_DIR) not in sys.path:
sys.path.insert(0, str(DB_DIR))
from db import connect # noqa: E402
from muse_db import connect
PROJECTION_KINDS = frozenset({"summary", "handoff", "embedding", "extraction", "dashboard"})
CREATOR = "confirm"

View File

@ -6,13 +6,8 @@
"""
import argparse
import json
import pathlib
import sys
DB_SCRIPTS = pathlib.Path(__file__).resolve().parents[2] / "access-database" / "scripts"
sys.path.insert(0, str(DB_SCRIPTS))
from db import connect # noqa: E402
from muse_db import connect
TENANT = 1

View File

@ -21,12 +21,9 @@ import json
import sys
from pathlib import Path
# 复用 access-database Skill 锁死的 DSN,不另硬编码连接串
DB_SCRIPTS = Path(__file__).resolve().parents[2] / "access-database" / "scripts"
sys.path.insert(0, str(DB_SCRIPTS))
from db import connect # noqa: E402
from fact_delta import FactDeltaError, apply_accepted_deltas # noqa: E402
from projection_registry import ( # noqa: E402
from fact_delta import FactDeltaError, apply_accepted_deltas
from muse_db import connect
from projection_registry import (
mark_stale_before_revision, register_pending_projections,
)

View File

@ -18,9 +18,7 @@ import sys
import click
import psycopg
sys.path.insert(0, str(__import__("pathlib").Path(__file__).resolve().parents[2] / "call-content-model" / "scripts"))
# DSN 的真实来源是同目录的 parse_llm(升格执行器拆分后不再经 upgrade 转导)。
from parse_llm import DSN # noqa: E402
from muse_db import connect
def render_val(v, limit=6):
@ -44,7 +42,7 @@ def cli():
def export_patterns(top, combat_top, out):
"""五书范式卡终态样张:统计面貌 + 每型实例数 top 代表卡。"""
lines = []
with psycopg.connect(DSN) as conn:
with connect() as conn:
works = conn.execute(
"""SELECT DISTINCT draft_payload->'出处'->>'书名' FROM muse_knowledge_draft
WHERE source_type='parse_book' AND deleted=FALSE ORDER BY 1""").fetchall()
@ -108,7 +106,7 @@ def export_patterns(top, combat_top, out):
def export_upgrade(work_id, top, out):
"""单书升格实体卡终态样张:统计面貌 + 头部实体全字段生长轨迹 + 各型代表。"""
lines = []
with psycopg.connect(DSN) as conn:
with connect() as conn:
title = conn.execute("SELECT title FROM muse_content_work WHERE id=%s", (work_id,)).fetchone()[0]
cards = conn.execute(
"""SELECT d.id, d.draft_payload FROM muse_knowledge_draft d

View File

@ -13,10 +13,9 @@ import sys
import click
import psycopg
sys.path.insert(0, str(__import__("pathlib").Path(__file__).resolve().parents[2] / "call-content-model" / "scripts"))
# DSN/TENANT 的真实来源是同目录的 parse_llm(升格执行器拆分后不再经 upgrade 转导);_is_garbage 是死 import,删。
from parse_llm import DSN, TENANT # noqa: E402
from parse_ingest import IP_LEAK_WORDS # noqa: E402
from muse_db import connect
from parse_llm import TENANT
from parse_ingest import IP_LEAK_WORDS
def sec_chapter(conn, wid):
@ -151,7 +150,7 @@ def sec_health(conn):
@click.command()
@click.option("--work-id", type=int, default=0, help="只体检指定书(0=全部)")
def health(work_id):
with psycopg.connect(DSN) as conn:
with connect() as conn:
works = conn.execute(
"SELECT id, title FROM muse_content_work WHERE deleted=FALSE"
+ (" AND id=%s" % work_id if work_id else "") + " ORDER BY id").fetchall()

View File

@ -20,14 +20,10 @@ import click
import psycopg
from psycopg.types.json import Jsonb
# 受控点依赖:嵌入走 embed-knowledge,归并判定走 call-content-model
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[2] / "embed-knowledge" / "scripts"))
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[2] / "call-content-model" / "scripts"))
from embed_drafts import _session, build_embed_text, embed_texts # noqa: E402
from llm import chat_governed, extract_json # noqa: E402 # 归并判定走全局额度治理入口
from muse_embed import _session, build_embed_text, embed_texts
from muse_llm import chat_governed, extract_json # 归并判定走全局额度治理入口
DSN = ("postgresql://root:f6710e2d0294eb1c10e26a805a64bc54@100.64.0.8:5433/muse-example"
"?keepalives=1&keepalives_idle=15&keepalives_interval=5&keepalives_count=3")
from muse_db import connect
TENANT, ACTOR = 1, "1"
PATTERN_TYPES = {"craft", "combat", "emotion", "scene_pattern", "trope"} # 拍板①:首轮只拆五型
NGRAM = 15 # 脱敏红线:≥15 连续字与原文重合=违规(deconstruct-book)
@ -211,7 +207,7 @@ def cli():
@click.option("--to", type=int, required=True)
def init_tasks(work_id, from_, to):
"""按章建任务行(幂等),并把参考书档案 parse_scope/parse_status 置为拆书中。"""
with psycopg.connect(DSN) as conn:
with connect() as conn:
chs = conn.execute(
"""SELECT id, order_no FROM muse_content_chapter
WHERE tenant_id=%s AND work_id=%s AND order_no BETWEEN %s AND %s AND deleted=FALSE
@ -239,7 +235,7 @@ def init_tasks(work_id, from_, to):
def scaffold(work_id, chapter_order, file_):
"""章级入库:{细纲, 实体:[{型,名称,一句话摘要}], 线索:[{型,短名,线索,证据}]};比例约束校验。"""
data = norm_scaffold(json.loads(pathlib.Path(file_).read_text()))
with psycopg.connect(DSN) as conn:
with connect() as conn:
ch_id, src = chapter_of(conn, work_id, chapter_order)
outline = (data.get("细纲") or "").strip()
if not outline:
@ -294,7 +290,7 @@ def patterns(work_id, chapter_order, file_):
if not isinstance(raw, list):
raise click.ClickException("patterns 文件须为卡片数组")
cards = [norm_card(c) for c in raw]
with psycopg.connect(DSN) as conn:
with connect() as conn:
ch_id, src = chapter_of(conn, work_id, chapter_order)
contracts = {t: field_contract(conn, t) for t in PATTERN_TYPES}
book = conn.execute("SELECT title FROM muse_content_work WHERE id=%s", (work_id,)).fetchone()[0]
@ -373,7 +369,7 @@ def cards(work_id, from_order, file_, model):
if isinstance(raw, dict):
raw = raw.get("cards") or raw.get("卡") or []
cards_in = [norm_card(c) for c in raw]
with psycopg.connect(DSN) as conn:
with connect() as conn:
win = conn.execute(
"""SELECT to_order FROM example_parse_outline
WHERE tenant_id=%s AND work_id=%s AND from_order=%s AND deleted=FALSE""",
@ -566,7 +562,7 @@ def recluster(ptype, dry_run, limit, sample, model):
写段 全新短连接批量落库(母卡累积 payload / 输家卡 deleted / 输家嵌入 deleted),一次提交。
双保险:embedding 只初筛出候选,是否同一手法一律由 merge_judge 定夺,拿不准 keep(宁重复不误并)。"""
# ── 读段:短连接读完即释放 ──
with psycopg.connect(DSN) as rconn:
with connect() as rconn:
rows = rconn.execute(
"""SELECT d.id, d.draft_payload FROM muse_knowledge_draft d
WHERE d.tenant_id=%s AND d.source_type='parse_book' AND d.deleted=FALSE
@ -643,7 +639,7 @@ def recluster(ptype, dry_run, limit, sample, model):
# ── 写段:全新短连接批量落库(红线:软删,绝不物理删)──
if not dry_run and plan:
movers = {lid for lid, _lo, _s in plan} # 被吸并过的母卡(去重,每张只写一次最终态)
with psycopg.connect(DSN) as wconn:
with connect() as wconn:
for lid in movers:
wconn.execute("UPDATE muse_knowledge_draft SET draft_payload=%s, updater=%s WHERE id=%s",
(Jsonb(cards[lid]), ACTOR, lid))
@ -713,7 +709,7 @@ def recluster(ptype, dry_run, limit, sample, model):
@click.option("--work-id", type=int)
def progress(work_id):
"""进度统计(审查面)。"""
with psycopg.connect(DSN) as conn:
with connect() as conn:
where = " AND t.work_id=%s" if work_id else ""
args = [TENANT] + ([work_id] if work_id else [])
rows = conn.execute(f"""

View File

@ -23,14 +23,10 @@ import sys
import click
import psycopg
# 统一走 call-content-model 入口(trust_env/重试/<think>剥离/JSON 容错都在那边)
# 敏感/额度降级链已上收 llm.chat_governed(全局统一治理),本模块不再自持降级链
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[2] / "call-content-model" / "scripts"))
from llm import chat_governed, extract_json # noqa: E402
from parse_outline import ensure_outline_coverage # noqa: E402
from muse_db import connect
from muse_llm import chat_governed, extract_json
from parse_outline import ensure_outline_coverage
DSN = ("postgresql://root:f6710e2d0294eb1c10e26a805a64bc54@100.64.0.8:5433/muse-example"
"?keepalives=1&keepalives_idle=15&keepalives_interval=5&keepalives_count=3")
TENANT = 1
HERE = pathlib.Path(__file__).resolve().parent
TMP = pathlib.Path("/tmp/muse-parse")
@ -394,7 +390,7 @@ def cli():
def chapters(work_id, from_, to, model, incomplete_only):
"""章级 pass:逐章一次 M3(细纲+实体+范式候选线索)。正文只过这一遍。"""
initialize_tasks(work_id, from_, to, incomplete_only)
with psycopg.connect(DSN) as conn:
with connect() as conn:
title = conn.execute("SELECT title FROM muse_content_work WHERE id=%s", (work_id,)).fetchone()[0]
targets = chapter_orders(conn, work_id, from_, to, incomplete_only)
if incomplete_only:
@ -402,7 +398,7 @@ def chapters(work_id, from_, to, model, incomplete_only):
total_in = total_out = 0
cache_hits = cache_misses = 0
for ch in targets:
with psycopg.connect(DSN) as conn:
with connect() as conn:
row = conn.execute(
"""SELECT c.id, c.title, b.content_text, t.scaffold_status
FROM muse_content_chapter c
@ -420,7 +416,7 @@ def chapters(work_id, from_, to, model, incomplete_only):
try:
def full_extract():
"""仅在缓存未命中时查询判重索引并执行正文完整抽取。"""
with psycopg.connect(DSN) as prev_conn:
with connect() as prev_conn:
prev = [e for (ents,) in prev_conn.execute(
"""SELECT s.entities FROM example_parse_scaffold s
JOIN muse_content_chapter c ON c.id=s.chapter_id
@ -447,7 +443,7 @@ def chapters(work_id, from_, to, model, incomplete_only):
click.echo(f" {out}")
except SensitiveHardStop as e:
# 降级链(主+2 备)全撞敏感——按创始人指令硬停该书解析并汇报,不跳过、不硬扛
with psycopg.connect(DSN) as conn:
with connect() as conn:
conn.execute(
"""UPDATE example_parse_task SET scaffold_status='failed', error_message=%s
WHERE tenant_id=%s AND work_id=%s AND chapter_id=(
@ -472,7 +468,7 @@ def chapters(work_id, from_, to, model, incomplete_only):
@click.option("--redo", is_flag=True, help="窗内已有活卡也重出(默认跳过=断点续跑)")
def cards(work_id, from_order, model, redo):
"""窗级出卡:逐窗一次 M3 聚类归并(窗=example_parse_outline 行,先跑 parse_outline window)。"""
with psycopg.connect(DSN) as conn:
with connect() as conn:
title = conn.execute("SELECT title FROM muse_content_work WHERE id=%s", (work_id,)).fetchone()[0]
contracts = load_contracts(conn) # prompt 与 ingest 守卫同源(库内 schema 快照)
sql = """SELECT from_order, to_order, outline_text, window_no FROM example_parse_outline
@ -485,7 +481,7 @@ def cards(work_id, from_order, model, redo):
if not wins:
raise click.ClickException("无大纲窗行——先跑 parse_outline.py window(窗是出卡的切分依据)")
if from_order is None:
with psycopg.connect(DSN) as conn:
with connect() as conn:
bounds = conn.execute(
"""SELECT min(order_no), max(order_no) FROM muse_content_chapter
WHERE tenant_id=%s AND work_id=%s AND deleted=FALSE""",
@ -500,7 +496,7 @@ def cards(work_id, from_order, model, redo):
)
total_in = total_out = 0
for a, b, stage_ol, wno in wins:
with psycopg.connect(DSN) as conn:
with connect() as conn:
# 断点续跑:窗内已有活卡(本窗出的)则跳过
if not redo and conn.execute(
"""SELECT 1 FROM muse_knowledge_draft WHERE tenant_id=%s AND source_type='parse_book'

View File

@ -7,14 +7,12 @@
前置:窗内章须已有脚手架细纲(example_parse_scaffold),缺则报缺不硬抽。
"""
import json
import pathlib
import sys
import click
import psycopg
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[2] / "call-content-model" / "scripts"))
from llm import chat_governed, extract_json, SensitiveError # noqa: E402
from muse_llm import chat_governed, extract_json, SensitiveError
# 敏感/额度降级链已上收 llm.chat_governed(BUDGET_CHAIN 全局统一 + 额度治理);本模块不再自持降级链。
@ -27,8 +25,7 @@ def chat_degrade(prompt, model):
raise SensitiveError("chat_governed 全局降级链全部耗尽(内容安全或模型不可用)")
return content, usage
DSN = ("postgresql://root:f6710e2d0294eb1c10e26a805a64bc54@100.64.0.8:5433/muse-example"
"?keepalives=1&keepalives_idle=15&keepalives_interval=5&keepalives_count=3")
from muse_db import connect
TENANT, ACTOR = 1, "1"
MAX_WINDOW_GENERATION_ATTEMPTS = 2
@ -165,7 +162,7 @@ def cli():
@click.option("--to", "to_", type=int, help="结束章(限定已拆域,防止切窗吞进没有细纲的章;默认全书)")
def window(work_id, window, model, from_, to_):
"""按字数切窗聚合大纲(窗内章须已有细纲;已有窗大纲的窗跳过=断点续跑)。"""
with psycopg.connect(DSN) as conn:
with connect() as conn:
title = conn.execute("SELECT title FROM muse_content_work WHERE id=%s", (work_id,)).fetchone()[0]
all_rows = load_chapters(conn, work_id)
last_order = all_rows[-1][0] if all_rows else 0 # 全书末章(书末残窗判断用)
@ -242,7 +239,7 @@ def _do_window(conn, work_id, title, win_no, chs, model, done):
@click.option("--model", default="MiniMax-M3", show_default=True)
def check(work_id, model):
"""终检:逐窗细纲对账 + 全书大纲连贯性纵览(结果写 check_status/check_note)。"""
with psycopg.connect(DSN) as conn:
with connect() as conn:
title = conn.execute("SELECT title FROM muse_content_work WHERE id=%s", (work_id,)).fetchone()[0]
wins = conn.execute(
"""SELECT id, window_no, from_order, to_order, outline_text, check_status

View File

@ -12,12 +12,11 @@ import json
import sys
import click
import psycopg
sys.path.insert(0, str(__import__("pathlib").Path(__file__).resolve().parent))
sys.path.insert(0, str(__import__("pathlib").Path(__file__).resolve().parents[2] / "call-content-model" / "scripts"))
from muse_db import connect # noqa: E402
from parse_llm import (m3_json, scaffold_prompt, ingest, SensitiveHardStop, # noqa: E402
DSN, TENANT)
TENANT)
MODEL = "MiniMax-M3"
@ -74,7 +73,7 @@ def salvage_chapter(conn, work_id, title, ch, ch_title, text, prev):
@click.option("--work-id", type=int, default=0, help="只跑指定书(0=全部)")
@click.option("--limit", type=int, default=0, help="最多抢救几章(0=不限)")
def main(work_id, limit):
with psycopg.connect(DSN) as conn:
with connect() as conn:
cond = "AND t.work_id=%s" % work_id if work_id else ""
rows = conn.execute(f"""
SELECT t.work_id, w.title, c.order_no, c.title, b.content_text
@ -88,7 +87,7 @@ def main(work_id, limit):
saved = failed = 0
for wid, title, ch, ch_title, text in rows[:limit or None]:
# 前文实体名清单(复用章级压缩口径)
with psycopg.connect(DSN) as conn:
with connect() as conn:
prev = [e for (es,) in conn.execute(
"""SELECT s.entities FROM example_parse_scaffold s
JOIN muse_content_chapter c ON c.id=s.chapter_id

View File

@ -13,18 +13,9 @@ merge_model_findings 注入,同样过合同校验;诊断不修改任何正
import argparse
import hashlib
import json
import sys
from pathlib import Path
# 共享执行骨架(humanization 副本)与数据库通道(access-database)的引导
SCRIPT_DIR = Path(__file__).resolve().parent
AGENT_ROOT = SCRIPT_DIR.parents[3]
for _p in (AGENT_ROOT / "humanization" / "src",
AGENT_ROOT / ".claude" / "skills" / "access-database" / "scripts"):
if str(_p) not in sys.path:
sys.path.insert(0, str(_p))
from deai import diagnose, load # noqa: E402
from deai import diagnose, load
TENANT_ID = 1
CREATOR = "1"
@ -46,7 +37,7 @@ def load_active_library(from_db: bool = False) -> tuple[dict, str]:
不静默回退 Git 文件资产;只有显式 --offline 才读文件。
"""
if from_db:
from db import connect
from muse_db import connect
from deai import load_db
with connect(readonly=True) as conn:
@ -84,7 +75,7 @@ def _run_id(*parts: str) -> str:
def persist_diagnosis(artifact: dict, *, text: str, from_db: bool = False,
creator: str = CREATOR, tenant_id: int = TENANT_ID) -> dict:
"""诊断运行落库:example_run(幂等 upsert)+ example_quality_result(append-only)。"""
from db import connect
from muse_db import connect
if not isinstance(text, str) or not text:
raise DiagnoseContractError("落库诊断文本为空")

View File

@ -7,6 +7,8 @@ description: 使用固定 Qwen3 嵌入模型将知识草稿或实体批量写入
对应 muse API 面:AI 网关(嵌入)。通道事实见 [`db/连接信息.md`](../../../db/连接信息.md):BASE `http://100.64.0.8:3000`、模型 `Qwen/Qwen3-Embedding-8B`、请求体 `"dimensions":1024`(实测生效)、**禁系统代理**(`trust_env=False`)。
库实现装为共享包 `muse-embed`(`-e ./muse-embed`):拆书与检索侧 `from muse_embed import embed_texts, build_embed_text`,`scripts/embed_drafts.py` 只是本 Skill 的 CLI。
## 用法
```bash

View File

@ -1,427 +1,12 @@
#!/usr/bin/env python3
"""embed-knowledge Skill:知识行批量嵌入(New-API / Qwen3-Embedding-8B / 1024 维)。
合同见同 skill SKILL.md;通道事实见 db/连接信息.md。失败原样报错不静默。
"""
import hashlib
import json
"""embed-knowledge Skill CLI。库实现安装为 muse_embed,这里只做入参与出错的壳。"""
import sys
import time
import click
import psycopg
import requests
DSN = ("postgresql://root:f6710e2d0294eb1c10e26a805a64bc54@100.64.0.8:5433/muse-example"
"?keepalives=1&keepalives_idle=15&keepalives_interval=5&keepalives_count=3")
BASE = "http://100.64.0.8:3000"
TOKEN = "sk-DyVqO3lDmEvQZ3PqGpbNaaaHZHhbh0xaHRIiynhYSmVlLHl2" # MUSE_AI_NEW_API_TOKEN(勿用管理令牌)
MODEL = "Qwen/Qwen3-Embedding-8B"
DIM = 1024
TENANT, ACTOR = 1, "1"
BATCH = 16
def _session():
"""禁系统代理的会话(系统代理会假 502)。"""
s = requests.Session()
s.trust_env = False
s.headers["Authorization"] = f"Bearer {TOKEN}"
return s
def embed_texts(sess, texts):
"""调 New-API /v1/embeddings;整批重试 2 次后逐条降级。返回 (向量列表, 失败索引集)。"""
def call(batch):
r = sess.post(f"{BASE}/v1/embeddings", json={
"model": MODEL, "input": batch, "dimensions": DIM}, timeout=120)
r.raise_for_status()
data = r.json()["data"]
# 响应 index 是请求槽位,不能排序后压缩;缺项、重复或越界都必须让本次调用失败并进入重试。
vectors = [None] * len(batch)
seen = set()
for item in data:
index = item["index"]
if type(index) is not int or not 0 <= index < len(batch):
raise ValueError(f"embedding 响应 index 越界或非整数:{index!r}")
if index in seen:
raise ValueError(f"embedding 响应 index 重复:{index}")
vectors[index] = item["embedding"]
seen.add(index)
if len(seen) != len(batch):
missing = sorted(set(range(len(batch))) - seen)
raise ValueError(f"embedding 响应缺少 index:{missing}")
return vectors
for attempt in range(3):
try:
return call(texts), set()
except Exception as e:
if attempt < 2:
time.sleep(2 ** attempt)
continue
# 整批三败 → 逐条降级,坏行记错不断批
vecs, bad = [], set()
for i, t in enumerate(texts):
try:
vecs.append(call([t])[0])
except Exception as ee:
vecs.append(None)
bad.add(i)
click.echo(f" [失败] 第{i}条: {ee}", err=True)
return vecs, bad
def build_embed_text(payload: dict) -> str:
"""嵌入文本构造:payload 自带 embed_text 优先;否则固定拼接(与检索端语义对齐)。"""
if payload.get("embed_text"):
return payload["embed_text"]
# 型取值补 type 键:升格卡 payload 用 type 存型(非 型/target_type),漏认会产出「【】名称…」丢型文本,
# 令升格卡向量与检索端跨型语义错位;补一段式回退(additive,不动 型/target_type 既有行为)。
t = payload.get("型") or payload.get("type") or payload.get("target_type", "")
name = payload.get("名称") or payload.get("name", "")
brief = payload.get("一句话摘要") or payload.get("brief", "")
fields = payload.get("字段") or payload.get("fields") or {}
body = "\n".join(f"{k}:{v}" for k, v in fields.items() if v and k not in ("名称", "一句话摘要"))
return f"【{t}】{name}:{brief}\n{body}"[:4000]
def _content_hash(text):
"""统一生成向量幂等键,候选筛选与写前复验必须共用同一规则。"""
return hashlib.sha256(f"{text}|{MODEL}".encode()).hexdigest()
class EmbeddingOwnershipConflict(RuntimeError):
"""同 hash 唯一行已归实体或其他活跃 draft,禁止迁移 owner。"""
def _embedding_owner_action(conn, draft_id, content_hash, *, lock=False):
"""判断同 hash 唯一行应幂等跳过还是写入;写段可锁行封住预查后的竞态。"""
lock_clause = " FOR UPDATE OF e" if lock else ""
owner = conn.execute(
"""SELECT e.draft_id, e.entity_id, e.deleted,
COALESCE(d.deleted, TRUE), d.tenant_id
FROM example_knowledge_embedding e
LEFT JOIN muse_knowledge_draft d ON d.id=e.draft_id
WHERE e.tenant_id=%s AND e.content_hash=%s AND e.model=%s""" + lock_clause,
(TENANT, content_hash, MODEL),
).fetchone()
if not owner:
return "write"
owner_draft_id, owner_entity_id, embedding_deleted, owner_deleted, owner_tenant = owner
# entity owner 是确认后的正式归属,任何 draft 都不得把它降级抢回。
if owner_entity_id is not None:
raise EmbeddingOwnershipConflict(
f"同 hash 唯一行已归 entity:hash={content_hash},entity={owner_entity_id},"
f"candidate={draft_id}"
)
# 只有当前租户、当前 draft、两侧都 active 才是真正的幂等命中。
if owner_draft_id == draft_id:
if owner_tenant != TENANT:
raise EmbeddingOwnershipConflict(
f"同 hash 当前 owner 租户不匹配:hash={content_hash},"
f"owner_tenant={owner_tenant},candidate_tenant={TENANT}"
)
if not embedding_deleted and not owner_deleted:
return "skip"
return "write"
# 空 owner、owner 行缺失或 owner draft 已软删时,可由当前活跃 draft 接管唯一行。
if owner_draft_id is None or owner_deleted:
return "write"
raise EmbeddingOwnershipConflict(
f"同 hash 唯一行已归其他 active draft:hash={content_hash},"
f"owner={owner_draft_id},candidate={draft_id}"
)
def _write_embedding(conn, draft_id, content_hash, text, vector):
"""在调用方单 draft 事务内锁定活性与 owner,条件写入并校验最终归属。"""
# 写事务先按固定表顺序取得 ROW EXCLUSIVE 锁,避免与 reset 的多表锁形成交叉等待。
conn.execute(
"LOCK TABLE muse_knowledge_draft, example_knowledge_embedding IN ROW EXCLUSIVE MODE"
)
# 取得表锁后再锁 candidate draft:embed 先到时 reset 的七表 SHARE ROW EXCLUSIVE 会等待;
# reset 先到时本查询等待其提交,随后读取 deleted=TRUE 并拒绝陈旧写入。
candidate = conn.execute(
"""SELECT tenant_id, deleted, status, draft_payload FROM muse_knowledge_draft
WHERE id=%s FOR UPDATE""",
(draft_id,),
).fetchone()
if not candidate:
click.echo(f" [跳过] draft={draft_id} 写前已不存在,未写向量", err=True)
return False
candidate_tenant, candidate_deleted, candidate_status, current_payload = candidate
if candidate_tenant != TENANT:
raise EmbeddingOwnershipConflict(
f"draft 租户不匹配:draft={draft_id},tenant={candidate_tenant},expected={TENANT}"
)
if candidate_deleted:
click.echo(f" [跳过] draft={draft_id} 写前已软删,未写向量", err=True)
return False
if candidate_status != "pending":
click.echo(
f" [跳过] draft={draft_id} 写前 status={candidate_status},非 pending,未写向量",
err=True,
)
return False
# HTTP 期间 payload 可能被 parse/confirm 更新;锁内必须按当前 payload 重构文本与 hash,
# 只要与 HTTP 请求所依据的快照不同,就丢弃陈旧向量,绝不覆盖并发产生的新结果。
current_text = build_embed_text(current_payload or {})
current_hash = _content_hash(current_text)
if current_text != text or current_hash != content_hash:
click.echo(
f" [跳过] draft={draft_id} 写前 payload/hash 漂移,"
f"expected_hash={content_hash} current_hash={current_hash},未写向量",
err=True,
)
return False
# 锁定该 draft 的全部活向量,保证 entity 归属和“每 draft 唯一活向量”在同一事务内判定。
live_embeddings = conn.execute(
"""SELECT id, content_hash, model, entity_id FROM example_knowledge_embedding
WHERE tenant_id=%s AND draft_id=%s AND deleted=FALSE
FOR UPDATE""",
(TENANT, draft_id),
).fetchall()
if len(live_embeddings) > 1:
raise EmbeddingOwnershipConflict(
f"draft={draft_id} 存在多条活向量,状态异常,禁止自动修复:{live_embeddings}"
)
entity_rows = [
(row_id, row_hash, row_model, entity_id)
for row_id, row_hash, row_model, entity_id in live_embeddings
if entity_id is not None
]
if entity_rows:
raise EmbeddingOwnershipConflict(
f"draft={draft_id} 存在 entity_id 非空旧活向量,禁止覆盖:{entity_rows}"
)
if any(
row_hash == content_hash and row_model == MODEL
for _, row_hash, row_model, _ in live_embeddings):
click.echo(f" [跳过] draft={draft_id} 同 hash 活向量已由当前 draft 持有")
return False
action = _embedding_owner_action(conn, draft_id, content_hash, lock=True)
if action == "skip":
click.echo(f" [跳过] draft={draft_id} 同 hash 活向量已由当前 draft 持有")
return False
if live_embeddings:
# 当前 payload 已通过锁内 hash 重验,因此其余 hash 均为该 draft 的过期向量;
# 只允许软删 draft owner,entity owner 已在上方失败关闭。
conn.execute(
"""UPDATE example_knowledge_embedding SET deleted=TRUE, updater=%s
WHERE tenant_id=%s AND draft_id=%s AND deleted=FALSE
AND entity_id IS NULL AND (content_hash!=%s OR model!=%s)""",
(ACTOR, TENANT, draft_id, content_hash, MODEL),
)
# 条件 UPSERT 是行锁检查后的第二道防线:当预查时唯一行尚不存在、随后被并发插入时,
# 仅允许当前 owner 或已失活 owner 迁移;entity/其他 active draft 均令 RETURNING 为空。
upserted = conn.execute(
"""INSERT INTO example_knowledge_embedding
(draft_id, content_hash, embed_text, model, dimensions, embedding,
creator, updater, tenant_id)
VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s)
ON CONFLICT (tenant_id, content_hash, model)
DO UPDATE SET draft_id=EXCLUDED.draft_id,
embed_text=EXCLUDED.embed_text,
model=EXCLUDED.model,
dimensions=EXCLUDED.dimensions,
embedding=EXCLUDED.embedding,
deleted=FALSE,
updater=EXCLUDED.updater
WHERE example_knowledge_embedding.entity_id IS NULL
AND (example_knowledge_embedding.draft_id=EXCLUDED.draft_id
OR NOT EXISTS (
SELECT 1 FROM muse_knowledge_draft owner
WHERE owner.id=example_knowledge_embedding.draft_id
AND owner.deleted=FALSE))
RETURNING draft_id""",
(draft_id, content_hash, text, MODEL, DIM, json.dumps(vector),
ACTOR, ACTOR, TENANT),
).fetchone()
if not upserted or upserted[0] != draft_id:
raise EmbeddingOwnershipConflict(
f"同 hash 唯一行未绑定当前 draft:hash={content_hash},candidate={draft_id}"
)
return True
def _load_bulk_candidates(conn, work_id, limit, source_type=None):
"""读取 pending draft 的全部活向量,在 Python 中按当前文本和模型筛选补嵌候选。
拆书草稿的 ``work_id`` 仍表示参考书,历史调用因此按 ``source_id`` 筛选。
章后抽卡直接把作品写入 draft.work_id,必须用显式 source_type 切换到该口径,
避免同一个 CLI 参数在两类数据上产生歧义。
"""
sql = """SELECT d.id, d.draft_payload,
e.id, e.content_hash, e.model, e.entity_id
FROM muse_knowledge_draft d
LEFT JOIN example_knowledge_embedding e
ON e.tenant_id=%s AND e.draft_id=d.id AND e.deleted=FALSE
WHERE d.tenant_id=%s AND d.deleted=FALSE AND d.status='pending'"""
args = [TENANT, TENANT]
if source_type == "chapter_extract":
if work_id is None:
raise ValueError("source_type=chapter_extract 必须同时指定 --work-id")
sql += " AND d.work_id=%s AND d.source_type=%s"
args.extend([work_id, source_type])
elif work_id is not None:
sql += " AND d.source_id=%s"
args.append(work_id)
# 必须先取得每个 draft 的全部活向量,不能在 SQL 层 LIMIT 后漏掉旧 hash 或异常状态。
sql += " ORDER BY d.id, e.id"
rows = conn.execute(sql, args).fetchall()
grouped = {}
for draft_id, payload, embedding_id, row_hash, row_model, entity_id in rows:
draft = grouped.setdefault(draft_id, {"payload": payload, "embeddings": []})
if embedding_id is not None:
draft["embeddings"].append((embedding_id, row_hash, row_model, entity_id))
candidates = []
failures_by_draft = {}
repair_targets = {}
for draft_id in sorted(grouped):
draft = grouped[draft_id]
text = build_embed_text(draft["payload"] or {})
content_hash = _content_hash(text)
live_embeddings = draft["embeddings"]
if len(live_embeddings) == 1:
_, row_hash, row_model, entity_id = live_embeddings[0]
if entity_id is None and row_hash == content_hash and row_model == MODEL:
continue
# 所有非健康目标都参与同批 hash 冲突检查,不能因其中一条先被判异常而放行另一条。
repair_targets.setdefault(content_hash, []).append(draft_id)
if len(live_embeddings) > 1:
failures_by_draft.setdefault(draft_id, []).append(
f"存在多条活向量,状态异常,禁止自动修复:{live_embeddings}"
)
continue
if live_embeddings:
_, row_hash, row_model, entity_id = live_embeddings[0]
if entity_id is not None:
failures_by_draft.setdefault(draft_id, []).append(
f"活向量已归 entity={entity_id},禁止 draft 补嵌迁移 owner"
)
continue
candidates.append((draft_id, content_hash, text))
# 相同目标 hash 的多个 draft 不能靠执行顺序决定 owner;冲突检查必须发生在 limit 之前。
conflicted_drafts = set()
for content_hash, draft_ids in repair_targets.items():
if len(draft_ids) < 2:
continue
reason = (
f"同批目标 hash 冲突:hash={content_hash},drafts={draft_ids},"
"禁止按执行顺序抢 owner"
)
for draft_id in draft_ids:
failures_by_draft.setdefault(draft_id, []).append(reason)
conflicted_drafts.add(draft_id)
candidates = [candidate for candidate in candidates if candidate[0] not in conflicted_drafts]
# limit 只能限制后续 HTTP/写入;先对完整候选集预查目标 hash owner,避免范围外冲突被隐藏。
prechecked_candidates = []
for draft_id, content_hash, text in candidates:
try:
action = _embedding_owner_action(conn, draft_id, content_hash)
except EmbeddingOwnershipConflict as exc:
failures_by_draft.setdefault(draft_id, []).append(str(exc))
continue
if action != "skip":
prechecked_candidates.append((draft_id, content_hash, text))
# 全量只读预检完成后释放事务,再截取实际处理行;每个 chunk 仍会再次预查以封住其后竞态。
conn.commit()
if limit and limit > 0:
prechecked_candidates = prechecked_candidates[:int(limit)]
failures = [
(draft_id, ";".join(reasons))
for draft_id, reasons in sorted(failures_by_draft.items())
]
return prechecked_candidates, failures
def _run_bulk(conn, sess, work_id, limit, source_type=None):
"""执行一次 bulk 补嵌;HTTP 前后均保持既有 owner、锁和 stale-write 边界。"""
rows, read_failures = _load_bulk_candidates(conn, work_id, limit, source_type)
for draft_id, reason in read_failures:
click.echo(f" [失败] draft={draft_id}: {reason}", err=True)
if read_failures:
details = ";".join(
f"draft={draft_id}: {reason}" for draft_id, reason in read_failures
)
raise EmbeddingOwnershipConflict(f"bulk 候选存在确定性冲突,已失败关闭:{details}")
done = skip = fail = 0
click.echo(f"待补嵌草稿: {len(rows)} 条(筛选失败 {len(read_failures)} 条)")
for i in range(0, len(rows), BATCH):
chunk = rows[i:i + BATCH]
metas = []
for draft_id, content_hash, text in chunk:
action = _embedding_owner_action(conn, draft_id, content_hash)
if action == "skip":
skip += 1
click.echo(f" [跳过] draft={draft_id} 同 hash 活向量已由当前 draft 持有")
continue
metas.append((draft_id, content_hash, text))
# owner 预查只用于避免无效 HTTP;HTTP 期间不持数据库事务或表锁。
conn.commit()
if not metas:
continue
texts = [meta[2] for meta in metas]
try:
vecs, bad = embed_texts(sess, texts)
except Exception as exc:
for draft_id, _, _ in metas:
fail += 1
click.echo(f" [失败] draft={draft_id}: HTTP 嵌入失败:{exc}", err=True)
continue
bad = set(bad or ())
for j, (draft_id, content_hash, text) in enumerate(metas):
if j in bad:
fail += 1
click.echo(f" [失败] draft={draft_id}: HTTP 返回 bad,保留旧向量", err=True)
continue
try:
vector = vecs[j]
except (IndexError, TypeError):
fail += 1
click.echo(f" [失败] draft={draft_id}: HTTP 返回向量缺项,保留旧向量", err=True)
continue
if vector is None:
fail += 1
click.echo(f" [失败] draft={draft_id}: HTTP 返回空向量,保留旧向量", err=True)
continue
# 每个 draft 独立事务:确定性冲突回滚当前事务并向上抛,使命令以非零状态退出。
with conn.transaction():
written = _write_embedding(conn, draft_id, content_hash, text, vector)
if written:
done += 1
else:
skip += 1
click.echo(
f" 进度 {min(i + BATCH, len(rows))}/{len(rows)}"
f"(新嵌{done} 跳过{skip} 失败{fail})"
)
click.echo(f"完成:新嵌 {done}、跳过 {skip}、失败 {fail}")
return {"done": done, "skip": skip, "fail": fail}
from muse_db import connect
from muse_embed import EmbeddingOwnershipConflict, _run_bulk, _session, embed_texts
@click.command()
@ -439,7 +24,7 @@ def main(work_id, source_type, limit, probe):
click.echo(f"维度={len(v)} 前5维={[round(x, 4) for x in v[:5]]}")
return
with psycopg.connect(DSN) as conn:
with connect() as conn:
_run_bulk(conn, sess, work_id, limit, source_type)

View File

@ -13,30 +13,15 @@ import argparse
import copy
import hashlib
import json
import sys
from pathlib import Path
SCRIPT_DIR = Path(__file__).resolve().parent
AGENT_ROOT = SCRIPT_DIR.parents[3]
for _p in (
AGENT_ROOT / "humanization" / "src",
AGENT_ROOT / ".claude" / "skills" / "access-database" / "scripts",
):
if str(_p) not in sys.path:
sys.path.insert(0, str(_p))
from deai.baseline import draft_ledger # noqa: E402
from deai.schemas import validate # noqa: E402
from deai.baseline import BaselineContractError, draft_ledger, validate_ledger
TENANT_ID = 1
CREATOR = "1"
SCHEMA_VERSION = "voice-baseline-v1"
class BaselineContractError(ValueError):
"""声音账合同失败:结构、来源或确认门未通过。"""
def _sha256_text(text: str) -> str:
return hashlib.sha256(text.encode("utf-8")).hexdigest()
@ -45,64 +30,6 @@ def _canonical_json(value: dict) -> str:
return json.dumps(value, ensure_ascii=False, sort_keys=True, separators=(",", ":"))
def validate_ledger(ledger: dict, *, work_ref: str) -> None:
"""校验声音账结构、作品绑定与角色归属冲突。"""
if not isinstance(ledger, dict):
raise BaselineContractError("声音账必须是 JSON 对象")
try:
validate(ledger, "voice_baseline")
except ValueError as exc:
raise BaselineContractError(str(exc)) from exc
if ledger.get("schema_version") != SCHEMA_VERSION:
raise BaselineContractError(f"schema_version 必须是 {SCHEMA_VERSION}")
if ledger.get("work_ref") != work_ref:
raise BaselineContractError("声音账 work_ref 与目标作品不一致")
if ledger.get("status", "candidate") not in {"candidate", "canonical"}:
raise BaselineContractError("声音账 status 只能是 candidate/canonical")
narrator = ledger.get("narrator")
if not isinstance(narrator, dict):
raise BaselineContractError("narrator 必须是对象")
for key in ("sentence_habits", "punctuation_habits"):
if not isinstance(narrator.get(key, []), list):
raise BaselineContractError(f"narrator.{key} 必须是数组")
if "metrics" in narrator and not isinstance(narrator["metrics"], dict):
raise BaselineContractError("narrator.metrics 必须是对象")
if "exemplar_passages" in narrator and not isinstance(narrator["exemplar_passages"], list):
raise BaselineContractError("narrator.exemplar_passages 必须是数组")
for key, typ, what in (
("characters", dict, "角色声音档案"),
("untouchable_verbal_tics", dict, "不可改口癖"),
("protected_spans", list, "保护片段"),
("blacklist", list, "作品级黑名单"),
):
value = ledger.get(key)
if not isinstance(value, typ):
raise BaselineContractError(f"{what}({key})缺失或类型错误")
for key in ("passing_samples", "unknown_fields", "sources"):
if key in ledger and not isinstance(ledger[key], list):
raise BaselineContractError(f"{key} 必须是数组")
ownership: dict[str, str] = {}
for who, info in ledger["characters"].items():
if not isinstance(info, dict):
raise BaselineContractError(f"characters.{who} 必须是对象")
for key in ("verbal_tics", "sample_lines"):
values = info.get(key, [])
if not isinstance(values, list) or not all(isinstance(item, str) and item for item in values):
raise BaselineContractError(f"characters.{who}.{key} 必须是非空字符串数组")
for item in values:
previous = ownership.setdefault(item, who)
if previous != who:
raise BaselineContractError(f"声音样本「{item}」同时归属 {previous}/{who},须作者裁决")
for who, tics in ledger["untouchable_verbal_tics"].items():
if not isinstance(tics, list) or not all(isinstance(item, str) and item for item in tics):
raise BaselineContractError(f"{who} 的口癖必须是非空字符串数组")
for tic in tics:
previous = ownership.setdefault(tic, who)
if previous != who:
raise BaselineContractError(f"口癖「{tic}」同时归属 {previous}/{who},须作者裁决")
def ground_ledger(ledger: dict, source_text: str) -> list[str]:
"""所有声明为原文样例的内容必须能逐字回到已确认正文。"""
failures = []
@ -129,7 +56,7 @@ def ground_ledger(ledger: dict, source_text: str) -> list[str]:
def load_canonical_sources(work_id: int, *, max_chapters: int | None = None,
tenant_id: int = TENANT_ID) -> tuple[str, list[dict]]:
"""从正式库读取 Canonical 正文;文件不是生产权威。"""
from db import connect
from muse_db import connect
if work_id <= 0:
raise BaselineContractError("work_id 必须为正整数")
@ -178,42 +105,18 @@ def load_canonical_sources(work_id: int, *, max_chapters: int | None = None,
def load_current_baseline(work_ref: str, *, tenant_id: int = TENANT_ID) -> dict | None:
"""读取当前声音账并复核 ledger hash;多条 current 视为数据库状态损坏。"""
from db import connect
"""读取当前声音账;连接经 muse_db,校验在 deai.load_db。"""
from muse_db import connect
from deai.load_db import load_current_baseline as _load
with connect(readonly=True) as conn:
rows = conn.execute(
"SELECT ledger,ledger_sha256,version FROM example_voice_baseline "
"WHERE tenant_id=%s AND work_ref=%s AND deleted=FALSE AND superseded=FALSE "
"ORDER BY version DESC",
(tenant_id, work_ref),
).fetchall()
if not rows:
return None
if len(rows) != 1:
raise BaselineContractError(f"作品 {work_ref} 存在 {len(rows)} 条 current 声音账")
ledger = dict(rows[0][0])
expected_hash = rows[0][1]
# 兼容 v1 旧脚本使用 json.dumps(sort_keys=True, 带空格) 计算的历史 hash;
# 新版本写入规范 JSON hash,读取时两种格式都必须与数据库一致。
candidate_hashes = {
_sha256_text(_canonical_json(ledger)),
_sha256_text(json.dumps(ledger, ensure_ascii=False, sort_keys=True)),
}
if expected_hash not in candidate_hashes:
raise BaselineContractError(f"作品 {work_ref} 当前声音账 hash 不一致")
ledger["database_version"] = rows[0][2]
ledger["database_ledger_sha256"] = expected_hash
# 历史已确认行可能未写 status;current 表本身由 reviewer + superseded 门确认,读取时补 canonical 投影。
ledger.setdefault("status", "canonical")
validate_ledger(ledger, work_ref=work_ref)
return ledger
return _load(conn, work_ref, tenant_id=tenant_id)
def persist_baseline(ledger: dict, *, source_text: str, reviewer: str, note: str = "",
creator: str = CREATOR, tenant_id: int = TENANT_ID) -> dict:
"""人工确认后追加新版本并取代旧版本;历史行保留。"""
from db import connect
from muse_db import connect
if not reviewer:
raise BaselineContractError("基线必须人工确认(reviewer 不得为空)")
@ -262,7 +165,7 @@ def persist_baseline(ledger: dict, *, source_text: str, reviewer: str, note: str
def persist_draft_run(ledger: dict, *, work_id: int | None = None,
creator: str = CREATOR, tenant_id: int = TENANT_ID) -> dict:
"""候选账也留运行事实,但不写入 current 基线表。"""
from db import connect
from muse_db import connect
validate_ledger(ledger, work_ref=ledger.get("work_ref", ""))
ledger_sha = _sha256_text(_canonical_json(ledger))

View File

@ -35,12 +35,9 @@ from pathlib import Path
from typing import Any, Callable, Mapping, Protocol
SCRIPT_DIR = Path(__file__).resolve().parent
EXECUTION_DIR = SCRIPT_DIR.parents[1] / "execute-claude-task" / "scripts"
QUALITY_GATE_DIR = SCRIPT_DIR.parents[1] / "score-content-quality" / "scripts"
if str(SCRIPT_DIR) not in sys.path:
sys.path.insert(0, str(SCRIPT_DIR))
if str(EXECUTION_DIR) not in sys.path:
sys.path.insert(0, str(EXECUTION_DIR))
if str(QUALITY_GATE_DIR) not in sys.path:
sys.path.insert(0, str(QUALITY_GATE_DIR))

View File

@ -106,7 +106,6 @@ from ._common import (
RAW_LEASE_CLEANUP_MARGIN_SECONDS,
READ_CONTEXT_DIR,
REQUIRED_ARMS,
EXECUTION_DIR,
EVIDENCE_DIR,
ReplayInterrupted,
ReplayJudge,

View File

@ -22,7 +22,6 @@ from typing import Any, Callable, Mapping, Protocol
SCRIPT_DIR = Path(__file__).resolve().parents[1] # 包目录的上一级仍是 scripts/
SKILLS_DIR = SCRIPT_DIR.parents[1]
EXECUTION_DIR = SKILLS_DIR / "execute-claude-task" / "scripts"
EVIDENCE_DIR = SKILLS_DIR / "record-run-evidence" / "scripts"
CONTINUATION_DIR = SKILLS_DIR / "write-next-chapter" / "scripts"
READ_CONTEXT_DIR = SKILLS_DIR / "assemble-context" / "scripts"
@ -30,7 +29,6 @@ DETECT_DIR = SKILLS_DIR / "check-content-consistency" / "scripts"
QUALITY_GATE_DIR = SKILLS_DIR / "score-content-quality" / "scripts"
SNAPSHOT_DIR = SKILLS_DIR / "freeze-context" / "scripts"
for import_path in (
EXECUTION_DIR,
EVIDENCE_DIR,
CONTINUATION_DIR,
READ_CONTEXT_DIR,

View File

@ -10,7 +10,7 @@ disable-model-invocation: true
## 入口
`scripts/claude_runtime.py` 只接受显式冻结的 `ExecutionProfile`,负责 fresh process、沙箱、最小环境、硬 deadline、结构化输出和联合执行回执。任何模型、预算、schema、prompt、退出状态或输出绑定不完整都失败关闭;错误对象不得携带 stderr 原文。
共享包 `claude_runtime`(`-e ./muse-claude-runtime`)只接受显式冻结的 `ExecutionProfile`,负责 fresh process、沙箱、最小环境、硬 deadline、结构化输出和联合执行回执。任何模型、预算、schema、prompt、退出状态或输出绑定不完整都失败关闭;错误对象不得携带 stderr 原文。调用方 `from claude_runtime import run_claude`,不得 `sys.path` 指向本 Skill 的 `scripts/`。
真实调用必须持有 `Popen` 进程句柄并使用独立进程组。deadline、SIGTERM、SIGINT 或父进程异常发生时,先终止并等待整个模型进程组,再向调用方返回失败回执,禁止遗留继续运行或计费的子进程。

View File

@ -0,0 +1 @@
本 Skill 的 Claude 调用实现已安装为 `claude_runtime` 包(`-e ./muse-claude-runtime`)。调用方 `from claude_runtime import run_claude`,本目录不再放置可 import 的运行时模块。

View File

@ -18,23 +18,17 @@ from psycopg.types.json import Jsonb
HERE = pathlib.Path(__file__).resolve().parent
SKILLS = HERE.parents[1]
for import_path in (
SKILLS / "call-content-model" / "scripts",
SKILLS / "record-run-evidence" / "scripts",
):
if str(import_path) not in sys.path:
sys.path.insert(0, str(import_path))
from llm import chat_governed, cost_usd, extract_json # noqa: E402
from muse_llm import chat_governed, cost_usd, extract_json
from muse_db import connect # noqa: E402
from record_failed_run import record_failure # noqa: E402
from run_registry import finish_run, new_run_id, start_run # noqa: E402
DB_SCRIPTS = SKILLS / "access-database" / "scripts"
if str(DB_SCRIPTS) not in sys.path:
sys.path.insert(0, str(DB_SCRIPTS))
from db import connect # noqa: E402
TENANT, ACTOR = 1, "1"
MODEL = "MiniMax-M3"
ENTITY_TYPES = frozenset({

View File

@ -34,21 +34,16 @@ from copy import deepcopy
from numbers import Real
import click
import psycopg
# 复用章级管线的敏感降级链与 llm 入口(trust_env/重试/JSON 容错同源)。
# 拆分后 upgrade 与 parse-book 分属两个 skill:parse_llm(敏感降级链+llm 入口+DSN/TENANT)留
# parse-book 字节不动,本脚本单向跨 skill 引用(upgrade→parse-book,合法);upgrade_work_lock 与本脚本同目录。
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent))
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[2] / "deconstruct-book" / "scripts"))
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[2] / "call-content-model" / "scripts"))
from parse_llm import m3_json, SensitiveHardStop, IDENTITY, TENANT, DSN # noqa: E402
from muse_db import DSN, connect # noqa: E402
from parse_llm import m3_json, SensitiveHardStop, IDENTITY, TENANT # noqa: E402
from upgrade_work_lock import UpgradeWorkLockUnavailable, upgrade_work_lock # noqa: E402
# 语义判重(P1)复用 embed-knowledge 的嵌入通道(同模型同维、与检索端语义对齐)——
# 只在开启 --semantic-dedup 时才真调,默认关(试跑期嵌入延后,见文件头注释);
# build_embed_text/MODEL/DIM/ACTOR 供最终嵌入短事务复用检索端同源文本构造器与列常量。
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[2] / "embed-knowledge" / "scripts"))
from embed_drafts import (_session as _embed_session, embed_texts, build_embed_text, # noqa: E402
from muse_embed import (_session as _embed_session, embed_texts, build_embed_text,
MODEL as EMBED_MODEL, DIM as EMBED_DIM, ACTOR as EMBED_ACTOR)
# ── 窗切割参数(方案 §五B:3–5 万字/窗、10–15 章,取保守双闸防观察输出过载)──
@ -3368,7 +3363,7 @@ def prejudge_semantic(obs, name_map, presence, embed_sess, work_id, call):
vecs, bad = embed_texts(embed_sess, [_entity_embed_text(e) for e in cands])
# c. 短连接:逐候选用现成向量召回近邻(recall SQL 半),查完即关(三段式)
cand_nbrs = {}
with psycopg.connect(DSN) as conn:
with connect() as conn:
for i, ent in enumerate(cands):
if i in bad or i >= len(vecs) or vecs[i] is None:
continue # 嵌入失败的候选优雅跳过(判重回退纯机械立卡)
@ -3811,7 +3806,7 @@ def prepare_touched_cards(sess, touched):
"""短读后关闭连接,再发嵌入 HTTP;返回全部 touched 快照供 final 独立复验。"""
snapshots = {}
with psycopg.connect(DSN) as conn:
with connect() as conn:
for did in sorted(touched):
row = conn.execute(
"""SELECT draft_payload, revision FROM muse_knowledge_draft
@ -3908,7 +3903,7 @@ def embed_touched_cards(sess, work_id, touched):
"""兼容独立调用:严格 prepare/apply,普通异常也向上抛出,不再告警放行。"""
prepared = prepare_touched_cards(sess, touched)
with psycopg.connect(DSN) as conn:
with connect() as conn:
conn.execute(
"LOCK TABLE muse_knowledge_draft, example_knowledge_embedding IN ROW EXCLUSIVE MODE"
)
@ -3938,7 +3933,7 @@ def windows(work_id):
"""机械切正文窗(幂等)。"""
try:
with upgrade_work_lock(DSN, TENANT, work_id):
with psycopg.connect(DSN) as conn:
with connect() as conn:
total, new = cut_windows(conn, work_id)
click.echo(f"work={work_id} 切窗完成:全书 {total} 窗(本次新建 {new} 行)")
except UpgradeWorkLockUnavailable as exc:
@ -4048,7 +4043,7 @@ def _compute_entity_updates(plan, expected_revisions, contracts, title, a, b, te
for offset in range(0, len(items), UPDATE_BATCH):
batch = items[offset:offset + UPDATE_BATCH]
cards = []
with psycopg.connect(DSN) as conn:
with connect() as conn:
for ref, points in batch:
if ref < 0:
payload, revision = deepcopy(plan["payloads"][ref]), 0
@ -4201,7 +4196,7 @@ def _read_relation_snapshot(work_id, refs, onstage):
wanted = set(refs) | {did for _, (did, kind, _) in onstage.items() if kind == "character"}
snapshots, characters = {}, []
with psycopg.connect(DSN) as conn:
with connect() as conn:
for did in sorted(wanted):
payload, revision = _read_upgrade_draft_snapshot(conn, did)
snapshots[did] = (payload, revision)
@ -4297,7 +4292,7 @@ def _apply_relation_stage(conn, work_id, win_no, relation_items, characters, rel
def _finalize_window(work_id, win_no, input_sha, relation_state_sha, prepared):
"""最终短事务原子完成向量写入与 done;semantic 关闭时 prepared 为空直接完成。"""
with psycopg.connect(DSN) as conn:
with connect() as conn:
_lock_upgrade_domains(conn)
_, marker = _assert_window_marker(
conn, work_id, win_no, "relation", input_sha, relation_state_sha,
@ -4447,7 +4442,7 @@ def recover_legacy_failed(work_id, window_no, preview, execute, confirmation_sha
if preview == execute:
raise click.ClickException("必须且只能指定 --preview 或 --execute")
if preview:
with psycopg.connect(DSN) as conn:
with connect() as conn:
conn.execute("SET TRANSACTION ISOLATION LEVEL REPEATABLE READ, READ ONLY")
snapshot = _capture_legacy_failed_snapshot(conn, work_id, window_no)
_echo_legacy_recovery_snapshot(snapshot, "preview")
@ -4458,7 +4453,7 @@ def recover_legacy_failed(work_id, window_no, preview, execute, confirmation_sha
raise click.ClickException("--execute 必须显式提供 --confirm-no-live-process")
try:
with upgrade_work_lock(DSN, TENANT, work_id):
with psycopg.connect(DSN) as conn:
with connect() as conn:
_lock_upgrade_domains(conn)
snapshot = _capture_legacy_failed_snapshot(conn, work_id, window_no, lock=True)
if snapshot["confirmation_sha"] != confirmation_sha:
@ -4517,7 +4512,7 @@ def repair_card_quality(work_id, draft_id, preview, execute, confirmation_sha,
raise click.ClickException(str(exc)) from exc
if preview:
try:
with psycopg.connect(DSN) as conn:
with connect() as conn:
conn.execute("SET TRANSACTION ISOLATION LEVEL REPEATABLE READ, READ ONLY")
snapshot = _quality_repair_capture_snapshot(conn, work_id, draft_id)
click.echo(json.dumps(
@ -4534,7 +4529,7 @@ def repair_card_quality(work_id, draft_id, preview, execute, confirmation_sha,
raise click.ClickException("--execute 必须显式提供 --confirm-no-live-process")
try:
with upgrade_work_lock(DSN, TENANT, work_id):
with psycopg.connect(DSN) as conn:
with connect() as conn:
conn.execute("SET TRANSACTION ISOLATION LEVEL REPEATABLE READ, READ ONLY")
snapshot = _quality_repair_capture_snapshot(conn, work_id, draft_id)
if _quality_repair_confirmation_sha(snapshot) != confirmation_sha:
@ -4557,7 +4552,7 @@ def repair_card_quality(work_id, draft_id, preview, execute, confirmation_sha,
f"actual={len(vectors) if isinstance(vectors, list) else type(vectors).__name__}"
)
_quality_repair_validate_vector(vectors[0])
with psycopg.connect(DSN) as conn:
with connect() as conn:
_lock_upgrade_domains(conn)
current = _quality_repair_capture_snapshot(conn, work_id, draft_id, lock=True)
if _quality_repair_confirmation_sha(current) != confirmation_sha:
@ -4734,7 +4729,7 @@ def repair_presence_duplicates(work_id, preview, execute, confirmation_sha,
)
if preview:
try:
with psycopg.connect(DSN) as conn:
with connect() as conn:
conn.execute("SET TRANSACTION ISOLATION LEVEL REPEATABLE READ, READ ONLY")
snapshot = _presence_dedupe_capture_snapshot(conn, work_id)
click.echo(json.dumps(
@ -4750,7 +4745,7 @@ def repair_presence_duplicates(work_id, preview, execute, confirmation_sha,
raise click.ClickException("--execute 必须显式提供 --confirm-no-live-process")
try:
with upgrade_work_lock(DSN, TENANT, work_id):
with psycopg.connect(DSN) as conn:
with connect() as conn:
_lock_upgrade_domains(conn)
snapshot = _presence_dedupe_capture_snapshot(conn, work_id, lock=True)
current_sha = _presence_dedupe_confirmation_sha(snapshot)
@ -4853,7 +4848,7 @@ def real_pg_rollback_smoke(work_id):
try:
with upgrade_work_lock(DSN, TENANT, work_id):
with psycopg.connect(DSN) as conn:
with connect() as conn:
_lock_upgrade_domains(conn)
row = conn.execute(
"""SELECT id, window_no, status, error_message
@ -4899,7 +4894,7 @@ def real_pg_rollback_smoke(work_id):
else:
raise RuntimeError("draft revision 漂移未被 _assert_window_marker 拒绝")
conn.rollback()
with psycopg.connect(DSN) as verify_conn:
with connect() as verify_conn:
restored_window_row = verify_conn.execute(
"""SELECT to_jsonb(w) FROM example_upgrade_window w
WHERE w.tenant_id=%s AND w.work_id=%s AND w.window_no=%s""",
@ -4949,7 +4944,7 @@ def _compensate_window(
original = f"{type(error).__name__}: {str(error)}"[:160]
return f"{COMPENSATION_FAILED_PREFIX} 原始异常={original}; {detail}"[:500]
with psycopg.connect(DSN) as conn:
with connect() as conn:
_lock_upgrade_domains(conn)
row = conn.execute(
"""SELECT id, status, error_message FROM example_upgrade_window
@ -5042,7 +5037,7 @@ def _compensate_window(
def _recover_processing_windows(work_id):
"""启动时先恢复 processing;exact 才 undo,漂移或非法 marker 持久化后立即停止。"""
with psycopg.connect(DSN) as conn:
with connect() as conn:
windows = conn.execute(
"""SELECT window_no FROM example_upgrade_window
WHERE tenant_id=%s AND work_id=%s AND status='processing' AND deleted=FALSE
@ -5073,7 +5068,7 @@ def _run(work_id, max_windows, max_calls, model, redo_window, semantic_on,
except CompensationFenceConflict as exc:
raise click.ClickException(str(exc)) from exc
with psycopg.connect(DSN) as conn:
with connect() as conn:
title = conn.execute(
"SELECT title FROM muse_content_work WHERE id=%s", (work_id,)
).fetchone()[0]
@ -5115,7 +5110,7 @@ def _run(work_id, max_windows, max_calls, model, redo_window, semantic_on,
attempt_state_sha = None
attempt_state_counts = None
try:
with psycopg.connect(DSN) as conn:
with connect() as conn:
window, chapters, schemas, input_title = _capture_window_input(
conn, work_id, win_no, validate=validate_window_input,
)
@ -5222,7 +5217,7 @@ def _run(work_id, max_windows, max_calls, model, redo_window, semantic_on,
call,
)
with psycopg.connect(DSN) as conn:
with connect() as conn:
ref_map, appearances, aliases, entity_touched, entity_state_sha = (
_apply_entity_stage(
conn,
@ -5255,7 +5250,7 @@ def _run(work_id, max_windows, max_calls, model, redo_window, semantic_on,
relation_rows,
call,
)
with psycopg.connect(DSN) as conn:
with connect() as conn:
relation_touched, relation_state_sha = _apply_relation_stage(
conn,
work_id,
@ -5353,7 +5348,7 @@ def run(work_id, max_windows, max_calls, model, redo_window, semantic_on):
@click.option("--work-id", type=int, required=True)
def status(work_id):
"""升格进度:窗状态/卡数/留档数。"""
with psycopg.connect(DSN) as conn:
with connect() as conn:
title = conn.execute("SELECT title FROM muse_content_work WHERE id=%s",
(work_id,)).fetchone()[0]
w = conn.execute(

View File

@ -23,10 +23,7 @@ from psycopg.rows import dict_row
from build_snapshot import filter_milestones, filter_outline_windows, normalize_chapter
DSN = (
"postgresql://root:f6710e2d0294eb1c10e26a805a64bc54@100.64.0.8:5433/muse-example"
"?keepalives=1&keepalives_idle=15&keepalives_interval=5&keepalives_count=3"
)
from muse_db import DSN
TENANT_ID = 1
REPO_ROOT = Path(__file__).resolve().parents[4]
DEFAULT_SNAPSHOT_VERSION = "next_fine_outline_replay_v0"

View File

@ -15,8 +15,7 @@ import click
import psycopg
from psycopg.types.json import Jsonb
DSN = ("postgresql://root:f6710e2d0294eb1c10e26a805a64bc54@100.64.0.8:5433/muse-example"
"?keepalives=1&keepalives_idle=15&keepalives_interval=5&keepalives_count=3")
from muse_db import connect
# keepalive 防 Tailscale 半死连接(2026-07-13 实测:逐行插入两万次往返曾卡死 16 分钟)
TENANT, ACTOR, OWNER = 1, "1", 1 # 实验写入约定:系统主账号
@ -251,7 +250,7 @@ def import_book(path: pathlib.Path, force: bool):
command_id = f"import-{file_hash[:16]}"
total_words = sum(len(re.sub(r'\s', '', c['text'])) for c in chapters)
with psycopg.connect(DSN) as conn:
with connect() as conn:
exist = conn.execute(
"SELECT id FROM muse_content_work WHERE tenant_id=%s AND title=%s AND deleted=FALSE",
(TENANT, meta["title"])).fetchone()

View File

@ -7,8 +7,7 @@ import sys
import click
import psycopg
DSN = ("postgresql://root:f6710e2d0294eb1c10e26a805a64bc54@100.64.0.8:5433/muse-example"
"?keepalives=1&keepalives_idle=15&keepalives_interval=5&keepalives_count=3")
from muse_db import connect
TENANT = 1
# 语义垃圾嫌疑模式(扫描计数用;真正删除走 clean-book-text 的 LLM 检测+代码执行)
@ -25,7 +24,7 @@ JUNK_PATTERNS = {
@click.option("--work-id", type=int, multiple=True, help="不给则全部")
@click.option("--sample-titles", default=5, show_default=True)
def main(work_id, sample_titles):
with psycopg.connect(DSN) as conn:
with connect() as conn:
works = conn.execute(
f"""SELECT id, title FROM muse_content_work WHERE tenant_id=%s AND deleted=FALSE
{'AND id = ANY(%s)' if work_id else ''} ORDER BY id""",

View File

@ -390,9 +390,10 @@ def capture_code_identity(git_commit: str | None = None) -> dict[str, Any]:
"parse_upgrade.py": extraction_scripts / "upgrade.py",
# parse_llm 留 parse-book(字节不动);路径单向跨 skill 定位,逻辑名 "parse_llm.py" 不变。
"parse_llm.py": here.parents[1] / "deconstruct-book" / "scripts" / "parse_llm.py",
"embed_drafts.py": here.parents[1] / "embed-knowledge" / "scripts" / "embed_drafts.py",
# 嵌入与模型调用的实现已装成共享运行时包;逻辑名不变,指向包内实现而非 Skill 的薄 CLI。
"embed_drafts.py": here.parents[3] / "muse-embed" / "src" / "muse_embed.py",
"upgrade_work_lock.py": extraction_scripts / "upgrade_work_lock.py",
"llm.py": here.parents[1] / "call-content-model" / "scripts" / "llm.py",
"llm.py": here.parents[3] / "muse-llm" / "src" / "muse_llm.py",
# 逻辑名保留 "parse-book/SKILL.md"(稳定契约键);随 __file__ 实际指向 upgrade/SKILL.md。
"parse-book/SKILL.md": extraction_scripts.parent / "SKILL.md",
}
@ -808,9 +809,10 @@ def _default_connect(dsn: str, **kwargs: Any) -> Any:
def _load_db_config() -> tuple[str, int]:
"""延迟复用 parse-book 既有数据库配置;本脚本和清单均不复制密码。"""
"""连接串来自共享的 muse_db,租户口径仍随 parse-book;本脚本和清单均不复制密码。"""
from parse_llm import DSN, TENANT
from muse_db import DSN
from parse_llm import TENANT
return DSN, TENANT

View File

@ -31,15 +31,14 @@ import click
import psycopg
# 复用作品抽取器的守卫/工具(剥前缀、剥尾残、垃圾拦截、内嵌章号、生命周期推断、章号排序键、
# 窗源文加载、SOURCE_TYPE/TENANT/DSN)——迁移与抽取同一套清洗口径,不另立标准。
# 窗源文加载、SOURCE_TYPE/TENANT)——迁移与抽取同一套清洗口径,不另立标准。
HERE = pathlib.Path(__file__).resolve().parent
sys.path.insert(0, str(HERE))
sys.path.insert(0, str(HERE.parents[1] / "extract-work-knowledge" / "scripts"))
sys.path.insert(0, str(HERE.parents[1] / "call-content-model" / "scripts"))
import upgrade as pu # noqa: E402
from llm import chat_governed, extract_json # noqa: E402
from muse_llm import chat_governed, extract_json
from muse_db import connect # noqa: E402
DSN = pu.DSN # 带 keepalives(防 Tailscale 长空转掐断),与抽取同源
SOURCE_TYPE = pu.SOURCE_TYPE
TENANT = pu.TENANT
@ -280,7 +279,7 @@ def main(work_id, ids, limit, use_llm, max_llm_windows, samples):
id_list = [int(x) for x in ids.split(",") if x.strip()] if ids else None
# ── 短连接①:读窗号上限 + 待迁卡(读完即释放)──
with psycopg.connect(DSN) as conn:
with connect() as conn:
win_max_map = load_window_maxima(conn)
cards = load_target_cards(conn, work_id, id_list, limit)
click.echo(f"待迁卡:{len(cards)} 张(演变类字段含 [窗N] 前缀条目)")
@ -307,7 +306,7 @@ def main(work_id, ids, limit, use_llm, max_llm_windows, samples):
if use_llm and pending_by_work:
for wid, wmap in pending_by_work.items():
win_nos = sorted(wmap)[:max_llm_windows] # 控额度:只精确化前 N 个窗
with psycopg.connect(DSN) as conn: # 短连接读源文
with connect() as conn: # 短连接读源文
title = conn.execute("SELECT title FROM muse_content_work WHERE id=%s",
(wid,)).fetchone()[0]
win_ranges = load_win_ranges(conn, wid, win_nos)

View File

@ -10,10 +10,8 @@ import json
import re
import sys
import psycopg
DSN = ("postgresql://root:f6710e2d0294eb1c10e26a805a64bc54@100.64.0.8:5433/muse-example"
"?keepalives=1")
from muse_db import connect
TENANT = 1
PREFIX_RE = re.compile(r"^(?:\[[窗本][^\]]{0,6}\]\s*)+") # 含 [窗本窗] 等模型自造变体
WIN_NUMS = re.compile(r"\[窗(\d+)\]")
@ -56,7 +54,7 @@ def rebuild_presence(chapters, names):
def main():
stats = {"前缀清洗卡": 0, "关系迁移卡": 0, "出场章重建卡": 0, "删别名行": 0, "删占位字段": 0}
with psycopg.connect(DSN) as conn:
with connect() as conn:
load_valid_keys(conn)
for work_id in (4, 8):
chapters = conn.execute(

View File

@ -26,7 +26,6 @@ import sys
import pathlib
import click
import psycopg
from psycopg.rows import dict_row
# 复用作品抽取 Skill 的连接、租户与同书锁;维护侧不另造兼容实现。
@ -38,7 +37,8 @@ sys.path.insert(0, str(
))
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[2] / "deconstruct-book" / "scripts"))
import backup_upgrade_work as backup # noqa: E402
from parse_llm import DSN, TENANT # noqa: E402
from muse_db import DSN, connect # noqa: E402
from parse_llm import TENANT # noqa: E402
from upgrade_work_lock import UpgradeWorkLockUnavailable, upgrade_work_lock # noqa: E402
SOURCE_TYPE = "upgrade_book"
@ -119,7 +119,7 @@ def _assert_code_identity_matches_manifest(manifest, current_identity):
def _reset(work_id, execute, manifest=None, code_identity=None):
"""在调用方已持有同书锁时预览或执行重抽清理。"""
with psycopg.connect(DSN) as conn:
with connect() as conn:
if execute:
# 输入源与七域表锁、七域核对及完整输入绑定均在同一事务内,先于 destructive SQL。
conn.execute(RESET_TABLE_LOCK_SQL)

View File

@ -10,15 +10,11 @@ payload = 对应 section_type 的 schema 字段结构化内容(JSON),由 p
--dry-run 试跑:插入/翻态后回滚,校验但不落库。
"""
import json
import sys
from pathlib import Path
import yaml
# 复用 access-database Skill 锁死的 DSN
DB_SCRIPTS = Path(__file__).resolve().parents[2] / "access-database" / "scripts"
sys.path.insert(0, str(DB_SCRIPTS))
from db import connect # noqa: E402
from muse_db import connect
CREATOR = "planning"
SECTION_TYPES = ("setting", "outline", "state", "assembly", "fine_outline")

View File

@ -13,13 +13,10 @@ import sys
ROOT = pathlib.Path(__file__).resolve().parents[2]
for path in (
ROOT / "access-database" / "scripts",
ROOT / "record-run-evidence" / "scripts",
):
if str(path) not in sys.path:
sys.path.insert(0, str(path))
from db import connect # noqa: E402
_evidence_scripts = ROOT / "record-run-evidence" / "scripts"
if str(_evidence_scripts) not in sys.path:
sys.path.insert(0, str(_evidence_scripts))
from muse_db import connect # noqa: E402
from run_registry import finish_run, start_run # noqa: E402
from persist_raw import _bare_sha256, _check_no_secrets # noqa: E402

View File

@ -11,13 +11,10 @@ import sys
ROOT = pathlib.Path(__file__).resolve().parents[2]
for path in (
ROOT / "access-database" / "scripts",
ROOT / "record-run-evidence" / "scripts",
):
if str(path) not in sys.path:
sys.path.insert(0, str(path))
from db import connect # noqa: E402
_evidence_scripts = ROOT / "record-run-evidence" / "scripts"
if str(_evidence_scripts) not in sys.path:
sys.path.insert(0, str(_evidence_scripts))
from muse_db import connect # noqa: E402
CREATOR = "planning-receipt-repair"

View File

@ -9,21 +9,13 @@ from __future__ import annotations
import argparse
import hashlib
import json
import sys
from pathlib import Path
SCRIPT_DIR = Path(__file__).resolve().parent
AGENT_ROOT = SCRIPT_DIR.parents[3]
for _p in (
AGENT_ROOT / "humanization" / "src",
AGENT_ROOT / ".claude" / "skills" / "access-database" / "scripts",
AGENT_ROOT / ".claude" / "skills" / "establish-voice-baseline" / "scripts",
):
if str(_p) not in sys.path:
sys.path.insert(0, str(_p))
from deai import load # noqa: E402
from deai.schemas import validate # noqa: E402
from deai import load
from deai.schemas import validate
from deai.baseline import validate_ledger
from deai.load_db import load_current_baseline
from muse_db import connect
TENANT_ID = 1
CREATOR = "1"
@ -67,14 +59,11 @@ def render_writer_constraints(contract: dict) -> list[str]:
def _load_db_ledger(work_ref: str) -> dict | None:
from establish_voice_baseline import load_current_baseline
return load_current_baseline(work_ref)
with connect(readonly=True) as conn:
return load_current_baseline(conn, work_ref)
def _validate_voice_ledger(voice_ledger: dict, work_ref: str) -> None:
from establish_voice_baseline import validate_ledger
try:
validate_ledger(voice_ledger, work_ref=work_ref)
except ValueError as exc:
@ -88,7 +77,7 @@ def load_runtime_library(load_database: bool) -> tuple[dict, dict, str, str]:
落库前的新鲜度检查必须与合同声明的来源一致。
"""
if load_database:
from db import connect
from muse_db import connect
from deai import load_db
with connect(readonly=True) as conn:
@ -190,7 +179,7 @@ def build_prevention_contract(work_ref: str, *, voice_ledger: dict | None = None
def persist_prevention(contract: dict, *, creator: str = CREATOR, tenant_id: int = TENANT_ID) -> dict:
"""上下文合同构建也是一次运行:example_run 留痕。"""
from db import connect
from muse_db import connect
try:
validate(contract, "prevention")

View File

@ -12,16 +12,13 @@ I3 评测候选的质量结果可被生产视图过滤(带评测标记,不
I4 oracle 读侧红线:oracle/标准答案以 kind='oracle' 标记,生产模型输入不得包含(read-context 生产路径保证;此处校验标记完整)。
I5 COMPLETED 轮次封存:一轮声明的 raw 集合 == 实际落库集合(lease 声明的 content_hashes 数 == 实际 raw_content 行数)。
跑法:.venv/bin/python .claude/skills/execute-claude-task/scripts/invariant_checks.py [--run-id X]
跑法:.venv/bin/python .claude/skills/record-run-evidence/scripts/invariant_checks.py [--run-id X]
"""
import argparse
import json
import sys
from pathlib import Path
DB_SCRIPTS = Path(__file__).resolve().parents[2] / "access-database" / "scripts"
sys.path.insert(0, str(DB_SCRIPTS))
from db import connect # noqa: E402
from muse_db import connect
def _count(conn, sql, params=()):

View File

@ -79,7 +79,7 @@ def persist_call(event, *, creator=CREATOR, dry_run=False):
``event`` 由 llm.chat 生成,至少包含 prompt/response、模型、usage 和调用方字段。
返回各证据行 id;dry-run 只验证事务并回滚。
"""
from db import connect
from muse_db import connect
prompt = event.get("prompt")
response = event.get("response")

View File

@ -10,13 +10,9 @@ raw 进库可看全文(看板可读),仓外 vault 降级为可选备份;
import hashlib
import json
import re
import sys
from pathlib import Path
# 复用 access-database Skill 锁死的 DSN
DB_SCRIPTS = Path(__file__).resolve().parents[2] / "access-database" / "scripts"
sys.path.insert(0, str(DB_SCRIPTS))
from db import connect # noqa: E402
from muse_db import connect
CREATOR = "runtime"
KINDS = ("prompt", "response", "source_text", "oracle", "supplier")

View File

@ -7,13 +7,9 @@ raw 指针,不复制供应商响应,不把失败伪装成通过。
import argparse
import hashlib
import json
import pathlib
import sys
DB_SCRIPTS = pathlib.Path(__file__).resolve().parents[2] / "access-database" / "scripts"
sys.path.insert(0, str(DB_SCRIPTS))
from db import connect # noqa: E402
from muse_db import connect
CREATOR = "runtime-failure-receipt"

View File

@ -6,13 +6,9 @@
"""
import argparse
import json
import pathlib
import sys
DB_SCRIPTS = pathlib.Path(__file__).resolve().parents[2] / "access-database" / "scripts"
sys.path.insert(0, str(DB_SCRIPTS))
from db import connect # noqa: E402
from muse_db import connect
CREATOR = "runtime-receipt-repair"

View File

@ -8,15 +8,10 @@ from contextlib import contextmanager
from datetime import datetime
import json
import re
import sys
import uuid
from pathlib import Path
DB_SCRIPTS = Path(__file__).resolve().parents[2] / "access-database" / "scripts"
if str(DB_SCRIPTS) not in sys.path:
sys.path.insert(0, str(DB_SCRIPTS))
from db import connect # noqa: E402
from muse_db import connect
CREATOR = "runtime"

View File

@ -16,11 +16,9 @@ import sys
import click
import psycopg
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[2] / "call-content-model" / "scripts"))
from llm import chat_governed, extract_json, BUDGET_CHAIN # noqa: E402
from muse_llm import chat_governed, extract_json, BUDGET_CHAIN
DSN = ("postgresql://root:f6710e2d0294eb1c10e26a805a64bc54@100.64.0.8:5433/muse-example"
"?keepalives=1&keepalives_idle=15&keepalives_interval=5&keepalives_count=3")
from muse_db import connect
TENANT, ACTOR = 1, "1"
GOLDEN = pathlib.Path(__file__).resolve().parents[1] / "golden"
@ -116,7 +114,7 @@ def review(work_id, batch, model, shard, dry_run):
tag = f"[片{si}/{sn}]" if shard else ""
# 读卡:短连接读完即释放。关键——LLM 阶段(可十几分钟)绝不持有 DB 连接,
# 否则连接空转会被 Tailscale 掐断、最后写库时连接已死→整片回滚(实测坑,2026-07-16)。
with psycopg.connect(DSN) as rconn:
with connect() as rconn:
rows = load_cards(rconn, work_id)
if shard:
rows = [r for r in rows if r[0] % sn == si] # r[0]=卡 id,按 id 取模切片
@ -192,7 +190,7 @@ def review(work_id, batch, model, shard, dry_run):
to_write.append((cid, payload))
# —— 写库阶段:全新短连接、快速批量写、连接全程活跃(无 LLM 夹在中间→不空转超时)——
if not dry_run:
with psycopg.connect(DSN) as wconn:
with connect() as wconn:
for cid, payload in to_write:
wconn.execute("UPDATE muse_knowledge_draft SET draft_payload=%s, updater=%s WHERE id=%s",
(json.dumps(payload, ensure_ascii=False), ACTOR, cid))
@ -209,7 +207,7 @@ def export(work_id, out):
"""全部活卡导出为人读样张(创始人确认门的入口物料——他看文件,不读数据库)。
每卡:审核判定/三角色判词/字段全文/实例章号/审计标记;按书分节、判定排序。"""
order = {"pass": 0, "revise": 1, "reject": 2, None: 3}
with psycopg.connect(DSN) as conn:
with connect() as conn:
rows = conn.execute(
"""SELECT w.title, d.id, d.draft_payload FROM muse_knowledge_draft d
JOIN muse_content_work w ON w.id=d.source_id
@ -258,7 +256,7 @@ def export(work_id, out):
def calibrate(work_id, batch):
"""与 golden/scores.json 金标准对照:输出逐卡偏差与整体一致性(起量前校准用)。"""
golden = json.loads((GOLDEN / "scores.json").read_text()) # {卡名: 金标准均分}
with psycopg.connect(DSN) as conn:
with connect() as conn:
rows = load_cards(conn, work_id)
diffs, lines = [], []
for _, payload in rows:

View File

@ -12,22 +12,13 @@ dimension=ai_flavor_revision,绑候选稿 sha256);--offline 才不写库
import argparse
import hashlib
import json
import sys
from pathlib import Path
SCRIPT_DIR = Path(__file__).resolve().parent
AGENT_ROOT = SCRIPT_DIR.parents[3]
for _p in (AGENT_ROOT / "humanization" / "src",
AGENT_ROOT / ".claude" / "skills" / "access-database" / "scripts",
AGENT_ROOT / ".claude" / "skills" / "establish-voice-baseline" / "scripts"):
if str(_p) not in sys.path:
sys.path.insert(0, str(_p))
from deai import load, report as report_mod # noqa: E402
from establish_voice_baseline import load_current_baseline # noqa: E402
from deai.pairwise import PairwiseNotExecuted # noqa: E402
from deai.patch import PatchError # noqa: E402
from deai.pipeline import DowngradedToAudit, RevisionNotAuthorized, run_patch # noqa: E402
from deai import load, report as report_mod
from deai.load_db import load_current_baseline
from deai.pairwise import PairwiseNotExecuted
from deai.patch import PatchError
from deai.pipeline import DowngradedToAudit, RevisionNotAuthorized, run_patch
TENANT_ID = 1
CREATOR = "1"
@ -50,8 +41,8 @@ def _load_json(path: Path, what: str) -> dict:
def load_revision_library(from_db: bool) -> tuple[dict, dict, str]:
"""修订用规则库:生产读数据库(失败关闭,不回退 Git),离线读文件。"""
if from_db:
from db import connect
from deai import load_db
from muse_db import connect
with connect(readonly=True) as conn:
samples = load_db.load_samples_from_db(conn)
@ -93,7 +84,7 @@ def persist_revision(*, work_ref: str, text_hash: str, candidate_text: str | Non
audit: dict | None, conclusion: str, detail: dict,
creator: str = CREATOR, tenant_id: int = TENANT_ID) -> dict:
"""修订运行落库:候选稿哈希绑定评判,append-only 记账。"""
from db import connect
from muse_db import connect
if not work_ref or not isinstance(detail, dict):
raise ReviseContractError("修订落库缺少 work_ref/detail")
@ -139,7 +130,7 @@ def persist_revision(*, work_ref: str, text_hash: str, candidate_text: str | Non
def _persist_downgrade(*, work_ref: str, text_hash: str, reason: str) -> dict:
"""降级也是运行事实:落 example_run,避免「静默没发生」。"""
from db import connect
from muse_db import connect
run_id = _run_id(work_ref, text_hash, "downgrade")
detail = {"status": "downgraded_to_audit", "reason": reason, "work_ref": work_ref}
@ -181,7 +172,9 @@ def main(argv: list[str] | None = None) -> int:
snapshot = _load_json(args.snapshot, "事实快照") if args.snapshot else None
ledger = _load_json(args.voice_ledger, "声音账") if args.voice_ledger else None
if ledger is None and not args.offline:
ledger = load_current_baseline(args.work_ref)
from muse_db import connect
with connect(readonly=True) as conn:
ledger = load_current_baseline(conn, args.work_ref)
pairwise = _load_json(args.pairwise, "成对选择记录") if args.pairwise else None
record, audit = run_revision(
text, artifact=artifact, patches=patches_raw, task_contract=task_contract,

View File

@ -12,16 +12,9 @@
from __future__ import annotations
import json
import pathlib
import sys
from typing import Any, Mapping
SCRIPT_DIR = pathlib.Path(__file__).resolve().parent
DB_DIR = SCRIPT_DIR.parents[1] / "access-database" / "scripts"
if str(DB_DIR) not in sys.path:
sys.path.insert(0, str(DB_DIR))
from db import connect # noqa: E402
from muse_db import connect
KINDS = frozenset({"lesson", "win"})
STATUSES = frozenset({"proposed", "reviewing", "promoted", "rejected"})

View File

@ -11,10 +11,8 @@ import sys
from typing import Any, Mapping, Protocol, Sequence, runtime_checkable
SCRIPT_DIR = pathlib.Path(__file__).resolve().parent
EXECUTION_DIR = SCRIPT_DIR.parents[1] / "execute-claude-task" / "scripts"
for _import_dir in (SCRIPT_DIR, EXECUTION_DIR):
if str(_import_dir) not in sys.path:
sys.path.insert(0, str(_import_dir))
if str(SCRIPT_DIR) not in sys.path:
sys.path.insert(0, str(SCRIPT_DIR))
from writer_rubric import ( # noqa: E402
DIMENSIONS,

View File

@ -4,18 +4,15 @@
合同见同目录 SKILL.md。查询嵌入与知识行同模型同维(复用 embed-knowledge 的实现)。
"""
import json
import pathlib
import sys
from typing import Any, Callable
import click
import psycopg
# 复用 embed-knowledge 的通道实现(同模型同维,语义对齐)
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[2] / "embed-knowledge" / "scripts"))
from embed_drafts import _session, embed_texts # noqa: E402
from muse_embed import _session, embed_texts
DSN = "postgresql://root:f6710e2d0294eb1c10e26a805a64bc54@100.64.0.8:5433/muse-example"
from muse_db import DSN
TENANT = 1

View File

@ -14,12 +14,10 @@ import sys
from typing import Any, Callable
SCRIPT_DIR = pathlib.Path(__file__).resolve().parent
DB_DIR = SCRIPT_DIR.parents[1] / "access-database" / "scripts"
for path in (SCRIPT_DIR, DB_DIR):
if str(path) not in sys.path:
sys.path.insert(0, str(path))
if str(SCRIPT_DIR) not in sys.path:
sys.path.insert(0, str(SCRIPT_DIR))
from db import connect as _default_connect # noqa: E402
from muse_db import connect as _default_connect # noqa: E402
from run_writer_pipeline import CasToken, PipelineError # noqa: E402

View File

@ -15,12 +15,11 @@ from typing import Any, Mapping
SCRIPT_DIR = pathlib.Path(__file__).resolve().parent
EVIDENCE_DIR = SCRIPT_DIR.parents[1] / "record-run-evidence" / "scripts"
READ_CONTEXT_DIR = SCRIPT_DIR.parents[1] / "assemble-context" / "scripts"
DB_DIR = SCRIPT_DIR.parents[1] / "access-database" / "scripts"
for path in (EVIDENCE_DIR, READ_CONTEXT_DIR, DB_DIR):
for path in (EVIDENCE_DIR, READ_CONTEXT_DIR):
if str(path) not in sys.path:
sys.path.insert(0, str(path))
from db import connect # noqa: E402
from muse_db import connect # noqa: E402
from persist_context_freeze import persist_freeze # noqa: E402
from run_registry import finish_run, start_run # noqa: E402

View File

@ -11,10 +11,8 @@ from typing import Any, Callable, Mapping, Sequence
SCRIPT_DIR = pathlib.Path(__file__).resolve().parent
READ_CONTEXT_DIR = SCRIPT_DIR.parents[1] / "assemble-context" / "scripts"
EXECUTION_DIR = SCRIPT_DIR.parents[1] / "execute-claude-task" / "scripts"
for import_path in (READ_CONTEXT_DIR, EXECUTION_DIR):
if str(import_path) not in sys.path:
sys.path.insert(0, str(import_path))
if str(READ_CONTEXT_DIR) not in sys.path:
sys.path.insert(0, str(READ_CONTEXT_DIR))
from claude_runtime import ( # noqa: E402
ClaudeRuntimeError,

1
.gitignore vendored
View File

@ -4,6 +4,7 @@ works/*/评审/
# python 虚拟环境(依赖清单在 requirements.txt,uv 一键重建)
.venv/
__pycache__/
*.egg-info/
# IDE 本地配置不入库
.idea/
# 运行 Skill 产生的 Python 缓存不属于源码

View File

@ -47,7 +47,12 @@ agent-example/
│ ├── ddl/ # 可审计 DDL / 迁移文件
│ ├── 表映射.md
│ └── 连接信息.md
├── humanization/ # 去 AI 味 Skill 能力域(运行时规则/样例以数据库为权威;YAML 为迁移种子/离线夹具)
├── humanization/ # 去 AI 味共享运行时库(muse-deai)
├── muse-db/ # 共享连接模块(muse_db:锁死 DSN 与只读/可写会话)
├── muse-llm/ # New-API 内容调用库(muse_llm:额度窗、降级链、JSON 容错)
├── muse-embed/ # 知识嵌入库(muse_embed:会话、向量请求、draft 写入)
├── muse-claude-runtime/ # 受控 Claude CLI 运行时(claude_runtime:冻结 profile、沙箱、回执)
├── dashboard/ # 只读看板;经 muse_db.connect(readonly=True) 读库
├── harness/ # 项目验证与外部评测索引、清单和调度支架
├── tests/ # 系统能力 Skill 实现测试(按 Skill 归档)
├── knowledge/ # 仓内参考资产;未经绑定、授权不得进入上下文
@ -80,7 +85,7 @@ agent-example/
**判据是产不产生系统事实,不是有没有 `scripts/`**:`plan-chapter`、`expand-scene`、`polish-prose`、`rewrite-selection`、`optimize-content-quality` 以模型判断为主、自身不带 Tool,落库由它们调用的 Skill 的 `scripts/` 承担,仍是系统能力 Skill。两类都是本仓正式 skill,都可在创作生命周期内被角色取用(参照 Skill 同样参与检测与评审),取用边界只由各自 `SKILL.md` 声明。
`humanization/` 是“去 AI 味与人感”Skill 家族的能力域:`src/deai/` 是共享实现。规则与样例的运行时权威是 `example_ai_flavor_rule` / `example_ai_flavor_sample`(DDL-111,`humanization/tools/seed_rules_db.py` 种子同步,生产读取失败关闭,不静默回退 Git);案例卡与声音账同样入库。仓内 YAML/JSON 是迁移种子、离线夹具和结构合同;规则生命周期变更经 YAML 评测/激活后同步入库。规则记录不各自注册为 Skill,Skill 负责动作和消费边界。
`humanization/` 是“去 AI 味与人感”Skill 家族的能力域:`src/deai/` 是共享运行时库(包名 `muse-deai`,经 `requirements.txt` 的 `-e ./humanization` 安装)。被两个以上 Skill 或看板消费的确定性实现一律装成顶层可安装包,所属 Skill 只留 CLI:`muse-db`(连接)、`muse-llm`(模型调用与额度窗)、`muse-embed`(嵌入)、`muse-claude-runtime`(受控 Claude 运行时),均在 `requirements.txt` 以 `-e ./<包>` 安装。调用方 `import` 已安装的包,不得 `sys.path` 指向 `access-database/scripts`、`call-content-model/scripts`、`embed-knowledge/scripts`、`execute-claude-task/scripts`、`establish-voice-baseline/scripts` 或 `humanization/src`;门禁见 [`tests/architecture/test_import_boundaries.py`](tests/architecture/test_import_boundaries.py)。规则与样例的运行时权威是 `example_ai_flavor_rule` / `example_ai_flavor_sample`(DDL-111,`humanization/tools/seed_rules_db.py` 种子同步,生产读取失败关闭,不静默回退 Git);案例卡与声音账同样入库。仓内 YAML/JSON 是迁移种子、离线夹具和结构合同;规则生命周期变更经 YAML 评测/激活后同步入库。规则记录不各自注册为 Skill,Skill 负责动作和消费边界。`humanization/tests`、`tools`、`eval` 仍按包内惯例装载源码树。
15 个参照 Skill 来自 7 本写作书的方法论单元按创作领域合并(`SKILL.md` 入口 + `references/` 全量内容 + `scripts/` 工作表与清单),供 writer、planner、judge 等角色在对应创作阶段按需取用;它们不设维护门禁。蒸馏与裁剪的历史留痕见 `docs/2026-08-19-craft-distillation-trace.md`(原 `craft/` 目录已清理,原料与旧 SoT 由 git 历史保留)。
@ -114,7 +119,7 @@ Skill 领域列表的新增、删除、改名或主领域调整,必须同时
## 6. 模型边界
- 清洗、抽卡、范式拆取及其模型调用统一走 `call-content-model` Skill,不裸调 New-API。治理政策固定为 5 小时额度窗:MiniMax 模型累计花费上限 `$24`,全模型成功调用上限 `6000`;运行适配器、正式配置和账本是额度合同的事实源,`.claude/skills/call-content-model/scripts/llm.py` 是实现,`test_quota.py` 只提供回归证据;模型链切换必须由该 Skill 治理并留下日志。
- 清洗、抽卡、范式拆取及其模型调用统一走 `call-content-model` Skill,不裸调 New-API。治理政策固定为 5 小时额度窗:MiniMax 模型累计花费上限 `$24`,全模型成功调用上限 `6000`;运行适配器、正式配置和账本是额度合同的事实源,共享库 `muse_llm` 与 `muse_db.WINDOW_BUDGET_USD` / `WINDOW_CALL_CAP` 是实现,Skill CLI 只做入口,`test_quota.py` 只提供回归证据;模型链切换必须由该治理入口留下日志。
- 角色模型归属:`planner`/`writer`/`judge` 固定 `opus`;`extractor`/`detector` 可用其它模型(非必须降级)。拆书/导入侧抽取经 `call-content-model`/`deconstruct-book` Skill 走 MiniMax-M3,不走角色 model 派发;创作期章后抽取作为角色派发,可用 `opus`。
- 确定性脚本、合同校验、快照冻结、泄漏审计和报告生成不调用模型;除非对应 `SKILL.md` 明确声明模型步骤,不得把机械任务升级为模型任务。
- Claude 生成或评测只在对应任务 SoT、显式预算、固定执行配置和原文用途授权全部满足后运行;任一前置门失败都应关闭执行。

View File

@ -0,0 +1 @@
看板离线 JSON 夹具(数据库不可用时的回退展示)。AI 味案例的正式来源仍是 muse-example。

View File

@ -21,18 +21,12 @@ from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
from urllib.parse import parse_qs, urlencode, urlparse
import psycopg
from muse_db import WINDOW_BUDGET_USD, WINDOW_CALL_CAP, connect
# 与 access-database Skill 同一库,但独立只读连接(合同 §2:连接串与写通道分开,只读是机械门)
DSN = "postgresql://root:f6710e2d0294eb1c10e26a805a64bc54@100.64.0.8:5433/muse-example"
HOST, PORT = "127.0.0.1", 8765
# 额度治理上限(call-content-model Skill 合同:5 小时窗 $24 / 6000 次)
QUOTA_USD_CAP, QUOTA_CALL_CAP = 24.0, 6000
# AI 味案例的正式来源是 muse-example;离线 JSON 只作为数据库不可用时的恢复/回退证据。
AI_FLAVOR_REPORT_DIR = (Path(__file__).resolve().parents[1]
/ ".claude/skills/capture-ai-flavor-cases/references/fixtures")
AI_FLAVOR_REPORT_DIR = Path(__file__).resolve().parent / "fixtures"
AI_FLAVOR_REVALIDATION_SCHEMA = "ai-flavor-revalidation-v1"
AI_FLAVOR_CASE_SCHEMA = "ai-flavor-case-v1"
AI_FLAVOR_TENANT_ID = 1
@ -58,14 +52,12 @@ _AI_FLAVOR_REVALIDATION_LABELS = {
}
def ro_connect():
"""只读连接:会话级锁死只读,写语句会被 PostgreSQL 拒绝(机械门,不靠自觉)。"""
return psycopg.connect(DSN, options="-c default_transaction_read_only=on")
def q(sql, params=()):
"""只读查询,返回 (列名, 行)。每次请求实时查库,不缓存(合同 §7)。"""
with ro_connect() as conn:
"""只读查询,返回 (列名, 行)。每次请求实时查库,不缓存(合同 §7)。
readonly=True 是会话级锁死:写语句会被 PostgreSQL 拒绝(机械门,不靠自觉)。
"""
with connect(readonly=True) as conn:
cur = conn.execute(sql, params)
cols = [d.name for d in cur.description] if cur.description else []
return cols, cur.fetchall()
@ -1081,8 +1073,8 @@ def _quota_block():
f"<div class='cap'>{text}(上限 {cap:,.0f})</div></div>")
return (f"<h2>额度水位 <span style='font-weight:400;color:var(--muted);font-size:12px'>额度窗口 <code>{esc(wk)}</code></span></h2>"
f"<div class='card'><div style='padding:14px 16px'>"
f"{meter(usd, QUOTA_USD_CAP, f'模型 <code>MiniMax</code> 花费 ${usd:.4f} / ${QUOTA_USD_CAP:.0f}')}"
f"{meter(calls, QUOTA_CALL_CAP, f'成功调用 {calls:,} / {QUOTA_CALL_CAP:,} 次')}</div></div>")
f"{meter(usd, WINDOW_BUDGET_USD, f'模型 <code>MiniMax</code> 花费 ${usd:.4f} / ${WINDOW_BUDGET_USD:.0f}')}"
f"{meter(calls, WINDOW_CALL_CAP, f'成功调用 {calls:,} / {WINDOW_CALL_CAP:,} 次')}</div></div>")
def view_home():

View File

@ -143,6 +143,19 @@ class DashboardDisplayTest(unittest.TestCase):
self.assertIn("按作品汇总(1 本)", text)
self.assertNotIn("按作品汇总(8 本)", text)
def test随包发布的离线回退夹具可被回退加载器解析(self):
"""离线回退夹具归看板自己所有;夹具坏了要在这里暴露,不能到断库时才发现。"""
report, error = server._load_ai_flavor_report()
self.assertIsNone(error)
self.assertEqual("ai-flavor-inventory-v1", report["schema_version"])
revalidation, error = server._load_ai_flavor_revalidation_report()
self.assertIsNone(error)
self.assertTrue(revalidation["usable"])
self.assertEqual(
{"cards": 788, "verified": 788, "stale": 0, "unavailable": 0, "card_mismatch": 0},
revalidation["totals"],
)
if __name__ == "__main__":
unittest.main()

View File

@ -18,7 +18,6 @@ SCRIPT_DIR = Path(__file__).resolve().parent
SKILLS = SCRIPT_DIR.parents[1] / ".claude" / "skills"
for sub in (
"write-next-chapter/scripts",
"execute-claude-task/scripts",
"assemble-context/scripts",
):
p = str(SKILLS / sub)

View File

@ -9,13 +9,11 @@ import sys
from pathlib import Path
SCRIPT_DIR = Path(__file__).resolve().parent
DB_SCRIPTS = SCRIPT_DIR.parents[1] / ".claude" / "skills" / "access-database" / "scripts"
PLANNING_SCRIPTS = SCRIPT_DIR.parents[1] / ".claude" / "skills" / "plan-story" / "scripts"
for p in (str(DB_SCRIPTS), str(PLANNING_SCRIPTS)):
if p not in sys.path:
sys.path.insert(0, p)
if str(PLANNING_SCRIPTS) not in sys.path:
sys.path.insert(0, str(PLANNING_SCRIPTS))
from db import connect # noqa: E402
from muse_db import connect # noqa: E402
from persist_planning import write_section, confirm_section # noqa: E402
WORK_TITLE = "深渊机神"

View File

@ -16,13 +16,11 @@ import sys
from pathlib import Path
SCRIPT_DIR = Path(__file__).resolve().parent
DB_SCRIPTS = SCRIPT_DIR.parents[1] / ".claude" / "skills" / "access-database" / "scripts"
PLANNING_SCRIPTS = SCRIPT_DIR.parents[1] / ".claude" / "skills" / "plan-story" / "scripts"
for p in (str(DB_SCRIPTS), str(PLANNING_SCRIPTS)):
if p not in sys.path:
sys.path.insert(0, p)
if str(PLANNING_SCRIPTS) not in sys.path:
sys.path.insert(0, str(PLANNING_SCRIPTS))
from db import connect # noqa: E402
from muse_db import connect # noqa: E402
from persist_planning import write_section, confirm_section # noqa: E402
WORK_ID = 12

View File

@ -18,13 +18,11 @@ import sys
from pathlib import Path
SCRIPT_DIR = Path(__file__).resolve().parent
DB_SCRIPTS = SCRIPT_DIR.parents[1] / ".claude" / "skills" / "access-database" / "scripts"
PLANNING_SCRIPTS = SCRIPT_DIR.parents[1] / ".claude" / "skills" / "plan-story" / "scripts"
for p in (str(DB_SCRIPTS), str(PLANNING_SCRIPTS)):
if p not in sys.path:
sys.path.insert(0, p)
if str(PLANNING_SCRIPTS) not in sys.path:
sys.path.insert(0, str(PLANNING_SCRIPTS))
from db import connect # noqa: E402
from muse_db import connect # noqa: E402
from persist_planning import write_section, confirm_section # noqa: E402
WORK_ID = 12

View File

@ -35,9 +35,7 @@ SKILLS = AGENT_ROOT / ".claude" / "skills"
for sub in (
"assemble-context/scripts",
"write-next-chapter/scripts",
"execute-claude-task/scripts",
"record-run-evidence/scripts",
"access-database/scripts",
"check-content-consistency/scripts",
"decide-candidate/scripts",
"prevent-ai-flavor/scripts",
@ -47,7 +45,7 @@ for sub in (
if p not in sys.path:
sys.path.insert(0, p)
from db import connect, DSN # noqa: E402
from muse_db import connect, DSN # noqa: E402
from assemble_writer_context import assemble_context # noqa: E402
from prevent_ai_flavor import ( # noqa: E402
PreventionContractError, build_prevention_contract, persist_prevention, render_writer_constraints,

View File

@ -21,14 +21,13 @@ for sub in (
"assemble-context/scripts",
"write-next-chapter/scripts",
"record-run-evidence/scripts",
"access-database/scripts",
"prevent-ai-flavor/scripts",
):
p = str(SKILLS / sub)
if p not in sys.path:
sys.path.insert(0, p)
from db import connect, DSN # noqa: E402
from muse_db import connect, DSN # noqa: E402
from assemble_writer_context import assemble_context # noqa: E402
from prevent_ai_flavor import ( # noqa: E402
PreventionContractError, build_prevention_contract, persist_prevention, render_writer_constraints,

View File

@ -45,8 +45,6 @@ AGENT_ROOT = SCRIPT_DIR.parents[1]
SKILLS = AGENT_ROOT / ".claude" / "skills"
for sub in (
"write-next-chapter/scripts",
"execute-claude-task/scripts",
"access-database/scripts",
"decide-candidate/scripts",
"assemble-context/scripts",
):
@ -54,7 +52,7 @@ for sub in (
if p not in sys.path:
sys.path.insert(0, p)
from db import connect # noqa: E402
from muse_db import connect # noqa: E402
from claude_runtime import run_claude # noqa: E402
from run_writer import build_writer_execution_profile # noqa: E402
from writer_contract import han_count, normalize_text, validate_writer_draft # noqa: E402

View File

@ -154,6 +154,23 @@
"classification_confidence": "high",
"classification_basis": "Reads the research coverage YAML and checks capability owners, implementation paths, and status values."
},
{
"path": "tests/architecture/test_import_boundaries.py",
"scope": "domain",
"owner_skill_or_domain": "architecture",
"kind": "tool_contract",
"evidence_level": "static_structure",
"requires": [
"offline",
"filesystem"
],
"side_effects": [
"none"
],
"skill_behavior_eval": false,
"classification_confidence": "high",
"classification_basis": "Scans Skill and dashboard Python files for forbidden sys.path injections into shared runtime implementations (humanization/src, access-database/scripts, call-content-model/scripts, embed-knowledge/scripts, execute-claude-task/scripts, establish-voice-baseline/scripts) and Skill imports from the dashboard."
},
{
"path": "humanization/tests/test_humanization_v2.py",
"scope": "domain",

View File

@ -25,6 +25,8 @@ eval/ 判别试跑与回归探针:../.venv/bin/python eval/run_eval.py
config.yaml 模型角色分离配置(改写模型 ≠ 选择模型,代码强制)
```
Skill 通过已安装的 `muse-deai` 消费 `src/deai/`(`agent-example/requirements.txt` 的 `-e ./humanization`),不得再把 `humanization/src` 插入 `sys.path`。本目录的 `tests/`、`tools/`、`eval/` 仍可直接装载源码树。
## 与五个技能 skill 的对应
| 技能 | skill | 消费本目录什么 |

View File

@ -223,4 +223,72 @@ def run_voice_gate(original: str, candidate: str, ledger: dict | None) -> dict:
}
__all__ = ["profile_text", "draft_ledger", "run_voice_gate"]
class BaselineContractError(ValueError):
"""声音账合同失败:结构、来源或确认门未通过。"""
SCHEMA_VERSION = "voice-baseline-v1"
def validate_ledger(ledger: dict, *, work_ref: str) -> None:
"""校验声音账结构、作品绑定与角色归属冲突。"""
from .schemas import validate
if not isinstance(ledger, dict):
raise BaselineContractError("声音账必须是 JSON 对象")
try:
validate(ledger, "voice_baseline")
except ValueError as exc:
raise BaselineContractError(str(exc)) from exc
if ledger.get("schema_version") != SCHEMA_VERSION:
raise BaselineContractError(f"schema_version 必须是 {SCHEMA_VERSION}")
if ledger.get("work_ref") != work_ref:
raise BaselineContractError("声音账 work_ref 与目标作品不一致")
if ledger.get("status", "candidate") not in {"candidate", "canonical"}:
raise BaselineContractError("声音账 status 只能是 candidate/canonical")
narrator = ledger.get("narrator")
if not isinstance(narrator, dict):
raise BaselineContractError("narrator 必须是对象")
for key in ("sentence_habits", "punctuation_habits"):
if not isinstance(narrator.get(key, []), list):
raise BaselineContractError(f"narrator.{key} 必须是数组")
if "metrics" in narrator and not isinstance(narrator["metrics"], dict):
raise BaselineContractError("narrator.metrics 必须是对象")
if "exemplar_passages" in narrator and not isinstance(narrator["exemplar_passages"], list):
raise BaselineContractError("narrator.exemplar_passages 必须是数组")
for key, typ, what in (
("characters", dict, "角色声音档案"),
("untouchable_verbal_tics", dict, "不可改口癖"),
("protected_spans", list, "保护片段"),
("blacklist", list, "作品级黑名单"),
):
value = ledger.get(key)
if not isinstance(value, typ):
raise BaselineContractError(f"{what}({key})缺失或类型错误")
for key in ("passing_samples", "unknown_fields", "sources"):
if key in ledger and not isinstance(ledger[key], list):
raise BaselineContractError(f"{key} 必须是数组")
ownership: dict[str, str] = {}
for who, info in ledger["characters"].items():
if not isinstance(info, dict):
raise BaselineContractError(f"characters.{who} 必须是对象")
for key in ("verbal_tics", "sample_lines"):
values = info.get(key, [])
if not isinstance(values, list) or not all(isinstance(item, str) and item for item in values):
raise BaselineContractError(f"characters.{who}.{key} 必须是非空字符串数组")
for item in values:
previous = ownership.setdefault(item, who)
if previous != who:
raise BaselineContractError(f"声音样本「{item}」同时归属 {previous}/{who},须作者裁决")
for who, tics in ledger["untouchable_verbal_tics"].items():
if not isinstance(tics, list) or not all(isinstance(item, str) and item for item in tics):
raise BaselineContractError(f"{who} 的口癖必须是非空字符串数组")
for tic in tics:
previous = ownership.setdefault(tic, who)
if previous != who:
raise BaselineContractError(f"口癖「{tic}」同时归属 {previous}/{who},须作者裁决")
__all__ = ["profile_text", "draft_ledger", "run_voice_gate", "validate_ledger",
"BaselineContractError", "SCHEMA_VERSION"]

View File

@ -117,7 +117,41 @@ def load_rules_from_db(conn, *, samples: dict | None = None, cards: dict | None
return rules
def load_current_baseline(conn, work_ref: str, *, tenant_id: int = TENANT_ID) -> dict | None:
"""读取当前声音账并复核 ledger hash;多条 current 视为数据库状态损坏。
连接由调用方经 muse_db 传入,本函数不自己开连接。
"""
from .baseline import validate_ledger, BaselineContractError
rows = conn.execute(
"SELECT ledger,ledger_sha256,version FROM example_voice_baseline "
"WHERE tenant_id=%s AND work_ref=%s AND deleted=FALSE AND superseded=FALSE "
"ORDER BY version DESC",
(tenant_id, work_ref),
).fetchall()
if not rows:
return None
if len(rows) != 1:
raise BaselineContractError(f"作品 {work_ref} 存在 {len(rows)} 条 current 声音账")
ledger = dict(rows[0][0])
expected_hash = rows[0][1]
candidate_hashes = {
canonical_sha(ledger),
hashlib.sha256(
json.dumps(ledger, ensure_ascii=False, sort_keys=True).encode("utf-8")
).hexdigest(),
}
if expected_hash not in candidate_hashes:
raise BaselineContractError(f"作品 {work_ref} 当前声音账 hash 不一致")
ledger["database_version"] = rows[0][2]
ledger["database_ledger_sha256"] = expected_hash
ledger.setdefault("status", "canonical")
validate_ledger(ledger, work_ref=work_ref)
return ledger
__all__ = [
"LoadError", "TENANT_ID", "canonical_sha", "rule_row", "sample_row",
"load_samples_from_db", "load_rules_from_db",
"load_samples_from_db", "load_rules_from_db", "load_current_baseline",
]

View File

@ -14,7 +14,7 @@ ROOT = pathlib.Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / "src"))
from deai import evaluation, gates, load # noqa: E402
from deai.baseline import draft_ledger, profile_text, run_voice_gate # noqa: E402
from deai.baseline import BaselineContractError, draft_ledger, profile_text, run_voice_gate, validate_ledger # noqa: E402
from deai.carriers import carrier_at, carrier_ranges # noqa: E402
from deai.diagnose import run_deterministic_rules # noqa: E402
@ -58,10 +58,6 @@ class HumanizationV2Test(unittest.TestCase):
self.assertEqual(profile_text(text), profile_text(text))
def test_baseline_duplicate_voice_ownership_is_rejected_by_skill_validator(self):
establish = load_script(
"establish_v2",
ROOT.parent / ".claude/skills/establish-voice-baseline/scripts/establish_voice_baseline.py",
)
ledger = {
"schema_version": "voice-baseline-v1",
"work_ref": "synthetic:demo",
@ -73,8 +69,8 @@ class HumanizationV2Test(unittest.TestCase):
"untouchable_verbal_tics": {"甲": ["嗯"], "乙": ["嗯"]},
"protected_spans": [], "blacklist": [],
}
with self.assertRaisesRegex(establish.BaselineContractError, "同时归属"):
establish.validate_ledger(ledger, work_ref="synthetic:demo")
with self.assertRaisesRegex(BaselineContractError, "同时归属"):
validate_ledger(ledger, work_ref="synthetic:demo")
def test_carrier_scope_masks_dialogue_and_marks_carve_out(self):
samples = load.load_samples()

View File

@ -6,7 +6,6 @@ import sys
PROJECT_ROOT = pathlib.Path(__file__).resolve().parents[2]
sys.path.insert(0, str(PROJECT_ROOT / "humanization" / "src"))
sys.path.insert(0, str(PROJECT_ROOT / ".claude" / "skills" / "access-database" / "scripts"))
from deai import load, load_db # noqa: E402
@ -16,7 +15,7 @@ def main():
print("BLOCKED: set MUSE_REAL_PG_RULE_SMOKE=1 to run the real PostgreSQL rule smoke")
return 2
from db import connect # noqa: E402
from muse_db import connect # noqa: E402
file_samples = load.load_samples()
file_rules = load.load_rules(samples=file_samples)

View File

@ -19,7 +19,6 @@ TOOL_DIR = Path(__file__).resolve().parent
AGENT_ROOT = TOOL_DIR.parent.parent
for _p in (
AGENT_ROOT / "humanization" / "src",
AGENT_ROOT / ".claude" / "skills" / "access-database" / "scripts",
):
if str(_p) not in sys.path:
sys.path.insert(0, str(_p))
@ -200,7 +199,7 @@ def main(argv: list[str] | None = None) -> int:
}
print(json.dumps(plan, ensure_ascii=False))
return 0
from db import connect # noqa: E402
from muse_db import connect # noqa: E402
with connect() as conn:
with conn.transaction():

View File

@ -0,0 +1,14 @@
[project]
name = "muse-claude-runtime"
version = "0.1.0"
description = "受控 Claude CLI 调用运行时:冻结 profile、沙箱、联合回执"
requires-python = ">=3.10"
dependencies = []
[build-system]
requires = ["setuptools>=61"]
build-backend = "setuptools.build_meta"
[tool.setuptools]
package-dir = {"" = "src"}
py-modules = ["claude_runtime"]

View File

@ -853,10 +853,10 @@ def _default_persist_call(event: Mapping[str, Any]):
"""按需加载运行证据写入器,保持离线 fake runner 无数据库副作用。"""
import sys
# 本文件在 muse-claude-runtime/src,不再与 Skill scripts 同级;parents[2] 是 agent-example 根。
evidence_dir = (
pathlib.Path(__file__).resolve().parents[2]
/ "record-run-evidence"
/ "scripts"
/ ".claude" / "skills" / "record-run-evidence" / "scripts"
)
if str(evidence_dir) not in sys.path:
sys.path.insert(0, str(evidence_dir))

13
muse-db/pyproject.toml Normal file
View File

@ -0,0 +1,13 @@
[project]
name = "muse-db"
version = "0.1.0"
description = "muse-example 共享连接模块:锁死 DSN 与只读/可写会话入口"
requires-python = ">=3.10"
dependencies = ["psycopg[binary]"]
[build-system]
requires = ["setuptools>=61"]
build-backend = "setuptools.build_meta"
[tool.setuptools.packages.find]
where = ["src"]

View File

@ -0,0 +1,36 @@
# -*- coding: utf-8 -*-
"""muse-example 的共享连接模块。
Skill 脚本与只读看板都从这里取连接,不各自硬编码连接串:
连接事实与凭据来源是 db/连接信息.md(内网 Tailscale 段,凭据明文入仓为既定政策)。
DSN 锁死 muse-example,严禁触碰共享 PG 上的其他库(muse_local / muse_slice_live / *_test)。
keepalives 是既定口径:Tailscale 上长空转会被掐断,逐行插入两万次往返曾卡死 16 分钟。
"""
import psycopg
__version__ = "0.1.0"
DSN = (
"postgresql://root:f6710e2d0294eb1c10e26a805a64bc54@100.64.0.8:5433/muse-example"
"?keepalives=1&keepalives_idle=15&keepalives_interval=5&keepalives_count=3"
)
__all__ = ["DSN", "connect", "WINDOW_BUDGET_USD", "WINDOW_CALL_CAP"]
# 每窗 MiniMax 花费上限 / 全模型调用上限:muse_llm 治理与看板展示共用,禁止两处各写一份。
WINDOW_BUDGET_USD = 24.0
WINDOW_CALL_CAP = 6000
def connect(readonly: bool = False, **kwargs):
"""统一连接入口(即开即关,不持长事务)。
readonly=True 会话级锁死只读,写语句被 PostgreSQL 直接拒——query 命令与只读看板用它,
这是机械门,不靠调用方自觉。调用方已显式传 options 时不覆盖。
其余 psycopg 参数(row_factory 等)原样透传。
"""
if readonly:
kwargs.setdefault("options", "-c default_transaction_read_only=on")
return psycopg.connect(DSN, **kwargs)

14
muse-embed/pyproject.toml Normal file
View File

@ -0,0 +1,14 @@
[project]
name = "muse-embed"
version = "0.1.0"
description = "知识嵌入库:会话、向量请求与 draft 写入"
requires-python = ">=3.10"
dependencies = ["requests", "click", "psycopg[binary]", "muse-db"]
[build-system]
requires = ["setuptools>=61"]
build-backend = "setuptools.build_meta"
[tool.setuptools]
package-dir = {"" = "src"}
py-modules = ["muse_embed"]

View File

@ -0,0 +1,420 @@
#!/usr/bin/env python3
"""muse_embed:知识行批量嵌入库(New-API / Qwen3-Embedding-8B / 1024 维)。
合同见同 skill SKILL.md;通道事实见 db/连接信息.md。失败原样报错不静默。
"""
import hashlib
import json
import time
import click
import requests
BASE = "http://100.64.0.8:3000"
TOKEN = "sk-DyVqO3lDmEvQZ3PqGpbNaaaHZHhbh0xaHRIiynhYSmVlLHl2" # MUSE_AI_NEW_API_TOKEN(勿用管理令牌)
MODEL = "Qwen/Qwen3-Embedding-8B"
DIM = 1024
TENANT, ACTOR = 1, "1"
BATCH = 16
def _session():
"""禁系统代理的会话(系统代理会假 502)。"""
s = requests.Session()
s.trust_env = False
s.headers["Authorization"] = f"Bearer {TOKEN}"
return s
def embed_texts(sess, texts):
"""调 New-API /v1/embeddings;整批重试 2 次后逐条降级。返回 (向量列表, 失败索引集)。"""
def call(batch):
r = sess.post(f"{BASE}/v1/embeddings", json={
"model": MODEL, "input": batch, "dimensions": DIM}, timeout=120)
r.raise_for_status()
data = r.json()["data"]
# 响应 index 是请求槽位,不能排序后压缩;缺项、重复或越界都必须让本次调用失败并进入重试。
vectors = [None] * len(batch)
seen = set()
for item in data:
index = item["index"]
if type(index) is not int or not 0 <= index < len(batch):
raise ValueError(f"embedding 响应 index 越界或非整数:{index!r}")
if index in seen:
raise ValueError(f"embedding 响应 index 重复:{index}")
vectors[index] = item["embedding"]
seen.add(index)
if len(seen) != len(batch):
missing = sorted(set(range(len(batch))) - seen)
raise ValueError(f"embedding 响应缺少 index:{missing}")
return vectors
for attempt in range(3):
try:
return call(texts), set()
except Exception:
if attempt < 2:
time.sleep(2 ** attempt)
continue
# 整批三败 → 逐条降级,坏行记错不断批
vecs, bad = [], set()
for i, t in enumerate(texts):
try:
vecs.append(call([t])[0])
except Exception as ee:
vecs.append(None)
bad.add(i)
click.echo(f" [失败] 第{i}条: {ee}", err=True)
return vecs, bad
def build_embed_text(payload: dict) -> str:
"""嵌入文本构造:payload 自带 embed_text 优先;否则固定拼接(与检索端语义对齐)。"""
if payload.get("embed_text"):
return payload["embed_text"]
# 型取值补 type 键:升格卡 payload 用 type 存型(非 型/target_type),漏认会产出「【】名称…」丢型文本,
# 令升格卡向量与检索端跨型语义错位;补一段式回退(additive,不动 型/target_type 既有行为)。
t = payload.get("型") or payload.get("type") or payload.get("target_type", "")
name = payload.get("名称") or payload.get("name", "")
brief = payload.get("一句话摘要") or payload.get("brief", "")
fields = payload.get("字段") or payload.get("fields") or {}
body = "\n".join(f"{k}:{v}" for k, v in fields.items() if v and k not in ("名称", "一句话摘要"))
return f"【{t}】{name}:{brief}\n{body}"[:4000]
def _content_hash(text):
"""统一生成向量幂等键,候选筛选与写前复验必须共用同一规则。"""
return hashlib.sha256(f"{text}|{MODEL}".encode()).hexdigest()
class EmbeddingOwnershipConflict(RuntimeError):
"""同 hash 唯一行已归实体或其他活跃 draft,禁止迁移 owner。"""
def _embedding_owner_action(conn, draft_id, content_hash, *, lock=False):
"""判断同 hash 唯一行应幂等跳过还是写入;写段可锁行封住预查后的竞态。"""
lock_clause = " FOR UPDATE OF e" if lock else ""
owner = conn.execute(
"""SELECT e.draft_id, e.entity_id, e.deleted,
COALESCE(d.deleted, TRUE), d.tenant_id
FROM example_knowledge_embedding e
LEFT JOIN muse_knowledge_draft d ON d.id=e.draft_id
WHERE e.tenant_id=%s AND e.content_hash=%s AND e.model=%s""" + lock_clause,
(TENANT, content_hash, MODEL),
).fetchone()
if not owner:
return "write"
owner_draft_id, owner_entity_id, embedding_deleted, owner_deleted, owner_tenant = owner
# entity owner 是确认后的正式归属,任何 draft 都不得把它降级抢回。
if owner_entity_id is not None:
raise EmbeddingOwnershipConflict(
f"同 hash 唯一行已归 entity:hash={content_hash},entity={owner_entity_id},"
f"candidate={draft_id}"
)
# 只有当前租户、当前 draft、两侧都 active 才是真正的幂等命中。
if owner_draft_id == draft_id:
if owner_tenant != TENANT:
raise EmbeddingOwnershipConflict(
f"同 hash 当前 owner 租户不匹配:hash={content_hash},"
f"owner_tenant={owner_tenant},candidate_tenant={TENANT}"
)
if not embedding_deleted and not owner_deleted:
return "skip"
return "write"
# 空 owner、owner 行缺失或 owner draft 已软删时,可由当前活跃 draft 接管唯一行。
if owner_draft_id is None or owner_deleted:
return "write"
raise EmbeddingOwnershipConflict(
f"同 hash 唯一行已归其他 active draft:hash={content_hash},"
f"owner={owner_draft_id},candidate={draft_id}"
)
def _write_embedding(conn, draft_id, content_hash, text, vector):
"""在调用方单 draft 事务内锁定活性与 owner,条件写入并校验最终归属。"""
# 写事务先按固定表顺序取得 ROW EXCLUSIVE 锁,避免与 reset 的多表锁形成交叉等待。
conn.execute(
"LOCK TABLE muse_knowledge_draft, example_knowledge_embedding IN ROW EXCLUSIVE MODE"
)
# 取得表锁后再锁 candidate draft:embed 先到时 reset 的七表 SHARE ROW EXCLUSIVE 会等待;
# reset 先到时本查询等待其提交,随后读取 deleted=TRUE 并拒绝陈旧写入。
candidate = conn.execute(
"""SELECT tenant_id, deleted, status, draft_payload FROM muse_knowledge_draft
WHERE id=%s FOR UPDATE""",
(draft_id,),
).fetchone()
if not candidate:
click.echo(f" [跳过] draft={draft_id} 写前已不存在,未写向量", err=True)
return False
candidate_tenant, candidate_deleted, candidate_status, current_payload = candidate
if candidate_tenant != TENANT:
raise EmbeddingOwnershipConflict(
f"draft 租户不匹配:draft={draft_id},tenant={candidate_tenant},expected={TENANT}"
)
if candidate_deleted:
click.echo(f" [跳过] draft={draft_id} 写前已软删,未写向量", err=True)
return False
if candidate_status != "pending":
click.echo(
f" [跳过] draft={draft_id} 写前 status={candidate_status},非 pending,未写向量",
err=True,
)
return False
# HTTP 期间 payload 可能被 parse/confirm 更新;锁内必须按当前 payload 重构文本与 hash,
# 只要与 HTTP 请求所依据的快照不同,就丢弃陈旧向量,绝不覆盖并发产生的新结果。
current_text = build_embed_text(current_payload or {})
current_hash = _content_hash(current_text)
if current_text != text or current_hash != content_hash:
click.echo(
f" [跳过] draft={draft_id} 写前 payload/hash 漂移,"
f"expected_hash={content_hash} current_hash={current_hash},未写向量",
err=True,
)
return False
# 锁定该 draft 的全部活向量,保证 entity 归属和“每 draft 唯一活向量”在同一事务内判定。
live_embeddings = conn.execute(
"""SELECT id, content_hash, model, entity_id FROM example_knowledge_embedding
WHERE tenant_id=%s AND draft_id=%s AND deleted=FALSE
FOR UPDATE""",
(TENANT, draft_id),
).fetchall()
if len(live_embeddings) > 1:
raise EmbeddingOwnershipConflict(
f"draft={draft_id} 存在多条活向量,状态异常,禁止自动修复:{live_embeddings}"
)
entity_rows = [
(row_id, row_hash, row_model, entity_id)
for row_id, row_hash, row_model, entity_id in live_embeddings
if entity_id is not None
]
if entity_rows:
raise EmbeddingOwnershipConflict(
f"draft={draft_id} 存在 entity_id 非空旧活向量,禁止覆盖:{entity_rows}"
)
if any(
row_hash == content_hash and row_model == MODEL
for _, row_hash, row_model, _ in live_embeddings):
click.echo(f" [跳过] draft={draft_id} 同 hash 活向量已由当前 draft 持有")
return False
action = _embedding_owner_action(conn, draft_id, content_hash, lock=True)
if action == "skip":
click.echo(f" [跳过] draft={draft_id} 同 hash 活向量已由当前 draft 持有")
return False
if live_embeddings:
# 当前 payload 已通过锁内 hash 重验,因此其余 hash 均为该 draft 的过期向量;
# 只允许软删 draft owner,entity owner 已在上方失败关闭。
conn.execute(
"""UPDATE example_knowledge_embedding SET deleted=TRUE, updater=%s
WHERE tenant_id=%s AND draft_id=%s AND deleted=FALSE
AND entity_id IS NULL AND (content_hash!=%s OR model!=%s)""",
(ACTOR, TENANT, draft_id, content_hash, MODEL),
)
# 条件 UPSERT 是行锁检查后的第二道防线:当预查时唯一行尚不存在、随后被并发插入时,
# 仅允许当前 owner 或已失活 owner 迁移;entity/其他 active draft 均令 RETURNING 为空。
upserted = conn.execute(
"""INSERT INTO example_knowledge_embedding
(draft_id, content_hash, embed_text, model, dimensions, embedding,
creator, updater, tenant_id)
VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s)
ON CONFLICT (tenant_id, content_hash, model)
DO UPDATE SET draft_id=EXCLUDED.draft_id,
embed_text=EXCLUDED.embed_text,
model=EXCLUDED.model,
dimensions=EXCLUDED.dimensions,
embedding=EXCLUDED.embedding,
deleted=FALSE,
updater=EXCLUDED.updater
WHERE example_knowledge_embedding.entity_id IS NULL
AND (example_knowledge_embedding.draft_id=EXCLUDED.draft_id
OR NOT EXISTS (
SELECT 1 FROM muse_knowledge_draft owner
WHERE owner.id=example_knowledge_embedding.draft_id
AND owner.deleted=FALSE))
RETURNING draft_id""",
(draft_id, content_hash, text, MODEL, DIM, json.dumps(vector),
ACTOR, ACTOR, TENANT),
).fetchone()
if not upserted or upserted[0] != draft_id:
raise EmbeddingOwnershipConflict(
f"同 hash 唯一行未绑定当前 draft:hash={content_hash},candidate={draft_id}"
)
return True
def _load_bulk_candidates(conn, work_id, limit, source_type=None):
"""读取 pending draft 的全部活向量,在 Python 中按当前文本和模型筛选补嵌候选。
拆书草稿的 ``work_id`` 仍表示参考书,历史调用因此按 ``source_id`` 筛选。
章后抽卡直接把作品写入 draft.work_id,必须用显式 source_type 切换到该口径,
避免同一个 CLI 参数在两类数据上产生歧义。
"""
sql = """SELECT d.id, d.draft_payload,
e.id, e.content_hash, e.model, e.entity_id
FROM muse_knowledge_draft d
LEFT JOIN example_knowledge_embedding e
ON e.tenant_id=%s AND e.draft_id=d.id AND e.deleted=FALSE
WHERE d.tenant_id=%s AND d.deleted=FALSE AND d.status='pending'"""
args = [TENANT, TENANT]
if source_type == "chapter_extract":
if work_id is None:
raise ValueError("source_type=chapter_extract 必须同时指定 --work-id")
sql += " AND d.work_id=%s AND d.source_type=%s"
args.extend([work_id, source_type])
elif work_id is not None:
sql += " AND d.source_id=%s"
args.append(work_id)
# 必须先取得每个 draft 的全部活向量,不能在 SQL 层 LIMIT 后漏掉旧 hash 或异常状态。
sql += " ORDER BY d.id, e.id"
rows = conn.execute(sql, args).fetchall()
grouped = {}
for draft_id, payload, embedding_id, row_hash, row_model, entity_id in rows:
draft = grouped.setdefault(draft_id, {"payload": payload, "embeddings": []})
if embedding_id is not None:
draft["embeddings"].append((embedding_id, row_hash, row_model, entity_id))
candidates = []
failures_by_draft = {}
repair_targets = {}
for draft_id in sorted(grouped):
draft = grouped[draft_id]
text = build_embed_text(draft["payload"] or {})
content_hash = _content_hash(text)
live_embeddings = draft["embeddings"]
if len(live_embeddings) == 1:
_, row_hash, row_model, entity_id = live_embeddings[0]
if entity_id is None and row_hash == content_hash and row_model == MODEL:
continue
# 所有非健康目标都参与同批 hash 冲突检查,不能因其中一条先被判异常而放行另一条。
repair_targets.setdefault(content_hash, []).append(draft_id)
if len(live_embeddings) > 1:
failures_by_draft.setdefault(draft_id, []).append(
f"存在多条活向量,状态异常,禁止自动修复:{live_embeddings}"
)
continue
if live_embeddings:
_, row_hash, row_model, entity_id = live_embeddings[0]
if entity_id is not None:
failures_by_draft.setdefault(draft_id, []).append(
f"活向量已归 entity={entity_id},禁止 draft 补嵌迁移 owner"
)
continue
candidates.append((draft_id, content_hash, text))
# 相同目标 hash 的多个 draft 不能靠执行顺序决定 owner;冲突检查必须发生在 limit 之前。
conflicted_drafts = set()
for content_hash, draft_ids in repair_targets.items():
if len(draft_ids) < 2:
continue
reason = (
f"同批目标 hash 冲突:hash={content_hash},drafts={draft_ids},"
"禁止按执行顺序抢 owner"
)
for draft_id in draft_ids:
failures_by_draft.setdefault(draft_id, []).append(reason)
conflicted_drafts.add(draft_id)
candidates = [candidate for candidate in candidates if candidate[0] not in conflicted_drafts]
# limit 只能限制后续 HTTP/写入;先对完整候选集预查目标 hash owner,避免范围外冲突被隐藏。
prechecked_candidates = []
for draft_id, content_hash, text in candidates:
try:
action = _embedding_owner_action(conn, draft_id, content_hash)
except EmbeddingOwnershipConflict as exc:
failures_by_draft.setdefault(draft_id, []).append(str(exc))
continue
if action != "skip":
prechecked_candidates.append((draft_id, content_hash, text))
# 全量只读预检完成后释放事务,再截取实际处理行;每个 chunk 仍会再次预查以封住其后竞态。
conn.commit()
if limit and limit > 0:
prechecked_candidates = prechecked_candidates[:int(limit)]
failures = [
(draft_id, ";".join(reasons))
for draft_id, reasons in sorted(failures_by_draft.items())
]
return prechecked_candidates, failures
def _run_bulk(conn, sess, work_id, limit, source_type=None):
"""执行一次 bulk 补嵌;HTTP 前后均保持既有 owner、锁和 stale-write 边界。"""
rows, read_failures = _load_bulk_candidates(conn, work_id, limit, source_type)
for draft_id, reason in read_failures:
click.echo(f" [失败] draft={draft_id}: {reason}", err=True)
if read_failures:
details = ";".join(
f"draft={draft_id}: {reason}" for draft_id, reason in read_failures
)
raise EmbeddingOwnershipConflict(f"bulk 候选存在确定性冲突,已失败关闭:{details}")
done = skip = fail = 0
click.echo(f"待补嵌草稿: {len(rows)} 条(筛选失败 {len(read_failures)} 条)")
for i in range(0, len(rows), BATCH):
chunk = rows[i:i + BATCH]
metas = []
for draft_id, content_hash, text in chunk:
action = _embedding_owner_action(conn, draft_id, content_hash)
if action == "skip":
skip += 1
click.echo(f" [跳过] draft={draft_id} 同 hash 活向量已由当前 draft 持有")
continue
metas.append((draft_id, content_hash, text))
# owner 预查只用于避免无效 HTTP;HTTP 期间不持数据库事务或表锁。
conn.commit()
if not metas:
continue
texts = [meta[2] for meta in metas]
try:
vecs, bad = embed_texts(sess, texts)
except Exception as exc:
for draft_id, _, _ in metas:
fail += 1
click.echo(f" [失败] draft={draft_id}: HTTP 嵌入失败:{exc}", err=True)
continue
bad = set(bad or ())
for j, (draft_id, content_hash, text) in enumerate(metas):
if j in bad:
fail += 1
click.echo(f" [失败] draft={draft_id}: HTTP 返回 bad,保留旧向量", err=True)
continue
try:
vector = vecs[j]
except (IndexError, TypeError):
fail += 1
click.echo(f" [失败] draft={draft_id}: HTTP 返回向量缺项,保留旧向量", err=True)
continue
if vector is None:
fail += 1
click.echo(f" [失败] draft={draft_id}: HTTP 返回空向量,保留旧向量", err=True)
continue
# 每个 draft 独立事务:确定性冲突回滚当前事务并向上抛,使命令以非零状态退出。
with conn.transaction():
written = _write_embedding(conn, draft_id, content_hash, text, vector)
if written:
done += 1
else:
skip += 1
click.echo(
f" 进度 {min(i + BATCH, len(rows))}/{len(rows)}"
f"(新嵌{done} 跳过{skip} 失败{fail})"
)
click.echo(f"完成:新嵌 {done}、跳过 {skip}、失败 {fail}")
return {"done": done, "skip": skip, "fail": fail}

14
muse-llm/pyproject.toml Normal file
View File

@ -0,0 +1,14 @@
[project]
name = "muse-llm"
version = "0.1.0"
description = "New-API 内容调用库:额度窗、降级链、JSON 容错"
requires-python = ">=3.10"
dependencies = ["requests", "json-repair", "muse-db"]
[build-system]
requires = ["setuptools>=61"]
build-backend = "setuptools.build_meta"
[tool.setuptools]
package-dir = {"" = "src"}
py-modules = ["muse_llm"]

382
muse-llm/src/muse_llm.py Normal file
View File

@ -0,0 +1,382 @@
#!/usr/bin/env python3
"""muse_llm:New-API 统一调用库(默认 MiniMax-M3)。
管线内所有内容生产型 LLM 调用(清洗探测/拆书抽取)必须经此入口:
- trust_env=False(本机代理环境变量会劫持内网直连,教训固化);
- 超时 + 指数退避重试;<think> 剥离;JSON 三级容错提取(json-repair 兜底);
- 每次调用向 stderr 打印 token 用量与耗时(成本审计),stdout 只出内容。
"""
import json
import pathlib
import re
import sys
import time
from muse_db import WINDOW_BUDGET_USD, WINDOW_CALL_CAP, connect
import requests
BASE = "http://100.64.0.8:3000"
# New-API 普通令牌(仓库政策允许明文;严禁换管理令牌打 /v1)
TOKEN = "sk-DyVqO3lDmEvQZ3PqGpbNaaaHZHhbh0xaHRIiynhYSmVlLHl2"
DEFAULT_MODEL = "MiniMax-M3"
# ── 额度治理常量(B: 把散在各调用方的降级链上收到 chat_governed 统一治理)──
# 额度账本走 muse_db 短连接(keepalives 已在共享 DSN 内)
MINIMAX_MODELS = {"MiniMax-M3", "MiniMax-M2.7"} # 计入每窗预算的模型
BUDGET_CHAIN = ["MiniMax-M3", "MiniMax-M2.7", "glm-5.2", "deepseek-v4-flash"] # 全局统一降级链
# WINDOW_BUDGET_USD / WINDOW_CALL_CAP 来自 muse_db,看板与本库共用
# 上游实测上限:请求前主动裁剪,避免依赖不同渠道含混甚至错误的 HTTP 400 文案再猜测重发。
# M3 / deepseek 未观察到该限制,故不在表内、不主动裁剪。
MODEL_MAX_TOKENS = {
"MiniMax-M2.7": 196608,
"glm-5.2": 12000,
}
# 费率兜底(model_ratio, completion_ratio, cache_ratio),与 New-API /api/pricing 一致(2026-07-16 快照)
PRICING_FALLBACK = {
"MiniMax-M3": (0.15, 4.0, 0.2),
"MiniMax-M2.7": (0.15, 4.0, 0.2),
"glm-5.2": (0.5634, 3.5, 0.25),
"deepseek-v4-flash": (0.07, 2.0, 0.071428571429),
}
class SensitiveError(Exception):
"""上游内容安全拦截(响应体含 sensitive,如 new_sensitive 1026)。
同模型退避重试必再触发(放量实测每敏感章空烧 3 次),故不在此退避,
立即抛给上层走模型降级链(创始人 2026-07-14:M3→MiniMax-M2.7→deepseek-v4-flash)。"""
class PlanQuotaExhausted(Exception):
"""上游模型渠道的 Token Plan 已耗尽。
该错误在同一额度窗内重试不会恢复,必须立即交给治理层熔断当前模型;它与普通限流 429
不同,普通 429 仍保留指数退避重试。"""
def _default_persist_call(event):
"""按需加载运行证据持久化器,避免离线调用被迫连库。"""
# 本文件在 muse-llm/src,不再与 Skill scripts 同级;parents[2] 是 agent-example 根。
evidence_scripts = (
pathlib.Path(__file__).resolve().parents[2]
/ ".claude" / "skills" / "record-run-evidence" / "scripts"
)
if str(evidence_scripts) not in sys.path:
sys.path.insert(0, str(evidence_scripts))
from persist_llm_call import persist_call
return persist_call(event)
def chat(prompt, model=DEFAULT_MODEL, max_tokens=512000, temperature=0.2,
retries=2, timeout=900, system=None, top_p=None, *, run_id=None,
caller=None, requested_model_id=None, persist_call=None):
"""单轮对话,返回 (content, usage)。网络错/5xx/普通 429 指数退避重试。
content 已剥离 <think>…</think>(推理模型可能把思考混进正文)。
system:身份段与任务材料分离(角色遵从更稳、身份段利于上游缓存)。
top_p:随 temperature 分化实验用(M 家族官方推荐 1.0/0.95,eval A/B 后定版)。
max_tokens 默认 512000;仅对有实测硬上限的 M2.7/GLM 请求前主动裁剪。
预扣费机制备忘:New-API 按 max_tokens 预扣(512k 预扣 $0.15375/次,网关已验证接受该值;
结算按实际用量,余额充足时预扣不产生额外成本)——**余额须 ≥ 并发路数 × $0.154**,
否则触发 403「预扣费额度失败」(2026-07-15 余额见底实测坐实此机制)。
"""
if persist_call is None and (run_id or caller):
persist_call = _default_persist_call
if persist_call is not None and not callable(persist_call):
raise TypeError("persist_call 必须是可调用对象")
s = requests.Session()
s.trust_env = False # 本机代理 env 会劫持内网直连
messages = ([{"role": "system", "content": system}] if system else []) \
+ [{"role": "user", "content": prompt}]
model_cap = MODEL_MAX_TOKENS.get(model)
effective_max_tokens = min(max_tokens, model_cap) if model_cap is not None else max_tokens
if effective_max_tokens != max_tokens:
print(f"[llm] {model} max_tokens={max_tokens} 主动裁为模型上限 {effective_max_tokens}",
file=sys.stderr)
payload = {
"model": model,
"messages": messages,
"max_tokens": effective_max_tokens,
"temperature": temperature,
}
if top_p is not None:
payload["top_p"] = top_p
prompt_raw = json.dumps({"messages": messages, **payload},
ensure_ascii=False, sort_keys=True, separators=(",", ":"))
requested_model_id = requested_model_id or model
last_err = None
for attempt in range(retries + 1):
try:
t0 = time.time()
r = s.post(f"{BASE}/v1/chat/completions",
headers={"Authorization": f"Bearer {TOKEN}"},
json=payload, timeout=timeout)
# Token Plan 耗尽不是瞬时限流:同模型退避只会白等 8/16 秒,立即交治理层按窗熔断。
if r.status_code == 429 and "Token Plan 用量上限" in r.text:
raise PlanQuotaExhausted(f"Token Plan 已耗尽 HTTP 429: {r.text[:200]}")
# 内容安全拦截:同模型退避重试必再敏感,立即抛 SensitiveError 交上层
# 降级换模型,不在此浪费退避(否则一敏感章空烧 3 次,实测占放量请求 23%)
if r.status_code >= 500 and "sensitive" in r.text.lower():
raise SensitiveError(f"内容安全拦截 HTTP {r.status_code}: {r.text[:150]}")
# 429/5xx 属于可重试的服务端瞬时问题
if r.status_code in (429,) or r.status_code >= 500:
last_err = f"HTTP {r.status_code}: {r.text[:200]}"
raise requests.RequestException(last_err)
r.raise_for_status()
data = r.json()
content = data["choices"][0]["message"]["content"] or ""
content = re.sub(r"<think>.*?</think>", "", content, flags=re.S).strip()
usage = data.get("usage", {})
# 缓存命中数(OpenAI 式 prompt_tokens_details.cached_tokens)——验证前缀缓存是否生效、省了多少
cached = (usage.get("prompt_tokens_details") or {}).get("cached_tokens", 0)
print(f"[llm] {model} in={usage.get('prompt_tokens', '?')} "
f"cached={cached} out={usage.get('completion_tokens', '?')} "
f"耗时{time.time() - t0:.0f}s finish={data['choices'][0].get('finish_reason')}",
file=sys.stderr)
if persist_call is not None:
persist_call({
"window_key": window_key(_now()),
"run_id": run_id,
"caller": caller or "",
"requested_model_id": requested_model_id,
"actual_model_id": model,
"usage": usage,
"cost_usd": cost_usd(model, usage),
"stop_reason": data["choices"][0].get("finish_reason"),
"duration_ms": max(0, int(round((time.time() - t0) * 1000))),
"prompt": prompt_raw,
"response": json.dumps(data, ensure_ascii=False, sort_keys=True,
separators=(",", ":"), default=str),
"role": caller,
})
return content, usage
except (requests.RequestException, KeyError, json.JSONDecodeError) as e:
last_err = str(e)
if attempt < retries:
wait = 8 * (2 ** attempt)
print(f"[llm] 第{attempt + 1}次失败({last_err[:120]}),{wait}s 后重试",
file=sys.stderr)
time.sleep(wait)
raise RuntimeError(f"LLM 调用重试耗尽: {last_err}")
def extract_json(text):
"""JSON 三级容错提取:直接解析 → 首尾括号截取 → json-repair 兜底。
opus 试拆实测过两类 JSON 病(中文引号、缺逗号)——任何模型都可能犯,统一在此兜住。
"""
try:
return json.loads(text)
except json.JSONDecodeError:
pass
# 剥 markdown 代码围栏后按最外层大括号/中括号截取
t = re.sub(r"^```(?:json)?\s*|\s*```$", "", text.strip(), flags=re.M)
for a, b in (("{", "}"), ("[", "]")):
i, j = t.find(a), t.rfind(b)
if i != -1 and j > i:
frag = t[i:j + 1]
try:
return json.loads(frag)
except json.JSONDecodeError:
import json_repair
return json_repair.loads(frag)
import json_repair
return json_repair.loads(t)
# ══ 额度治理层(chat_governed)══
# WHY 上收:此前每个调用方各写一套模型降级链(parse_llm/parse_outline/review_cards 各一份,
# 链名/顺序还不一致),既无法全局限预算、也无法跨进程共享"这一窗烧了多少/调了多少次"。
# 统一到 chat_governed 后:一本共享账本按 5 小时窗计钱计次,MiniMax 超 $24/窗自动切非 MiniMax 链,
# 全窗调用达 6000 次自动睡到下一窗续跑——降级策略只此一处,调用方只管拿结果。
_PRICING_CACHE = None
# 仅保存当前额度窗内已确认 Token Plan 耗尽的模型。进程重启会自然重探;跨窗也会清空重探。
_PLAN_QUOTA_OPEN = {}
def _plan_quota_open_models(wk):
"""返回当前窗已熔断模型集合,并清除其他窗口的陈旧状态。"""
stale = [key for key in _PLAN_QUOTA_OPEN if key != wk]
for key in stale:
del _PLAN_QUOTA_OPEN[key]
return _PLAN_QUOTA_OPEN.setdefault(wk, set())
def get_pricing():
"""返回 {model: (model_ratio, completion_ratio, cache_ratio)}。
进程内只拉一次 /api/pricing;拉取失败或字段异常时回退硬编码,绝不因定价接口抖动崩管线。"""
global _PRICING_CACHE
if _PRICING_CACHE is not None:
return _PRICING_CACHE
# WHY 先复制兜底再逐字段覆盖:定价接口只是"锦上添花",任何一环出问题都必须能退回硬编码,
# 让成本核算继续跑;只认能解析成正数的字段,脏数据/0/负数一律不覆盖(算废预算比抖动更危险)。
merged = {m: list(r) for m, r in PRICING_FALLBACK.items()}
try:
s = requests.Session()
s.trust_env = False # 与 chat 同源:本机代理 env 会劫持内网直连
r = s.get(f"{BASE}/api/pricing", timeout=10)
r.raise_for_status()
by_name = {row.get("model_name"): row
for row in (r.json().get("data") or []) if isinstance(row, dict)}
for m in merged:
row = by_name.get(m)
if not row:
continue
for idx, key in enumerate(("model_ratio", "completion_ratio", "cache_ratio")):
try:
v = float(row.get(key))
except (TypeError, ValueError):
continue # 字段缺失/非数:保留兜底值
if v > 0:
merged[m][idx] = v
except Exception as e:
# 网络/HTTP/JSON 任何异常:整体回退硬编码(不吃半拉子覆盖的脏账)
print(f"[llm] /api/pricing 拉取失败({type(e).__name__}),用兜底费率", file=sys.stderr)
_PRICING_CACHE = {m: tuple(r) for m, r in PRICING_FALLBACK.items()}
return _PRICING_CACHE
_PRICING_CACHE = {m: tuple(r) for m, r in merged.items()}
return _PRICING_CACHE
def cost_usd(model, usage):
"""按 New-API 口径算单次调用美元成本($1 = 500000 配额单位)。
cost = model_ratio × ((prompt-cached) + cached×cache_ratio + completion×completion_ratio) / 500000"""
pricing = get_pricing()
if model in pricing:
model_ratio, completion_ratio, cache_ratio = pricing[model]
else:
# 未知模型宁高估勿漏计(漏计会让预算穿底),用 M3 费率兜底并告警
model_ratio, completion_ratio, cache_ratio = pricing["MiniMax-M3"]
print(f"[llm] cost_usd 未知模型 {model},用 MiniMax-M3 费率兜底计价", file=sys.stderr)
prompt = usage.get("prompt_tokens", 0) or 0
completion = usage.get("completion_tokens", 0) or 0
cached = (usage.get("prompt_tokens_details") or {}).get("cached_tokens", 0) or 0
billable = (prompt - cached) + cached * cache_ratio + completion * completion_ratio
return model_ratio * billable / 500000
def _now():
from datetime import datetime
return datetime.now() # 单独封装便于单测打桩
def window_key(dt):
"""把时刻归到所属窗口边界键。窗口起点 0/5/10/15/20 点,末窗 20-24=4h。"""
wh = (dt.hour // 5) * 5 # 0..4→0,5..9→5,10..14→10,15..19→15,20..23→20
return f"{dt:%Y-%m-%d}T{wh:02d}"
def seconds_to_next_window(dt):
"""距下一窗边界的秒数(<5→05:00,<10→10:00,<15→15:00,<20→20:00,否则次日00:00)。"""
from datetime import timedelta
h = dt.hour
if h < 5:
boundary = dt.replace(hour=5, minute=0, second=0, microsecond=0)
elif h < 10:
boundary = dt.replace(hour=10, minute=0, second=0, microsecond=0)
elif h < 15:
boundary = dt.replace(hour=15, minute=0, second=0, microsecond=0)
elif h < 20:
boundary = dt.replace(hour=20, minute=0, second=0, microsecond=0)
else:
boundary = (dt + timedelta(days=1)).replace(hour=0, minute=0, second=0, microsecond=0)
# 至少 1 秒:边界精确命中时避免 0/负导致空睡后原地打转
return max(1, int((boundary - dt).total_seconds()))
def _read_window(wk):
"""读某窗账本,返回 (minimax_usd:float, total_calls:int);无行返回 (0.0,0)。短连接即关。"""
# WHY 短连接:LLM/sleep 期间绝不持 DB 连接(Tailscale 长事务空转会被掐断),读完立刻释放
with connect() as c:
row = c.execute(
"SELECT minimax_usd, total_calls FROM example_llm_quota WHERE window_key=%s",
(wk,)).fetchone()
if not row:
return 0.0, 0
return float(row[0]), int(row[1])
def _bump_window(wk, add_usd):
"""原子累加:该窗 minimax_usd += add_usd、total_calls += 1,返回累加后的 (usd,calls)。
单语句 upsert,多分片共用一本账靠 PG 行锁串行化。短连接即关。"""
with connect() as c:
row = c.execute(
"""INSERT INTO example_llm_quota (window_key, minimax_usd, total_calls, updated_at)
VALUES (%s, %s, 1, now())
ON CONFLICT (window_key) DO UPDATE
SET minimax_usd = example_llm_quota.minimax_usd + EXCLUDED.minimax_usd,
total_calls = example_llm_quota.total_calls + 1, updated_at = now()
RETURNING minimax_usd, total_calls""",
(wk, add_usd)).fetchone()
return float(row[0]), int(row[1]) # psycopg 返回 Decimal,转 float
def chat_governed(prompt, model=DEFAULT_MODEL, system=None, max_tokens=512000,
temperature=0.2, top_p=None, *, run_id=None, caller=None,
persist_call=None):
"""全局额度治理下的对话入口,返回 (content, usage, actual_model)。
契约:成功→三元组;全链耗尽(所有模型敏感/不可用)→(None,None,None)。
model 参数仅作兼容保留:实际用哪个模型由全局额度策略决定,不由调用方指定。
策略(每次调用前):
1) 读本窗账本;本窗 total_calls ≥ WINDOW_CALL_CAP → 打日志、睡到下一窗边界(不持DB连接)、重读续跑;
2) 本窗 minimax_usd ≥ WINDOW_BUDGET_USD → 降级链去掉 MiniMax 前缀(只剩 glm-5.2→deepseek),否则用全链;
3) 跳过本窗已确认 Token Plan 耗尽的模型;其余模型沿链调用,敏感/不可用时换下一个;成功即止;
4) 成功后 _bump_window(本窗, MiniMax模型才计成本否则0),返回三元组;全链失败返回 (None,None,None)。"""
from datetime import timedelta
while True:
wk = window_key(_now())
usd, calls = _read_window(wk) # 短连接读完即释放,下面 LLM/sleep 阶段不持连接
# 1) 调用数达上限:睡到下一窗边界再重来(睡眠期间不持任何 DB 连接)
if calls >= WINDOW_CALL_CAP:
now2 = _now()
secs = seconds_to_next_window(now2)
# 目标窗边界:secs 经 int() 截断可能落在边界前 <1s(如 04:59:59),+1s 归整到整分,
# 否则 %H 会把 04:59:59 显示成上一整点"04"、误导成非法边界(窗边界只有 00/05/10/15/20)
wake = (now2 + timedelta(seconds=secs + 1)).replace(second=0, microsecond=0)
print(f"[llm] 本窗 {wk} 已达 {calls} 次调用上限(≥{WINDOW_CALL_CAP}),"
f"睡 {secs // 60} 分钟到下一窗 {wake:%H:%M} 续跑", file=sys.stderr)
time.sleep(secs)
continue # 醒来重读账本:跨过窗边界后是新窗,calls 归 0
# 2) 预算耗尽:本窗改用非 MiniMax 链;否则用全链
if usd >= WINDOW_BUDGET_USD:
chain = [m for m in BUDGET_CHAIN if m not in MINIMAX_MODELS]
print(f"[llm] 本窗 {wk} MiniMax 花费 ${usd:.4f} 已达预算上限 ${WINDOW_BUDGET_USD},"
f"本窗改用非 MiniMax 链 {chain}", file=sys.stderr)
else:
chain = list(BUDGET_CHAIN)
plan_quota_open = _plan_quota_open_models(wk)
skipped = [m for m in chain if m in plan_quota_open]
if skipped:
print(f"[llm] 本窗 {wk} 跳过 Token Plan 已耗尽模型 {skipped}", file=sys.stderr)
chain = [m for m in chain if m not in plan_quota_open]
# 3) 沿链逐个模型调用;撞敏感/不可用换下一个
for m in chain:
try:
content, usage = chat(
prompt,
model=m,
system=system,
max_tokens=max_tokens,
temperature=temperature,
top_p=top_p,
run_id=run_id,
caller=caller,
requested_model_id=model,
persist_call=persist_call,
)
except PlanQuotaExhausted as e:
plan_quota_open.add(m)
print(f"[llm] 治理链 {m} Token Plan 本窗耗尽,立即熔断并降级下一个:{str(e)[:80]}",
file=sys.stderr)
continue
except (SensitiveError, RuntimeError) as e:
print(f"[llm] 治理链 {m} 失败({type(e).__name__}: {str(e)[:80]}),降级下一个",
file=sys.stderr)
continue
# 4) 成功记账:只有 MiniMax 计入 $24/窗 预算,其余模型成本计 0(只占调用数)
add = cost_usd(m, usage) if m in MINIMAX_MODELS else 0.0
_bump_window(wk, add) # 全新短连接原子累加,写完即释放
return content, usage, m
# 全链走完仍无成功:交上层处置(拆书硬停 / 判重保守 keep / 审核标 blocked)
return None, None, None

View File

@ -1,7 +1,13 @@
# skill 脚本层通用依赖(公约见 README §四能力清单)
# 初始化: uv venv .venv && uv pip install --python .venv/bin/python -r requirements.txt
# 初始化: 在 agent-example/ 根执行
# uv venv .venv && uv pip install --python .venv/bin/python -r requirements.txt
psycopg[binary]
requests
pyyaml
click
json-repair
-e ./muse-db
-e ./muse-claude-runtime
-e ./muse-llm
-e ./muse-embed
-e ./humanization

View File

@ -0,0 +1,81 @@
#!/usr/bin/env python3
"""门禁:Skill 不得靠 sys.path 去别人的 scripts/ 或 humanization/src 拿实现;看板不得 import Skill。
被多个 Skill 或看板消费的实现装成顶层可安装包(muse-db / muse-llm / muse-embed /
muse-claude-runtime / muse-deai),调用方 import 已安装的包。拥有该实现的 Skill 自己的
scripts/ 不受限(CLI 与库同属一个能力域)。
扫描范围只含 .claude/skills 与 dashboard;测试、humanization/tests/tools/eval 不在范围内。
"""
from __future__ import annotations
import pathlib
import re
import unittest
ROOT = pathlib.Path(__file__).resolve().parents[2]
SKILLS = ROOT / ".claude" / "skills"
DASHBOARD = ROOT / "dashboard"
# (禁止注入的路径, 提供替代实现的包, 豁免的 Skill 目录名)
FORBIDDEN_PATHS = (
("humanization/src", "muse-deai(import deai)", None),
("access-database/scripts", "muse-db(from muse_db import connect)", "access-database"),
("call-content-model/scripts", "muse-llm(import muse_llm)", "call-content-model"),
("embed-knowledge/scripts", "muse-embed(import muse_embed)", "embed-knowledge"),
("execute-claude-task/scripts", "muse-claude-runtime(import claude_runtime)",
"execute-claude-task"),
("establish-voice-baseline/scripts", "muse-deai(deai.baseline / deai.load_db)",
"establish-voice-baseline"),
)
def _path_pattern(path: str) -> re.Pattern[str]:
"""同时匹配裸路径与 pathlib 拼接形态:``a/b`` 与 ``"a" / "b"``。"""
head, tail = path.split("/")
return re.compile(rf"""{re.escape(head)}(?:/|["']\s*/\s*["']){re.escape(tail)}""")
class ImportBoundaryTest(unittest.TestCase):
def test_skills_do_not_syspath_into_shared_implementations(self):
for path, replacement, owner in FORBIDDEN_PATHS:
with self.subTest(path=path):
pattern = _path_pattern(path)
prefix = f".claude/skills/{owner}/" if owner else None
offenders = [
rel for rel in self._skill_files()
if not (prefix and rel.startswith(prefix))
and pattern.search((ROOT / rel).read_text(encoding="utf-8"))
]
self.assertEqual(
offenders,
[],
f"Skill 必须消费 {replacement},不得 sys.path 指向 {path}:\n"
+ "\n".join(offenders),
)
def test_dashboard_does_not_import_skills(self):
offenders: list[str] = []
for path in DASHBOARD.rglob("*.py"):
if path.name.startswith("test_"):
continue
text = path.read_text(encoding="utf-8")
rel = str(path.relative_to(ROOT))
if re.search(r"sys\.path", text) and ".claude/skills" in text:
offenders.append(rel)
if re.search(r"^from db import|^import db\b", text, re.M):
offenders.append(rel)
if re.search(r"\b(muse_llm|claude_runtime|deai)\b", text):
offenders.append(rel)
self.assertEqual(
offenders,
[],
"看板只读共享运行时包,不得 sys.path 注入 Skill 或 import db.py:\n" + "\n".join(offenders),
)
def _skill_files(self) -> list[str]:
return sorted(p.relative_to(ROOT).as_posix() for p in SKILLS.rglob("*.py"))
if __name__ == "__main__":
unittest.main()

Some files were not shown because too many files have changed in this diff Show More