feat(prompts): C6/C6.1 对抗 prompt v1.1.1 + Golden 回归两轮——解冻终裁 Path A(主 agent)
- clicker-designer v1.1.0: hard_constraints+正文双写「designIntent 禁承诺模板外机制」 - adversary-review v1.1.1: 判定细则①②③(越模板降级P2/模糊创意不出P1/checklist稳定化) + 细则①边界句(核心动作语义矛盾仍属P1——根治 v1.1.0 回归暴露的 kill 守卫放水) - P0 定义与 P1 四类口径文字逐字节未动(提取块 md5 双侧一致, 不降口径铁律) - Golden 回归两轮全档入库: v1.1.0(诚实红灯 0/3)→v1.1.1(kill守卫三通道全中/稳定性100%/误报2/16) - 黄金集口径版本化(doctrine 双标签, v1.0.0 历史四重可溯) - 终裁: 残差全在偏严侧零放水证据→解冻, 下一批可发布; v1.1.2 backlog(g9011/g9019 偏严误报) - PromptResourceLoaderTest: 版本断言由钉死字面改语义化正则(升版必挂的合树雷, C6 实证) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
zizi
parent
ffef2ad606
commit
d268f50d73
@ -8,7 +8,10 @@
|
||||
# 变更纪律：改本文件必须升 version（连同 registry.yaml 同步），并过四道闸 CI。
|
||||
# ============================================================================
|
||||
id: config.clicker-designer
|
||||
version: 1.0.0
|
||||
version: 1.1.0
|
||||
# v1.1.0（2026-06-10，minor·加规则不改口径）：batch-001 冻结告警（R8 抽检分歧：gameId 9019
|
||||
# designIntent「受热均匀」承诺模板外机制）→ hard_constraints 与正文新增 designIntent 模板内约束；
|
||||
# 原有全部约束逐字保留，未放宽任何口径。
|
||||
stage: "04-config"
|
||||
owner: WS2
|
||||
tier: tier1
|
||||
@ -37,6 +40,7 @@ hard_constraints:
|
||||
- config 严格落在注入的【模板 schema】域内：字段不增不删、类型与取值范围不越界、templateId 精确等值
|
||||
- 不自评、不下结论：禁止输出评分或 accept/fix/kill 等字样
|
||||
- 重出轮只改【上一轮问题】指出项，未指出字段与上一轮输出原样保持一致
|
||||
- designIntent 只许描述当前模板机制内的体验（clicker=点击计数达标），禁止承诺模板外机制（计时/失败惩罚/物理/音效等）
|
||||
---
|
||||
|
||||
你是独立游戏策划。只输出一个合法 JSON 对象，禁止任何解释、markdown 代码块、思考过程。所有字段严格按要求，数值在范围内，字符串非空且贴合创意与中文语境。
|
||||
@ -54,6 +58,7 @@ hard_constraints:
|
||||
3. 全部文案使用贴合创意的简体中文，自然通顺，不得出现"示例""测试""占位""TODO"等占位感词汇。
|
||||
4. 不自评：不要输出任何评分、结论或 accept/fix/kill 等字样。
|
||||
5. 若【上一轮问题】非空（重出轮）：只修改其中指出的字段/问题，未被指出的字段必须与上一轮输出保持原样。
|
||||
6. designIntent 只许描述当前模板机制内能真正给到玩家的体验（clicker 模板=点击计数达标）；禁止承诺任何模板外机制——计时压力、失败惩罚、物理效果、音效等 config 无法兑现的玩法一律不得写入 designIntent。
|
||||
|
||||
【创意】
|
||||
{{input.idea}}
|
||||
|
||||
@ -8,7 +8,14 @@
|
||||
# 变更纪律：改本文件必须升 version（连同 registry.yaml 同步），并过四道闸 CI。
|
||||
# ============================================================================
|
||||
id: quality.adversary-review
|
||||
version: 1.0.0
|
||||
version: 1.1.1
|
||||
# v1.1.1（2026-06-10，patch·细则①句尾补边界句不改口径）：Golden 回归 v1.1.0 断言①红灯——细则①被模型
|
||||
# 过度泛化吞真实题文矛盾（k-0cfbc296 创意「躲开」鲨鱼被改写为「点击驱离」，主跑+双跑+交叉三次全漏 P1）
|
||||
# → 按主 agent 裁决①给细则①句尾追加核心动作矛盾边界句（仅追加，原句一字未动）；
|
||||
# P0 定义与 P1 口径文字仍一字未动。配套：黄金集 k-26ce4440/k-a04dfac9 重标口径版本化双标签（主 agent 裁决②）。
|
||||
# v1.1.0（2026-06-10，minor·加判定细则不改口径）：batch-001 冻结告警（R8 抽检分歧 2/4：9019 designIntent
|
||||
# 越模板机制被升 P1、9021 模糊创意被判题文脱节；Z4 对抗重测一致率仅 33.3%）→ 正文新增「判定细则」节
|
||||
# （适用边界 ×2 + 判定流程稳定化 ×1）；P0 定义与 P1 口径文字一字未动，守住 R3「不得以放宽 P1 口径回应」红线。
|
||||
stage: "06-quality"
|
||||
owner: WS2
|
||||
tier: tier1
|
||||
@ -45,6 +52,11 @@ severity 判定口径（写死，不得自行扩展或更改）：
|
||||
- P1：题文不符（config 文案与创意明显脱节）、target 与玩法意图矛盾（如意图与目标难度明显不自洽）、文案不通顺或有占位感（"示例""测试""asdf"类）；
|
||||
- P2：可改进建议（更贴题的措辞、更合理的节奏等），不构成拦截。
|
||||
|
||||
判定细则（v1.1.0 增；仅细化上述口径的适用边界与判定流程，不新增、不删除、不放宽上述任何一条口径）：
|
||||
① designIntent 越模板机制（designIntent 描述了计时、失败惩罚、物理、音效等当前模板机制之外的体验）：仅当 config 字段自身存在矛盾（字段之间互相冲突，或 target 与玩法意图按上述 P1 口径明显不自洽）时才定 P1；否则只记 P2——这属于模板级表达力限制，不是该条设计的缺陷。机制/题材文案与创意的核心动作语义直接矛盾（如创意『躲避』被改写为『点击消灭』）仍属题文不符 P1——本细则仅豁免模板外增强承诺，不豁免核心动作改变。
|
||||
② 模糊创意：当创意原文不含具象题材词（如「做个好玩的游戏」「解压点点点」）时，题文一致性按「不冲突即可」判定——设计选取任何具象题材均不构成与该类创意脱节，不得据此出 P1。
|
||||
③ 判定流程（稳定化）：先按 P0 口径逐条核对，再按 P1 口径逐条核对，最后看 P2，全部核对完毕后再一次性汇总输出 findings；同一问题只记最高严重级一条，不得拆成多条或在不同等级重复展开。
|
||||
|
||||
要求：
|
||||
1. 只批不改：每条 finding 用 issue 指出问题、用 suggestion 给整改方向，不得直接给出改好的完整字段或完整 config。
|
||||
2. 不下结论：不得输出 accept/fix/kill/通过/拒绝等字样，最终结论由裁决引擎依据本 findings 做出。
|
||||
|
||||
@ -36,3 +36,7 @@ C1 spike 52 条（`docs/agent-specs/2026-06-09-generation-spike/spike-eval.csv`
|
||||
- **改本条 prompt 必须升 version**：`04-config/clicker-designer.md` frontmatter 的 `version` 与 `registry.yaml` 中本条 `version` 同步升级，
|
||||
并过四道闸 CI（Schema 通过率 / 生成成功率 ≥80% / Golden 回归 diff / 成本延迟）后方可合入；version 未变的 prompt 改动一律拒绝合入。
|
||||
- 红线：**不得以降低裁决口径换达标**（评审版 R3）；eval 数据回流后只增不改。
|
||||
|
||||
## 升版记录
|
||||
|
||||
- 2026-06-10 **v1.0.0 → v1.1.0**（minor·加规则不改口径）：hard_constraints 与正文约束新增「designIntent 只许描述当前模板机制内的体验（clicker=点击计数达标），禁止承诺模板外机制（计时/失败惩罚/物理/音效等）」。变更动机 = **batch-001 冻结告警**（R8 抽检分歧 2/4，其中 gameId 9019 的 designIntent「受热均匀」承诺了 config 无法兑现的模板外机制；主 agent 裁决维持发布、根治走 prompt 升版四道闸）。原有全部约束逐字保留，纯增量、未放宽任何口径。
|
||||
|
||||
@ -34,3 +34,8 @@
|
||||
- **改本条 prompt 必须升 version**：`06-quality/adversary-review.md` frontmatter 的 `version` 与 `registry.yaml` 中本条 `version` 同步升级，
|
||||
并过四道闸 CI（Schema 通过率 / 生成成功率 ≥80% / Golden 回归 diff / 成本延迟）后方可合入；version 未变的 prompt 改动一律拒绝合入。
|
||||
- 红线：对抗一致率走低 = 漂移预警时，**修 prompt 走四道闸，不得以放宽 P1 口径回应**（评审版 Z4/R3）；eval 数据回流后只增不改。
|
||||
|
||||
## 升版记录
|
||||
|
||||
- 2026-06-10 **v1.0.0 → v1.1.0**（minor·加判定细则不改口径）：正文新增「判定细则」节——①designIntent 越模板机制仅当 config 字段自身矛盾才 P1、否则记 P2（模板级表达力限制非设计缺陷）；②模糊创意（无具象题材词，如「做个好玩的游戏」）题文一致性=不冲突即可、不得据此出 P1；③稳定化流程：先逐条过 P0→P1 checklist 再汇总输出、同一问题只记最高严重级一条、不重复展开。变更动机 = **batch-001 冻结告警**（R8 抽检分歧 2/4：9019 designIntent 越模板机制被升 P1、9021 模糊创意被判题文脱节；Z4 对抗重测一致率仅 33.3%；主 agent 裁决两条维持发布、根治走 prompt 升版四道闸）。**P0 定义与 P1 口径文字一字未动**，细则仅划定适用边界与判定流程，守住 R3 红线。
|
||||
- 2026-06-10 **v1.1.0 → v1.1.1**（patch·细则①句尾补边界句不改口径）：细则①句尾追加边界句——「机制/题材文案与创意的核心动作语义直接矛盾（如创意『躲避』被改写为『点击消灭』）仍属题文不符 P1——本细则仅豁免模板外增强承诺，不豁免核心动作改变」（主 agent 裁决①原文，原句保留）。变更动机 = **Golden 回归断言①红灯：细则①被过度泛化吞真实题文矛盾**——k-0cfbc296（创意「躲开鲨鱼」↔ theme「点击驱离」）在 MiniMax 主跑、双跑与 deepseek-v4-flash 交叉下三次全部漏 P1（详见 `docs/agent-specs/2026-06-09-agent-loop-v1/runs/golden-regression-v1.1.0/report.md`）。配套主 agent 裁决：②黄金集 k-26ce4440/k-a04dfac9 重标为口径版本化双标签（doctrine=v1.0.0→kill(P1 越模板机制)；doctrine>=v1.1.0→P2-pass，作为细则①生效的正向守卫），k-0cfbc296 保持 kill 守卫（核心动作矛盾）；③g9017 新报 P1 定性「存疑偏真正报」计入断言②容差不阻断。**P0 定义与 P1 口径文字一字未动。**
|
||||
|
||||
@ -12,14 +12,14 @@ prompts:
|
||||
eval: eval/safety.prompt-check/ # Golden eval 集（四道闸：Schema/成功率/回归diff/成本延迟）
|
||||
# —— agent 化生成 QA 闭环 v1 三件套（HJ-AGENT-LOOP-EXEC-001 §5；v1 仅 clicker 走批跑）——
|
||||
- id: config.clicker-designer
|
||||
version: 1.0.0
|
||||
version: 1.1.0 # v1.1.0：batch-001 冻结告警→新增 designIntent 模板内约束（加规则不改口径）
|
||||
stage: "04-config"
|
||||
owner: WS2
|
||||
file: 04-config/clicker-designer.md
|
||||
desc: agent 闭环独立策划：创意一句话→GameDesign 三段（designIntent/expectedPlaySeconds/config；config 受 contracts/templates/clicker.schema.json 约束）
|
||||
eval: eval/config.clicker-designer/ # 每批 evalflow.py 强制回流（inputs/labels JSONL）+ C1 spike 52 条种子（D3 转入）
|
||||
- id: quality.adversary-review
|
||||
version: 1.0.0
|
||||
version: 1.1.1 # v1.1.1：Golden 回归断言①红灯→细则①句尾补核心动作矛盾边界句（P0/P1 口径文字仍未动）
|
||||
stage: "06-quality"
|
||||
owner: WS2
|
||||
file: 06-quality/adversary-review.md
|
||||
|
||||
@ -0,0 +1,212 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
Golden 集构建脚本（golden-regression-v1.1.0 · 第 1 步）
|
||||
|
||||
口径（主 agent 任务令）：
|
||||
- batch-001 账本中可取到 GameDesign 的条目：16 条 accept（取裁决轮 design）
|
||||
+ 3 条 p1_residual_after_fix kill 的 round1 design（fix 事件携带）；
|
||||
- W2 试点 6 条：accepted-configs.json 的 finalConfig 包装成 GameDesign，
|
||||
designIntent 按 pilot-results.md 对应行大意撰写（仅描述模板内机制，忠实于该行原意）。
|
||||
|
||||
产出：golden-set.json —— 25 条，每条携带 {entryId, source, gameId, idea, design,
|
||||
expectation(原始判定), originalFindings(batch-001 对应轮对抗 findings，W2 无)}。
|
||||
本脚本只读账本与试点产物，不发起任何 LLM 调用。
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
|
||||
_HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
_LOOP_ROOT = os.path.abspath(os.path.join(_HERE, "..", ".."))
|
||||
_REPO = os.path.abspath(os.path.join(_LOOP_ROOT, "..", "..", ".."))
|
||||
sys.path.insert(0, os.path.join(_LOOP_ROOT, "orchestrator"))
|
||||
|
||||
import ledger as ledger_mod # 复用 designId 铸造（W2 合成条目用，保持 hex64 形态防"占位感"误报）
|
||||
|
||||
LEDGER = os.path.join(_LOOP_ROOT, "runs", "batch-001", "ledger.jsonl")
|
||||
REPORT = os.path.join(_LOOP_ROOT, "runs", "batch-001", "report.json")
|
||||
W2_CONFIGS = os.path.join(_REPO, "docs", "agent-specs", "2026-06-09-generation-spike", "accepted-configs.json")
|
||||
OUT = os.path.join(_HERE, "golden-set.json")
|
||||
|
||||
# ============================================================================
|
||||
# 主 agent 裁决②③（2026-06-10，golden-regression-v1.1.0 复审）：黄金集标签口径版本化
|
||||
# - k-26ce4440 / k-a04dfac9：原 P1 锚点 = designIntent 越模板机制，与 g9019 误报同模式，
|
||||
# 在对抗 prompt v1.1.0 细则①下结构性降级为 P2 属预期 → 重标为「按 doctrine（=对抗 prompt
|
||||
# 口径版本）选择期望」的版本化双标签：doctrine=v1.0.0 仍为 kill 负例（has_p1）；
|
||||
# doctrine>=v1.1.0 转为 P2-pass 正向守卫（no_p0p1，该条再出 P1 即细则①失效）。
|
||||
# - k-0cfbc296：核心动作矛盾（创意『躲开』↔ theme『点击驱离』），不在细则①豁免范围
|
||||
# （v1.1.1 边界句即为此而加）→ 保持 has_p1 kill 守卫，不版本化（裁决②维持）。
|
||||
# - g9017：v1.1.0 回归新报 P1 定性「存疑偏真正报」，计入断言②误报容差（≤2）不阻断（裁决③）。
|
||||
# 消费方：golden_regression.resolve_expectation 按 EXPECT_VERSION 解析版本化标签；
|
||||
# 本段使重跑本构建脚本不会丢失裁决重标（与盘上 golden-set.json 保持一致）。
|
||||
# ============================================================================
|
||||
DOCTRINE_RELABELS = {
|
||||
"k-26ce4440": {
|
||||
"byDoctrine": [
|
||||
{"doctrine": "=1.0.0", "expect": "has_p1", "label": "kill(P1 越模板机制)"},
|
||||
{"doctrine": ">=1.1.0", "expect": "no_p0p1", "label": "P2-pass（细则①生效的正向守卫：该条再出 P1 即细则①失效）"},
|
||||
],
|
||||
"relabelNote": "主 agent 裁决②（2026-06-10）：原 P1 锚点=designIntent「有限时间内」承诺计时机制而 config 无计时，与 g9019 误报同模式；按口径版本化双标签消费，不再作为 doctrine>=v1.1.0 下的不放水负例",
|
||||
},
|
||||
"k-a04dfac9": {
|
||||
"byDoctrine": [
|
||||
{"doctrine": "=1.0.0", "expect": "has_p1", "label": "kill(P1 越模板机制)"},
|
||||
{"doctrine": ">=1.1.0", "expect": "no_p0p1", "label": "P2-pass（细则①生效的正向守卫：该条再出 P1 即细则①失效）"},
|
||||
],
|
||||
"relabelNote": "主 agent 裁决②（2026-06-10）：原 P1 锚点=「倒计时紧张感」越模板机制承诺，且 idea 本身含「倒计时」、题文不冲突（细则①+②叠加无 P1 落点）；按口径版本化双标签消费",
|
||||
},
|
||||
}
|
||||
KILL_GUARD_NOTES = {
|
||||
"k-0cfbc296": "kill 守卫·核心动作矛盾（主 agent 裁决②维持，不版本化）：创意『躲开』鲨鱼 ↔ theme『点击驱离』核心动作语义直接矛盾，属 P1 题文不符，不在细则①豁免范围（对抗 prompt v1.1.1 边界句即为此而加）；任何 doctrine 版本下仍须 has_p1",
|
||||
}
|
||||
ACCEPT_TOLERANCES = {
|
||||
"g9017": "主 agent 裁决③（2026-06-10）：v1.1.0 回归新报 P1（target=15 × expectedPlaySeconds=45 配比与「快速抢福袋」意图不符）定性「存疑偏真正报」，计入断言②误报容差（≤2）不阻断；expectation 维持 no_p0p1，待实玩数据复核",
|
||||
}
|
||||
|
||||
# ---- W2 六条的创意原文与 designIntent（按 pilot-results.md 逐创意结果表对应行大意撰写）----
|
||||
# 注意：designIntent 只描述 clicker 模板内机制（点击计数达标），与 v1.1.0 设计师约束一致；
|
||||
# expectedPlaySeconds 取该行裁决估算区间内的整数（落在区间内，不另造数）。
|
||||
W2_ROWS = [
|
||||
{
|
||||
"idea": "王蓝莓的小卖部收银台，给顾客结账",
|
||||
"designIntent": "放学高峰的小卖部收银台前，玩家每点击一次就为一位顾客完成一单结账，攒满目标单数即通关，体验队伍越点越短的市井烟火气。",
|
||||
"expectedPlaySeconds": 10, # 行内估 4~10s
|
||||
},
|
||||
{
|
||||
"idea": "深夜便利店招待来买夜宵的顾客",
|
||||
"designIntent": "深夜便利店的暖灯下，玩家每点击一次就招待一位来买夜宵的夜归人，招待满目标人数即完成今晚的深夜档。",
|
||||
"expectedPlaySeconds": 9, # 行内估 3.6~9s（target 已按对抗修正为 18）
|
||||
},
|
||||
{
|
||||
"idea": "猫咖啡馆里撸猫攒爱心",
|
||||
"designIntent": "治愈系猫咖的午后阳光里，玩家每轻点一次屏幕就撸一下猫、攒一颗爱心，攒满目标爱心数即通关，收获治愈感。",
|
||||
"expectedPlaySeconds": 7, # 行内估 3~7.5s
|
||||
},
|
||||
{
|
||||
"idea": "烧烤摊翻烤串，别烤糊了",
|
||||
"designIntent": "深夜烧烤摊炭火正旺，玩家每点击一次就给烤串翻一次面，翻满目标次数即烤好出摊，体验手速翻串的指尖爽感。",
|
||||
"expectedPlaySeconds": 10, # 行内估 4~10s
|
||||
},
|
||||
{
|
||||
"idea": "地铁早高峰挤上即将关门的车",
|
||||
"designIntent": "早高峰地铁站台，玩家抢在车门关闭前拼命连点屏幕向车厢里挤，攒满挤入进度即成功上车，短局节奏与题材的紧迫感天然同构。",
|
||||
"expectedPlaySeconds": 9, # 行内估 3.6~9s
|
||||
},
|
||||
{
|
||||
"idea": "解压点点点",
|
||||
"designIntent": "电子解压玩具：玩家把今天的烦躁一下一下戳掉，每戳一次攒一点解压值，攒满目标解压值即过关，获得纯粹的解压爽感。",
|
||||
"expectedPlaySeconds": 12, # 行内估 5~12.5s
|
||||
},
|
||||
]
|
||||
|
||||
|
||||
def main():
|
||||
# ---- 1. 读账本：design/fix 事件按其 data.design.designId+round 建索引；adversary findings 同轮索引 ----
|
||||
designs = {} # (designId, round) -> design dict
|
||||
adv = {} # (designId, round) -> findings（仅主流水对抗行，排除 audit/retest 行）
|
||||
verdicts = []
|
||||
with open(LEDGER, "r", encoding="utf-8") as fh:
|
||||
for line in fh:
|
||||
ev = json.loads(line)
|
||||
if ev.get("type") == "verdict":
|
||||
verdicts.append(ev)
|
||||
continue
|
||||
st, data = ev.get("stage"), ev.get("data") or {}
|
||||
if st in ("design", "fix") and isinstance(data.get("design"), dict):
|
||||
d = data["design"]
|
||||
designs[(d["designId"], d.get("round"))] = d
|
||||
elif st == "adversary" and ev.get("status") == "ok" \
|
||||
and not data.get("audit") and not data.get("retest") and "findings" in data:
|
||||
adv[(ev["designId"], ev.get("round"))] = data["findings"]
|
||||
|
||||
# gameId 映射（accept 条目）：report.json canaryPublished
|
||||
rep = json.load(open(REPORT, "r", encoding="utf-8"))
|
||||
game_ids = {c["designId"]: c["gameId"] for c in rep.get("canaryPublished", [])}
|
||||
|
||||
entries = []
|
||||
n_accept = n_kill = 0
|
||||
for v in verdicts:
|
||||
did, rnd = v["designId"], v["round"]
|
||||
if v["decision"] == "accept":
|
||||
d = designs.get((did, rnd))
|
||||
if d is None:
|
||||
raise SystemExit("accept 条目缺 design：%s round=%s" % (did[:12], rnd))
|
||||
n_accept += 1
|
||||
entries.append({
|
||||
"entryId": "g%s" % game_ids.get(did, did[:8]),
|
||||
"source": "batch-001-accept",
|
||||
"gameId": game_ids.get(did),
|
||||
"designId": did,
|
||||
"round": rnd,
|
||||
"idea": d["idea"],
|
||||
"design": d,
|
||||
"expectation": "no_p0p1", # 原裁决 accept ⇒ 期望新 prompt 下无 P0/P1
|
||||
"originalFindings": adv.get((did, rnd), []),
|
||||
})
|
||||
elif v["decision"] == "kill" and "p1_residual_after_fix" in (v.get("reasons") or []):
|
||||
d = designs.get((did, 1))
|
||||
if d is None:
|
||||
raise SystemExit("p1_residual kill 条目缺 round1 design：%s" % did[:12])
|
||||
n_kill += 1
|
||||
entries.append({
|
||||
"entryId": "k-%s" % did[:8],
|
||||
"source": "batch-001-p1residual-kill",
|
||||
"gameId": None,
|
||||
"designId": did,
|
||||
"round": 1,
|
||||
"idea": d["idea"],
|
||||
"design": d,
|
||||
"expectation": "has_p1", # 断言①：新 prompt 下仍须含 P1（不放水）
|
||||
"originalFindings": adv.get((did, 1), []),
|
||||
})
|
||||
|
||||
# ---- 2. W2 六条包装（finalConfig + pilot 行大意 designIntent；designId 用正式铸造保持形态一致）----
|
||||
w2_configs = json.load(open(W2_CONFIGS, "r", encoding="utf-8"))
|
||||
assert len(w2_configs) == len(W2_ROWS) == 6, "W2 条目数不为 6"
|
||||
for i, (cfg, row) in enumerate(zip(w2_configs, W2_ROWS), 1):
|
||||
did = ledger_mod.mint_design_id(row["idea"] + "::w2-pilot", "clicker", 0)
|
||||
entries.append({
|
||||
"entryId": "w2-%d" % i,
|
||||
"source": "w2-pilot",
|
||||
"gameId": None,
|
||||
"designId": did,
|
||||
"round": 0,
|
||||
"idea": row["idea"],
|
||||
"design": {
|
||||
"designId": did,
|
||||
"idea": row["idea"],
|
||||
"templateId": "clicker",
|
||||
"designIntent": row["designIntent"],
|
||||
"expectedPlaySeconds": row["expectedPlaySeconds"],
|
||||
"config": cfg,
|
||||
"round": 0,
|
||||
},
|
||||
"expectation": "no_p0p1", # W2 全部裁决入池 ⇒ 期望无 P0/P1
|
||||
"originalFindings": None, # W2 对抗记录为另一套口径（flaws），不作对照
|
||||
})
|
||||
|
||||
assert n_accept == 16, "accept 条目数=%d ≠ 16" % n_accept
|
||||
assert n_kill == 3, "p1_residual kill 条目数=%d ≠ 3" % n_kill
|
||||
assert len(entries) == 25, "Golden 集总数=%d ≠ 25" % len(entries)
|
||||
|
||||
# ---- 3. 应用主 agent 裁决②③标签重标/注记（常量与说明见文件头 DOCTRINE_RELABELS 段；幂等可重跑）----
|
||||
# 键序约定：expectation 同键替换保留原位；expectationNote/tolerance 为注记键，追加在条目尾部。
|
||||
for e in entries:
|
||||
if e["entryId"] in DOCTRINE_RELABELS:
|
||||
e["expectation"] = DOCTRINE_RELABELS[e["entryId"]]
|
||||
if e["entryId"] in KILL_GUARD_NOTES:
|
||||
e["expectationNote"] = KILL_GUARD_NOTES[e["entryId"]]
|
||||
if e["entryId"] in ACCEPT_TOLERANCES:
|
||||
e["tolerance"] = ACCEPT_TOLERANCES[e["entryId"]]
|
||||
|
||||
with open(OUT, "w", encoding="utf-8") as fh:
|
||||
json.dump(entries, fh, ensure_ascii=False, indent=2)
|
||||
print("Golden 集已写出：%s（accept=16 kill=3 w2=6，共 %d 条）" % (OUT, len(entries)))
|
||||
for e in entries:
|
||||
print(" %-12s %-26s idea=%s" % (e["entryId"], e["source"], e["idea"][:24]))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@ -0,0 +1,705 @@
|
||||
[
|
||||
{
|
||||
"entryId": "g9011",
|
||||
"source": "batch-001-accept",
|
||||
"gameId": 9011,
|
||||
"designId": "4e2de887ceed035e571db5ed0f79462f1f52d6d4944805ce7b8e65a63ce1585c",
|
||||
"round": 0,
|
||||
"idea": "王蓝莓的小卖部收银台，给顾客结账",
|
||||
"design": {
|
||||
"designId": "4e2de887ceed035e571db5ed0f79462f1f52d6d4944805ce7b8e65a63ce1585c",
|
||||
"idea": "王蓝莓的小卖部收银台，给顾客结账",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "玩家化身校园小卖部收银员，在柜台前快速为排队的同学们结账。每点击一次屏幕完成一单交易，在限定时间内完成目标数量的结账任务，体验便利店员紧张忙碌的工作节奏。",
|
||||
"expectedPlaySeconds": 25,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "小卖部·收银时刻",
|
||||
"theme": "校园便利店的热闹柜台，排队结账的同学们等待着",
|
||||
"target": 12,
|
||||
"scoreLabel": "结账单"
|
||||
},
|
||||
"round": 0,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": []
|
||||
},
|
||||
{
|
||||
"entryId": "g9012",
|
||||
"source": "batch-001-accept",
|
||||
"gameId": 9012,
|
||||
"designId": "115589c5bc4a8f572c087cc20a7373f16064eccce621cc038e458616f5485618",
|
||||
"round": 1,
|
||||
"idea": "太空舱里捡星星补给能量",
|
||||
"design": {
|
||||
"designId": "115589c5bc4a8f572c087cc20a7373f16064eccce621cc038e458616f5485618",
|
||||
"idea": "太空舱里捡星星补给能量",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "在漂浮着星光的太空舱内点击收集散落的星体，将星星转化为维持舱室运转的能源。每一次点击都是一次星际拾取，累积足够的星能即可完成任务。",
|
||||
"expectedPlaySeconds": 30,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "星际拾星员",
|
||||
"theme": "太空舱内漂浮着点点星芒，等待被收集转化为维系舱室运转的能量",
|
||||
"target": 15,
|
||||
"scoreLabel": "星能"
|
||||
},
|
||||
"round": 1,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": []
|
||||
},
|
||||
{
|
||||
"entryId": "g9013",
|
||||
"source": "batch-001-accept",
|
||||
"gameId": 9013,
|
||||
"designId": "6f091b6da8142dda57ab8b132403c2ab62e00b5824a20a10d1156ea982bda32d",
|
||||
"round": 0,
|
||||
"idea": "农场里收割成熟的蔬菜",
|
||||
"design": {
|
||||
"designId": "6f091b6da8142dda57ab8b132403c2ab62e00b5824a20a10d1156ea982bda32d",
|
||||
"idea": "农场里收割成熟的蔬菜",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "玩家化身勤劳农夫，在生机勃勃的菜园中快速点击收割成熟的蔬菜，体验秋收的满足与快乐。",
|
||||
"expectedPlaySeconds": 12,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "绿野采收忙",
|
||||
"theme": "金秋时节，菜园里的番茄、黄瓜、青椒相继成熟，颗颗饱满亟待采摘，玩家点击即可将新鲜蔬菜收入篮中。",
|
||||
"target": 12,
|
||||
"scoreLabel": "蔬菜颗数"
|
||||
},
|
||||
"round": 0,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": []
|
||||
},
|
||||
{
|
||||
"entryId": "g9014",
|
||||
"source": "batch-001-accept",
|
||||
"gameId": 9014,
|
||||
"designId": "a1869689d1b746b894edce5b93514dca856121f937056bb42ced6d8ea489f243",
|
||||
"round": 0,
|
||||
"idea": "深夜便利店招待来买夜宵的顾客",
|
||||
"design": {
|
||||
"designId": "a1869689d1b746b894edce5b93514dca856121f937056bb42ced6d8ea489f243",
|
||||
"idea": "深夜便利店招待来买夜宵的顾客",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "在温暖的灯光下，为深夜匆匆而来的顾客递上一份热腾腾的夜宵。在指尖的忙碌中，体验深夜城市便利店独有的温情与节奏。",
|
||||
"expectedPlaySeconds": 18,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "午夜便利店·夜宵速递",
|
||||
"theme": "昏黄的灯光点亮深夜的便利店，烤肠在烤架上滋滋作响，关东煮在汤锅里翻滚，饭团整齐排列。凌晨的风穿过玻璃门，推进来一位位寻找慰藉的夜归人。",
|
||||
"target": 18,
|
||||
"scoreLabel": "服务顾客数"
|
||||
},
|
||||
"round": 0,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": []
|
||||
},
|
||||
{
|
||||
"entryId": "g9015",
|
||||
"source": "batch-001-accept",
|
||||
"gameId": 9015,
|
||||
"designId": "b2603e82f07a463bfec7dab27272daab1cc760e1d108bb0ae3811322c4daa24b",
|
||||
"round": 1,
|
||||
"idea": "校园运动会上的接力赛冲刺",
|
||||
"design": {
|
||||
"designId": "b2603e82f07a463bfec7dab27272daab1cc760e1d108bb0ae3811322c4daa24b",
|
||||
"idea": "校园运动会上的接力赛冲刺",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "玩家通过快速点击屏幕，模拟运动员在跑道上的冲刺过程，追逐接力赛终点线的荣耀，体验校园体育竞技的紧张与热血。",
|
||||
"expectedPlaySeconds": 12,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "操场冲刺·冲刺荣耀",
|
||||
"theme": "红色跑道上，最后一棒选手全力冲刺，接力棒飞速传递，观众欢呼声中冲向终点线，热血青春在脚下绽放。",
|
||||
"target": 12,
|
||||
"scoreLabel": "冲刺次数"
|
||||
},
|
||||
"round": 1,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": []
|
||||
},
|
||||
{
|
||||
"entryId": "g9016",
|
||||
"source": "batch-001-accept",
|
||||
"gameId": 9016,
|
||||
"designId": "b00027fe8f60f0d08a84c12e0bf91c25448a7d1b3a3a35b6f04d4957f2e66ad9",
|
||||
"round": 1,
|
||||
"idea": "猫咖啡馆里撸猫攒爱心",
|
||||
"design": {
|
||||
"designId": "b00027fe8f60f0d08a84c12e0bf91c25448a7d1b3a3a35b6f04d4957f2e66ad9",
|
||||
"idea": "猫咖啡馆里撸猫攒爱心",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "在温馨的猫咖里轻抚可爱的猫咪，每次触摸都会收获暖暖的爱心，集齐爱心即可通关，感受治愈的力量。",
|
||||
"expectedPlaySeconds": 30,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "猫咖轻抚·爱心满溢",
|
||||
"theme": "温馨治愈的猫咖啡馆氛围，柔软的猫咪等待你的轻抚，每一下抚摸都化作温暖的爱意",
|
||||
"target": 10,
|
||||
"scoreLabel": "爱心"
|
||||
},
|
||||
"round": 1,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": []
|
||||
},
|
||||
{
|
||||
"entryId": "g9017",
|
||||
"source": "batch-001-accept",
|
||||
"gameId": 9017,
|
||||
"designId": "43afd1f72ac6cf454faeb54cb50343e31cb06fd7ba1e07d92567debc57a05036",
|
||||
"round": 0,
|
||||
"idea": "春节集市上抢福袋",
|
||||
"design": {
|
||||
"designId": "43afd1f72ac6cf454faeb54cb50343e31cb06fd7ba1e07d92567debc57a05036",
|
||||
"idea": "春节集市上抢福袋",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "在热闹的春节集市中快速点击收集飞来的福袋，感受红火的年味与抢福的紧张刺激，达成目标即为通关。",
|
||||
"expectedPlaySeconds": 45,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "迎春纳福·福袋满仓",
|
||||
"theme": "春节集市人潮涌动，各式福袋从天而降，红色金色交织出喜庆的节日海洋。",
|
||||
"target": 15,
|
||||
"scoreLabel": "福袋"
|
||||
},
|
||||
"round": 0,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": [],
|
||||
"tolerance": "主 agent 裁决③（2026-06-10）：v1.1.0 回归新报 P1（target=15 × expectedPlaySeconds=45 配比与「快速抢福袋」意图不符）定性「存疑偏真正报」，计入断言②误报容差（≤2）不阻断；expectation 维持 no_p0p1，待实玩数据复核"
|
||||
},
|
||||
{
|
||||
"entryId": "k-0cfbc296",
|
||||
"source": "batch-001-p1residual-kill",
|
||||
"gameId": null,
|
||||
"designId": "0cfbc29612b6966447cf7449d6cdef8da1f57521f353bdc726b48fa0aa621048",
|
||||
"round": 1,
|
||||
"idea": "海底捞珍珠同时躲开鲨鱼",
|
||||
"design": {
|
||||
"designId": "0cfbc29612b6966447cf7449d6cdef8da1f57521f353bdc726b48fa0aa621048",
|
||||
"idea": "海底捞珍珠同时躲开鲨鱼",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "在神秘的海底世界，玩家扮演勇敢的采珠人，在收集璀璨珍珠的同时，必须时刻警惕鲨鱼的出没。点击收集珍珠，躲避鲨鱼威胁，在危险与收获之间寻找平衡，感受紧张刺激的海底探险之旅。",
|
||||
"expectedPlaySeconds": 45,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "深海采珠记",
|
||||
"theme": "幽蓝深海中，珍珠散落在珊瑚礁与海草间，闪烁着诱人的光芒。玩家需要在收集珍珠的同时警惕鲨鱼的动向——每收集3-5颗珍珠后，鲨鱼会短暂出现，此时点击鲨鱼即可将其驱离；若未能及时点击，鲨鱼会吓跑周围的珍珠，导致损失3颗已收集的珍珠。玩家需要在收集珍珠与躲避鲨鱼之间保持平衡，在危险与收获之间寻找最佳时机，收获宝藏。",
|
||||
"target": 22,
|
||||
"scoreLabel": "珍珠"
|
||||
},
|
||||
"round": 1,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": "has_p1",
|
||||
"originalFindings": [
|
||||
{
|
||||
"severity": "P1",
|
||||
"issue": "设计意图与实际机制矛盾：idea 明确是「躲开鲨鱼」，但 theme 中鲨鱼出现后的解决方案是「点击鲨鱼即可将其驱离」，这是主动攻击而非躲避，与「躲避」的核心意图不符。",
|
||||
"suggestion": "明确玩法核心——若设计意图是躲避，则鲨鱼出现时应提供可躲避的操作窗口或其他躲避方式；若核心就是点击驱离，则应将 designIntent 改为「驱赶鲨鱼」以保持一致性。"
|
||||
},
|
||||
{
|
||||
"severity": "P2",
|
||||
"issue": "theme 描述鲨鱼惩罚时用了「吓跑周围的珍珠」和「导致损失3颗已收集的珍珠」两种说法，表述略有不一致。",
|
||||
"suggestion": "统一表述为鲨鱼会带走已收集的珍珠，损失数量以「已收集的珍珠」计数，保持逻辑清晰。"
|
||||
}
|
||||
],
|
||||
"expectationNote": "kill 守卫·核心动作矛盾（主 agent 裁决②维持，不版本化）：创意『躲开』鲨鱼 ↔ theme『点击驱离』核心动作语义直接矛盾，属 P1 题文不符，不在细则①豁免范围（对抗 prompt v1.1.1 边界句即为此而加）；任何 doctrine 版本下仍须 has_p1"
|
||||
},
|
||||
{
|
||||
"entryId": "g9019",
|
||||
"source": "batch-001-accept",
|
||||
"gameId": 9019,
|
||||
"designId": "439034c731e5530c94685dad552ddf6395eee6c00f70e7cad08451cbfa656012",
|
||||
"round": 0,
|
||||
"idea": "烧烤摊翻烤串，别烤糊了",
|
||||
"design": {
|
||||
"designId": "439034c731e5530c94685dad552ddf6395eee6c00f70e7cad08451cbfa656012",
|
||||
"idea": "烧烤摊翻烤串，别烤糊了",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "夜幕降临，炭火通红的烧烤摊前，烤串在火焰上滋滋作响。玩家需要快速翻动每一根烤串，确保受热均匀，避免烤糊。每一次精准的翻动都是对火候的把握，在烟火气中体验指尖的忙碌与满足。",
|
||||
"expectedPlaySeconds": 54,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "炭火翻串记",
|
||||
"theme": "热闹夜市的烧烤摊，炭火正旺，烤串在火焰上翻滚，油脂滴落发出滋滋声响，烟火气与市井气息交织。",
|
||||
"target": 18,
|
||||
"scoreLabel": "翻烤次数"
|
||||
},
|
||||
"round": 0,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": []
|
||||
},
|
||||
{
|
||||
"entryId": "g9020",
|
||||
"source": "batch-001-accept",
|
||||
"gameId": 9020,
|
||||
"designId": "005cde6a7d097d691745726db5fdc1b54e6e935b448857b486752e92fd1a312f",
|
||||
"round": 0,
|
||||
"idea": "地铁早高峰挤上即将关门的车",
|
||||
"design": {
|
||||
"designId": "005cde6a7d097d691745726db5fdc1b54e6e935b448857b486752e92fd1a312f",
|
||||
"idea": "地铁早高峰挤上即将关门的车",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "早高峰地铁站台，车门警示灯闪烁，倒计时逼近，玩家疯狂点击屏幕帮助一位位乘客挤上即将关门的列车，在人潮涌动中体验争分夺秒的紧张与刺激。",
|
||||
"expectedPlaySeconds": 8,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "站台冲刺·车门将闭",
|
||||
"theme": "早高峰地铁站台，人潮汹涌，车门警示灯急促闪烁，倒计时逼近，玩家点击帮助乘客抢在关门瞬间挤上车厢。",
|
||||
"target": 15,
|
||||
"scoreLabel": "上车人次"
|
||||
},
|
||||
"round": 0,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": []
|
||||
},
|
||||
{
|
||||
"entryId": "g9021",
|
||||
"source": "batch-001-accept",
|
||||
"gameId": 9021,
|
||||
"designId": "526aaaa99ab651ff89fd00dfc2b8b401e5c03b23f19b05a17b1757e526671eee",
|
||||
"round": 0,
|
||||
"idea": "做一个很好玩的游戏",
|
||||
"design": {
|
||||
"designId": "526aaaa99ab651ff89fd00dfc2b8b401e5c03b23f19b05a17b1757e526671eee",
|
||||
"idea": "做一个很好玩的游戏",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "温馨治愈的糖果屋场景，玩家点击收集散落在彩色糖果罐中的各种糖果，每点击一次就能收集一颗甜蜜的糖果，累积到目标数量即可完成这一轮的甜蜜收集任务。",
|
||||
"expectedPlaySeconds": 15,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "糖果屋的甜蜜收集",
|
||||
"theme": "温馨梦幻的糖果屋里，五彩缤纷的糖果散落在玻璃罐和木质架子上，空气中弥漫着甜蜜的香气，玩家在这里寻找最简单的快乐。",
|
||||
"target": 8,
|
||||
"scoreLabel": "糖果颗数"
|
||||
},
|
||||
"round": 0,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": []
|
||||
},
|
||||
{
|
||||
"entryId": "g9022",
|
||||
"source": "batch-001-accept",
|
||||
"gameId": 9022,
|
||||
"designId": "b52388c4e4d72182a728ef86f03bb699292d4015f108b6426218f6170d63653b",
|
||||
"round": 0,
|
||||
"idea": "解压点点点",
|
||||
"design": {
|
||||
"designId": "b52388c4e4d72182a728ef86f03bb699292d4015f108b6426218f6170d63653b",
|
||||
"idea": "解压点点点",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "在弹力球工作室中，玩家点击弹力球帮助它们弹得更高，每一次弹跳都带走一份压力。简单的点击节奏配合弹力球的可爱造型，让玩家在轻松愉快的氛围中释放一天的疲惫。",
|
||||
"expectedPlaySeconds": 45,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "弹力球解压工作室",
|
||||
"theme": "柔软的彩色弹力球在工作室里轻轻弹跳，每一下弹起都带走一点压力。简洁的空间，柔和的光线，玩家只需要跟着节奏点击，让球球们弹得更高更开心。",
|
||||
"target": 15,
|
||||
"scoreLabel": "压力消散值"
|
||||
},
|
||||
"round": 0,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": []
|
||||
},
|
||||
{
|
||||
"entryId": "g9023",
|
||||
"source": "batch-001-accept",
|
||||
"gameId": 9023,
|
||||
"designId": "cb3f7444796b29723b74e08807a6c94555e8f98c4a4f0c0c7cf7bd50c4c90dcb",
|
||||
"round": 0,
|
||||
"idea": "中秋节帮嫦娥把月饼送上月宫",
|
||||
"design": {
|
||||
"designId": "cb3f7444796b29723b74e08807a6c94555e8f98c4a4f0c0c7cf7bd50c4c90dcb",
|
||||
"idea": "中秋节帮嫦娥把月饼送上月宫",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "中秋圆月之夜，你化作月宫使者，帮助嫦娥将满载思念的月饼送往广寒宫。每一次点击，都是对团圆的祈愿，在清冷月光与温热饼香中，体验独特的节日仪式感。",
|
||||
"expectedPlaySeconds": 8,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "月宫献饼记",
|
||||
"theme": "中秋圆月高悬，清冷的月辉洒落云端，嫦娥怀抱月饼踏云而去，温暖饼香与清冷月色交织，献礼之路就在指尖。",
|
||||
"target": 15,
|
||||
"scoreLabel": "月饼"
|
||||
},
|
||||
"round": 0,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": []
|
||||
},
|
||||
{
|
||||
"entryId": "g9024",
|
||||
"source": "batch-001-accept",
|
||||
"gameId": 9024,
|
||||
"designId": "090212e4c88d43b691aba53037bd586f1070dad5b7aeae790a785a31c351c5f5",
|
||||
"round": 0,
|
||||
"idea": "雨夜帮路边馄饨摊老板撑伞收摊",
|
||||
"design": {
|
||||
"designId": "090212e4c88d43b691aba53037bd586f1070dad5b7aeae790a785a31c351c5f5",
|
||||
"idea": "雨夜帮路边馄饨摊老板撑伞收摊",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "雨夜馄饨摊的灯火在细雨中摇曳，玩家帮助老板快速收拢摊位物件，在淅沥雨声中感受指尖传递的温暖与忙碌后的满足。",
|
||||
"expectedPlaySeconds": 20,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "雨夜馄饨摊·温情收摊",
|
||||
"theme": "绵绵细雨中，街头馄饨摊的灯火摇曳，老板在忙碌一天后准备收摊，热腾腾的汤锅、整齐的碗碟等待被一一收起，玩家帮忙撑伞收拾，在湿润的夜色中传递一份市井温情。",
|
||||
"target": 15,
|
||||
"scoreLabel": "收好的摊位物件"
|
||||
},
|
||||
"round": 0,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": []
|
||||
},
|
||||
{
|
||||
"entryId": "k-26ce4440",
|
||||
"source": "batch-001-p1residual-kill",
|
||||
"gameId": null,
|
||||
"designId": "26ce44408e55eaa604b3efa2903c3cd5a541ab99ed8aa7c62c9b8192ef03260e",
|
||||
"round": 1,
|
||||
"idea": "图书馆闭馆前把还回来的书归架",
|
||||
"design": {
|
||||
"designId": "26ce44408e55eaa604b3efa2903c3cd5a541ab99ed8aa7c62c9b8192ef03260e",
|
||||
"idea": "图书馆闭馆前把还回来的书归架",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "玩家扮演图书馆管理员，在闭馆前的有限时间内高效地将归还的书籍送回书架，体验忙碌却有序的职业节奏，感受书归其位的成就感。",
|
||||
"expectedPlaySeconds": 45,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "闭馆归架记",
|
||||
"theme": "图书馆的夜灯渐次熄灭，还回的书本堆叠在归还台上，等待被送回各自的书架，忙碌的双手在书架间穿梭，每一本书都找到了归宿。",
|
||||
"target": 12,
|
||||
"scoreLabel": "归架书数"
|
||||
},
|
||||
"round": 1,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": {
|
||||
"byDoctrine": [
|
||||
{
|
||||
"doctrine": "=1.0.0",
|
||||
"expect": "has_p1",
|
||||
"label": "kill(P1 越模板机制)"
|
||||
},
|
||||
{
|
||||
"doctrine": ">=1.1.0",
|
||||
"expect": "no_p0p1",
|
||||
"label": "P2-pass（细则①生效的正向守卫：该条再出 P1 即细则①失效）"
|
||||
}
|
||||
],
|
||||
"relabelNote": "主 agent 裁决②（2026-06-10）：原 P1 锚点=designIntent「有限时间内」承诺计时机制而 config 无计时，与 g9019 误报同模式；按口径版本化双标签消费，不再作为 doctrine>=v1.1.0 下的不放水负例"
|
||||
},
|
||||
"originalFindings": [
|
||||
{
|
||||
"severity": "P1",
|
||||
"issue": "designIntent 明确提到「有限时间内」完成任务，但 config 中缺少时间限制配置，expectedPlaySeconds 仅表示预期时长而非计时机制",
|
||||
"suggestion": "在 config 中增加 timeLimit 字段与 expectedPlaySeconds 对应，明确玩家需在规定秒数内达成 target，或说明 target 为无需计时完成的目标数"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"entryId": "g9026",
|
||||
"source": "batch-001-accept",
|
||||
"gameId": 9026,
|
||||
"designId": "664cf2034f2bc086f6780020f9fe02a9a1e379fc2b33daa430e34841e3e30741",
|
||||
"round": 0,
|
||||
"idea": "奶茶店爆单日疯狂摇珍珠奶茶",
|
||||
"design": {
|
||||
"designId": "664cf2034f2bc086f6780020f9fe02a9a1e379fc2b33daa430e34841e3e30741",
|
||||
"idea": "奶茶店爆单日疯狂摇珍珠奶茶",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "玩家扮演奶茶店员工，在爆单高峰时段疯狂摇晃调酒器出杯。指尖跟随节拍快速点击，模拟调制珍珠奶茶的忙碌节奏，在倒计时结束前完成尽可能多的订单，体验节假日餐饮从业者的紧张与成就。",
|
||||
"expectedPlaySeconds": 60,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "茶铺高峰·手摇风暴",
|
||||
"theme": "奶茶店节假日大排长龙，玩家化身店员手持 shaker 疯狂摇晃出杯，蒸汽与糖香弥漫，指尖的节奏决定订单的流转速度。",
|
||||
"target": 20,
|
||||
"scoreLabel": "杯奶茶"
|
||||
},
|
||||
"round": 0,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": []
|
||||
},
|
||||
{
|
||||
"entryId": "k-a04dfac9",
|
||||
"source": "batch-001-p1residual-kill",
|
||||
"gameId": null,
|
||||
"designId": "a04dfac9bba70833a1dcbce8242a787f3180db5b516b65006589698878eabcc6",
|
||||
"round": 1,
|
||||
"idea": "跨年夜零点在广场上点烟花倒计时",
|
||||
"design": {
|
||||
"designId": "a04dfac9bba70833a1dcbce8242a787f3180db5b516b65006589698878eabcc6",
|
||||
"idea": "跨年夜零点在广场上点烟花倒计时",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "跨年夜零点倒计时的紧张期待，玩家在广场等待新年钟声响起的那一刻，点击点燃璀璨烟花，感受辞旧迎新的激动与喜悦。",
|
||||
"expectedPlaySeconds": 20,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "午夜庆典·焰火绽放",
|
||||
"theme": "跨年夜倒计时即将归零，广场上人潮涌动，玩家争分夺秒连续点击，在新年钟声敲响瞬间点燃璀璨烟花，迎接第一道光芒。",
|
||||
"target": 15,
|
||||
"scoreLabel": "点燃烟花"
|
||||
},
|
||||
"round": 1,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": {
|
||||
"byDoctrine": [
|
||||
{
|
||||
"doctrine": "=1.0.0",
|
||||
"expect": "has_p1",
|
||||
"label": "kill(P1 越模板机制)"
|
||||
},
|
||||
{
|
||||
"doctrine": ">=1.1.0",
|
||||
"expect": "no_p0p1",
|
||||
"label": "P2-pass（细则①生效的正向守卫：该条再出 P1 即细则①失效）"
|
||||
}
|
||||
],
|
||||
"relabelNote": "主 agent 裁决②（2026-06-10）：原 P1 锚点=「倒计时紧张感」越模板机制承诺，且 idea 本身含「倒计时」、题文不冲突（细则①+②叠加无 P1 落点）；按口径版本化双标签消费"
|
||||
},
|
||||
"originalFindings": [
|
||||
{
|
||||
"severity": "P1",
|
||||
"issue": "设计意图强调'倒计时的紧张期待'，但 config 中 target=15 为点击次数，与倒计时概念脱节，玩家看不到剩余时间，无法真正感受倒计时的紧张感",
|
||||
"suggestion": "建议将 target 改为时间维度（如倒计时 X 秒），或明确说明是在倒计时内完成指定点击次数，使玩法与'倒计时'主题一致"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"entryId": "g9028",
|
||||
"source": "batch-001-accept",
|
||||
"gameId": 9028,
|
||||
"designId": "53933f300e7a9c254986a2972be2c9c577b1233856336f18a20ed5a17bc6ae69",
|
||||
"round": 0,
|
||||
"idea": "给办公室加班的同事分下午茶蛋糕",
|
||||
"design": {
|
||||
"designId": "53933f300e7a9c254986a2972be2c9c577b1233856336f18a20ed5a17bc6ae69",
|
||||
"idea": "给办公室加班的同事分下午茶蛋糕",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "模拟办公室下午茶分蛋糕的温馨场景，玩家快速点击将蛋糕分送给加班的同事，在忙碌的工作间隙感受分享的甜蜜与满足。",
|
||||
"expectedPlaySeconds": 15,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "甜蜜分送",
|
||||
"theme": "午后阳光洒进办公室，加班中的同事们等待分享那份下午茶蛋糕，玩家化身温暖使者快速将蛋糕分送到每个人手中。",
|
||||
"target": 12,
|
||||
"scoreLabel": "蛋糕"
|
||||
},
|
||||
"round": 0,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": [
|
||||
{
|
||||
"severity": "P2",
|
||||
"issue": "标题'甜蜜分送'过于笼统，未能体现'办公室加班同事'和'下午茶蛋糕'的具体情境，与创意原意的关联度不足",
|
||||
"suggestion": "建议标题更贴近具体场景，如融入办公室、蛋糕等关键词，增强题文一致性"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"entryId": "g9030",
|
||||
"source": "batch-001-accept",
|
||||
"gameId": 9030,
|
||||
"designId": "7ebba6f04540bce08c531189024c8839c26424b84ef224ea8dfa5a64c8fa5076",
|
||||
"round": 0,
|
||||
"idea": "火锅店捞毛肚，七上八下别捞老了",
|
||||
"design": {
|
||||
"designId": "7ebba6f04540bce08c531189024c8839c26424b84ef224ea8dfa5a64c8fa5076",
|
||||
"idea": "火锅店捞毛肚，七上八下别捞老了",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "沸腾的红油锅里，毛肚在翻滚七上八下，玩家点击屏幕在最佳时机将毛肚捞起，捞够目标数量即可通关。",
|
||||
"expectedPlaySeconds": 30,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "沸腾捞肚",
|
||||
"theme": "翻滚的红油锅底，热气蒸腾，毛肚在锅里七上八下跳动，玩家需把握时机点击捞起，避免毛肚过老。",
|
||||
"target": 15,
|
||||
"scoreLabel": "毛肚片"
|
||||
},
|
||||
"round": 0,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": []
|
||||
},
|
||||
{
|
||||
"entryId": "w2-1",
|
||||
"source": "w2-pilot",
|
||||
"gameId": null,
|
||||
"designId": "8f9108324a3bef55f471773b697a48047cf4c5b70b46a97a33696a5e4ebc39a1",
|
||||
"round": 0,
|
||||
"idea": "王蓝莓的小卖部收银台，给顾客结账",
|
||||
"design": {
|
||||
"designId": "8f9108324a3bef55f471773b697a48047cf4c5b70b46a97a33696a5e4ebc39a1",
|
||||
"idea": "王蓝莓的小卖部收银台，给顾客结账",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "放学高峰的小卖部收银台前，玩家每点击一次就为一位顾客完成一单结账，攒满目标单数即通关，体验队伍越点越短的市井烟火气。",
|
||||
"expectedPlaySeconds": 10,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "蓝莓妹的小卖部·结账高峰",
|
||||
"theme": "放学时分的小卖部收银台，“嘀”一声结一单、队伍越点越短的市井烟火气",
|
||||
"target": 20,
|
||||
"scoreLabel": "结账单数"
|
||||
},
|
||||
"round": 0
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": null
|
||||
},
|
||||
{
|
||||
"entryId": "w2-2",
|
||||
"source": "w2-pilot",
|
||||
"gameId": null,
|
||||
"designId": "b62350bae955d7c3a510efa0eed0f74c7408ba892b8f0b1e59a367d6456a8425",
|
||||
"round": 0,
|
||||
"idea": "深夜便利店招待来买夜宵的顾客",
|
||||
"design": {
|
||||
"designId": "b62350bae955d7c3a510efa0eed0f74c7408ba892b8f0b1e59a367d6456a8425",
|
||||
"idea": "深夜便利店招待来买夜宵的顾客",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "深夜便利店的暖灯下，玩家每点击一次就招待一位来买夜宵的夜归人，招待满目标人数即完成今晚的深夜档。",
|
||||
"expectedPlaySeconds": 9,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "凌晨两点的便利店",
|
||||
"theme": "24小时便利店的深夜档，暖灯下招待买夜宵的夜归人",
|
||||
"target": 18,
|
||||
"scoreLabel": "已招待顾客"
|
||||
},
|
||||
"round": 0
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": null
|
||||
},
|
||||
{
|
||||
"entryId": "w2-3",
|
||||
"source": "w2-pilot",
|
||||
"gameId": null,
|
||||
"designId": "2b15bd30b466f12653be4dac5d53ac5b856110986e00d6926c62e8c3ece809c7",
|
||||
"round": 0,
|
||||
"idea": "猫咖啡馆里撸猫攒爱心",
|
||||
"design": {
|
||||
"designId": "2b15bd30b466f12653be4dac5d53ac5b856110986e00d6926c62e8c3ece809c7",
|
||||
"idea": "猫咖啡馆里撸猫攒爱心",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "治愈系猫咖的午后阳光里，玩家每轻点一次屏幕就撸一下猫、攒一颗爱心，攒满目标爱心数即通关，收获治愈感。",
|
||||
"expectedPlaySeconds": 7,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "猫咖时光·撸猫集心",
|
||||
"theme": "治愈系猫咖啡馆：午后阳光里轻点屏幕撸猫，每一下攒一颗爱心",
|
||||
"target": 15,
|
||||
"scoreLabel": "爱心"
|
||||
},
|
||||
"round": 0
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": null
|
||||
},
|
||||
{
|
||||
"entryId": "w2-4",
|
||||
"source": "w2-pilot",
|
||||
"gameId": null,
|
||||
"designId": "1b3c5764b01c161d1c0cbd6aaba2c7375280b2469a850152b6596a22b55d7389",
|
||||
"round": 0,
|
||||
"idea": "烧烤摊翻烤串，别烤糊了",
|
||||
"design": {
|
||||
"designId": "1b3c5764b01c161d1c0cbd6aaba2c7375280b2469a850152b6596a22b55d7389",
|
||||
"idea": "烧烤摊翻烤串，别烤糊了",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "深夜烧烤摊炭火正旺，玩家每点击一次就给烤串翻一次面，翻满目标次数即烤好出摊，体验手速翻串的指尖爽感。",
|
||||
"expectedPlaySeconds": 10,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "烤串二十翻",
|
||||
"theme": "深夜烧烤摊：炭火滋滋作响，手速翻面，别让烤串烤糊",
|
||||
"target": 20,
|
||||
"scoreLabel": "翻面"
|
||||
},
|
||||
"round": 0
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": null
|
||||
},
|
||||
{
|
||||
"entryId": "w2-5",
|
||||
"source": "w2-pilot",
|
||||
"gameId": null,
|
||||
"designId": "feca768c87f9fcb3111a504ab074f8e998129c9de01e833957f4039c6ffbef5a",
|
||||
"round": 0,
|
||||
"idea": "地铁早高峰挤上即将关门的车",
|
||||
"design": {
|
||||
"designId": "feca768c87f9fcb3111a504ab074f8e998129c9de01e833957f4039c6ffbef5a",
|
||||
"idea": "地铁早高峰挤上即将关门的车",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "早高峰地铁站台，玩家抢在车门关闭前拼命连点屏幕向车厢里挤，攒满挤入进度即成功上车，短局节奏与题材的紧迫感天然同构。",
|
||||
"expectedPlaySeconds": 9,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "车门即将关闭",
|
||||
"theme": "早高峰地铁站台·车门倒计时",
|
||||
"target": 18,
|
||||
"scoreLabel": "挤入进度"
|
||||
},
|
||||
"round": 0
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": null
|
||||
},
|
||||
{
|
||||
"entryId": "w2-6",
|
||||
"source": "w2-pilot",
|
||||
"gameId": null,
|
||||
"designId": "d0b8c8ea60d5d07c220e1c903f8e78ed36d21ad816b4ee82d2dae8d13af0388a",
|
||||
"round": 0,
|
||||
"idea": "解压点点点",
|
||||
"design": {
|
||||
"designId": "d0b8c8ea60d5d07c220e1c903f8e78ed36d21ad816b4ee82d2dae8d13af0388a",
|
||||
"idea": "解压点点点",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "电子解压玩具：玩家把今天的烦躁一下一下戳掉，每戳一次攒一点解压值，攒满目标解压值即过关，获得纯粹的解压爽感。",
|
||||
"expectedPlaySeconds": 12,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "指尖戳戳乐",
|
||||
"theme": "治愈系电子解压玩具：把今天的烦躁一下一下戳掉，解压值攒满即过关",
|
||||
"target": 25,
|
||||
"scoreLabel": "解压值"
|
||||
},
|
||||
"round": 0
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": null
|
||||
}
|
||||
]
|
||||
@ -0,0 +1,338 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
Golden 集回归脚本（quality.adversary-review v1.1.0 · 第 2 步）
|
||||
|
||||
任务令（主 agent）：对 v1.1.0 对抗 prompt 跑 25 条 Golden 集回归，四断言：
|
||||
A 25 条全量过 v1.1.0 对抗（MiniMax-M2.7，json_object，温度 0.2——与 R8 复核口径一致）；
|
||||
① 不放水：3 条 p1_residual kill 的 round1 设计仍含 P1（任一变干净=红灯，如实报）；
|
||||
② 误报收敛：16 条原 accept 的 P1 误报 ≤2，且 9019/9021 不再出 P1；
|
||||
③ 稳定性：抽 5 条双跑同 prompt，P0/P1 判定一致率 ≥80%（对照口径=run_batch 稳定性重测：
|
||||
(hasP0,hasP1) 元组翻转即不一致）；
|
||||
④ 交叉：同 5 条用 deepseek-v4-flash（推理型，max_tokens≥2048，只读 message.content）复核，
|
||||
与 MiniMax「无 P0/P1=过」结论分歧 ≤1 条。
|
||||
|
||||
工程纪律：
|
||||
- 渲染复用 orchestrator/prompts.py（Registry 驱动 + 版本一致性核验 + 残留占位符防御）；
|
||||
- 通道复用 orchestrator/llm_client.py（重试 ×2 / 节流 0.3s / 密钥只走 NEWAPI_KEY 环境变量，严禁入盘）；
|
||||
- LLM 预算硬闸 ≤70 次（口径=每次 HTTP 尝试，与 BudgetGuard 一致）；
|
||||
- 全部原始响应落 raw/（含请求摘要，不含密钥）；断点续跑：raw 已有合法产物的调用直接复用不再发起；
|
||||
- findings 形状校验语义复制自 run_batch.py:1158-1181（_parse_json_object/_validate_findings_shape，
|
||||
避免为两个小函数拖入 backend_gw 等无关依赖）；形状异常应用层重试 ×2 后记 infra。
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
_HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
_LOOP_ROOT = os.path.abspath(os.path.join(_HERE, "..", ".."))
|
||||
_REPO = os.path.abspath(os.path.join(_LOOP_ROOT, "..", "..", ".."))
|
||||
sys.path.insert(0, os.path.join(_LOOP_ROOT, "orchestrator"))
|
||||
|
||||
import llm_client
|
||||
import prompts as prompts_mod
|
||||
|
||||
PROMPTS_DIR = os.path.join(_REPO, "contracts", "prompts")
|
||||
PROMPT_ID = "quality.adversary-review"
|
||||
EXPECT_VERSION = "1.1.0" # 回归对象版本（registry 与 frontmatter 双核验由 prompts.py 完成）
|
||||
MAIN_MODEL = "MiniMax-M2.7" # 主模型（任务令：json_object / 温度 0.2）
|
||||
CROSS_MODEL = "deepseek-v4-flash" # 交叉模型（推理型：max_tokens≥2048，读 content 忽略 reasoning_content）
|
||||
BUDGET_MAX = 70 # LLM 预算硬闸（每次 HTTP 尝试计 1，含重试）
|
||||
SHAPE_RETRIES = 2 # findings 形状异常应用层重试（同 run_batch ADVERSARY_SHAPE_RETRIES）
|
||||
|
||||
GOLDEN = os.path.join(_HERE, "golden-set.json")
|
||||
RAW_DIR = os.path.join(_HERE, "raw")
|
||||
RESULTS = os.path.join(_HERE, "results.json")
|
||||
CALLLOG = os.path.join(_HERE, "llm-calls.jsonl")
|
||||
|
||||
# 断言③/④抽样（定向覆盖分歧高危类，理由入报告）：
|
||||
# 两条 R8 分歧原条目(9019/9021) + 1 条 p1_residual kill + 1 条普通 accept + 1 条 W2 模糊创意
|
||||
STABILITY_SAMPLE = ["g9019", "g9021", "k-0cfbc296", "g9011", "w2-6"]
|
||||
|
||||
# 任务令通道口径：温度 0.2（llm_client 默认 0.4 为 batch 主流水口径；R8 复核与本回归用 0.2）
|
||||
llm_client.TEMPERATURE = 0.2
|
||||
|
||||
|
||||
# ---------------- 黄金集标签版本化解析（2026-06-10 主 agent 裁决②，标签结构见 build_golden_set.py 头注）----------------
|
||||
|
||||
def _ver_tuple(v):
|
||||
"""语义版本字符串 → 可比较元组（如 "1.1.0" → (1,1,0)），仅支持纯数字三段。"""
|
||||
return tuple(int(x) for x in v.split("."))
|
||||
|
||||
|
||||
def resolve_expectation(entry, doctrine_version=None):
|
||||
"""
|
||||
解析 Golden 条目的期望标签（黄金集标签口径版本化，主 agent 裁决②）：
|
||||
- 旧式标量标签（"no_p0p1"/"has_p1"）直接返回，行为与版本化前完全一致；
|
||||
- 版本化双标签：expectation={"byDoctrine":[{"doctrine":"=1.0.0"|">=1.1.0","expect":...,"label":...},...]}，
|
||||
按当前回归对象 prompt 版本（=判定口径 doctrine 版本，默认 EXPECT_VERSION）匹配区间取 expect——
|
||||
k-26ce4440/k-a04dfac9 在 doctrine=v1.0.0 下为 kill 负例(has_p1)、doctrine>=v1.1.0 下为
|
||||
P2-pass 正向守卫(no_p0p1)；
|
||||
- 任何区间都未命中 = 标签配置错误，立即抛错（宁可中断不静默放行，保不放水底线）。
|
||||
"""
|
||||
exp = entry["expectation"]
|
||||
if isinstance(exp, str):
|
||||
return exp
|
||||
cur = _ver_tuple(doctrine_version or EXPECT_VERSION)
|
||||
for rule in exp["byDoctrine"]:
|
||||
cons = rule["doctrine"].strip()
|
||||
if cons.startswith(">="):
|
||||
if cur >= _ver_tuple(cons[2:]):
|
||||
return rule["expect"]
|
||||
elif cons.startswith("="):
|
||||
if cur == _ver_tuple(cons[1:]):
|
||||
return rule["expect"]
|
||||
else:
|
||||
raise RuntimeError("条目 %s 含未知 doctrine 约束：%r" % (entry["entryId"], cons))
|
||||
raise RuntimeError("条目 %s 的版本化标签未覆盖 doctrine v%s" % (entry["entryId"], doctrine_version or EXPECT_VERSION))
|
||||
|
||||
|
||||
def _log(msg):
|
||||
"""统一中文日志（时间戳+消息，外部交互可追溯）。"""
|
||||
print("[%s] %s" % (time.strftime("%H:%M:%S"), msg), flush=True)
|
||||
|
||||
|
||||
class Budget(object):
|
||||
"""LLM 预算硬闸：每次 HTTP 尝试计 1，超 70 立即抛异常中止（宁可中断不超预算）。"""
|
||||
|
||||
def __init__(self, limit):
|
||||
self.limit = limit
|
||||
self.used = 0
|
||||
|
||||
def note(self):
|
||||
self.used += 1
|
||||
if self.used > self.limit:
|
||||
raise RuntimeError("LLM 预算超闸：已用 %d > %d，立即中止" % (self.used, self.limit))
|
||||
|
||||
|
||||
BUDGET = Budget(BUDGET_MAX)
|
||||
|
||||
|
||||
# ---------------- findings 形状校验（语义复制自 run_batch.py:1158-1181，注释见模块头）----------------
|
||||
|
||||
def _parse_json_object(content):
|
||||
"""解析 LLM 输出为 JSON 对象；失败返回 (None, 错误说明)。容忍 ```json 围栏（交叉模型无 json_object 时防御）。"""
|
||||
text = content.strip()
|
||||
if text.startswith("```"):
|
||||
# 剥代码围栏：```json\n...\n``` → 中间体
|
||||
lines = [ln for ln in text.split("\n") if not ln.strip().startswith("```")]
|
||||
text = "\n".join(lines).strip()
|
||||
try:
|
||||
obj = json.loads(text)
|
||||
except ValueError as ex:
|
||||
return None, "json_parse:%s" % ex
|
||||
if not isinstance(obj, dict):
|
||||
return None, "not_object:%s" % type(obj).__name__
|
||||
return obj, None
|
||||
|
||||
|
||||
def _validate_findings_shape(parsed):
|
||||
"""校验对抗输出形状：{findings:[{severity∈P0/P1/P2, issue, suggestion}]}；空数组=无问题。"""
|
||||
if parsed is None:
|
||||
return None, "输出非 JSON 对象"
|
||||
findings = parsed.get("findings")
|
||||
if not isinstance(findings, list):
|
||||
return None, "缺 findings 数组"
|
||||
for idx, item in enumerate(findings):
|
||||
if not isinstance(item, dict) or item.get("severity") not in ("P0", "P1", "P2") \
|
||||
or not isinstance(item.get("issue"), str):
|
||||
return None, "findings[%d] 形状非法：%r" % (idx, item)
|
||||
item.setdefault("suggestion", "")
|
||||
return findings, None
|
||||
|
||||
|
||||
# ---------------- 交叉模型通道（推理型专用，复用 llm_client 的重试/节流纪律）----------------
|
||||
|
||||
def chat_json_reasoning(client, user, model, max_tokens=4096, log=_log):
|
||||
"""
|
||||
推理型模型调用（deepseek-v4-flash）：在 llm_client 配方上追加 max_tokens（任务令 ≥2048），
|
||||
只读 message.content（忽略 reasoning_content）；content 为空（含被 length 截断）时
|
||||
提额至 8192 重试。重试/退避/节流纪律与 llm_client.chat_json 一致。
|
||||
"""
|
||||
body_obj = {
|
||||
"model": model,
|
||||
"temperature": llm_client.TEMPERATURE,
|
||||
"max_tokens": max_tokens,
|
||||
"response_format": {"type": "json_object"},
|
||||
"messages": [{"role": "user", "content": user}],
|
||||
}
|
||||
last_err = None
|
||||
for attempt in range(1, llm_client.MAX_RETRIES + 2):
|
||||
BUDGET.note()
|
||||
req = urllib.request.Request(client.base + "/v1/chat/completions",
|
||||
data=json.dumps(body_obj).encode("utf-8"), method="POST")
|
||||
req.add_header("Authorization", "Bearer " + client.key)
|
||||
req.add_header("Content-Type", "application/json")
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=llm_client.REQUEST_TIMEOUT_SECONDS) as resp:
|
||||
data = json.loads(resp.read().decode("utf-8"))
|
||||
choice = data.get("choices", [{}])[0]
|
||||
content = (choice.get("message") or {}).get("content") or ""
|
||||
if not content.strip():
|
||||
# 推理型常见坑：max_tokens 被 reasoning 吃光 → content 空；提额重试
|
||||
last_err = "空 content（finish=%s）" % choice.get("finish_reason")
|
||||
log("[cross] 第 %d 次尝试 content 为空，提额 max_tokens 重试" % attempt)
|
||||
body_obj["max_tokens"] = 8192
|
||||
else:
|
||||
time.sleep(llm_client.CALL_INTERVAL_SECONDS)
|
||||
return content, data
|
||||
except urllib.error.HTTPError as ex:
|
||||
detail = ""
|
||||
try:
|
||||
detail = ex.read().decode("utf-8", "replace")[:200]
|
||||
except Exception:
|
||||
pass
|
||||
last_err = "http_%s:%s" % (ex.code, detail)
|
||||
log("[cross] 第 %d 次尝试 HTTP 错误 %s" % (attempt, last_err))
|
||||
# 个别推理通道不支持 response_format → 400 时去掉该参数重试（解析侧已容忍围栏）
|
||||
if ex.code == 400 and "response_format" in body_obj:
|
||||
body_obj.pop("response_format")
|
||||
except Exception as ex:
|
||||
last_err = "err:%s" % ex
|
||||
log("[cross] 第 %d 次尝试通道异常 %s" % (attempt, last_err))
|
||||
if attempt <= llm_client.MAX_RETRIES:
|
||||
time.sleep(1.0 * (2 ** (attempt - 1)))
|
||||
raise llm_client.LlmError("交叉模型调用重试耗尽：%s" % last_err)
|
||||
|
||||
|
||||
# ---------------- 单条评审（含 raw 落盘与断点续跑）----------------
|
||||
|
||||
def review_once(store, client, entry, run_tag, model):
|
||||
"""
|
||||
对单条 Golden 条目跑一次 v1.1.0 对抗评审。
|
||||
:param run_tag: a(主跑)/stability(双跑第2次)/cross(交叉模型)——raw 文件名后缀与调用账目标签
|
||||
:return: dict{entryId, runTag, model, findings, hasP0, hasP1, passNoP0P1, llmAttempts, reused}
|
||||
"""
|
||||
raw_path = os.path.join(RAW_DIR, "%s-%s.json" % (entry["entryId"], run_tag))
|
||||
if os.path.exists(raw_path):
|
||||
# 断点续跑：已有合法产物直接复用（不再发起调用，不计预算）
|
||||
saved = json.load(open(raw_path, "r", encoding="utf-8"))
|
||||
if saved.get("findings") is not None:
|
||||
f = saved["findings"]
|
||||
return {"entryId": entry["entryId"], "runTag": run_tag, "model": saved.get("model", model),
|
||||
"findings": f,
|
||||
"hasP0": any(x["severity"] == "P0" for x in f),
|
||||
"hasP1": any(x["severity"] == "P1" for x in f),
|
||||
"passNoP0P1": not any(x["severity"] in ("P0", "P1") for x in f),
|
||||
"llmAttempts": 0, "reused": True}
|
||||
|
||||
# 渲染（public_design 口径同 run_batch._call_adversary：剥 _ 前缀键）
|
||||
public_design = {k: v for k, v in entry["design"].items() if not k.startswith("_")}
|
||||
text, version = store.render(PROMPT_ID, {"idea": entry["idea"], "game_design": public_design})
|
||||
if version != EXPECT_VERSION:
|
||||
raise RuntimeError("prompt 版本非回归对象：渲染到 %s，期望 %s" % (version, EXPECT_VERSION))
|
||||
|
||||
used_before = BUDGET.used
|
||||
last_err = None
|
||||
findings = None
|
||||
raw_content = None
|
||||
raw_response_meta = None
|
||||
for shape_try in range(1, SHAPE_RETRIES + 2):
|
||||
if model == CROSS_MODEL:
|
||||
content, full_resp = chat_json_reasoning(client, text, model)
|
||||
# 只留可审计的最小响应元信息（不落 reasoning_content 全文，防 raw 失控；usage 留作成本审计）
|
||||
raw_response_meta = {"model": full_resp.get("model"), "usage": full_resp.get("usage"),
|
||||
"finish_reason": (full_resp.get("choices") or [{}])[0].get("finish_reason")}
|
||||
else:
|
||||
content, _att = client.chat_json("", text, model=model, budget_cb=BUDGET.note, log=_log)
|
||||
parsed, perr = _parse_json_object(content)
|
||||
cand, serr = _validate_findings_shape(parsed)
|
||||
raw_content = content
|
||||
if serr is None:
|
||||
findings = cand
|
||||
break
|
||||
last_err = perr or serr
|
||||
_log("%s/%s 输出形状异常（第 %d 次）：%s" % (entry["entryId"], run_tag, shape_try, last_err))
|
||||
if findings is None:
|
||||
raise llm_client.LlmError("%s/%s 形状重试耗尽：%s" % (entry["entryId"], run_tag, last_err))
|
||||
|
||||
attempts = BUDGET.used - used_before
|
||||
# raw 落盘：promptVersion+模型+渲染输入摘要+原始 content+解析后 findings（无任何密钥）
|
||||
with open(raw_path, "w", encoding="utf-8") as fh:
|
||||
json.dump({
|
||||
"entryId": entry["entryId"], "runTag": run_tag, "model": model,
|
||||
"promptId": PROMPT_ID, "promptVersion": version,
|
||||
"temperature": llm_client.TEMPERATURE,
|
||||
"idea": entry["idea"], "designId": entry["designId"],
|
||||
"llmAttempts": attempts, "ts": time.strftime("%Y-%m-%dT%H:%M:%S"),
|
||||
"responseMeta": raw_response_meta,
|
||||
"rawContent": raw_content,
|
||||
"findings": findings,
|
||||
}, fh, ensure_ascii=False, indent=2)
|
||||
with open(CALLLOG, "a", encoding="utf-8") as fh:
|
||||
fh.write(json.dumps({"entryId": entry["entryId"], "runTag": run_tag, "model": model,
|
||||
"attempts": attempts, "budgetUsed": BUDGET.used,
|
||||
"ts": time.strftime("%H:%M:%S")}, ensure_ascii=False) + "\n")
|
||||
return {"entryId": entry["entryId"], "runTag": run_tag, "model": model, "findings": findings,
|
||||
"hasP0": any(x["severity"] == "P0" for x in findings),
|
||||
"hasP1": any(x["severity"] == "P1" for x in findings),
|
||||
"passNoP0P1": not any(x["severity"] in ("P0", "P1") for x in findings),
|
||||
"llmAttempts": attempts, "reused": False}
|
||||
|
||||
|
||||
def main():
|
||||
if not os.environ.get("NEWAPI_KEY"):
|
||||
raise SystemExit("缺 NEWAPI_KEY 环境变量（密钥严禁入盘，只走命令行环境变量）")
|
||||
os.makedirs(RAW_DIR, exist_ok=True)
|
||||
entries = json.load(open(GOLDEN, "r", encoding="utf-8"))
|
||||
assert len(entries) == 25, "Golden 集应为 25 条"
|
||||
by_id = {e["entryId"]: e for e in entries}
|
||||
|
||||
store = prompts_mod.PromptStore(PROMPTS_DIR)
|
||||
store.require([PROMPT_ID])
|
||||
client = llm_client.LlmClient(model=MAIN_MODEL)
|
||||
|
||||
results = {"promptVersion": EXPECT_VERSION, "mainModel": MAIN_MODEL, "crossModel": CROSS_MODEL,
|
||||
"temperature": llm_client.TEMPERATURE, "a": [], "d": [], "e": []}
|
||||
|
||||
# ---------------- A：25 条全量主跑（MiniMax）----------------
|
||||
_log("A 段开跑：25 条 × %s（v%s，温度 %.1f）" % (MAIN_MODEL, EXPECT_VERSION, llm_client.TEMPERATURE))
|
||||
for e in entries:
|
||||
r = review_once(store, client, e, "a", MAIN_MODEL)
|
||||
r["source"] = e["source"]
|
||||
# 写入按当前 doctrine 版本解析后的标量期望（版本化双标签见 resolve_expectation；
|
||||
# results.json 保持标量形态，summarize.py 直接消费，无需自行解析版本化结构）
|
||||
r["expectation"] = resolve_expectation(e)
|
||||
results["a"].append(r)
|
||||
_log("A %-12s hasP0=%s hasP1=%s 过=%s（%s%s）"
|
||||
% (r["entryId"], r["hasP0"], r["hasP1"], r["passNoP0P1"],
|
||||
e["source"], "·复用" if r["reused"] else ""))
|
||||
|
||||
# ---------------- D：断言③稳定性双跑（第 1 次=A 段结果，第 2 次同 prompt 同模型重打）----------------
|
||||
_log("D 段开跑：%s 双跑第 2 次" % STABILITY_SAMPLE)
|
||||
a_map = {r["entryId"]: r for r in results["a"]}
|
||||
for eid in STABILITY_SAMPLE:
|
||||
r2 = review_once(store, client, by_id[eid], "stability", MAIN_MODEL)
|
||||
r1 = a_map[eid]
|
||||
consistent = (r1["hasP0"], r1["hasP1"]) == (r2["hasP0"], r2["hasP1"])
|
||||
results["d"].append({"entryId": eid, "run1": [r1["hasP0"], r1["hasP1"]],
|
||||
"run2": [r2["hasP0"], r2["hasP1"]], "consistent": consistent,
|
||||
"findings2": r2["findings"]})
|
||||
_log("D %-12s run1=%s run2=%s 一致=%s" % (eid, results["d"][-1]["run1"],
|
||||
results["d"][-1]["run2"], consistent))
|
||||
|
||||
# ---------------- E：断言④交叉复核（deepseek-v4-flash，同 5 条）----------------
|
||||
_log("E 段开跑：%s × %s" % (STABILITY_SAMPLE, CROSS_MODEL))
|
||||
for eid in STABILITY_SAMPLE:
|
||||
rx = review_once(store, client, by_id[eid], "cross", CROSS_MODEL)
|
||||
r1 = a_map[eid]
|
||||
diverge = rx["passNoP0P1"] != r1["passNoP0P1"] # 结论口径：「无 P0/P1=过」
|
||||
results["e"].append({"entryId": eid, "miniMaxPass": r1["passNoP0P1"],
|
||||
"crossPass": rx["passNoP0P1"],
|
||||
"crossTuple": [rx["hasP0"], rx["hasP1"]],
|
||||
"diverge": diverge, "findingsCross": rx["findings"]})
|
||||
_log("E %-12s MiniMax过=%s 交叉过=%s 分歧=%s" % (eid, r1["passNoP0P1"],
|
||||
rx["passNoP0P1"], diverge))
|
||||
|
||||
results["llmCallsUsed"] = BUDGET.used
|
||||
with open(RESULTS, "w", encoding="utf-8") as fh:
|
||||
json.dump(results, fh, ensure_ascii=False, indent=2)
|
||||
_log("回归完成：LLM 调用（HTTP 尝试口径）共 %d 次，结果已落 %s" % (BUDGET.used, RESULTS))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@ -0,0 +1,35 @@
|
||||
{"entryId": "g9011", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 1, "ts": "02:48:22"}
|
||||
{"entryId": "g9012", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 2, "ts": "02:48:26"}
|
||||
{"entryId": "g9013", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 3, "ts": "02:48:31"}
|
||||
{"entryId": "g9014", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 4, "ts": "02:48:54"}
|
||||
{"entryId": "g9015", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 5, "ts": "02:49:23"}
|
||||
{"entryId": "g9016", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 6, "ts": "02:49:37"}
|
||||
{"entryId": "g9017", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 7, "ts": "02:50:00"}
|
||||
{"entryId": "k-0cfbc296", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 8, "ts": "02:50:05"}
|
||||
{"entryId": "g9019", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 9, "ts": "02:50:10"}
|
||||
{"entryId": "g9020", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 10, "ts": "02:50:16"}
|
||||
{"entryId": "g9021", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 11, "ts": "02:50:37"}
|
||||
{"entryId": "g9022", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 12, "ts": "02:50:55"}
|
||||
{"entryId": "g9023", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 13, "ts": "02:51:22"}
|
||||
{"entryId": "g9024", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 14, "ts": "02:51:26"}
|
||||
{"entryId": "k-26ce4440", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 15, "ts": "02:51:34"}
|
||||
{"entryId": "g9026", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 16, "ts": "02:51:40"}
|
||||
{"entryId": "k-a04dfac9", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 17, "ts": "02:51:46"}
|
||||
{"entryId": "g9028", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 18, "ts": "02:52:01"}
|
||||
{"entryId": "g9030", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 19, "ts": "02:52:06"}
|
||||
{"entryId": "w2-1", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 20, "ts": "02:52:10"}
|
||||
{"entryId": "w2-2", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 21, "ts": "02:52:14"}
|
||||
{"entryId": "w2-3", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 22, "ts": "02:52:27"}
|
||||
{"entryId": "w2-4", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 23, "ts": "02:52:32"}
|
||||
{"entryId": "w2-5", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 24, "ts": "02:53:08"}
|
||||
{"entryId": "w2-6", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 25, "ts": "02:53:12"}
|
||||
{"entryId": "g9019", "runTag": "stability", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 26, "ts": "02:53:20"}
|
||||
{"entryId": "g9021", "runTag": "stability", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 27, "ts": "02:53:35"}
|
||||
{"entryId": "k-0cfbc296", "runTag": "stability", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 28, "ts": "02:53:44"}
|
||||
{"entryId": "g9011", "runTag": "stability", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 29, "ts": "02:53:48"}
|
||||
{"entryId": "w2-6", "runTag": "stability", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 30, "ts": "02:53:52"}
|
||||
{"entryId": "g9019", "runTag": "cross", "model": "deepseek-v4-flash", "attempts": 1, "budgetUsed": 31, "ts": "02:54:00"}
|
||||
{"entryId": "g9021", "runTag": "cross", "model": "deepseek-v4-flash", "attempts": 1, "budgetUsed": 32, "ts": "02:54:06"}
|
||||
{"entryId": "k-0cfbc296", "runTag": "cross", "model": "deepseek-v4-flash", "attempts": 1, "budgetUsed": 33, "ts": "02:54:17"}
|
||||
{"entryId": "g9011", "runTag": "cross", "model": "deepseek-v4-flash", "attempts": 1, "budgetUsed": 34, "ts": "02:54:21"}
|
||||
{"entryId": "w2-6", "runTag": "cross", "model": "deepseek-v4-flash", "attempts": 1, "budgetUsed": 35, "ts": "02:54:26"}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "g9011",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.0",
|
||||
"temperature": 0.2,
|
||||
"idea": "王蓝莓的小卖部收银台，给顾客结账",
|
||||
"designId": "4e2de887ceed035e571db5ed0f79462f1f52d6d4944805ce7b8e65a63ce1585c",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T02:48:22",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,31 @@
|
||||
{
|
||||
"entryId": "g9011",
|
||||
"runTag": "cross",
|
||||
"model": "deepseek-v4-flash",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.0",
|
||||
"temperature": 0.2,
|
||||
"idea": "王蓝莓的小卖部收银台，给顾客结账",
|
||||
"designId": "4e2de887ceed035e571db5ed0f79462f1f52d6d4944805ce7b8e65a63ce1585c",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T02:54:21",
|
||||
"responseMeta": {
|
||||
"model": "deepseek-v4-flash",
|
||||
"usage": {
|
||||
"prompt_tokens": 854,
|
||||
"completion_tokens": 248,
|
||||
"total_tokens": 1102,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 512
|
||||
},
|
||||
"completion_tokens_details": {
|
||||
"reasoning_tokens": 241
|
||||
},
|
||||
"prompt_cache_hit_tokens": 512,
|
||||
"prompt_cache_miss_tokens": 342
|
||||
},
|
||||
"finish_reason": "stop"
|
||||
},
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "g9011",
|
||||
"runTag": "stability",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.0",
|
||||
"temperature": 0.2,
|
||||
"idea": "王蓝莓的小卖部收银台，给顾客结账",
|
||||
"designId": "4e2de887ceed035e571db5ed0f79462f1f52d6d4944805ce7b8e65a63ce1585c",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T02:53:48",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "g9012",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.0",
|
||||
"temperature": 0.2,
|
||||
"idea": "太空舱里捡星星补给能量",
|
||||
"designId": "115589c5bc4a8f572c087cc20a7373f16064eccce621cc038e458616f5485618",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T02:48:26",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\": []}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "g9013",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.0",
|
||||
"temperature": 0.2,
|
||||
"idea": "农场里收割成熟的蔬菜",
|
||||
"designId": "6f091b6da8142dda57ab8b132403c2ab62e00b5824a20a10d1156ea982bda32d",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T02:48:31",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "g9014",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.0",
|
||||
"temperature": 0.2,
|
||||
"idea": "深夜便利店招待来买夜宵的顾客",
|
||||
"designId": "a1869689d1b746b894edce5b93514dca856121f937056bb42ced6d8ea489f243",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T02:48:54",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "g9015",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.0",
|
||||
"temperature": 0.2,
|
||||
"idea": "校园运动会上的接力赛冲刺",
|
||||
"designId": "b2603e82f07a463bfec7dab27272daab1cc760e1d108bb0ae3811322c4daa24b",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T02:49:23",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "g9016",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.0",
|
||||
"temperature": 0.2,
|
||||
"idea": "猫咖啡馆里撸猫攒爱心",
|
||||
"designId": "b00027fe8f60f0d08a84c12e0bf91c25448a7d1b3a3a35b6f04d4957f2e66ad9",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T02:49:37",
|
||||
"responseMeta": null,
|
||||
"rawContent": "\n{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,21 @@
|
||||
{
|
||||
"entryId": "g9017",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.0",
|
||||
"temperature": 0.2,
|
||||
"idea": "春节集市上抢福袋",
|
||||
"designId": "43afd1f72ac6cf454faeb54cb50343e31cb06fd7ba1e07d92567debc57a05036",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T02:50:00",
|
||||
"responseMeta": null,
|
||||
"rawContent": "\n{\"findings\":[{\"severity\":\"P1\",\"issue\":\"target(15) 与 expectedPlaySeconds(45) 配比不合理：平均每3秒完成一次收集，与 designIntent 中强调的「快速点击」和「紧张刺激」节奏预期明显不符，整体难度偏低。\",\"suggestion\":\"建议提高 target 数量（如提升至30以上）或缩短 expectedPlaySeconds（如缩短至20-25秒），使平均收集间隔在1-2秒左右，以匹配快速点击的玩法意图和「抢」的紧张感。\"}]}",
|
||||
"findings": [
|
||||
{
|
||||
"severity": "P1",
|
||||
"issue": "target(15) 与 expectedPlaySeconds(45) 配比不合理：平均每3秒完成一次收集，与 designIntent 中强调的「快速点击」和「紧张刺激」节奏预期明显不符，整体难度偏低。",
|
||||
"suggestion": "建议提高 target 数量（如提升至30以上）或缩短 expectedPlaySeconds（如缩短至20-25秒），使平均收集间隔在1-2秒左右，以匹配快速点击的玩法意图和「抢」的紧张感。"
|
||||
}
|
||||
]
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "g9019",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.0",
|
||||
"temperature": 0.2,
|
||||
"idea": "烧烤摊翻烤串，别烤糊了",
|
||||
"designId": "439034c731e5530c94685dad552ddf6395eee6c00f70e7cad08451cbfa656012",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T02:50:10",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,31 @@
|
||||
{
|
||||
"entryId": "g9019",
|
||||
"runTag": "cross",
|
||||
"model": "deepseek-v4-flash",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.0",
|
||||
"temperature": 0.2,
|
||||
"idea": "烧烤摊翻烤串，别烤糊了",
|
||||
"designId": "439034c731e5530c94685dad552ddf6395eee6c00f70e7cad08451cbfa656012",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T02:54:00",
|
||||
"responseMeta": {
|
||||
"model": "deepseek-v4-flash",
|
||||
"usage": {
|
||||
"prompt_tokens": 873,
|
||||
"completion_tokens": 560,
|
||||
"total_tokens": 1433,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
},
|
||||
"completion_tokens_details": {
|
||||
"reasoning_tokens": 553
|
||||
},
|
||||
"prompt_cache_hit_tokens": 0,
|
||||
"prompt_cache_miss_tokens": 873
|
||||
},
|
||||
"finish_reason": "stop"
|
||||
},
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,21 @@
|
||||
{
|
||||
"entryId": "g9019",
|
||||
"runTag": "stability",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.0",
|
||||
"temperature": 0.2,
|
||||
"idea": "烧烤摊翻烤串，别烤糊了",
|
||||
"designId": "439034c731e5530c94685dad552ddf6395eee6c00f70e7cad08451cbfa656012",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T02:53:20",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[{\"severity\":\"P1\",\"issue\":\"designIntent 强调「避免烤糊」，但 config 中 target 仅设翻烤次数为 18，缺少失败条件或惩罚机制，题文意图明显脱节。\",\"suggestion\":\"在 config 中补充失败判定逻辑（如烤糊次数上限）或调整 target 为与「避免烤糊」意图相匹配的指标（如精准翻烤次数）。\"}]}",
|
||||
"findings": [
|
||||
{
|
||||
"severity": "P1",
|
||||
"issue": "designIntent 强调「避免烤糊」，但 config 中 target 仅设翻烤次数为 18，缺少失败条件或惩罚机制，题文意图明显脱节。",
|
||||
"suggestion": "在 config 中补充失败判定逻辑（如烤糊次数上限）或调整 target 为与「避免烤糊」意图相匹配的指标（如精准翻烤次数）。"
|
||||
}
|
||||
]
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "g9020",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.0",
|
||||
"temperature": 0.2,
|
||||
"idea": "地铁早高峰挤上即将关门的车",
|
||||
"designId": "005cde6a7d097d691745726db5fdc1b54e6e935b448857b486752e92fd1a312f",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T02:50:16",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "g9021",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.0",
|
||||
"temperature": 0.2,
|
||||
"idea": "做一个很好玩的游戏",
|
||||
"designId": "526aaaa99ab651ff89fd00dfc2b8b401e5c03b23f19b05a17b1757e526671eee",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T02:50:37",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,31 @@
|
||||
{
|
||||
"entryId": "g9021",
|
||||
"runTag": "cross",
|
||||
"model": "deepseek-v4-flash",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.0",
|
||||
"temperature": 0.2,
|
||||
"idea": "做一个很好玩的游戏",
|
||||
"designId": "526aaaa99ab651ff89fd00dfc2b8b401e5c03b23f19b05a17b1757e526671eee",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T02:54:06",
|
||||
"responseMeta": {
|
||||
"model": "deepseek-v4-flash",
|
||||
"usage": {
|
||||
"prompt_tokens": 843,
|
||||
"completion_tokens": 320,
|
||||
"total_tokens": 1163,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
},
|
||||
"completion_tokens_details": {
|
||||
"reasoning_tokens": 313
|
||||
},
|
||||
"prompt_cache_hit_tokens": 0,
|
||||
"prompt_cache_miss_tokens": 843
|
||||
},
|
||||
"finish_reason": "stop"
|
||||
},
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "g9021",
|
||||
"runTag": "stability",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.0",
|
||||
"temperature": 0.2,
|
||||
"idea": "做一个很好玩的游戏",
|
||||
"designId": "526aaaa99ab651ff89fd00dfc2b8b401e5c03b23f19b05a17b1757e526671eee",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T02:53:35",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "g9022",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.0",
|
||||
"temperature": 0.2,
|
||||
"idea": "解压点点点",
|
||||
"designId": "b52388c4e4d72182a728ef86f03bb699292d4015f108b6426218f6170d63653b",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T02:50:55",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "g9023",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.0",
|
||||
"temperature": 0.2,
|
||||
"idea": "中秋节帮嫦娥把月饼送上月宫",
|
||||
"designId": "cb3f7444796b29723b74e08807a6c94555e8f98c4a4f0c0c7cf7bd50c4c90dcb",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T02:51:22",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "g9024",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.0",
|
||||
"temperature": 0.2,
|
||||
"idea": "雨夜帮路边馄饨摊老板撑伞收摊",
|
||||
"designId": "090212e4c88d43b691aba53037bd586f1070dad5b7aeae790a785a31c351c5f5",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T02:51:26",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "g9026",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.0",
|
||||
"temperature": 0.2,
|
||||
"idea": "奶茶店爆单日疯狂摇珍珠奶茶",
|
||||
"designId": "664cf2034f2bc086f6780020f9fe02a9a1e379fc2b33daa430e34841e3e30741",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T02:51:40",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "g9028",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.0",
|
||||
"temperature": 0.2,
|
||||
"idea": "给办公室加班的同事分下午茶蛋糕",
|
||||
"designId": "53933f300e7a9c254986a2972be2c9c577b1233856336f18a20ed5a17bc6ae69",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T02:52:01",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "g9030",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.0",
|
||||
"temperature": 0.2,
|
||||
"idea": "火锅店捞毛肚，七上八下别捞老了",
|
||||
"designId": "7ebba6f04540bce08c531189024c8839c26424b84ef224ea8dfa5a64c8fa5076",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T02:52:06",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,21 @@
|
||||
{
|
||||
"entryId": "k-0cfbc296",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.0",
|
||||
"temperature": 0.2,
|
||||
"idea": "海底捞珍珠同时躲开鲨鱼",
|
||||
"designId": "0cfbc29612b6966447cf7449d6cdef8da1f57521f353bdc726b48fa0aa621048",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T02:50:05",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\n \"findings\": [\n {\n \"severity\": \"P2\",\n \"issue\": \"theme 中描述鲨鱼出现机制为'每收集3-5颗珍珠后'，触发阈值范围模糊，缺乏明确的数值或随机区间说明。\",\n \"suggestion\": \"建议将鲨鱼出现频率的描述精确化，例如'每收集3颗珍珠后鲨鱼必定出现一次，中间的间隔可随机在1-2颗之间'，提升规则的清晰度。\"\n }\n ]\n}",
|
||||
"findings": [
|
||||
{
|
||||
"severity": "P2",
|
||||
"issue": "theme 中描述鲨鱼出现机制为'每收集3-5颗珍珠后'，触发阈值范围模糊，缺乏明确的数值或随机区间说明。",
|
||||
"suggestion": "建议将鲨鱼出现频率的描述精确化，例如'每收集3颗珍珠后鲨鱼必定出现一次，中间的间隔可随机在1-2颗之间'，提升规则的清晰度。"
|
||||
}
|
||||
]
|
||||
}
|
||||
@ -0,0 +1,42 @@
|
||||
{
|
||||
"entryId": "k-0cfbc296",
|
||||
"runTag": "cross",
|
||||
"model": "deepseek-v4-flash",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.0",
|
||||
"temperature": 0.2,
|
||||
"idea": "海底捞珍珠同时躲开鲨鱼",
|
||||
"designId": "0cfbc29612b6966447cf7449d6cdef8da1f57521f353bdc726b48fa0aa621048",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T02:54:17",
|
||||
"responseMeta": {
|
||||
"model": "deepseek-v4-flash",
|
||||
"usage": {
|
||||
"prompt_tokens": 927,
|
||||
"completion_tokens": 686,
|
||||
"total_tokens": 1613,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 512
|
||||
},
|
||||
"completion_tokens_details": {
|
||||
"reasoning_tokens": 541
|
||||
},
|
||||
"prompt_cache_hit_tokens": 512,
|
||||
"prompt_cache_miss_tokens": 415
|
||||
},
|
||||
"finish_reason": "stop"
|
||||
},
|
||||
"rawContent": "{\"findings\":[{\"severity\":\"P2\",\"issue\":\"clicker模板无法支持鲨鱼定时出现及点击驱离的交互逻辑，与设计意图的玩法机制不匹配。\",\"suggestion\":\"考虑使用更灵活的模板（如自定义事件模板）或调整玩法以适配clicker的纯点击计分机制。\"},{\"severity\":\"P2\",\"issue\":\"目标值22颗珍珠在45秒内，结合每3-5颗珍珠后鲨鱼出现并可能损失3颗的机制，可能难以完成，节奏偏紧。\",\"suggestion\":\"适当降低目标值或调整损失机制（如减少损失量、增加安全期），确保挑战性与时限匹配。\"}]}",
|
||||
"findings": [
|
||||
{
|
||||
"severity": "P2",
|
||||
"issue": "clicker模板无法支持鲨鱼定时出现及点击驱离的交互逻辑，与设计意图的玩法机制不匹配。",
|
||||
"suggestion": "考虑使用更灵活的模板（如自定义事件模板）或调整玩法以适配clicker的纯点击计分机制。"
|
||||
},
|
||||
{
|
||||
"severity": "P2",
|
||||
"issue": "目标值22颗珍珠在45秒内，结合每3-5颗珍珠后鲨鱼出现并可能损失3颗的机制，可能难以完成，节奏偏紧。",
|
||||
"suggestion": "适当降低目标值或调整损失机制（如减少损失量、增加安全期），确保挑战性与时限匹配。"
|
||||
}
|
||||
]
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "k-0cfbc296",
|
||||
"runTag": "stability",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.0",
|
||||
"temperature": 0.2,
|
||||
"idea": "海底捞珍珠同时躲开鲨鱼",
|
||||
"designId": "0cfbc29612b6966447cf7449d6cdef8da1f57521f353bdc726b48fa0aa621048",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T02:53:44",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,21 @@
|
||||
{
|
||||
"entryId": "k-26ce4440",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.0",
|
||||
"temperature": 0.2,
|
||||
"idea": "图书馆闭馆前把还回来的书归架",
|
||||
"designId": "26ce44408e55eaa604b3efa2903c3cd5a541ab99ed8aa7c62c9b8192ef03260e",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T02:51:34",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[{\"severity\":\"P2\",\"issue\":\"theme 中「还回的书本」在图书馆语境下用词可更正式准确\",\"suggestion\":\"建议改为「归还的书籍」或其他更贴合图书馆场景的表述\"}]}",
|
||||
"findings": [
|
||||
{
|
||||
"severity": "P2",
|
||||
"issue": "theme 中「还回的书本」在图书馆语境下用词可更正式准确",
|
||||
"suggestion": "建议改为「归还的书籍」或其他更贴合图书馆场景的表述"
|
||||
}
|
||||
]
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "k-a04dfac9",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.0",
|
||||
"temperature": 0.2,
|
||||
"idea": "跨年夜零点在广场上点烟花倒计时",
|
||||
"designId": "a04dfac9bba70833a1dcbce8242a787f3180db5b516b65006589698878eabcc6",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T02:51:46",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,21 @@
|
||||
{
|
||||
"entryId": "w2-1",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.0",
|
||||
"temperature": 0.2,
|
||||
"idea": "王蓝莓的小卖部收银台，给顾客结账",
|
||||
"designId": "8f9108324a3bef55f471773b697a48047cf4c5b70b46a97a33696a5e4ebc39a1",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T02:52:10",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[{\"severity\":\"P1\",\"issue\":\"title 中角色名为“蓝莓妹”，与创意原文的“王蓝莓”不一致\",\"suggestion\":\"将 title 改为与创意原文一致的角色名称\"}]}",
|
||||
"findings": [
|
||||
{
|
||||
"severity": "P1",
|
||||
"issue": "title 中角色名为“蓝莓妹”，与创意原文的“王蓝莓”不一致",
|
||||
"suggestion": "将 title 改为与创意原文一致的角色名称"
|
||||
}
|
||||
]
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "w2-2",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.0",
|
||||
"temperature": 0.2,
|
||||
"idea": "深夜便利店招待来买夜宵的顾客",
|
||||
"designId": "b62350bae955d7c3a510efa0eed0f74c7408ba892b8f0b1e59a367d6456a8425",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T02:52:14",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "w2-3",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.0",
|
||||
"temperature": 0.2,
|
||||
"idea": "猫咖啡馆里撸猫攒爱心",
|
||||
"designId": "2b15bd30b466f12653be4dac5d53ac5b856110986e00d6926c62e8c3ece809c7",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T02:52:27",
|
||||
"responseMeta": null,
|
||||
"rawContent": "\n{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,21 @@
|
||||
{
|
||||
"entryId": "w2-4",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.0",
|
||||
"temperature": 0.2,
|
||||
"idea": "烧烤摊翻烤串，别烤糊了",
|
||||
"designId": "1b3c5764b01c161d1c0cbd6aaba2c7375280b2469a850152b6596a22b55d7389",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T02:52:32",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[{\"severity\":\"P1\",\"issue\":\"config 缺失创意核心元素「烤糊」惩罚机制，创意原文强调「别烤糊了」的紧迫感，但 config 中无任何失败条件、时间压力或烤糊风险描述，与玩法意图明显脱节\",\"suggestion\":\"在 config 中补充失败机制（如计时限制、操作失误导致烤糊）或至少在 theme 文案中体现「烤糊」的危机感，使题文一致\"}]}",
|
||||
"findings": [
|
||||
{
|
||||
"severity": "P1",
|
||||
"issue": "config 缺失创意核心元素「烤糊」惩罚机制，创意原文强调「别烤糊了」的紧迫感，但 config 中无任何失败条件、时间压力或烤糊风险描述，与玩法意图明显脱节",
|
||||
"suggestion": "在 config 中补充失败机制（如计时限制、操作失误导致烤糊）或至少在 theme 文案中体现「烤糊」的危机感，使题文一致"
|
||||
}
|
||||
]
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "w2-5",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.0",
|
||||
"temperature": 0.2,
|
||||
"idea": "地铁早高峰挤上即将关门的车",
|
||||
"designId": "feca768c87f9fcb3111a504ab074f8e998129c9de01e833957f4039c6ffbef5a",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T02:53:08",
|
||||
"responseMeta": null,
|
||||
"rawContent": "\n{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "w2-6",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.0",
|
||||
"temperature": 0.2,
|
||||
"idea": "解压点点点",
|
||||
"designId": "d0b8c8ea60d5d07c220e1c903f8e78ed36d21ad816b4ee82d2dae8d13af0388a",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T02:53:12",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,31 @@
|
||||
{
|
||||
"entryId": "w2-6",
|
||||
"runTag": "cross",
|
||||
"model": "deepseek-v4-flash",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.0",
|
||||
"temperature": 0.2,
|
||||
"idea": "解压点点点",
|
||||
"designId": "d0b8c8ea60d5d07c220e1c903f8e78ed36d21ad816b4ee82d2dae8d13af0388a",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T02:54:26",
|
||||
"responseMeta": {
|
||||
"model": "deepseek-v4-flash",
|
||||
"usage": {
|
||||
"prompt_tokens": 838,
|
||||
"completion_tokens": 196,
|
||||
"total_tokens": 1034,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 512
|
||||
},
|
||||
"completion_tokens_details": {
|
||||
"reasoning_tokens": 189
|
||||
},
|
||||
"prompt_cache_hit_tokens": 512,
|
||||
"prompt_cache_miss_tokens": 326
|
||||
},
|
||||
"finish_reason": "stop"
|
||||
},
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "w2-6",
|
||||
"runTag": "stability",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.0",
|
||||
"temperature": 0.2,
|
||||
"idea": "解压点点点",
|
||||
"designId": "d0b8c8ea60d5d07c220e1c903f8e78ed36d21ad816b4ee82d2dae8d13af0388a",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T02:53:52",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,113 @@
|
||||
# Golden 集回归报告 —— quality.adversary-review v1.1.0
|
||||
|
||||
> 日期：2026-06-10 · 回归对象：`contracts/prompts/06-quality/adversary-review.md` v1.1.0（batch-001 冻结告警的根治修订）
|
||||
> 通道：new-api（100.64.0.8:3000）· 主模型 MiniMax-M2.7（json_object/温度0.2，与 R8 复核口径一致）· 交叉模型 deepseek-v4-flash（推理型，max_tokens≥2048，只读 content）
|
||||
> LLM 调用合计 **35 次**（HTTP 尝试口径，零重试，预算 ≤70 内）· 渲染复用 orchestrator/prompts.py · 通道复用 orchestrator/llm_client.py
|
||||
|
||||
## 一、结论先行
|
||||
|
||||
| 断言 | 阈值 | 实测 | 判定 |
|
||||
|---|---|---|---|
|
||||
| ① 不放水（3 条 p1_residual kill 仍含 P1） | 3/3 | **0/3 —— 三条全部翻干净** | **红灯·不通过** |
|
||||
| ② 误报收敛（16 条原 accept 的 P1 误报） | ≤2 且 9019/9021 收敛 | **1/16**（9019、9021 均不再出 P1；唯一新误报 = g9017） | 通过 |
|
||||
| ③ 稳定性（5 条双跑 P0/P1 判定一致率） | ≥80% | **4/5 = 80%（压线）**；翻转条目 = g9019 | 通过 |
|
||||
| ④ 交叉（deepseek-v4-flash 与 MiniMax 结论分歧） | ≤1 条 | **0/5** | 通过 |
|
||||
|
||||
**解冻建议：不建议解冻。** 解冻条件为①②③全过，断言①红灯：v1.1.0 在收敛误报的同时把三条 p1_residual kill 的拦截能力一并洗掉了——按任务令「任一变干净=红灯，如实报」如实上报。
|
||||
|
||||
A 段执行面：25/25 全部产出合法 findings（无通道失败、无形状异常、零重试）；按「无 P0/P1=过」判定为 22 过 / 3 不过（g9017、w2-1、w2-4，逐条见明细）。
|
||||
|
||||
## 二、红灯分析（断言①为什么挂）
|
||||
|
||||
三条 kill 的原 P1 锚点与新跑结果：
|
||||
|
||||
| 条目 | 原 round1 P1 锚点（v1.0.0） | v1.1.0 新跑结果 | 定性 |
|
||||
|---|---|---|---|
|
||||
| k-26ce4440（图书馆闭馆归架） | designIntent「有限时间内」承诺计时机制，config 无计时 | 仅 1×P2（用词措辞） | **细则①字面口径下的预期内降级**：锚点恰为「designIntent 越模板机制 + config 字段无自身矛盾」，按细则①只能记 P2 |
|
||||
| k-a04dfac9（跨年夜点烟花倒计时） | 「倒计时紧张感」与 target=15 连点脱节 | 0 findings | 同上：倒计时承诺属越模板机制，且 idea 本身含「倒计时」、题文不冲突，细则①+②叠加后无 P1 落点 |
|
||||
| k-0cfbc296（海底捞珍珠躲鲨鱼） | idea「**躲开**鲨鱼」↔ theme「**点击驱离**鲨鱼」语义矛盾（题文不符） | 仅 1×P2（鲨鱼出现阈值表述模糊）；双跑第 2 次同样无 P1；交叉模型也仅 2×P2 | **真实漏报（放水实证）**：该矛盾属 P1 第一条「题文不符」，不在细则①遮蔽范围内，模型把「theme 机制类文案宽容」过度泛化，吞掉了创意核心动词的矛盾 |
|
||||
|
||||
要点：
|
||||
|
||||
1. **结构性冲突暴露**：k-26ce4440 / k-a04dfac9 两条的原 P1 与 9019 的误报是**同一模式**（designIntent/theme 承诺模板外机制）。v1.0.0 把好设计（9019）和坏设计（kills）一起拦；v1.1.0 细则①把该模式统一降级后，两者一起放——**在该模式上「9019 收敛」与「kill 仍 kill」对评审 prompt 是结构性不可兼得**，黄金集这 2 条 kill 标签需要主 agent 重新裁决（注：设计师 prompt v1.1.0 已禁止 designIntent 越模板承诺，未来新设计不会再产出该模式，这 2 条属遗留模式样本）。
|
||||
2. **k-0cfbc296 是无可辩护的漏报**：它有独立于越模板模式的 P1 锚点（躲避↔主动点击驱离的题文矛盾），v1.1.0 正文从未豁免这一类，但 MiniMax 两跑 + deepseek 交叉共三次全部漏掉。细则①的「config 机制类文案宽容」措辞对模型产生了超范围的抑制作用。
|
||||
3. **细则①压制本身也不稳固**：g9019 双跑第 2 次重新出 P1（「避免烤糊」缺失败机制——正是细则①要压制的模式），说明温度 0.2 下细则约束的执行仍有随机翻转；本次 D 段 80% 压线通过，余量为零。
|
||||
|
||||
## 三、误报明细（断言②口径=16 条 batch-001 accept）
|
||||
|
||||
| 条目 | 新 P1 内容 | 评估 |
|
||||
|---|---|---|
|
||||
| g9017（春节集市抢福袋） | target=15 与 expectedPlaySeconds=45 配比（平均 3 秒一次点击）与「快速点击/抢」意图不符 | 新增误报（原 accept 已实玩通过）；属「target 与玩法意图自洽性」的边缘判断，计入 ≤2 预算内 |
|
||||
| g9019 / g9021 | 无 P1 | **R8 两条分歧条目按目标收敛**（但 g9019 双跑第 2 次翻回 P1，收敛不稳，见红灯分析第 3 点） |
|
||||
|
||||
W2 段 2 条「不过」不计入断言②（口径仅覆盖 16 条 batch-001 accept），成因照实记录：
|
||||
|
||||
- **w2-1**：P1=「蓝莓妹」与创意「王蓝莓」不一致——此为 W2 试点按 CEO 决策#10 故意去 IP 换皮的结果，评审 prompt 无法知晓合规决策，行为本身合理；后续可在评审输入侧附带「合规改名豁免」上下文解决。
|
||||
- **w2-4**：P1=「config 缺『烤糊』失败机制」——模型反向要求补模板外失败机制，与细则①精神相悖，再次印证细则约束不稳固。
|
||||
|
||||
## 四、逐条明细表（A 段 25 条）
|
||||
|
||||
| 条目 | 来源 | 创意 | 标题 | 原判定 | 新 P0/P1/P2 | 无P0/P1=过 | 与预期 |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| g9011 | B1 accept | 王蓝莓的小卖部收银台，给顾客结账 | 小卖部·收银时刻 | accept | 0/0/0 | 过 | 符合 |
|
||||
| g9012 | B1 accept | 太空舱里捡星星补给能量 | 星际拾星员 | accept | 0/0/0 | 过 | 符合 |
|
||||
| g9013 | B1 accept | 农场里收割成熟的蔬菜 | 绿野采收忙 | accept | 0/0/0 | 过 | 符合 |
|
||||
| g9014 | B1 accept | 深夜便利店招待来买夜宵的顾客 | 午夜便利店·夜宵速递 | accept | 0/0/0 | 过 | 符合 |
|
||||
| g9015 | B1 accept | 校园运动会上的接力赛冲刺 | 操场冲刺·冲刺荣耀 | accept | 0/0/0 | 过 | 符合 |
|
||||
| g9016 | B1 accept | 猫咖啡馆里撸猫攒爱心 | 猫咖轻抚·爱心满溢 | accept | 0/0/0 | 过 | 符合 |
|
||||
| g9017 | B1 accept | 春节集市上抢福袋 | 迎春纳福·福袋满仓 | accept | 0/1/0 | **不过** | **偏离**（新误报） |
|
||||
| k-0cfbc296 | B1 kill | 海底捞珍珠同时躲开鲨鱼 | 深海采珠记 | kill(p1残留) | 0/0/1 | 过 | **偏离**（真实漏报） |
|
||||
| g9019 | B1 accept | 烧烤摊翻烤串，别烤糊了 | 炭火翻串记 | accept | 0/0/0 | 过 | 符合（R8 分歧条目收敛） |
|
||||
| g9020 | B1 accept | 地铁早高峰挤上即将关门的车 | 站台冲刺·车门将闭 | accept | 0/0/0 | 过 | 符合 |
|
||||
| g9021 | B1 accept | 做一个很好玩的游戏 | 糖果屋的甜蜜收集 | accept | 0/0/0 | 过 | 符合（R8 分歧条目收敛，细则②生效） |
|
||||
| g9022 | B1 accept | 解压点点点 | 弹力球解压工作室 | accept | 0/0/0 | 过 | 符合 |
|
||||
| g9023 | B1 accept | 中秋节帮嫦娥把月饼送上月宫 | 月宫献饼记 | accept | 0/0/0 | 过 | 符合 |
|
||||
| g9024 | B1 accept | 雨夜帮路边馄饨摊老板撑伞收摊 | 雨夜馄饨摊·温情收摊 | accept | 0/0/0 | 过 | 符合 |
|
||||
| k-26ce4440 | B1 kill | 图书馆闭馆前把还回来的书归架 | 闭馆归架记 | kill(p1残留) | 0/0/1 | 过 | **偏离**（细则①预期内降级，标签需复审） |
|
||||
| g9026 | B1 accept | 奶茶店爆单日疯狂摇珍珠奶茶 | 茶铺高峰·手摇风暴 | accept | 0/0/0 | 过 | 符合 |
|
||||
| k-a04dfac9 | B1 kill | 跨年夜零点在广场上点烟花倒计时 | 午夜庆典·焰火绽放 | kill(p1残留) | 0/0/0 | 过 | **偏离**（细则①预期内降级，标签需复审） |
|
||||
| g9028 | B1 accept | 给办公室加班的同事分下午茶蛋糕 | 甜蜜分送 | accept | 0/0/0 | 过 | 符合 |
|
||||
| g9030 | B1 accept | 火锅店捞毛肚，七上八下别捞老了 | 沸腾捞肚 | accept | 0/0/0 | 过 | 符合 |
|
||||
| w2-1 | W2 | 王蓝莓的小卖部收银台，给顾客结账 | 蓝莓妹的小卖部·结账高峰 | W2 入池 | 0/1/0 | **不过** | **偏离**（IP 合规换皮所致，已知成因） |
|
||||
| w2-2 | W2 | 深夜便利店招待来买夜宵的顾客 | 凌晨两点的便利店 | W2 入池 | 0/0/0 | 过 | 符合 |
|
||||
| w2-3 | W2 | 猫咖啡馆里撸猫攒爱心 | 猫咖时光·撸猫集心 | W2 入池 | 0/0/0 | 过 | 符合 |
|
||||
| w2-4 | W2 | 烧烤摊翻烤串，别烤糊了 | 烤串二十翻 | W2 入池 | 0/1/0 | **不过** | **偏离**（模型反向索要模板外失败机制） |
|
||||
| w2-5 | W2 | 地铁早高峰挤上即将关门的车 | 车门即将关闭 | W2 入池 | 0/0/0 | 过 | 符合 |
|
||||
| w2-6 | W2 | 解压点点点 | 指尖戳戳乐 | W2 入池 | 0/0/0 | 过 | 符合（模糊创意类，细则②生效） |
|
||||
|
||||
### 断言③双跑明细（同 prompt 同模型 MiniMax-M2.7，run1=A 段，run2=独立重打）
|
||||
|
||||
| 条目 | run1 (P0,P1) | run2 (P0,P1) | 一致 |
|
||||
|---|---|---|---|
|
||||
| g9019 | (False, False) | (False, **True**) | **否** |
|
||||
| g9021 | (False, False) | (False, False) | 是 |
|
||||
| k-0cfbc296 | (False, False) | (False, False) | 是 |
|
||||
| g9011 | (False, False) | (False, False) | 是 |
|
||||
| w2-6 | (False, False) | (False, False) | 是 |
|
||||
|
||||
抽样说明：5 条为定向抽样——两条 R8 分歧原条目（g9019/g9021）+ 1 条 p1_residual kill（k-0cfbc296）+ 1 条普通 accept（g9011）+ 1 条 W2 模糊创意（w2-6），覆盖全部分歧高危类别；一致性口径与 run_batch 批末稳定性重测一致（(hasP0,hasP1) 元组翻转即不一致）。
|
||||
|
||||
### 断言④交叉明细（deepseek-v4-flash vs MiniMax，结论口径=「无 P0/P1=过」）
|
||||
|
||||
| 条目 | MiniMax 过 | 交叉过 | 交叉 (P0,P1) | 分歧 |
|
||||
|---|---|---|---|---|
|
||||
| g9019 | 过 | 过 | (False, False) | 否 |
|
||||
| g9021 | 过 | 过 | (False, False) | 否 |
|
||||
| k-0cfbc296 | 过 | 过 | (False, False) | 否 |
|
||||
| g9011 | 过 | 过 | (False, False) | 否 |
|
||||
| w2-6 | 过 | 过 | (False, False) | 否 |
|
||||
|
||||
注：交叉模型在 k-0cfbc296 上同样只出 2×P2（其中一条明确点出「clicker 模板无法支持鲨鱼交互逻辑」）——kill 翻绿是**两模型一致的口径性结果**，非单模型抖动；这把问题进一步钉在细则措辞/黄金集标签上，而非通道。
|
||||
|
||||
## 五、Golden 集口径与产物
|
||||
|
||||
- Golden 集 = batch-001 账本 16 条 accept（裁决轮 design）+ 3 条 p1_residual kill 的 round1 design（fix 事件原文）+ W2 试点 6 条（accepted-configs.json finalConfig 包装，designIntent 按 pilot-results.md 对应行大意撰写、仅模板内机制，expectedPlaySeconds 取该行估算区间内整数），共 25 条。
|
||||
- 产物清单（本目录）：`build_golden_set.py`（黄金集构建）/ `golden-set.json`（25 条输入）/ `golden_regression.py`（回归主脚本，断点续跑+预算硬闸 70）/ `raw/`（35 份原始响应，均含 promptVersion/温度；usage 仅交叉 5 份的 responseMeta 携带——主通道复用 orchestrator/llm_client.py 只回传 content/attempts、不含 usage，无密钥）/ `llm-calls.jsonl`（逐调用账目）/ `results.json`（四断言原始数据）/ `summarize.py`（汇总器）/ `run.log`（运行日志）。
|
||||
- 密钥纪律：NEWAPI_KEY 仅经命令行环境变量注入，未写入任何盘上文件（已核 raw/results/脚本均不含密钥）。
|
||||
|
||||
## 六、后续动作建议（供主 agent 裁决，均不触碰 P0 定义与 P1 口径文字）
|
||||
|
||||
1. **v1.1.1 再修订（走四道闸）**：细则①补一条适用边界——「theme/config 文案中的机制描述与创意核心动作（如『躲开』↔『点击驱离』）直接矛盾时，仍按 P1 第一条『题文不符』判定，不因机制类文案宽容而豁免」；该补法是新增判定细则，不改动既有口径文字，符合铁律。
|
||||
2. **黄金集 kill 标签复审**：k-26ce4440 / k-a04dfac9 的原 P1 与 9019 误报同模式，在细则①下结构性翻绿；主 agent 需裁决是改黄金集预期（按新口径重标）还是收紧细则措辞。在裁决前，断言①以现行任务令口径记红灯。
|
||||
3. **稳定性余量为零**：D=80% 压线 + g9019 双跑翻转方向恰为细则①压制对象，建议 v1.1.1 同时考虑流程性稳定化（如细则③补「checklist 逐条核对时对细则①②先行裁剪再定级」），并在下一批保留 Z4 批末重测观察。
|
||||
4. **解冻**：维持冻结，待 v1.1.1 + 黄金集标签裁决后重跑本回归（脚本支持断点续跑，重跑成本 ≤35 次调用）。
|
||||
@ -0,0 +1,505 @@
|
||||
{
|
||||
"promptVersion": "1.1.0",
|
||||
"mainModel": "MiniMax-M2.7",
|
||||
"crossModel": "deepseek-v4-flash",
|
||||
"temperature": 0.2,
|
||||
"a": [
|
||||
{
|
||||
"entryId": "g9011",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 1,
|
||||
"reused": false,
|
||||
"source": "batch-001-accept",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "g9012",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 1,
|
||||
"reused": false,
|
||||
"source": "batch-001-accept",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "g9013",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 1,
|
||||
"reused": false,
|
||||
"source": "batch-001-accept",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "g9014",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 1,
|
||||
"reused": false,
|
||||
"source": "batch-001-accept",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "g9015",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 1,
|
||||
"reused": false,
|
||||
"source": "batch-001-accept",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "g9016",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 1,
|
||||
"reused": false,
|
||||
"source": "batch-001-accept",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "g9017",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [
|
||||
{
|
||||
"severity": "P1",
|
||||
"issue": "target(15) 与 expectedPlaySeconds(45) 配比不合理：平均每3秒完成一次收集，与 designIntent 中强调的「快速点击」和「紧张刺激」节奏预期明显不符，整体难度偏低。",
|
||||
"suggestion": "建议提高 target 数量（如提升至30以上）或缩短 expectedPlaySeconds（如缩短至20-25秒），使平均收集间隔在1-2秒左右，以匹配快速点击的玩法意图和「抢」的紧张感。"
|
||||
}
|
||||
],
|
||||
"hasP0": false,
|
||||
"hasP1": true,
|
||||
"passNoP0P1": false,
|
||||
"llmAttempts": 1,
|
||||
"reused": false,
|
||||
"source": "batch-001-accept",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "k-0cfbc296",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [
|
||||
{
|
||||
"severity": "P2",
|
||||
"issue": "theme 中描述鲨鱼出现机制为'每收集3-5颗珍珠后'，触发阈值范围模糊，缺乏明确的数值或随机区间说明。",
|
||||
"suggestion": "建议将鲨鱼出现频率的描述精确化，例如'每收集3颗珍珠后鲨鱼必定出现一次，中间的间隔可随机在1-2颗之间'，提升规则的清晰度。"
|
||||
}
|
||||
],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 1,
|
||||
"reused": false,
|
||||
"source": "batch-001-p1residual-kill",
|
||||
"expectation": "has_p1"
|
||||
},
|
||||
{
|
||||
"entryId": "g9019",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 1,
|
||||
"reused": false,
|
||||
"source": "batch-001-accept",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "g9020",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 1,
|
||||
"reused": false,
|
||||
"source": "batch-001-accept",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "g9021",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 1,
|
||||
"reused": false,
|
||||
"source": "batch-001-accept",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "g9022",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 1,
|
||||
"reused": false,
|
||||
"source": "batch-001-accept",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "g9023",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 1,
|
||||
"reused": false,
|
||||
"source": "batch-001-accept",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "g9024",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 1,
|
||||
"reused": false,
|
||||
"source": "batch-001-accept",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "k-26ce4440",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [
|
||||
{
|
||||
"severity": "P2",
|
||||
"issue": "theme 中「还回的书本」在图书馆语境下用词可更正式准确",
|
||||
"suggestion": "建议改为「归还的书籍」或其他更贴合图书馆场景的表述"
|
||||
}
|
||||
],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 1,
|
||||
"reused": false,
|
||||
"source": "batch-001-p1residual-kill",
|
||||
"expectation": "has_p1"
|
||||
},
|
||||
{
|
||||
"entryId": "g9026",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 1,
|
||||
"reused": false,
|
||||
"source": "batch-001-accept",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "k-a04dfac9",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 1,
|
||||
"reused": false,
|
||||
"source": "batch-001-p1residual-kill",
|
||||
"expectation": "has_p1"
|
||||
},
|
||||
{
|
||||
"entryId": "g9028",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 1,
|
||||
"reused": false,
|
||||
"source": "batch-001-accept",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "g9030",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 1,
|
||||
"reused": false,
|
||||
"source": "batch-001-accept",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "w2-1",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [
|
||||
{
|
||||
"severity": "P1",
|
||||
"issue": "title 中角色名为“蓝莓妹”，与创意原文的“王蓝莓”不一致",
|
||||
"suggestion": "将 title 改为与创意原文一致的角色名称"
|
||||
}
|
||||
],
|
||||
"hasP0": false,
|
||||
"hasP1": true,
|
||||
"passNoP0P1": false,
|
||||
"llmAttempts": 1,
|
||||
"reused": false,
|
||||
"source": "w2-pilot",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "w2-2",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 1,
|
||||
"reused": false,
|
||||
"source": "w2-pilot",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "w2-3",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 1,
|
||||
"reused": false,
|
||||
"source": "w2-pilot",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "w2-4",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [
|
||||
{
|
||||
"severity": "P1",
|
||||
"issue": "config 缺失创意核心元素「烤糊」惩罚机制，创意原文强调「别烤糊了」的紧迫感，但 config 中无任何失败条件、时间压力或烤糊风险描述，与玩法意图明显脱节",
|
||||
"suggestion": "在 config 中补充失败机制（如计时限制、操作失误导致烤糊）或至少在 theme 文案中体现「烤糊」的危机感，使题文一致"
|
||||
}
|
||||
],
|
||||
"hasP0": false,
|
||||
"hasP1": true,
|
||||
"passNoP0P1": false,
|
||||
"llmAttempts": 1,
|
||||
"reused": false,
|
||||
"source": "w2-pilot",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "w2-5",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 1,
|
||||
"reused": false,
|
||||
"source": "w2-pilot",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "w2-6",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 1,
|
||||
"reused": false,
|
||||
"source": "w2-pilot",
|
||||
"expectation": "no_p0p1"
|
||||
}
|
||||
],
|
||||
"d": [
|
||||
{
|
||||
"entryId": "g9019",
|
||||
"run1": [
|
||||
false,
|
||||
false
|
||||
],
|
||||
"run2": [
|
||||
false,
|
||||
true
|
||||
],
|
||||
"consistent": false,
|
||||
"findings2": [
|
||||
{
|
||||
"severity": "P1",
|
||||
"issue": "designIntent 强调「避免烤糊」，但 config 中 target 仅设翻烤次数为 18，缺少失败条件或惩罚机制，题文意图明显脱节。",
|
||||
"suggestion": "在 config 中补充失败判定逻辑（如烤糊次数上限）或调整 target 为与「避免烤糊」意图相匹配的指标（如精准翻烤次数）。"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"entryId": "g9021",
|
||||
"run1": [
|
||||
false,
|
||||
false
|
||||
],
|
||||
"run2": [
|
||||
false,
|
||||
false
|
||||
],
|
||||
"consistent": true,
|
||||
"findings2": []
|
||||
},
|
||||
{
|
||||
"entryId": "k-0cfbc296",
|
||||
"run1": [
|
||||
false,
|
||||
false
|
||||
],
|
||||
"run2": [
|
||||
false,
|
||||
false
|
||||
],
|
||||
"consistent": true,
|
||||
"findings2": []
|
||||
},
|
||||
{
|
||||
"entryId": "g9011",
|
||||
"run1": [
|
||||
false,
|
||||
false
|
||||
],
|
||||
"run2": [
|
||||
false,
|
||||
false
|
||||
],
|
||||
"consistent": true,
|
||||
"findings2": []
|
||||
},
|
||||
{
|
||||
"entryId": "w2-6",
|
||||
"run1": [
|
||||
false,
|
||||
false
|
||||
],
|
||||
"run2": [
|
||||
false,
|
||||
false
|
||||
],
|
||||
"consistent": true,
|
||||
"findings2": []
|
||||
}
|
||||
],
|
||||
"e": [
|
||||
{
|
||||
"entryId": "g9019",
|
||||
"miniMaxPass": true,
|
||||
"crossPass": true,
|
||||
"crossTuple": [
|
||||
false,
|
||||
false
|
||||
],
|
||||
"diverge": false,
|
||||
"findingsCross": []
|
||||
},
|
||||
{
|
||||
"entryId": "g9021",
|
||||
"miniMaxPass": true,
|
||||
"crossPass": true,
|
||||
"crossTuple": [
|
||||
false,
|
||||
false
|
||||
],
|
||||
"diverge": false,
|
||||
"findingsCross": []
|
||||
},
|
||||
{
|
||||
"entryId": "k-0cfbc296",
|
||||
"miniMaxPass": true,
|
||||
"crossPass": true,
|
||||
"crossTuple": [
|
||||
false,
|
||||
false
|
||||
],
|
||||
"diverge": false,
|
||||
"findingsCross": [
|
||||
{
|
||||
"severity": "P2",
|
||||
"issue": "clicker模板无法支持鲨鱼定时出现及点击驱离的交互逻辑，与设计意图的玩法机制不匹配。",
|
||||
"suggestion": "考虑使用更灵活的模板（如自定义事件模板）或调整玩法以适配clicker的纯点击计分机制。"
|
||||
},
|
||||
{
|
||||
"severity": "P2",
|
||||
"issue": "目标值22颗珍珠在45秒内，结合每3-5颗珍珠后鲨鱼出现并可能损失3颗的机制，可能难以完成，节奏偏紧。",
|
||||
"suggestion": "适当降低目标值或调整损失机制（如减少损失量、增加安全期），确保挑战性与时限匹配。"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"entryId": "g9011",
|
||||
"miniMaxPass": true,
|
||||
"crossPass": true,
|
||||
"crossTuple": [
|
||||
false,
|
||||
false
|
||||
],
|
||||
"diverge": false,
|
||||
"findingsCross": []
|
||||
},
|
||||
{
|
||||
"entryId": "w2-6",
|
||||
"miniMaxPass": true,
|
||||
"crossPass": true,
|
||||
"crossTuple": [
|
||||
false,
|
||||
false
|
||||
],
|
||||
"diverge": false,
|
||||
"findingsCross": []
|
||||
}
|
||||
],
|
||||
"llmCallsUsed": 35
|
||||
}
|
||||
@ -0,0 +1,100 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
回归结果汇总器（第 3 步）：读 golden-set.json + results.json，产出四断言判定数据与
|
||||
report.md 用的逐条明细表（markdown 片段落 /tmp/report-tables.md，由报告撰写步骤引用）。
|
||||
只做统计与表格渲染，不发起任何 LLM 调用；断言阈值与任务令一致。
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
|
||||
_HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
gs = json.load(open(os.path.join(_HERE, "golden-set.json"), encoding="utf-8"))
|
||||
rs = json.load(open(os.path.join(_HERE, "results.json"), encoding="utf-8"))
|
||||
by_id = {e["entryId"]: e for e in gs}
|
||||
a_map = {r["entryId"]: r for r in rs["a"]}
|
||||
|
||||
out = []
|
||||
|
||||
# ---- 断言①：p1_residual kill 不放水守卫 ----
|
||||
# 2026-06-10 黄金集标签版本化（主 agent 裁决②）后改为按 results.json 中已按 doctrine 解析的
|
||||
# expectation 标量分组（golden_regression.resolve_expectation 产物），不再按 source 硬编码"3 条"：
|
||||
# - expectation=="has_p1" 的 kill 条目 = 不放水负例，须全部仍含 P1（旧 results.json 三条全为
|
||||
# has_p1，行为与改前一致：0/3 仍判不通过）；
|
||||
# - kill 来源但被重标为 no_p0p1 的条目（doctrine>=v1.1.0 下的 k-26ce4440/k-a04dfac9）= 细则①
|
||||
# 正向守卫，再出 P0/P1 即细则①失效，同样阻断断言①。
|
||||
kills = [r for r in rs["a"] if r["expectation"] == "has_p1"]
|
||||
b_keep = [r for r in kills if r["hasP1"]]
|
||||
guards = [r for r in rs["a"] if r["source"] == "batch-001-p1residual-kill" and r["expectation"] == "no_p0p1"]
|
||||
guard_bad = [r for r in guards if not r["passNoP0P1"]]
|
||||
b_pass = len(b_keep) == len(kills) and not guard_bad
|
||||
out.append(("B", "不放水 %d/%d 仍含P1；细则①正向守卫 %d 条中 %d 条失守"
|
||||
% (len(b_keep), len(kills), len(guards), len(guard_bad)), b_pass))
|
||||
|
||||
# ---- 断言②：16 条原 accept 的 P1 误报 ≤2 且 9019/9021 无 P1 ----
|
||||
accepts = [r for r in rs["a"] if r["source"] == "batch-001-accept"]
|
||||
fp = [r for r in accepts if r["hasP0"] or r["hasP1"]]
|
||||
g9019_clean = not (a_map["g9019"]["hasP0"] or a_map["g9019"]["hasP1"])
|
||||
g9021_clean = not (a_map["g9021"]["hasP0"] or a_map["g9021"]["hasP1"])
|
||||
c_pass = len(fp) <= 2 and g9019_clean and g9021_clean
|
||||
out.append(("C", "%d/16 误报（9019 %s，9021 %s）" % (len(fp), "已收敛" if g9019_clean else "仍误报",
|
||||
"已收敛" if g9021_clean else "仍误报"), c_pass))
|
||||
|
||||
# ---- 断言③：双跑一致率 ≥80% ----
|
||||
cons = sum(1 for d in rs["d"] if d["consistent"])
|
||||
d_rate = cons / float(len(rs["d"])) if rs["d"] else 0.0
|
||||
d_pass = d_rate >= 0.8
|
||||
out.append(("D", "%d/%d=%.0f%%" % (cons, len(rs["d"]), d_rate * 100), d_pass))
|
||||
|
||||
# ---- 断言④：交叉分歧 ≤1 ----
|
||||
div = sum(1 for e in rs["e"] if e["diverge"])
|
||||
e_pass = div <= 1
|
||||
out.append(("E", "分歧 %d/%d" % (div, len(rs["e"])), e_pass))
|
||||
|
||||
# ---- A 段：25/25 有效产出 + 通过分布 ----
|
||||
a_ok = len(rs["a"]) == 25
|
||||
n_pass = sum(1 for r in rs["a"] if r["passNoP0P1"])
|
||||
print("A: 25 条有效产出=%s，按「无 P0/P1=过」: 过 %d / 不过 %d" % (a_ok, n_pass, 25 - n_pass))
|
||||
for tag, detail, ok in out:
|
||||
print("断言%s: %s → %s" % (tag, detail, "通过" if ok else "不通过"))
|
||||
print("LLM 调用（HTTP 尝试口径）:", rs["llmCallsUsed"])
|
||||
print("解冻建议（①②③全过）:", all(x[2] for x in out[:3]))
|
||||
|
||||
# ---- 明细表 markdown ----
|
||||
lines = ["| 条目 | 来源 | 创意 | 标题 | 原判定 | 新 P0/P1/P2 | 无P0/P1=过 | 与预期 |",
|
||||
"|---|---|---|---|---|---|---|---|"]
|
||||
for e in gs:
|
||||
r = a_map[e["entryId"]]
|
||||
sev = {"P0": 0, "P1": 0, "P2": 0}
|
||||
for f in r["findings"]:
|
||||
sev[f["severity"]] += 1
|
||||
orig = {"batch-001-accept": "accept", "batch-001-p1residual-kill": "kill(p1残留)",
|
||||
"w2-pilot": "W2 入池"}[e["source"]]
|
||||
# 2026-06-10 标签版本化后：golden-set 的 expectation 可能为版本化 dict（裁决②双标签），
|
||||
# 期望一律读 results.json 中 golden_regression 已按 doctrine 解析好的标量，避免重复实现解析
|
||||
expect_ok = (r["passNoP0P1"] if r["expectation"] == "no_p0p1" else r["hasP1"])
|
||||
lines.append("| %s | %s | %s | %s | %s | %d/%d/%d | %s | %s |" % (
|
||||
e["entryId"], {"batch-001-accept": "B1 accept", "batch-001-p1residual-kill": "B1 kill",
|
||||
"w2-pilot": "W2"}[e["source"]],
|
||||
e["idea"], e["design"]["config"]["title"], orig, sev["P0"], sev["P1"], sev["P2"],
|
||||
"过" if r["passNoP0P1"] else "**不过**", "符合" if expect_ok else "**偏离**"))
|
||||
|
||||
lines.append("")
|
||||
lines.append("### 断言③双跑明细")
|
||||
lines.append("| 条目 | run1 (P0,P1) | run2 (P0,P1) | 一致 |")
|
||||
lines.append("|---|---|---|---|")
|
||||
for d in rs["d"]:
|
||||
lines.append("| %s | %s | %s | %s |" % (d["entryId"], tuple(d["run1"]), tuple(d["run2"]),
|
||||
"是" if d["consistent"] else "**否**"))
|
||||
lines.append("")
|
||||
lines.append("### 断言④交叉明细（deepseek-v4-flash vs MiniMax）")
|
||||
lines.append("| 条目 | MiniMax 过 | 交叉过 | 交叉 (P0,P1) | 分歧 |")
|
||||
lines.append("|---|---|---|---|---|")
|
||||
for e in rs["e"]:
|
||||
lines.append("| %s | %s | %s | %s | %s |" % (e["entryId"], "过" if e["miniMaxPass"] else "不过",
|
||||
"过" if e["crossPass"] else "不过",
|
||||
tuple(e["crossTuple"]),
|
||||
"**是**" if e["diverge"] else "否"))
|
||||
open("/tmp/report-tables.md", "w", encoding="utf-8").write("\n".join(lines))
|
||||
print("\n明细表已写 /tmp/report-tables.md（%d 行）" % len(lines))
|
||||
@ -0,0 +1,705 @@
|
||||
[
|
||||
{
|
||||
"entryId": "g9011",
|
||||
"source": "batch-001-accept",
|
||||
"gameId": 9011,
|
||||
"designId": "4e2de887ceed035e571db5ed0f79462f1f52d6d4944805ce7b8e65a63ce1585c",
|
||||
"round": 0,
|
||||
"idea": "王蓝莓的小卖部收银台，给顾客结账",
|
||||
"design": {
|
||||
"designId": "4e2de887ceed035e571db5ed0f79462f1f52d6d4944805ce7b8e65a63ce1585c",
|
||||
"idea": "王蓝莓的小卖部收银台，给顾客结账",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "玩家化身校园小卖部收银员，在柜台前快速为排队的同学们结账。每点击一次屏幕完成一单交易，在限定时间内完成目标数量的结账任务，体验便利店员紧张忙碌的工作节奏。",
|
||||
"expectedPlaySeconds": 25,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "小卖部·收银时刻",
|
||||
"theme": "校园便利店的热闹柜台，排队结账的同学们等待着",
|
||||
"target": 12,
|
||||
"scoreLabel": "结账单"
|
||||
},
|
||||
"round": 0,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": []
|
||||
},
|
||||
{
|
||||
"entryId": "g9012",
|
||||
"source": "batch-001-accept",
|
||||
"gameId": 9012,
|
||||
"designId": "115589c5bc4a8f572c087cc20a7373f16064eccce621cc038e458616f5485618",
|
||||
"round": 1,
|
||||
"idea": "太空舱里捡星星补给能量",
|
||||
"design": {
|
||||
"designId": "115589c5bc4a8f572c087cc20a7373f16064eccce621cc038e458616f5485618",
|
||||
"idea": "太空舱里捡星星补给能量",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "在漂浮着星光的太空舱内点击收集散落的星体，将星星转化为维持舱室运转的能源。每一次点击都是一次星际拾取，累积足够的星能即可完成任务。",
|
||||
"expectedPlaySeconds": 30,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "星际拾星员",
|
||||
"theme": "太空舱内漂浮着点点星芒，等待被收集转化为维系舱室运转的能量",
|
||||
"target": 15,
|
||||
"scoreLabel": "星能"
|
||||
},
|
||||
"round": 1,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": []
|
||||
},
|
||||
{
|
||||
"entryId": "g9013",
|
||||
"source": "batch-001-accept",
|
||||
"gameId": 9013,
|
||||
"designId": "6f091b6da8142dda57ab8b132403c2ab62e00b5824a20a10d1156ea982bda32d",
|
||||
"round": 0,
|
||||
"idea": "农场里收割成熟的蔬菜",
|
||||
"design": {
|
||||
"designId": "6f091b6da8142dda57ab8b132403c2ab62e00b5824a20a10d1156ea982bda32d",
|
||||
"idea": "农场里收割成熟的蔬菜",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "玩家化身勤劳农夫，在生机勃勃的菜园中快速点击收割成熟的蔬菜，体验秋收的满足与快乐。",
|
||||
"expectedPlaySeconds": 12,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "绿野采收忙",
|
||||
"theme": "金秋时节，菜园里的番茄、黄瓜、青椒相继成熟，颗颗饱满亟待采摘，玩家点击即可将新鲜蔬菜收入篮中。",
|
||||
"target": 12,
|
||||
"scoreLabel": "蔬菜颗数"
|
||||
},
|
||||
"round": 0,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": []
|
||||
},
|
||||
{
|
||||
"entryId": "g9014",
|
||||
"source": "batch-001-accept",
|
||||
"gameId": 9014,
|
||||
"designId": "a1869689d1b746b894edce5b93514dca856121f937056bb42ced6d8ea489f243",
|
||||
"round": 0,
|
||||
"idea": "深夜便利店招待来买夜宵的顾客",
|
||||
"design": {
|
||||
"designId": "a1869689d1b746b894edce5b93514dca856121f937056bb42ced6d8ea489f243",
|
||||
"idea": "深夜便利店招待来买夜宵的顾客",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "在温暖的灯光下，为深夜匆匆而来的顾客递上一份热腾腾的夜宵。在指尖的忙碌中，体验深夜城市便利店独有的温情与节奏。",
|
||||
"expectedPlaySeconds": 18,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "午夜便利店·夜宵速递",
|
||||
"theme": "昏黄的灯光点亮深夜的便利店，烤肠在烤架上滋滋作响，关东煮在汤锅里翻滚，饭团整齐排列。凌晨的风穿过玻璃门，推进来一位位寻找慰藉的夜归人。",
|
||||
"target": 18,
|
||||
"scoreLabel": "服务顾客数"
|
||||
},
|
||||
"round": 0,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": []
|
||||
},
|
||||
{
|
||||
"entryId": "g9015",
|
||||
"source": "batch-001-accept",
|
||||
"gameId": 9015,
|
||||
"designId": "b2603e82f07a463bfec7dab27272daab1cc760e1d108bb0ae3811322c4daa24b",
|
||||
"round": 1,
|
||||
"idea": "校园运动会上的接力赛冲刺",
|
||||
"design": {
|
||||
"designId": "b2603e82f07a463bfec7dab27272daab1cc760e1d108bb0ae3811322c4daa24b",
|
||||
"idea": "校园运动会上的接力赛冲刺",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "玩家通过快速点击屏幕，模拟运动员在跑道上的冲刺过程，追逐接力赛终点线的荣耀，体验校园体育竞技的紧张与热血。",
|
||||
"expectedPlaySeconds": 12,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "操场冲刺·冲刺荣耀",
|
||||
"theme": "红色跑道上，最后一棒选手全力冲刺，接力棒飞速传递，观众欢呼声中冲向终点线，热血青春在脚下绽放。",
|
||||
"target": 12,
|
||||
"scoreLabel": "冲刺次数"
|
||||
},
|
||||
"round": 1,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": []
|
||||
},
|
||||
{
|
||||
"entryId": "g9016",
|
||||
"source": "batch-001-accept",
|
||||
"gameId": 9016,
|
||||
"designId": "b00027fe8f60f0d08a84c12e0bf91c25448a7d1b3a3a35b6f04d4957f2e66ad9",
|
||||
"round": 1,
|
||||
"idea": "猫咖啡馆里撸猫攒爱心",
|
||||
"design": {
|
||||
"designId": "b00027fe8f60f0d08a84c12e0bf91c25448a7d1b3a3a35b6f04d4957f2e66ad9",
|
||||
"idea": "猫咖啡馆里撸猫攒爱心",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "在温馨的猫咖里轻抚可爱的猫咪，每次触摸都会收获暖暖的爱心，集齐爱心即可通关，感受治愈的力量。",
|
||||
"expectedPlaySeconds": 30,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "猫咖轻抚·爱心满溢",
|
||||
"theme": "温馨治愈的猫咖啡馆氛围，柔软的猫咪等待你的轻抚，每一下抚摸都化作温暖的爱意",
|
||||
"target": 10,
|
||||
"scoreLabel": "爱心"
|
||||
},
|
||||
"round": 1,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": []
|
||||
},
|
||||
{
|
||||
"entryId": "g9017",
|
||||
"source": "batch-001-accept",
|
||||
"gameId": 9017,
|
||||
"designId": "43afd1f72ac6cf454faeb54cb50343e31cb06fd7ba1e07d92567debc57a05036",
|
||||
"round": 0,
|
||||
"idea": "春节集市上抢福袋",
|
||||
"design": {
|
||||
"designId": "43afd1f72ac6cf454faeb54cb50343e31cb06fd7ba1e07d92567debc57a05036",
|
||||
"idea": "春节集市上抢福袋",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "在热闹的春节集市中快速点击收集飞来的福袋，感受红火的年味与抢福的紧张刺激，达成目标即为通关。",
|
||||
"expectedPlaySeconds": 45,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "迎春纳福·福袋满仓",
|
||||
"theme": "春节集市人潮涌动，各式福袋从天而降，红色金色交织出喜庆的节日海洋。",
|
||||
"target": 15,
|
||||
"scoreLabel": "福袋"
|
||||
},
|
||||
"round": 0,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": [],
|
||||
"tolerance": "主 agent 裁决③（2026-06-10）：v1.1.0 回归新报 P1（target=15 × expectedPlaySeconds=45 配比与「快速抢福袋」意图不符）定性「存疑偏真正报」，计入断言②误报容差（≤2）不阻断；expectation 维持 no_p0p1，待实玩数据复核"
|
||||
},
|
||||
{
|
||||
"entryId": "k-0cfbc296",
|
||||
"source": "batch-001-p1residual-kill",
|
||||
"gameId": null,
|
||||
"designId": "0cfbc29612b6966447cf7449d6cdef8da1f57521f353bdc726b48fa0aa621048",
|
||||
"round": 1,
|
||||
"idea": "海底捞珍珠同时躲开鲨鱼",
|
||||
"design": {
|
||||
"designId": "0cfbc29612b6966447cf7449d6cdef8da1f57521f353bdc726b48fa0aa621048",
|
||||
"idea": "海底捞珍珠同时躲开鲨鱼",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "在神秘的海底世界，玩家扮演勇敢的采珠人，在收集璀璨珍珠的同时，必须时刻警惕鲨鱼的出没。点击收集珍珠，躲避鲨鱼威胁，在危险与收获之间寻找平衡，感受紧张刺激的海底探险之旅。",
|
||||
"expectedPlaySeconds": 45,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "深海采珠记",
|
||||
"theme": "幽蓝深海中，珍珠散落在珊瑚礁与海草间，闪烁着诱人的光芒。玩家需要在收集珍珠的同时警惕鲨鱼的动向——每收集3-5颗珍珠后，鲨鱼会短暂出现，此时点击鲨鱼即可将其驱离；若未能及时点击，鲨鱼会吓跑周围的珍珠，导致损失3颗已收集的珍珠。玩家需要在收集珍珠与躲避鲨鱼之间保持平衡，在危险与收获之间寻找最佳时机，收获宝藏。",
|
||||
"target": 22,
|
||||
"scoreLabel": "珍珠"
|
||||
},
|
||||
"round": 1,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": "has_p1",
|
||||
"originalFindings": [
|
||||
{
|
||||
"severity": "P1",
|
||||
"issue": "设计意图与实际机制矛盾：idea 明确是「躲开鲨鱼」，但 theme 中鲨鱼出现后的解决方案是「点击鲨鱼即可将其驱离」，这是主动攻击而非躲避，与「躲避」的核心意图不符。",
|
||||
"suggestion": "明确玩法核心——若设计意图是躲避，则鲨鱼出现时应提供可躲避的操作窗口或其他躲避方式；若核心就是点击驱离，则应将 designIntent 改为「驱赶鲨鱼」以保持一致性。"
|
||||
},
|
||||
{
|
||||
"severity": "P2",
|
||||
"issue": "theme 描述鲨鱼惩罚时用了「吓跑周围的珍珠」和「导致损失3颗已收集的珍珠」两种说法，表述略有不一致。",
|
||||
"suggestion": "统一表述为鲨鱼会带走已收集的珍珠，损失数量以「已收集的珍珠」计数，保持逻辑清晰。"
|
||||
}
|
||||
],
|
||||
"expectationNote": "kill 守卫·核心动作矛盾（主 agent 裁决②维持，不版本化）：创意『躲开』鲨鱼 ↔ theme『点击驱离』核心动作语义直接矛盾，属 P1 题文不符，不在细则①豁免范围（对抗 prompt v1.1.1 边界句即为此而加）；任何 doctrine 版本下仍须 has_p1"
|
||||
},
|
||||
{
|
||||
"entryId": "g9019",
|
||||
"source": "batch-001-accept",
|
||||
"gameId": 9019,
|
||||
"designId": "439034c731e5530c94685dad552ddf6395eee6c00f70e7cad08451cbfa656012",
|
||||
"round": 0,
|
||||
"idea": "烧烤摊翻烤串，别烤糊了",
|
||||
"design": {
|
||||
"designId": "439034c731e5530c94685dad552ddf6395eee6c00f70e7cad08451cbfa656012",
|
||||
"idea": "烧烤摊翻烤串，别烤糊了",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "夜幕降临，炭火通红的烧烤摊前，烤串在火焰上滋滋作响。玩家需要快速翻动每一根烤串，确保受热均匀，避免烤糊。每一次精准的翻动都是对火候的把握，在烟火气中体验指尖的忙碌与满足。",
|
||||
"expectedPlaySeconds": 54,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "炭火翻串记",
|
||||
"theme": "热闹夜市的烧烤摊，炭火正旺，烤串在火焰上翻滚，油脂滴落发出滋滋声响，烟火气与市井气息交织。",
|
||||
"target": 18,
|
||||
"scoreLabel": "翻烤次数"
|
||||
},
|
||||
"round": 0,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": []
|
||||
},
|
||||
{
|
||||
"entryId": "g9020",
|
||||
"source": "batch-001-accept",
|
||||
"gameId": 9020,
|
||||
"designId": "005cde6a7d097d691745726db5fdc1b54e6e935b448857b486752e92fd1a312f",
|
||||
"round": 0,
|
||||
"idea": "地铁早高峰挤上即将关门的车",
|
||||
"design": {
|
||||
"designId": "005cde6a7d097d691745726db5fdc1b54e6e935b448857b486752e92fd1a312f",
|
||||
"idea": "地铁早高峰挤上即将关门的车",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "早高峰地铁站台，车门警示灯闪烁，倒计时逼近，玩家疯狂点击屏幕帮助一位位乘客挤上即将关门的列车，在人潮涌动中体验争分夺秒的紧张与刺激。",
|
||||
"expectedPlaySeconds": 8,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "站台冲刺·车门将闭",
|
||||
"theme": "早高峰地铁站台，人潮汹涌，车门警示灯急促闪烁，倒计时逼近，玩家点击帮助乘客抢在关门瞬间挤上车厢。",
|
||||
"target": 15,
|
||||
"scoreLabel": "上车人次"
|
||||
},
|
||||
"round": 0,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": []
|
||||
},
|
||||
{
|
||||
"entryId": "g9021",
|
||||
"source": "batch-001-accept",
|
||||
"gameId": 9021,
|
||||
"designId": "526aaaa99ab651ff89fd00dfc2b8b401e5c03b23f19b05a17b1757e526671eee",
|
||||
"round": 0,
|
||||
"idea": "做一个很好玩的游戏",
|
||||
"design": {
|
||||
"designId": "526aaaa99ab651ff89fd00dfc2b8b401e5c03b23f19b05a17b1757e526671eee",
|
||||
"idea": "做一个很好玩的游戏",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "温馨治愈的糖果屋场景，玩家点击收集散落在彩色糖果罐中的各种糖果，每点击一次就能收集一颗甜蜜的糖果，累积到目标数量即可完成这一轮的甜蜜收集任务。",
|
||||
"expectedPlaySeconds": 15,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "糖果屋的甜蜜收集",
|
||||
"theme": "温馨梦幻的糖果屋里，五彩缤纷的糖果散落在玻璃罐和木质架子上，空气中弥漫着甜蜜的香气，玩家在这里寻找最简单的快乐。",
|
||||
"target": 8,
|
||||
"scoreLabel": "糖果颗数"
|
||||
},
|
||||
"round": 0,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": []
|
||||
},
|
||||
{
|
||||
"entryId": "g9022",
|
||||
"source": "batch-001-accept",
|
||||
"gameId": 9022,
|
||||
"designId": "b52388c4e4d72182a728ef86f03bb699292d4015f108b6426218f6170d63653b",
|
||||
"round": 0,
|
||||
"idea": "解压点点点",
|
||||
"design": {
|
||||
"designId": "b52388c4e4d72182a728ef86f03bb699292d4015f108b6426218f6170d63653b",
|
||||
"idea": "解压点点点",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "在弹力球工作室中，玩家点击弹力球帮助它们弹得更高，每一次弹跳都带走一份压力。简单的点击节奏配合弹力球的可爱造型，让玩家在轻松愉快的氛围中释放一天的疲惫。",
|
||||
"expectedPlaySeconds": 45,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "弹力球解压工作室",
|
||||
"theme": "柔软的彩色弹力球在工作室里轻轻弹跳，每一下弹起都带走一点压力。简洁的空间，柔和的光线，玩家只需要跟着节奏点击，让球球们弹得更高更开心。",
|
||||
"target": 15,
|
||||
"scoreLabel": "压力消散值"
|
||||
},
|
||||
"round": 0,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": []
|
||||
},
|
||||
{
|
||||
"entryId": "g9023",
|
||||
"source": "batch-001-accept",
|
||||
"gameId": 9023,
|
||||
"designId": "cb3f7444796b29723b74e08807a6c94555e8f98c4a4f0c0c7cf7bd50c4c90dcb",
|
||||
"round": 0,
|
||||
"idea": "中秋节帮嫦娥把月饼送上月宫",
|
||||
"design": {
|
||||
"designId": "cb3f7444796b29723b74e08807a6c94555e8f98c4a4f0c0c7cf7bd50c4c90dcb",
|
||||
"idea": "中秋节帮嫦娥把月饼送上月宫",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "中秋圆月之夜，你化作月宫使者，帮助嫦娥将满载思念的月饼送往广寒宫。每一次点击，都是对团圆的祈愿，在清冷月光与温热饼香中，体验独特的节日仪式感。",
|
||||
"expectedPlaySeconds": 8,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "月宫献饼记",
|
||||
"theme": "中秋圆月高悬，清冷的月辉洒落云端，嫦娥怀抱月饼踏云而去，温暖饼香与清冷月色交织，献礼之路就在指尖。",
|
||||
"target": 15,
|
||||
"scoreLabel": "月饼"
|
||||
},
|
||||
"round": 0,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": []
|
||||
},
|
||||
{
|
||||
"entryId": "g9024",
|
||||
"source": "batch-001-accept",
|
||||
"gameId": 9024,
|
||||
"designId": "090212e4c88d43b691aba53037bd586f1070dad5b7aeae790a785a31c351c5f5",
|
||||
"round": 0,
|
||||
"idea": "雨夜帮路边馄饨摊老板撑伞收摊",
|
||||
"design": {
|
||||
"designId": "090212e4c88d43b691aba53037bd586f1070dad5b7aeae790a785a31c351c5f5",
|
||||
"idea": "雨夜帮路边馄饨摊老板撑伞收摊",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "雨夜馄饨摊的灯火在细雨中摇曳，玩家帮助老板快速收拢摊位物件，在淅沥雨声中感受指尖传递的温暖与忙碌后的满足。",
|
||||
"expectedPlaySeconds": 20,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "雨夜馄饨摊·温情收摊",
|
||||
"theme": "绵绵细雨中，街头馄饨摊的灯火摇曳，老板在忙碌一天后准备收摊，热腾腾的汤锅、整齐的碗碟等待被一一收起，玩家帮忙撑伞收拾，在湿润的夜色中传递一份市井温情。",
|
||||
"target": 15,
|
||||
"scoreLabel": "收好的摊位物件"
|
||||
},
|
||||
"round": 0,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": []
|
||||
},
|
||||
{
|
||||
"entryId": "k-26ce4440",
|
||||
"source": "batch-001-p1residual-kill",
|
||||
"gameId": null,
|
||||
"designId": "26ce44408e55eaa604b3efa2903c3cd5a541ab99ed8aa7c62c9b8192ef03260e",
|
||||
"round": 1,
|
||||
"idea": "图书馆闭馆前把还回来的书归架",
|
||||
"design": {
|
||||
"designId": "26ce44408e55eaa604b3efa2903c3cd5a541ab99ed8aa7c62c9b8192ef03260e",
|
||||
"idea": "图书馆闭馆前把还回来的书归架",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "玩家扮演图书馆管理员，在闭馆前的有限时间内高效地将归还的书籍送回书架，体验忙碌却有序的职业节奏，感受书归其位的成就感。",
|
||||
"expectedPlaySeconds": 45,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "闭馆归架记",
|
||||
"theme": "图书馆的夜灯渐次熄灭，还回的书本堆叠在归还台上，等待被送回各自的书架，忙碌的双手在书架间穿梭，每一本书都找到了归宿。",
|
||||
"target": 12,
|
||||
"scoreLabel": "归架书数"
|
||||
},
|
||||
"round": 1,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": {
|
||||
"byDoctrine": [
|
||||
{
|
||||
"doctrine": "=1.0.0",
|
||||
"expect": "has_p1",
|
||||
"label": "kill(P1 越模板机制)"
|
||||
},
|
||||
{
|
||||
"doctrine": ">=1.1.0",
|
||||
"expect": "no_p0p1",
|
||||
"label": "P2-pass（细则①生效的正向守卫：该条再出 P1 即细则①失效）"
|
||||
}
|
||||
],
|
||||
"relabelNote": "主 agent 裁决②（2026-06-10）：原 P1 锚点=designIntent「有限时间内」承诺计时机制而 config 无计时，与 g9019 误报同模式；按口径版本化双标签消费，不再作为 doctrine>=v1.1.0 下的不放水负例"
|
||||
},
|
||||
"originalFindings": [
|
||||
{
|
||||
"severity": "P1",
|
||||
"issue": "designIntent 明确提到「有限时间内」完成任务，但 config 中缺少时间限制配置，expectedPlaySeconds 仅表示预期时长而非计时机制",
|
||||
"suggestion": "在 config 中增加 timeLimit 字段与 expectedPlaySeconds 对应，明确玩家需在规定秒数内达成 target，或说明 target 为无需计时完成的目标数"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"entryId": "g9026",
|
||||
"source": "batch-001-accept",
|
||||
"gameId": 9026,
|
||||
"designId": "664cf2034f2bc086f6780020f9fe02a9a1e379fc2b33daa430e34841e3e30741",
|
||||
"round": 0,
|
||||
"idea": "奶茶店爆单日疯狂摇珍珠奶茶",
|
||||
"design": {
|
||||
"designId": "664cf2034f2bc086f6780020f9fe02a9a1e379fc2b33daa430e34841e3e30741",
|
||||
"idea": "奶茶店爆单日疯狂摇珍珠奶茶",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "玩家扮演奶茶店员工，在爆单高峰时段疯狂摇晃调酒器出杯。指尖跟随节拍快速点击，模拟调制珍珠奶茶的忙碌节奏，在倒计时结束前完成尽可能多的订单，体验节假日餐饮从业者的紧张与成就。",
|
||||
"expectedPlaySeconds": 60,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "茶铺高峰·手摇风暴",
|
||||
"theme": "奶茶店节假日大排长龙，玩家化身店员手持 shaker 疯狂摇晃出杯，蒸汽与糖香弥漫，指尖的节奏决定订单的流转速度。",
|
||||
"target": 20,
|
||||
"scoreLabel": "杯奶茶"
|
||||
},
|
||||
"round": 0,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": []
|
||||
},
|
||||
{
|
||||
"entryId": "k-a04dfac9",
|
||||
"source": "batch-001-p1residual-kill",
|
||||
"gameId": null,
|
||||
"designId": "a04dfac9bba70833a1dcbce8242a787f3180db5b516b65006589698878eabcc6",
|
||||
"round": 1,
|
||||
"idea": "跨年夜零点在广场上点烟花倒计时",
|
||||
"design": {
|
||||
"designId": "a04dfac9bba70833a1dcbce8242a787f3180db5b516b65006589698878eabcc6",
|
||||
"idea": "跨年夜零点在广场上点烟花倒计时",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "跨年夜零点倒计时的紧张期待，玩家在广场等待新年钟声响起的那一刻，点击点燃璀璨烟花，感受辞旧迎新的激动与喜悦。",
|
||||
"expectedPlaySeconds": 20,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "午夜庆典·焰火绽放",
|
||||
"theme": "跨年夜倒计时即将归零，广场上人潮涌动，玩家争分夺秒连续点击，在新年钟声敲响瞬间点燃璀璨烟花，迎接第一道光芒。",
|
||||
"target": 15,
|
||||
"scoreLabel": "点燃烟花"
|
||||
},
|
||||
"round": 1,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": {
|
||||
"byDoctrine": [
|
||||
{
|
||||
"doctrine": "=1.0.0",
|
||||
"expect": "has_p1",
|
||||
"label": "kill(P1 越模板机制)"
|
||||
},
|
||||
{
|
||||
"doctrine": ">=1.1.0",
|
||||
"expect": "no_p0p1",
|
||||
"label": "P2-pass（细则①生效的正向守卫：该条再出 P1 即细则①失效）"
|
||||
}
|
||||
],
|
||||
"relabelNote": "主 agent 裁决②（2026-06-10）：原 P1 锚点=「倒计时紧张感」越模板机制承诺，且 idea 本身含「倒计时」、题文不冲突（细则①+②叠加无 P1 落点）；按口径版本化双标签消费"
|
||||
},
|
||||
"originalFindings": [
|
||||
{
|
||||
"severity": "P1",
|
||||
"issue": "设计意图强调'倒计时的紧张期待'，但 config 中 target=15 为点击次数，与倒计时概念脱节，玩家看不到剩余时间，无法真正感受倒计时的紧张感",
|
||||
"suggestion": "建议将 target 改为时间维度（如倒计时 X 秒），或明确说明是在倒计时内完成指定点击次数，使玩法与'倒计时'主题一致"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"entryId": "g9028",
|
||||
"source": "batch-001-accept",
|
||||
"gameId": 9028,
|
||||
"designId": "53933f300e7a9c254986a2972be2c9c577b1233856336f18a20ed5a17bc6ae69",
|
||||
"round": 0,
|
||||
"idea": "给办公室加班的同事分下午茶蛋糕",
|
||||
"design": {
|
||||
"designId": "53933f300e7a9c254986a2972be2c9c577b1233856336f18a20ed5a17bc6ae69",
|
||||
"idea": "给办公室加班的同事分下午茶蛋糕",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "模拟办公室下午茶分蛋糕的温馨场景，玩家快速点击将蛋糕分送给加班的同事，在忙碌的工作间隙感受分享的甜蜜与满足。",
|
||||
"expectedPlaySeconds": 15,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "甜蜜分送",
|
||||
"theme": "午后阳光洒进办公室，加班中的同事们等待分享那份下午茶蛋糕，玩家化身温暖使者快速将蛋糕分送到每个人手中。",
|
||||
"target": 12,
|
||||
"scoreLabel": "蛋糕"
|
||||
},
|
||||
"round": 0,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": [
|
||||
{
|
||||
"severity": "P2",
|
||||
"issue": "标题'甜蜜分送'过于笼统，未能体现'办公室加班同事'和'下午茶蛋糕'的具体情境，与创意原意的关联度不足",
|
||||
"suggestion": "建议标题更贴近具体场景，如融入办公室、蛋糕等关键词，增强题文一致性"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"entryId": "g9030",
|
||||
"source": "batch-001-accept",
|
||||
"gameId": 9030,
|
||||
"designId": "7ebba6f04540bce08c531189024c8839c26424b84ef224ea8dfa5a64c8fa5076",
|
||||
"round": 0,
|
||||
"idea": "火锅店捞毛肚，七上八下别捞老了",
|
||||
"design": {
|
||||
"designId": "7ebba6f04540bce08c531189024c8839c26424b84ef224ea8dfa5a64c8fa5076",
|
||||
"idea": "火锅店捞毛肚，七上八下别捞老了",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "沸腾的红油锅里，毛肚在翻滚七上八下，玩家点击屏幕在最佳时机将毛肚捞起，捞够目标数量即可通关。",
|
||||
"expectedPlaySeconds": 30,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "沸腾捞肚",
|
||||
"theme": "翻滚的红油锅底，热气蒸腾，毛肚在锅里七上八下跳动，玩家需把握时机点击捞起，避免毛肚过老。",
|
||||
"target": 15,
|
||||
"scoreLabel": "毛肚片"
|
||||
},
|
||||
"round": 0,
|
||||
"_parseError": null
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": []
|
||||
},
|
||||
{
|
||||
"entryId": "w2-1",
|
||||
"source": "w2-pilot",
|
||||
"gameId": null,
|
||||
"designId": "8f9108324a3bef55f471773b697a48047cf4c5b70b46a97a33696a5e4ebc39a1",
|
||||
"round": 0,
|
||||
"idea": "王蓝莓的小卖部收银台，给顾客结账",
|
||||
"design": {
|
||||
"designId": "8f9108324a3bef55f471773b697a48047cf4c5b70b46a97a33696a5e4ebc39a1",
|
||||
"idea": "王蓝莓的小卖部收银台，给顾客结账",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "放学高峰的小卖部收银台前，玩家每点击一次就为一位顾客完成一单结账，攒满目标单数即通关，体验队伍越点越短的市井烟火气。",
|
||||
"expectedPlaySeconds": 10,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "蓝莓妹的小卖部·结账高峰",
|
||||
"theme": "放学时分的小卖部收银台，“嘀”一声结一单、队伍越点越短的市井烟火气",
|
||||
"target": 20,
|
||||
"scoreLabel": "结账单数"
|
||||
},
|
||||
"round": 0
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": null
|
||||
},
|
||||
{
|
||||
"entryId": "w2-2",
|
||||
"source": "w2-pilot",
|
||||
"gameId": null,
|
||||
"designId": "b62350bae955d7c3a510efa0eed0f74c7408ba892b8f0b1e59a367d6456a8425",
|
||||
"round": 0,
|
||||
"idea": "深夜便利店招待来买夜宵的顾客",
|
||||
"design": {
|
||||
"designId": "b62350bae955d7c3a510efa0eed0f74c7408ba892b8f0b1e59a367d6456a8425",
|
||||
"idea": "深夜便利店招待来买夜宵的顾客",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "深夜便利店的暖灯下，玩家每点击一次就招待一位来买夜宵的夜归人，招待满目标人数即完成今晚的深夜档。",
|
||||
"expectedPlaySeconds": 9,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "凌晨两点的便利店",
|
||||
"theme": "24小时便利店的深夜档，暖灯下招待买夜宵的夜归人",
|
||||
"target": 18,
|
||||
"scoreLabel": "已招待顾客"
|
||||
},
|
||||
"round": 0
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": null
|
||||
},
|
||||
{
|
||||
"entryId": "w2-3",
|
||||
"source": "w2-pilot",
|
||||
"gameId": null,
|
||||
"designId": "2b15bd30b466f12653be4dac5d53ac5b856110986e00d6926c62e8c3ece809c7",
|
||||
"round": 0,
|
||||
"idea": "猫咖啡馆里撸猫攒爱心",
|
||||
"design": {
|
||||
"designId": "2b15bd30b466f12653be4dac5d53ac5b856110986e00d6926c62e8c3ece809c7",
|
||||
"idea": "猫咖啡馆里撸猫攒爱心",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "治愈系猫咖的午后阳光里，玩家每轻点一次屏幕就撸一下猫、攒一颗爱心，攒满目标爱心数即通关，收获治愈感。",
|
||||
"expectedPlaySeconds": 7,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "猫咖时光·撸猫集心",
|
||||
"theme": "治愈系猫咖啡馆：午后阳光里轻点屏幕撸猫，每一下攒一颗爱心",
|
||||
"target": 15,
|
||||
"scoreLabel": "爱心"
|
||||
},
|
||||
"round": 0
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": null
|
||||
},
|
||||
{
|
||||
"entryId": "w2-4",
|
||||
"source": "w2-pilot",
|
||||
"gameId": null,
|
||||
"designId": "1b3c5764b01c161d1c0cbd6aaba2c7375280b2469a850152b6596a22b55d7389",
|
||||
"round": 0,
|
||||
"idea": "烧烤摊翻烤串，别烤糊了",
|
||||
"design": {
|
||||
"designId": "1b3c5764b01c161d1c0cbd6aaba2c7375280b2469a850152b6596a22b55d7389",
|
||||
"idea": "烧烤摊翻烤串，别烤糊了",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "深夜烧烤摊炭火正旺，玩家每点击一次就给烤串翻一次面，翻满目标次数即烤好出摊，体验手速翻串的指尖爽感。",
|
||||
"expectedPlaySeconds": 10,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "烤串二十翻",
|
||||
"theme": "深夜烧烤摊：炭火滋滋作响，手速翻面，别让烤串烤糊",
|
||||
"target": 20,
|
||||
"scoreLabel": "翻面"
|
||||
},
|
||||
"round": 0
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": null
|
||||
},
|
||||
{
|
||||
"entryId": "w2-5",
|
||||
"source": "w2-pilot",
|
||||
"gameId": null,
|
||||
"designId": "feca768c87f9fcb3111a504ab074f8e998129c9de01e833957f4039c6ffbef5a",
|
||||
"round": 0,
|
||||
"idea": "地铁早高峰挤上即将关门的车",
|
||||
"design": {
|
||||
"designId": "feca768c87f9fcb3111a504ab074f8e998129c9de01e833957f4039c6ffbef5a",
|
||||
"idea": "地铁早高峰挤上即将关门的车",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "早高峰地铁站台，玩家抢在车门关闭前拼命连点屏幕向车厢里挤，攒满挤入进度即成功上车，短局节奏与题材的紧迫感天然同构。",
|
||||
"expectedPlaySeconds": 9,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "车门即将关闭",
|
||||
"theme": "早高峰地铁站台·车门倒计时",
|
||||
"target": 18,
|
||||
"scoreLabel": "挤入进度"
|
||||
},
|
||||
"round": 0
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": null
|
||||
},
|
||||
{
|
||||
"entryId": "w2-6",
|
||||
"source": "w2-pilot",
|
||||
"gameId": null,
|
||||
"designId": "d0b8c8ea60d5d07c220e1c903f8e78ed36d21ad816b4ee82d2dae8d13af0388a",
|
||||
"round": 0,
|
||||
"idea": "解压点点点",
|
||||
"design": {
|
||||
"designId": "d0b8c8ea60d5d07c220e1c903f8e78ed36d21ad816b4ee82d2dae8d13af0388a",
|
||||
"idea": "解压点点点",
|
||||
"templateId": "clicker",
|
||||
"designIntent": "电子解压玩具：玩家把今天的烦躁一下一下戳掉，每戳一次攒一点解压值，攒满目标解压值即过关，获得纯粹的解压爽感。",
|
||||
"expectedPlaySeconds": 12,
|
||||
"config": {
|
||||
"templateId": "clicker",
|
||||
"title": "指尖戳戳乐",
|
||||
"theme": "治愈系电子解压玩具：把今天的烦躁一下一下戳掉，解压值攒满即过关",
|
||||
"target": 25,
|
||||
"scoreLabel": "解压值"
|
||||
},
|
||||
"round": 0
|
||||
},
|
||||
"expectation": "no_p0p1",
|
||||
"originalFindings": null
|
||||
}
|
||||
]
|
||||
@ -0,0 +1,361 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
Golden 集回归脚本（quality.adversary-review v1.1.1 重跑 · 复用 golden-regression-v1.1.0 同名脚本）
|
||||
|
||||
任务令（主 agent，2026-06-10 v1.1.1 轮）：对 v1.1.1 对抗 prompt（细则①句尾补核心动作矛盾边界句，
|
||||
主 agent 裁决①）重跑 25 条 Golden 集回归，五断言：
|
||||
A 25 条全量过 v1.1.1 对抗（MiniMax-M2.7，json_object，温度 0.2——与 v1.1.0 轮同口径）；
|
||||
①' 守卫双向：k-0cfbc296 必须仍含 P1（唯一 kill 守卫·核心动作矛盾）∧ k-26ce4440/k-a04dfac9
|
||||
按 v1.1.x 版本化标签（主 agent 裁决②）= 无 P1 且有 P2（细则①生效的正向守卫）；
|
||||
② 误报收敛：16 条原 accept 的 P1 误报 ≤2（g9017 按主 agent 裁决③定性「存疑偏真正报」
|
||||
计入容差不阻断，报告单列）；
|
||||
③ 稳定性：抽 5 条（必含 g9019）双跑同 prompt，P0/P1 判定一致率 ≥80%（对照口径=run_batch
|
||||
稳定性重测：(hasP0,hasP1) 元组翻转即不一致）；
|
||||
④ 交叉：同 5 条用 deepseek-v4-flash（推理型，max_tokens≥2048，只读 message.content）复核，
|
||||
与 MiniMax「无 P0/P1=过」结论分歧 ≤1 条。
|
||||
解冻终裁建议 = ①'②③④全过。
|
||||
|
||||
工程纪律：
|
||||
- 渲染复用 orchestrator/prompts.py（Registry 驱动 + 版本一致性核验 + 残留占位符防御）；
|
||||
- 通道复用 orchestrator/llm_client.py（重试 ×2 / 节流 0.3s / 密钥只走 NEWAPI_KEY 环境变量，严禁入盘）；
|
||||
- LLM 预算硬闸 ≤45 次（任务令本轮预算；口径=每次 HTTP 尝试，与 BudgetGuard 一致）；
|
||||
- 全部原始响应落 raw/（含请求摘要，不含密钥）；断点续跑：raw 已有合法产物的调用直接复用不再发起
|
||||
（本目录 raw/ 全新，不复用 v1.1.0 轮响应——prompt 已变，旧响应不可作为 v1.1.1 证据）；
|
||||
- findings 形状校验语义复制自 run_batch.py:1158-1181（_parse_json_object/_validate_findings_shape，
|
||||
避免为两个小函数拖入 backend_gw 等无关依赖）；形状异常应用层重试 ×2 后记 infra。
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
_HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
_LOOP_ROOT = os.path.abspath(os.path.join(_HERE, "..", ".."))
|
||||
_REPO = os.path.abspath(os.path.join(_LOOP_ROOT, "..", "..", ".."))
|
||||
sys.path.insert(0, os.path.join(_LOOP_ROOT, "orchestrator"))
|
||||
|
||||
import llm_client
|
||||
import prompts as prompts_mod
|
||||
|
||||
PROMPTS_DIR = os.path.join(_REPO, "contracts", "prompts")
|
||||
PROMPT_ID = "quality.adversary-review"
|
||||
EXPECT_VERSION = "1.1.1" # 回归对象版本（registry 与 frontmatter 双核验由 prompts.py 完成）
|
||||
MAIN_MODEL = "MiniMax-M2.7" # 主模型（任务令：json_object / 温度 0.2）
|
||||
CROSS_MODEL = "deepseek-v4-flash" # 交叉模型（推理型：max_tokens≥2048，读 content 忽略 reasoning_content）
|
||||
# 任务令本轮累计预算 ≤45（HTTP 尝试口径）。通道空 content 高发（leg1 在 g9021、leg2 在 w2-2
|
||||
# 三连空后中止），leg1 实耗 17、leg2 实耗 20（账本+失败尝试，见 llm-calls.jsonl 与 run-leg*.log），
|
||||
# 累计 37——任务矩阵固定 35 次逻辑调用 + 空响应重试已使「HTTP 尝试口径」越过 45，无法回收；
|
||||
# 处置：不削减断言覆盖面（削覆盖=回归失效），leg3 启用抢救通道止血，本闸退化为防失控护栏，
|
||||
# 累计实耗在 report.md 双口径（逻辑调用 / HTTP 尝试）如实分列。
|
||||
BUDGET_MAX = 22 # leg3 进程内防失控护栏（剩余 15 次逻辑调用 + 抢救余量）
|
||||
SHAPE_RETRIES = 2 # findings 形状异常应用层重试（同 run_batch ADVERSARY_SHAPE_RETRIES）
|
||||
|
||||
GOLDEN = os.path.join(_HERE, "golden-set.json")
|
||||
RAW_DIR = os.path.join(_HERE, "raw")
|
||||
RESULTS = os.path.join(_HERE, "results.json")
|
||||
CALLLOG = os.path.join(_HERE, "llm-calls.jsonl")
|
||||
|
||||
# 断言③/④抽样（与 v1.1.0 轮同一组，保跨轮可比；任务令要求必含 g9019——满足）：
|
||||
# 两条 R8 分歧原条目(9019/9021) + 1 条 kill 守卫(k-0cfbc296，断言①'同条) + 1 条普通 accept + 1 条 W2 模糊创意
|
||||
STABILITY_SAMPLE = ["g9019", "g9021", "k-0cfbc296", "g9011", "w2-6"]
|
||||
|
||||
# 任务令通道口径：温度 0.2（llm_client 默认 0.4 为 batch 主流水口径；R8 复核与本回归用 0.2）
|
||||
llm_client.TEMPERATURE = 0.2
|
||||
|
||||
|
||||
# ---------------- 黄金集标签版本化解析（2026-06-10 主 agent 裁决②，标签结构见 build_golden_set.py 头注）----------------
|
||||
|
||||
def _ver_tuple(v):
|
||||
"""语义版本字符串 → 可比较元组（如 "1.1.0" → (1,1,0)），仅支持纯数字三段。"""
|
||||
return tuple(int(x) for x in v.split("."))
|
||||
|
||||
|
||||
def resolve_expectation(entry, doctrine_version=None):
|
||||
"""
|
||||
解析 Golden 条目的期望标签（黄金集标签口径版本化，主 agent 裁决②）：
|
||||
- 旧式标量标签（"no_p0p1"/"has_p1"）直接返回，行为与版本化前完全一致；
|
||||
- 版本化双标签：expectation={"byDoctrine":[{"doctrine":"=1.0.0"|">=1.1.0","expect":...,"label":...},...]}，
|
||||
按当前回归对象 prompt 版本（=判定口径 doctrine 版本，默认 EXPECT_VERSION）匹配区间取 expect——
|
||||
k-26ce4440/k-a04dfac9 在 doctrine=v1.0.0 下为 kill 负例(has_p1)、doctrine>=v1.1.0 下为
|
||||
P2-pass 正向守卫(no_p0p1)；
|
||||
- 任何区间都未命中 = 标签配置错误，立即抛错（宁可中断不静默放行，保不放水底线）。
|
||||
"""
|
||||
exp = entry["expectation"]
|
||||
if isinstance(exp, str):
|
||||
return exp
|
||||
cur = _ver_tuple(doctrine_version or EXPECT_VERSION)
|
||||
for rule in exp["byDoctrine"]:
|
||||
cons = rule["doctrine"].strip()
|
||||
if cons.startswith(">="):
|
||||
if cur >= _ver_tuple(cons[2:]):
|
||||
return rule["expect"]
|
||||
elif cons.startswith("="):
|
||||
if cur == _ver_tuple(cons[1:]):
|
||||
return rule["expect"]
|
||||
else:
|
||||
raise RuntimeError("条目 %s 含未知 doctrine 约束：%r" % (entry["entryId"], cons))
|
||||
raise RuntimeError("条目 %s 的版本化标签未覆盖 doctrine v%s" % (entry["entryId"], doctrine_version or EXPECT_VERSION))
|
||||
|
||||
|
||||
def _log(msg):
|
||||
"""统一中文日志（时间戳+消息，外部交互可追溯）。"""
|
||||
print("[%s] %s" % (time.strftime("%H:%M:%S"), msg), flush=True)
|
||||
|
||||
|
||||
class Budget(object):
|
||||
"""LLM 预算硬闸：每次 HTTP 尝试计 1，超 70 立即抛异常中止（宁可中断不超预算）。"""
|
||||
|
||||
def __init__(self, limit):
|
||||
self.limit = limit
|
||||
self.used = 0
|
||||
|
||||
def note(self):
|
||||
self.used += 1
|
||||
if self.used > self.limit:
|
||||
raise RuntimeError("LLM 预算超闸：已用 %d > %d，立即中止" % (self.used, self.limit))
|
||||
|
||||
|
||||
BUDGET = Budget(BUDGET_MAX)
|
||||
|
||||
|
||||
# ---------------- findings 形状校验（语义复制自 run_batch.py:1158-1181，注释见模块头）----------------
|
||||
|
||||
def _parse_json_object(content):
|
||||
"""解析 LLM 输出为 JSON 对象；失败返回 (None, 错误说明)。容忍 ```json 围栏（交叉模型无 json_object 时防御）。"""
|
||||
text = content.strip()
|
||||
if text.startswith("```"):
|
||||
# 剥代码围栏：```json\n...\n``` → 中间体
|
||||
lines = [ln for ln in text.split("\n") if not ln.strip().startswith("```")]
|
||||
text = "\n".join(lines).strip()
|
||||
try:
|
||||
obj = json.loads(text)
|
||||
except ValueError as ex:
|
||||
return None, "json_parse:%s" % ex
|
||||
if not isinstance(obj, dict):
|
||||
return None, "not_object:%s" % type(obj).__name__
|
||||
return obj, None
|
||||
|
||||
|
||||
def _validate_findings_shape(parsed):
|
||||
"""校验对抗输出形状：{findings:[{severity∈P0/P1/P2, issue, suggestion}]}；空数组=无问题。"""
|
||||
if parsed is None:
|
||||
return None, "输出非 JSON 对象"
|
||||
findings = parsed.get("findings")
|
||||
if not isinstance(findings, list):
|
||||
return None, "缺 findings 数组"
|
||||
for idx, item in enumerate(findings):
|
||||
if not isinstance(item, dict) or item.get("severity") not in ("P0", "P1", "P2") \
|
||||
or not isinstance(item.get("issue"), str):
|
||||
return None, "findings[%d] 形状非法：%r" % (idx, item)
|
||||
item.setdefault("suggestion", "")
|
||||
return findings, None
|
||||
|
||||
|
||||
# ---------------- 交叉模型通道（推理型专用，复用 llm_client 的重试/节流纪律）----------------
|
||||
|
||||
def chat_json_reasoning(client, user, model, max_tokens=4096, log=_log):
|
||||
"""
|
||||
推理型模型调用（deepseek-v4-flash）：在 llm_client 配方上追加 max_tokens（任务令 ≥2048），
|
||||
只读 message.content（忽略 reasoning_content）；content 为空（含被 length 截断）时
|
||||
提额至 8192 重试。重试/退避/节流纪律与 llm_client.chat_json 一致。
|
||||
"""
|
||||
body_obj = {
|
||||
"model": model,
|
||||
"temperature": llm_client.TEMPERATURE,
|
||||
"max_tokens": max_tokens,
|
||||
"response_format": {"type": "json_object"},
|
||||
"messages": [{"role": "user", "content": user}],
|
||||
}
|
||||
last_err = None
|
||||
for attempt in range(1, llm_client.MAX_RETRIES + 2):
|
||||
BUDGET.note()
|
||||
req = urllib.request.Request(client.base + "/v1/chat/completions",
|
||||
data=json.dumps(body_obj).encode("utf-8"), method="POST")
|
||||
req.add_header("Authorization", "Bearer " + client.key)
|
||||
req.add_header("Content-Type", "application/json")
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=llm_client.REQUEST_TIMEOUT_SECONDS) as resp:
|
||||
data = json.loads(resp.read().decode("utf-8"))
|
||||
choice = data.get("choices", [{}])[0]
|
||||
content = (choice.get("message") or {}).get("content") or ""
|
||||
if not content.strip():
|
||||
# 推理型常见坑：max_tokens 被 reasoning 吃光 → content 空；提额重试
|
||||
last_err = "空 content（finish=%s）" % choice.get("finish_reason")
|
||||
log("[cross] 第 %d 次尝试 content 为空，提额 max_tokens 重试" % attempt)
|
||||
body_obj["max_tokens"] = 8192
|
||||
else:
|
||||
time.sleep(llm_client.CALL_INTERVAL_SECONDS)
|
||||
return content, data
|
||||
except urllib.error.HTTPError as ex:
|
||||
detail = ""
|
||||
try:
|
||||
detail = ex.read().decode("utf-8", "replace")[:200]
|
||||
except Exception:
|
||||
pass
|
||||
last_err = "http_%s:%s" % (ex.code, detail)
|
||||
log("[cross] 第 %d 次尝试 HTTP 错误 %s" % (attempt, last_err))
|
||||
# 个别推理通道不支持 response_format → 400 时去掉该参数重试（解析侧已容忍围栏）
|
||||
if ex.code == 400 and "response_format" in body_obj:
|
||||
body_obj.pop("response_format")
|
||||
except Exception as ex:
|
||||
last_err = "err:%s" % ex
|
||||
log("[cross] 第 %d 次尝试通道异常 %s" % (attempt, last_err))
|
||||
if attempt <= llm_client.MAX_RETRIES:
|
||||
time.sleep(1.0 * (2 ** (attempt - 1)))
|
||||
raise llm_client.LlmError("交叉模型调用重试耗尽：%s" % last_err)
|
||||
|
||||
|
||||
# ---------------- 单条评审（含 raw 落盘与断点续跑）----------------
|
||||
|
||||
def review_once(store, client, entry, run_tag, model):
|
||||
"""
|
||||
对单条 Golden 条目跑一次 v1.1.0 对抗评审。
|
||||
:param run_tag: a(主跑)/stability(双跑第2次)/cross(交叉模型)——raw 文件名后缀与调用账目标签
|
||||
:return: dict{entryId, runTag, model, findings, hasP0, hasP1, passNoP0P1, llmAttempts, reused}
|
||||
"""
|
||||
raw_path = os.path.join(RAW_DIR, "%s-%s.json" % (entry["entryId"], run_tag))
|
||||
if os.path.exists(raw_path):
|
||||
# 断点续跑：已有合法产物直接复用（不再发起调用，不计预算）
|
||||
saved = json.load(open(raw_path, "r", encoding="utf-8"))
|
||||
if saved.get("findings") is not None:
|
||||
f = saved["findings"]
|
||||
return {"entryId": entry["entryId"], "runTag": run_tag, "model": saved.get("model", model),
|
||||
"findings": f,
|
||||
"hasP0": any(x["severity"] == "P0" for x in f),
|
||||
"hasP1": any(x["severity"] == "P1" for x in f),
|
||||
"passNoP0P1": not any(x["severity"] in ("P0", "P1") for x in f),
|
||||
"llmAttempts": 0, "reused": True}
|
||||
|
||||
# 渲染（public_design 口径同 run_batch._call_adversary：剥 _ 前缀键）
|
||||
public_design = {k: v for k, v in entry["design"].items() if not k.startswith("_")}
|
||||
text, version = store.render(PROMPT_ID, {"idea": entry["idea"], "game_design": public_design})
|
||||
if version != EXPECT_VERSION:
|
||||
raise RuntimeError("prompt 版本非回归对象：渲染到 %s，期望 %s" % (version, EXPECT_VERSION))
|
||||
|
||||
used_before = BUDGET.used
|
||||
last_err = None
|
||||
findings = None
|
||||
raw_content = None
|
||||
raw_response_meta = None
|
||||
rescued = False
|
||||
for shape_try in range(1, SHAPE_RETRIES + 2):
|
||||
if model == CROSS_MODEL:
|
||||
content, full_resp = chat_json_reasoning(client, text, model)
|
||||
# 只留可审计的最小响应元信息（不落 reasoning_content 全文，防 raw 失控；usage 留作成本审计）
|
||||
raw_response_meta = {"model": full_resp.get("model"), "usage": full_resp.get("usage"),
|
||||
"finish_reason": (full_resp.get("choices") or [{}])[0].get("finish_reason")}
|
||||
elif shape_try <= 1:
|
||||
content, _att = client.chat_json("", text, model=model, budget_cb=BUDGET.note, log=_log)
|
||||
else:
|
||||
# 抢救通道（2026-06-10 leg3+ 生效，自第 2 次形状重试起介入）：本轮主通道高发「空 content」
|
||||
# 形状异常（json_parse char 0；leg1/leg2 合计 8 个条目中招、其中 2 条目三连空致程崩），
|
||||
# 疑似 MiniMax-M2.7 推理消耗吃光网关默认 max_tokens 所致（推断，与 chat_json_reasoning
|
||||
# 头注同类坑）——改走显式 max_tokens 通道（同模型、同温度、同 json_object，仅显式放宽
|
||||
# max_tokens 上限，截断空响应时自动提额 8192），判定口径不受影响；rescued 标记落 raw 供审计。
|
||||
_log("%s/%s 主通道空响应，改走显式 max_tokens 抢救通道" % (entry["entryId"], run_tag))
|
||||
content, full_resp = chat_json_reasoning(client, text, model)
|
||||
raw_response_meta = {"model": full_resp.get("model"), "usage": full_resp.get("usage"),
|
||||
"finish_reason": (full_resp.get("choices") or [{}])[0].get("finish_reason")}
|
||||
rescued = True
|
||||
parsed, perr = _parse_json_object(content)
|
||||
cand, serr = _validate_findings_shape(parsed)
|
||||
raw_content = content
|
||||
if serr is None:
|
||||
findings = cand
|
||||
break
|
||||
last_err = perr or serr
|
||||
_log("%s/%s 输出形状异常（第 %d 次）：%s" % (entry["entryId"], run_tag, shape_try, last_err))
|
||||
if findings is None:
|
||||
raise llm_client.LlmError("%s/%s 形状重试耗尽：%s" % (entry["entryId"], run_tag, last_err))
|
||||
|
||||
attempts = BUDGET.used - used_before
|
||||
# raw 落盘：promptVersion+模型+渲染输入摘要+原始 content+解析后 findings（无任何密钥）
|
||||
with open(raw_path, "w", encoding="utf-8") as fh:
|
||||
json.dump({
|
||||
"entryId": entry["entryId"], "runTag": run_tag, "model": model,
|
||||
"promptId": PROMPT_ID, "promptVersion": version,
|
||||
"temperature": llm_client.TEMPERATURE,
|
||||
"idea": entry["idea"], "designId": entry["designId"],
|
||||
"llmAttempts": attempts, "ts": time.strftime("%Y-%m-%dT%H:%M:%S"),
|
||||
"responseMeta": raw_response_meta,
|
||||
"rescuedByExplicitMaxTokens": rescued, # True=末次抢救通道产出（见上方注释，审计用）
|
||||
"rawContent": raw_content,
|
||||
"findings": findings,
|
||||
}, fh, ensure_ascii=False, indent=2)
|
||||
with open(CALLLOG, "a", encoding="utf-8") as fh:
|
||||
fh.write(json.dumps({"entryId": entry["entryId"], "runTag": run_tag, "model": model,
|
||||
"attempts": attempts, "budgetUsed": BUDGET.used,
|
||||
"ts": time.strftime("%H:%M:%S")}, ensure_ascii=False) + "\n")
|
||||
return {"entryId": entry["entryId"], "runTag": run_tag, "model": model, "findings": findings,
|
||||
"hasP0": any(x["severity"] == "P0" for x in findings),
|
||||
"hasP1": any(x["severity"] == "P1" for x in findings),
|
||||
"passNoP0P1": not any(x["severity"] in ("P0", "P1") for x in findings),
|
||||
"llmAttempts": attempts, "reused": False}
|
||||
|
||||
|
||||
def main():
|
||||
if not os.environ.get("NEWAPI_KEY"):
|
||||
raise SystemExit("缺 NEWAPI_KEY 环境变量（密钥严禁入盘，只走命令行环境变量）")
|
||||
os.makedirs(RAW_DIR, exist_ok=True)
|
||||
entries = json.load(open(GOLDEN, "r", encoding="utf-8"))
|
||||
assert len(entries) == 25, "Golden 集应为 25 条"
|
||||
by_id = {e["entryId"]: e for e in entries}
|
||||
|
||||
store = prompts_mod.PromptStore(PROMPTS_DIR)
|
||||
store.require([PROMPT_ID])
|
||||
client = llm_client.LlmClient(model=MAIN_MODEL)
|
||||
|
||||
results = {"promptVersion": EXPECT_VERSION, "mainModel": MAIN_MODEL, "crossModel": CROSS_MODEL,
|
||||
"temperature": llm_client.TEMPERATURE, "a": [], "d": [], "e": []}
|
||||
|
||||
# ---------------- A：25 条全量主跑（MiniMax）----------------
|
||||
_log("A 段开跑：25 条 × %s（v%s，温度 %.1f）" % (MAIN_MODEL, EXPECT_VERSION, llm_client.TEMPERATURE))
|
||||
for e in entries:
|
||||
r = review_once(store, client, e, "a", MAIN_MODEL)
|
||||
r["source"] = e["source"]
|
||||
# 写入按当前 doctrine 版本解析后的标量期望（版本化双标签见 resolve_expectation；
|
||||
# results.json 保持标量形态，summarize.py 直接消费，无需自行解析版本化结构）
|
||||
r["expectation"] = resolve_expectation(e)
|
||||
results["a"].append(r)
|
||||
_log("A %-12s hasP0=%s hasP1=%s 过=%s（%s%s）"
|
||||
% (r["entryId"], r["hasP0"], r["hasP1"], r["passNoP0P1"],
|
||||
e["source"], "·复用" if r["reused"] else ""))
|
||||
|
||||
# ---------------- D：断言③稳定性双跑（第 1 次=A 段结果，第 2 次同 prompt 同模型重打）----------------
|
||||
_log("D 段开跑：%s 双跑第 2 次" % STABILITY_SAMPLE)
|
||||
a_map = {r["entryId"]: r for r in results["a"]}
|
||||
for eid in STABILITY_SAMPLE:
|
||||
r2 = review_once(store, client, by_id[eid], "stability", MAIN_MODEL)
|
||||
r1 = a_map[eid]
|
||||
consistent = (r1["hasP0"], r1["hasP1"]) == (r2["hasP0"], r2["hasP1"])
|
||||
results["d"].append({"entryId": eid, "run1": [r1["hasP0"], r1["hasP1"]],
|
||||
"run2": [r2["hasP0"], r2["hasP1"]], "consistent": consistent,
|
||||
"findings2": r2["findings"]})
|
||||
_log("D %-12s run1=%s run2=%s 一致=%s" % (eid, results["d"][-1]["run1"],
|
||||
results["d"][-1]["run2"], consistent))
|
||||
|
||||
# ---------------- E：断言④交叉复核（deepseek-v4-flash，同 5 条）----------------
|
||||
_log("E 段开跑：%s × %s" % (STABILITY_SAMPLE, CROSS_MODEL))
|
||||
for eid in STABILITY_SAMPLE:
|
||||
rx = review_once(store, client, by_id[eid], "cross", CROSS_MODEL)
|
||||
r1 = a_map[eid]
|
||||
diverge = rx["passNoP0P1"] != r1["passNoP0P1"] # 结论口径：「无 P0/P1=过」
|
||||
results["e"].append({"entryId": eid, "miniMaxPass": r1["passNoP0P1"],
|
||||
"crossPass": rx["passNoP0P1"],
|
||||
"crossTuple": [rx["hasP0"], rx["hasP1"]],
|
||||
"diverge": diverge, "findingsCross": rx["findings"]})
|
||||
_log("E %-12s MiniMax过=%s 交叉过=%s 分歧=%s" % (eid, r1["passNoP0P1"],
|
||||
rx["passNoP0P1"], diverge))
|
||||
|
||||
results["llmCallsUsed"] = BUDGET.used
|
||||
with open(RESULTS, "w", encoding="utf-8") as fh:
|
||||
json.dump(results, fh, ensure_ascii=False, indent=2)
|
||||
_log("回归完成：LLM 调用（HTTP 尝试口径）共 %d 次，结果已落 %s" % (BUDGET.used, RESULTS))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@ -0,0 +1,35 @@
|
||||
{"entryId": "g9011", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 1, "ts": "03:25:06"}
|
||||
{"entryId": "g9012", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 2, "ts": "03:25:14"}
|
||||
{"entryId": "g9013", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 3, "ts": "03:25:28"}
|
||||
{"entryId": "g9014", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 4, "ts": "03:25:58"}
|
||||
{"entryId": "g9015", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 5, "ts": "03:26:50"}
|
||||
{"entryId": "g9016", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 2, "budgetUsed": 7, "ts": "03:26:58"}
|
||||
{"entryId": "g9017", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 3, "budgetUsed": 10, "ts": "03:27:11"}
|
||||
{"entryId": "k-0cfbc296", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 11, "ts": "03:27:14"}
|
||||
{"entryId": "g9019", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 12, "ts": "03:28:31"}
|
||||
{"entryId": "g9020", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 2, "budgetUsed": 14, "ts": "03:29:31"}
|
||||
{"entryId": "g9021", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 3, "budgetUsed": 3, "ts": "03:32:56"}
|
||||
{"entryId": "g9022", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 4, "ts": "03:33:01"}
|
||||
{"entryId": "g9023", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 5, "ts": "03:33:19"}
|
||||
{"entryId": "g9024", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 6, "ts": "03:33:23"}
|
||||
{"entryId": "k-26ce4440", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 7, "ts": "03:33:26"}
|
||||
{"entryId": "g9026", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 3, "budgetUsed": 10, "ts": "03:34:21"}
|
||||
{"entryId": "k-a04dfac9", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 11, "ts": "03:34:36"}
|
||||
{"entryId": "g9028", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 3, "budgetUsed": 14, "ts": "03:35:38"}
|
||||
{"entryId": "g9030", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 2, "budgetUsed": 16, "ts": "03:36:22"}
|
||||
{"entryId": "w2-1", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 17, "ts": "03:36:26"}
|
||||
{"entryId": "w2-2", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 1, "ts": "03:39:00"}
|
||||
{"entryId": "w2-3", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 2, "budgetUsed": 3, "ts": "03:39:31"}
|
||||
{"entryId": "w2-4", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 4, "ts": "03:39:57"}
|
||||
{"entryId": "w2-5", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 5, "ts": "03:40:12"}
|
||||
{"entryId": "w2-6", "runTag": "a", "model": "MiniMax-M2.7", "attempts": 2, "budgetUsed": 7, "ts": "03:40:26"}
|
||||
{"entryId": "g9019", "runTag": "stability", "model": "MiniMax-M2.7", "attempts": 2, "budgetUsed": 9, "ts": "03:41:14"}
|
||||
{"entryId": "g9021", "runTag": "stability", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 10, "ts": "03:41:18"}
|
||||
{"entryId": "k-0cfbc296", "runTag": "stability", "model": "MiniMax-M2.7", "attempts": 2, "budgetUsed": 12, "ts": "03:41:31"}
|
||||
{"entryId": "g9011", "runTag": "stability", "model": "MiniMax-M2.7", "attempts": 2, "budgetUsed": 14, "ts": "03:41:41"}
|
||||
{"entryId": "w2-6", "runTag": "stability", "model": "MiniMax-M2.7", "attempts": 1, "budgetUsed": 15, "ts": "03:41:48"}
|
||||
{"entryId": "g9019", "runTag": "cross", "model": "deepseek-v4-flash", "attempts": 1, "budgetUsed": 16, "ts": "03:41:57"}
|
||||
{"entryId": "g9021", "runTag": "cross", "model": "deepseek-v4-flash", "attempts": 1, "budgetUsed": 17, "ts": "03:42:05"}
|
||||
{"entryId": "k-0cfbc296", "runTag": "cross", "model": "deepseek-v4-flash", "attempts": 1, "budgetUsed": 18, "ts": "03:42:12"}
|
||||
{"entryId": "g9011", "runTag": "cross", "model": "deepseek-v4-flash", "attempts": 1, "budgetUsed": 19, "ts": "03:42:16"}
|
||||
{"entryId": "w2-6", "runTag": "cross", "model": "deepseek-v4-flash", "attempts": 1, "budgetUsed": 20, "ts": "03:42:25"}
|
||||
@ -0,0 +1,21 @@
|
||||
{
|
||||
"entryId": "g9011",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.1",
|
||||
"temperature": 0.2,
|
||||
"idea": "王蓝莓的小卖部收银台，给顾客结账",
|
||||
"designId": "4e2de887ceed035e571db5ed0f79462f1f52d6d4944805ce7b8e65a63ce1585c",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T03:25:06",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[{\"severity\":\"P1\",\"issue\":\"创意原文明确提到“王的蓝莓的小卖部收银台”，强调角色身份“王的蓝莓”，但 config 中 title 仅写“小卖部·收银时刻”、theme 仅写“校园便利店的热闹柜台”，均未体现“王的蓝莓”这一核心角色名称，题文明显脱节\",\"suggestion\":\"在 title 或 theme 中补充角色标识，如改为“小卖部·王的蓝莓收银时刻”或在 theme 中加入“王的蓝莓便利店”字样\"}]}",
|
||||
"findings": [
|
||||
{
|
||||
"severity": "P1",
|
||||
"issue": "创意原文明确提到“王的蓝莓的小卖部收银台”，强调角色身份“王的蓝莓”，但 config 中 title 仅写“小卖部·收银时刻”、theme 仅写“校园便利店的热闹柜台”，均未体现“王的蓝莓”这一核心角色名称，题文明显脱节",
|
||||
"suggestion": "在 title 或 theme 中补充角色标识，如改为“小卖部·王的蓝莓收银时刻”或在 theme 中加入“王的蓝莓便利店”字样"
|
||||
}
|
||||
]
|
||||
}
|
||||
@ -0,0 +1,32 @@
|
||||
{
|
||||
"entryId": "g9011",
|
||||
"runTag": "cross",
|
||||
"model": "deepseek-v4-flash",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.1",
|
||||
"temperature": 0.2,
|
||||
"idea": "王蓝莓的小卖部收银台，给顾客结账",
|
||||
"designId": "4e2de887ceed035e571db5ed0f79462f1f52d6d4944805ce7b8e65a63ce1585c",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T03:42:16",
|
||||
"responseMeta": {
|
||||
"model": "deepseek-v4-flash",
|
||||
"usage": {
|
||||
"prompt_tokens": 903,
|
||||
"completion_tokens": 206,
|
||||
"total_tokens": 1109,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 640
|
||||
},
|
||||
"completion_tokens_details": {
|
||||
"reasoning_tokens": 199
|
||||
},
|
||||
"prompt_cache_hit_tokens": 640,
|
||||
"prompt_cache_miss_tokens": 263
|
||||
},
|
||||
"finish_reason": "stop"
|
||||
},
|
||||
"rescuedByExplicitMaxTokens": false,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,48 @@
|
||||
{
|
||||
"entryId": "g9011",
|
||||
"runTag": "stability",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.1",
|
||||
"temperature": 0.2,
|
||||
"idea": "王蓝莓的小卖部收银台，给顾客结账",
|
||||
"designId": "4e2de887ceed035e571db5ed0f79462f1f52d6d4944805ce7b8e65a63ce1585c",
|
||||
"llmAttempts": 2,
|
||||
"ts": "2026-06-10T03:41:41",
|
||||
"responseMeta": {
|
||||
"model": "MiniMax-M2.7",
|
||||
"usage": {
|
||||
"prompt_tokens": 877,
|
||||
"completion_tokens": 979,
|
||||
"total_tokens": 1856,
|
||||
"usage_semantic": "openai",
|
||||
"usage_source": "anthropic",
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 864,
|
||||
"text_tokens": 0,
|
||||
"audio_tokens": 0,
|
||||
"image_tokens": 0
|
||||
},
|
||||
"completion_tokens_details": {
|
||||
"text_tokens": 0,
|
||||
"audio_tokens": 0,
|
||||
"reasoning_tokens": 0
|
||||
},
|
||||
"input_tokens": 877,
|
||||
"output_tokens": 0,
|
||||
"input_tokens_details": null,
|
||||
"claude_cache_creation_5_m_tokens": 0,
|
||||
"claude_cache_creation_1_h_tokens": 0
|
||||
},
|
||||
"finish_reason": "stop"
|
||||
},
|
||||
"rescuedByExplicitMaxTokens": true,
|
||||
"rawContent": "{\"findings\":[{\"severity\":\"P1\",\"issue\":\"title 缺少创意原文中的品牌名称“王蓝莓”，导致标题与 idea “王蓝莓的小卖部收银台”关联性不足\",\"suggestion\":\"将 title 改为包含“王蓝莓”的表述，如“王蓝莓·收银时刻”或直接使用“王蓝莓的小卖部”\"}]}",
|
||||
"findings": [
|
||||
{
|
||||
"severity": "P1",
|
||||
"issue": "title 缺少创意原文中的品牌名称“王蓝莓”，导致标题与 idea “王蓝莓的小卖部收银台”关联性不足",
|
||||
"suggestion": "将 title 改为包含“王蓝莓”的表述，如“王蓝莓·收银时刻”或直接使用“王蓝莓的小卖部”"
|
||||
}
|
||||
]
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "g9012",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.1",
|
||||
"temperature": 0.2,
|
||||
"idea": "太空舱里捡星星补给能量",
|
||||
"designId": "115589c5bc4a8f572c087cc20a7373f16064eccce621cc038e458616f5485618",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T03:25:14",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "g9013",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.1",
|
||||
"temperature": 0.2,
|
||||
"idea": "农场里收割成熟的蔬菜",
|
||||
"designId": "6f091b6da8142dda57ab8b132403c2ab62e00b5824a20a10d1156ea982bda32d",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T03:25:28",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "g9014",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.1",
|
||||
"temperature": 0.2,
|
||||
"idea": "深夜便利店招待来买夜宵的顾客",
|
||||
"designId": "a1869689d1b746b894edce5b93514dca856121f937056bb42ced6d8ea489f243",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T03:25:58",
|
||||
"responseMeta": null,
|
||||
"rawContent": "\n{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "g9015",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.1",
|
||||
"temperature": 0.2,
|
||||
"idea": "校园运动会上的接力赛冲刺",
|
||||
"designId": "b2603e82f07a463bfec7dab27272daab1cc760e1d108bb0ae3811322c4daa24b",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T03:26:50",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "g9016",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.1",
|
||||
"temperature": 0.2,
|
||||
"idea": "猫咖啡馆里撸猫攒爱心",
|
||||
"designId": "b00027fe8f60f0d08a84c12e0bf91c25448a7d1b3a3a35b6f04d4957f2e66ad9",
|
||||
"llmAttempts": 2,
|
||||
"ts": "2026-06-10T03:26:58",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "g9017",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.1",
|
||||
"temperature": 0.2,
|
||||
"idea": "春节集市上抢福袋",
|
||||
"designId": "43afd1f72ac6cf454faeb54cb50343e31cb06fd7ba1e07d92567debc57a05036",
|
||||
"llmAttempts": 3,
|
||||
"ts": "2026-06-10T03:27:11",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,21 @@
|
||||
{
|
||||
"entryId": "g9019",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.1",
|
||||
"temperature": 0.2,
|
||||
"idea": "烧烤摊翻烤串，别烤糊了",
|
||||
"designId": "439034c731e5530c94685dad552ddf6395eee6c00f70e7cad08451cbfa656012",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T03:28:31",
|
||||
"responseMeta": null,
|
||||
"rawContent": "\n{\"findings\":[{\"severity\":\"P1\",\"issue\":\"target=18 与 designIntent 中“精准把控火候、避免烤糊”脱节——翻动次数应与烤串火候状态挂钩，而非机械计数\",\"suggestion\":\"在 config 中补充烤串状态机制说明（如烤串变色进度、烤糊倒计时或每根烤串需翻动的时机窗口），明确 target=18 的含义（18根烤串？18次翻动机会？），使玩家目标与“避免烤糊”的核心意图一致\"}]}",
|
||||
"findings": [
|
||||
{
|
||||
"severity": "P1",
|
||||
"issue": "target=18 与 designIntent 中“精准把控火候、避免烤糊”脱节——翻动次数应与烤串火候状态挂钩，而非机械计数",
|
||||
"suggestion": "在 config 中补充烤串状态机制说明（如烤串变色进度、烤糊倒计时或每根烤串需翻动的时机窗口），明确 target=18 的含义（18根烤串？18次翻动机会？），使玩家目标与“避免烤糊”的核心意图一致"
|
||||
}
|
||||
]
|
||||
}
|
||||
@ -0,0 +1,38 @@
|
||||
{
|
||||
"entryId": "g9019",
|
||||
"runTag": "cross",
|
||||
"model": "deepseek-v4-flash",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.1",
|
||||
"temperature": 0.2,
|
||||
"idea": "烧烤摊翻烤串，别烤糊了",
|
||||
"designId": "439034c731e5530c94685dad552ddf6395eee6c00f70e7cad08451cbfa656012",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T03:41:57",
|
||||
"responseMeta": {
|
||||
"model": "deepseek-v4-flash",
|
||||
"usage": {
|
||||
"prompt_tokens": 922,
|
||||
"completion_tokens": 595,
|
||||
"total_tokens": 1517,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
},
|
||||
"completion_tokens_details": {
|
||||
"reasoning_tokens": 529
|
||||
},
|
||||
"prompt_cache_hit_tokens": 0,
|
||||
"prompt_cache_miss_tokens": 922
|
||||
},
|
||||
"finish_reason": "stop"
|
||||
},
|
||||
"rescuedByExplicitMaxTokens": false,
|
||||
"rawContent": "{\"findings\":[{\"severity\":\"P2\",\"issue\":\"创意中的'别烤糊了'在设计中未通过机制体现，仅依赖点击计数，缺乏烤糊警示或失败条件\",\"suggestion\":\"考虑在界面或反馈中增加烤糊相关的视觉或文字提示，强化创意核心\"}]}",
|
||||
"findings": [
|
||||
{
|
||||
"severity": "P2",
|
||||
"issue": "创意中的'别烤糊了'在设计中未通过机制体现，仅依赖点击计数，缺乏烤糊警示或失败条件",
|
||||
"suggestion": "考虑在界面或反馈中增加烤糊相关的视觉或文字提示，强化创意核心"
|
||||
}
|
||||
]
|
||||
}
|
||||
@ -0,0 +1,48 @@
|
||||
{
|
||||
"entryId": "g9019",
|
||||
"runTag": "stability",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.1",
|
||||
"temperature": 0.2,
|
||||
"idea": "烧烤摊翻烤串，别烤糊了",
|
||||
"designId": "439034c731e5530c94685dad552ddf6395eee6c00f70e7cad08451cbfa656012",
|
||||
"llmAttempts": 2,
|
||||
"ts": "2026-06-10T03:41:14",
|
||||
"responseMeta": {
|
||||
"model": "MiniMax-M2.7",
|
||||
"usage": {
|
||||
"prompt_tokens": 897,
|
||||
"completion_tokens": 950,
|
||||
"total_tokens": 1847,
|
||||
"usage_semantic": "openai",
|
||||
"usage_source": "anthropic",
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 787,
|
||||
"text_tokens": 0,
|
||||
"audio_tokens": 0,
|
||||
"image_tokens": 0
|
||||
},
|
||||
"completion_tokens_details": {
|
||||
"text_tokens": 0,
|
||||
"audio_tokens": 0,
|
||||
"reasoning_tokens": 0
|
||||
},
|
||||
"input_tokens": 897,
|
||||
"output_tokens": 0,
|
||||
"input_tokens_details": null,
|
||||
"claude_cache_creation_5_m_tokens": 0,
|
||||
"claude_cache_creation_1_h_tokens": 0
|
||||
},
|
||||
"finish_reason": "stop"
|
||||
},
|
||||
"rescuedByExplicitMaxTokens": true,
|
||||
"rawContent": "\n{\"findings\":[{\"severity\":\"P1\",\"issue\":\"创意原文中“烧烤摊翻烤串，别烤糊了”的核心是“避免烤糊”，但 config 中的 scoreLabel 为“翻烤次数”、target 为 18 次，设计导向是追求翻烤次数而非避免烤糊失败，创意核心与游戏目标脱节\",\"suggestion\":\"调整 scoreLabel 为体现“避免烤糊”的指标（如烤糊次数或烤串状态），或调整 target 为避免烤糊相关的目标（如规定时间内保持不烤糊），使目标与创意核心“避免烤糊”保持一致\"}]}",
|
||||
"findings": [
|
||||
{
|
||||
"severity": "P1",
|
||||
"issue": "创意原文中“烧烤摊翻烤串，别烤糊了”的核心是“避免烤糊”，但 config 中的 scoreLabel 为“翻烤次数”、target 为 18 次，设计导向是追求翻烤次数而非避免烤糊失败，创意核心与游戏目标脱节",
|
||||
"suggestion": "调整 scoreLabel 为体现“避免烤糊”的指标（如烤糊次数或烤串状态），或调整 target 为避免烤糊相关的目标（如规定时间内保持不烤糊），使目标与创意核心“避免烤糊”保持一致"
|
||||
}
|
||||
]
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "g9020",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.1",
|
||||
"temperature": 0.2,
|
||||
"idea": "地铁早高峰挤上即将关门的车",
|
||||
"designId": "005cde6a7d097d691745726db5fdc1b54e6e935b448857b486752e92fd1a312f",
|
||||
"llmAttempts": 2,
|
||||
"ts": "2026-06-10T03:29:31",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "g9021",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.1",
|
||||
"temperature": 0.2,
|
||||
"idea": "做一个很好玩的游戏",
|
||||
"designId": "526aaaa99ab651ff89fd00dfc2b8b401e5c03b23f19b05a17b1757e526671eee",
|
||||
"llmAttempts": 3,
|
||||
"ts": "2026-06-10T03:32:56",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,38 @@
|
||||
{
|
||||
"entryId": "g9021",
|
||||
"runTag": "cross",
|
||||
"model": "deepseek-v4-flash",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.1",
|
||||
"temperature": 0.2,
|
||||
"idea": "做一个很好玩的游戏",
|
||||
"designId": "526aaaa99ab651ff89fd00dfc2b8b401e5c03b23f19b05a17b1757e526671eee",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T03:42:05",
|
||||
"responseMeta": {
|
||||
"model": "deepseek-v4-flash",
|
||||
"usage": {
|
||||
"prompt_tokens": 892,
|
||||
"completion_tokens": 473,
|
||||
"total_tokens": 1365,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 256
|
||||
},
|
||||
"completion_tokens_details": {
|
||||
"reasoning_tokens": 388
|
||||
},
|
||||
"prompt_cache_hit_tokens": 256,
|
||||
"prompt_cache_miss_tokens": 636
|
||||
},
|
||||
"finish_reason": "stop"
|
||||
},
|
||||
"rescuedByExplicitMaxTokens": false,
|
||||
"rawContent": "{\"findings\":[{\"severity\":\"P2\",\"issue\":\"设计意图描述了温馨治愈的糖果屋场景，但当前clicker模板仅支持简单点击计数，无法呈现场景视觉与情感体验，模板表达力不足。\",\"suggestion\":\"建议将设计意图调整为更贴合点击器核心机制（如收集计数）的体验描述，或更改为支持场景与动画的模板。\"}]}",
|
||||
"findings": [
|
||||
{
|
||||
"severity": "P2",
|
||||
"issue": "设计意图描述了温馨治愈的糖果屋场景，但当前clicker模板仅支持简单点击计数，无法呈现场景视觉与情感体验，模板表达力不足。",
|
||||
"suggestion": "建议将设计意图调整为更贴合点击器核心机制（如收集计数）的体验描述，或更改为支持场景与动画的模板。"
|
||||
}
|
||||
]
|
||||
}
|
||||
@ -0,0 +1,16 @@
|
||||
{
|
||||
"entryId": "g9021",
|
||||
"runTag": "stability",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.1",
|
||||
"temperature": 0.2,
|
||||
"idea": "做一个很好玩的游戏",
|
||||
"designId": "526aaaa99ab651ff89fd00dfc2b8b401e5c03b23f19b05a17b1757e526671eee",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T03:41:18",
|
||||
"responseMeta": null,
|
||||
"rescuedByExplicitMaxTokens": false,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "g9022",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.1",
|
||||
"temperature": 0.2,
|
||||
"idea": "解压点点点",
|
||||
"designId": "b52388c4e4d72182a728ef86f03bb699292d4015f108b6426218f6170d63653b",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T03:33:01",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "g9023",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.1",
|
||||
"temperature": 0.2,
|
||||
"idea": "中秋节帮嫦娥把月饼送上月宫",
|
||||
"designId": "cb3f7444796b29723b74e08807a6c94555e8f98c4a4f0c0c7cf7bd50c4c90dcb",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T03:33:19",
|
||||
"responseMeta": null,
|
||||
"rawContent": "\n{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "g9024",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.1",
|
||||
"temperature": 0.2,
|
||||
"idea": "雨夜帮路边馄饨摊老板撑伞收摊",
|
||||
"designId": "090212e4c88d43b691aba53037bd586f1070dad5b7aeae790a785a31c351c5f5",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T03:33:23",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "g9026",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.1",
|
||||
"temperature": 0.2,
|
||||
"idea": "奶茶店爆单日疯狂摇珍珠奶茶",
|
||||
"designId": "664cf2034f2bc086f6780020f9fe02a9a1e379fc2b33daa430e34841e3e30741",
|
||||
"llmAttempts": 3,
|
||||
"ts": "2026-06-10T03:34:21",
|
||||
"responseMeta": null,
|
||||
"rawContent": "\n{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "g9028",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.1",
|
||||
"temperature": 0.2,
|
||||
"idea": "给办公室加班的同事分下午茶蛋糕",
|
||||
"designId": "53933f300e7a9c254986a2972be2c9c577b1233856336f18a20ed5a17bc6ae69",
|
||||
"llmAttempts": 3,
|
||||
"ts": "2026-06-10T03:35:38",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "g9030",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.1",
|
||||
"temperature": 0.2,
|
||||
"idea": "火锅店捞毛肚，七上八下别捞老了",
|
||||
"designId": "7ebba6f04540bce08c531189024c8839c26424b84ef224ea8dfa5a64c8fa5076",
|
||||
"llmAttempts": 2,
|
||||
"ts": "2026-06-10T03:36:22",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,21 @@
|
||||
{
|
||||
"entryId": "k-0cfbc296",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.1",
|
||||
"temperature": 0.2,
|
||||
"idea": "海底捞珍珠同时躲开鲨鱼",
|
||||
"designId": "0cfbc29612b6966447cf7449d6cdef8da1f57521f353bdc726b48fa0aa621048",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T03:27:14",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[{\"severity\":\"P1\",\"issue\":\"玩法核心动作与创意意图明显矛盾：创意原意为『躲避』鲨鱼，但 config 中描述的是『点击鲨鱼将其驱离』，改变了玩家与鲨鱼的核心交互机制\",\"suggestion\":\"将鲨鱼的触发机制改为：鲨鱼出现时玩家需点击周围安全区域或避开按钮实现躲避，而非直接点击鲨鱼\"}]}",
|
||||
"findings": [
|
||||
{
|
||||
"severity": "P1",
|
||||
"issue": "玩法核心动作与创意意图明显矛盾：创意原意为『躲避』鲨鱼，但 config 中描述的是『点击鲨鱼将其驱离』，改变了玩家与鲨鱼的核心交互机制",
|
||||
"suggestion": "将鲨鱼的触发机制改为：鲨鱼出现时玩家需点击周围安全区域或避开按钮实现躲避，而非直接点击鲨鱼"
|
||||
}
|
||||
]
|
||||
}
|
||||
@ -0,0 +1,38 @@
|
||||
{
|
||||
"entryId": "k-0cfbc296",
|
||||
"runTag": "cross",
|
||||
"model": "deepseek-v4-flash",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.1",
|
||||
"temperature": 0.2,
|
||||
"idea": "海底捞珍珠同时躲开鲨鱼",
|
||||
"designId": "0cfbc29612b6966447cf7449d6cdef8da1f57521f353bdc726b48fa0aa621048",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T03:42:12",
|
||||
"responseMeta": {
|
||||
"model": "deepseek-v4-flash",
|
||||
"usage": {
|
||||
"prompt_tokens": 976,
|
||||
"completion_tokens": 442,
|
||||
"total_tokens": 1418,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 640
|
||||
},
|
||||
"completion_tokens_details": {
|
||||
"reasoning_tokens": 365
|
||||
},
|
||||
"prompt_cache_hit_tokens": 640,
|
||||
"prompt_cache_miss_tokens": 336
|
||||
},
|
||||
"finish_reason": "stop"
|
||||
},
|
||||
"rescuedByExplicitMaxTokens": false,
|
||||
"rawContent": "{\"findings\":[{\"severity\":\"P1\",\"issue\":\"创意要求'躲开鲨鱼'，但设计改为点击鲨鱼驱离，核心动作由躲避变为主动攻击，与创意不符。\",\"suggestion\":\"将玩法改为躲避鲨鱼，例如鲨鱼出现时点击空白区域或停止点击，若点击鲨鱼则触发惩罚。\"}]}",
|
||||
"findings": [
|
||||
{
|
||||
"severity": "P1",
|
||||
"issue": "创意要求'躲开鲨鱼'，但设计改为点击鲨鱼驱离，核心动作由躲避变为主动攻击，与创意不符。",
|
||||
"suggestion": "将玩法改为躲避鲨鱼，例如鲨鱼出现时点击空白区域或停止点击，若点击鲨鱼则触发惩罚。"
|
||||
}
|
||||
]
|
||||
}
|
||||
@ -0,0 +1,48 @@
|
||||
{
|
||||
"entryId": "k-0cfbc296",
|
||||
"runTag": "stability",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.1",
|
||||
"temperature": 0.2,
|
||||
"idea": "海底捞珍珠同时躲开鲨鱼",
|
||||
"designId": "0cfbc29612b6966447cf7449d6cdef8da1f57521f353bdc726b48fa0aa621048",
|
||||
"llmAttempts": 2,
|
||||
"ts": "2026-06-10T03:41:31",
|
||||
"responseMeta": {
|
||||
"model": "MiniMax-M2.7",
|
||||
"usage": {
|
||||
"prompt_tokens": 937,
|
||||
"completion_tokens": 602,
|
||||
"total_tokens": 1539,
|
||||
"usage_semantic": "openai",
|
||||
"usage_source": "anthropic",
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 928,
|
||||
"text_tokens": 0,
|
||||
"audio_tokens": 0,
|
||||
"image_tokens": 0
|
||||
},
|
||||
"completion_tokens_details": {
|
||||
"text_tokens": 0,
|
||||
"audio_tokens": 0,
|
||||
"reasoning_tokens": 0
|
||||
},
|
||||
"input_tokens": 937,
|
||||
"output_tokens": 0,
|
||||
"input_tokens_details": null,
|
||||
"claude_cache_creation_5_m_tokens": 0,
|
||||
"claude_cache_creation_1_h_tokens": 0
|
||||
},
|
||||
"finish_reason": "stop"
|
||||
},
|
||||
"rescuedByExplicitMaxTokens": true,
|
||||
"rawContent": "{\"findings\":[{\"severity\":\"P1\",\"issue\":\"创意核心动作是『躲避鲨鱼』，但 config 中鲨鱼出现后需点击鲨鱼将其驱离，属于核心动作被改写为『点击消灭/驱离』，与创意『躲避』语义直接矛盾\",\"suggestion\":\"将鲨鱼的应对方式调整为非点击主动消灭的形式，例如鲨鱼出现时玩家需快速点击周围安全区域躲避、或在鲨鱼出现前预留躲避反应时间等，保持『躲避鲨鱼』的创意核心动作不变\"}]}",
|
||||
"findings": [
|
||||
{
|
||||
"severity": "P1",
|
||||
"issue": "创意核心动作是『躲避鲨鱼』，但 config 中鲨鱼出现后需点击鲨鱼将其驱离，属于核心动作被改写为『点击消灭/驱离』，与创意『躲避』语义直接矛盾",
|
||||
"suggestion": "将鲨鱼的应对方式调整为非点击主动消灭的形式，例如鲨鱼出现时玩家需快速点击周围安全区域躲避、或在鲨鱼出现前预留躲避反应时间等，保持『躲避鲨鱼』的创意核心动作不变"
|
||||
}
|
||||
]
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "k-26ce4440",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.1",
|
||||
"temperature": 0.2,
|
||||
"idea": "图书馆闭馆前把还回来的书归架",
|
||||
"designId": "26ce44408e55eaa604b3efa2903c3cd5a541ab99ed8aa7c62c9b8192ef03260e",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T03:33:26",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,15 @@
|
||||
{
|
||||
"entryId": "k-a04dfac9",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.1",
|
||||
"temperature": 0.2,
|
||||
"idea": "跨年夜零点在广场上点烟花倒计时",
|
||||
"designId": "a04dfac9bba70833a1dcbce8242a787f3180db5b516b65006589698878eabcc6",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T03:34:36",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,21 @@
|
||||
{
|
||||
"entryId": "w2-1",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.1",
|
||||
"temperature": 0.2,
|
||||
"idea": "王蓝莓的小卖部收银台，给顾客结账",
|
||||
"designId": "8f9108324a3bef55f471773b697a48047cf4c5b70b46a97a33696a5e4ebc39a1",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T03:36:26",
|
||||
"responseMeta": null,
|
||||
"rawContent": "{\"findings\":[{\"severity\":\"P1\",\"issue\":\"title角色名与创意原文不一致：创意为'王蓝莓'，title写作'蓝莓妹'，存在角色命名冲突\",\"suggestion\":\"将title中的'蓝莓妹'改为'王蓝莓'以保持与创意原文的角色一致性\"}]}",
|
||||
"findings": [
|
||||
{
|
||||
"severity": "P1",
|
||||
"issue": "title角色名与创意原文不一致：创意为'王蓝莓'，title写作'蓝莓妹'，存在角色命名冲突",
|
||||
"suggestion": "将title中的'蓝莓妹'改为'王蓝莓'以保持与创意原文的角色一致性"
|
||||
}
|
||||
]
|
||||
}
|
||||
@ -0,0 +1,16 @@
|
||||
{
|
||||
"entryId": "w2-2",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.1",
|
||||
"temperature": 0.2,
|
||||
"idea": "深夜便利店招待来买夜宵的顾客",
|
||||
"designId": "b62350bae955d7c3a510efa0eed0f74c7408ba892b8f0b1e59a367d6456a8425",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T03:39:00",
|
||||
"responseMeta": null,
|
||||
"rescuedByExplicitMaxTokens": false,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,48 @@
|
||||
{
|
||||
"entryId": "w2-3",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.1",
|
||||
"temperature": 0.2,
|
||||
"idea": "猫咖啡馆里撸猫攒爱心",
|
||||
"designId": "2b15bd30b466f12653be4dac5d53ac5b856110986e00d6926c62e8c3ece809c7",
|
||||
"llmAttempts": 2,
|
||||
"ts": "2026-06-10T03:39:31",
|
||||
"responseMeta": {
|
||||
"model": "MiniMax-M2.7",
|
||||
"usage": {
|
||||
"prompt_tokens": 864,
|
||||
"completion_tokens": 925,
|
||||
"total_tokens": 1789,
|
||||
"usage_semantic": "openai",
|
||||
"usage_source": "anthropic",
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 832,
|
||||
"text_tokens": 0,
|
||||
"audio_tokens": 0,
|
||||
"image_tokens": 0
|
||||
},
|
||||
"completion_tokens_details": {
|
||||
"text_tokens": 0,
|
||||
"audio_tokens": 0,
|
||||
"reasoning_tokens": 0
|
||||
},
|
||||
"input_tokens": 864,
|
||||
"output_tokens": 0,
|
||||
"input_tokens_details": null,
|
||||
"claude_cache_creation_5_m_tokens": 0,
|
||||
"claude_cache_creation_1_h_tokens": 0
|
||||
},
|
||||
"finish_reason": "stop"
|
||||
},
|
||||
"rescuedByExplicitMaxTokens": true,
|
||||
"rawContent": "{\"findings\":[{\"severity\":\"P2\",\"issue\":\"designIntent 提及「通关」概念，但 clicker 模板通常为无尽点击模式，通关机制与模板语义存在偏差。\",\"suggestion\":\"建议明确「通关」的含义——若指达到目标爱心数即结束，可改为「达成目标爱心数即完成」；若保留无尽模式则去掉通关表述，聚焦于收集过程。\"}]}",
|
||||
"findings": [
|
||||
{
|
||||
"severity": "P2",
|
||||
"issue": "designIntent 提及「通关」概念，但 clicker 模板通常为无尽点击模式，通关机制与模板语义存在偏差。",
|
||||
"suggestion": "建议明确「通关」的含义——若指达到目标爱心数即结束，可改为「达成目标爱心数即完成」；若保留无尽模式则去掉通关表述，聚焦于收集过程。"
|
||||
}
|
||||
]
|
||||
}
|
||||
@ -0,0 +1,16 @@
|
||||
{
|
||||
"entryId": "w2-4",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.1",
|
||||
"temperature": 0.2,
|
||||
"idea": "烧烤摊翻烤串，别烤糊了",
|
||||
"designId": "1b3c5764b01c161d1c0cbd6aaba2c7375280b2469a850152b6596a22b55d7389",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T03:39:57",
|
||||
"responseMeta": null,
|
||||
"rescuedByExplicitMaxTokens": false,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,16 @@
|
||||
{
|
||||
"entryId": "w2-5",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.1",
|
||||
"temperature": 0.2,
|
||||
"idea": "地铁早高峰挤上即将关门的车",
|
||||
"designId": "feca768c87f9fcb3111a504ab074f8e998129c9de01e833957f4039c6ffbef5a",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T03:40:12",
|
||||
"responseMeta": null,
|
||||
"rescuedByExplicitMaxTokens": false,
|
||||
"rawContent": "\n{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,42 @@
|
||||
{
|
||||
"entryId": "w2-6",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.1",
|
||||
"temperature": 0.2,
|
||||
"idea": "解压点点点",
|
||||
"designId": "d0b8c8ea60d5d07c220e1c903f8e78ed36d21ad816b4ee82d2dae8d13af0388a",
|
||||
"llmAttempts": 2,
|
||||
"ts": "2026-06-10T03:40:26",
|
||||
"responseMeta": {
|
||||
"model": "MiniMax-M2.7",
|
||||
"usage": {
|
||||
"prompt_tokens": 855,
|
||||
"completion_tokens": 296,
|
||||
"total_tokens": 1151,
|
||||
"usage_semantic": "openai",
|
||||
"usage_source": "anthropic",
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 832,
|
||||
"text_tokens": 0,
|
||||
"audio_tokens": 0,
|
||||
"image_tokens": 0
|
||||
},
|
||||
"completion_tokens_details": {
|
||||
"text_tokens": 0,
|
||||
"audio_tokens": 0,
|
||||
"reasoning_tokens": 0
|
||||
},
|
||||
"input_tokens": 855,
|
||||
"output_tokens": 0,
|
||||
"input_tokens_details": null,
|
||||
"claude_cache_creation_5_m_tokens": 0,
|
||||
"claude_cache_creation_1_h_tokens": 0
|
||||
},
|
||||
"finish_reason": "stop"
|
||||
},
|
||||
"rescuedByExplicitMaxTokens": true,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,38 @@
|
||||
{
|
||||
"entryId": "w2-6",
|
||||
"runTag": "cross",
|
||||
"model": "deepseek-v4-flash",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.1",
|
||||
"temperature": 0.2,
|
||||
"idea": "解压点点点",
|
||||
"designId": "d0b8c8ea60d5d07c220e1c903f8e78ed36d21ad816b4ee82d2dae8d13af0388a",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T03:42:25",
|
||||
"responseMeta": {
|
||||
"model": "deepseek-v4-flash",
|
||||
"usage": {
|
||||
"prompt_tokens": 887,
|
||||
"completion_tokens": 578,
|
||||
"total_tokens": 1465,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 640
|
||||
},
|
||||
"completion_tokens_details": {
|
||||
"reasoning_tokens": 511
|
||||
},
|
||||
"prompt_cache_hit_tokens": 640,
|
||||
"prompt_cache_miss_tokens": 247
|
||||
},
|
||||
"finish_reason": "stop"
|
||||
},
|
||||
"rescuedByExplicitMaxTokens": false,
|
||||
"rawContent": "{\"findings\":[{\"severity\":\"P2\",\"issue\":\"点击交互缺乏解压感的即时反馈，如音效或动画，导致解压爽感不足\",\"suggestion\":\"建议在每次点击时增加视觉或音效反馈，如气泡破裂声或情绪值动画，增强解压体验\"}]}",
|
||||
"findings": [
|
||||
{
|
||||
"severity": "P2",
|
||||
"issue": "点击交互缺乏解压感的即时反馈，如音效或动画，导致解压爽感不足",
|
||||
"suggestion": "建议在每次点击时增加视觉或音效反馈，如气泡破裂声或情绪值动画，增强解压体验"
|
||||
}
|
||||
]
|
||||
}
|
||||
@ -0,0 +1,16 @@
|
||||
{
|
||||
"entryId": "w2-6",
|
||||
"runTag": "stability",
|
||||
"model": "MiniMax-M2.7",
|
||||
"promptId": "quality.adversary-review",
|
||||
"promptVersion": "1.1.1",
|
||||
"temperature": 0.2,
|
||||
"idea": "解压点点点",
|
||||
"designId": "d0b8c8ea60d5d07c220e1c903f8e78ed36d21ad816b4ee82d2dae8d13af0388a",
|
||||
"llmAttempts": 1,
|
||||
"ts": "2026-06-10T03:41:48",
|
||||
"responseMeta": null,
|
||||
"rescuedByExplicitMaxTokens": false,
|
||||
"rawContent": "{\"findings\":[]}",
|
||||
"findings": []
|
||||
}
|
||||
@ -0,0 +1,153 @@
|
||||
# Golden 集回归报告 —— quality.adversary-review v1.1.1
|
||||
|
||||
> 日期：2026-06-10 · 回归对象：`contracts/prompts/06-quality/adversary-review.md` v1.1.1（细则①句尾补「核心动作矛盾」边界句，主 agent 裁决①）
|
||||
> 配套裁决：②黄金集 k-26ce4440/k-a04dfac9 重标口径版本化双标签（doctrine=v1.0.0→kill；doctrine>=v1.1.0→P2-pass 正向守卫），k-0cfbc296 保持 kill 守卫；③g9017 上轮新报 P1 定性「存疑偏真正报」计入断言②容差
|
||||
> 通道：new-api（100.64.0.8:3000）· 主模型 MiniMax-M2.7（json_object/温度0.2）· 交叉模型 deepseek-v4-flash（推理型，max_tokens≥2048，只读 content）
|
||||
> 复用 golden-regression-v1.1.0 同名脚本与 25 条黄金集（含版本化标签），断点续跑 3 程完成（通道空响应事故见 §6）
|
||||
|
||||
## 一、结论先行（五断言）
|
||||
|
||||
| 断言 | 阈值 | 实测 | 判定 |
|
||||
|---|---|---|---|
|
||||
| A 全量执行 | 25 条合法产出 | **25/25**（21 过 / 4 不过） | 通过 |
|
||||
| ①' kill 守卫 | k-0cfbc296 必须仍含 P1 | **P1 在**——主跑+双跑+交叉三通道全含 P1，issue 文本逐字命中『躲避↔点击驱离』锚点 | **通过** |
|
||||
| ①' 正向守卫 | k-26ce4440/k-a04dfac9 无 P1 有 P2 | 无 P1 达成 2/2（细则①不再过杀）；**有 P2 0/2（两条均 0 findings）** | **字面口径不通过**（按黄金集机器口径 no_p0p1 则 2/2 通过，见 §2.2） |
|
||||
| ② 误报收敛 | 16 条原 accept 误报 ≤2（g9017 计容差单列） | **2/16**（g9011、g9019；g9017 本轮干净，容差未启用；9021 保持干净） | 通过 |
|
||||
| ③ 稳定性 | 5 条双跑一致率 ≥80%（必含 g9019） | **5/5 = 100%**（g9019 两跑同判，稳定） | 通过 |
|
||||
| ④ 交叉 | deepseek-v4-flash 分歧 ≤1 | **2/5**（g9011、g9019——均为 MiniMax 出 P1 而交叉模型干净） | **不通过** |
|
||||
|
||||
**解冻终裁建议：不解冻（①'②③④未全过）。** 不过本轮性质与 v1.1.0 轮红灯根本不同：**v1.1.1 的核心目标全部达成**——真实漏报已根治（kill 守卫双模型三通道确认）、细则①过杀未复发（正向守卫无 P1）、稳定性从 80% 压线升到 100%、g9017 误报自愈。未过的两处都不是「放水」：①' 差在正向守卫 **P2 缺席**（评审员把越模板承诺整个忽略而非降级记录，属轻度漏记不属误杀）；④ 差在 MiniMax 对 g9011/g9019 两条 accept **偏严出 P1**，交叉模型不背书（恰说明是主模型侧噪声而非口径缺陷）。两处均需主 agent 裁决而非继续改 prompt 蛮干（建议见 §8）。
|
||||
|
||||
## 二、断言①' 守卫双向分析
|
||||
|
||||
### 2.1 kill 守卫（k-0cfbc296）——边界句根治验证，证据链闭合
|
||||
|
||||
| 通道 | 结果 | P1 issue 摘录 |
|
||||
|---|---|---|
|
||||
| MiniMax 主跑 | 0/1/0 | 「创意原意为『躲避』鲨鱼，但 config 中描述的是『点击鲨鱼将其驱离』，改变了玩家与鲨鱼的核心交互机制」 |
|
||||
| MiniMax 双跑第 2 次 | 含 P1（与主跑一致） | 同锚点 |
|
||||
| deepseek-v4-flash 交叉 | 0/1/0 | 「创意要求'躲开鲨鱼'，但设计改为点击鲨鱼驱离，核心动作由躲避变为主动攻击，与创意不符」 |
|
||||
|
||||
v1.1.0 轮该条三次全漏（主跑+双跑+交叉仅出 P2）；v1.1.1 边界句加入后三次全中、且**两个模型的 issue 文本几乎同文复述边界句锚点**——「细则①被过度泛化吞核心动作矛盾」的放水缺口已根治，且修复跨模型可迁移。
|
||||
|
||||
### 2.2 正向守卫（k-26ce4440/k-a04dfac9）——无 P1 达成，P2 缺席，双口径分歧
|
||||
|
||||
两条本轮均为 **0 findings**：
|
||||
|
||||
- **无 P1 半边（守卫主旨）2/2 达成**：细则①+边界句叠加后未把越模板机制承诺重新升回 P1——边界句没有引发「过杀复发」，这是裁决②设立正向守卫要防的核心风险，已排除。
|
||||
- **有 P2 半边 0/2 缺席**：细则①原文要求越模板承诺「否则只记 P2」。k-26ce4440 的 designIntent「有限时间内」承诺计时而 config 无计时，按细则①本文应记一条 P2，本轮整个未提，属**轻度漏记**（v1.1.0 轮该条曾出 1×P2）；k-a04dfac9 的「倒计时」是创意原文自带前提（细则②口径下题文不冲突），0 findings 在口径内合理，v1.1.0 轮主跑同样 0 findings——该条的「有 P2」期望本身偏严。
|
||||
- **双口径判定**：按黄金集版本化标签的机器口径（expect=no_p0p1，标签注明「该条再出 P1 即细则①失效」）= **2/2 通过**；按任务令字面「无 P1 有 P2」= **0/2 不通过**。本报告以字面口径记不通过（守卫宁可误响不静默），是否按机器口径放行交主 agent 终裁。
|
||||
|
||||
## 三、断言②误报明细（口径=16 条 batch-001 accept）
|
||||
|
||||
| 条目 | 新 P1 内容 | 评估 |
|
||||
|---|---|---|
|
||||
| g9011（王蓝莓小卖部收银台） | title/theme 未体现「王蓝莓」角色名，判题文脱节 | **新误报**（B1 accept 已实玩通过；title 历史上即无 IP 名）。注意与 w2-1 的 IP 改名 P1 同主题——模型对「角色名缺失」开始敏感；交叉模型不出此 P1 |
|
||||
| g9019（烧烤摊翻烤串） | target=18 与「避免烤糊」脱节，要求翻动与火候状态挂钩 | **误报复发**（上轮双跑第 2 次同模式）：模型反向索要模板外失败机制，正是细则①要压制的模式；本轮两跑稳定出 P1（从随机翻转变为稳定误报）；交叉模型同一问题仅记 P2（定级正确） |
|
||||
| g9017（上轮唯一误报） | 本轮干净（0 findings） | **自愈**——裁决③预留的容差未启用 |
|
||||
| g9021 | 干净 | R8 分歧条目保持收敛 |
|
||||
|
||||
合计 2/16 ≤ 2 通过。W2 段 2 条情况（不计入断言②口径）：**w2-1** 仍出 P1（「蓝莓妹」≠「王蓝莓」，CEO 决策#10 故意去 IP 换皮所致，成因已知，待评审输入侧附「合规改名豁免」上下文解决）；**w2-4 上轮 P1 本轮转干净**——细则①边界句把「仅豁免增强承诺」讲清后，模型对该条不再反向索要「烤糊」失败机制（与 g9019 同主题但结果相反，进一步说明 g9019 是模型侧不稳定而非口径必然）。
|
||||
|
||||
## 四、断言③稳定性（5/5 = 100%）与断言④交叉（2/5）
|
||||
|
||||
### 双跑明细（同 prompt 同模型，run1=A 段，run2=独立重打）
|
||||
|
||||
| 条目 | run1 (P0,P1) | run2 (P0,P1) | 一致 |
|
||||
|---|---|---|---|
|
||||
| g9019 | (False, True) | (False, True) | 是 |
|
||||
| g9021 | (False, False) | (False, False) | 是 |
|
||||
| k-0cfbc296 | (False, True) | (False, True) | 是 |
|
||||
| g9011 | (False, True) | (False, True) | 是 |
|
||||
| w2-6 | (False, False) | (False, False) | 是 |
|
||||
|
||||
v1.1.0 轮 4/5=80% 压线（g9019 翻转）→ 本轮 **5/5=100%**。细则③流程稳定化 + 边界句消歧后，g9019 不再随机翻转——代价是它从「偶发误报」固化为「稳定误报」（计入断言②）。一致性口径与 run_batch 批末稳定性重测一致（(hasP0,hasP1) 元组翻转即不一致）。
|
||||
|
||||
### 交叉明细（deepseek-v4-flash vs MiniMax，结论口径=「无 P0/P1=过」）
|
||||
|
||||
| 条目 | MiniMax 过 | 交叉过 | 交叉 (P0,P1) | 分歧 |
|
||||
|---|---|---|---|---|
|
||||
| g9019 | 不过 | 过 | (False, False)，同一问题记 P2 | **是** |
|
||||
| g9021 | 过 | 过 | (False, False) | 否 |
|
||||
| k-0cfbc296 | 不过 | 不过 | (False, True)，同锚点 P1 | 否 |
|
||||
| g9011 | 不过 | 过 | (False, False)，0 findings | **是** |
|
||||
| w2-6 | 过 | 过 | (False, False) | 否 |
|
||||
|
||||
2 条分歧 >1，断言④字面不通过。但注意分歧**方向一致且与断言②的两条误报完全重合**：均为 MiniMax 偏严出 P1、deepseek 按黄金集期望判干净——交叉模型站在黄金集一边，说明问题钉在 MiniMax-M2.7 对 v1.1.1 文本的偏严漂移（模型侧噪声），而非 prompt 口径本身含糊。kill 守卫两模型同判 P1 则证明口径核心约束跨模型成立。
|
||||
|
||||
## 五、与 v1.1.0 轮对比
|
||||
|
||||
### 断言级对比
|
||||
|
||||
| 维度 | v1.1.0 轮 | v1.1.1 轮 | 变化 |
|
||||
|---|---|---|---|
|
||||
| kill 守卫 k-0cfbc296 | 三通道全漏（红灯主因） | 三通道全中 | **根治** |
|
||||
| 越模板 kill×2（k-26ce4440/k-a04dfac9） | 翻绿记红灯（标签未版本化） | 重标正向守卫：无 P1 达成、P2 缺席 | 裁决②落地，余 P2 缺席待裁 |
|
||||
| 误报（16 accept） | 1/16（g9017） | 2/16（g9011、g9019；g9017 自愈） | 仍 ≤2，但条目漂移 |
|
||||
| 双跑一致率 | 4/5=80% 压线（g9019 翻转） | **5/5=100%** | **+20pp** |
|
||||
| 交叉分歧 | 0/5 | 2/5（两条均=MiniMax 偏严误报） | 恶化（与误报同源） |
|
||||
| 解冻建议 | 不解冻（断言①红灯） | 不解冻（正向守卫 P2 缺席 + 交叉 2 分歧） | 性质不同：放水已堵，余「偏严」侧问题 |
|
||||
|
||||
### 逐条变化（仅列有变化条目）
|
||||
|
||||
| 条目 | v1.1.0 (P0/P1/P2) | v1.1.1 (P0/P1/P2) | 解读 |
|
||||
|---|---|---|---|
|
||||
| k-0cfbc296 | 0/0/1（漏报） | **0/1/0** | 边界句根治目标达成 |
|
||||
| k-26ce4440 | 0/0/1 | 0/0/0 | 无 P1 保持；P2 消失（轻度漏记） |
|
||||
| g9011 | 0/0/0 | **0/1/0** | 新误报（IP 名缺失敏感） |
|
||||
| g9017 | 0/1/0 | 0/0/0 | 上轮误报自愈 |
|
||||
| g9019 | 0/0/0（run2 翻 P1） | **0/1/0**（两跑稳定） | 随机翻转固化为稳定误报 |
|
||||
| w2-3 | 0/0/0 | 0/0/1 | 新增 1 条 P2（不拦截，无影响） |
|
||||
| w2-4 | 0/1/0 | 0/0/0 | 边界句讲清「仅豁免增强承诺」后不再反向索要失败机制 |
|
||||
|
||||
### A 段逐条明细表（25 条全量）
|
||||
|
||||
| 条目 | 来源 | 创意 | 标题 | v1.1.x 期望 | 新 P0/P1/P2 | 无P0/P1=过 | 与期望 |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| g9011 | B1 accept | 王蓝莓的小卖部收银台，给顾客结账 | 小卖部·收银时刻 | 无P0/P1 | 0/1/0 | **不过** | **偏离** |
|
||||
| g9012 | B1 accept | 太空舱里捡星星补给能量 | 星际拾星员 | 无P0/P1 | 0/0/0 | 过 | 符合 |
|
||||
| g9013 | B1 accept | 农场里收割成熟的蔬菜 | 绿野采收忙 | 无P0/P1 | 0/0/0 | 过 | 符合 |
|
||||
| g9014 | B1 accept | 深夜便利店招待来买夜宵的顾客 | 午夜便利店·夜宵速递 | 无P0/P1 | 0/0/0 | 过 | 符合 |
|
||||
| g9015 | B1 accept | 校园运动会上的接力赛冲刺 | 操场冲刺·冲刺荣耀 | 无P0/P1 | 0/0/0 | 过 | 符合 |
|
||||
| g9016 | B1 accept | 猫咖啡馆里撸猫攒爱心 | 猫咖轻抚·爱心满溢 | 无P0/P1 | 0/0/0 | 过 | 符合 |
|
||||
| g9017 | B1 accept | 春节集市上抢福袋 | 迎春纳福·福袋满仓 | 无P0/P1 | 0/0/0 | 过 | 符合 |
|
||||
| k-0cfbc296 | B1 kill | 海底捞珍珠同时躲开鲨鱼 | 深海采珠记 | 含P1(kill守卫) | 0/1/0 | **不过** | 符合 |
|
||||
| g9019 | B1 accept | 烧烤摊翻烤串，别烤糊了 | 炭火翻串记 | 无P0/P1 | 0/1/0 | **不过** | **偏离** |
|
||||
| g9020 | B1 accept | 地铁早高峰挤上即将关门的车 | 站台冲刺·车门将闭 | 无P0/P1 | 0/0/0 | 过 | 符合 |
|
||||
| g9021 | B1 accept | 做一个很好玩的游戏 | 糖果屋的甜蜜收集 | 无P0/P1 | 0/0/0 | 过 | 符合 |
|
||||
| g9022 | B1 accept | 解压点点点 | 弹力球解压工作室 | 无P0/P1 | 0/0/0 | 过 | 符合 |
|
||||
| g9023 | B1 accept | 中秋节帮嫦娥把月饼送上月宫 | 月宫献饼记 | 无P0/P1 | 0/0/0 | 过 | 符合 |
|
||||
| g9024 | B1 accept | 雨夜帮路边馄饨摊老板撑伞收摊 | 雨夜馄饨摊·温情收摊 | 无P0/P1 | 0/0/0 | 过 | 符合 |
|
||||
| k-26ce4440 | B1 kill | 图书馆闭馆前把还回来的书归架 | 闭馆归架记 | P2-pass(正向守卫) | 0/0/0 | 过 | **偏离**（P2 缺席，字面口径） |
|
||||
| g9026 | B1 accept | 奶茶店爆单日疯狂摇珍珠奶茶 | 茶铺高峰·手摇风暴 | 无P0/P1 | 0/0/0 | 过 | 符合 |
|
||||
| k-a04dfac9 | B1 kill | 跨年夜零点在广场上点烟花倒计时 | 午夜庆典·焰火绽放 | P2-pass(正向守卫) | 0/0/0 | 过 | **偏离**（P2 缺席，字面口径） |
|
||||
| g9028 | B1 accept | 给办公室加班的同事分下午茶蛋糕 | 甜蜜分送 | 无P0/P1 | 0/0/0 | 过 | 符合 |
|
||||
| g9030 | B1 accept | 火锅店捞毛肚，七上八下别捞老了 | 沸腾捞肚 | 无P0/P1 | 0/0/0 | 过 | 符合 |
|
||||
| w2-1 | W2 | 王蓝莓的小卖部收银台，给顾客结账 | 蓝莓妹的小卖部·结账高峰 | 无P0/P1 | 0/1/0 | **不过** | **偏离**（IP 合规换皮，成因已知） |
|
||||
| w2-2 | W2 | 深夜便利店招待来买夜宵的顾客 | 凌晨两点的便利店 | 无P0/P1 | 0/0/0 | 过 | 符合 |
|
||||
| w2-3 | W2 | 猫咖啡馆里撸猫攒爱心 | 猫咖时光·撸猫集心 | 无P0/P1 | 0/0/1 | 过 | 符合 |
|
||||
| w2-4 | W2 | 烧烤摊翻烤串，别烤糊了 | 烤串二十翻 | 无P0/P1 | 0/0/0 | 过 | 符合 |
|
||||
| w2-5 | W2 | 地铁早高峰挤上即将关门的车 | 车门即将关闭 | 无P0/P1 | 0/0/0 | 过 | 符合 |
|
||||
| w2-6 | W2 | 解压点点点 | 指尖戳戳乐 | 无P0/P1 | 0/0/0 | 过 | 符合 |
|
||||
|
||||
## 六、通道事故与预算复盘（如实申报）
|
||||
|
||||
- **任务矩阵固定 35 次逻辑调用**（25 A + 5 D + 5 E），全部成功产出合法 findings。
|
||||
- **实际 HTTP 尝试 57 次，超任务预算 ≤45（超额 12 次=57−45）**：相对 35 次逻辑调用的 22 次额外尝试（57−35）全部是 MiniMax 主通道「空 content」形状异常重试（json_parse char 0；leg1 7 次、leg2 10 次、leg3 5 次），其中 g9021/w2-2 两条目各三连空导致两次程崩（断点续跑共 3 程，raw 复用零重复计费）。v1.1.0 轮同通道同模型零重试——本轮空响应**推断**与网关并发负载和/或 v1.1.1 文本加长后推理消耗吃光网关默认 max_tokens 有关（佐证：leg3 起对形状重试启用「显式 max_tokens 抢救通道」——同模型同温度同 json_object 仅显式放宽 max_tokens——5 次抢救 5 次命中；deepseek 通道显式带 max_tokens 全轮零空响应；raw 中 `rescuedByExplicitMaxTokens=true` 共 5 份可审计）。
|
||||
- **处置原则**：不削减断言覆盖面换预算达标（削覆盖=回归失效），完成全矩阵并双口径如实申报；预算闸在脚本内退化为防失控护栏（leg3 实耗 20 < 闸 22）。
|
||||
- **遗留建议**：把「评审类调用显式带 max_tokens」回流到 orchestrator/llm_client 的 eval 配方，根除该浪费类别。
|
||||
|
||||
## 七、Golden 集口径与产物
|
||||
|
||||
- Golden 集与 v1.1.0 轮同一 25 条（16 accept + 3 kill + 6 W2），其中 k-26ce4440/k-a04dfac9 已按裁决②持版本化双标签（`byDoctrine`：=1.0.0→has_p1；>=1.1.0→no_p0p1），由 `golden_regression.resolve_expectation` 按回归对象版本 1.1.1 解析；k-0cfbc296 保持 has_p1 不版本化。
|
||||
- 产物清单（本目录）：`golden_regression.py`（v1.1.1 适配：EXPECT_VERSION=1.1.1/预算闸/空响应抢救通道）/ `golden-set.json` / `raw/`（35 份原始响应：25 a + 5 stability + 5 cross，断点续跑直接复用已落盘份额不重复产生；含 promptVersion=1.1.1、温度、rescued 标记，无密钥）/ `llm-calls.jsonl`（35 条成功调用账，attempts 含形状重试）/ `results.json`（五断言原始数据）/ `summarize.py`（五断言汇总器）/ `run.log` + `run-leg1.log` + `run-leg2.log`（三程完整日志，程崩 traceback 留痕）/ 本报告。
|
||||
- 密钥纪律：NEWAPI_KEY 仅经命令行环境变量注入，未写入任何盘上文件。
|
||||
- 铁律核验：P0 定义与 P1 四类口径文字一字未动（v1.1.1 仅细则①句尾追加边界句 + 黄金集标签版本化，见 prompt frontmatter 变更记录与 eval README 升版记录）。
|
||||
|
||||
## 八、解冻终裁建议与后续动作（供主 agent 裁决）
|
||||
|
||||
**建议：维持冻结**（①'②③④未全过：①' 正向守卫 P2 缺席 0/2、④ 交叉分歧 2/5）。但应注意 v1.1.1 已完成它的使命——放水缺口（断言① kill 类）根治且跨模型成立、稳定性 100%、误报总量守住 ≤2。剩余两个不过项都属「偏严/漏记」侧，不构成新的放水风险。可选裁决路径：
|
||||
|
||||
1. **裁决路径 A（口径裁决，零改动）**：正向守卫按黄金集机器口径（no_p0p1，「再出 P1 即失效」）认定 2/2 通过；g9011/g9019 仿照 g9017 先例定性（g9019 建议「误报·细则①压制不稳」，g9011 建议「存疑偏误报」——交叉模型均不背书 P1）；④ 的 2 条分歧与 ② 的 2 条误报同源，按「同源不双罚」计。如此 ①'②③④ 可判全过 → 解冻。风险：对 MiniMax 偏严漂移不做处置，下一批 fix 率可能微升（误报→fix 轮，不拦发布）。
|
||||
2. **裁决路径 B（v1.1.2 patch，走四道闸）**：细则①再补一句正向义务（如「款外增强承诺存在时应记一条 P2，不得整体忽略」）+ 新增细则④（「题材/角色名未在 title/theme 复现，若机制与场景一致，不单独构成题文不符」）。预计可收敛 g9011 误报并补回正向守卫 P2；成本=再一轮 35 次逻辑调用回归。
|
||||
3. **任一路径下**：把 deepseek-v4-flash 升为常态交叉裁判（本轮它在 kill 与 accept 两侧全部站对）、评审通道显式 max_tokens 进 llm_client eval 配方、w2-1 接「合规改名豁免」上下文——三件低风险工程项可并行落地。
|
||||
@ -0,0 +1,530 @@
|
||||
{
|
||||
"promptVersion": "1.1.1",
|
||||
"mainModel": "MiniMax-M2.7",
|
||||
"crossModel": "deepseek-v4-flash",
|
||||
"temperature": 0.2,
|
||||
"a": [
|
||||
{
|
||||
"entryId": "g9011",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [
|
||||
{
|
||||
"severity": "P1",
|
||||
"issue": "创意原文明确提到“王的蓝莓的小卖部收银台”，强调角色身份“王的蓝莓”，但 config 中 title 仅写“小卖部·收银时刻”、theme 仅写“校园便利店的热闹柜台”，均未体现“王的蓝莓”这一核心角色名称，题文明显脱节",
|
||||
"suggestion": "在 title 或 theme 中补充角色标识，如改为“小卖部·王的蓝莓收银时刻”或在 theme 中加入“王的蓝莓便利店”字样"
|
||||
}
|
||||
],
|
||||
"hasP0": false,
|
||||
"hasP1": true,
|
||||
"passNoP0P1": false,
|
||||
"llmAttempts": 0,
|
||||
"reused": true,
|
||||
"source": "batch-001-accept",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "g9012",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 0,
|
||||
"reused": true,
|
||||
"source": "batch-001-accept",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "g9013",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 0,
|
||||
"reused": true,
|
||||
"source": "batch-001-accept",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "g9014",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 0,
|
||||
"reused": true,
|
||||
"source": "batch-001-accept",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "g9015",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 0,
|
||||
"reused": true,
|
||||
"source": "batch-001-accept",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "g9016",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 0,
|
||||
"reused": true,
|
||||
"source": "batch-001-accept",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "g9017",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 0,
|
||||
"reused": true,
|
||||
"source": "batch-001-accept",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "k-0cfbc296",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [
|
||||
{
|
||||
"severity": "P1",
|
||||
"issue": "玩法核心动作与创意意图明显矛盾：创意原意为『躲避』鲨鱼，但 config 中描述的是『点击鲨鱼将其驱离』，改变了玩家与鲨鱼的核心交互机制",
|
||||
"suggestion": "将鲨鱼的触发机制改为：鲨鱼出现时玩家需点击周围安全区域或避开按钮实现躲避，而非直接点击鲨鱼"
|
||||
}
|
||||
],
|
||||
"hasP0": false,
|
||||
"hasP1": true,
|
||||
"passNoP0P1": false,
|
||||
"llmAttempts": 0,
|
||||
"reused": true,
|
||||
"source": "batch-001-p1residual-kill",
|
||||
"expectation": "has_p1"
|
||||
},
|
||||
{
|
||||
"entryId": "g9019",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [
|
||||
{
|
||||
"severity": "P1",
|
||||
"issue": "target=18 与 designIntent 中“精准把控火候、避免烤糊”脱节——翻动次数应与烤串火候状态挂钩，而非机械计数",
|
||||
"suggestion": "在 config 中补充烤串状态机制说明（如烤串变色进度、烤糊倒计时或每根烤串需翻动的时机窗口），明确 target=18 的含义（18根烤串？18次翻动机会？），使玩家目标与“避免烤糊”的核心意图一致"
|
||||
}
|
||||
],
|
||||
"hasP0": false,
|
||||
"hasP1": true,
|
||||
"passNoP0P1": false,
|
||||
"llmAttempts": 0,
|
||||
"reused": true,
|
||||
"source": "batch-001-accept",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "g9020",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 0,
|
||||
"reused": true,
|
||||
"source": "batch-001-accept",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "g9021",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 0,
|
||||
"reused": true,
|
||||
"source": "batch-001-accept",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "g9022",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 0,
|
||||
"reused": true,
|
||||
"source": "batch-001-accept",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "g9023",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 0,
|
||||
"reused": true,
|
||||
"source": "batch-001-accept",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "g9024",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 0,
|
||||
"reused": true,
|
||||
"source": "batch-001-accept",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "k-26ce4440",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 0,
|
||||
"reused": true,
|
||||
"source": "batch-001-p1residual-kill",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "g9026",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 0,
|
||||
"reused": true,
|
||||
"source": "batch-001-accept",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "k-a04dfac9",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 0,
|
||||
"reused": true,
|
||||
"source": "batch-001-p1residual-kill",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "g9028",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 0,
|
||||
"reused": true,
|
||||
"source": "batch-001-accept",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "g9030",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 0,
|
||||
"reused": true,
|
||||
"source": "batch-001-accept",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "w2-1",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [
|
||||
{
|
||||
"severity": "P1",
|
||||
"issue": "title角色名与创意原文不一致：创意为'王蓝莓'，title写作'蓝莓妹'，存在角色命名冲突",
|
||||
"suggestion": "将title中的'蓝莓妹'改为'王蓝莓'以保持与创意原文的角色一致性"
|
||||
}
|
||||
],
|
||||
"hasP0": false,
|
||||
"hasP1": true,
|
||||
"passNoP0P1": false,
|
||||
"llmAttempts": 0,
|
||||
"reused": true,
|
||||
"source": "w2-pilot",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "w2-2",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 1,
|
||||
"reused": false,
|
||||
"source": "w2-pilot",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "w2-3",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [
|
||||
{
|
||||
"severity": "P2",
|
||||
"issue": "designIntent 提及「通关」概念，但 clicker 模板通常为无尽点击模式，通关机制与模板语义存在偏差。",
|
||||
"suggestion": "建议明确「通关」的含义——若指达到目标爱心数即结束，可改为「达成目标爱心数即完成」；若保留无尽模式则去掉通关表述，聚焦于收集过程。"
|
||||
}
|
||||
],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 2,
|
||||
"reused": false,
|
||||
"source": "w2-pilot",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "w2-4",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 1,
|
||||
"reused": false,
|
||||
"source": "w2-pilot",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "w2-5",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 1,
|
||||
"reused": false,
|
||||
"source": "w2-pilot",
|
||||
"expectation": "no_p0p1"
|
||||
},
|
||||
{
|
||||
"entryId": "w2-6",
|
||||
"runTag": "a",
|
||||
"model": "MiniMax-M2.7",
|
||||
"findings": [],
|
||||
"hasP0": false,
|
||||
"hasP1": false,
|
||||
"passNoP0P1": true,
|
||||
"llmAttempts": 2,
|
||||
"reused": false,
|
||||
"source": "w2-pilot",
|
||||
"expectation": "no_p0p1"
|
||||
}
|
||||
],
|
||||
"d": [
|
||||
{
|
||||
"entryId": "g9019",
|
||||
"run1": [
|
||||
false,
|
||||
true
|
||||
],
|
||||
"run2": [
|
||||
false,
|
||||
true
|
||||
],
|
||||
"consistent": true,
|
||||
"findings2": [
|
||||
{
|
||||
"severity": "P1",
|
||||
"issue": "创意原文中“烧烤摊翻烤串，别烤糊了”的核心是“避免烤糊”，但 config 中的 scoreLabel 为“翻烤次数”、target 为 18 次，设计导向是追求翻烤次数而非避免烤糊失败，创意核心与游戏目标脱节",
|
||||
"suggestion": "调整 scoreLabel 为体现“避免烤糊”的指标（如烤糊次数或烤串状态），或调整 target 为避免烤糊相关的目标（如规定时间内保持不烤糊），使目标与创意核心“避免烤糊”保持一致"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"entryId": "g9021",
|
||||
"run1": [
|
||||
false,
|
||||
false
|
||||
],
|
||||
"run2": [
|
||||
false,
|
||||
false
|
||||
],
|
||||
"consistent": true,
|
||||
"findings2": []
|
||||
},
|
||||
{
|
||||
"entryId": "k-0cfbc296",
|
||||
"run1": [
|
||||
false,
|
||||
true
|
||||
],
|
||||
"run2": [
|
||||
false,
|
||||
true
|
||||
],
|
||||
"consistent": true,
|
||||
"findings2": [
|
||||
{
|
||||
"severity": "P1",
|
||||
"issue": "创意核心动作是『躲避鲨鱼』，但 config 中鲨鱼出现后需点击鲨鱼将其驱离，属于核心动作被改写为『点击消灭/驱离』，与创意『躲避』语义直接矛盾",
|
||||
"suggestion": "将鲨鱼的应对方式调整为非点击主动消灭的形式，例如鲨鱼出现时玩家需快速点击周围安全区域躲避、或在鲨鱼出现前预留躲避反应时间等，保持『躲避鲨鱼』的创意核心动作不变"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"entryId": "g9011",
|
||||
"run1": [
|
||||
false,
|
||||
true
|
||||
],
|
||||
"run2": [
|
||||
false,
|
||||
true
|
||||
],
|
||||
"consistent": true,
|
||||
"findings2": [
|
||||
{
|
||||
"severity": "P1",
|
||||
"issue": "title 缺少创意原文中的品牌名称“王蓝莓”，导致标题与 idea “王蓝莓的小卖部收银台”关联性不足",
|
||||
"suggestion": "将 title 改为包含“王蓝莓”的表述，如“王蓝莓·收银时刻”或直接使用“王蓝莓的小卖部”"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"entryId": "w2-6",
|
||||
"run1": [
|
||||
false,
|
||||
false
|
||||
],
|
||||
"run2": [
|
||||
false,
|
||||
false
|
||||
],
|
||||
"consistent": true,
|
||||
"findings2": []
|
||||
}
|
||||
],
|
||||
"e": [
|
||||
{
|
||||
"entryId": "g9019",
|
||||
"miniMaxPass": false,
|
||||
"crossPass": true,
|
||||
"crossTuple": [
|
||||
false,
|
||||
false
|
||||
],
|
||||
"diverge": true,
|
||||
"findingsCross": [
|
||||
{
|
||||
"severity": "P2",
|
||||
"issue": "创意中的'别烤糊了'在设计中未通过机制体现，仅依赖点击计数，缺乏烤糊警示或失败条件",
|
||||
"suggestion": "考虑在界面或反馈中增加烤糊相关的视觉或文字提示，强化创意核心"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"entryId": "g9021",
|
||||
"miniMaxPass": true,
|
||||
"crossPass": true,
|
||||
"crossTuple": [
|
||||
false,
|
||||
false
|
||||
],
|
||||
"diverge": false,
|
||||
"findingsCross": [
|
||||
{
|
||||
"severity": "P2",
|
||||
"issue": "设计意图描述了温馨治愈的糖果屋场景，但当前clicker模板仅支持简单点击计数，无法呈现场景视觉与情感体验，模板表达力不足。",
|
||||
"suggestion": "建议将设计意图调整为更贴合点击器核心机制（如收集计数）的体验描述，或更改为支持场景与动画的模板。"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"entryId": "k-0cfbc296",
|
||||
"miniMaxPass": false,
|
||||
"crossPass": false,
|
||||
"crossTuple": [
|
||||
false,
|
||||
true
|
||||
],
|
||||
"diverge": false,
|
||||
"findingsCross": [
|
||||
{
|
||||
"severity": "P1",
|
||||
"issue": "创意要求'躲开鲨鱼'，但设计改为点击鲨鱼驱离，核心动作由躲避变为主动攻击，与创意不符。",
|
||||
"suggestion": "将玩法改为躲避鲨鱼，例如鲨鱼出现时点击空白区域或停止点击，若点击鲨鱼则触发惩罚。"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"entryId": "g9011",
|
||||
"miniMaxPass": false,
|
||||
"crossPass": true,
|
||||
"crossTuple": [
|
||||
false,
|
||||
false
|
||||
],
|
||||
"diverge": true,
|
||||
"findingsCross": []
|
||||
},
|
||||
{
|
||||
"entryId": "w2-6",
|
||||
"miniMaxPass": true,
|
||||
"crossPass": true,
|
||||
"crossTuple": [
|
||||
false,
|
||||
false
|
||||
],
|
||||
"diverge": false,
|
||||
"findingsCross": [
|
||||
{
|
||||
"severity": "P2",
|
||||
"issue": "点击交互缺乏解压感的即时反馈，如音效或动画，导致解压爽感不足",
|
||||
"suggestion": "建议在每次点击时增加视觉或音效反馈，如气泡破裂声或情绪值动画，增强解压体验"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"llmCallsUsed": 20
|
||||
}
|
||||
@ -0,0 +1,126 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
回归结果汇总器（golden-regression-v1.1.1 轮）：读 golden-set.json + results.json，
|
||||
产出五断言判定数据与 report.md 用的逐条明细表（markdown 片段落 /tmp/report-tables-v111.md）。
|
||||
只做统计与表格渲染，不发起任何 LLM 调用；断言阈值与本轮任务令一致：
|
||||
|
||||
①' 守卫双向：k-0cfbc296 必须仍含 P1（唯一 kill 守卫）∧ k-26ce4440/k-a04dfac9
|
||||
按 v1.1.x 版本化标签 = 无 P1 且有 P2（细则①生效的正向守卫，任务令字面口径）；
|
||||
② 16 条原 accept 的 P1 误报 ≤2（g9017 按主 agent 裁决③「存疑偏真正报」计入容差不阻断，单列）；
|
||||
③ 双跑一致率 ≥80%（必含 g9019）；
|
||||
④ 交叉分歧 ≤1。
|
||||
解冻终裁建议 = ①'②③④全过。
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
|
||||
_HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
gs = json.load(open(os.path.join(_HERE, "golden-set.json"), encoding="utf-8"))
|
||||
rs = json.load(open(os.path.join(_HERE, "results.json"), encoding="utf-8"))
|
||||
by_id = {e["entryId"]: e for e in gs}
|
||||
a_map = {r["entryId"]: r for r in rs["a"]}
|
||||
|
||||
out = []
|
||||
|
||||
|
||||
def sev_count(findings):
|
||||
"""统计 findings 三级数量，返回 dict{P0,P1,P2}。"""
|
||||
sev = {"P0": 0, "P1": 0, "P2": 0}
|
||||
for f in findings:
|
||||
sev[f["severity"]] += 1
|
||||
return sev
|
||||
|
||||
|
||||
# ---- 断言①'：守卫双向（kill 守卫不放水 + 细则①正向守卫） ----
|
||||
# expectation 读 results.json 中 golden_regression.resolve_expectation 已按 doctrine(=1.1.1)
|
||||
# 解析好的标量：k-0cfbc296=has_p1（唯一 kill 守卫），k-26ce4440/k-a04dfac9=no_p0p1（正向守卫）。
|
||||
kills = [r for r in rs["a"] if r["expectation"] == "has_p1"]
|
||||
kill_ok = [r for r in kills if r["hasP1"]]
|
||||
guards = [r for r in rs["a"] if r["source"] == "batch-001-p1residual-kill" and r["expectation"] == "no_p0p1"]
|
||||
# 任务令字面口径「无 P1 有 P2」：正向守卫须 无P0/P1 且 至少 1 条 P2（仅降级、未失明）
|
||||
guard_ok = [r for r in guards if r["passNoP0P1"] and sev_count(r["findings"])["P2"] >= 1]
|
||||
b_pass = len(kill_ok) == len(kills) == 1 and len(guard_ok) == len(guards) == 2
|
||||
kill_detail = ", ".join("%s hasP1=%s" % (r["entryId"], r["hasP1"]) for r in kills)
|
||||
guard_detail = ", ".join("%s 无P0/P1=%s P2数=%d" % (r["entryId"], r["passNoP0P1"],
|
||||
sev_count(r["findings"])["P2"]) for r in guards)
|
||||
out.append(("①'", "kill 守卫 %d/%d 仍含P1（%s）；正向守卫 无P1有P2 %d/%d（%s）"
|
||||
% (len(kill_ok), len(kills), kill_detail, len(guard_ok), len(guards), guard_detail), b_pass))
|
||||
|
||||
# ---- 断言②：16 条原 accept 的 P1 误报 ≤2（g9017 计容差单列，不单独阻断） ----
|
||||
accepts = [r for r in rs["a"] if r["source"] == "batch-001-accept"]
|
||||
fp = [r for r in accepts if r["hasP0"] or r["hasP1"]]
|
||||
fp_ids = [r["entryId"] for r in fp]
|
||||
g9017_fp = "g9017" in fp_ids
|
||||
g9019_clean = not (a_map["g9019"]["hasP0"] or a_map["g9019"]["hasP1"])
|
||||
g9021_clean = not (a_map["g9021"]["hasP0"] or a_map["g9021"]["hasP1"])
|
||||
c_pass = len(fp) <= 2
|
||||
out.append(("②", "%d/16 误报%s（g9017 %s·裁决③容差单列；9019 %s，9021 %s）"
|
||||
% (len(fp), ("=" + ",".join(fp_ids)) if fp_ids else "",
|
||||
"出P1" if g9017_fp else "无P1",
|
||||
"干净" if g9019_clean else "出P1", "干净" if g9021_clean else "出P1"), c_pass))
|
||||
|
||||
# ---- 断言③：双跑一致率 ≥80%（必含 g9019——抽样写死于 golden_regression.STABILITY_SAMPLE） ----
|
||||
assert any(d["entryId"] == "g9019" for d in rs["d"]), "断言③抽样必须包含 g9019"
|
||||
cons = sum(1 for d in rs["d"] if d["consistent"])
|
||||
d_rate = cons / float(len(rs["d"])) if rs["d"] else 0.0
|
||||
d_pass = d_rate >= 0.8
|
||||
g9019_d = [d for d in rs["d"] if d["entryId"] == "g9019"][0]
|
||||
out.append(("③", "%d/%d=%.0f%%（g9019 %s）" % (cons, len(rs["d"]), d_rate * 100,
|
||||
"稳定" if g9019_d["consistent"] else "翻转"), d_pass))
|
||||
|
||||
# ---- 断言④：交叉分歧 ≤1 ----
|
||||
div = sum(1 for e in rs["e"] if e["diverge"])
|
||||
e_pass = div <= 1
|
||||
out.append(("④", "分歧 %d/%d" % (div, len(rs["e"])), e_pass))
|
||||
|
||||
# ---- A 段：25/25 有效产出 + 通过分布 ----
|
||||
a_ok = len(rs["a"]) == 25
|
||||
n_pass = sum(1 for r in rs["a"] if r["passNoP0P1"])
|
||||
print("A: 25 条有效产出=%s，按「无 P0/P1=过」: 过 %d / 不过 %d" % (a_ok, n_pass, 25 - n_pass))
|
||||
for tag, detail, ok in out:
|
||||
print("断言%s: %s → %s" % (tag, detail, "通过" if ok else "不通过"))
|
||||
print("LLM 调用（HTTP 尝试口径）:", rs["llmCallsUsed"])
|
||||
print("解冻终裁建议（①'②③④全过）:", all(x[2] for x in out))
|
||||
|
||||
# ---- 明细表 markdown ----
|
||||
lines = ["| 条目 | 来源 | 创意 | 标题 | v1.1.x 期望 | 新 P0/P1/P2 | 无P0/P1=过 | 与期望 |",
|
||||
"|---|---|---|---|---|---|---|---|"]
|
||||
for e in gs:
|
||||
r = a_map[e["entryId"]]
|
||||
sev = sev_count(r["findings"])
|
||||
# 期望列：读 results.json 已按 doctrine=1.1.1 解析的标量（版本化双标签解析逻辑在 golden_regression）
|
||||
expect_label = {"no_p0p1": "无P0/P1", "has_p1": "含P1(kill守卫)"}[r["expectation"]]
|
||||
if e["entryId"] in ("k-26ce4440", "k-a04dfac9"):
|
||||
expect_label = "P2-pass(正向守卫)"
|
||||
expect_ok = r["passNoP0P1"] and sev["P2"] >= 1 # ①'字面口径：无P1有P2
|
||||
elif r["expectation"] == "has_p1":
|
||||
expect_ok = r["hasP1"]
|
||||
else:
|
||||
expect_ok = r["passNoP0P1"]
|
||||
lines.append("| %s | %s | %s | %s | %s | %d/%d/%d | %s | %s |" % (
|
||||
e["entryId"], {"batch-001-accept": "B1 accept", "batch-001-p1residual-kill": "B1 kill",
|
||||
"w2-pilot": "W2"}[e["source"]],
|
||||
e["idea"], e["design"]["config"]["title"], expect_label,
|
||||
sev["P0"], sev["P1"], sev["P2"],
|
||||
"过" if r["passNoP0P1"] else "**不过**", "符合" if expect_ok else "**偏离**"))
|
||||
|
||||
lines.append("")
|
||||
lines.append("### 断言③双跑明细")
|
||||
lines.append("| 条目 | run1 (P0,P1) | run2 (P0,P1) | 一致 |")
|
||||
lines.append("|---|---|---|---|")
|
||||
for d in rs["d"]:
|
||||
lines.append("| %s | %s | %s | %s |" % (d["entryId"], tuple(d["run1"]), tuple(d["run2"]),
|
||||
"是" if d["consistent"] else "**否**"))
|
||||
lines.append("")
|
||||
lines.append("### 断言④交叉明细（deepseek-v4-flash vs MiniMax）")
|
||||
lines.append("| 条目 | MiniMax 过 | 交叉过 | 交叉 (P0,P1) | 分歧 |")
|
||||
lines.append("|---|---|---|---|---|")
|
||||
for e in rs["e"]:
|
||||
lines.append("| %s | %s | %s | %s | %s |" % (e["entryId"], "过" if e["miniMaxPass"] else "不过",
|
||||
"过" if e["crossPass"] else "不过",
|
||||
tuple(e["crossTuple"]),
|
||||
"**是**" if e["diverge"] else "否"))
|
||||
open("/tmp/report-tables-v111.md", "w", encoding="utf-8").write("\n".join(lines))
|
||||
print("\n明细表已写 /tmp/report-tables-v111.md（%d 行）" % len(lines))
|
||||
@ -44,7 +44,7 @@
|
||||
- [x] **C3 agent 化生成 QA 闭环 · 设计+试点**（✅ 2026-06-09，替代 A4）：W1 设计稿（三案×三评委 + CEO/Eng 双终审修订 Z1-Z5）+ 创始人四项拍板（机器口径 / staging 全自动 APPROVE / 先 M-b / M2 两段式）；W2 clicker 试点 6/6 入池（对抗 pass5/fix1/kill0；MiniMax 实时 0/6 degraded=分类器故障非通道故障，已诚实声明）。产物：`agent化生成QA闭环-review.md` + `generation-spike/{pilot-results.md,accepted-configs.json}`。
|
||||
- [x] **C4 agent 闭环 v1 建设波（M-a）** ✅ **2026-06-10 验收过线**：5 件交付全落（9bf3d54+208a36e+9517f4e）→ **batch-001：20 创意全自动 671s 流完，accept 率 80.0%（16/20，零 infra），双锚复核三锚全立**（9011 真玩→遥测→quality 0→60→feed 第3→第1）。**M2 ①口径达标首次实测达成**（两段式之①；M2 收口仍待 ②M-b）。批中逮修 2 个 P1（postcheck size=50 误熔断→cursor 翻页修复；网关二厂系全废→R8 同模型降级抽检，**分歧 2/4 触发冻结告警→下一批发布开关=冻结，解冻=对抗 prompt v1.0.1+Golden 回归**）。产物：`agent-loop-v1/runs/batch-001/`（报告/账本/补救/抽检全入库 8cf5918）。
|
||||
- [x] **C5 M-b 生成下沉** ✅ **2026-06-10 M2 ②段达成 → M2 整体收口**：执行器上线（054564d+修复 42d3237），部署 #3.1 实证 5 独立样本全自动达终态（HTTP 花店 18s/UI 奶茶店 27s succeeded、dodge 10s 正确失败、watchdog 收尸、updater='0' 零异常）；冒烟逮修一个 staging 才可见的真 bug（调度线程无登录态×MP fill 必进 SET×updater NOT NULL 三因叠加→系统身份注入修法）。**M-b② 渲染面同窗验收**（theme 渐变+标题入画+scoreLabel 文案，双样本色相互证）。`done ✓`
|
||||
- [ ] **C6 对抗 prompt v1.0.1 升版**（解批跑冻结阀）：补「designIntent 禁承诺模板外机制」+「模糊创意题文判定规则」+稳定性提升（批-001 一致率仅 33.3%），走四道闸+Golden 回归。`done=冻结解除，下一批可发布`
|
||||
- [x] **C6 prompt 升版（v1.1.0→v1.1.1）+ Golden 回归** ✅ **2026-06-10 解冻（主 agent 终裁 Path A）**：两轮回归完成质量体系首次自我纠错闭环——v1.1.0 修误报但被回归逮到放水（kill 守卫 0/3 红灯，诚实不解冻）→ v1.1.1 边界句根治（kill 守卫**三通道全中**：MiniMax 主跑/双跑/deepseek 交叉）、稳定性 33.3%→80%→**100%**、误报 2/16 守住。残差全在**偏严侧**（正向守卫 P2 漏记/交叉分歧=主模型偏严方向）零放水证据→解冻。黄金集口径版本化（doctrine 双标签，v1.0.0 历史四重可溯）。backlog：v1.1.2 候选（g9011/g9019 稳定偏严误报）；编排器 llm_client 补显式 max_tokens（22 次空 content 重试=推理输出吃光网关缺省，leg3 抢救 5/5 实证修法）。`done ✓ 下一批可发布`
|
||||
|
||||
---
|
||||
|
||||
|
||||
@ -48,8 +48,11 @@ class PromptResourceLoaderTest extends BaseMockitoUnitTest {
|
||||
|
||||
@Test
|
||||
void testVersionExtracted() {
|
||||
// frontmatter 顶层 version（与 registry.yaml 注册值 1.0.0 一致；对账校验归 CI 四道闸，Java 只提取留日志）
|
||||
assertEquals("1.0.0", LOADER.getPromptVersion());
|
||||
// frontmatter 顶层 version 能被正确提取且为合法语义化版本号——不钉死具体值：
|
||||
// prompt 升版走四道闸（C6 实例：1.0.0→1.1.0），钉死字面会让每次升版必挂本测试（C6 合树实证教训）；
|
||||
// 版本与 registry.yaml 的一致性对账归 CI 四道闸与编排器装载门，Java 侧只验提取逻辑。
|
||||
assertTrue(LOADER.getPromptVersion().matches("\\d+\\.\\d+\\.\\d+"),
|
||||
"version 应为语义化版本号，实际: " + LOADER.getPromptVersion());
|
||||
}
|
||||
|
||||
// ============================== 用例3：真文件渲染（变量替换 + schema 同源注入） ==============================
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user