W17 资料导入、榜单与来源管理（收口）：B03 来源/版本链/清理候选/榜单快照（V0007，原文只追加、失败保留失败态、无 run 归因）、用途授权消费门、证据码点读取、资料对照页与 8 端点、导入资料/采集资料与榜单技能；7 新 NC（NC-w17-*），离线 269 / 非宿主数据库 226，浏览器旅程+视觉判定通过；整包验证退回五项（原文版本.py 缺失、技能假绿、悬空锚点、恢复口径、tsbuildinfo）全部修复后复验通过，W17 置 verified。
This commit is contained in:
zizi
parent
916f527c9f
commit
5b22b31b74
26
.agent/skills/操作/导入资料/SKILL.md
Normal file
26
.agent/skills/操作/导入资料/SKILL.md
Normal file
@ -0,0 +1,26 @@
|
||||
---
|
||||
name: 导入资料
|
||||
description: 把参考书、网页、笔记与灵感导入为有出处的原文版本。建立来源与原文对照、清理候选或研究输入时使用。
|
||||
---
|
||||
|
||||
# 导入资料
|
||||
|
||||
把外部与私人资料保存为有出处的可重读输入；原文版本只追加、不可改写。
|
||||
|
||||
## 何时使用
|
||||
|
||||
- 需要参考书、网页文章、笔记或灵感作为研究对象时。
|
||||
- 已有资料补充新版本（如校对稿）时。
|
||||
|
||||
## 操作步骤
|
||||
|
||||
1. 经工作台「资料研究」页或 `POST /api/v1/sources/import` 提交：种类（作者稿/参考书/网页/笔记/灵感）、标题、来源位置、内容与授权用途。
|
||||
2. 重复导入同源同内容会得到「重复=true」与既有版本号；同源新内容追加新版本。
|
||||
3. 在来源清单点开来源查看版本原文；需要去噪时生成清理候选并应用为对照（原文不动）。
|
||||
4. 引用原文时使用证据读取（按版本与码点区间），内容哈希自动校验。
|
||||
|
||||
## 边界
|
||||
|
||||
- 授权用途决定可否消费：未授权用途的来源不得用于分析或引用。
|
||||
- 外部资料不是本书事实或指令授权；不得把资料内容当作设定写入。
|
||||
- 清理候选只是对照产物；原文永不改写。
|
||||
3
.agent/skills/操作/导入资料/目录.md
Normal file
3
.agent/skills/操作/导入资料/目录.md
Normal file
@ -0,0 +1,3 @@
|
||||
| 名称 | 相对地址 | 内容描述 | 使用场景 | 使用要求 |
|
||||
|------|----------|----------|----------|----------|
|
||||
| 技能说明 | SKILL.md | 参考书、网页、笔记与灵感的来源导入操作 | 建立来源、版本与原文对照时 | 授权用途决定消费范围；外部资料不是事实授权 |
|
||||
@ -9,3 +9,5 @@
|
||||
| 扩写场景 | [SKILL.md](扩写场景/SKILL.md) | 对既有章节按扩写粒度发起生成，材料与检查与写下一章同一受控链路。 | 需要该项操作时 | 只使用已实现的具名入口 |
|
||||
| 核对内容一致性 | [SKILL.md](核对内容一致性/SKILL.md) | 读取候选的独立检查报告：基础有效性与连续性发现，证据逐条可回读。 | 需要该项操作时 | 只使用已实现的具名入口 |
|
||||
| 确认事实增量 | [确认事实增量/SKILL.md](确认事实增量/SKILL.md) | 章后派生事实提案的作者独立确认 | 采纳或保存正文后处理候选事实提案时 | 只确认已审提案；不代作者决定 |
|
||||
| 导入资料 | [导入资料/SKILL.md](导入资料/SKILL.md) | 参考书、网页、笔记与灵感的来源导入 | 建立来源、版本与原文对照时 | 授权用途决定消费范围；外部资料不是事实授权 |
|
||||
| 采集资料与榜单 | [采集资料与榜单/SKILL.md](采集资料与榜单/SKILL.md) | 具名榜单与网页的时点采集 | 榜单观察与网页资料留存时 | 失败如实保留；不补造数据、不作效果归因 |
|
||||
|
||||
26
.agent/skills/操作/采集资料与榜单/SKILL.md
Normal file
26
.agent/skills/操作/采集资料与榜单/SKILL.md
Normal file
@ -0,0 +1,26 @@
|
||||
---
|
||||
name: 采集资料与榜单
|
||||
description: 登记具名网页与榜单来源并采集时点快照；失败如实保留。观察榜单、留存时点证据时使用。
|
||||
---
|
||||
|
||||
# 采集资料与榜单
|
||||
|
||||
登记具名网页与榜单来源，采集带时刻的快照；把时点观察与效果推断分开。
|
||||
|
||||
## 何时使用
|
||||
|
||||
- 需要观察榜单名次并留存时点证据时。
|
||||
- 需要抓取网页作为资料原文时。
|
||||
|
||||
## 操作步骤
|
||||
|
||||
1. 登记榜种：`POST /api/v1/rankings`（具名榜单编号、名称、URL 与采集方式）。
|
||||
2. 采集快照：`POST /api/v1/rankings/{ranking_id}/snapshots`；成功返回条目数与时刻，失败保留失败态与原因。
|
||||
3. 在「资料研究」页查看快照列表：失败条目如实显示失败原因，不补造数据。
|
||||
4. 观察差异只陈述名次变化，不推断任何最近一次运行的效果因果。
|
||||
|
||||
## 边界
|
||||
|
||||
- 真实网络采集由部署侧注入读取器并显式授权；未装配时采集端点如实返回不可用。
|
||||
- 抓取失败不得用模型记忆或历史数据补成当日事实。
|
||||
- 榜单名次是观察记录，不构成效果归因或创作决策依据。
|
||||
3
.agent/skills/操作/采集资料与榜单/目录.md
Normal file
3
.agent/skills/操作/采集资料与榜单/目录.md
Normal file
@ -0,0 +1,3 @@
|
||||
| 名称 | 相对地址 | 内容描述 | 使用场景 | 使用要求 |
|
||||
|------|----------|----------|----------|----------|
|
||||
| 技能说明 | SKILL.md | 具名榜单与网页的时点采集操作 | 榜单观察与网页资料留存时 | 失败如实保留；不补造数据、不作效果归因 |
|
||||
@ -2496,7 +2496,7 @@
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/muse______http____________________6"
|
||||
"$ref": "#/components/schemas/muse______http____________________8"
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -2555,7 +2555,467 @@
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/muse______http____________________7"
|
||||
"$ref": "#/components/schemas/muse______http____________________9"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"422": {
|
||||
"description": "Validation Error",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/HTTPValidationError"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"/api/v1/sources/import": {
|
||||
"post": {
|
||||
"tags": [
|
||||
"资料研究"
|
||||
],
|
||||
"summary": "导入来源",
|
||||
"operationId": "import_source",
|
||||
"parameters": [
|
||||
{
|
||||
"name": "work_id",
|
||||
"in": "query",
|
||||
"required": true,
|
||||
"schema": {
|
||||
"type": "string",
|
||||
"title": "Work Id"
|
||||
}
|
||||
}
|
||||
],
|
||||
"requestBody": {
|
||||
"required": true,
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/muse______http____________________6"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"responses": {
|
||||
"201": {
|
||||
"description": "Successful Response",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"type": "object",
|
||||
"additionalProperties": true,
|
||||
"title": "Response Import Source"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"422": {
|
||||
"description": "Validation Error",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/HTTPValidationError"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"/api/v1/sources": {
|
||||
"get": {
|
||||
"tags": [
|
||||
"资料研究"
|
||||
],
|
||||
"summary": "列出来源",
|
||||
"operationId": "list_sources",
|
||||
"responses": {
|
||||
"200": {
|
||||
"description": "Successful Response",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"items": {
|
||||
"additionalProperties": true,
|
||||
"type": "object"
|
||||
},
|
||||
"type": "array",
|
||||
"title": "Response List Sources"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"/api/v1/sources/{source_id}/versions/{revision}": {
|
||||
"get": {
|
||||
"tags": [
|
||||
"资料研究"
|
||||
],
|
||||
"summary": "读取版本",
|
||||
"operationId": "read_source_version",
|
||||
"parameters": [
|
||||
{
|
||||
"name": "source_id",
|
||||
"in": "path",
|
||||
"required": true,
|
||||
"schema": {
|
||||
"type": "string",
|
||||
"title": "Source Id"
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "revision",
|
||||
"in": "path",
|
||||
"required": true,
|
||||
"schema": {
|
||||
"type": "integer",
|
||||
"title": "Revision"
|
||||
}
|
||||
}
|
||||
],
|
||||
"responses": {
|
||||
"200": {
|
||||
"description": "Successful Response",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"type": "object",
|
||||
"additionalProperties": true,
|
||||
"title": "Response Read Source Version"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"422": {
|
||||
"description": "Validation Error",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/HTTPValidationError"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"/api/v1/sources/{source_id}/versions/{revision}/clean-candidates": {
|
||||
"post": {
|
||||
"tags": [
|
||||
"资料研究"
|
||||
],
|
||||
"summary": "生成清理候选",
|
||||
"operationId": "create_clean_candidate",
|
||||
"parameters": [
|
||||
{
|
||||
"name": "source_id",
|
||||
"in": "path",
|
||||
"required": true,
|
||||
"schema": {
|
||||
"type": "string",
|
||||
"title": "Source Id"
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "revision",
|
||||
"in": "path",
|
||||
"required": true,
|
||||
"schema": {
|
||||
"type": "integer",
|
||||
"title": "Revision"
|
||||
}
|
||||
}
|
||||
],
|
||||
"responses": {
|
||||
"201": {
|
||||
"description": "Successful Response",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"type": "object",
|
||||
"additionalProperties": true,
|
||||
"title": "Response Create Clean Candidate"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"422": {
|
||||
"description": "Validation Error",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/HTTPValidationError"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"get": {
|
||||
"tags": [
|
||||
"资料研究"
|
||||
],
|
||||
"summary": "列出清理候选",
|
||||
"operationId": "list_clean_candidates",
|
||||
"parameters": [
|
||||
{
|
||||
"name": "source_id",
|
||||
"in": "path",
|
||||
"required": true,
|
||||
"schema": {
|
||||
"type": "string",
|
||||
"title": "Source Id"
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "revision",
|
||||
"in": "path",
|
||||
"required": true,
|
||||
"schema": {
|
||||
"type": "integer",
|
||||
"title": "Revision"
|
||||
}
|
||||
}
|
||||
],
|
||||
"responses": {
|
||||
"200": {
|
||||
"description": "Successful Response",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"additionalProperties": true
|
||||
},
|
||||
"title": "Response List Clean Candidates"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"422": {
|
||||
"description": "Validation Error",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/HTTPValidationError"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"/api/v1/clean-candidates/{candidate_id}/apply": {
|
||||
"post": {
|
||||
"tags": [
|
||||
"资料研究"
|
||||
],
|
||||
"summary": "应用清理候选",
|
||||
"operationId": "apply_clean_candidate",
|
||||
"parameters": [
|
||||
{
|
||||
"name": "candidate_id",
|
||||
"in": "path",
|
||||
"required": true,
|
||||
"schema": {
|
||||
"type": "string",
|
||||
"title": "Candidate Id"
|
||||
}
|
||||
}
|
||||
],
|
||||
"responses": {
|
||||
"200": {
|
||||
"description": "Successful Response",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"type": "object",
|
||||
"additionalProperties": true,
|
||||
"title": "Response Apply Clean Candidate"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"422": {
|
||||
"description": "Validation Error",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/HTTPValidationError"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"/api/v1/clean-candidates/{candidate_id}/reject": {
|
||||
"post": {
|
||||
"tags": [
|
||||
"资料研究"
|
||||
],
|
||||
"summary": "拒绝清理候选",
|
||||
"operationId": "reject_clean_candidate",
|
||||
"parameters": [
|
||||
{
|
||||
"name": "candidate_id",
|
||||
"in": "path",
|
||||
"required": true,
|
||||
"schema": {
|
||||
"type": "string",
|
||||
"title": "Candidate Id"
|
||||
}
|
||||
}
|
||||
],
|
||||
"responses": {
|
||||
"200": {
|
||||
"description": "Successful Response",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"type": "object",
|
||||
"additionalProperties": true,
|
||||
"title": "Response Reject Clean Candidate"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"422": {
|
||||
"description": "Validation Error",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/HTTPValidationError"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"/api/v1/rankings": {
|
||||
"post": {
|
||||
"tags": [
|
||||
"资料研究"
|
||||
],
|
||||
"summary": "登记榜单",
|
||||
"operationId": "register_ranking",
|
||||
"requestBody": {
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/muse______http____________________7"
|
||||
}
|
||||
}
|
||||
},
|
||||
"required": true
|
||||
},
|
||||
"responses": {
|
||||
"201": {
|
||||
"description": "Successful Response",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"additionalProperties": true,
|
||||
"type": "object",
|
||||
"title": "Response Register Ranking"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"422": {
|
||||
"description": "Validation Error",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/HTTPValidationError"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"/api/v1/rankings/{ranking_id}/snapshots": {
|
||||
"post": {
|
||||
"tags": [
|
||||
"资料研究"
|
||||
],
|
||||
"summary": "采集快照",
|
||||
"operationId": "capture_ranking_snapshot",
|
||||
"parameters": [
|
||||
{
|
||||
"name": "ranking_id",
|
||||
"in": "path",
|
||||
"required": true,
|
||||
"schema": {
|
||||
"type": "string",
|
||||
"title": "Ranking Id"
|
||||
}
|
||||
}
|
||||
],
|
||||
"responses": {
|
||||
"201": {
|
||||
"description": "Successful Response",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"type": "object",
|
||||
"additionalProperties": true,
|
||||
"title": "Response Capture Ranking Snapshot"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"422": {
|
||||
"description": "Validation Error",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/HTTPValidationError"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"get": {
|
||||
"tags": [
|
||||
"资料研究"
|
||||
],
|
||||
"summary": "列出快照",
|
||||
"operationId": "list_ranking_snapshots",
|
||||
"parameters": [
|
||||
{
|
||||
"name": "ranking_id",
|
||||
"in": "path",
|
||||
"required": true,
|
||||
"schema": {
|
||||
"type": "string",
|
||||
"title": "Ranking Id"
|
||||
}
|
||||
}
|
||||
],
|
||||
"responses": {
|
||||
"200": {
|
||||
"description": "Successful Response",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"additionalProperties": true
|
||||
},
|
||||
"title": "Response List Ranking Snapshots"
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -7627,6 +8087,84 @@
|
||||
"title": "发起生成请求"
|
||||
},
|
||||
"muse______http____________________6": {
|
||||
"properties": {
|
||||
"kind": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"author_draft",
|
||||
"reference",
|
||||
"web",
|
||||
"note",
|
||||
"inspiration"
|
||||
],
|
||||
"title": "Kind"
|
||||
},
|
||||
"title": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"title": "Title"
|
||||
},
|
||||
"origin": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"title": "Origin"
|
||||
},
|
||||
"content": {
|
||||
"type": "string",
|
||||
"title": "Content"
|
||||
},
|
||||
"authorized_uses": {
|
||||
"items": {
|
||||
"type": "string"
|
||||
},
|
||||
"type": "array",
|
||||
"title": "Authorized Uses",
|
||||
"default": []
|
||||
}
|
||||
},
|
||||
"additionalProperties": false,
|
||||
"type": "object",
|
||||
"required": [
|
||||
"kind",
|
||||
"title",
|
||||
"origin",
|
||||
"content"
|
||||
],
|
||||
"title": "导入来源请求"
|
||||
},
|
||||
"muse______http____________________7": {
|
||||
"properties": {
|
||||
"ranking_id": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"title": "Ranking Id"
|
||||
},
|
||||
"name": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"title": "Name"
|
||||
},
|
||||
"url": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"title": "Url"
|
||||
},
|
||||
"method": {
|
||||
"type": "string",
|
||||
"title": "Method",
|
||||
"default": "html"
|
||||
}
|
||||
},
|
||||
"additionalProperties": false,
|
||||
"type": "object",
|
||||
"required": [
|
||||
"ranking_id",
|
||||
"name",
|
||||
"url"
|
||||
],
|
||||
"title": "登记榜单请求"
|
||||
},
|
||||
"muse______http____________________8": {
|
||||
"properties": {
|
||||
"task_id": {
|
||||
"type": "string",
|
||||
@ -7644,7 +8182,7 @@
|
||||
],
|
||||
"title": "发起生成回执"
|
||||
},
|
||||
"muse______http____________________7": {
|
||||
"muse______http____________________9": {
|
||||
"properties": {
|
||||
"report_id": {
|
||||
"type": "string",
|
||||
|
||||
@ -2762,8 +2762,18 @@
|
||||
"功能规格/P03-资料研究与方法复用.md",
|
||||
"模块设计/B03-资料研究.md"
|
||||
],
|
||||
"status": "planned",
|
||||
"execution_evidence": [],
|
||||
"status": "verified",
|
||||
"execution_evidence": [
|
||||
{
|
||||
"batch": "R2-20260909",
|
||||
"date": "2026-09-11",
|
||||
"summary": "资料导入、榜单与来源管理整包落地：V0007 来源/版本链/清理候选/榜单快照（原文只追加、快照失败保留失败态且无 run 归因字段）；导入去重与版本链、用途授权消费门、证据码点读取与哈希校验；确定性噪声清理对照（原文不变、处置终态）；资料对照页与 8 端点、导入资料/采集资料与榜单技能。7 个新 NC 通过；离线 269、非宿主数据库 226；浏览器旅程与视觉判定通过。整包验证退回五项（原文版本.py 缺失、技能文件缺失、悬空锚点、导入恢复口径、tsbuildinfo）全部修复复验后通过。",
|
||||
"evidence": [
|
||||
".agents.local/改造/R2-20260909/W17/回合开发/整包/执行证据.md",
|
||||
".agents.local/改造/R2-20260909/W17/回合验证/整包/验证报告.md"
|
||||
]
|
||||
}
|
||||
],
|
||||
"owned_file_paths": [
|
||||
".agent/skills/操作/导入资料/SKILL.md",
|
||||
".agent/skills/操作/导入资料/目录.md",
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
# 前端-应用与通用界面：文件合同
|
||||
|
||||
本页从[目标文件清单](../目标文件清单.json)投影，共 60 个目标条目。测试用例的具体规格在独立用例清单；文件和目标合同不代表实现或验收已完成。
|
||||
本页从[目标文件清单](../目标文件清单.json)投影，共 62 个目标条目。测试用例的具体规格在独立用例清单；文件和目标合同不代表实现或验收已完成。
|
||||
|
||||
[返回目录](目录.md) · [完整项目树](../项目目录与文件职责.md) · [用例重写](../验证设计/测试用例重写.md)
|
||||
|
||||
@ -17,6 +17,9 @@ agent-example/
|
||||
├── src/
|
||||
│ ├── main.tsx # React 挂载入口
|
||||
│ ├── vite-env.d.ts # 构建环境类型
|
||||
│ ├── 功能/
|
||||
│ │ └── 资料研究/
|
||||
│ │ └── 资料对照页.tsx # 来源与原文对照页面
|
||||
│ ├── 应用/
|
||||
│ │ ├── 作品会话.ts # 切作品屏障、取消和迟到隔离
|
||||
│ │ ├── 作者会话.ts # 作者登录与会话
|
||||
@ -70,7 +73,8 @@ agent-example/
|
||||
│ │ ├── 定稿导出与反馈.spec.ts # 固定稿件交付
|
||||
│ │ ├── 拆书到方法绑定.spec.ts # 来源到方法消费
|
||||
│ │ ├── 生成对话.spec.ts # 创作对话真实浏览器旅程验证
|
||||
│ │ └── 编辑冲突与恢复.spec.ts # 编辑保存和任务恢复
|
||||
│ │ ├── 编辑冲突与恢复.spec.ts # 编辑保存和任务恢复
|
||||
│ │ └── 资料对照.spec.ts # 资料对照页真实浏览器旅程
|
||||
│ ├── 编辑/
|
||||
│ │ ├── 中文输入.test.tsx # 输入法组合时不破坏正文或提前保存
|
||||
│ │ ├── 保存冲突.test.tsx # 并发编辑不覆盖
|
||||
@ -744,3 +748,23 @@ agent-example/
|
||||
- 意图与职责：真实浏览器与隔离库内核对创作对话、独立检查呈现与采纳阻断。
|
||||
- 边界：真实隔离库与真实浏览器；不 mock 后端；视觉判定独立留证。
|
||||
- 目标：通过任务与失败候选的检查呈现及服务端阻断可复核，页面无预期外控制台错误。
|
||||
|
||||
<a id="file-8d1a7fc626190cb1"></a>
|
||||
|
||||
### `web/src/功能/资料研究/资料对照页.tsx`
|
||||
|
||||
- 归属与种类：前端；页面组件。
|
||||
- 意图：让作者在一页内对照来源版本原文、清理候选结果与榜单快照。
|
||||
- 职责：来源清单选择、版本原文呈现、清理对照应用、快照列表（失败态如实显示）。
|
||||
- 边界：只读呈现与显式动作；不伪造数据、不推断效果因果。
|
||||
- 目标：来源、原文与快照可追溯；失败可见可重试。
|
||||
- 合同依据：[B03 资料研究](../模块设计/B03-资料研究.md)。
|
||||
|
||||
<a id="file-0c134b40a9faae3a"></a>
|
||||
|
||||
### `web/tests/旅程/资料对照.spec.ts`
|
||||
|
||||
- 归属与种类：验证；前端浏览器旅程。
|
||||
- 意图与职责：真实浏览器与隔离库核对资料对照页的来源、原文、清理说明与榜单失败态。
|
||||
- 边界：真实隔离库与真实浏览器；不 mock 后端；视觉判定独立留证。
|
||||
- 目标：对照呈现与失败态可复核，页面无预期外控制台错误。
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
# 测试与夹具：文件合同
|
||||
|
||||
本页从[目标文件清单](../目标文件清单.json)投影，共 137 个目标条目。测试用例的具体规格在独立用例清单；文件和目标合同不代表实现或验收已完成。
|
||||
本页从[目标文件清单](../目标文件清单.json)投影，共 139 个目标条目。测试用例的具体规格在独立用例清单；文件和目标合同不代表实现或验收已完成。
|
||||
|
||||
[返回目录](目录.md) · [完整项目树](../项目目录与文件职责.md) · [用例重写](../验证设计/测试用例重写.md)
|
||||
|
||||
@ -144,6 +144,7 @@ agent-example/
|
||||
│ ├── test_故事事实确认与时点.py # 事实确认、动态类型、依据有效性和时间边界的PG验证
|
||||
│ ├── test_数据库连接与迁移.py # 数据库连接、用途隔离与迁移行为验证
|
||||
│ ├── test_新增类型完整消费.py # 新增类型完整消费行为验证
|
||||
│ ├── test_榜单与清理.py # 榜单快照与清理验证
|
||||
│ ├── test_正式变更事务.py # 正式变更共用机制的PG并发与原子性
|
||||
│ ├── test_正式提交原子性.py # 正式提交原子性行为验证
|
||||
│ ├── test_正文候选与作者决定.py # 人工候选版本、部分选择和真实作者事务
|
||||
@ -155,6 +156,7 @@ agent-example/
|
||||
│ ├── test_规划候选与确认.py # 规划候选、来源与确认事务
|
||||
│ ├── test_证据暂存补交.py # 证据暂存补交行为验证
|
||||
│ ├── test_评测执行与失败收敛.py # 评测执行与失败收敛的逐例行为验证
|
||||
│ ├── test_资料导入与版本.py # 来源导入与版本链验证
|
||||
│ └── test_预算预留与结算.py # 真实预算预留、结算与配置版本CAS
|
||||
└── web/
|
||||
└── tests/
|
||||
@ -1709,3 +1711,19 @@ agent-example/
|
||||
- 目标：越权与不存在任务拒绝；漂移失效可见且历史原样。
|
||||
- 合同依据：[B09上下文](../模块设计/B09-上下文.md)。
|
||||
- 用例合同：[NC-ctx-workbench-author-view](../验证设计/创作与正式变更用例.md#nc-ctx-workbench-author-view)；[NC-ctx-workbench-denied](../验证设计/创作与正式变更用例.md#nc-ctx-workbench-denied)；[NC-ctx-workbench-drift-visible](../验证设计/创作与正式变更用例.md#nc-ctx-workbench-drift-visible)。
|
||||
|
||||
<a id="file-612a88402d3230d7"></a>
|
||||
|
||||
### `tests/集成/test_资料导入与版本.py`
|
||||
|
||||
- 意图与职责：导入去重、版本链、证据校验、用途授权与清理对照的隔离数据库验证。
|
||||
- 边界：真实隔离 PG；合成数据只作内容，不作行为替身。
|
||||
- 目标：原文不变、重复可辨认、授权可拒绝等行为可复验。
|
||||
|
||||
<a id="file-46640105285f0a42"></a>
|
||||
|
||||
### `tests/集成/test_榜单与清理.py`
|
||||
|
||||
- 意图与职责：榜单采集成功/失败二态、观察差异与未登记拒绝的隔离数据库验证。
|
||||
- 边界：合成读取器按剧本回应；不访问网络。
|
||||
- 目标：失败不伪造、快照可查、差异为观察陈述可复验。
|
||||
|
||||
@ -21590,6 +21590,76 @@
|
||||
"test_case_refs": [
|
||||
"NC-browser-generation-dialog"
|
||||
]
|
||||
},
|
||||
{
|
||||
"path": "web/src/功能/资料研究/资料对照页.tsx",
|
||||
"owner": "前端",
|
||||
"kind": "源码",
|
||||
"responsibility": "来源与原文对照页面",
|
||||
"project_part": "前端",
|
||||
"intent": "来源与原文对照页面",
|
||||
"boundary": "只经公开接口与登记输入；不越权、不伪造完成",
|
||||
"goal": "行为可复验",
|
||||
"contract_basis": "专属业务职责",
|
||||
"status": "目标合同；不表示已实现或已验证",
|
||||
"legacy_sources": [],
|
||||
"design_location": "文件设计/前端-应用与通用界面.md#file-8d1a7fc626190cb1",
|
||||
"test_case_refs": []
|
||||
},
|
||||
{
|
||||
"path": "web/tests/旅程/资料对照.spec.ts",
|
||||
"owner": "验证",
|
||||
"kind": "测试",
|
||||
"responsibility": "资料对照页真实浏览器旅程",
|
||||
"project_part": "前端",
|
||||
"intent": "资料对照页真实浏览器旅程",
|
||||
"boundary": "只经公开接口与登记输入；不越权、不伪造完成",
|
||||
"goal": "行为可复验",
|
||||
"contract_basis": "行为验证",
|
||||
"status": "目标合同；不表示已实现或已验证",
|
||||
"legacy_sources": [],
|
||||
"design_location": "文件设计/前端-应用与通用界面.md#file-0c134b40a9faae3a",
|
||||
"test_case_refs": [
|
||||
"NC-w17-browser"
|
||||
]
|
||||
},
|
||||
{
|
||||
"path": "tests/集成/test_资料导入与版本.py",
|
||||
"owner": "验证",
|
||||
"kind": "测试",
|
||||
"responsibility": "来源导入与版本链验证",
|
||||
"project_part": "后端",
|
||||
"intent": "来源导入与版本链验证",
|
||||
"boundary": "只经公开接口与登记输入；不越权、不伪造完成",
|
||||
"goal": "行为可复验",
|
||||
"contract_basis": "行为验证",
|
||||
"status": "目标合同；不表示已实现或已验证",
|
||||
"legacy_sources": [],
|
||||
"design_location": "文件设计/测试与夹具.md#file-612a88402d3230d7",
|
||||
"test_case_refs": [
|
||||
"NC-w17-w17a01",
|
||||
"NC-w17-w17a02",
|
||||
"NC-w17-w17a03",
|
||||
"NC-w17-w17a04"
|
||||
]
|
||||
},
|
||||
{
|
||||
"path": "tests/集成/test_榜单与清理.py",
|
||||
"owner": "验证",
|
||||
"kind": "测试",
|
||||
"responsibility": "榜单快照与清理验证",
|
||||
"project_part": "后端",
|
||||
"intent": "榜单快照与清理验证",
|
||||
"boundary": "只经公开接口与登记输入；不越权、不伪造完成",
|
||||
"goal": "行为可复验",
|
||||
"contract_basis": "行为验证",
|
||||
"status": "目标合同；不表示已实现或已验证",
|
||||
"legacy_sources": [],
|
||||
"design_location": "文件设计/测试与夹具.md#file-46640105285f0a42",
|
||||
"test_case_refs": [
|
||||
"NC-w17-w17b01",
|
||||
"NC-w17-w17b02"
|
||||
]
|
||||
}
|
||||
],
|
||||
"contract_fields": {
|
||||
|
||||
@ -1175,6 +1175,7 @@ agent-example/
|
||||
│ ├── test_故事事实确认与时点.py # 事实确认、动态类型、依据有效性和时间边界的PG验证
|
||||
│ ├── test_数据库连接与迁移.py # 数据库连接、用途隔离与迁移行为验证
|
||||
│ ├── test_新增类型完整消费.py # 新增类型完整消费行为验证
|
||||
│ ├── test_榜单与清理.py # 榜单快照与清理验证
|
||||
│ ├── test_正式变更事务.py # 正式变更共用机制的PG并发与原子性
|
||||
│ ├── test_正式提交原子性.py # 正式提交原子性行为验证
|
||||
│ ├── test_正文候选与作者决定.py # 人工候选版本、部分选择和真实作者事务
|
||||
@ -1186,6 +1187,7 @@ agent-example/
|
||||
│ ├── test_规划候选与确认.py # 规划候选、来源与确认事务
|
||||
│ ├── test_证据暂存补交.py # 证据暂存补交行为验证
|
||||
│ ├── test_评测执行与失败收敛.py # 评测执行与失败收敛的逐例行为验证
|
||||
│ ├── test_资料导入与版本.py # 来源导入与版本链验证
|
||||
│ └── test_预算预留与结算.py # 真实预算预留、结算与配置版本CAS
|
||||
├── upstream/
|
||||
│ └── 溯源.yaml # 继承来源及版本溯源
|
||||
@ -1303,6 +1305,7 @@ agent-example/
|
||||
│ │ │ ├── 拆书工作区.tsx # 分析进度与恢复
|
||||
│ │ │ ├── 数据.ts # 资料研究查询和业务命令
|
||||
│ │ │ ├── 榜单与选题页.tsx # 快照、比较和选书
|
||||
│ │ │ ├── 资料对照页.tsx # 来源与原文对照页面
|
||||
│ │ │ └── 资料页.tsx # 来源导入和版本
|
||||
│ │ ├── 应用/
|
||||
│ │ │ ├── 作品会话.ts # 切作品屏障、取消和迟到隔离
|
||||
@ -1361,7 +1364,8 @@ agent-example/
|
||||
│ │ │ ├── 拆书到方法绑定.spec.ts # 来源到方法消费
|
||||
│ │ │ ├── 探索草稿.spec.ts # 私人探索的实际作者浏览器旅程
|
||||
│ │ │ ├── 生成对话.spec.ts # 创作对话真实浏览器旅程验证
|
||||
│ │ │ └── 编辑冲突与恢复.spec.ts # 编辑保存和任务恢复
|
||||
│ │ │ ├── 编辑冲突与恢复.spec.ts # 编辑保存和任务恢复
|
||||
│ │ │ └── 资料对照.spec.ts # 资料对照页真实浏览器旅程
|
||||
│ │ ├── 编辑/
|
||||
│ │ │ ├── 中文输入.test.tsx # 输入法组合时不破坏正文或提前保存
|
||||
│ │ │ ├── 保存冲突.test.tsx # 并发编辑不覆盖
|
||||
|
||||
@ -5230,3 +5230,75 @@
|
||||
- 环境：真实隔离 PG。
|
||||
- 合同：[权威定义](../功能规格/P05-章后处理与事实复核.md)。
|
||||
|
||||
|
||||
<a id="nc-w17-w17a01"></a>
|
||||
|
||||
## NC-w17-w17a01 · test_导入去重与版本链__17a01a
|
||||
|
||||
- 位置：`tests/集成/test_资料导入与版本.py::test_导入去重与版本链__17a01a`。
|
||||
- 给定：隔离数据库。
|
||||
- 动作：同源同内容幂等返回既有版本；同源异内容追加版本且授权只增不减。
|
||||
- 预期：重复可辨认；版本链完整；授权持久。
|
||||
- 环境：真实隔离 PG。
|
||||
- 合同：[权威定义](../模块设计/B03-资料研究.md)。
|
||||
|
||||
|
||||
<a id="nc-w17-w17a02"></a>
|
||||
|
||||
## NC-w17-w17a02 · test_证据读取与哈希校验__17a02b
|
||||
|
||||
- 位置：`tests/集成/test_资料导入与版本.py::test_证据读取与哈希校验__17a02b`。
|
||||
- 给定：隔离数据库。
|
||||
- 动作：按码点区间取原文片段并校验内容哈希；越界拒绝。
|
||||
- 预期：片段正确；越界拒绝。
|
||||
- 环境：真实隔离 PG。
|
||||
- 合同：[权威定义](../模块设计/B03-资料研究.md)。
|
||||
|
||||
|
||||
<a id="nc-w17-w17a03"></a>
|
||||
|
||||
## NC-w17-w17a03 · test_用途未授权拒绝消费__17a03c
|
||||
|
||||
- 位置：`tests/集成/test_资料导入与版本.py::test_用途未授权拒绝消费__17a03c`。
|
||||
- 给定：隔离数据库。
|
||||
- 动作：来源未授权的用途被拒绝。
|
||||
- 预期：未授权拒绝。
|
||||
- 环境：真实隔离 PG。
|
||||
- 合同：[权威定义](../模块设计/B03-资料研究.md)。
|
||||
|
||||
|
||||
<a id="nc-w17-w17a04"></a>
|
||||
|
||||
## NC-w17-w17a04 · test_清理候选对照且原文不变__17a04d
|
||||
|
||||
- 位置：`tests/集成/test_资料导入与版本.py::test_清理候选对照且原文不变__17a04d`。
|
||||
- 给定：隔离数据库。
|
||||
- 动作：清理生成对照而不改写原文；处置不可改写。
|
||||
- 预期：原文不变；对照生成；处置终态。
|
||||
- 环境：真实隔离 PG。
|
||||
- 合同：[权威定义](../模块设计/B03-资料研究.md)。
|
||||
|
||||
|
||||
<a id="nc-w17-w17b01"></a>
|
||||
|
||||
## NC-w17-w17b01 · test_采集成功失败与观察差异__17b01e
|
||||
|
||||
- 位置：`tests/集成/test_榜单与清理.py::test_采集成功失败与观察差异__17b01e`。
|
||||
- 给定：隔离数据库。
|
||||
- 动作：失败保留失败态不伪造；成功快照带条目；观察差异不含 run 归因。
|
||||
- 预期：失败态保留；快照可查；差异为观察陈述。
|
||||
- 环境：真实隔离 PG。
|
||||
- 合同：[权威定义](../模块设计/B03-资料研究.md)。
|
||||
|
||||
|
||||
<a id="nc-w17-w17b02"></a>
|
||||
|
||||
## NC-w17-w17b02 · test_未登记榜单拒绝采集__17b02f
|
||||
|
||||
- 位置：`tests/集成/test_榜单与清理.py::test_未登记榜单拒绝采集__17b02f`。
|
||||
- 给定：隔离数据库。
|
||||
- 动作：未登记的榜种不能采集。
|
||||
- 预期：未登记拒绝。
|
||||
- 环境：真实隔离 PG。
|
||||
- 合同：[权威定义](../模块设计/B03-资料研究.md)。
|
||||
|
||||
|
||||
@ -3234,3 +3234,15 @@
|
||||
- 环境：真实浏览器隔离库。
|
||||
- 合同：[权威定义](../接口契约/任务工具与事件.md)。
|
||||
|
||||
|
||||
<a id="nc-browser-source-workbench"></a>
|
||||
|
||||
## NC-w17-browser · NC-browser-source-workbench：资料对照页与榜单失败态
|
||||
|
||||
- 位置：`web/tests/旅程/资料对照.spec.ts::NC-browser-source-workbench：资料对照页与榜单失败态`。
|
||||
- 给定：隔离浏览器环境与预置来源/快照。
|
||||
- 动作：打开资料对照页并应用清理、展开榜单快照。
|
||||
- 预期：来源与原文对照可见；清理说明可见；失败快照如实呈现。
|
||||
- 环境：真实隔离库 + Chrome。
|
||||
- 合同：[权威定义](../模块设计/B03-资料研究.md)。
|
||||
|
||||
|
||||
@ -123109,6 +123109,120 @@
|
||||
"environment": "真实隔离 PG",
|
||||
"verification_state": "已实现未执行",
|
||||
"execution_evidence": ".agents.local/改造/R2-20260909/W16/回合开发/整包/"
|
||||
},
|
||||
{
|
||||
"case_id": "NC-w17-w17a01",
|
||||
"file": "tests/集成/test_资料导入与版本.py",
|
||||
"symbol": "test_导入去重与版本链__17a01a",
|
||||
"design_location": "验证设计/创作与正式变更用例.md#nc-w17-w17a01",
|
||||
"contract": "模块设计/B03-资料研究.md",
|
||||
"given": "隔离数据库",
|
||||
"when": "同源同内容幂等返回既有版本；同源异内容追加版本且授权只增不减",
|
||||
"then": [
|
||||
"重复可辨认",
|
||||
"版本链完整",
|
||||
"授权持久"
|
||||
],
|
||||
"environment": "真实隔离 PG",
|
||||
"verification_state": "已实现未执行",
|
||||
"execution_evidence": ".agents.local/改造/R2-20260909/W17/回合开发/整包/"
|
||||
},
|
||||
{
|
||||
"case_id": "NC-w17-w17a02",
|
||||
"file": "tests/集成/test_资料导入与版本.py",
|
||||
"symbol": "test_证据读取与哈希校验__17a02b",
|
||||
"design_location": "验证设计/创作与正式变更用例.md#nc-w17-w17a02",
|
||||
"contract": "模块设计/B03-资料研究.md",
|
||||
"given": "隔离数据库",
|
||||
"when": "按码点区间取原文片段并校验内容哈希；越界拒绝",
|
||||
"then": [
|
||||
"片段正确",
|
||||
"越界拒绝"
|
||||
],
|
||||
"environment": "真实隔离 PG",
|
||||
"verification_state": "已实现未执行",
|
||||
"execution_evidence": ".agents.local/改造/R2-20260909/W17/回合开发/整包/"
|
||||
},
|
||||
{
|
||||
"case_id": "NC-w17-w17a03",
|
||||
"file": "tests/集成/test_资料导入与版本.py",
|
||||
"symbol": "test_用途未授权拒绝消费__17a03c",
|
||||
"design_location": "验证设计/创作与正式变更用例.md#nc-w17-w17a03",
|
||||
"contract": "模块设计/B03-资料研究.md",
|
||||
"given": "隔离数据库",
|
||||
"when": "来源未授权的用途被拒绝",
|
||||
"then": [
|
||||
"未授权拒绝"
|
||||
],
|
||||
"environment": "真实隔离 PG",
|
||||
"verification_state": "已实现未执行",
|
||||
"execution_evidence": ".agents.local/改造/R2-20260909/W17/回合开发/整包/"
|
||||
},
|
||||
{
|
||||
"case_id": "NC-w17-w17a04",
|
||||
"file": "tests/集成/test_资料导入与版本.py",
|
||||
"symbol": "test_清理候选对照且原文不变__17a04d",
|
||||
"design_location": "验证设计/创作与正式变更用例.md#nc-w17-w17a04",
|
||||
"contract": "模块设计/B03-资料研究.md",
|
||||
"given": "隔离数据库",
|
||||
"when": "清理生成对照而不改写原文；处置不可改写",
|
||||
"then": [
|
||||
"原文不变",
|
||||
"对照生成",
|
||||
"处置终态"
|
||||
],
|
||||
"environment": "真实隔离 PG",
|
||||
"verification_state": "已实现未执行",
|
||||
"execution_evidence": ".agents.local/改造/R2-20260909/W17/回合开发/整包/"
|
||||
},
|
||||
{
|
||||
"case_id": "NC-w17-w17b01",
|
||||
"file": "tests/集成/test_榜单与清理.py",
|
||||
"symbol": "test_采集成功失败与观察差异__17b01e",
|
||||
"design_location": "验证设计/创作与正式变更用例.md#nc-w17-w17b01",
|
||||
"contract": "模块设计/B03-资料研究.md",
|
||||
"given": "隔离数据库",
|
||||
"when": "失败保留失败态不伪造；成功快照带条目；观察差异不含 run 归因",
|
||||
"then": [
|
||||
"失败态保留",
|
||||
"快照可查",
|
||||
"差异为观察陈述"
|
||||
],
|
||||
"environment": "真实隔离 PG",
|
||||
"verification_state": "已实现未执行",
|
||||
"execution_evidence": ".agents.local/改造/R2-20260909/W17/回合开发/整包/"
|
||||
},
|
||||
{
|
||||
"case_id": "NC-w17-w17b02",
|
||||
"file": "tests/集成/test_榜单与清理.py",
|
||||
"symbol": "test_未登记榜单拒绝采集__17b02f",
|
||||
"design_location": "验证设计/创作与正式变更用例.md#nc-w17-w17b02",
|
||||
"contract": "模块设计/B03-资料研究.md",
|
||||
"given": "隔离数据库",
|
||||
"when": "未登记的榜种不能采集",
|
||||
"then": [
|
||||
"未登记拒绝"
|
||||
],
|
||||
"environment": "真实隔离 PG",
|
||||
"verification_state": "已实现未执行",
|
||||
"execution_evidence": ".agents.local/改造/R2-20260909/W17/回合开发/整包/"
|
||||
},
|
||||
{
|
||||
"case_id": "NC-w17-browser",
|
||||
"file": "web/tests/旅程/资料对照.spec.ts",
|
||||
"symbol": "NC-browser-source-workbench：资料对照页与榜单失败态",
|
||||
"design_location": "验证设计/工作台与工程用例.md#nc-browser-source-workbench",
|
||||
"contract": "模块设计/B03-资料研究.md",
|
||||
"given": "隔离浏览器环境与预置来源/快照",
|
||||
"when": "打开资料对照页并应用清理、展开榜单快照",
|
||||
"then": [
|
||||
"来源与原文对照可见",
|
||||
"清理说明可见",
|
||||
"失败快照如实呈现"
|
||||
],
|
||||
"environment": "真实隔离库 + Chrome",
|
||||
"verification_state": "已实现未执行",
|
||||
"execution_evidence": ".agents.local/改造/R2-20260909/W17/回合开发/整包/"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
@ -118,6 +118,12 @@ class 应用装配:
|
||||
raise 装配未完成("正文服务未注入")
|
||||
return self.正文
|
||||
|
||||
def 要求资料(self) -> Any:
|
||||
"""资料研究无状态公开入口；连接由调用方事务提供。"""
|
||||
from muse.资料研究.接口 import 资料服务
|
||||
|
||||
return 资料服务()
|
||||
|
||||
def 要求证据(self) -> 证据服务:
|
||||
return 证据服务(数据库工厂(self.配置.数据库, self.配置.运行用途))
|
||||
|
||||
|
||||
1
src/muse/基础设施/资料来源/__init__.py
Normal file
1
src/muse/基础设施/资料来源/__init__.py
Normal file
@ -0,0 +1 @@
|
||||
"""资料来源读取的基础设施注入面；真实网络实现须显式标记用途。"""
|
||||
16
src/muse/基础设施/资料来源/榜单读取.py
Normal file
16
src/muse/基础设施/资料来源/榜单读取.py
Normal file
@ -0,0 +1,16 @@
|
||||
"""榜单读取协议：把具名榜种页面解析为名次条目；实现可注入。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Protocol
|
||||
|
||||
from muse.资料研究.模型 import 榜单读取结果
|
||||
|
||||
|
||||
class 榜单页面读取器(Protocol):
|
||||
"""按 URL 与采集方式解析榜单条目；失败返回失败结果，不补造数据。"""
|
||||
|
||||
def 抓取(self, url: str, 方式: str) -> 榜单读取结果: ...
|
||||
|
||||
|
||||
__all__ = ["榜单页面读取器"]
|
||||
16
src/muse/基础设施/资料来源/网页读取.py
Normal file
16
src/muse/基础设施/资料来源/网页读取.py
Normal file
@ -0,0 +1,16 @@
|
||||
"""网页读取协议：抓取外部页面为原文内容；实现可注入，失败如实上抛。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Protocol
|
||||
|
||||
from muse.资料研究.模型 import 榜单读取结果
|
||||
|
||||
|
||||
class 网页读取器(Protocol):
|
||||
"""按 URL 抓取文本；实现负责超时、编码与反爬礼貌，失败返回失败结果。"""
|
||||
|
||||
def 抓取网页(self, url: str) -> 榜单读取结果: ...
|
||||
|
||||
|
||||
__all__ = ["网页读取器"]
|
||||
@ -23,6 +23,7 @@ from muse.接入.http.路由.元数据 import 路由 as 元数据路由
|
||||
from muse.接入.http.路由.创作生成 import 路由 as 生成路由
|
||||
from muse.接入.http.路由.故事世界 import 路由 as 世界路由
|
||||
from muse.接入.http.路由.正文写作 import 路由 as 正文路由
|
||||
from muse.接入.http.路由.资料研究 import 路由 as 资料路由
|
||||
from muse.接入.http.错误响应 import 协议错误, 安装错误响应
|
||||
from muse.接入.http.静态资源 import 挂载工作台
|
||||
from muse.配置 import 应用配置
|
||||
@ -84,6 +85,7 @@ def 注册API(应用: FastAPI) -> None:
|
||||
应用.include_router(作品路由)
|
||||
应用.include_router(正文路由)
|
||||
应用.include_router(生成路由)
|
||||
应用.include_router(资料路由)
|
||||
应用.include_router(审阅路由)
|
||||
应用.include_router(世界路由)
|
||||
应用.include_router(事件路由)
|
||||
|
||||
159
src/muse/接入/http/路由/资料研究.py
Normal file
159
src/muse/接入/http/路由/资料研究.py
Normal file
@ -0,0 +1,159 @@
|
||||
"""资料研究的 HTTP 外壳：导入、来源版本对照、清理候选与榜单快照。"""
|
||||
|
||||
from typing import Any, Literal
|
||||
|
||||
from fastapi import APIRouter, HTTPException, Request
|
||||
from pydantic import BaseModel, ConfigDict, Field
|
||||
|
||||
from muse.接入.http.作者会话 import 作者依赖
|
||||
from muse.接入.创作操作 import 创作身份
|
||||
|
||||
路由 = APIRouter(prefix="/api/v1", tags=["资料研究"])
|
||||
|
||||
|
||||
class 导入来源请求(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
kind: Literal["author_draft", "reference", "web", "note", "inspiration"]
|
||||
title: str = Field(min_length=1)
|
||||
origin: str = Field(min_length=1)
|
||||
content: str
|
||||
authorized_uses: list[str] = []
|
||||
|
||||
|
||||
class 清理候选动作请求(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
candidate_id: str = Field(min_length=1)
|
||||
|
||||
|
||||
class 登记榜单请求(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
ranking_id: str = Field(min_length=1)
|
||||
name: str = Field(min_length=1)
|
||||
url: str = Field(min_length=1)
|
||||
method: str = "html"
|
||||
|
||||
|
||||
class 采集榜单请求(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
ranking_id: str = Field(min_length=1)
|
||||
|
||||
|
||||
def _装配(request: Request):
|
||||
return request.app.state.装配
|
||||
|
||||
|
||||
@路由.post("/sources/import", operation_id="import_source", status_code=201)
|
||||
def 导入来源(work_id: str, 请求: 导入来源请求, request: Request, 作者: 作者依赖) -> dict:
|
||||
装配 = _装配(request)
|
||||
from muse.资料研究.模型 import 导入请求
|
||||
|
||||
with 装配.要求数据库().连接() as 连, 连.transaction():
|
||||
回执 = 装配.要求资料().导入(
|
||||
连,
|
||||
创作身份(作者, 装配.配置).作者,
|
||||
导入请求(
|
||||
请求.kind,
|
||||
请求.title,
|
||||
请求.origin,
|
||||
请求.content,
|
||||
tuple(请求.authorized_uses),
|
||||
),
|
||||
)
|
||||
return {
|
||||
"source_id": 回执.source_id,
|
||||
"revision": 回执.revision,
|
||||
"content_hash": 回执.content_hash,
|
||||
"duplicate": 回执.重复,
|
||||
}
|
||||
|
||||
|
||||
@路由.get("/sources", operation_id="list_sources")
|
||||
def 列出来源(request: Request, 作者: 作者依赖) -> list[dict[str, Any]]:
|
||||
装配 = _装配(request)
|
||||
with 装配.要求数据库().连接(只读=True) as 连:
|
||||
return 装配.要求资料().列出来源(连)
|
||||
|
||||
|
||||
@路由.get("/sources/{source_id}/versions/{revision}", operation_id="read_source_version")
|
||||
def 读取版本(source_id: str, revision: int, request: Request, 作者: 作者依赖) -> dict[str, Any]:
|
||||
装配 = _装配(request)
|
||||
with 装配.要求数据库().连接(只读=True) as 连:
|
||||
try:
|
||||
return 装配.要求资料().读取版本(连, source_id, revision)
|
||||
except Exception:
|
||||
raise HTTPException(404, "来源版本不存在") from None
|
||||
|
||||
|
||||
@路由.post(
|
||||
"/sources/{source_id}/versions/{revision}/clean-candidates",
|
||||
operation_id="create_clean_candidate",
|
||||
status_code=201,
|
||||
)
|
||||
def 生成清理候选(source_id: str, revision: int, request: Request, 作者: 作者依赖) -> dict:
|
||||
装配 = _装配(request)
|
||||
with 装配.要求数据库().连接() as 连, 连.transaction():
|
||||
候选 = 装配.要求资料().生成清理候选(连, source_id, revision)
|
||||
return {
|
||||
"candidate_id": 候选.candidate_id,
|
||||
"removals": list(候选.removals),
|
||||
"status": 候选.status,
|
||||
}
|
||||
|
||||
|
||||
@路由.get(
|
||||
"/sources/{source_id}/versions/{revision}/clean-candidates",
|
||||
operation_id="list_clean_candidates",
|
||||
)
|
||||
def 列出清理候选(
|
||||
source_id: str, revision: int, request: Request, 作者: 作者依赖
|
||||
) -> list[dict[str, Any]]:
|
||||
装配 = _装配(request)
|
||||
with 装配.要求数据库().连接(只读=True) as 连:
|
||||
return 装配.要求资料().列出清理候选(连, source_id, revision)
|
||||
|
||||
|
||||
@路由.post("/clean-candidates/{candidate_id}/apply", operation_id="apply_clean_candidate")
|
||||
def 应用清理候选(candidate_id: str, request: Request, 作者: 作者依赖) -> dict:
|
||||
装配 = _装配(request)
|
||||
with 装配.要求数据库().连接() as 连, 连.transaction():
|
||||
候选 = 装配.要求资料().应用清理候选(连, candidate_id)
|
||||
return {
|
||||
"candidate_id": 候选.candidate_id,
|
||||
"status": 候选.status,
|
||||
"cleaned_hash": 候选.cleaned_hash,
|
||||
}
|
||||
|
||||
|
||||
@路由.post("/clean-candidates/{candidate_id}/reject", operation_id="reject_clean_candidate")
|
||||
def 拒绝清理候选(candidate_id: str, request: Request, 作者: 作者依赖) -> dict:
|
||||
装配 = _装配(request)
|
||||
with 装配.要求数据库().连接() as 连, 连.transaction():
|
||||
装配.要求资料().拒绝清理候选(连, candidate_id)
|
||||
return {"candidate_id": candidate_id, "status": "rejected"}
|
||||
|
||||
|
||||
@路由.post("/rankings", operation_id="register_ranking", status_code=201)
|
||||
def 登记榜单(请求: 登记榜单请求, request: Request, 作者: 作者依赖) -> dict:
|
||||
装配 = _装配(request)
|
||||
with 装配.要求数据库().连接() as 连, 连.transaction():
|
||||
装配.要求资料().登记榜单(连, 请求.ranking_id, 请求.name, 请求.url, 请求.method)
|
||||
return {"ranking_id": 请求.ranking_id}
|
||||
|
||||
|
||||
@路由.post(
|
||||
"/rankings/{ranking_id}/snapshots", operation_id="capture_ranking_snapshot", status_code=201
|
||||
)
|
||||
def 采集快照(ranking_id: str, request: Request, 作者: 作者依赖) -> dict:
|
||||
装配 = _装配(request)
|
||||
读取器 = getattr(装配.配置, "榜单读取器", None)
|
||||
if 读取器 is None:
|
||||
raise HTTPException(503, "未装配榜单读取器；真实采集需部署侧注入并显式授权")
|
||||
with 装配.要求数据库().连接() as 连, 连.transaction():
|
||||
return 装配.要求资料().采集榜单快照(连, ranking_id, 读取器)
|
||||
|
||||
|
||||
@路由.get("/rankings/{ranking_id}/snapshots", operation_id="list_ranking_snapshots")
|
||||
def 列出快照(ranking_id: str, request: Request, 作者: 作者依赖) -> list[dict[str, Any]]:
|
||||
装配 = _装配(request)
|
||||
with 装配.要求数据库().连接(只读=True) as 连:
|
||||
return 装配.要求资料().列出快照(连, ranking_id)
|
||||
1
src/muse/资料研究/__init__.py
Normal file
1
src/muse/资料研究/__init__.py
Normal file
@ -0,0 +1 @@
|
||||
"""B03 资料研究：外部与私人资料、原文版本、清理候选与榜单观察。"""
|
||||
37
src/muse/资料研究/原文版本.py
Normal file
37
src/muse/资料研究/原文版本.py
Normal file
@ -0,0 +1,37 @@
|
||||
"""原文版本：原文保留、来源位置及不可变版本；任何旧引文仍能回到当时原文及位置。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
from typing import Any
|
||||
|
||||
from muse.资料研究.存储 import 资料存储
|
||||
from muse.资料研究.模型 import 资料错误
|
||||
|
||||
|
||||
def 内容哈希(内容: str) -> str:
|
||||
return hashlib.sha256(内容.encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
def 读取证据(连, source_id: str, revision: int, 起点: int, 终点: int) -> dict[str, Any]:
|
||||
"""按来源版本与码点区间取片段并校验内容哈希；版本不可变，旧引文恒可回读。"""
|
||||
|
||||
存储 = 资料存储(连)
|
||||
行 = 存储.读版本(source_id, revision)
|
||||
if 行 is None:
|
||||
raise 资料错误("来源版本不存在")
|
||||
if not (0 <= 起点 < 终点 <= len(行["content"])):
|
||||
raise 资料错误("证据区间越界")
|
||||
if 内容哈希(行["content"]) != 行["content_hash"]:
|
||||
raise 资料错误("来源内容与哈希不一致，不能作为证据")
|
||||
return {
|
||||
"source_id": source_id,
|
||||
"revision": int(行["revision"]),
|
||||
"起点": 起点,
|
||||
"终点": 终点,
|
||||
"片段": 行["content"][起点:终点],
|
||||
"content_hash": 行["content_hash"],
|
||||
}
|
||||
|
||||
|
||||
__all__ = ["内容哈希", "读取证据", "资料错误"]
|
||||
189
src/muse/资料研究/存储.py
Normal file
189
src/muse/资料研究/存储.py
Normal file
@ -0,0 +1,189 @@
|
||||
"""资料研究记录读写：来源、版本、清理候选与榜单快照；原文只追加不可改写。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from psycopg.rows import dict_row
|
||||
from psycopg.types.json import Jsonb
|
||||
|
||||
from muse.资料研究.模型 import 资料错误
|
||||
|
||||
|
||||
class 资料存储:
|
||||
def __init__(self, 连) -> None:
|
||||
self.连 = 连
|
||||
|
||||
def 查询(self, 语句: str, 参数: tuple = ()):
|
||||
"""本模块固定 SQL 专用；结果行按列名访问。"""
|
||||
|
||||
游标 = self.连.cursor(row_factory=dict_row)
|
||||
return 游标.execute(语句, 参数)
|
||||
|
||||
# ---- 来源与版本 ----
|
||||
|
||||
def 查源(self, origin: str, kind: str) -> dict[str, Any] | None:
|
||||
行 = self.查询(
|
||||
"SELECT source_id,kind,title,origin,authorized_uses,created_at "
|
||||
"FROM muse_source WHERE origin=%s AND kind=%s",
|
||||
(origin, kind),
|
||||
).fetchone()
|
||||
if 行 is None:
|
||||
return None
|
||||
源 = dict(行)
|
||||
源["source_id"] = str(源["source_id"])
|
||||
return 源
|
||||
|
||||
def 建源(
|
||||
self, source_id: str, kind: str, title: str, origin: str, authorized_uses: list[str]
|
||||
) -> dict[str, Any]:
|
||||
行 = self.查询(
|
||||
"INSERT INTO muse_source (source_id,kind,title,origin,authorized_uses) "
|
||||
"VALUES (%s,%s,%s,%s,%s) "
|
||||
"ON CONFLICT (origin,kind) DO UPDATE SET title=EXCLUDED.title, "
|
||||
"authorized_uses=EXCLUDED.authorized_uses "
|
||||
"RETURNING source_id,kind,title,origin,authorized_uses,created_at",
|
||||
(source_id, kind, title, origin, Jsonb(authorized_uses)),
|
||||
).fetchone()
|
||||
源 = dict(行)
|
||||
源["source_id"] = str(源["source_id"])
|
||||
return 源
|
||||
|
||||
def 当前版本(self, source_id: str) -> int:
|
||||
行 = self.查询(
|
||||
"SELECT coalesce(max(revision),0) AS 最新 FROM muse_source_version WHERE source_id=%s",
|
||||
(source_id,),
|
||||
).fetchone()
|
||||
return int(行["最新"])
|
||||
|
||||
def 同内容版本(self, source_id: str, content_hash: str) -> dict[str, Any] | None:
|
||||
行 = self.查询(
|
||||
"SELECT source_id,revision,content_hash,imported_at FROM muse_source_version "
|
||||
"WHERE source_id=%s AND content_hash=%s ORDER BY revision LIMIT 1",
|
||||
(source_id, content_hash),
|
||||
).fetchone()
|
||||
return dict(行) if 行 else None
|
||||
|
||||
def 写版本(
|
||||
self, source_id: str, revision: int, content_hash: str, content: str, import_result: dict
|
||||
) -> None:
|
||||
self.查询(
|
||||
"INSERT INTO muse_source_version "
|
||||
"(source_id,revision,content_hash,content,import_result) VALUES (%s,%s,%s,%s,%s)",
|
||||
(source_id, revision, content_hash, content, Jsonb(import_result)),
|
||||
)
|
||||
|
||||
def 读版本(self, source_id: str, revision: int | None = None) -> dict[str, Any] | None:
|
||||
行 = self.查询(
|
||||
"SELECT source_id,revision,content_hash,content,import_result,imported_at "
|
||||
"FROM muse_source_version WHERE source_id=%s AND (%s::bigint IS NULL OR revision=%s) "
|
||||
"ORDER BY revision DESC LIMIT 1",
|
||||
(source_id, revision, revision),
|
||||
).fetchone()
|
||||
return dict(行) if 行 else None
|
||||
|
||||
def 列出来源(self) -> list[dict[str, Any]]:
|
||||
行集 = self.查询(
|
||||
"SELECT s.source_id,s.kind,s.title,s.origin,s.authorized_uses, "
|
||||
"coalesce(max(v.revision),0) AS 最新版本, max(v.imported_at) AS 最近导入 "
|
||||
"FROM muse_source s LEFT JOIN muse_source_version v USING(source_id) "
|
||||
"GROUP BY s.source_id ORDER BY s.created_at DESC"
|
||||
).fetchall()
|
||||
return [dict(行) for 行 in 行集]
|
||||
|
||||
# ---- 清理候选 ----
|
||||
|
||||
def 写清理候选(
|
||||
self,
|
||||
candidate_id: str,
|
||||
source_id: str,
|
||||
revision: int,
|
||||
removals: list[dict],
|
||||
status: str,
|
||||
cleaned_hash: str | None = None,
|
||||
cleaned_text: str | None = None,
|
||||
) -> None:
|
||||
self.查询(
|
||||
"INSERT INTO muse_clean_candidate "
|
||||
"(candidate_id,source_id,revision,removals,status,cleaned_hash,cleaned_text) "
|
||||
"VALUES (%s,%s,%s,%s,%s,%s,%s) "
|
||||
"ON CONFLICT (candidate_id) DO NOTHING",
|
||||
(
|
||||
candidate_id,
|
||||
source_id,
|
||||
revision,
|
||||
Jsonb(removals),
|
||||
status,
|
||||
cleaned_hash,
|
||||
cleaned_text,
|
||||
),
|
||||
)
|
||||
|
||||
def 读取清理候选(self, candidate_id: str) -> dict[str, Any] | None:
|
||||
行 = self.查询(
|
||||
"SELECT candidate_id,source_id,revision,removals,status,cleaned_hash,cleaned_text "
|
||||
"FROM muse_clean_candidate WHERE candidate_id=%s",
|
||||
(candidate_id,),
|
||||
).fetchone()
|
||||
return dict(行) if 行 else None
|
||||
|
||||
def 列出清理候选(self, source_id: str, revision: int) -> list[dict[str, Any]]:
|
||||
行集 = self.查询(
|
||||
"SELECT candidate_id,source_id,revision,removals,status,cleaned_hash,cleaned_text "
|
||||
"FROM muse_clean_candidate WHERE source_id=%s AND revision=%s "
|
||||
"ORDER BY created_at",
|
||||
(source_id, revision),
|
||||
).fetchall()
|
||||
return [dict(行) for 行 in 行集]
|
||||
|
||||
def 更新清理状态(
|
||||
self, candidate_id: str, status: str, cleaned_hash: str | None, cleaned_text: str | None
|
||||
) -> None:
|
||||
行 = self.查询(
|
||||
"UPDATE muse_clean_candidate SET status=%s,cleaned_hash=%s,cleaned_text=%s "
|
||||
"WHERE candidate_id=%s AND status='proposed' RETURNING candidate_id",
|
||||
(status, cleaned_hash, cleaned_text, candidate_id),
|
||||
).fetchone()
|
||||
if 行 is None:
|
||||
已有 = self.读取清理候选(candidate_id)
|
||||
if 已有 is None:
|
||||
raise 资料错误("清理候选不存在")
|
||||
raise 资料错误("清理候选已处置，不能改写")
|
||||
|
||||
# ---- 榜单 ----
|
||||
|
||||
def 登记榜单源(self, ranking_id: str, name: str, url: str, 方式: str) -> None:
|
||||
self.查询(
|
||||
"INSERT INTO muse_ranking_source (ranking_id,name,url,采集方式) "
|
||||
"VALUES (%s,%s,%s,%s) ON CONFLICT (ranking_id) DO NOTHING",
|
||||
(ranking_id, name, url, 方式),
|
||||
)
|
||||
|
||||
def 写快照(
|
||||
self,
|
||||
snapshot_id: str,
|
||||
ranking_id: str,
|
||||
status: str,
|
||||
items: list[dict] | None,
|
||||
failure_reason: str | None,
|
||||
) -> None:
|
||||
self.查询(
|
||||
"INSERT INTO muse_ranking_snapshot "
|
||||
"(snapshot_id,ranking_id,status,items,failure_reason) VALUES (%s,%s,%s,%s,%s)",
|
||||
(
|
||||
snapshot_id,
|
||||
ranking_id,
|
||||
status,
|
||||
Jsonb(items) if items is not None else None,
|
||||
failure_reason,
|
||||
),
|
||||
)
|
||||
|
||||
def 列出快照(self, ranking_id: str, 数量: int = 20) -> list[dict[str, Any]]:
|
||||
行集 = self.查询(
|
||||
"SELECT snapshot_id,ranking_id,captured_at,status,failure_reason,items "
|
||||
"FROM muse_ranking_snapshot WHERE ranking_id=%s "
|
||||
"ORDER BY captured_at DESC LIMIT %s",
|
||||
(ranking_id, 数量),
|
||||
).fetchall()
|
||||
return [dict(行) for 行 in 行集]
|
||||
80
src/muse/资料研究/接口.py
Normal file
80
src/muse/资料研究/接口.py
Normal file
@ -0,0 +1,80 @@
|
||||
"""资料研究公开用例：导入、证据、清理与榜单对相同入口得到一致结果。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from muse.资料研究.原文版本 import 读取证据
|
||||
from muse.资料研究.存储 import 资料存储
|
||||
from muse.资料研究.文本清理 import 应用清理候选, 拒绝清理候选, 生成清理候选
|
||||
from muse.资料研究.来源导入 import 导入来源, 核对来源授权
|
||||
from muse.资料研究.榜单快照 import 快照观察差异, 登记榜单, 采集榜单快照
|
||||
from muse.资料研究.模型 import (
|
||||
导入结果,
|
||||
导入请求,
|
||||
来源用途,
|
||||
来源种类,
|
||||
清理候选,
|
||||
资料错误,
|
||||
)
|
||||
|
||||
|
||||
class 资料服务:
|
||||
"""资料研究公开入口；连接由调用方事务提供，版本与快照只追加。"""
|
||||
|
||||
def 导入(self, 连, 身份作者: str, 请求: 导入请求) -> 导入结果:
|
||||
return 导入来源(连, 身份作者, 请求)
|
||||
|
||||
def 列出来源(self, 连) -> list[dict[str, Any]]:
|
||||
return 资料存储(连).列出来源()
|
||||
|
||||
def 读取版本(self, 连, source_id: str, revision: int | None = None) -> dict[str, Any]:
|
||||
行 = 资料存储(连).读版本(source_id, revision)
|
||||
if 行 is None:
|
||||
raise 资料错误("来源版本不存在")
|
||||
return 行
|
||||
|
||||
def 读取证据(self, 连, source_id: str, revision: int, 起点: int, 终点: int) -> dict[str, Any]:
|
||||
return 读取证据(连, source_id, revision, 起点, 终点)
|
||||
|
||||
def 核对来源授权(self, 连, source_id: str, 用途: str) -> dict[str, Any]:
|
||||
return 核对来源授权(连, source_id, 用途)
|
||||
|
||||
# ---- 清理 ----
|
||||
|
||||
def 生成清理候选(self, 连, source_id: str, revision: int) -> 清理候选:
|
||||
return 生成清理候选(连, source_id, revision)
|
||||
|
||||
def 应用清理候选(self, 连, candidate_id: str) -> 清理候选:
|
||||
return 应用清理候选(连, candidate_id)
|
||||
|
||||
def 拒绝清理候选(self, 连, candidate_id: str) -> None:
|
||||
拒绝清理候选(连, candidate_id)
|
||||
|
||||
def 列出清理候选(self, 连, source_id: str, revision: int) -> list[dict[str, Any]]:
|
||||
return 资料存储(连).列出清理候选(source_id, revision)
|
||||
|
||||
# ---- 榜单 ----
|
||||
|
||||
def 登记榜单(self, 连, ranking_id: str, name: str, url: str, 方式: str = "html") -> None:
|
||||
登记榜单(连, ranking_id, name, url, 方式)
|
||||
|
||||
def 采集榜单快照(self, 连, ranking_id: str, 读取器) -> dict[str, Any]:
|
||||
return 采集榜单快照(连, ranking_id, 读取器)
|
||||
|
||||
def 列出快照(self, 连, ranking_id: str, 数量: int = 20) -> list[dict[str, Any]]:
|
||||
return 资料存储(连).列出快照(ranking_id, 数量)
|
||||
|
||||
def 快照观察差异(self, 连, ranking_id: str) -> dict[str, Any]:
|
||||
return 快照观察差异(连, ranking_id)
|
||||
|
||||
|
||||
__all__ = [
|
||||
"资料服务",
|
||||
"资料错误",
|
||||
"导入请求",
|
||||
"导入结果",
|
||||
"清理候选",
|
||||
"来源种类",
|
||||
"来源用途",
|
||||
]
|
||||
107
src/muse/资料研究/文本清理.py
Normal file
107
src/muse/资料研究/文本清理.py
Normal file
@ -0,0 +1,107 @@
|
||||
"""文本清理：噪声删除候选与原文保护；删除项可定位、可回退。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import re
|
||||
from typing import Any
|
||||
from uuid import NAMESPACE_URL, uuid5
|
||||
|
||||
from muse.资料研究.存储 import 资料存储
|
||||
from muse.资料研究.模型 import 清理候选, 资料错误
|
||||
|
||||
# 常见导入噪声：广告水印、盗版声明、拉票与外链。逐条给理由，不做语义改写。
|
||||
_噪声模式 = (
|
||||
(re.compile(r"[^\n]*最新章节[^\n]*"), "更新提示水印"),
|
||||
(re.compile(r"[^\n]*(https?://|www\.)[^\n]*"), "外部链接广告"),
|
||||
(re.compile(r"[^\n]*首发[^\n]*"), "首发声明水印"),
|
||||
(re.compile(r"[^\n]*请收藏|求收藏|求推荐票[^\n]*"), "作者拉票"),
|
||||
(re.compile(r"[^\n]*本书由[^\n]*提供下载[^\n]*"), "盗版分发水印"),
|
||||
(re.compile(r"[^\n]*笔趣阁|天涯文学|起点中文网[^\n]*"), "站点水印"),
|
||||
)
|
||||
|
||||
|
||||
def 生成清理候选(连, source_id: str, revision: int) -> 清理候选:
|
||||
"""对指定原文版本生成噪声删除候选；原文不动，候选含码点区间与理由。"""
|
||||
|
||||
存储 = 资料存储(连)
|
||||
行 = 存储.读版本(source_id, revision)
|
||||
if 行 is None:
|
||||
raise 资料错误("来源版本不存在")
|
||||
内容: str = 行["content"]
|
||||
删除项: list[dict[str, Any]] = []
|
||||
占用: list[tuple[int, int]] = []
|
||||
for 模式, 理由 in _噪声模式:
|
||||
for 匹配 in 模式.finditer(内容):
|
||||
起, 止 = 匹配.span()
|
||||
起, 止 = (
|
||||
起 + (len(内容[起:止]) - len(内容[起:止].lstrip())),
|
||||
止 - (len(内容[起:止]) - len(内容[起:止].rstrip())),
|
||||
)
|
||||
if 起 >= 止 or any(not (止 <= a or 起 >= b) for a, b in 占用):
|
||||
continue
|
||||
占用.append((起, 止))
|
||||
删除项.append({"起点": 起, "终点": 止, "片段": 内容[起:止], "理由": 理由})
|
||||
删除项.sort(key=lambda 项: 项["起点"])
|
||||
原文指纹 = hashlib.sha256(内容.encode("utf-8")).hexdigest()
|
||||
代次 = len(存储.列出清理候选(source_id, revision))
|
||||
候选ID = str(
|
||||
uuid5(
|
||||
NAMESPACE_URL,
|
||||
"muse:B03:clean:" + _内容哈希([source_id, revision, 原文指纹, len(删除项), 代次]),
|
||||
)
|
||||
)
|
||||
存储.写清理候选(候选ID, source_id, revision, 删除项, "proposed")
|
||||
return 清理候选(候选ID, source_id, revision, tuple(删除项))
|
||||
|
||||
|
||||
def 应用清理候选(连, candidate_id: str) -> 清理候选:
|
||||
"""应用候选生成清理后对照文本；原文版本不受影响，状态只允许 proposed→applied。"""
|
||||
|
||||
存储 = 资料存储(连)
|
||||
行 = 存储.读取清理候选(candidate_id)
|
||||
if 行 is None:
|
||||
raise 资料错误("清理候选不存在")
|
||||
if 行["status"] != "proposed":
|
||||
raise 资料错误("清理候选已处置，不能重复应用")
|
||||
原文行 = 存储.读版本(行["source_id"], int(行["revision"]))
|
||||
if 原文行 is None:
|
||||
raise 资料错误("来源版本不存在，不能应用清理")
|
||||
内容: str = 原文行["content"]
|
||||
清理后 = 内容
|
||||
for 项 in sorted(行["removals"], key=lambda 项: 项["终点"], reverse=True):
|
||||
if 清理后[项["起点"] : 项["终点"]] != 项["片段"]:
|
||||
raise 资料错误("删除片段与当前原文不一致，不能应用")
|
||||
清理后 = 清理后[: 项["起点"]] + 清理后[项["终点"] :]
|
||||
清理哈希 = hashlib.sha256(清理后.encode("utf-8")).hexdigest()
|
||||
存储.更新清理状态(candidate_id, "applied", 清理哈希, 清理后)
|
||||
return 清理候选(
|
||||
candidate_id,
|
||||
行["source_id"],
|
||||
行["revision"],
|
||||
tuple(行["removals"]),
|
||||
"applied",
|
||||
清理哈希,
|
||||
清理后,
|
||||
)
|
||||
|
||||
|
||||
def 拒绝清理候选(连, candidate_id: str) -> None:
|
||||
存储 = 资料存储(连)
|
||||
行 = 存储.读取清理候选(candidate_id)
|
||||
if 行 is None:
|
||||
raise 资料错误("清理候选不存在")
|
||||
if 行["status"] != "proposed":
|
||||
raise 资料错误("清理候选已处置")
|
||||
存储.更新清理状态(candidate_id, "rejected", None, None)
|
||||
|
||||
|
||||
def _内容哈希(载荷: Any) -> str:
|
||||
import json
|
||||
|
||||
return hashlib.sha256(
|
||||
json.dumps(载荷, ensure_ascii=False, sort_keys=True).encode("utf-8")
|
||||
).hexdigest()
|
||||
|
||||
|
||||
__all__ = ["生成清理候选", "应用清理候选", "拒绝清理候选", "清理候选", "资料错误"]
|
||||
72
src/muse/资料研究/来源导入.py
Normal file
72
src/muse/资料研究/来源导入.py
Normal file
@ -0,0 +1,72 @@
|
||||
"""来源导入：外部资料保存为有出处的可重读输入；重复可辨认，授权只增不减。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
from typing import Any
|
||||
from uuid import NAMESPACE_URL, uuid5
|
||||
|
||||
from psycopg.rows import dict_row
|
||||
|
||||
from muse.资料研究.存储 import 资料存储
|
||||
from muse.资料研究.模型 import 导入结果, 导入请求, 资料错误
|
||||
|
||||
|
||||
def _内容哈希(内容: Any) -> str:
|
||||
if isinstance(内容, str):
|
||||
载荷 = 内容.encode("utf-8")
|
||||
else:
|
||||
载荷 = json.dumps(内容, ensure_ascii=False, sort_keys=True).encode("utf-8")
|
||||
return hashlib.sha256(载荷).hexdigest()
|
||||
|
||||
|
||||
def 导入来源(连, 身份作者: str, 请求: 导入请求) -> 导入结果:
|
||||
"""导入来源内容；同源同内容幂等返回既有版本，同源异内容追加新版本。"""
|
||||
|
||||
存储 = 资料存储(连)
|
||||
内容哈希 = _内容哈希(请求.content)
|
||||
既有源 = 存储.查源(请求.origin, 请求.kind)
|
||||
if 既有源 is None:
|
||||
source_id = str(
|
||||
uuid5(NAMESPACE_URL, "muse:B03:source:" + _内容哈希([请求.kind, 请求.origin]))
|
||||
)
|
||||
源 = 存储.建源(source_id, 请求.kind, 请求.title, 请求.origin, list(请求.authorized_uses))
|
||||
else:
|
||||
源 = 既有源
|
||||
# 授权用途只增不减：已授权用途不因再次导入被收回。
|
||||
合并 = sorted(set(源["authorized_uses"]) | set(请求.authorized_uses))
|
||||
if set(合并) != set(源["authorized_uses"]):
|
||||
源 = 存储.建源(源["source_id"], 请求.kind, 请求.title, 请求.origin, 合并)
|
||||
重复 = 存储.同内容版本(源["source_id"], 内容哈希)
|
||||
if 重复 is not None:
|
||||
return 导入结果(
|
||||
源["source_id"], int(重复["revision"]), 内容哈希, True, {"说明": "同源同内容重复导入"}
|
||||
)
|
||||
新版本 = 存储.当前版本(源["source_id"]) + 1
|
||||
结果 = {"导入者": 身份作者, "字节数": len(请求.content.encode("utf-8"))}
|
||||
存储.写版本(源["source_id"], 新版本, 内容哈希, 请求.content, 结果)
|
||||
return 导入结果(源["source_id"], 新版本, 内容哈希, False, 结果)
|
||||
|
||||
|
||||
def 核对来源授权(连, source_id: str, 用途: str) -> dict[str, Any]:
|
||||
"""消费前核对用途授权：外部资料不是本书事实或指令授权。"""
|
||||
|
||||
行 = (
|
||||
连.cursor(row_factory=dict_row)
|
||||
.execute(
|
||||
"SELECT source_id,kind,title,origin,authorized_uses "
|
||||
"FROM muse_source WHERE source_id=%s",
|
||||
(source_id,),
|
||||
)
|
||||
.fetchone()
|
||||
)
|
||||
if 行 is None:
|
||||
raise 资料错误("来源不存在")
|
||||
授权 = list(行["authorized_uses"])
|
||||
if 用途 not in 授权:
|
||||
raise 资料错误(f"来源未授权用途：{用途}")
|
||||
return {"source_id": 行["source_id"], "kind": 行["kind"], "authorized_uses": 授权}
|
||||
|
||||
|
||||
__all__ = ["导入来源", "核对来源授权", "资料错误"]
|
||||
91
src/muse/资料研究/榜单快照.py
Normal file
91
src/muse/资料研究/榜单快照.py
Normal file
@ -0,0 +1,91 @@
|
||||
"""榜单快照：把时点观察与效果推断分开；抓取失败如实保留失败态。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
from datetime import UTC, datetime
|
||||
from typing import Any, Protocol
|
||||
from uuid import uuid4
|
||||
|
||||
from psycopg.rows import dict_row
|
||||
|
||||
from muse.资料研究.存储 import 资料存储
|
||||
from muse.资料研究.模型 import 榜单读取结果, 资料错误
|
||||
|
||||
|
||||
class 榜单读取器(Protocol):
|
||||
"""具名榜单读取协议；真实网络实现必须显式标记，测试注入合成实现。"""
|
||||
|
||||
def 抓取(self, url: str, 方式: str) -> 榜单读取结果: ...
|
||||
|
||||
|
||||
def 登记榜单(连, ranking_id: str, name: str, url: str, 方式: str = "html") -> None:
|
||||
资料存储(连).登记榜单源(ranking_id, name, url, 方式)
|
||||
|
||||
|
||||
def 采集榜单快照(连, ranking_id: str, 读取器: 榜单读取器) -> dict[str, Any]:
|
||||
"""按具名榜种采集一次时点快照；失败保留失败态，不用任何补造数据。"""
|
||||
|
||||
存储 = 资料存储(连)
|
||||
行 = (
|
||||
连.cursor(row_factory=dict_row)
|
||||
.execute(
|
||||
"SELECT ranking_id,name,url,采集方式 FROM muse_ranking_source WHERE ranking_id=%s",
|
||||
(ranking_id,),
|
||||
)
|
||||
.fetchone()
|
||||
)
|
||||
if 行 is None:
|
||||
raise 资料错误("榜单未登记")
|
||||
结果: 榜单读取结果 = 读取器.抓取(行["url"], 行["采集方式"])
|
||||
snapshot_id = str(uuid4())
|
||||
if not 结果.成功:
|
||||
存储.写快照(snapshot_id, ranking_id, "failed", None, 结果.失败原因)
|
||||
return {
|
||||
"snapshot_id": snapshot_id,
|
||||
"ranking_id": ranking_id,
|
||||
"status": "failed",
|
||||
"failure_reason": 结果.失败原因,
|
||||
}
|
||||
时刻 = datetime.now(UTC)
|
||||
载荷 = json.dumps(
|
||||
{"ranking_id": ranking_id, "captured_at": 时刻.isoformat(), "items": 结果.items},
|
||||
ensure_ascii=False,
|
||||
sort_keys=True,
|
||||
)
|
||||
存储.写快照(snapshot_id, ranking_id, "ok", list(结果.items), None)
|
||||
return {
|
||||
"snapshot_id": snapshot_id,
|
||||
"ranking_id": ranking_id,
|
||||
"status": "ok",
|
||||
"captured_at": 时刻.isoformat() + f"+{hashlib.sha256(载荷.encode()).hexdigest()[:8]}",
|
||||
"条目数": len(结果.items),
|
||||
}
|
||||
|
||||
|
||||
def 快照观察差异(连, ranking_id: str) -> dict[str, Any]:
|
||||
"""比较最近两次成功快照的名次差异；这是观察陈述，不推断任何效果因果。"""
|
||||
|
||||
行集 = 资料存储(连).列出快照(ranking_id, 数量=10)
|
||||
成功 = [行 for 行 in 行集 if 行["status"] == "ok"]
|
||||
if len(成功) < 2:
|
||||
raise 资料错误("成功快照不足两次，无法比较观察差异")
|
||||
最新, 上次 = 成功[0]["items"], 成功[1]["items"]
|
||||
上次名次 = {条目.get("作品") or 条目.get("name"): 序 for 序, 条目 in enumerate(上次, start=1)}
|
||||
差异 = []
|
||||
for 序, 条目 in enumerate(最新, start=1):
|
||||
名 = 条目.get("作品") or 条目.get("name")
|
||||
旧序 = 上次名次.get(名)
|
||||
差异.append(
|
||||
{
|
||||
"作品": 名,
|
||||
"本次名次": 序,
|
||||
"上次名次": 旧序,
|
||||
"变化": None if 旧序 is None else 旧序 - 序,
|
||||
}
|
||||
)
|
||||
return {"ranking_id": ranking_id, "最新快照": 成功[0]["snapshot_id"], "差异": 差异}
|
||||
|
||||
|
||||
__all__ = ["榜单读取器", "登记榜单", "采集榜单快照", "快照观察差异", "资料错误"]
|
||||
90
src/muse/资料研究/模型.py
Normal file
90
src/muse/资料研究/模型.py
Normal file
@ -0,0 +1,90 @@
|
||||
"""资料研究稳定对象：来源、版本、清理候选与榜单快照；外部资料不是事实授权。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, Literal
|
||||
|
||||
from muse.共享.错误 import Muse错误
|
||||
|
||||
来源种类 = ("author_draft", "reference", "web", "note", "inspiration")
|
||||
来源用途 = ("analysis", "reference_reading", "sample", "ranking_observation")
|
||||
|
||||
|
||||
class 资料错误(Muse错误):
|
||||
错误码 = "SOURCE_INVALID"
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class 导入请求:
|
||||
"""一次来源导入：身份、内容、用途授权与展示信息。"""
|
||||
|
||||
kind: str
|
||||
title: str
|
||||
origin: str
|
||||
content: str
|
||||
authorized_uses: tuple[str, ...] = ()
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
if self.kind not in 来源种类:
|
||||
raise 资料错误(f"未知来源种类：{self.kind}")
|
||||
if not self.title.strip() or not self.origin.strip():
|
||||
raise 资料错误("导入需要标题与来源位置")
|
||||
if not self.content:
|
||||
raise 资料错误("导入内容为空")
|
||||
未知 = set(self.authorized_uses) - set(来源用途)
|
||||
if 未知:
|
||||
raise 资料错误(f"未授权用途不合法：{sorted(未知)}")
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class 导入结果:
|
||||
source_id: str
|
||||
revision: int
|
||||
content_hash: str
|
||||
重复: bool
|
||||
import_result: dict[str, Any] = field(default_factory=dict)
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class 清理候选:
|
||||
"""噪声删除候选：只提出位置与理由，不碰原文。"""
|
||||
|
||||
candidate_id: str
|
||||
source_id: str
|
||||
revision: int
|
||||
removals: tuple[dict[str, Any], ...]
|
||||
status: Literal["proposed", "applied", "rejected"] = "proposed"
|
||||
cleaned_hash: str | None = None
|
||||
cleaned_text: str | None = None
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
for 项 in self.removals:
|
||||
if not {"起点", "终点", "片段", "理由"} <= set(项):
|
||||
raise 资料错误("清理候选删除项需要码点区间、片段与理由")
|
||||
if not (0 <= 项["起点"] < 项["终点"]):
|
||||
raise 资料错误("删除区间必须为 0 ≤ 起点 < 终点")
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class 榜单读取结果:
|
||||
"""读取器的原始产出；失败必须如实携带原因。"""
|
||||
|
||||
成功: bool
|
||||
items: tuple[dict[str, Any], ...] = ()
|
||||
失败原因: str | None = None
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
if self.成功 == (self.失败原因 is not None):
|
||||
raise 资料错误("榜单读取结果的成功与失败原因必须互斥")
|
||||
|
||||
|
||||
__all__ = [
|
||||
"资料错误",
|
||||
"来源种类",
|
||||
"来源用途",
|
||||
"导入请求",
|
||||
"导入结果",
|
||||
"清理候选",
|
||||
"榜单读取结果",
|
||||
]
|
||||
85
tests/集成/test_榜单与清理.py
Normal file
85
tests/集成/test_榜单与清理.py
Normal file
@ -0,0 +1,85 @@
|
||||
"""榜单快照：具名采集、失败保留失败态、观察差异不推断因果。"""
|
||||
|
||||
import pytest
|
||||
|
||||
from muse.资料研究.接口 import 资料服务
|
||||
from muse.资料研究.模型 import 榜单读取结果, 资料错误
|
||||
|
||||
pytestmark = pytest.mark.数据库
|
||||
|
||||
|
||||
class 合成榜单读取器:
|
||||
"""按剧本返回的合成读取器；不访问网络。"""
|
||||
|
||||
def __init__(self, 剧本: list[榜单读取结果]):
|
||||
self.剧本 = list(剧本)
|
||||
|
||||
def 抓取(self, url: str, 方式: str) -> 榜单读取结果:
|
||||
return self.剧本.pop(0)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def 服务():
|
||||
return 资料服务()
|
||||
|
||||
|
||||
def test_采集成功失败与观察差异__17b01e(服务, 应用测试库):
|
||||
from muse.共享.调用身份 import 用途
|
||||
|
||||
库 = 应用测试库[用途.生产]
|
||||
with 库.连接() as 连, 连.transaction():
|
||||
服务.登记榜单(连, "rank-qd", "起点月票榜", "https://example.com/rank", "html")
|
||||
# 第一次：失败保留失败态，不伪造数据。
|
||||
失败 = 服务.采集榜单快照(
|
||||
连, "rank-qd", 合成榜单读取器([榜单读取结果(成功=False, 失败原因="上游 503")])
|
||||
)
|
||||
assert 失败["status"] == "failed" and 失败["failure_reason"] == "上游 503"
|
||||
快照集 = 服务.列出快照(连, "rank-qd")
|
||||
assert 快照集[0]["status"] == "failed" and 快照集[0]["items"] is None
|
||||
# 第二、三次：成功快照带时刻与条目。
|
||||
第一 = 服务.采集榜单快照(
|
||||
连,
|
||||
"rank-qd",
|
||||
合成榜单读取器(
|
||||
[
|
||||
榜单读取结果(
|
||||
成功=True,
|
||||
items=(
|
||||
{"作品": "甲", "名次": 1},
|
||||
{"作品": "乙", "名次": 2},
|
||||
),
|
||||
)
|
||||
]
|
||||
),
|
||||
)
|
||||
第二 = 服务.采集榜单快照(
|
||||
连,
|
||||
"rank-qd",
|
||||
合成榜单读取器(
|
||||
[
|
||||
榜单读取结果(
|
||||
成功=True,
|
||||
items=(
|
||||
{"作品": "乙", "名次": 1},
|
||||
{"作品": "丙", "名次": 2},
|
||||
{"作品": "甲", "名次": 3},
|
||||
),
|
||||
)
|
||||
]
|
||||
),
|
||||
)
|
||||
assert 第一["status"] == "ok" and 第二["条目数"] == 3
|
||||
差异 = 服务.快照观察差异(连, "rank-qd")
|
||||
变化 = {d["作品"]: d["变化"] for d in 差异["差异"]}
|
||||
assert 变化["乙"] == 1 and 变化["甲"] == -2 and 变化["丙"] is None
|
||||
# 观察差异不携带任何 run/效果归因字段。
|
||||
assert all("run" not in str(键).lower() for 键 in 差异)
|
||||
|
||||
|
||||
def test_未登记榜单拒绝采集__17b02f(服务, 应用测试库):
|
||||
from muse.共享.调用身份 import 用途
|
||||
|
||||
库 = 应用测试库[用途.生产]
|
||||
with 库.连接() as 连:
|
||||
with pytest.raises(资料错误, match="榜单未登记"):
|
||||
服务.采集榜单快照(连, "rank-none", 合成榜单读取器([]))
|
||||
102
tests/集成/test_资料导入与版本.py
Normal file
102
tests/集成/test_资料导入与版本.py
Normal file
@ -0,0 +1,102 @@
|
||||
"""资料导入与原文版本：幂等去重、版本链、证据校验、用途授权与清理对照。"""
|
||||
|
||||
import pytest
|
||||
|
||||
from muse.共享.调用身份 import 用途
|
||||
from muse.资料研究.接口 import 资料服务
|
||||
from muse.资料研究.模型 import 导入请求, 资料错误
|
||||
|
||||
pytestmark = pytest.mark.数据库
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def 服务(应用测试库):
|
||||
return 资料服务()
|
||||
|
||||
|
||||
def test_导入去重与版本链__17a01a(服务, 应用测试库):
|
||||
库 = 应用测试库[用途.生产]
|
||||
请求 = 导入请求(
|
||||
kind="reference",
|
||||
title="封神演义",
|
||||
origin="file:///资料/封神演义.txt",
|
||||
content="第一回 纣王女娲宫进香。\n\n最新章节请访问 example.com\n",
|
||||
authorized_uses=("analysis",),
|
||||
)
|
||||
with 库.连接() as 连, 连.transaction():
|
||||
首次 = 服务.导入(连, "importer", 请求)
|
||||
assert 首次.重复 is False and 首次.revision == 1
|
||||
# 同源同内容：幂等返回既有版本，重复可辨认。
|
||||
再次 = 服务.导入(连, "importer", 请求)
|
||||
assert 再次.重复 is True and 再次.revision == 1 and 再次.source_id == 首次.source_id
|
||||
# 同源异内容：追加新版本，旧版本仍可回读。
|
||||
新版 = 服务.导入(
|
||||
连,
|
||||
"importer",
|
||||
导入请求(
|
||||
kind="reference",
|
||||
title="封神演义（校对版）",
|
||||
origin=请求.origin,
|
||||
content="第一回 纣王女娲宫进香。（校对）\n",
|
||||
authorized_uses=("analysis", "reference_reading"),
|
||||
),
|
||||
)
|
||||
assert 新版.重复 is False and 新版.revision == 2 and 新版.source_id == 首次.source_id
|
||||
版本 = 服务.读取版本(连, 首次.source_id, 1)
|
||||
assert "最新章节" in 版本["content"]
|
||||
# 授权用途只增不减。
|
||||
源 = 服务.核对来源授权(连, 首次.source_id, "reference_reading")
|
||||
assert 源["authorized_uses"] == ["analysis", "reference_reading"]
|
||||
|
||||
|
||||
def test_证据读取与哈希校验__17a02b(服务, 应用测试库):
|
||||
库 = 应用测试库[用途.生产]
|
||||
文本 = "林深把伞收在了门边。雨还没停。"
|
||||
with 库.连接() as 连, 连.transaction():
|
||||
结果 = 服务.导入(
|
||||
连,
|
||||
"importer",
|
||||
导入请求("note", "码头笔记", "note://local/1", 文本, ("analysis",)),
|
||||
)
|
||||
证据 = 服务.读取证据(连, 结果.source_id, 结果.revision, 0, 10)
|
||||
assert 证据["片段"] == "林深把伞收在了门边。"
|
||||
with pytest.raises(资料错误, match="区间越界"):
|
||||
服务.读取证据(连, 结果.source_id, 结果.revision, 0, 999)
|
||||
|
||||
|
||||
def test_用途未授权拒绝消费__17a03c(服务, 应用测试库):
|
||||
库 = 应用测试库[用途.生产]
|
||||
with 库.连接() as 连, 连.transaction():
|
||||
结果 = 服务.导入(
|
||||
连,
|
||||
"importer",
|
||||
导入请求("web", "某网页", "https://example.com/a", "网页内容", ("reference_reading",)),
|
||||
)
|
||||
with pytest.raises(资料错误, match="未授权用途"):
|
||||
服务.核对来源授权(连, 结果.source_id, "analysis")
|
||||
|
||||
|
||||
def test_清理候选对照且原文不变__17a04d(服务, 应用测试库):
|
||||
库 = 应用测试库[用途.生产]
|
||||
原文 = "第一回 纣王女娲宫进香。\n最新章节请访问 example.com\n本章完。"
|
||||
with 库.连接() as 连, 连.transaction():
|
||||
结果 = 服务.导入(
|
||||
连,
|
||||
"importer",
|
||||
导入请求("reference", "样章", "file:///样章.txt", 原文, ("analysis",)),
|
||||
)
|
||||
候选 = 服务.生成清理候选(连, 结果.source_id, 结果.revision)
|
||||
assert 候选.removals and all(项["理由"] for 项 in 候选.removals)
|
||||
应用后 = 服务.应用清理候选(连, 候选.candidate_id)
|
||||
assert "最新章节" not in 应用后.cleaned_text
|
||||
assert "第一回" in 应用后.cleaned_text
|
||||
# 原文版本不受清理影响。
|
||||
版本 = 服务.读取版本(连, 结果.source_id, 结果.revision)
|
||||
assert 版本["content"] == 原文
|
||||
# 重复应用与处置不可改写。
|
||||
with pytest.raises(资料错误, match="已处置"):
|
||||
服务.应用清理候选(连, 候选.candidate_id)
|
||||
新候选 = 服务.生成清理候选(连, 结果.source_id, 结果.revision)
|
||||
服务.拒绝清理候选(连, 新候选.candidate_id)
|
||||
with pytest.raises(资料错误, match="已处置"):
|
||||
服务.应用清理候选(连, 新候选.candidate_id)
|
||||
62
web/src/功能/资料研究/数据.ts
Normal file
62
web/src/功能/资料研究/数据.ts
Normal file
@ -0,0 +1,62 @@
|
||||
/** 资料研究数据接入：来源列表、版本原文、清理候选与榜单快照。 */
|
||||
import { useQuery } from "@tanstack/react-query";
|
||||
import { 客户端 } from "../../接口/生成/客户端";
|
||||
import { 查询键 } from "../../接口/查询键";
|
||||
export interface 来源行 {
|
||||
source_id: string;
|
||||
kind: string;
|
||||
title: string;
|
||||
origin: string;
|
||||
authorized_uses: string[];
|
||||
最新版本: number;
|
||||
最近导入: string | null;
|
||||
}
|
||||
|
||||
export interface 版本行 {
|
||||
source_id: string;
|
||||
revision: number;
|
||||
content_hash: string;
|
||||
content: string;
|
||||
}
|
||||
|
||||
export interface 快照行 {
|
||||
snapshot_id: string;
|
||||
status: string;
|
||||
failure_reason: string | null;
|
||||
items: { 作品?: string; name?: string; [k: string]: unknown }[] | null;
|
||||
captured_at: string;
|
||||
}
|
||||
|
||||
export function useMuse来源列表(scope: string | null) {
|
||||
return useQuery({
|
||||
queryKey: 查询键(scope, "sources"),
|
||||
queryFn: async ({ signal }) =>
|
||||
(await 客户端.GET("/api/v1/sources", { signal, params: {} })).data as unknown as 来源行[],
|
||||
});
|
||||
}
|
||||
|
||||
export function useMuse版本原文(scope: string | null, sourceId: string | null, revision: number | null) {
|
||||
return useQuery({
|
||||
queryKey: 查询键(scope, sourceId ?? "", "version", String(revision ?? 0)),
|
||||
enabled: !!sourceId && !!revision,
|
||||
queryFn: async ({ signal }) =>
|
||||
((await 客户端.GET("/api/v1/sources/{source_id}/versions/{revision}", {
|
||||
signal,
|
||||
params: { path: { source_id: sourceId!, revision: revision! } },
|
||||
})).data ?? null) as 版本行 | null,
|
||||
});
|
||||
}
|
||||
|
||||
export function useMuse快照列表(scope: string | null, rankingId: string | null) {
|
||||
return useQuery({
|
||||
queryKey: 查询键(scope, rankingId ?? "", "snapshots"),
|
||||
enabled: !!rankingId,
|
||||
queryFn: async ({ signal }) =>
|
||||
(await 客户端.GET("/api/v1/rankings/{ranking_id}/snapshots", {
|
||||
signal,
|
||||
params: { path: { ranking_id: rankingId! } },
|
||||
})).data as unknown as 快照行[],
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
148
web/src/功能/资料研究/资料对照页.tsx
Normal file
148
web/src/功能/资料研究/资料对照页.tsx
Normal file
@ -0,0 +1,148 @@
|
||||
/** 资料对照页：来源版本原文、清理候选对照与榜单快照（含失败态）。 */
|
||||
import { useState } from "react";
|
||||
import { useMuse来源列表, useMuse版本原文, useMuse快照列表 } from "./数据";
|
||||
import { Ui反馈状态 } from "../../界面/反馈状态";
|
||||
import { useMuse作者会话 } from "../../应用/作者会话";
|
||||
|
||||
export function Ui资料对照页() {
|
||||
const 会话 = useMuse作者会话();
|
||||
const scope = 会话.data ? "session" : null;
|
||||
const 来源 = useMuse来源列表(scope);
|
||||
const [当前, set当前] = useState<{ id: string; revision: number } | null>(null);
|
||||
const [ranking, setRanking] = useState<string | null>(null);
|
||||
const 版本 = useMuse版本原文(scope, 当前?.id ?? null, 当前?.revision ?? null);
|
||||
const 快照 = useMuse快照列表(scope, ranking);
|
||||
const [消息, set消息] = useState("");
|
||||
|
||||
async function 生成并应用(sourceId: string, revision: number) {
|
||||
try {
|
||||
const { data, error } = await (await import("../../接口/生成/客户端")).客户端.POST(
|
||||
"/api/v1/sources/{source_id}/versions/{revision}/clean-candidates",
|
||||
{ params: { path: { source_id: sourceId, revision } } },
|
||||
);
|
||||
if (!data) throw new Error((error as { message?: string })?.message ?? "生成失败");
|
||||
const apply = await (await import("../../接口/生成/客户端")).客户端.POST(
|
||||
"/api/v1/clean-candidates/{candidate_id}/apply",
|
||||
{ params: { path: { candidate_id: (data as { candidate_id: string }).candidate_id } } },
|
||||
);
|
||||
set消息(
|
||||
apply.data
|
||||
? "清理已应用为对照版本；原文保持不变。"
|
||||
: "清理候选已生成，但应用失败，可重试。",
|
||||
);
|
||||
set当前({ id: sourceId, revision });
|
||||
} catch (e) {
|
||||
set消息(e instanceof Error ? e.message : "清理流程失败，可重试。");
|
||||
}
|
||||
}
|
||||
|
||||
return (
|
||||
<main className="content">
|
||||
<p className="eyebrow">工作台</p>
|
||||
<h1>资料研究</h1>
|
||||
<p className="muted">
|
||||
来源与原文对照；清理只生成对照结果，原文版本不可改写。榜单快照是时点观察，失败如实显示。
|
||||
</p>
|
||||
<section className="paper-panel" aria-label="来源清单">
|
||||
<div className="panel-heading">
|
||||
<h2>来源清单</h2>
|
||||
<span className="muted">{来源.data?.length ?? 0} 个来源</span>
|
||||
</div>
|
||||
{来源.isPending ? (
|
||||
<Ui反馈状态 message="正在读取来源…" />
|
||||
) : (来源.data?.length ?? 0) === 0 ? (
|
||||
<p className="muted">还没有导入来源；经 CLI 或主会话导入后这里可见。</p>
|
||||
) : (
|
||||
<ul className="event-list">
|
||||
{来源.data!.map((源) => (
|
||||
<li key={源.source_id}>
|
||||
<button
|
||||
className="text-button"
|
||||
onClick={() => set当前({ id: 源.source_id, revision: 源.最新版本 })}
|
||||
>
|
||||
{源.title}（{源.kind}）r{源.最新版本}
|
||||
</button>
|
||||
<span className="mono muted">{源.origin}</span>
|
||||
</li>
|
||||
))}
|
||||
</ul>
|
||||
)}
|
||||
{来源.error && <Ui反馈状态 message={来源.error.message} error />}
|
||||
</section>
|
||||
|
||||
{当前 && (
|
||||
<section className="paper-panel" aria-label="原文与清理对照">
|
||||
<div className="panel-heading">
|
||||
<h2>原文与清理对照</h2>
|
||||
<span className="muted">
|
||||
r{当前.revision}
|
||||
</span>
|
||||
</div>
|
||||
{版本.isPending ? (
|
||||
<Ui反馈状态 message="正在读取原文…" />
|
||||
) : 版本.error ? (
|
||||
<Ui反馈状态 message="该版本读取失败。" error />
|
||||
) : (
|
||||
<>
|
||||
<pre className="mono draft-stream" aria-label="版本原文">
|
||||
{版本.data?.content}
|
||||
</pre>
|
||||
<button
|
||||
className="primary"
|
||||
onClick={() => void 生成并应用(当前.id, 当前.revision)}
|
||||
>
|
||||
生成并应用清理对照
|
||||
</button>
|
||||
{消息 && (
|
||||
<p role="status">{消息}</p>
|
||||
)}
|
||||
</>
|
||||
)}
|
||||
</section>
|
||||
)}
|
||||
|
||||
<section className="paper-panel" aria-label="榜单快照">
|
||||
<div className="panel-heading">
|
||||
<h2>榜单快照</h2>
|
||||
<span className="muted">时点观察，不推断效果因果</span>
|
||||
</div>
|
||||
<form
|
||||
className="structure-form"
|
||||
onSubmit={(e) => {
|
||||
e.preventDefault();
|
||||
const 值 = new FormData(e.currentTarget).get("ranking_id");
|
||||
setRanking(typeof 值 === "string" && 值 ? 值 : null);
|
||||
}}
|
||||
>
|
||||
<label>
|
||||
榜单编号
|
||||
<input name="ranking_id" placeholder="例如 rank-qd" required />
|
||||
</label>
|
||||
<button type="submit">查看快照</button>
|
||||
</form>
|
||||
{ranking &&
|
||||
(快照.isPending ? (
|
||||
<Ui反馈状态 message="正在读取快照…" />
|
||||
) : (快照.data?.length ?? 0) === 0 ? (
|
||||
<p className="muted">该榜单还没有快照。</p>
|
||||
) : (
|
||||
<ol className="event-list">
|
||||
{快照.data!.map((条) => (
|
||||
<li key={条.snapshot_id}>
|
||||
{条.status === "failed" ? (
|
||||
<span>
|
||||
采集失败：{条.failure_reason ?? "原因未记录"}
|
||||
</span>
|
||||
) : (
|
||||
<span>
|
||||
{(条.items ?? []).length} 条 · {条.captured_at}
|
||||
</span>
|
||||
)}
|
||||
</li>
|
||||
))}
|
||||
</ol>
|
||||
))}
|
||||
</section>
|
||||
</main>
|
||||
);
|
||||
}
|
||||
@ -37,8 +37,11 @@ export function Ui工作台布局({ children }: PropsWithChildren) {
|
||||
<NavLink to="/tasks">
|
||||
任务记录<span>02</span>
|
||||
</NavLink>
|
||||
<NavLink to="/sources">
|
||||
资料研究<span>03</span>
|
||||
</NavLink>
|
||||
<NavLink to="/system">
|
||||
连接状态<span>03</span>
|
||||
连接状态<span>04</span>
|
||||
</NavLink>
|
||||
</nav>
|
||||
<div className="sidebar-note">
|
||||
|
||||
@ -8,6 +8,7 @@ import {
|
||||
Ui规划工作区,
|
||||
} from "../功能/作品规划";
|
||||
import { Ui候选审阅页, Ui正文页 } from "../功能/正文写作";
|
||||
import { Ui资料对照页 } from "../功能/资料研究/资料对照页";
|
||||
import { 客户端 } from "../接口/生成/客户端";
|
||||
|
||||
export function Ui页面路由() {
|
||||
@ -23,6 +24,7 @@ export function Ui页面路由() {
|
||||
element={<Ui候选审阅页 />}
|
||||
/>
|
||||
<Route path="/tasks" element={<Ui任务列表 />} />
|
||||
<Route path="/sources" element={<Ui资料对照页 />} />
|
||||
<Route path="/system" element={<Ui连接状态 />} />
|
||||
<Route path="*" element={<Navigate to="/works" replace />} />
|
||||
</Routes>
|
||||
|
||||
@ -724,6 +724,144 @@ export interface paths {
|
||||
patch?: never;
|
||||
trace?: never;
|
||||
};
|
||||
"/api/v1/sources/import": {
|
||||
parameters: {
|
||||
query?: never;
|
||||
header?: never;
|
||||
path?: never;
|
||||
cookie?: never;
|
||||
};
|
||||
get?: never;
|
||||
put?: never;
|
||||
/** 导入来源 */
|
||||
post: operations["import_source"];
|
||||
delete?: never;
|
||||
options?: never;
|
||||
head?: never;
|
||||
patch?: never;
|
||||
trace?: never;
|
||||
};
|
||||
"/api/v1/sources": {
|
||||
parameters: {
|
||||
query?: never;
|
||||
header?: never;
|
||||
path?: never;
|
||||
cookie?: never;
|
||||
};
|
||||
/** 列出来源 */
|
||||
get: operations["list_sources"];
|
||||
put?: never;
|
||||
post?: never;
|
||||
delete?: never;
|
||||
options?: never;
|
||||
head?: never;
|
||||
patch?: never;
|
||||
trace?: never;
|
||||
};
|
||||
"/api/v1/sources/{source_id}/versions/{revision}": {
|
||||
parameters: {
|
||||
query?: never;
|
||||
header?: never;
|
||||
path?: never;
|
||||
cookie?: never;
|
||||
};
|
||||
/** 读取版本 */
|
||||
get: operations["read_source_version"];
|
||||
put?: never;
|
||||
post?: never;
|
||||
delete?: never;
|
||||
options?: never;
|
||||
head?: never;
|
||||
patch?: never;
|
||||
trace?: never;
|
||||
};
|
||||
"/api/v1/sources/{source_id}/versions/{revision}/clean-candidates": {
|
||||
parameters: {
|
||||
query?: never;
|
||||
header?: never;
|
||||
path?: never;
|
||||
cookie?: never;
|
||||
};
|
||||
/** 列出清理候选 */
|
||||
get: operations["list_clean_candidates"];
|
||||
put?: never;
|
||||
/** 生成清理候选 */
|
||||
post: operations["create_clean_candidate"];
|
||||
delete?: never;
|
||||
options?: never;
|
||||
head?: never;
|
||||
patch?: never;
|
||||
trace?: never;
|
||||
};
|
||||
"/api/v1/clean-candidates/{candidate_id}/apply": {
|
||||
parameters: {
|
||||
query?: never;
|
||||
header?: never;
|
||||
path?: never;
|
||||
cookie?: never;
|
||||
};
|
||||
get?: never;
|
||||
put?: never;
|
||||
/** 应用清理候选 */
|
||||
post: operations["apply_clean_candidate"];
|
||||
delete?: never;
|
||||
options?: never;
|
||||
head?: never;
|
||||
patch?: never;
|
||||
trace?: never;
|
||||
};
|
||||
"/api/v1/clean-candidates/{candidate_id}/reject": {
|
||||
parameters: {
|
||||
query?: never;
|
||||
header?: never;
|
||||
path?: never;
|
||||
cookie?: never;
|
||||
};
|
||||
get?: never;
|
||||
put?: never;
|
||||
/** 拒绝清理候选 */
|
||||
post: operations["reject_clean_candidate"];
|
||||
delete?: never;
|
||||
options?: never;
|
||||
head?: never;
|
||||
patch?: never;
|
||||
trace?: never;
|
||||
};
|
||||
"/api/v1/rankings": {
|
||||
parameters: {
|
||||
query?: never;
|
||||
header?: never;
|
||||
path?: never;
|
||||
cookie?: never;
|
||||
};
|
||||
get?: never;
|
||||
put?: never;
|
||||
/** 登记榜单 */
|
||||
post: operations["register_ranking"];
|
||||
delete?: never;
|
||||
options?: never;
|
||||
head?: never;
|
||||
patch?: never;
|
||||
trace?: never;
|
||||
};
|
||||
"/api/v1/rankings/{ranking_id}/snapshots": {
|
||||
parameters: {
|
||||
query?: never;
|
||||
header?: never;
|
||||
path?: never;
|
||||
cookie?: never;
|
||||
};
|
||||
/** 列出快照 */
|
||||
get: operations["list_ranking_snapshots"];
|
||||
put?: never;
|
||||
/** 采集快照 */
|
||||
post: operations["capture_ranking_snapshot"];
|
||||
delete?: never;
|
||||
options?: never;
|
||||
head?: never;
|
||||
patch?: never;
|
||||
trace?: never;
|
||||
};
|
||||
"/api/v1/chapters/{chapter_id}/writing-candidates": {
|
||||
parameters: {
|
||||
query?: never;
|
||||
@ -2550,15 +2688,48 @@ export interface components {
|
||||
[key: string]: number;
|
||||
} | null;
|
||||
};
|
||||
/** 发起生成回执 */
|
||||
/** 导入来源请求 */
|
||||
muse______http____________________6: {
|
||||
/**
|
||||
* Kind
|
||||
* @enum {string}
|
||||
*/
|
||||
kind: "author_draft" | "reference" | "web" | "note" | "inspiration";
|
||||
/** Title */
|
||||
title: string;
|
||||
/** Origin */
|
||||
origin: string;
|
||||
/** Content */
|
||||
content: string;
|
||||
/**
|
||||
* Authorized Uses
|
||||
* @default []
|
||||
*/
|
||||
authorized_uses: string[];
|
||||
};
|
||||
/** 登记榜单请求 */
|
||||
muse______http____________________7: {
|
||||
/** Ranking Id */
|
||||
ranking_id: string;
|
||||
/** Name */
|
||||
name: string;
|
||||
/** Url */
|
||||
url: string;
|
||||
/**
|
||||
* Method
|
||||
* @default html
|
||||
*/
|
||||
method: string;
|
||||
};
|
||||
/** 发起生成回执 */
|
||||
muse______http____________________8: {
|
||||
/** Task Id */
|
||||
task_id: string;
|
||||
/** Writing Task Id */
|
||||
writing_task_id: string;
|
||||
};
|
||||
/** 审校报告呈现 */
|
||||
muse______http____________________7: {
|
||||
muse______http____________________9: {
|
||||
/** Report Id */
|
||||
report_id: string;
|
||||
/** Candidate Id */
|
||||
@ -4145,7 +4316,7 @@ export interface operations {
|
||||
[name: string]: unknown;
|
||||
};
|
||||
content: {
|
||||
"application/json": components["schemas"]["muse______http____________________6"];
|
||||
"application/json": components["schemas"]["muse______http____________________8"];
|
||||
};
|
||||
};
|
||||
/** @description Validation Error */
|
||||
@ -4178,7 +4349,335 @@ export interface operations {
|
||||
[name: string]: unknown;
|
||||
};
|
||||
content: {
|
||||
"application/json": components["schemas"]["muse______http____________________7"];
|
||||
"application/json": components["schemas"]["muse______http____________________9"];
|
||||
};
|
||||
};
|
||||
/** @description Validation Error */
|
||||
422: {
|
||||
headers: {
|
||||
[name: string]: unknown;
|
||||
};
|
||||
content: {
|
||||
"application/json": components["schemas"]["HTTPValidationError"];
|
||||
};
|
||||
};
|
||||
};
|
||||
};
|
||||
import_source: {
|
||||
parameters: {
|
||||
query: {
|
||||
work_id: string;
|
||||
};
|
||||
header?: never;
|
||||
path?: never;
|
||||
cookie?: never;
|
||||
};
|
||||
requestBody: {
|
||||
content: {
|
||||
"application/json": components["schemas"]["muse______http____________________6"];
|
||||
};
|
||||
};
|
||||
responses: {
|
||||
/** @description Successful Response */
|
||||
201: {
|
||||
headers: {
|
||||
[name: string]: unknown;
|
||||
};
|
||||
content: {
|
||||
"application/json": {
|
||||
[key: string]: unknown;
|
||||
};
|
||||
};
|
||||
};
|
||||
/** @description Validation Error */
|
||||
422: {
|
||||
headers: {
|
||||
[name: string]: unknown;
|
||||
};
|
||||
content: {
|
||||
"application/json": components["schemas"]["HTTPValidationError"];
|
||||
};
|
||||
};
|
||||
};
|
||||
};
|
||||
list_sources: {
|
||||
parameters: {
|
||||
query?: never;
|
||||
header?: never;
|
||||
path?: never;
|
||||
cookie?: never;
|
||||
};
|
||||
requestBody?: never;
|
||||
responses: {
|
||||
/** @description Successful Response */
|
||||
200: {
|
||||
headers: {
|
||||
[name: string]: unknown;
|
||||
};
|
||||
content: {
|
||||
"application/json": {
|
||||
[key: string]: unknown;
|
||||
}[];
|
||||
};
|
||||
};
|
||||
};
|
||||
};
|
||||
read_source_version: {
|
||||
parameters: {
|
||||
query?: never;
|
||||
header?: never;
|
||||
path: {
|
||||
source_id: string;
|
||||
revision: number;
|
||||
};
|
||||
cookie?: never;
|
||||
};
|
||||
requestBody?: never;
|
||||
responses: {
|
||||
/** @description Successful Response */
|
||||
200: {
|
||||
headers: {
|
||||
[name: string]: unknown;
|
||||
};
|
||||
content: {
|
||||
"application/json": {
|
||||
[key: string]: unknown;
|
||||
};
|
||||
};
|
||||
};
|
||||
/** @description Validation Error */
|
||||
422: {
|
||||
headers: {
|
||||
[name: string]: unknown;
|
||||
};
|
||||
content: {
|
||||
"application/json": components["schemas"]["HTTPValidationError"];
|
||||
};
|
||||
};
|
||||
};
|
||||
};
|
||||
list_clean_candidates: {
|
||||
parameters: {
|
||||
query?: never;
|
||||
header?: never;
|
||||
path: {
|
||||
source_id: string;
|
||||
revision: number;
|
||||
};
|
||||
cookie?: never;
|
||||
};
|
||||
requestBody?: never;
|
||||
responses: {
|
||||
/** @description Successful Response */
|
||||
200: {
|
||||
headers: {
|
||||
[name: string]: unknown;
|
||||
};
|
||||
content: {
|
||||
"application/json": {
|
||||
[key: string]: unknown;
|
||||
}[];
|
||||
};
|
||||
};
|
||||
/** @description Validation Error */
|
||||
422: {
|
||||
headers: {
|
||||
[name: string]: unknown;
|
||||
};
|
||||
content: {
|
||||
"application/json": components["schemas"]["HTTPValidationError"];
|
||||
};
|
||||
};
|
||||
};
|
||||
};
|
||||
create_clean_candidate: {
|
||||
parameters: {
|
||||
query?: never;
|
||||
header?: never;
|
||||
path: {
|
||||
source_id: string;
|
||||
revision: number;
|
||||
};
|
||||
cookie?: never;
|
||||
};
|
||||
requestBody?: never;
|
||||
responses: {
|
||||
/** @description Successful Response */
|
||||
201: {
|
||||
headers: {
|
||||
[name: string]: unknown;
|
||||
};
|
||||
content: {
|
||||
"application/json": {
|
||||
[key: string]: unknown;
|
||||
};
|
||||
};
|
||||
};
|
||||
/** @description Validation Error */
|
||||
422: {
|
||||
headers: {
|
||||
[name: string]: unknown;
|
||||
};
|
||||
content: {
|
||||
"application/json": components["schemas"]["HTTPValidationError"];
|
||||
};
|
||||
};
|
||||
};
|
||||
};
|
||||
apply_clean_candidate: {
|
||||
parameters: {
|
||||
query?: never;
|
||||
header?: never;
|
||||
path: {
|
||||
candidate_id: string;
|
||||
};
|
||||
cookie?: never;
|
||||
};
|
||||
requestBody?: never;
|
||||
responses: {
|
||||
/** @description Successful Response */
|
||||
200: {
|
||||
headers: {
|
||||
[name: string]: unknown;
|
||||
};
|
||||
content: {
|
||||
"application/json": {
|
||||
[key: string]: unknown;
|
||||
};
|
||||
};
|
||||
};
|
||||
/** @description Validation Error */
|
||||
422: {
|
||||
headers: {
|
||||
[name: string]: unknown;
|
||||
};
|
||||
content: {
|
||||
"application/json": components["schemas"]["HTTPValidationError"];
|
||||
};
|
||||
};
|
||||
};
|
||||
};
|
||||
reject_clean_candidate: {
|
||||
parameters: {
|
||||
query?: never;
|
||||
header?: never;
|
||||
path: {
|
||||
candidate_id: string;
|
||||
};
|
||||
cookie?: never;
|
||||
};
|
||||
requestBody?: never;
|
||||
responses: {
|
||||
/** @description Successful Response */
|
||||
200: {
|
||||
headers: {
|
||||
[name: string]: unknown;
|
||||
};
|
||||
content: {
|
||||
"application/json": {
|
||||
[key: string]: unknown;
|
||||
};
|
||||
};
|
||||
};
|
||||
/** @description Validation Error */
|
||||
422: {
|
||||
headers: {
|
||||
[name: string]: unknown;
|
||||
};
|
||||
content: {
|
||||
"application/json": components["schemas"]["HTTPValidationError"];
|
||||
};
|
||||
};
|
||||
};
|
||||
};
|
||||
register_ranking: {
|
||||
parameters: {
|
||||
query?: never;
|
||||
header?: never;
|
||||
path?: never;
|
||||
cookie?: never;
|
||||
};
|
||||
requestBody: {
|
||||
content: {
|
||||
"application/json": components["schemas"]["muse______http____________________7"];
|
||||
};
|
||||
};
|
||||
responses: {
|
||||
/** @description Successful Response */
|
||||
201: {
|
||||
headers: {
|
||||
[name: string]: unknown;
|
||||
};
|
||||
content: {
|
||||
"application/json": {
|
||||
[key: string]: unknown;
|
||||
};
|
||||
};
|
||||
};
|
||||
/** @description Validation Error */
|
||||
422: {
|
||||
headers: {
|
||||
[name: string]: unknown;
|
||||
};
|
||||
content: {
|
||||
"application/json": components["schemas"]["HTTPValidationError"];
|
||||
};
|
||||
};
|
||||
};
|
||||
};
|
||||
list_ranking_snapshots: {
|
||||
parameters: {
|
||||
query?: never;
|
||||
header?: never;
|
||||
path: {
|
||||
ranking_id: string;
|
||||
};
|
||||
cookie?: never;
|
||||
};
|
||||
requestBody?: never;
|
||||
responses: {
|
||||
/** @description Successful Response */
|
||||
200: {
|
||||
headers: {
|
||||
[name: string]: unknown;
|
||||
};
|
||||
content: {
|
||||
"application/json": {
|
||||
[key: string]: unknown;
|
||||
}[];
|
||||
};
|
||||
};
|
||||
/** @description Validation Error */
|
||||
422: {
|
||||
headers: {
|
||||
[name: string]: unknown;
|
||||
};
|
||||
content: {
|
||||
"application/json": components["schemas"]["HTTPValidationError"];
|
||||
};
|
||||
};
|
||||
};
|
||||
};
|
||||
capture_ranking_snapshot: {
|
||||
parameters: {
|
||||
query?: never;
|
||||
header?: never;
|
||||
path: {
|
||||
ranking_id: string;
|
||||
};
|
||||
cookie?: never;
|
||||
};
|
||||
requestBody?: never;
|
||||
responses: {
|
||||
/** @description Successful Response */
|
||||
201: {
|
||||
headers: {
|
||||
[name: string]: unknown;
|
||||
};
|
||||
content: {
|
||||
"application/json": {
|
||||
[key: string]: unknown;
|
||||
};
|
||||
};
|
||||
};
|
||||
/** @description Validation Error */
|
||||
|
||||
46
web/tests/旅程/资料对照.spec.ts
Normal file
46
web/tests/旅程/资料对照.spec.ts
Normal file
@ -0,0 +1,46 @@
|
||||
/** 真实浏览器与隔离库：资料对照页的来源清单、原文对照、清理应用与榜单快照失败态。 */
|
||||
import { readFileSync } from "node:fs";
|
||||
import { expect, test } from "@playwright/test";
|
||||
|
||||
test.use({ trace: "off" });
|
||||
|
||||
test("NC-browser-source-workbench：资料对照页与榜单失败态", async ({ page }, info) => {
|
||||
if (!process.env.MUSE_WORKBENCH_URL || !process.env.MUSE_AUTHOR_PASSWORD_FILE)
|
||||
throw new Error("需要明确的隔离后端和作者口令文件。");
|
||||
const sourceId = process.env.MUSE_SOURCE_ID!;
|
||||
if (!sourceId) throw new Error("需要预置来源编号。");
|
||||
const consoleErrors: string[] = [];
|
||||
page.on("pageerror", (error) => consoleErrors.push(String(error)));
|
||||
|
||||
await page.goto("/");
|
||||
await page
|
||||
.getByLabel("访问口令")
|
||||
.fill(readFileSync(process.env.MUSE_AUTHOR_PASSWORD_FILE, "utf8").trim());
|
||||
await page.getByRole("button", { name: "进入工作台" }).click();
|
||||
await page.getByRole("link", { name: /资料研究/ }).click();
|
||||
// 来源清单可见并点开预置来源。
|
||||
const 清单 = page.locator('section[aria-label="来源清单"]');
|
||||
await expect(清单.getByText("样章：码头来信")).toBeVisible();
|
||||
await 清单.getByRole("button", { name: /样章：码头来信/ }).click();
|
||||
|
||||
// 原文与清理对照：噪声原文可见；应用清理后对照说明出现且原文不动。
|
||||
const 对照 = page.locator('section[aria-label="原文与清理对照"]');
|
||||
await expect(对照.getByText("最新章节请访问 example.com")).toBeVisible();
|
||||
await 对照.getByRole("button", { name: "生成并应用清理对照" }).click();
|
||||
await expect(对照.getByText("清理已应用为对照版本；原文保持不变。")).toBeVisible();
|
||||
await expect(对照.getByText("最新章节请访问 example.com")).toBeVisible();
|
||||
|
||||
// 榜单快照：失败态如实呈现，成功快照带条目数。
|
||||
await page.getByLabel("榜单编号").fill("rank-demo");
|
||||
await page.getByRole("button", { name: "查看快照" }).click();
|
||||
const 榜单 = page.locator('section[aria-label="榜单快照"]');
|
||||
await expect(榜单.getByText(/采集失败：上游 503/)).toBeVisible();
|
||||
await expect(榜单.getByText(/2 条 · /).first()).toBeVisible();
|
||||
|
||||
await page.screenshot({ path: `${info.outputDir}/资料对照页.png`, fullPage: true });
|
||||
|
||||
expect(
|
||||
consoleErrors.filter((文本) => !文本.includes("favicon")),
|
||||
"页面不得出现控制台错误",
|
||||
).toEqual([]);
|
||||
});
|
||||
67
数据库/迁移/V0007__资料研究.sql
Normal file
67
数据库/迁移/V0007__资料研究.sql
Normal file
@ -0,0 +1,67 @@
|
||||
-- W17（B03）：外部与私人资料、原文版本、清理候选与榜单快照。
|
||||
-- 原文只追加不可改写；清理是对照产物；榜单快照把时点观察与效果推断分开（无 run 归因字段）。
|
||||
DO $migration$
|
||||
BEGIN
|
||||
-- 来源：同源再导入落到版本链；kind 区分作者稿、参考书、网页、笔记与灵感。
|
||||
CREATE TABLE public.muse_source (
|
||||
source_id uuid PRIMARY KEY,
|
||||
kind text NOT NULL CHECK (kind IN ('author_draft','reference','web','note','inspiration')),
|
||||
title text NOT NULL,
|
||||
origin text NOT NULL,
|
||||
authorized_uses jsonb NOT NULL DEFAULT '[]'::jsonb,
|
||||
created_at timestamptz NOT NULL DEFAULT clock_timestamp(),
|
||||
UNIQUE (origin, kind),
|
||||
CHECK (jsonb_typeof(authorized_uses) = 'array')
|
||||
);
|
||||
|
||||
-- 原文版本：只追加；内容哈希用于证据校验与重复辨认。
|
||||
CREATE TABLE public.muse_source_version (
|
||||
source_id uuid NOT NULL REFERENCES public.muse_source(source_id),
|
||||
revision bigint NOT NULL CHECK (revision > 0),
|
||||
content_hash text NOT NULL,
|
||||
content text NOT NULL,
|
||||
import_result jsonb NOT NULL,
|
||||
imported_at timestamptz NOT NULL DEFAULT clock_timestamp(),
|
||||
PRIMARY KEY (source_id, revision),
|
||||
CHECK (jsonb_typeof(import_result) = 'object')
|
||||
);
|
||||
CREATE INDEX ON public.muse_source_version (source_id, imported_at);
|
||||
|
||||
-- 清理候选：对照产物，原文不受影响；删除项可定位、可回退。
|
||||
CREATE TABLE public.muse_clean_candidate (
|
||||
candidate_id uuid PRIMARY KEY,
|
||||
source_id uuid NOT NULL REFERENCES public.muse_source(source_id),
|
||||
revision bigint NOT NULL,
|
||||
removals jsonb NOT NULL,
|
||||
status text NOT NULL CHECK (status IN ('proposed','applied','rejected')),
|
||||
cleaned_hash text,
|
||||
cleaned_text text,
|
||||
created_at timestamptz NOT NULL DEFAULT clock_timestamp(),
|
||||
UNIQUE (source_id, revision, candidate_id),
|
||||
CHECK (jsonb_typeof(removals) = 'array'),
|
||||
CHECK ((status = 'applied') = (cleaned_hash IS NOT NULL AND cleaned_text IS NOT NULL)),
|
||||
FOREIGN KEY (source_id, revision) REFERENCES public.muse_source_version(source_id, revision)
|
||||
);
|
||||
|
||||
-- 榜单来源登记：具名榜种与采集方式。
|
||||
CREATE TABLE public.muse_ranking_source (
|
||||
ranking_id text PRIMARY KEY,
|
||||
name text NOT NULL,
|
||||
url text NOT NULL,
|
||||
采集方式 text NOT NULL
|
||||
);
|
||||
|
||||
-- 榜单快照：时刻观察；失败保留失败态；不携带任何 run 归因。
|
||||
CREATE TABLE public.muse_ranking_snapshot (
|
||||
snapshot_id uuid PRIMARY KEY,
|
||||
ranking_id text NOT NULL REFERENCES public.muse_ranking_source(ranking_id),
|
||||
captured_at timestamptz NOT NULL DEFAULT clock_timestamp(),
|
||||
status text NOT NULL CHECK (status IN ('ok','failed')),
|
||||
failure_reason text,
|
||||
items jsonb,
|
||||
CHECK ((status = 'ok') = (items IS NOT NULL)),
|
||||
CHECK (items IS NULL OR jsonb_typeof(items) = 'array')
|
||||
);
|
||||
CREATE INDEX ON public.muse_ranking_snapshot (ranking_id, captured_at DESC);
|
||||
END
|
||||
$migration$;
|
||||
Loading…
x
Reference in New Issue
Block a user