框架: clean守卫细化——章尾孤悬星号线连带清理+盗版占位残留特征词(1040-1060验证窗实测)
This commit is contained in:
parent
bcd502688b
commit
228ba42a33
@ -31,7 +31,8 @@ GARBAGE_LINE_PAT = re.compile(
|
||||
r"|感谢|打赏|盟主|书友|书评|拜谢|恳求|打滚|鸣谢"
|
||||
r"|(?:月|推荐|评价|催更|票)票|起点币|求(?:收藏|推荐|订阅|票|月票)|拉票|冲榜"
|
||||
r"|龙套|上架|[aA]签|签约|加更|[一二三四五六七八九十]更|更新时间|码字|存稿|请假|感言"
|
||||
r"|www|http|[\._]{2,}|→|♂|÷|一秒记住|记住本站|无弹窗|手机版|首发|最新章节|整理|转载",
|
||||
r"|www|http|[\._]{2,}|→|♂|÷|一秒记住|记住本站|无弹窗|手机版|首发|最新章节|整理|转载"
|
||||
r"|手打中|稍等片刻|重新刷新|获取最新", # 盗版站占位残留(1040-1060 验证窗实测)
|
||||
re.I)
|
||||
|
||||
|
||||
@ -164,6 +165,9 @@ def main(work_id, batch, file_, model, dry_run, report_md):
|
||||
stats["删除字数"] -= ch["removed"]
|
||||
stats["拒绝"].append(f"#{no}: 累计删除 {ch['removed']} 字超章长20%上限,整章回退")
|
||||
continue
|
||||
# 章尾孤悬分隔线清理:删掉章尾 ps 段后,其前置星号线会孤悬(前50章实测32处
|
||||
# 星号线全为 ps 前置分隔;场景分隔线后必有正文不会孤悬,故此规则天然安全)
|
||||
ch["text"] = re.sub(r"(?:\s*\n\s*[**]{3,}\s*)+$", "\n", ch["text"])
|
||||
ch["text"] = re.sub(r"\n{3,}", "\n\n", ch["text"]) # 删除后合并多余空行
|
||||
dirty.append((no, ch))
|
||||
|
||||
@ -203,7 +207,9 @@ def main(work_id, batch, file_, model, dry_run, report_md):
|
||||
f"(弹性命中 {stats['弹性命中']}、邻章命中 {stats['邻章命中']});"
|
||||
f"守卫拒绝 {len(stats['拒绝'])} 条(见文末)。",
|
||||
"",
|
||||
"> 看法:确认每条【删】段是独立垃圾、上下文(灰引文)是完整正文即可放行。", ""]
|
||||
"> 看法:确认每条【删】段是独立垃圾、上下文(灰引文)是完整正文即可放行。",
|
||||
"> 收尾规则(落库时自动执行,不逐条列出):被删 ps 段的前置星号分隔线若孤悬章尾",
|
||||
"> 将连带清除;被清洗章的三连以上换行归一为一个空行(⏎ 符号即原文换行)。", ""]
|
||||
seq = 0
|
||||
for no, ch in dirty:
|
||||
for exact, n, note, before, after in ch["audit"]:
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user