框架: chat()撞模型max_tokens上限自适应降档重发(M2.7上限196608实测,修降级链400断链)
This commit is contained in:
parent
66bad3157b
commit
c39b2e9f3e
@ -59,6 +59,17 @@ def chat(prompt, model=DEFAULT_MODEL, max_tokens=512000, temperature=0.2,
|
||||
r = s.post(f"{BASE}/v1/chat/completions",
|
||||
headers={"Authorization": f"Bearer {TOKEN}"},
|
||||
json=payload, timeout=timeout)
|
||||
# 各模型 max_tokens 上限不一(实测 M3 收 512k、M2.7 上限 196608 报 400):
|
||||
# 撞上限自适应降档重发,保证降级链换模型时不被参数掀翻。
|
||||
# 必须在敏感检查之前处理——重发后的响应仍要完整走敏感/429/5xx 判定链
|
||||
if r.status_code == 400 and "max tokens" in r.text.lower():
|
||||
cur = payload.get("max_tokens", 0)
|
||||
payload["max_tokens"] = 196608 if cur > 196608 else 12000
|
||||
print(f"[llm] {model} max_tokens={cur} 超模型上限,降为 {payload['max_tokens']} 重发",
|
||||
file=sys.stderr)
|
||||
r = s.post(f"{BASE}/v1/chat/completions",
|
||||
headers={"Authorization": f"Bearer {TOKEN}"},
|
||||
json=payload, timeout=timeout)
|
||||
# 内容安全拦截:同模型退避重试必再敏感,立即抛 SensitiveError 交上层
|
||||
# 降级换模型,不在此浪费退避(否则一敏感章空烧 3 次,实测占放量请求 23%)
|
||||
if r.status_code >= 500 and "sensitive" in r.text.lower():
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user