自学习 AI Harness — Discover AI 深度拆解 Shanghai AI Lab 论文
Harness engineering 是 2026-06-08 后 AI 圈新范式：Anthropic Boris 自己说 "stop prompting Claude, build loops that prompt themselves"。Anthropic 把"intelligence"从 LLM 移到 harness（参考 Fable 5 现象：70% refusal = 模型被换 harness）。Shanghai AI Lab 提出 Self-Harness Optimization —— 让 LLM 自己优化自己的 harness（不改权重），用 frozen model + 数学 loop 检测 failure pattern → 提案 → 验证 → 更新。
"Is Fable 5 really a new LLM, or since it's input a vision language model — is this really a new object or has it just a better harness?"
"Tropic is moving the intelligence out from their LLM system into the harness system."
| 任务类型 | Fable 5 | Claude 4.6 |
|---|---|---|
| Video game | ✅ 好 | ✅ 好 |
| HTML 生成 | ✅ 好 | ✅ 好 |
| 科学 task | ❌ 70%+ refusal | ✅ work |
| 价格 | 2x | 1x |
"We use the term harness to denote the non-parametric scaffolding that governs how a fixed language model is deployed as an agent."
简化：论文把 "everything that is not the LLM" 都叫 harness，包含 scaffold。
| 维度 | Scaffold | Harness |
|---|---|---|
| 本质 | Expertise as workflow | Execution substrate |
| 问 | "How should this problem be solved?" | "With which tool? How often retry? When stop?" |
| 包含 | Skills + workflows | Tool calling / file I/O / code exec / memory / loop control |
| 类比 | OSC: 知识 | OSC: runtime layer |
| Source of expertise | ✅ 是 | ❌ 否（只是 operational layer） |
"LLM running inside an iterative loop, governed by a harness, and shaped by a scaffold of skills and workflows."
System / execution / verification / failure recovery instruction
Tool calling / file I/O / code execution
Context compaction / past experience
Runtime control / routing
"Since it's a self-learning procedure, you know this is like a cool back library versions, you don't want to diverse too much here from your given probability distribution. So, you just make baby steps."
为什么不大幅修改：保持分布稳定 · 避免 catastrophic forgetting · 验证局部最优化（小改易验证）
| 字段 | 含义 |
|---|---|
| Cluster size | 失败记录数 |
| Representative task | 代表任务 |
| Shared trace symptoms | trace 共同症状 |
| Verifier evidence | verifier 证据 |
| Agent mechanisms | agent 行为 |
"Order clusters by estimated actionability — what can we solve immediately, what's simplest to correct, how can we optimize performance immediately? We don't waste 30-min or 1-hour runs to find what's wrong."
| # | 元素 |
|---|---|
| 1 | System prompt |
| 2 | Memory source |
| 3 | Sub-agent |
| 4 | Skills |
| 5 | Bootstrap instruction |
| 6 | Execution instruction |
| 7 | Verification instruction |
| 8 | Failure recovery instruction |
| 9 | Runtime control policy |
| 10 | Routing policy |
→ Self-harness optimization 改的就是这 10 个 element
"继续重复 same failed path，stuck in loop"
"You appear to be stuck in a tool loop. Stop repeating the same failed path. Summarize the evidence already collected. Choose the smallest remaining implementation step and then run one targeted verification."
不是改 LLM 权重，而是改 harness 的 instruction → 立即可读 / 立即生效
比 fine-tune 快 + 比 prompt 更结构化
"Of course, since the tasks are rather simple and you have a simple harness, a self-evolving harness, these are the easy wins."
"For a higher complexity system, there's nothing particular to it. It is more or less more of the same."
Sam 暗示 Anthropic "把 intelligence 移到 harness"（Fable 5 现象）
Sam "inference underinvested" → 被验证：harness engineering 是 inference 高阶
K1 = "harness 设计" 优秀 case（17 MCP tools + 3 源 retrieval）
Self-harness = 进一步让 harness 自我进化
K1 可以加 self-harness loop → 自动检测 failure → 优化 CLI
IndyDevDan "Claude Code = LLM + 迭代 loop + harness"
"Milestone + commit" = harness 的人工版本
Self-harness = Milestone + commit 的自动化
4 harness 元素：prompts / tools / memory / policy → memory + policy 是 routing 关键
Self-harness optimization 可以优化 "policy" → router policy 自学习
"估计 actionability" 排序 = routing decision log 分析
OpenSwarm = 多个 harness 协作
每个 harness 可以自学习 → 自演化 multi-agent
"OpenSkill"（Discover AI 之前）= self-evolution skill loop
"Stop prompting Claude and build loops that prompt them self."
→ Harness engineering = 2026-06-08 后新范式
→ Self-harness = loop 的 self-improvement 版本