2024 年 41 分钟访谈：Scale 哲学 + ChatGPT 故事 + AI 作为新 Utility + 3 个未来 Fork
AI 时代唯一"反共识但正确"的认知 = "scale 永远比共识认为的更有效"。Sam 用 YC 批次规模、神经网络 scaling、OpenAI 内部组织为例：找到"小规模已经在 working 的东西" → 推到前所未有的规模 → 几乎 always 出 interesting results。
教育是 Sam 自承的最大 prediction error："我以为 ChatGPT 上线 1 年内教育会大改，3.5 年过去了没看到任何系统性变化。"
"I offer no theory that I find satisfying to explain it... but empirically it does seem to be true, which is all of the most interesting things I have observed in my career in watching other things happen. All of the most interesting ones have had something to do with emergent properties that scale or scale continuing to provide returns far beyond what the consensus thinks will work."
| 案例 | 共识认为 | 实际发生 |
|---|---|---|
| 神经网络 scaling | "已知能 scale，不需要再试" | LLM 能力持续突破 |
| YC batch size | "太多公司，应该缩到 10 个/batch" | 批次网络效应 — 50+ 公司/batch 反而更好 |
| OpenAI compute scale | "10K-100K GPUs 不可能" | GPT-4 训练需要 ~25K H100s 集群 |
"Stuff breaks at accelerating rate and in an unpredictable way as you scale"
"There are always very smart people who say why you shouldn't do this"
突破需要 first principles reasoning，类比推理不 work
"We did not evolve to be good at thinking about exponentials. People have a hard time imagining that scaling laws are going to continue exponentially."
"Under the YC principle of see what your users love and do that."
"When something really starts growing and it's not very good, you have like a guaranteed hit on your hands."
| 时间 | 事件 |
|---|---|
| 早期 | Codex 进展一般 |
| 2024 初 | Codeex 真正变好（5.5 是 inflection point） |
| 2024+ | 用户做 "incredible things" |
"这 pipeline 有点怪，不太像 optimal solution。我们肯定会 major rewrite，但不知道何时。"
"We are in the process of creating a new utility. This doesn't happen very often. You know, electricity is a utility, internet's a utility, water, I guess there's not a lot of these."
"Even if we're totally right that intelligence is going to become this new utility... I kind of don't think the right way for us to analogize that is 'we're selling intelligence' because people are just like somehow not resonating. I don't know what our equivalent of 'we're selling you light at night' is going to be."
| 视角 | 主张 | 抽象层级 |
|---|---|---|
| Jensen | Compute 是 utility | 硬件层（chips） |
| Sam | Intelligence 是 utility | 服务层（tokens） |
| 真实答案 | 消费者看到 tokens，hardware 被抽象掉 | — |
"Pay for cell phone bill... you think about access to the whole system and the particular hardware at the base station and how it connects to the internet. You don't think about that as much."
| 时间 | 目标 | 意义 |
|---|---|---|
| 2026 年 9 月 | 用 500K A100-equivalent GPUs 作 AI research intern | Compute scale 极限 |
| 2028 年 3 月 | Full end-to-end very talented researcher（能发明完整新架构） | AGI 实质突破 |
这些是 Sam 2024 年在 Stanford 讲的目标，可能已更新。但 framework 仍 valid。
OpenAI 6/8 文章 "Built to Benefit Everyone" 跟 Sam 在 Stanford 讲的目标完全一致：
"How much is this technology going to be very widely democratized versus how much is it going to sit in a few companies."
Sam 概率：80% 民主路径
"How specifically how we distribute compute."
H100 / Blackwell 5x 价差 · "Compute shortage forever"
UBI vs ownership vs capitalism？
Sam 偏好 citizen wealth fund（不是现金分红）
"The risk of keeping this concentrated in a handful of companies even though we would be one of these companies is not something we should tolerate."
| 现状（2024 视角） | Sam 预测 |
|---|---|
| H100 / Blackwell 大幅短缺 | "As long as we can continue to make progress... there will be a shortage forever" |
| Long-term vs spot 价差 5x | Demand 永远 > supply |
| "几乎全部 gone for this year" | "100 个 personal agents 一直 work for you" |
"People assume we will make big inference gains on the hardware we have... I also think there is a tsunami of hardware coming but maybe the demand tsunami is even bigger."
"I would love to see... that we find a way to have something like a citizens wealth fund in the country or in the world eventually where you basically own a slice of capitalism."
LLMs 已经远超人类在某些方面
长 horizon + 高 judgment 任务
昨天模型 disprove 一个数学 conjecture（smart scientists 之前说"不会发生"）
但 robotics 显然需要 · 赌 LLM scaling 不 work 感觉 misguided
"Field held back by a generation of scientists who just were way too certain on what scaling was not going to produce and then some people just looked at the graphs and said, 'Well, it looks like it's continuing beautifully. Let's keep going.'"
"The data is quite strong on our side and I don't think it'd be that fun to say I told you so."
"You were like she was nervous. You're still going on about it. Like the data is quite strong on our side."
"If you make your identity about a particular thing is going to work or not work and then the science disproves you and you're too hung up on your identity, you can't let it go. You can't see the truth. I think this is a form of insanity."
"ChatGPT 上线 → 1 年学生 cheating → 教育系统大改 → 教得更好"
"I struggle to point to any significant systemic change that I've seen in the education system at large in the three and a half years since ChatGPT launched."
"If we continue to teach and evaluate students as if we were in a pre-agi world, it's not going to work and it is going to lead to like atrophy of learning how to think."
Sam 自述靠 writing 来 think
同样 meta skill
"Machines can do better, but useful to teach"