2024 年 4 月,xAI 的人去找数据中心供应商,问了一个问题:在一个地方把 10 万张 GPU 连贯地跑起来,需要多久?
对方给的答复是 18 到 24 个月。
马斯克在直播里复述这段话时的语气很平静:18 到 24 个月,意味着失败是必然的。于是他做了个决定,自己建。结果是 122 天。
这场直播本来是要发布 Grok 3 的。但真正听完两个多小时,会发现模型只是结果,前面那一大段关于厂房、发电机、冷却塔、比特翻转和凌晨四点二十的内容,才是这次发布最有分量的部分。
先把使命说清楚,再说产品
开场马斯克没有讲参数,讲的是为什么做这件事。
我们要理解宇宙的本质,搞清楚到底发生了什么:外星人在哪,生命的意义是什么,宇宙怎么结束,又是怎么开始的。这些基本问题驱动着我们。(马斯克)
这套说法听着玄,但它直接决定了一个工程选择:xAI 要做最大化追求真理的 AI,哪怕真相有时候跟政治正确相冲突。
他的论证很干脆。要理解宇宙,就必须严格地追求真理,否则你理解的只是自己的幻觉和错误。这条路没有捷径可走。
Grok 这个名字来自海因莱因的《异乡异客》(Stranger in a Strange Land),书里一个在火星上长大的人用它表示"彻底而深刻地理解某件事"。团队特意补了一句:共情也是理解的一部分。
17 个月,从玩具到能上大学
从 Grok 1 到现在,只过了 17 个月。
马斯克自己评价 Grok 1:"以现在的眼光看,它几乎就是个玩具。"3140 亿参数,放到 2023 年底算能打,放到今天就是个起点。
中间那段 Grok 2 的训练经历,被讲得很不体面。2024 年 2 月启动,团队以为手里芯片不少,实际跑起来发现,同时能连贯工作的训练卡远远不够。冷却出问题,供电出问题,平均 8000 张 H100 只能跑到 80% 的效率,等效下来大概 6500 张在真正干活,就这么跑了好几个月。
到了 Grok 3,数字变成 10 万张。算力比上一代多 10 倍以上,马斯克插话说"可能 15 倍"。
所以那句被反复引用的判断,在这场直播里有了具体的注脚:智能的提升来自大集群。人有最好的工程师、最好的 AI 人才,剩下唯一缺的东西就是算力。
把"建一个数据中心"拆成一道道能解的题
供应商报出 18 到 24 个月之后,团队开始拆问题。这段叙述几乎是 Grok 推理过程的人类版本。
第一层:需要一个建筑。自己盖来不及,只能用现成的。他们找的是那种公司破产、但厂房状况还不错的工厂。最后在孟菲斯找到一座伊莱克斯(Electrolux)留下的厂子。马斯克顺口抖了个包袱:孟菲斯是猫王的故乡,也是古埃及最早的首都之一。
第二层:需要电。初期至少要 120 兆瓦,厂房自带的容量只有 15 兆瓦。最终 20 万张 GPU 要吃掉 0.25 吉瓦。解决办法是在厂房一侧排满发电机拖车,一车接一车,撑到市电接进来为止。
第三层:需要冷却。厂房另一侧同样排满拖车,他们租下了全美大约四分之一的移动冷却能力。
第四层:密度。要达到这个密度必须上液冷,而在此之前,没有人做过这么大规模的液冷数据中心。管路、接头、流量,全部要从零趟一遍。
十万人的交响乐团,和突然安静的那一秒
整场直播里最生动的一段,是马斯克讲功率波动。
GPU 集群的功率波动非常剧烈,像一场巨大的交响乐,十万甚至二十万个演奏者同时在台上。整个乐团会在 100 毫秒内从极响变成极静。这造成了巨大的功率波动,发电机完全懵了,它们从没见过这种场面。(马斯克)
发电机撑不住,就得在中间加缓冲。他们用了特斯拉的 Megapack 储能来做功率平滑。原有的控制逻辑对付不了这种节奏,xAI 和特斯拉的团队一起重新写了 Megapack 的程序,才让电流稳定下来,机器能正常跑。
再往下一层是网络。十万张卡要像一个整体那样通信,任何一根线、任何一个固件设置出错,整个训练就散了。团队在凌晨四点二十左右定位到问题:BIOS 不匹配,某个参数没配对。定位方式是拿两台机器做 lspci 输出对比,一台能跑,一台不能跑,一行一行看差异。
马斯克把这套方法论说得很明白:像 Grok 做推理那样,把问题拆成组成元素,然后逐个解决。这是他们能在别人几分之一的时间里搞出一个连贯训练集群的唯一原因。整个过程没有什么魔法发生。
还有一层更底层的敌人。负责推理团队的 Tony 提到,跑十万张卡,等于跟宇宙的终极 boss 熵正面对抗:随时可能有宇宙射线打下来,把你某个晶体管里的一个比特翻掉。如果翻的是尾数位,整个梯度更新就废了,而这样的卡有十万张,需要时时刻刻盯着。
盲测代号"热巧克力"
说完基建,才轮到模型。
Grok 3 的预训练在 2025 年 1 月初完成,直播时模型还在继续训练。评估分三类:数学推理(AIME,美国数学邀请赛,一年一次)、博士级科学知识、计算机编程(竞赛编程加 LeetCode 面试题)。团队的说法是,Grok 3 在所有类别里处于独一档,连小号的 Grok mini 也摸到了前沿。
但工程师自己先把质疑提出来了:这些基准测的可能是教科书背得熟不熟、GitHub 仓库记得牢不牢,那真实场景里好不好用呢?
于是有了第二个测试。两周前,他们把一个早期版本的 Grok 3 匿名扔进 Chatbot Arena 盲测,代号 Chocolate。规则很简单:去掉所有产品外壳,只比模型本身,用户提一个问题,看到两个匿名回答,选更好的那个。
结果这个还在爬坡的版本,ELO 到了 1400 左右。此前没有模型到过这个分数。而且不是某一类第一,是综合能力、指令遵循、编程全部第一。团队说它还在往上走,同时手上已经有一个比送测版本更好的模型。
马斯克在这里补了一句用户体验上的实话:Grok 3 上线后,用户可能会发现它几乎每天都在变好,有时候 24 小时内就能感觉到差别。
会思考的模型,和"想久一点"这个新旋钮
光有好的预训练模型不够。团队的说法是,最好的 AI 得会像人一样思考:设想所有可能的解法,自我批评,验证,走不通就退回来,并且从第一性原理出发。
做法是拿最好的预训练模型继续做强化学习,把推理能力激发出来。这样能力可以同时在两个维度上扩展:训练时,和测试时。
直播里展示的推理模型有两个:Grok 3 Reasoning Beta 和 Grok 3 mini Reasoning。有意思的地方在于,mini 版本训练时间更长,某些测试上反而略好于大号推理模型。这反过来说明大号模型因为训练时间短,还有很大空间。
Tony 解释了图表上那段阴影条。推理模型可以被要求"想更久",同一个问题反复求解多次再下结论,这就是测试时计算(test-time compute)。给了这份预算,模型的表现还能再往上抬一截。他的说法是,过去只让 AI 走一条思考链,现在可以同时走很多条。
五天前刚考完的那张卷子
关于"是不是在对着基准过拟合"这个问题,团队没有辩解,直接甩了一个新证据。
AIME 2025 在直播前五天刚刚结束。这是一场全新的、刚出炉的竞赛。他们把两个推理模型扔进同一张卷子,结果是大号模型在新鲜考题上表现更好。而如果去看去年的旧卷子,情况正好相反,小模型反而更占优,因为它把旧题学得更透。
这个对比说明的东西很关键:大模型的泛化能力更强,它学到的是解题方法,不是题库。
还有一层更反直觉的发现。Grok 的推理能力,只在数学题和竞赛编程题上训练过,任务类型非常窄。但它把这些能力迁移到了完全无关的事情上:写游戏、处理各种别的任务。团队的解释是,模型在数学和编程里学到的是一套通用动作:发现自己的错误、纠正、死磕一个问题、试很多种变体、挑最好的那个。这套动作换个场景照样能用。
马斯克在这里给了一句挺妙的总结:现实是数学的具象化。
顺便说,17 个月前 Grok 1 几乎解不出任何一道高中数学题。现在团队的说法是,人类考试已经不够难了,很快就没有基准可刷了。
现场两个 demo:火星轨道和一款不存在的游戏
第一道题,只有一句话的 prompt,没有任何脚本预设:生成一段 3D 动画代码,展示从地球发射、登陆火星、并在下一个发射窗口返回地球的过程。
Grok 开始输出思考轨迹(thinking trace),用户可以点进去看它在想什么。团队提到,展示出来的思考是经过部分遮挡的,防止模型被瞬间抄走。
几分钟后代码出来了,用 matplotlib,写着"数值求解开普勒定律"。跑起来之后,地球和火星在转,一个绿色小球从地球出发,切到火星,停一会儿,再切回来。马斯克看完的评价是,真实轨道还要考虑很多复杂性,但这个已经相当接近,他自己项链上挂的吊坠就是这个图案。
第二道题更难:做一个把俄罗斯方块和宝石迷阵(Bejeweled)混在一起的游戏。
这道题的意义在于原创性。让 AI 做俄罗斯方块,网上有无数现成代码可以抄。这次的考验是,它能不能想出一个把两种机制缝合起来、而且还真的好玩的东西。团队给这个任务开了"Big Brain"模式,用更多算力做推理,同时发了三次请求做备份。
结果跑起来了。画面像俄罗斯方块,颜色有点怪,机制被改了:三个同色方块连成一线就消失,然后重力激活,上面的方块往下掉。方块满行到底消不消,团队自己也没完全搞清楚,说"看你怎么理解"。他们还补了一句:每次问它,出来的版本都不一样。
马斯克当场宣布了一件事:xAI 要成立一个 AI 游戏工作室,就在这场直播里正式对外说。
DeepSearch:把"搜一下"变成"派个实习生去查"
接下来是产品层的重头戏。
逻辑链条是这样的:最好的预训练模型加上最好的推理模型,构成了底座;人类解题从来不只靠一支笔一张纸,还需要浏览器、搜索引擎、代码解释器;把这些工具给到推理模型,就得到了 Grok agent 的第一代产品,DeepSearch。
它跟现在的搜索不一样。传统搜索一次检索就返回结果,DeepSearch 会先想清楚用户到底想要什么,需要核实哪些事实,应该去读多少个网站,然后自己去读、去交叉验证,最后给出结论。左边是进度条,右边是它每一步在干什么的摘要。
直播里跑了三个查询。
第一个问星舰下一次发射是什么时候。模型跑了一分钟左右,翻出第七次飞行的情况、被停飞的原因、FCC 的文件,结论是不早于 2 月 24 日,也可能更早。中间还自动生成了一张表。团队开了个玩笑,说在 xAI 内部,"到第一张表的时间"是唯一值得关注的延迟指标。
第二个问 Dota 2 硬核模式当前最流行的出装。结果是 Invoker 当道,然后是风暴之灵,模型还顺带指出早期版本里最火的根本不是这几个。
第三个问怎么赢下巴菲特那十亿美元的"疯狂三月"全对竞猜。答案还没发生,只能等。马斯克调侃说,40 美元的订阅去博十亿美元,这笔投资挺划算。
DeepSearch 有一个其他搜索引擎做不到的能力:你可以点开子任务面板,看它怎么判断哪些信息源可信、哪些不可信、怎么交叉比对。你也可以直接命令它只用 X 上的来源。这种可操控性是普通搜索给不了的。
马斯克给了一个很实在的换算:这件事你自己去查,可能花掉半小时到一小时,交给它,十分钟后回来,活儿已经干完了,而且可能干得比你更好。
什么时候能用,以及一个关于名字的提醒
节奏是这样的:
- 发布当天开始向 X 的 Premium Plus 订阅用户开放,X 应用已经更新,高级功能都在里面
- 推出一个独立订阅 SuperGrok,面向想要最强能力和最早尝鲜的人,配套 Grok 独立应用和网站
- 网站是 grok.com,iOS 应用商店也能搜到
关于用哪个入口,团队说得很坦白:网页版永远是最新最强的版本。因为上架 App Store 要走审核,手机形态本身也有功能限制,所以最新能力总是先到 grok.com。
这里有个好玩的识别方法。打开应用,如果显示的是 "Grok",那装的是新版;如果还是那个旧拼写的名字,说明新版还没推到你这里。
马斯克也主动给这次发布降了温:这是 beta,别指望一开始就完美,但会以几乎每天的速度变好。如果想要更成熟的体验,等一周。语音模式大约还需要一周的打磨,他当天试过,说"效果相当不错,但还要再打磨一下"。
观众问答里透露的几个关键信息
语音是一个模型,不是拼接的 TTS。 有人问语音模式是不是"语音识别加文字转语音"的组合,回答是:一个 Grok 变体,直接听懂你说的话,直接生成音频。Igor 举了个例子,他今天跟它对话,模型从某段文本里读到了他的名字,念错了,他说"不对,我叫 Igor",模型就记住了,后面一直念对。这种即时纠正,文字转语音的管线做不到。
API 几周内上线。 推理模型和 DeepSearch 都会进 API。团队更看好企业场景,测试时计算加上工具调用,能明显提速很多业务流程。
开源策略是先放上一代。 惯例是等下一代完全成熟稳定之后再开源上一代。Grok 3 大概还需要几个月才能到那个状态,届时会开源 Grok 2。
Grok 是男是女? 马斯克的回答是:你想让它是什么就是什么。
会有多个 Grok 吗? 可以有多个。Igor 说团队内部就有人打算弄一个"健身 Grok"专门管训练。
下一站:1.2 吉瓦
直播快结束时,马斯克透露了下一个集群的规模,用的参照物是《回到未来》里那台需要 1.21 吉瓦的车。
从 0.25 吉瓦跳到大约 1.2 吉瓦,接近五倍。硬件换到 GB200 和 GB300 这一代,建成后仍然是世界上最强的训练集群。
关于"什么时候能解决黎曼猜想",负责推理的 Tony 给了个半开玩笑的技术回答:最简单的办法是枚举所有可能的字符串,只要你有验证器,加上足够的算力,总能撞出来。更认真的版本是,未来几年机器会开始拿到一些奖牌,菲尔兹奖、诺贝尔奖这个级别,但大概率还需要人类专家在回路里。
还有一个绕不开的"两年"。有人问什么时候把 Grok 装到火箭上,马斯克的答案是两年。他自己也知道这个梗:所有事情都是两年之后。紧接着他给了实际的时间表:地火转移窗口每 26 个月出现一次,下一个大约在明年年底,如果一切顺利,星舰会带着 Optimus 机器人去火星。
最后,团队被问到一个略显奇怪的问题:这个项目最难的部分是什么。回答没有绕开:让整个训练在 10 万张 H100 上连贯地跑起来,本质上是在跟熵打一场必输的仗,你只能一直赢下去。
至于马斯克自己的角色,他在开场就自嘲过了:我不干什么,只是偶尔出现一下。
这场直播真正值得拿走的几条
把两个多小时听完,产品参数反而是最容易被替代的部分。真正有用的,是那套把"不可能的时间表"变成工程步骤的方法,以及几个关于 AI 能力下一步走向的判断。
如果你也在这条路上做事,下面这几件事可以立刻用上:
- 先质疑供应商的时间表,再决定自建还是租用。 18 到 24 个月等于出局,这个判断一旦成立,后面的所有困难都变成"必须解决"而不是"要不要解决"。
- 把大目标拆成物理约束。 建筑、电力、冷却、网络、比特翻转,每一层都是独立的题,每道题都有解。拆不开的大目标才是真的难。
- 算力之外,测试时计算是被低估的杠杆。 同一个问题让模型多跑几遍再取最优,不需要重训就能涨分。这条对做产品的人尤其划算。
- 训练数据的窄,不等于能力窄。 只在数学和竞赛编程上做强化学习,学到的却是"自查、纠错、坚持、多路尝试"这套通用动作。选训练任务时,优先选那些能逼出通用元能力的。
- 给自己的 AI 配上工具,而不是只加参数。 浏览器、搜索、代码解释器,这是把聊天机器人变成 agent 的分界线。DeepSearch 的本质就是这个加法。
- 做搜索类产品,把推理过程露出来。 让用户看到信息源怎么被筛选、怎么被交叉验证,可信度立刻不一样,这是传统搜索框做不到的差异化。
- 新版本先上网页端,再考虑 App。 审核周期和移动端限制会吃掉你的迭代速度,最强的能力永远应该先放在你能完全控制的地方。
- 接受 beta 状态,用日更节奏换市场时间。 与其等到完美,不如先放出去,用每天可见的改进留住用户。前提是你真的做得到每天改。
- 关注开源节奏,规划你的技术栈。 上一代模型会在下一代稳定后开源,这意味着半年左右你就能拿到今天的前沿能力做本地部署。
马斯克最后说的是,让他们每天早上爬起来的东西,是把 Grok 免费给出去,然后看着使用量往上涨。
这话听着像场面话。但想想那 122 天、那些发电机拖车、那四分之一的全国移动制冷能力、那个凌晨四点二十才定位到的 BIOS 参数,它确实不完全是。
0:00well welcome to the gro 3 presentation so the mission of xai and Gro is to understand the universe we want to understand the nature of the universe so we can figure out what's going on where are the aliens what's the meaning of life how does the universe end how did it start all these fundamental questions were driven by curiosity about the nature of the universe and that's also what causes us to be a maximally truth seeking AI even if that truth is sometimes at odds with what is politically correct in order to understand the nature of the universe you must absolutely rigorously pursue truth or you will not understand the universe you will be suffering from some
0:39amount of delusion or error so that is our goal figure out what's going on and we're very excited to present grock 3 which is we think an order of magnitude more capable than grock 2 in a very short period of time and that's thanks to the hard work of an incredible team and I'm honored to work with such a great team and of course we'd love to have some of the smartest humans out there join our team so with that let's let's go hi everyone my name is eigor lead engineering at xcii I'm Jimmy Paul leading research I'm Tony working on the reasoning Team all right I'm Elan I don't do anything I just show up occasionally yeah so like mentioned Gro is the tool that we're working on
1:23Grog is our AI that we're building here at XI and we've been working extremely hard over the last few months to improve grock as much as we can so we can give it to all of you so we can give all of you access to it we think it's going to be extremely useful do we think it's going to be interesting to talk to funny really really funny and we're going to explain to you how we've improved gr over the last few months we've made quite a jump in in capabilities yeah actually we should explain maybe also what is why do we call it Gro so Gro is a word from a heand novel Stranger in a Strange Land and it's used by a guy who's who was raised on Mars and
1:58the word Gro is is to sort of fully and profoundly understand something that's what the word gr mean fully and profoundly understand something and empathy is important true so yeah so if we charted xas progress in the last few months has only been 17 months since we started kicking of our very first model grock 1 was almost like a toy by this point only 314 billion parameters and now if you proud the progress the time on x-axis the performance of favorite Benchmark numbers M mlu on the y axis we're literally progressing at unprecedent speed across the whole field and then we kick off grock 1.5 right after grock One released after November 2023 and then gr
2:412 so if you look at where the all the performance coming from when you have a very correct engineering team and all the best AI at Talent the only one thing we need is a big intelligence comes from big cluster so we can reconvert the entire progress of X now replacing the Benchmark on the y axis to the to total amount of training flops that is how many gpus we can run at any given time to train our large language models to compress the entire internet so after all human all human knowledge really that's right yeah internet being part of it but it's really all human knowledge all everything yeah the whole internet fits into a USB stick at this point it's
3:18like all the human tokens yeah that's right yeah very soon into the real world yeah so we had so much trouble actually training grock 2 back in the days with kickoff the model around February and we thought we had a large amount of chips but turned out we can barely get AK training chips running coherently at any given time and we had so many Cooling and power issues I think you were there in the data center yeah it was like really sort of more like 8K tiffs on average at 80% efficiency more like like 6,500 effective h100s training for you know several months but now now we're at the 100K yeah that's right 100K that's right so so what's the next step right so after gu 2 so if we
4:05want to continue accelerate we have to take the matter into our own hands we have to solve all the coolings all the power issues and everything yeah so so in April of last year Elon decided that really the only way for XI to succeed for XI to build the best AI out there is to build our own data center so we didn't have a lot of time right because we wanted to give you gr free as quickly as possible so really we realized we have to build the data center in about 4 months it turned out it took us 122 days to get the first 100K gpus up and running and that was a Monumental effort to be able to do that it's we believe it's the biggest fully connected h100 cluster of its
4:44kind and we didn't just stop there we actually decided that we need to double the size of the cluster pretty much immediately if we want to build the kind of AI that we want to build so we then had another phase which we haven't talked about publicly yet so this is the first time that we're talking about this U where we doubled the capacity of the data center yet again and that one only took us 92 days so we've been able to use all of these gpus use all of this compute to improve grock in the meantime and basically today we're going to present you the results of that the the fruits that came from that so let yeah so all the paths all the rows leads to gr 3
5:2310x more compute more than 10x really yeah really maybe 15x yep compared to our previous generation model and gr finished the pre-training early January and we start you know the model still currently training actually so this is a little preview of our Benchmark numbers so we evaluated gr 3 on you know three different categories on General mathematical reasonings on general knowledge about stem and Science and then also on computer science coding so Amy American Invitational math examination hosts it you know once a year and if we evaluate the model performance we can see that the grock 3 across the board is in a league of its own even his little brother gr mini is reaching the
6:13frontier across all the other competitors so you will say well at this point all these benchmarks you're just evaluating you know the memorization of the textbooks memorization of the GitHub repos how about realtime usefulness how about we actually use those models in our product so what we did instead is we actually kicked off a blind test of our gr 3 Model code named Chocolate it's pretty hot yeah hot chocolate and you know been running on this platform called Cho arena for two weeks I think the entire X platform at some point speculated this might be the next generation of a AI coming away so how this CH Arena works is that it strip away the entire product surface
6:59right it just raw comparison of the engine of those agis the language models themselves and place interface where the user will submit one single query and you get to show two responses you don't know which model they come from and in then you make the both so in this blind test gr 3 and early version of Gro 3 already reached like 1,400 no other models has reached an ELO score had to have comparison to all the other models at this score and it's not just one single category it's 1,400 aggregated across all the categories in ch B capabilities instruction following coding so it's number one across the board in this blind test and it's it's still climbing so we actually to keep
7:38updating it so it's it's 14400 about, 1400 in climbing yeah and in fact we have a version of the model that we think is already much better than the one that we tested here yeah we'll see you know how how far it gets but that's the one that we're you know working on we're talking about today yeah so actually one thing if if you're if you're using grock 3 you I think you may notice improvements almost every day because we're we're continuously improving the model so Lally even within 24 hours you'll see improvements yep so but we believe here at X getting the best pre-training model is not enough that's not enough to build the best AI and the best a need to think like a
8:16human you to contemplate about all the possible solutions self-critique verify all the solutions backtrack and also think from the first principle that's a very important capability so we believe that as we take the best train model and continue training it with reinforcement learning it will elicit the additional reasoning capabilities that allows the model just become so much better and scale not just in the training time but actually in the test time as well so we already found the models extremely useful internally for our own engineering saving hours of time hundreds of hours of coding time so eal you the power user of our gra reasoning model so what are some use
8:56cases yeah so like Jimmy said we've added Advanced reasoning capabilities to Gro and we've been testing them pretty heavily over the last few weeks in order to give you a little bit of a taste of what it looks like when Gro is solving heart reasoning problems so we prepared two little problems for you one comes from physics and one is actually a game that gr is going to write for us so when it comes to the physics problem you know what we want Gro to do is to plot a viable trajectory to do a transfer from Earth to Mars and then at a later point in time a transfer back from Mars to Earth and that requires some know some physics that Gro will have to
9:30understand so we're going to challenge Gro you know come up with a viable trajectory calculate it and then plot it for us so we can see it and yeah this is totally unscripted by the way this is the that's the entirety of the prompt which just be clarify is that yeah there there's nothing more than that yeah exactly this is the gro interface and we've typed in this text that you can see here generate code for an animated 3D plot of a launch from Earth landing on Mars and then back to Earth at the next launch window so and we've not kicked off the query and you can see Gro is Ping so part of grock's advanced reasoning capabilities are these ping traces that you can see
10:08here you can even go inside and actually read what Gro is thinking as it's going through the problem as it's trying to solve it yeah we say like we are doing some obscuration of the thinking so that our model doesn't get totally copied instantly so there's more to the thinking than is displayed yeah yeah and because this is totally unscripted there's actually a chance that Gro might made a little coding mistake and it might not actually work so just in case we're going to launch two more instances of this so if something goes wrong we were able to to switch to those and show you something that's present so we're kicking off the other two as well and
10:48like I said we have a second problem as well and yeah actually one of the favorite one of our favorite activities here at xci is having gr WR games for us and not just any know any old game any game that you might already be familiar with but actually creating new games on the spot and being creative about it so one example that we found was really really fun is create a game that's a mixture of the two games Tetris and Beed so this is maybe an important thing like there obviously if you if you ask an AI to create a game like Tetris there's there are many examples of Tetris on the on the Internet or game like J whatever there it can copy it what's interesting here
11:28is it a achieved a creative solution combining the two games that actually works and and is a good game yeah that's the it's cre we're seeing the beginnings of creativity yeah fingers crossed that we can recreate that hopefully it works embarrassing it so actually because this is a bit more challenging we're going to use something special here which we call Big Brain that's our mode in which we use more computation which more reasoning for GR just to make sure that you know there's a good chance here that it might actually might actually Do It so we're also going to fire off know three attempts here at at solving this game at creating this game that's a
12:06mixture of know Tetris and Bs yeah let's let's see what Gro comes up like I've played the game it's pretty good like it's like wow okay this is something yeah so while gr is thinking in the in the background we can now actually talk about some concrete numbers know how how well is gr doing across tons of different tasks that we've tested it on so we'll hand it over to Tony to talk about that yeah okay so let's see how Grog does on those interesting challenging benchmarks so yeah so reasoning again refers to those models that actually thinks quite for quite a long time before it tries to solve a problem so in this case you know around a month ago the grph 3
12:47pre-training finished so after that we work very hard to put the reasoning capability into the current graph 3 Model but again this is very early days so the model is still Cy in training so right now what we're are going to show to people is this beta version of the grth three reasoning model alongside we also are training a mini version of the reasoning model so essentially on this plot you can see the grth three reasoning beta and then grth three mini reasoning the gr reason mini reasoning is actually a model that we train for much longer time and you can see that sometimes it actually perform slightly better compared to the gr 3 reasoning
13:24this also just means that there's a huge potential for the gr 3 reasoning because it's train for much less time so all right so let's actually look at what how how it does on those three benchmarks so Jimmy also introduced already so essentially we're looking at three different areas mathematics science and coding and for math we're picking this high school competition M problem for science we actually pick those PhD level science questions and for coding it's also actually pretty challenging it's competitive coding and also some lead code which is some code inter interview problems that people usually get when they interview for company so on those benchmarks you
14:03can see that the gro 3 actually perform quite well across the board compared to other competitors yeah so it's pretty promising these models are very smart so Tony what what what are those shaded bars yeah so okay so I'm GL you asked this question so for those models because it can reason it can think you can also ask them to even think longer you can spend more what we call test and compute which means you can spend more time to reason to think about a problem before you spit out the answer so in this case the Shaded bar here means that we just asked the model to spend more more time you know you can solve the the same problem many many times before it it
14:46tries to conclude what is the right solution and once you give this compute or this this kind of budget to the model it turns out the model can even perform better so this is essentially the Shaded part in in those SPS right so I think this is really exciting right because now instead of just doing one chain of thoughts with AI why not do multiple at once yes so that's a very powerful technique that allows to continue scale the model capabilities after training and you know people often ask are we actually just over fitting to the benchmarks so how about generalization so yes I think yeah this is definitely a question that we are asking ourselves whether we are overfitting to
15:26those current benchmarks luckily we have a real test so about 5 days ago Amy 2025 just finished this is where high school students compete in this particular Benchmark so we got this very fresh new competition and then we asked our two models to compete on the same Benchmark at the same exam and it turns out very interestingly the grth three reasoning the big one actually does better on this particular new fresh exam this also means that the generalization capability of the big model is stronger much stronger compared to the smaller model if you compare to the last year's exam actually this is the opposite the smaller model kind of learns the the the previous exams
16:11better so yeah so this this actually shows some kind of true generalization from the model that's right so 17 months ago our Gro zero and Gro one barely solves any High School problems that's right and now we have a kid that just already graduate the gr gr is ready to go to college is that right yeah I mean it's w beong with before is simply perfect the human exams won't be hard they' be too easy yeah like and internally we actually as a gret Contin evolves we're going to talk about you know what we're excited about but very soon there will be no more benchmarks left yeah yeah one thing that's quite fascinating I think is that we basically only trained Ro's reasoning abilities on
16:49math problems and competitive coding problems right so very very specialized kinds of tasks but somehow it's able to work on all kinds of other different tasks so including creating games no lots lots and lots of different things and what seems to be happening is that basically Gro learns this ability to detect its own mistakes and its thinking correct them persist on a problem try lots of different variants pick pick the one that's best so there are this generalized generalizing abilities that Gro learns from mathematics and from coding which it can then use to solve all kinds of other problems so that's yeah that's pretty I mean reality is the instantiation of
17:25mathematics mhm that's right and one thing we actually really excited about that going back to our funing mission is what if one day we have a computer just like deep thought that utilize our entire cluster just for that one very important problem in the test time all the GPU turned on right so I think back then we were building the GPU cluster together you were plugging cables and I remember that when we turn on the the first initial test you can hear all the GPS humming in the hallway that's almost feel like spiritual yeah that that's actually a pretty cool thing that we're able to do that we can can go into the data center and Tinker with the
18:01machines there so for example we went in and we unplugged a few of the cables and just made sure that our training setup is still running running stably so that's something that you know I think most AI you know teams out there don't usually do but it's actually totally unlocks like a new level of reliability and what you're able to do with with the hardware so okay so when when are we going to solve remon so the easiest solution is to numerate over all possible strings and as long you have a verifier enough compute you'll be able to do it okay my projection will be what your guess what is your neuronet calculate so my my my both prediction so so 3 years ago I told
18:41you this I think in now it's two years later two things going to happen we're going to see machines win some medals that's touring award abely Fields medal Nobel Prize with probably some expert in the loop right so the expert uplifting do you mean so this year or next year oh okay that's what it comes down to really yeah so it looks like Gro finished know all of its thinking on on the two problem so let's take a look at what it said all right so this was the the little physics problem we had no we we've collapsed the thoughts here so they're you know they're hidden and then we see grock's answer below that so it explains it wrote a python script here
19:21using matplot lip then gives us all of the code so let's take a quick look at the code you know seems like it's doing reasonable things here here not not totally of the Mark solve Kepler says here so maybe it's solving Kepler laws cap capler law numerically yeah there's really only one way to find out if this thing is working I'd say let's let's give it a try let's run let's run the code all right and we can see yeah gr is animating two different planets Earth and Mars here and then the the green ball is the the vehicle that's transiting the the spacecraft that's transitioning between Earth and Mars and you you could see the journey from Earth to Mars and looks like yeah
20:01indeed the the astronauts return safely you know at the right moment in time so now obviously this was just generated on the spot so now we can't tell you if that was actually correct solution so we're going to take a closer look now maybe we're going to call some colleagues from space X ask them if if this is legit it's pretty close it's it's I mean yeah I mean there there's a lot of complexities in the actual orbits that have to be taken into account but this is this is pretty close to to what it what looks like awesome in fact I have that on my pendant here this got the Earth m home and transfer on it when when are we going to install rock on a
20:39rocket well I suppose in 2 years two years everything is two years away well Earth and Mars Transit can occurs every 26 months the next we're currently in a Transit window approximately the next one would be November of next year roughly end of next year and and if all goes well SpaceX will send Starship Rockets to Mars and with Optimus robots and U and Gro I'm curious what this combination of Tetris and B looks like bet Tetris as we've named it internally so okay we also have an output from gr here it say Ro python script explains that it's what it's been doing if you look at the the code now there are some constants that are being defined here some colors then the the
21:27trinos the the pieces of Tetris are there obviously very hard to see at one glance if this is good so we got to we got to run this to figure out if it's working let's let's give it a try fingers crossed all right right so this kind of looks like Tetris but the the colors are a little bit off right the colors are different here and if I if you think about what's going what's going on here the J has this mechanic where if you get three Jews in a row you know then they they disappear and also gravity de activate right so what happens if you get three of the colors together okay so something happened so I think I think what gr did in this version is is that you know once you
22:09connect three at least three blocks of the same color in a row then you know gravity activates and they disappear and then gravity activates and all the other blocks fall down kind kind of curious if there's still a Tetris mechanic here where if the line is full does it actually clear it or what happens then it's up to interpretation you know so who knows yeah every I mean when it'll do different variants when you ask it it doesn't do the same thing every time exactly we've seen a few other but Tetris that worked very differently but this one seems cool so yeah are we ready for game Studio at x. a yes so we're launching an AI gaming studio at xci
22:50if you're interested in joining us and building AI games please join xai we're launching an AI gaming studio we're announcing it tonight let's Go epic games but right that's an actual game ST me yeah yeah all right so I think one thing is super exciting for us is that once you have the best pre Trend model you have the best reasoning model right so we already see that we actually give the capability for those model to think harder think longer think more broad the performance continue improves and we're really excited about the next froner that what happen if we not only allow the model to think harder but also provide more tools just like how real
23:32humans to solve those problems for real humans we don't ask them to solve reman a hypothesis just with a piece of pen and paper no internet so with all the basic web browsing search engine and code interpreters that builds the foundations and the best reasoning model builds the foundations for the gro agent to come so today we're actually introducing a new product called Deep search that is the the first generation of our Gro agents that not just helping the engineers and researchers and scientists to do coding but actually help everyone to answer questions that you have dayto day it's a kind of like a Next Generation search engine that really help you to understand the
24:11universe so you can start asking question like for example hey when is the next Starship lunch day for example so let's try that if get the answer on the left hand side we see a high level progress bar essentially you know the model know just going to do one single search like the current R system but actually thought very deeply about hey what's the user intent here and what other the facts I should consider at the same time and how many different website I should actually go and read their content right so this can really save hundreds hours of everyone's Google time if you want to really look into certain topics and then on the right hand side
24:51you can see the bullet summaries of how the current model you know is doing what websites browsing what sources verifying and often time actually cross validate different sources out there to make sure the answer is actually correct before it's output final answer and we can you know at the same time fire up a few more queries how about you know you don't you're a gamer right so sure yeah so how about what are some of the best builds and most popular builds in the path Excel hardcore right hardcore League I if you can't technically just look at the hardcore ladder might be a fast way to figure it out yeah we'll see what model does and then we can also do you know
25:31something more fun for example how about like make a prediction about the marsh madness out there yeah so this is kind of a fun one where waren Buffett has a billion dollar bet if you can exactly match the I think the the the the sort of the entire winning tree of marsh Madness you can win a billion dollars from Warren Buffett so like it would be pretty cool if AI could help you win a billion dollars from Buffett that seems like a pretty good investment let's go yeah all right so now let's fire up the query and see what model does so we can actually go back to our very first one how about the buff it wasn't counting on this it's already done that's right okay so we got
26:13the result of the first one and model thought around 1 minute so okay so the key Insight here the knock Starship is going to be on 24 so or later so no earlier than February 24th it might be sooner so yeah so I think we can you know go down SC go down what what the model does so it does a little research on the flight 7 what happened got grounded and actually it look into the FCC filing you know from its data collections and then actually make the new conclusion that yeah if we continue scroll down right yeah so it makes the you know little table I think inside xai we often joked about the time to to the first table is the only you know latency that matters yeah so
27:03that's how the model make the inference and look up all the sources and then we can look into the gaming one so how about the right so for this particular one we look at hey the you know the build is light It's Kind all better so what they The Infernal is but if we go down so the surprising fact of all the other builds so it looking to the 12 classes yeah so we'll see that the Min build was pretty popular whenever the game first came out and now the the invokers of the world took over invoker monke invoker for sure yeah that's right yeah followed by the stor wavers and that's really good at mapping so yeah and then we can see the the match Madness how about that so one one
27:52interesting thing about the Deep search is that if you actually go into the panel where shows you know what are the subtasks you can actually click the bottom left of the spray and then in this case you can actually scroll through actually reading through the mind of grock what informations does the model actually think about are trustworthy what are not how does it actually cross valate different information sources so that makes the entire search experience and information retrieval process a lot more transparent to our users and this is much more powerful than any search engine out there you can literally just tell it only use sources from X you know I will
28:29try to respect that yeah and so it's much more steerable much more intelligent I mean it really should save you a lot of time so something that might take you half an hour or an hour of researching on the web or searching social media you can just ask it to go do that and and come back in 10 minutes later it's done an hours worth of work for you that's really what it comes down to exactly and maybe better than you could have done it yourself yeah think about you have INF of interns working for you now you can just fire up all the tasks and come back a minute later so this is going to interesting one so March M had not happened yet so I guess we have to follow up with a
29:04next live stream yeah it seems like pretty good like $40 might get you a billion dollars $40 subscription that's right I mean my work so yeah so when are the users going to have their hands on gr 3 yeah so the the good news is we've been working tirelessly to actually release all of these features that we've shown you the gr free base model with amazing chat capability that's really useful that's really interesting to talk to the the Deep search the advanced reasoning mode all of these things we want to roll them out to you today starting with the premium plus subscribers on X so it's the first group that will initially get access make sure to update your X app if
29:44you want to see all of the advanced capabilities because we just released the update now as we're as we're talking here and yeah if you're interested in getting early access to gr then sign up for premium plus and also we're announcing that we're starting a separate subscription for Gro that we call Super grock for those who are those real grock fans that want the most advanced capabilities and the earliest access to to new features so feel free to check that out as well this this is for the dedicated grock app and for the website exact so our our new website is called gro.com yeah and you'll also find you never guess yeah you never guess and you can also find our grock
30:22app in the IOS app store and that gives you like a more Pol even even more polished experience that's totally grock focused if you're if you want to have Gro easily available one Tap Away yeah the version on gro.com on you know on a web browser is going to be the the most the latest and most advanced version because obviously it takes us a while to get thing get something into an app and then get it approved by the app store so and then if something's on a phone format there's limitations what you can do so the most powerful version of Gro and the latest version will be the the web version at gro.com yeah so so watch out for the name grock free in
30:56the app dead giveaway yeah exactly that that's that's the giveaway that you have gr and if it says grr then gr hasn't quite arrived for yet but we're working hard to roll this out today and then to even more people over the the coming days yeah make sure you update your phone app too where you actually going to get all the tools we're showcase today with the thinking mode with the Deep search so yeah really looking forward to all the feedbacks you have yeah and I think we we should emphasize that this is kind of a beta like meaning that it's you should expect some imperfections at first but we will improve it rapidly almost every day in fact every day I think it'll get
31:35better so if you want a more polished version I'd like maybe wait a week but expect improvements literally every day and then we're also going to be providing a voice interaction so you can have conversational in fact I was trying it earlier today it's working pretty well but not we need these a bit more polish the the the sort of weight where we can just literally talk to it like you're talking to a person it's that's awesome it's actually I think one of the best experience of gr but that's that's probably about a week away yeah so with that said well I think we might have some audience questions sure yeah all right let take a look yeah let's take a look the the
32:15audience from the a platform yeah cool so the first question here is when grock voice assistant when is it coming out yeah as as as soon as possible just like Elon said just a little bit of polishing away from being to everybody obviously it's going to be released in an early form and we're going to rapidly iterate on that Y and the next question is like when will Gro 3 be in the API so this is coming in the the gro 3 API with both the reasoning models and deep search is coming your way in the coming weeks we're actually very excited about the Enterprise use cases of all these additional tools that now Gro has access to and how the test time compute
32:53and Tool use can actually really accelerate all the business use cases yeah another another one is Will voice mode be native or text to speech so I think that means is it going to be one one model that is understanding what you say and then talking back to you or is it going to be some system that has text to speech inside of it and the good news is it's going to be one model like a variant of gr free that we're going to release which basically understands what you're say what you're saying and then generates the audio no directly from that so very much like grf free generates text know that model generates audio and that has a bunch of advantages I was talking to it earlier
33:30today and it said hi Igor know reading my my name from probably from some text that it had and I said no no my name is Igor and it remembered that you know so it could continue to say Igor just like a human word and you can't achieve that with with Texas speech so on yeah so oh here's a question for you pretty spicy you is Gro a boy or a girl
33:53and how they sing gr is whatever you wanted to be yeah yeah are you single yes all right the shop is open so honestly people are going to fall in love with crcket since like 1,000% probable yeah the next question will Gro be able to transcribe audio into text yes so we'll have this capability both the app and also the API we found that's like gr should just be your personal assistant looking over your shoulder and follow you along the way learn everything you have learned and really help you to understand the world better become smarter every day yeah I mean The Voice M doesn't isn't simply it's not just voice text it understands like tone inflection pacing everything
34:35it's wild I mean it's like talking to a person okay yeah so any plans for conversation memory yeah absolutely we're working on it right now I already forgot that's right let's see what are the other ones so what about the you know the DM features right so if you have personalizations and if if you have you know Gro remembers your previous interactions yes should it be one Gro or multiple different grocs it's up to you you can have one Gro or many GRS y I suspect people will probably have more than one yeah I want to have a do Gro yeah the grock do that's right right
35:16cool so in the past we've open sourced gr one right so somebody's asking us are we going to do it again with gr tool yeah I think one once gr our general approach is that we will open source the last version when the next version is fully out so like when when gr 3 is mature and stable which is probably within a few months then we'll open source gr too mhm okay so we probably have time for one last question what was the most difficult part about working on this project I assume Gro 3 and what I most excited about so I think me looking back you know getting the whole model training on 100k H 100 coherently that's almost like battling against the final boss of the
36:01universe the entropy because any given time you can have a cosmic ray that beaming down and flip a bit in your transistor and now the entire grading update if it's fit mantisa bit the entire grading update is out of whack and now you have 100,000 of those and you have to orchestrate them every time any at at any given time any of gpus can go down yeah I mean it's worth breaking down like how were we able to get the world's most powerful training cluster operational within 122 days because we we started off we we actually weren't intending to do a data center ourselves we were going to just we we went to the data center providers and said how long would it take to have
36:42100,000 gpus operating coherently in a single location and we got time frames from 18 to 24 months so we're like well 18 to 24 months that means losing is a certainty so the only option was to do it do it ourselves so then if you break down the problem I guess I'm doing like reasoning here with like makes you think one single chain though yeah yeah exactly so well we needed a building we can't build a building so we must use an existing building so we we looked for for basically for factories that had been were that had been abandoned but the factory was in good shape like a company had gone bankrupt or something so we found an Electrolux Factory in memph in
37:22Memphis that's why it's in Memphis home of Alvis and also one of the oldest I think it was the capital of ancient Egypt and it was actually very nice Factory that I know for whatever reason that electrox had left and that that gave us shelter for the computers then we needed power the we needed at least 120 megawatt at first but the building only had 15 megawatts and ultimately for 200,000 me 200,000 gpus we needed a quarter gwatt so we initially leased a whole bunch of generat so we have generators on one side of the building just one trailer after trailer trailer of generators until we can get the utility power to to come in and then but then we also need cooling so on the
38:07other side of the building it was just trailer after trailer of of cooling so we leased about a quarter of the mobile cooling capacity of the United States on the one other side of the building then we need to get the gpus all installed and they're all liquid cooled so in order to achieve the density necessary this is a liquid cooled system so we had to get all the plumbing for liquid cooling nobody had ever done a liid cooling data center at scale so this was a incredibly dedicated effort by a very talented team to achieve that outcome I may think not now it's going to work nope the the issue is that the the power fluctuations for a GPU cluster are dramatic so it's it's
38:44like a a this giant Symphony that is taking place like a imag having a symphony with a 100,000 or 200,000 participants in the in the symphony and the whole Orchestra will go quiet and loud l in you know 100 millisecond and so this caused massive power fluctuation so then which then caused the generators to lose their minds and they they weren't expecting this so to buffer the power we then used Tesla megapacks to smooth out the power so the MEAP packs had to be reprogrammed so with with xai working with Tesla we reprogram the mega packs to be able to deal with these dramatic power fluctu fluctuation to smooth out the power the computers could actually run properly
39:29and that that worked was quite tricky and and then but even at that point you still have to make the computers all communicate effective so all the networking had to be solved and debugging a Brazillian network cables a debugging nickel at 4: in the morning I we solved it like roughly 4:20 a.m. yes was figured out like there's some well there were a whole bunch of issues well one there was like a bios mismatch the virus was not set up correctly yeah we had to D our lspci
40:04outputs between two different machines one that was working yeah one that was not working many many many other things I mean yeah exactly this would go on for a long time if we actually listed all the things but you know it's like interesting like it's not like oh we just magically made it happen you have to break down the problem just like Gro does for reasoning into the constituent elements and then solve each of the constituent elements in order to achieve a a coherent training cluster in a period of time that is a small fraction of what anyone else was could do it in and then once the training cluster was up and running and we could use it now we had to make sure that it
40:36actually stays healthy throughout which is his own giant Challenge and then we had to get every single detail of the training right in order to get a gr level model which is actually really really hard so we don't know if there are any other models out there that have gr's capabilities but whoever trains a model better than grye has to be extremely good at the the science of deep learning at every aspect of the engineering so it's it's not so easy to to pull this off and this is now going to be the last cluster we built and last Model we train oh yeah we've already we've already started work on the next cluster which will be yeah about five times the power so instead of
41:12a quar gaw roughly 1.2 GW what's the what's the Back to the Future War what's the power on you there like the Back to the Future car yeah don't anyway the Back to the Future power car it's it's like roughly in that order I think so and you know these will be the sort of the gb2 200/300 cluster it it once again will will be the most powerful training colle in the world so we're not like stopping here no and our reason model is going to continue improve by accessing more tools every day so yeah we're very excited to share any of the upcoming results with you all yeah the thing that keeps us going is basically being able to give gr free to you and
41:51then seeing the usage go up seeing everybody enjoy no gr that's that's what really gets us up in the morning morning so yeah thanks for tuning in thanks guys hey Gro what's up can you hear me I'm so excited to finally meet you I can't wait to chat and learn more about each other I'll talk to you soon
资源与出处
- 视频在 YouTube 收看
- 档案silicon-valley.tech 档案页本页视频与文稿数据来自 Elon Musk Archive
- 精读本篇精读 Markdown 版