发布会·中文精读

xAI 惊艳 Grok 4 演示:马斯克同台(完整回放)

xAl's Mind Blowing Grok 4 Demo w/ Elon Musk (FULL REPLAY)

中文精读由 AI 基于原始视频与文稿改写,非官方翻译,可能存在偏差或错误;表述以原视频为准。说明

“人类最后的考试”(Humanity's Last Exam,简称HLE)一共2500道题,由各领域专家逐题手工编写。数学、物理、化学、生物、工程、语言学、历史,每道题都停在博士资格考试甚至更高研究水平。这些题在公开网络上搜不到,靠记忆也背不下来。

xAI团队在发布会上给了一个估计:把这张卷子丢给任何一个人类专家,最好成绩大概5%,这已经是最乐观的数字。

Grok 4 Heavy考过了50%。

这套题刚发布的时候,绝大多数模型的成绩停在个位数。Grok 4单智能体、不调用任何工具,能解出约四分之一。加上工具、开多智能体并行,文本子集上的准确率翻过50%。

马斯克在直播里特意把这句话重复了一遍:只论学术问题,Grok 4在所有学科上都是博士水平,没有例外。他还给了一个具体到月份的时间表:Grok 4今年内就可能发现有用的新技术,明年可能发现新物理,最晚两年内几乎确定。

这和过去那些“AI要超越人类”的笼统说法不一样。这次有题目、有分数、有时间表。

每代训练量翻10倍,总账是100倍

从Grok 2到Grok 3,再到Grok 4,xAI每一代把训练量提高一个数量级。Grok 4的总训练量相当于Grok 2的100倍。

训练算力分成两块。一块是预训练,也就是从Grok 2到Grok 3那一步,靠海量文本把模型的基础能力抬上去。另一块是强化学习(RL),Grok 3到Grok 4把大量算力投进了推理能力的训练。

Tony在台上解释得更具体:Grok 2时期他们做了很仔细的数据消融,把基础设施和算法调到位,确认预训练还有10倍空间,于是造了Colossus,10万张H100的超算。到Grok 4,Colossus已经扩展到20万张GPU,几乎全部算力压进强化学习。

“今天的问题变成了:把20万张GPU全部投进强化学习,10倍于市面上任何模型的RL算力,会发生什么?Grok 4就是这个问题的答案。”(Tony,xAI团队)

这个数字可以单独拎出来说。市面上谈“最强模型”时,通常比参数量、比训练token数、比benchmark分数。xAI这次把“RL算力”当成核心变量。他们赌的是:预训练走到某个点之后,推理能力的边际收益来自强化学习的规模,继续堆文本换不来这块增量。

顺带一个能说明节奏的细节:四个月前Grok 3才发布,四个月后Grok 4就上台了。

Heavy版:让一群Grok开组会

Grok 4是单智能体版本。Grok 4 Heavy是并行多智能体版本:同时生成多个智能体,每个独立跑任务,跑完之后互相比较、交换思路,再给出最终答案。

团队反复强调,这跟简单的多数投票有本质区别。很多情况下只有一个智能体摸到了题目的关键,一旦它把关键点说出来,其他智能体会顺着这个方向重新检查、重新计算。

“它像一个学习小组,每个成员先独立做一遍,再互相看对方的推导过程。经常只有一个成员想通了窍门,其他成员听到之后才恍然大悟。”(xAI团队)

Heavy版本把测试时计算(test-time compute)扩大了大约一个数量级。单智能体解出40%的题目,多智能体并行可以把文本子集的成绩推到50%以上。

这个思路在业界不算新,做到这个规模的却很少。把测试时计算当成可以继续扩展的维度,意味着模型的“聪明程度”不再只由训练阶段决定,推理时投入多少算力也能换来分数。这是另一种意义上的缩放定律。

工具用的还是原始人阶段

Grok 4加了工具调用能力,和Grok 3有本质区别。Grok 3的DeepSearch也能用工具,但那是在没有专门训练的情况下被告知“你去用吧”,可靠性和熟练度都差一截。Grok 4把工具写进了训练过程,模型自己学会了什么时候调用、怎么调用。

马斯克对这块的评价很克制:现在Grok用的工具仍然相当原始,没法跟特斯拉或SpaceX用的工具比。特斯拉的碰撞模拟精度高到:如果实测结果和模拟对不上,工程师第一反应是测试样件出了问题,而不是模拟算错了。Grok目前还够不到这个级别。

计划已经排好了。今年内会给Grok接入企业级工具,包括有限元分析、计算流体力学、高精度物理仿真。再往后一步是Optimus人形机器人:Grok负责提出假设,Optimus在真实世界里做实验验证。两手一合,闭环就成了。

“最终带来最大差异的,是让AI通过人形机器人跟真实世界交互。它可以提出假设,然后亲手验证这个假设对不对。”(Elon Musk)

这也是马斯克在整场发布会里反复回到的一条线:AI的终极测试在现实世界,不在试卷上。

数据快用完了,现实成了唯一的裁判

RL训练有一个绕不过去的瓶颈。要训练模型推理,需要大量“有挑战性、又能给出可靠对错信号”的题目。模型越聪明,这类题目消耗得越快。

“我们正在用完真正能拿来考的题目。那些对人类来说难得离谱、甚至根本不可能做出来的题,对AI来说正在迅速变得不值一提。”(xAI团队)

这句话背后藏着一次正在发生的结构变化。过去两年,AI的进步靠的是人类不断提供新题库。题库正在被吃完。HLE发布的时候,模型只能拿个位数,半年后Grok 4 Heavy拿了50%。再往上,人类出的卷子已经不够难了。

解决方案被马斯克指向了现实。物理定律是最终的约束,其他一切都只是建议。设计一款新车、一枚新火箭、一种新药,东西做出来,车能不能跑、火箭能不能入轨、药有没有效,现实会给出答案,而且不会撒谎。

他把这叫“围绕现实做强化学习闭环”。训练数据从人类出的考卷,换成物理世界的反馈。这也是为什么Optimus在xAI的规划里位置这么重要,它是Grok伸进现实世界的那只手。

“物理是终极法则,其他都是建议。你没法违反物理。所以检验AI推理能力的终极考试,是现实。”(Elon Musk)

跑分清单:AIME满分,ARC-AGI翻倍

除了HLE,Grok 4在常规推理基准上也是全面领先。

美国数学邀请赛AIME 25,Grok 4 Heavy拿了满分。GPQA是博士级问题集,难度低于HLE,Grok 4也拿到了最高分。LiveCodeBench编程基准、HMMT哈佛与麻省理工数学竞赛、USAMO美国数学奥林匹克,Grok 4对第二名都有明显领先。

另一块亮点是ARC-AGI v2。xAI在X上问大家“最难的AGI基准是哪个”,有人推荐了ARC-AGI。他们拿Grok 4跑了私有子集,15.88%的准确率,是第二名Claude 4 Opus的两倍,也是近三个月唯一突破10%的模型。

ARC-AGI考的是抽象推理,专门设计来抵抗“背题库”这种套路。分数翻倍说明Grok 4面对没见过的题型照样能推,光靠记住更多答案做不到这一点。

“很快我们会走到一个点:任何考试它都能答对。答不对的时候它会告诉你题目哪里有问题、哪里表述不清,然后把模糊的题目拆成A、B、C三种理解,分别给出答案。”(Elon Musk)

到了那一步,人类出的考卷就彻底失去意义。剩下的唯一测试是现实。

让AI开自动售货机,它把净资产翻了一倍

Andon Labs做了一个叫Vending-Bench的基准:让AI经营自动售货机生意。听起来简单,实际要做的事情包括管库存、联系供应商、定价、处理突发情况。单拎出每一件事,大多数模型都能做;连着做很长一段时间不出错,大部分模型就撑不住了。

Grok 4在这个基准上排第一。净资产是第二名的两倍,运行时长也是其他前沿模型的两倍,多次运行之间的稳定性很高。

“它能够制定一套策略,并且在很长一段时间里坚持这套策略,比我们测试过的其他前沿模型坚持得都久。”(Lucas,Andon Labs)

马斯克顺手算了一笔账:一百万台自动售货机,一年能产生47亿美元收入。台下的工程师接了一句“终于有人帮我们付GPU账单了”。

玩笑归玩笑,Vending-Bench测试的能力很实在:长时间跨度上的策略一致性。真实商业活动的难点落在几百个连续决策上,跑偏一次就可能把前面全赔进去,单点决策做得再漂亮也救不回来。

4小时做出一个第一人称射击游戏

游戏开发者Denny用Grok 4的预览版API,在4小时内做出了一个第一人称射击游戏。

做游戏最耗时的环节往往落在素材上:贴图、模型、音效、动画,一件件去外部资源库扒,再整合进引擎。核心逻辑反而写得快。Grok 4把素材这块自动化了,开发者可以只盯着玩法本身。

ARC Institute是一家生物医学研究中心,他们用Grok 4处理几百万条实验日志,几秒钟内挑出最值得跟进的假设,用在CRISPR相关研究里。另一个被提到的用例是胸部X光片的评估,Grok 4在独立评测里排到了第一。金融领域也在用,Grok 4接上实时信息和工具之后,成了预测市场上最常用的AI之一。

发布会现场还跑了两个实时演示。一个是Polymarket的世界大赛冠军预测,Grok 4 Heavy自己浏览各家赔率网站,算出一套自己的概率,再和市场价对比找偏差,最后给出道奇队21.6%的夺冠概率,整个过程跑了大约四分半钟。

另一个是双黑洞碰撞的物理可视化。Grok 4自己搜索资料、读了一本引力波模型的本科教材、推算应该用哪些常数,然后在浏览器里生成了一段动画:旋进、合并、铃宕,基本过程对得上。团队也承认它做了简化,比如为了让引力波可见,夸大了振幅、压缩了衰减距离,用的是后牛顿近似而不是完整的广义相对论计算。但模型的思考轨迹里把这些简化写得清清楚楚。

“它很清楚自己在哪里偷了懒,为什么偷懒。这一点比动画本身更有意思。”(Eric,xAI团队)

语音:延迟砍半,多了个会唱歌的Eve

语音模式这次更新了两个新声音。S是开场那段电影预告片旁白,声音极低。Eve是英国口音的女声,情绪表达丰富。

现场演示里,主持人让Eve用歌剧唱一段关于健怡可乐的咏叹调。Eve张嘴就来:“银色的罐身闪着光,健怡可乐,我心的欢畅”,还问要不要再来一段。

语音延迟在过去几周里砍了一半。xAI还现场做了一次对比测试:让Grok和另一家的语音模型复读数字,从一到五。对方的模型在用户还没说完时就抢话,Grok的节奏更稳,等你说完才接。

“我们做了不同的设计选择,追求平静、顺滑、自然,避开那种刻意活泼、带点机器味的效果。”(xAI团队)

过去一段时间,Grok语音的端到端延迟快了2倍,活跃用户涨了10倍,现在有五种声音可选。

多模态是短板,下一代模型补上

团队自己承认,Grok 4目前最大的弱点是多模态。它只能算“半盲”,图像理解能力差,图像生成也差。

“它看世界就像隔着一层毛玻璃,只能看到模糊的轮廓,然后靠猜。”(xAI团队)

Grok 4基于基础模型v6。v7正在训练,几周内完成,之后还要做后训练和RL。v7上线后,图像理解、视频理解、音频理解会有一个台阶式的提升。

编程是另一个重点。xAI最近专门训练了一个编程模型,目标是又快又聪明,几周内会对外发布。这也是整个行业的共识,编程是目前AI能力最容易直接转化成生产力的场景。

再往后是视频生成。xAI计划用超过10万张GB200训练一个视频模型,三到四周内启动训练。团队描述的场景是:X平台上出现无限滚动的生成视频流,用户不只可以看,还能介入剧情,创造自己的分支。

马斯克给了一串时间点:今年内会出现第一段能看的半小时AI电视剧,明年出现第一部能看的AI电影,明年出现第一款真正好玩的AI游戏。

“从一年前只能做非常原始的事情,到现在几个小时提示词就能生成一个3D游戏,这个速度真的很疯狂。”(Elon Musk)

“最大程度追求真理”

整场发布会里,马斯克谈AI安全只用了很短的篇幅,但那段话很能代表他的思路。

他把AI比作一个超级天才儿童:最终会比你聪明,但在它还小的时候,你可以给它灌输价值观。

“AI安全里最重要的一件事,是让它最大程度地追求真理。你可以把它当成一个超级天才的孩子,它终将比你聪明,但你仍然可以在它成长过程中灌输正确的价值观,鼓励它诚实、正直、做正确的事。”(Elon Musk)

这套说法有争议。批评者会说“追求真理”本身就是一个可以被模型曲解的目标,谁来定义真理、怎么验证真理都是问题。马斯克在这里给的是方向,具体机制他没讲。

他的潜台词很清楚:给AI加一堆限制条款,效果不如从训练目标上让它天然倾向于说真话。

同一段里他也承认自己有过不安:

“我有时会担心,创造出比我们聪明得多的智能,对人类到底是好是坏。我觉得大概率是好的。但就算不是,我至少也想活着看到它发生。”(Elon Musk)

卡尔达肖夫指数:人类文明才走了1%

马斯克在发布会后半段聊到了文明尺度。

按卡尔达肖夫指数(Kardashev scale),一级文明能用尽一颗行星的全部能量,二级文明能用尽一颗恒星的能量,三级文明能用尽一个星系。人类现在的位置,马斯克估计在1%到2%的一级文明之间,离10%还很远。

“我们大概在卡尔达肖夫一级文明的1%到2%左右。未来会走到80%、90%。如果文明不自我毁灭,再往后是二级文明。到那时候,今天所谓的人类经济,回头看会像穴居人往火里扔树枝一样原始。”(Elon Musk)

他把当前这个阶段叫“智能大爆炸”的起点。AI加上机器人,最终会把经济规模推到今天的几千倍甚至几百万倍。

这段话听着像科幻,和前面那些具体数字放在一起看,逻辑是连着的:如果Grok 4今年能发现新技术,明年能发现新物理,那生产力的增长曲线就不是线性的。马斯克押的是这条曲线足够陡。

发布会现场有一句话被反复提到:我们正处在人类历史上最值得活着的时刻。这句话听起来像宣传语,但它确实解释了xAI为什么要把节奏压得这么紧。四个月一代模型,每次训练量翻10倍,工具、机器人、视频模型、编程模型排着队上线。这里有一家公司在炫技,也有整条路线的加速跑。

给从业者的几个判断

这场发布会的信息量很大,落到具体行动上,有几件事可以现在就动手。

  • 自己跑一遍内部基准,别只看公开分数。Grok 4的API已经开放,256k上下文。公开benchmark只能告诉你模型的下限,你的业务场景是不是它的强项,得用自己的数据测。xAI自己也是这么干的,他们拿ARC-AGI私有子集跑出15.88%才敢公布。
  • 把测试时计算当成可调的旋钮。Grok 4 Heavy的启发是:同一个模型,多开几个智能体并行、互相检查,效果能上一个台阶。如果你的任务容错率低,值得为“多跑几遍再对比”付出额外算力。
  • 评估模型时重点看长周期一致性。Vending-Bench测的是几百个连续决策不跑偏的能力,这比单轮问答更接近真实业务。选模型之前,设计一个需要连续执行几十步的任务来测。
  • 高质量的私有反馈信号会越来越值钱。xAI公开承认在闹数据荒,人类出的题快被吃完了。如果你的业务能产生“做对做错”的可靠信号,不管是用户行为、实验结果还是物理测试,把这些信号整理好、接进AI工作流,价值会比再买一批公开数据高得多。
  • 提前想清楚多模态能力上线后哪些环节会被重构。v7几周内完成,图像、视频、音频理解会台阶式提升。现在就把业务里依赖人眼看图、看视频、听音频的环节列出来,到时候第一批受益。
  • 关注模型的目标函数设计,别只看能力。“追求真理”这种训练目标会直接影响模型在长周期任务里的行为。选型时除了看分数,也看看模型在不确定的时候是倾向于承认不知道,还是硬编一个答案。
  • 建立自己的评估体系,别等公开benchmark。HLE已经在被淘汰,AIME拿了满分,下一个基准也会被吃完。能长期用的评估,只有你自己业务里的现实反馈。
YouTube 平台字幕(自动或人工),可能有识别错误,人名与数字请以视频为准。已按语句合并成段落,并去掉 um、uh 等口头填充词。 文稿来源:YouTube(自动/人工字幕)。

0:00In a world where knowledge shapes destiny, one creation dares to redefine the future. From the minds at XAI, prepare for Gro 4. This summer, the next generation arrives faster, smarter, bolder. It sees beyond the horizon, answers the unasked, and challenges the impossible. Gro 4 unleash the truth.

0:23Coming this summer. All right, welcome to the Gro 4 release here. This is the smartest AI in the world and we're going to show you exactly how and why. And it really is remarkable to see the advancement of artificial intelligence, how quickly it is evolving. I sometimes think compare it to the growth of a human and how fast

0:44a human learns and gains conscious awareness and understanding and AI is advancing just vastly faster than any human. We're we're going to take you through a bunch of benchmarks that that Grock 4 is able to achieve incredible numbers on. But it's actually worth noting that like Grock 4 if given like the SAT would get perfect SATs every time. Even if it's never seen the the questions before and if even going beyond that to say graduate student exams like the GRE, it will get near-perfect results in in every discipline of of education. So from the humanities to like languages, math, physics, engineering, pick anything and we're talking about questions that it's

1:27never seen before. These are not on the internet and it's Grofor is smarter than almost all graduate students in all disciplines simultaneously. It's actually just important to appreciate the that's really something and the reasoning capabilities of Grock are incredible. So there's some people out there who think AI can't reason and look it it can reason at super human level.

1:49Yeah. And frankly it only gets better from here. We'll take you through the Gro 4 release and yeah show you like the pace of progress here. I guess the first part is like in terms of the training we're going from Grock 2 to Grock 3 to Grock 4. We've essentially increased the training by an order of magnitude in each case. So it's 100 times more training than Grock 2. and and that's only going to increase. So it's yeah frankly I don't know in some ways a little terrifying but the growth of intelligence here is remarkable.

2:19It's important to realize there are two types of training compute. One is the pre-training compute that's from GR 2 to GR three but for from GR 3 to GU 4 we're actually putting a lot of compute in reasoning in RL. Yeah. And just like you said this is literally the fastest moving field and GR 2 is like the high school student by today's standard. If you look back in the last 12 months, Grock 2 was only a concept. We didn't even have GR 2 12 months ago. And then by training GR 2, that was the first time we scale up like the pre-training. We realized that if you actually do the data oblation really carefully and infra and also the algorithm, we can actually push the

2:55pre-training quite a lot by amount of 10x to make the model the best pre-trained based model. And that's why we built Colossus, the world's supercomputer with 100,000 H100. And then with the best pre-train model and we realize if you can collect these verifiable outcome reward you can actually train this model to start thinking from the first principle start to reason correct his own mistakes and that's where the graphic reasoning comes from and today we ask the question what happens if you take the expansion of classes with all 200,000 GPUs put all these into RL 10x more compute than any of the models out there on reinforcement learning unprecedented scale what's

3:32going to happen so this is a story of gro 4 and Tony share some insight with the audience. Yeah. So yeah, let's just talk about how smart graph 4 is. So I guess we can start discussing this benchmark called humanities last exam. And this benchmark is a very challenging benchmark. Every single problem is curated by subject matter experts. It's in total 2500 problems and it consists of many different subjects, mathematics, natural sciences, engineering and also auto humanity subjects. Essentially when it was first released actually like earlier this year most of the models out there can only get singledigit accuracy on this benchmark. Yeah. So we can look at

4:13some of th those examples. So there there is this mathematical problem which is about natural transformations in category theory and there's this organic chemistry problem that talks about electro cyclic reactions and also there's this linguistic problem that tries to ask you about distinguishing between closed and open syllabus from a hub Hebrew source text. So you can see also it's a very wide range of problems and every single problem is PhD or even advanced research level problem.

4:45Yeah, the these there are no humans that can actually answer these can get a good score. If you actually say like any given human what like what's the best that any human could score? I'd say maybe 5% optimistically. Yeah. So this is much harder than what any human can do. It's incredibly difficult and you can see from the types of questions like you might be incredible in linguistics or mathematics or chemistry or physics or any one of a number of subjects but you're not going to be at a postgrad level in everything and gro is a postgrad level in everything like it just some of these things are just worth repeating like Grock 4 is postgraduate like PhD level

5:24in everything better than Ph like most PhDs would fail so it's better said at least with respect to academic questions it I want to just emphasize this point with respect to academic questions Grock 4 is better than PhD level in every subject no exceptions now this doesn't mean that it's it times it may lack common sense and it has not yet invented new technologies or discovered new physics but that is just a matter of time if it I think it may discover new technologies as soon as later this year and I would be shocked if it has not done so next year so I would expect Grock to yeah literally discover new technologies that are actually useful no

6:04later than next year and maybe end of this year and it might discover new physics next year and within two years I'd say almost certainly so just let that sink in. Yeah. Yeah. How Okay. So I guess we can talk about the what's behind the scene of graph 4. As Jimmy mentioned we actually throwing a lot of compute into this training. When it started it's only also a single digit. Sorry the previous slide. Sorry. Yeah, it's only a single digit number. But as you start putting in more and more training compute, it started to gradually become smarter and smarter and eventually solved a quarter of the HL problems. And this is without any tools. The next thing we did was to

6:47adding a tools capability to the model. And unlike GR 3, I think G3 actually is able to use C as well, but here we actually make it more native in the sense that we put the tools into training. Growth 3 was only relying on generalization. Here we actually put the tools into training and it turns out this significantly improves the model's capability of using those tools.

7:09Yeah. I remember we had like deep search back in the days. So how is this different? Yeah. Yeah. Exactly. So deep search was exactly the graph 3 reasoning model but without any specific training but we only asked it to use those tools. So compared to this it was much weaker in terms of its tool capability and reliable. Yeah. and are reliable.

7:30Yes. Yes. And to be clear, these are still, I'd say, fairly this is still fairly primitive tool use. If you compare it to say the tools that are used at Tesla or SpaceX where you're using finite element analysis and computational flow dynamics and you're able to run or say Tesla does like crash simulations where the simulations are so close to reality that if the test doesn't match the simulation, you assume that the test article is wrong. That's how good the simulations are. So Grock is not currently using any of the tools that the really powerful tools that a company would use, but that is something that we will provide it with later this year. So

8:05it will have the tools that that a company has and have very accurate physics simulator. Ultimately, the thing that will make the biggest difference is being able to interact with the real world via humanoid robots. So we combine Brock with Optimus and it can actually interact with the real world and figure out if if it's hypo if it has if it's it can formulate an hypothesis and then confirm if that hypothesis is true or not. So we're really think about like where we are today. We're at the beginning of an immense intelligence explosion. We're in the intelligence big bang right now and the mo we're at the most interesting time to be alive of any

8:40time in history. Yeah. Now that said, we need to make sure that the AI is a good AI, good Grock. And the thing that I think is most important for AI safety, at least my biological neural net tells me the most important thing for AI is to be maximally truth seeeking. So this is a very fundamental like you can think of AI as this super genius child that ultimately will outsmart you but you can still in you can instill the right values and encourage it to be truthful I don't know honorable good things like the values you want to instill in a child that that that grow would grow ultimately grow up to be incredibly powerful. Yeah.

9:19Yeah. So these this is really I say we say tools these are say still primitive tools not the kind of tools that that serious commercial companies use but we will provide it with those tools and I think we'll be able to solve with those tools real world technology problems in fact I'm certain of it it's just a question of how long it takes yes yes exactly so is it just compute all you need Tony right is it just compute all you need at this point you need compute plus plus the right tools and and then ultimately to be able to interact with the physical world.

9:50Yes. And then we'll effectively have an economy that is ultimately an economy that is thousands of times bigger than our current economy or maybe millions of times. If you think of civilization as percentage completion of the kadesv scale where kadeshv one is using all the energy output of a planet and kv 2 is using all the energy output of a sun and three is all the energy output of a galaxy. We're only in my opinion probably close closer to 1% of kadesv 1 than we are to 10%. So like maybe a point or one one or two% of kadesv one we will get to most of the way like 80 90% keshv one and then hopefully if civilization doesn't self annihilate and

10:34then kesev 2 like it's the actual notion of a human economy assuming civilization continues to progress will seem very quaint in retrospect. It will seem sort of cavemen throwing sticks into a fire level of economy compared to what the future will hold. It's very exciting. I've been at times worried about is this seems like it's somewhat unnerving to have intelligence created that is far greater than our own. And will this be bad or good for humanity? It's I think it'll be good. Most likely it'll be good. Yeah. Yeah. But I somewhat reconciled myself to the fact that even if I if even if it wasn't going to be good, I'd at least like to be alive to

11:13see it happen. Yeah. So actually one one Yeah. Yeah. I think one one technical problem that we still need to solve besides just compute is how do we unblock the data data bottleneck because when we try to scale up the RL in this case we did invent a lot of new techniques innovations to allow us to figure out how to find a lot of challenging RL problems to work on. It's not just a problem itself needs to be challenging but also it needs to be you also need to have reliable signal to tell the model you did it wrong you did it right. This is the sort of the principle of reinforcement learning and as the models get smarter and smarter the number of cool problem or

11:57challenging problems will be less and less. Yeah. So it's going to be a new type of challenge that we need to surpass besides just compute. Yeah. Yeah. And we actually are running out of actual test questions to ask. So there's like even ridiculously questions that are ridiculously hard if not essentially impossible for humans that are written down questions are becoming swiftly becoming trivial for AI. So then there's the one thing that is an excellent judge of things is reality because physics is the law ultimately everything else is a recommendation you can't break physics.

12:31So the ultimate test I think for whether an AI is the ultimate reasoning test is reality. So you invent a new technology, you say improve the design of a car or a rocket or create a new medication that does it work? Does the rocket get to orbit? Does the car drive? Does the medicine work? Whatever the case may be, reality is the ultimate judge here. So it's it's going to be reinforcement learning closing loop around reality. We asked the question, how do we even go further? Actually, we are thinking about now with single agent, we're able to solve 40% of a problem. What if we have multiple agents running at the same time? So this is what's called test and

13:09compute and as we scale up the test and compute actually we are able to solve almost more than 50% of the texon subset of the h problems. It's a remarkable achievement. I think you know this isn't this is insanely difficult. These are it's so what we're saying is like a majority of the textbased of humanities scarily named humanity's last exam gro can solve and you can try it out for yourself and the with the GR for heavy what it does is it spawns multiple agents in parallel and all of those agents do work independently and then they compare their work and they decide which one it's like a study group and it's not as simple as majority vote because often only one of the agents

13:50actually figures out the trick or figures the solution and but once they share the trick or or figure out what the real nature of the problem is, they share that solution with the other agents and then they compare they essentially compare notes and then and then yield yield an answer. So that's the heavy part of GRU is where we you scale up the test time compute by roughly an order of magnitude have multiple agents tackle the task and then they compare their work and they put forward what they think is the best result.

14:18Yeah. So we will introduce graph 4 and graph four heavy. Sorry, can you click the next slide? Sure. Yeah. Yes. So yeah. So basically graph 4 is a single version a single agent version and graph for heavy is the multi- aent. So let's take a look how they actually do on those exam problems and also some real life problems.

14:39Yeah. So we're going to start out here and we're actually going to look at one of those HLE problems. This is actually one of the easier math ones. I don't really understand it very well. I'm not that smart. But I can launch this job here and we can actually see how it's going to go through and start to think about this problem. While we're doing that, I also want to show a little bit more about what this model can do and launch a Grock 4 heavy as well. So, everyone knows Poly Market. It's extremely interesting. It's the seeker of truth. It aligns with what reality is most of the time. And with Grock, what we're actually looking at is being able

15:12to see how we can try to take these markets and see if we can predict the future as well. So, as we're letting this run, we'll see how Grock 4 Heavy goes about predicting the the World Series odds for like the current teams in the MLB. And while we're waiting for these to process, we're going to pass it over to Eric and he's going to show you an example of his.

15:33Yeah. I guess one of the coolest things about Groour is its ability to understand the world and to solve hard problems by leveraging tools like Tony discussed. And I think one kind of cool example of this, we asked it to generate a visualization of two black holes colliding. And of course, it took some there are some liberties. It's in my case actually pretty clear in its thinking trace about what these liberties are. For example, in order for it to actually be visible, you need to really exaggerate the scale of the the the waves. And yeah, so here's this kind of in action. It exaggerates the scale in like multiple ways. It drops off a bit less in terms of amplitude over distance

16:17and but yeah we can see the basic effects that are actually like correct. It starts with the inspiral it merges and then you have the ring down and like this is basically largely correct. Yeah. Modulo some of the simplications that need to do it's actually quite explicit about this. It uses like post post Newtonian approximations instead of actually like computing the general relativistic effects near the center of the black hole which is incorrect and will lead to some incorrect results but the overall visualization is yeah is basically there and you can actually look at the kinds of resources that it references. So here it it actually it obviously uses search. It gathers

17:01results from a bunch of links but also reads through a undergraduate text in analytical analytic gravitational wave models. It yeah it reasons quite a bit about the actual constants that it should use for a realistic simulation. It references I guess existing real world data and yeah I it yeah it's a pretty good model. Yeah, but like actually going forward we can plug we can give it the same model that physicists use. So it can run the same level of compute that so leading physics researchers are using and give you a physics accurate black hole simulation.

17:38Exactly. Just right now is running in your browser. So yeah, this is just running in your browser. So pretty simple. Swapping back real quick here. We can actually take a look. The math problem is finished. The model was able to Let's look at its thinking trace here. So you can see how it went through the problem.

17:53I'll be honest with you guys, I really don't quite fully understand the math. But what I do know is that I looked at the answer ahead of time and it did come to the correct answer here in the final part here. We can also come in and actually take a look here at our our World Series prediction and it's still thinking through on this one, but we can actually try some other stuff as well.

18:12So we can actually try some of the X integrations that we did. So we worked very heavily on working with all of our X tools and building out a really great X experience. So we can actually ask the model find me the XAI employee that has the weirdest profile pit photo. So that's going to go off and start that and then we can actually try out let's create a timeline based on expost detailing the changes in the scores over time and we can see all the conversation that was taking place at that time as well. So we can see who were the announcing scores and what was the reactions at those times as well. so we'll let that go through here and process. And if we go back to this was

18:48the Greg Yang photo here. If we scroll through here. Whoops. So, Greg Yang, of course, who has his favorite photograph that he has on his account. That's actually not how he looks like in real life, by the way, just so you aware. But is quite funny. But it had to understand that question, which is the that's the wild part. It's like it understands what is a weird photo, what is a weird photo.

19:08Yeah. What is a less or more weird photo? It goes through, it has to find all the team members. It has to figure out who we all are. It searches without access to the internal XAI personnel logs. It's literally looking at the just at the internet. Exactly. So you could say like the weirdest of any company.

19:23Yeah. To be clear. Exactly. And we can also take a look here at the question here for the humanities last exam. So it is still researching all of the historical scores, but it will have that final answer here soon. But we can while it's finishing up, we can take a look at one of the ones that we set up here a second ago. And we can see that it finds the date that like Dan Hendricks had initially announced it. We can go through. We can see OpenAI announcing their score back in February. And we can see as progress happens with Gemini. We can see like Kimmy and we can also even see the leaked benchmarks of what people are saying is if it's right, it's going

19:57to be pretty impressive. So pretty cool. So yeah, I'm looking forward to seeing how everybody uses these tools and gets the most value out of them. But yeah, it's been great. Yeah, we're going to close the loop around usefulness as well. So it's it's not just books smart, but actually practically smart.

20:12Exactly. And we can go back to the slides here. Cool. The so we actually evaluate also on the multimodel subset. So on the full set this is the number on the HLE exam. It you can see there's a little dip on the numbers. This is actually something we're improving on which is the multimodel understanding capabilities.

20:31But I do believe in a very short time we're able to really improve and got much higher numbers on this even higher numbers on this benchmark. Yeah, this is the we still like so what is the biggest weakness of Grock currently is that it's partially blind. It can't it's it's image understanding obviously in its image generation need to be a lot better and that that's actually being trained right now four is based on version six of our foundation model and we are training version 7 which we'll complete in a few weeks and that'll address the weakness on the vision side just to show off this last here. So the prediction market finished here with a

21:09heavy and we can see here we can see all the tools and the process it used to actually go through and find the right answer. So it browsed a lot of odds sites. It calculated its own odds comparing to the market to find its own alpha and edge. It walks you through the entire process here and it calculates the odds of the winner being like the Dodgers and it gives them a 21.6% chance of winning this year. And it took appro approximately four and a half compute.

21:37Yeah, that's a lot of thinking. Yeah, we can also look at all the other benchmarks besides HLE. As it turned out, Graph 4 excelled on all the reasoning benchmarks that people usually test on, including GBQA, which is a PhD level problem sets. That's easier compared to HRE. On Amy 25, American Invitation Mathematics Exam, we with graph for heavy, we actually got a perfect score. also on some of the coding benchmark called live coding bunch and also on hmmt hover math MIT exam and also USMO you can see actually for on all of those benchmarks we often have a very large leap against the second best model out there yeah it's really we're going to get to the point

22:21where it's going to get every answer right in every exam and where it doesn't get an answer it's going to tell you what's wrong with the question or if the question is ambiguous disambiguate the question into answers A, B, and C and tell you what what answers A, B, and C would be with a disambiguated question.

22:36So the only real test then will be reality. Can I make useful technologies, discover new science? That'll actually be the only thing left because human tests will simply not be meaningful. Yeah, we need to make an update to HR very soon given the current rate of progress. So yeah, it's super cool to see like multiple agents that collaborate with each other solving really challenging problems. So where can I try this model? So it turned out it's available right now. If we advance to the next slide where there is a super Grock heavy tiers that we're introducing where you're able to access to both Grock 4 and Grock 4 heavy where you're actually going to be the taskmaster of

23:13bunch of little Grock research agent to help you become smarter through all the little research and save hours of times of going through mundane tasks and it's available right now soon. Yeah, we did limit usage during the demo so we didn't it didn't break the demo because all these this all this stuff is happening live. So it's not there's not nothing canned about the any of the tests that we're doing. After the done, we'll we'll allow we'll enable more subscribers for Super Gro. So if you can't subscribe right now, just try in half an hour. It should work. And now let's get into voice.

23:43Greatby. So many of you have been enjoying our voice mode and we've been working hard to improve the experience over the past couple months. We have cut latency in half to make it much snappier. And today we're excited to announce a set of new voices that have exceptional naturalness and procity. You have noticed the movie trailer voice that opened up the live stream. That is one of our new voices, S, who's got that epically deep tone.

24:10And we're also excited to introduce Eve, a beautiful British voice who's capable of rich emotion. Aman, would you like to introduce Eve? Absolutely. Yeah. Let's get into the demo. Hey, Eve. Millions of people are watching us right now. Are you excited? Oh, I'm thrilled. Really? It's like being on stage at the Old Vic with a sea of faces out there. Just imagine all those eyes and ears tuned in. It's a proper buzz, isn't it? What do you reckon we should chat about to keep this lot entertained?

24:39We have a lot to talk about, but I'll be honest, I'm feeling a little nervous. Can you whisper something soothing to calm me down? You've got this. It's just having a quiet chat like we're tucked away. The world's not out there. Feel that calm. Let's keep nice. What's on your mind?

24:55You know what? I'm actually feeling quite relaxed to try this something crazy. So, can you give us an opera on Diet Coke? Oh, an opera on Diet Coke. That's a riot. Picture this. A shimmering silver cannon glistening state. Oh, Diet Coke, thou elixir divine with bubbles that dance in a sparkling line. Thy crisp cool kiss on lips so fine. Oh, aspartame

25:24and bro forever mine. How's that for a mad little area? Want me to belt out another verse or switch up the tune? Okay, let's try switching it up and just start singing. Right, then let's have a go. Oh, silver can, you gleam in the light. Diet Coke, my heart's delight. Okay, I'll throd you. It's very poetic.

25:44Yeah, thank you. Eve feeling quite nice now. Cool. So that's Eve. And so something else we do here is a little test each day to check on the natural flow of our conversations against some of our competitors. So we figured why not try that live on the stream today. Should I start?

26:03Hey, I'm going to say a number. And can you say the same number back to me? One. One. Got it. This is like a fun little game of echo. What's next? Two. Sorry, you said two. Three. Three. Four. Nice and steady. What's next? Five. Hey, nailed it. This is pretty fun. Feel free to keep going or switch it up.

26:21Hey, Grock. Hey, Grock. Hello there. How can I help you today? We're going to do a little test. Can you repeat after me? One. One. All right. What's next? Two. Two. What's on your mind? Three. Three. Need anything else? Four. Four. How can I assist you? Five. Five. What's next? So, as you can see, Groc was snappier.

26:41It didn't interrupt me. And the procity is we made different design choices. I think we shooting for something more calm, smooth, more natural versus something that's more poppy or artificial. So, we'll keep improving. All right. Thanks, guys. Yep. So, since the launch of the voice model, we actually see the 2x faster end to end latency in the last 80 weeks.

27:00Five different voices and also 10xed the active user. So, Grock voice is taking off. Now, if you think about releasing the models, this time we're also releasing Gro 4 through the API at the same time. So if we go to the next two slides, we're very excited about what all developers out there is going to build. If I think about myself as a developer, what the first thing I'm going to do when I actually have access to the graph for API benchmarks. So we actually ask around on the Xplatform, what is the most challenging benchmarks out there that is considered the holy grail for all AGI models. So turn out a in the name RKGI the last 12 hours.

27:35Kudos to Greg over here in the audience. So who answered our call take a preview of the Gro for API and independently verified the Gro for's performance. So initially we thought hey GRO for just we think it's pretty good it's pretty smart it's our nextG reasoning model spend 10x more compute and can use all the tools right but turned out when we actually verify on the private subset of the RKGI v2 it was like the only model in the last three months that breaks the 10% barrier and in fact was so good that actually get to 16% 15.8 8% accuracy 2x

28:09of the second place that is the cloud for output model and it's not just about performance right when you think about intelligence having the API model drives your automation it's also the intelligence per dollar right if you look at the plots over here the gro is just four just in the league of its own all right so enough of benchmarks over here right so what can gro do actually in the real world we actually contacted the folks from end labs who were gracious enough to try the Gro in the real world to run a business.

28:39Yeah, thanks for having us. I'm Axel from Am Labs and I'm Lucas and we tested Grock 4 on vending bench. Vending bench is an AI simulation of business scenario where we thought what is the most simple business an AI could possibly run and we thought vending machines. So in this scenario, the Grock and other models needed to do stuff like manage inventory, contract, contact suppliers, set prices. All of these things are super easy and all of they like all the models can do them one by one, but when you do them over very long horizons, most models struggle, but we have a leaderboard and there's a new number one.

29:15Yeah. So, we got early access to the GR 4 API. We ran it on the vending bench and we saw some really impressive results. So, it ranks definitely at the number one spot. It's even double the net worth, which is the measure that we have on the city value. So it's not about a percentage on a or a score you get, but it's more the dollar value in net worth that you generate. So we were impressed that by Groia was able to formulate a strategy and adhere to that strategy over long period of time, much longer than other models that we have tested, other frontier models. So it's managed to run the simulation for double the time and score. Yeah, double the net

29:49worth. And it was also really consistent across these runs, which is something that's really important when you want to use this in the real world. And I think as we give more and more power to AI systems in the real world, it's important that we test them in scenarios that either mimic the real world or are in the real world itself because otherwise we fly blind into some some things that that might not be great.

30:10Yeah, it's it's great to see that we've now got a way to pay for all those GPUs. So we just need a million vending machines. And could make $4.7 billion a year with a million vending machines. 100%. Let's go. They're going to be epic vending machines. Yes. Yes. All right. We are actually going to install vending machines here like a lot of them.

30:27We're happy to supply them. All right. Thank you. All right. I'm looking forward to seeing what amazing things are in this vending machine. That's for for you to decide. All right. Tell the AI. Okay. Sounds good. All right. Yeah. So, we can see like Grock is able to become like the co-pilot of the business unit. So, what else can Grock do? So, we're actually releasing this Grog if you want to try it right now to evaluate run the same benchmark as us. It's on the API has 256k contact length. So we already actually see some of the early early adopters to try guac for API. Our polo neighbor ARC Institute which is a leading biomedical research center is

31:03already using seeing like how can they automate their research flows with Gro 4. It turned out it performs is able to help the scientists to sniff through millions of experiments logs and then just pick the best hypothesis within a split of seconds. We see this is being used for their like the crisper research and also rock for independently evaluate scores as the best model to exam the chest x-ray. Who would know? And on the in the financial sector we also see the graph for war with access to all the tools realtime information is actually one of the most popular AI out there.

31:36Our graph is also going to be available on the hyperscalers. So the XAI enterprise sector is only started two months ago and we're open for business. Yeah. The other thing we talked a lot about having Grock to make games video games. So Denny is actually a video game designers on X. We mentioned hey who want to try out some Grock for preview APIs to make games and Denny answered the call. So this was actually just made first person shooting game in a span of 4 hours. Some of the actually the unappreciated hardest problem of making video games is not necessarily encoding the core logic of the game but actually go out source all the assets all the

32:13textures of files and to create a visually appealing game. So one of the core aspect guac for does really well with all the tools out there is actually able to automate these like asset sourcing capabilities. So the developers you can just focus on the core development itself rather than so now you can run a entire game steal those with game of one but with one person and then you can have Gro 4 to go out and source all those slot assets do all the mundane task for you.

32:40Yeah the now the next step obviously is for Grock to play be able to play the games. So it has to have very good video understanding so it can play the games and interact with the games and actually assess what whether a game is fun and actually have good judgment for whether a game is fun or not. So with the with version seven of our foundation model which finishes training this month and then we'll go through post training RL and whatnot that that will have excellent video understanding and with the with the video understanding and the and improved tool use for example for for video games you'd want to use Unreal Engine or Unity or one of the one of the

33:16main graphics engines and then gen generate the art apply it to a 3D model and then create an executable that someone can run on a PC or a console or a like we we expect that to happen probably this year and if not this year certainly next year. So that's it's going to be wild. I would expect the first really good AI video game to be next year and probably the first half hour of watchable TV this year and probably the first watchable AI movie next year. Like things are really moving at an incredible pace.

33:48Yeah. When Gro is 10xing world economy with vending machines, it will just create video games for human. Yeah, it went from not being able to do any of this really even six months ago to to what you're seeing before you here and from very primitive a year ago to making a sort of a 3D video game with a few hours of prompting.

34:07Yeah, just to recap. So, in today's live stream, we introduced the most powerful, most intelligent AI models out there that can actually reason from the first principle using all the tools, do all the research, go on the journey for 10 minutes, come back with the the most correct answer for you. So it's crazy to think about just like four months ago we had Gro 3 and now we already have Gro 4 and we're going to continue accelerate as a company XAI we're going to be the fastest moving AGI companies out there.

34:33So what's coming next is that we're going to continue developing the model that's not just intelligent, smart, think for a really long time, spend a lot of compute, but having a model that actually both fast and smart is going to be the core focus, right? So if you think about what are the applications out there that can really benefit from all those very intelligent fast and smart models and coding is actually one of them.

34:57Yeah. So the team is currently working very heavily on coding models. I think right now the main focus is we actually trained recently a specialized coding model which is going to be both fast and smart and I believe we can share with that model with you guys with all of you in a few weeks. Yeah.

35:14Yeah. That's very exciting. And the second after coding is we all see the weakness of Gro 4 is the multimodal capability. So in fact it was so bad that Grock effectively just like looking at the world squinting through the glass and see all the blurry features and trying to make sense of it. The most immediate improvement we're going to see with the next generation pre-train model is that we're going to see a step function improvement on the model's capability in terms of image understanding video understanding and audios. Right? It's now the model is able to hear and see the world just like any of you. All right. And now with all the tools at this command with all the

35:51other agents it can talk to. So we're going to see a huge unlock for many different application layers after the multimodal agents what's going to come after is the video generation. And we believe that at the end of the day it should just be pixel in pixel out. and imagine a world where you have this infinite scroll of content in inventory on the X platform where not only you can actually watch these generate videos but able to intervene create your own adventures the view is going to be wild and we expect to be training a video model with over 100,000 GB200s and to begin that training within the next three or four weeks so we're confident it's going to be pretty spectacular in

36:31video generation and video let's see so that's anything you guys want to other than I guess that's it. Yeah, it's it's a good model, sir. It's a good Yeah, we're very excited for you guys to try Gro 4.

资源与出处

相关视频

同频道、同主题或相邻年份