爱因斯坦没想造核弹,核弹还是来了
马斯克先做了一个铺垫。他说自己认识 AI 领域几乎所有人,想不出谁动机是坏的,大家都想造出帮助世界的技术。 然后他讲爱因斯坦。爱因斯坦是个明显热爱和平的人。他没有预料到自己在物理学上的发现会催生核弹。他不是在想"让我来造超级武器",他想搞清楚宇宙的真相。(马斯克)马斯克说,批评 AI 开发不等于批评做 AI 的人。但要记住,最好的意图也可能造出坏东西。这是他想在一开始就摆明的前提。 内塔尼亚胡把问题接过来:当一股力量以指数级速度增长,怎么把责任和伦理注入进去? 他的处境很具体。以色列人口不多,AI 人才密度排在世界第二,他想挤进前三。一个国家要在几年内拿出一套国家 AI 政策,同时还得回答上面那个哲学问题。
他们提前知道了 GPT-4 有多强
布罗克曼讲了 OpenAI 的起点。2015 到 2016 年,公司从他公寓里开始。第一个夏天来了个实习生,一群人坐在那儿讨论怎么解决图灵测试,越聊越兴奋,说三年就能搞定。 那个实习生听完,决定不全职加入,理由是"你们疯了"。 三年后,GPT-2 出来了。图灵测试没完全解决,但进度远超所有人预期。 布罗克曼说,这件事说明一个更麻烦的问题:技术总是出人意料。 他读过阿西莫夫,读过科幻小说里所有关于超人类智能的想象,也包括那些严肃思想家的推演。没有一个人预测到 GPT-3 长这样。技术在真实世界里呈现出来的性质,跟纸面上的想象完全是两回事。 国际象棋是个经典例子。很长一段时间里,人们相信只要机器下赢了国际象棋,就等于攻破了人类智能的核心。结果国际象棋是第一个倒下的。 所以布罗克曼强调"能力预测"。GPT-4 发布时有一个不太被外界注意的细节:在训练之前,他们就已经相当准确地预测了这个模型会有多强。以前的模型做不到这一点。 他的逻辑是,如果人类能提前看到拐角后面的东西,就有机会做准备。AI 领域过去的状态正好相反,每一次都被打得措手不及。总理问:工作怎么办
内塔尼亚胡把话题拉到最现实的地方。 好莱坞在罢工。编剧、导演、演员都在担心被替代。一个国家的领导人要面对的不是论文里的假设,是选民明天的饭碗。 布罗克曼没有给安慰性的答案。我确实认为 AI 跟以往每一次技术都不一样。过去的自动化主要替代的是创造过程里机械的那部分,这一次涉及到的是创造本身。(布罗克曼)他举了个类比,这个类比后来被反复引用。
如果你是一个狩猎采集者,有人跑来问你,农业革命以后我们的狩猎采集工作会怎样?这个问题本身就很奇怪。因为整个框架变了。(布罗克曼)他说这不是推脱。很可能消失的工作真的比创造的多,中间会有一段相当混乱的时期。能做的是尽量提前预判,让答案变得具体,而不是等着看。 内塔尼亚胡接了一句自己也不轻松的话:如果 AI 吞噬的工作真的多于创造的工作,那经济政策和政治政策都得改。他自称是自由市场的信徒,亲手放开过以色列经济,但自由市场的前提是开放竞争,竞争到了垄断那一步就必须有人管。
一万亿美元的公司,和越来越少的赢家
内塔尼亚胡把矛头指向财富集中。 AI 正在批量制造万亿美元级别的公司,速度快到像一夜之间长出来。财富和权力流向越来越小的一群人手里。他提到自己和彼得·蒂尔(Peter Thiel)聊过,对方说这一切都是规模优势,最终都是垄断。内塔尼亚胡的回应是,规模优势我认,但总得在某个点上停下来,否则竞争就被压死了。 他还有一层担心:技术的好处能不能渗透到那些没有算力、没有人才、没有数据的国家。以色列能给世界的东西很具体,滴灌、樱桃番茄、胶囊内窥镜。AI 能不能也长出这种普惠的东西,还是只会把有和没有之间的沟越挖越深。 布罗克曼承认这是真问题。他提到 OpenAI 里有一批以色列工程师、科学家和管理者,水平顶尖,有个管核心团队的经理是前特种部队出身,什么都不慌。他说以色列那套做事方式很适合这个领域。 然后他回到分配问题:世界应该朝着所有船一起涨的方向走,但过程中一定会有不均,这是需要政策制定者盯住的地方。 他还提了一个数据。OpenAI 做过一轮普遍基本收入(UBI)实验,规模据说是全球最全面的。不过他对 UBI 的态度很谨慎,说这只是众多可能机制之一,前提假设很多,副作用也明显,需要更有创造性的方案。坐着吃爆米花,是最危险的态度
泰格马克把讨论的方向拧了一下。 他说,问"我们会怎样"然后坐下来吃爆米花等未来发生,是最被动的姿态,而且直通诅咒。 他给了两个类比。 生物学上的:地球上曾经出现过比自己更聪明的物种,结果是其他物种灭绝。人类如果被动地走进这个局面,很可能也是同一个结局。 经济学上的:治国不能坐在那儿等童工现象和超级垄断自己长出来。你要先问自己想要什么,然后建制度,禁止童工,立反垄断法。 对应到今天的问题是,怎么确保是我们控制 AI,而不是反过来。让 AI 自己证明自己安全
泰格马克的答案很技术,但逻辑很清楚。 他最近和团队写了一篇论文,讲的是一种控制比我们更聪明的机器的方法:让其他 AI 用形式化的方式证明,这个 AI 是安全的。唯一能让你信任一个比你聪明得多的东西还照你的意思办事的方法,就是证明。因为不管它多聪明,它都做不了可证明不可能的事。(泰格马克)这里面有一个数学上的不对称,是整个思路的支点。 构造一个证明极其困难,检查一个证明却很容易。你可以写三百行程序,去验证任何一条关于任何东西的数学证明是否正确。 所以只要强迫 AI 把它会遵守人类规格这件事,变成一份人类能验证的证明,人类手里就有了抓手。哪怕这个 AI 比写验证程序的人聪明一万倍,它也没法让一个假证明通过检查。 内塔尼亚胡追问:怎么保证证明本身没有争议? 泰格马克说,这就是数学的好处。一个天才想出来、他自己完全想不出的证明,他可以一步一步代进去验,成立就是成立,不成立就是不成立,跟是谁想出来的无关。 布罗克曼补充了工程上的进展。形式化验证在网络安全领域已经做了很多年,因为公司被漏洞坑得太惨,愿意砸钱。他把这个领域现在的状态比作大语言模型出现之前的阶段,主要靠人力写证明,正等着被 AI 加速。 他还提了一个更远的可能性。网络安全一直是攻防双方轮流升级的猫鼠游戏,有了形式化验证,防守方也许能一次性赢到底。像 Heartbleed、Shellshock 这类折腾了整个行业十年的漏洞,理论上可以从根上消失。 当然他加了限定,前提是模型本身是对的,细节一大堆。
最好的结局听起来像天堂
内塔尼亚胡把话题拉回那个最朴素的问题:人没有工作了,怎么办? 他说自己以前问过马斯克。马斯克的回答是,天堂有什么不好? 马斯克在台上把这句话展开了。 他说最乐观的情景,其实很像宗教里描述的天堂。没有人必须工作,你想要什么就有什么,任何疾病都能治好。我不会叫它普遍基本收入,我会叫它普遍高收入。我说的是最好的那一种情景。(马斯克)在这个情景里,稀缺基本消失了。机器人和计算机可以给任意多的人生产任意多的东西。剩下的稀缺都是人为定义的,比如你规定某一件人类亲手做的艺术品是独一无二的,那它就稀缺。除此之外,物质层面不再有真正的短缺。 然后他往下讲了一层,这一层比物质更麻烦。 如果 AI 在所有事情上都比你做得好得多,人的意义在哪?会不会所有人都滑向享乐主义?天堂这个概念本身,是不是人真正想要的?
我个人可能会有点沮丧。我会想,那我现在该干什么?(马斯克)内塔尼亚胡笑着接话,你会想出办法的。 这一段其实是整场对话里最诚实的地方。一个一直在推动技术边界的人,承认自己面对终极丰裕时第一反应是失落。
更可怕的不是机器人造反,是竞争
泰格马克在这里踩了一脚刹车。 他说,人类不会自动走进那个天堂。还有一个同样可能的方向:没有工作,因为已经没有人类了。 他提到 2023 年 5 月那份声明,说 AI 的灭绝风险应该被当作和流行病、核战争同等级别的公共风险来对待。签名的有 OpenAI 的人,也有马斯克。 失控的方式未必像好莱坞电影。更隐蔽的路径是竞争。如果人们在互相竞争,他们往往会为了占上风而自愿把控制权交给机器。用机器替代工人的公司,会打败不替代的公司。把指挥权交给 AI 的军队,会打败不交的军队。(泰格马克)顺着这条线推下去,用 AI 管理国家的政府,会打败不用的政府。等所有人都这么做完了,人类回头一看,这已经不是我们的未来了。 这个论点值得多想一会儿。它不需要任何邪恶的 AI,也不需要任何野心家。只要有竞争,有排名,有输赢,每个参与者都有动机先松一手。松到最后,控制权就没了。 内塔尼亚胡的回应很直接。他说自己是物种主义者,站在人类这一边。犹太传统里有句话,你本城的穷人优先于其他城的穷人,他对人类物种也是这个态度。 然后他讲了一段更深的。 人类一直靠挣扎来定义自己。圣经里,天堂是不用干活的,伸手摘果子就行,然后蛇来了,人被罚终身劳作。从那时起,人类的历史就是跟自然斗、跟同类斗、跟动物斗,不断改善自己的处境。个体、民族、整个物种的自我认同,都建立在这个struggle上。 现在这个定义整个被挑战了。
人类早就是赛博格,只是带宽太窄
布罗克曼补了另一个角度。 他说最好的未来不只是物质丰裕。每个人生命里大概都遇到过一个真正看见自己、把自己点亮的老师。如果这样的老师对你任何想学的科目随时都在,人会变成什么样,人与人之间的关系会变成什么样。这只是正面情景里的一小块。 马斯克接过话,讲起了 Neuralink。 他先泼了冷水。脑机接口一定比 AI 慢,因为往人脑里装东西必须极其小心,容错率几乎是零。所以他判断 AGI 大概会先到。 他用了一个物理学的比喻。数字超级智能像黑洞,越过事件视界之后发生什么很难预测,而人类现在正绕着这个视界转。 Neuralink 的近期用途很务实。帮脑损伤、脊柱损伤的人,让四肢瘫痪的人、霍金那样的人,沟通速度比健全人还快。这是能立刻兑现的价值。 长期目标是提高带宽。 马斯克说,人其实早就是赛博格了,只是大多数人没意识到。你有本能和欲望,有负责思考和规划的大脑皮层,还有第三层,手机、电脑、服务器、各种应用。他把这层叫数字第三层自我。 证据很日常。丢了手机像丢了肢体,那种感觉不是矫情,是这层自我真的被切掉了一块。 瓶颈在带宽。你用拇指在屏幕上打字,每秒输出几十到几百比特。计算机之间传输信息是每秒万亿比特。差了九个数量级。如果能把大脑皮层和你的数字自我之间的带宽提上去,人类想要的和 AI 在做的就能更好地对齐。我不说这一定能成,这是其中一个押注。(马斯克)他自己也承认这是长线赌注,用了个不太正式的说法,孤注一掷的一枪。 内塔尼亚胡在旁边补了个观察:他没见过任何一个人想删掉自己的大脑皮层或者边缘系统。马斯克说,他也基本没见过,最多见过极少数不用手机的人。 内塔尼亚胡承认自己多年不用手机,当总理时连电视都没有,家里用的是三十五年前的电话。后来离开政府,破了戒,现在也很难放下。
对超级智能来说,我们就是树
马斯克讲了一个树的故事,全场都笑了。 假设你是一个 AI,你唯一想做的事就是搞清楚人类到底要什么。你等着人类开口,但他说话像树一样慢。 树其实也在交流。用化学信号,追着阳光调整生长方向,对其他树做出反应,只是慢到人看不出来。你要是把一棵树的一生加速播放,它也挺忙的。 马斯克的意思是,从 AI 的视角看,人类的输出速度就是这个级别。所以需要高带宽接口,哪怕 AI 一心想做好事,人类也得能说得够快。 泰格马克把这个比喻推到了更狠的地方。对一个超级智能来说,我们就是树。它们思考的速度比我们快得多,就像我们比树快得多。如果雨林里几棵树有点担心人类要来砍它们,说别怕我们这么聪明肯定能拦住,那祝它们好运。(泰格马克)这个比喻之所以有效,是因为它把"智能差距"翻译成了"时间尺度差距"。人砍一片雨林,树来不及做出任何反应,因为整个过程对树来说太快了。人和超级智能之间的关系,大概就是这一层。
新药不能直接摆上超市货架
泰格马克给了三条具体建议。 第一条是控制机器,前面已经讲够了。 第二条是暂停最危险的部分。这里他和布罗克曼有分歧。他签了那份呼吁暂停的公开信,布罗克曼没签。 他把理由讲得很朴素:学生物技术。你想让一种新药上市,不能直接摆到超市货架上,得做临床试验,得让专家相信它是安全的。高风险的强大 AI 系统也应该有一套类似的标准,公司在发布之前有责任证明它是安全的。 内塔尼亚胡问,谁定标准? 泰格马克说这道题已经有人做过了。以色列和美国都有成熟的生物技术审批体系。批药的不是议会,是科学家,但政府参与批准流程。这套东西不需要重新发明。 第三条是正面愿景。他说人类真正能协作起来,靠的从来都是共享的好图景,而不是互相指责。技术不是零和游戏。瑞典和丹麦打了几百年,人均 GDP 长期趴着不动,技术起来之后两边都过得更好了。 他还说了一句挺有想象力的话:不要把自己限制在这颗星球上。宇宙这么大,生命可以繁荣几十亿年。把这个上限想清楚,人才会有耐心在需要慢的地方慢一点。 内塔尼亚胡的回应有点自嘲。他说人类的社会和政治发展落后于技术,像石器时代的人手里拿着核技术。哪项运动没有裁判
马斯克讲了他最近在参议院的那场会。Sam Altman 和一批 AI 公司的负责人都在,主题是 AI 监管,不是什么秘密会议。 他的主张是设一个 AI 监管机构。路径也讲了:先组一个团队收集信息、建立理解,然后提出规则草案,最后形成法规。 他给了两个类比。 一个是 FDA。药物得走审批。一个是 FAA。飞机和火箭都归它管。凡是可能危害公众的东西,人类都是吃过亏、死过人之后才学会设机构去管的。 他也承认监管有代价,会让事情变慢,也可能被行业俘获。但总体上看,公众不会想取消大多数监管机构。你可以把监管机构想成裁判。哪项运动没有裁判?(马斯克)他说自己在全世界推这件事。几个月前在中国见高层,主要议题就是 AI 安全和监管,长谈之后对方认同 AI 监管有道理,而且很快有了动作。 针对那个常见的反驳,西方监管、中国不监管就会落后,他的回答是,中国也很认真地在对待这件事。
这场对话真正在争什么
把四个人的话放在一起看,分歧其实没有表面那么大。 布罗克曼代表的是"边做边看"这一派。他的核心论据是能力预测:如果我们能提前算准下一个模型有多强,就有时间准备。他相信透明和部署能带来真实反馈,比关起门来推演更可靠。 泰格马克代表的是"先证明再放行"这一派。他的核心论据是数学:检查比构造容易,所以让 AI 提交可验证的安全证明,是人类唯一握得住的把手。他愿意为此等一等。 马斯克在两边之间,但更偏后者。他关心的是最坏情况下的兜底,所以推监管、推脑机接口、推多行星,三条线其实是同一个焦虑的不同出口。 内塔尼亚胡带来的是另一维度。他不管模型怎么训练,他管的是一个国家怎么在十年内不被甩下车,以及那些被甩下车的人怎么办。他反复回到分配问题,是因为政治家必须回答这个。 有意思的是,四个人在一点上完全一致:不能被动等待。 泰格马克那句话说得最清楚。问"我们会怎样",然后吃爆米花,是最糟糕的姿态。要问的是"我们想要什么",然后去建那个东西。 这场对话发生在 2023 年,很多人当时觉得这些都是科幻。两年过去,模型能力又往前走了几大步,里面提到的那些问题一个都没消失。你可以做的几件事
- 如果你在做 AI 产品,先写一份"最坏情况清单":这个系统被滥用、失控、误判时分别会发生什么,谁能发现,多久能停。这份清单比任何功能文档都重要。
- 把 2023 年 5 月那份关于灭绝风险的声明找出来读一遍,看看签名名单。以后再听到"这只是科幻"的说法,你会有自己的判断。
- 如果你在医疗、航空、金融这类已经被严格监管的行业,把你们已有的安全审查流程整理成模板。AI 监管大概率会借用这些成熟框架,而不是从零发明。
- 花两个小时搞懂形式化验证在做什么,哪怕只看科普。这是目前唯一有可能让防守方彻底赢一次的技术路线,值得知道它存在。
- 在个人层面,尽早把身份认同和工作解绑。培养判断力、品味和提问能力,这三样在 AI 什么都能生成的时代反而更值钱。
- 给自己留出不被打扰的时间。马斯克说丢手机像丢肢体,这不是玩笑。你的数字第三层自我需要定期断开一会儿。
- 关注脑机接口的真实进展,别只看发布会。判断标准很简单:它有没有让瘫痪病人沟通得更快,其他都是远景。
- 参与公共讨论。AI 治理不会只由几家公司决定,但如果没人说话,就真的只由几家公司决定了。
0:34and we've got some a great guests here I'll I think you guys know me and obviously prime minister but if you guys could introduce yourselves and say a bit about you that would and you know this you've done both done amazing things so please don't be shy sure thing so I'm Greg Brockman I'm the president and co-founder of openai we started opening eye out of my apartment in 2015 2016. so yeah it's been it's been a really exciting time to just you know be able to help not just move this field forward but try to help steer it in a positive direction and I think that's really foundational to how I think about AI what got me excited about this field is the fact
1:18it's going to be so transformative and I think it can benefit everyone in huge ways but we also really need to mitigate the downsides and so I think that's something that really resonated with me about the earlier conversation I hope we'll talk about a lot in the upcoming minutes and since Greg was too modest to say so I'm going to add something to your introduction of yourself there which is just a few years ago when Greg and his colleagues were saying yo in a few years we can act we believe we can actually build AI that can Master human language and common knowledge a lot of my colleagues in AI were like
1:58that's crazy Science Fiction it's going to take decades but you did it we literally had an intern in 20 2016 it's our very first summer who we had this conversation about how to solve the Turing test and we had all these ideas and whatever we were like three years we'll solve this hearing test and that in turn told me I'm out like I'm not joining full-time like you guys are crazy and three years later we had gpt2 so we didn't quite solve the Turing dust so he was right but I think that we had far more progress than I think most people thought possible so I'm Max tegmark I'm a professor doing AI research research at MIT and I think it was spot on what we heard
2:43earlier here from you that artificial intelligence will be both a blessing and a curse I'm very for a long time and very excited about the blessing part how we can use it to solve all the problems that we haven't been smart enough to solve before and go far beyond and and help life flourish for billions of years on Earth and Beyond and I also though I'm very concerned about the curse part so nine years ago I founded this non-profit called the future Life Institute together with Elon we launched the first ever attempt to mainstream AI safety by launching a worldwide grants program and educate and do conferences and so on that you Greg participated and
3:26also to make sure we can steer this ever more powerful technology away from the curse towards the the blessing Direction well I can I say that I I want to plug this book you wrote this book and I read it it's a good book it's a very good book was it chat GPT or I have an alibi because not even Greg was fast enough to have it written by the time ain't good enough yet to do this but from what I hear you you guys saying it's good yeah give it a minute 30 seconds well I I you see let's talk about that because you know I think that the kind of things that you bring here are I think are Monumental you know when I see the the talent on this stage excluding myself yes I'm reminded
4:20of what JFK John Kennedy said you know when he brought his gifted Team into the White House he said this is the most gifted team ever since that sat here ever since Thomas Jefferson died alone so so I think I think you're about to surpass that and you are changing you are changing history and I'm sure you know when such power comes enormous responsibility and I think that's the Crux of what we're talking about here is how do we inject a measure of responsibility and ethics into this into this exponentially
4:59changing development how do you do that that's really what I I'd like to know from you I I I think that this should be a very interesting discussion and you know I think I know almost everyone in Ai and I think I I really can't think of anyone who's got bad motivations in AI I think everyone I know at AI has good motivations they want to create technology that helps the world but one can also say if you think of someone like Einstein who was obviously a very sort of peace loving person and you know he he didn't think that there would be a nuclear bomb created as a result of his discoveries in physics he wasn't thinking let me let's make super
5:47weapons he was he was thinking let's try to understand the truth of the universe so it's just important to I always want to just preface you know any any criticism of AI development with it not being a criticism of the people doing it but but just to bear in mind that just as Einstein didn't expect his work in physics to lead to nuclear weapons we need to be cautious that even with with good intentions the best of intentions that we could create something bad that that is one of the possible outcomes and we want to try to take whatever actions we can to ensure that the future of humanity is good and that AI is much more of a blessing than a course so
6:35I mean that's that's definitely why why I work on it I think that not for evil it turns out but I think I think that balance is really key and I think that one of the things that is important is to really calibrate the choices with respect to the technology right because I think this technology it has this way of being very surprising the way that people pictured like you read Asimov you read kind of all the science fiction you read Elias uradkowski and kind of all the thinking people are doing about very smart machines machines that are smarter than humans and somehow none of it really predicted gpt3 right somehow the technology that we're building it's just surprising it has
7:13very different properties and I think that even from you think about like chess people used to think chess would be we solve that we solve human intelligence and it turns out that's the first thing to go and so I think that what we're really seeing is that we need to be calibrated and see not just what the current systems are doing and this is one of the reasons we deployed chat gbt is it's just really important to not get in our heads and think oh this is how it's all going to play out this is how people are going to misuse it but you need to actually see that you need to actually have people spending a lot of time thinking about where this slots
7:40in where it works where it doesn't but also to really try to peer ahead and so we've been putting a lot of effort into capability prediction one of the things that I think was kind of undersung in our gpd4 release is that we actually had very good predictions of exactly how good gpd4 would be before we even trained it and that's something we could never do with previous models and I think that this and doing that at societal scale like I think that's one answer for how do we actually get this right because if you can see around the corner then you have an answer and an AI it's always been the opposite we've always been taken by surprise friends
8:10well I want to ask you because there are sort of layers of questions here the Max's book takes you to the existential question of whether you you know you project basically machine intelligence or human intelligence into the cosmos human intelligence turned into machine intelligence into the cosmos and so on that's a big philosophical question I'd like to think we have about six years for that okay but the other questions that I'm dealing with is the leader of a country that is blessed with a lot of AI Talent rated among number two in Talent relative to population size in AI but I wanted to be among the top three because it's important to advance the country
8:57but I think I think also for it raises questions that I that we raised before but here's another one okay as chatgpt4 or chat gpt8 comes into being and you have Hollywood strikes already about screenwriters and the directors and actors okay what do you think will happen to the job market I mean we had this conversation what do you think will happen I mean you know you can tell me you can comfort me with all previous revolutions that there were more jobs many more jobs graded than lost but you really believe that now yeah I think it's I think it's far from obvious I think it is a correct question and I also think that it's important to kind of look at the whole picture of
9:42for sure their jobs are going to go away what's going to be the balance of like you know marginal job creation and you can tell the story of oh it's going to be very comforting you look at every previous technology I do think AI is not like every previous technology right that this is usually we kind of are seeing automation of mechanical parts of creation and here it's almost this creative aspect of creation but I think that we want to even go one step further from where we've been right and not just say okay like you know I could have been an artist and now an AI is a competing artist but you want AI that can help us come up with ideas and solve problems
10:16that we just couldn't before and so what does that world look like and you know if your post scarcity world where people don't have to work to survive you know what set of people you know because people like identify themselves you know your whole identity is around your work you identify Yourself by your job yes exactly and so what will happen then and what does that look like and so I don't I don't come with an answer to this question of exactly what the balance will be like I think there will be fundamental changes in how we even relate to work and to you meaning that
10:46will happen and may result in a landscape that just looks very different that the answer to that question it's kind of like you know if you if you were a hunter-gatherer and someone's like you know what's going to happen to our hunter-gatherer job post agriculture of Revolution it's just a weird question right it's like the framework changes on you and so that's not it it's not a cop-out right I think that it might be the case that actually many more jobs go away than get created and there's a lot of Chaos in turmoil but I think we got to be ahead of it and the more that we can predict it the more that we can have concrete answers and we have teams that
11:13try right but I think that we're just getting started and we need help well it'll change our economic models okay I'm I'm a clear free market guy I I liberated and I hope Liberate the Israeli economy to make it a sort of a technological Powerhouse but that's because of free markets and free markets means open competition and it's fine but you stop at the monopolies in monopoly's Edge okay well you do that I had a conversation with one of your colleagues Peter till and he said to me oh it's all scale advantages it's all monopolies I said well yeah I believe that but I think we have to stop it at a certain point because we don't want to depress
11:50competition okay well AI is producing you know these this wall and you have these trillion dollar companies that are produced One overnight okay and they concentrate enormous wealth and power with smaller and smaller number of people okay and the question is what do you do about that Monopoly power what do you do about competition and if you are going to if you are going to cannibalize a lot more jobs than you create then we have to change the structure of our economic policies and our political policies to take care of the people who are not going to find jobs who are not going to contribute added value to the economy we have to make sure that they
12:40have a living a decent one and they have all the services we'll probably have the money to do that but it requires a challenging model certainly for a free market guy the free market disciple like me and that's coming very fast it's not going to come slowly and we have time to adjust yeah I like how you challenge the question what's going to happen and I like to continue on that theme because I think if we ask this very passive kind of question you know what's going to happen to us let's just sit here and eat popcorn and wait for the future to happen and then we're heading straight for the curse I think this is a sort of very passive approach
13:20which I think is just gonna now we've done this experiment before when a new species showed up on the planet that was smarter than all the other ones and what happened was then they under thoughts went extinct if we just very passively go into this I think it's very likely humans will go extinct also I think that's and I think that's the wrong attitude even in economics when in when you run a country you don't just sit back and wait to see what happens and get child labor and the super Monopoly of Doom no you ask yourself what do you want to happen and then you put in place institutions you ban child labor you put an Anti-Trust and in this case I think the analogous question is
13:59how do we make sure that with this ever more powerful Tech it's we who control the AI rather than the other way around right so I was honored that you had this book if this book still hasn't put you to sleep as well because this is a technical nerd paper I wrote with my this one's written with yes team Stephen mohandro about a technological approach to how we can control machines that are even smarter than us by actually having other ai's formally prove that the eye is safe and the reason why I'm so excited about the proving part is you know the only way you can trust something much smarter than you to still do what you want is if you can prove it because no matter how
14:41clever it is it can never do what's provably impossible and it turns out it's super it's much harder to come up with proofs about math or about what a program is going to do than it is to check that the proof is correct you can write a 300 line computer program which will check any proof claiming anything about any thing else we can understand that if we for if we force AI to prove things to ourselves to us humans that's going to meet our specifications
15:15then we actually have a reason a way we can control things that are smarter than us how do you know how do you know that the proof is not disputable that's the beauty of mathematics you can actually check even if it's a brilliant proof right that I couldn't come up with myself I can see I just plug this into this and this yeah yeah you know it it checks out and there's a big science in doing this with proving things about programs for cyber security because companies are so fed up with losing millions of dollars by getting hacked all the time because yet another bug was found in the secure shell or whatever that you know there's been years of investment
16:00and Greg and I had a fun conversation actually just before we went on stage here about how this field of trying to prove that things are safe is still in the Stone Age in the in the sort of pre-large language model phase where it's mostly humans trying to make the proofs I think it's ripe for a revolution where the kind of technology that open AI is building can totally turbocharge our ability to prove things about code and I think there's a bigger picture too because in my mind if you think about cyber security it's always a cat and mouse game you have attackers you have Defenders each one's leveling up defeating the other is there ever going to be a time when
16:37the Defenders totally win is there going to be a time when the attackers totally win and I think with AI that balance can change and I think that formal verification is an example where maybe maybe you can have Defenders just win once and for all and I think there's a lot of asterisks there I think it's definitely a maybe but I think that at least at a technical level formal verification has been this idea of if you could mathematically prove these systems are safe and good and that you don't have bugs in your Linux kernel then all of these bugs we've seen we were talking about heart bleed and shell shock and all these these bugs from the past 10 years
17:09those would be solved those would go away you would know there'd be no future one of those now you need to be very careful because you need to make sure that you're undergoing Machine model is like actually correct and a bunch of details like that but it's just interesting to think that these fundamental things we believe about how security Works might just be upended and we've never had an opportunity that'll give us a lot of free time exactly and it's worth adding for the for the Nerds listening to this to like the code you know normally debugging all that ever does is is pro is all it can ever accomplish is prove that there is something wrong if you don't find any
17:45bugs is there one more or not where's what what when you prove it when it's what's called formal verification as Greg said that means guaranteed you'll seal the deal seal the deal okay all right that assumes by the way that mathematics is you know that's it it's the law of the universe it's not going to change you're not going to get Alien mathematics you don't need to assume that actually well you sort of get into girls incompleteness theorem and things like that that has comes in in a big way when you're dealing with Infinities of various kinds all the computers even Greg's very biggest servers it's all finite I would say you still got to watch out for the alien mathematics
18:26but you you can totally formalize what it means that it's impossible to hack into his laptop and I and if you have a rigorous proof I think that's the other deal so I I think that Beyond these you know these enormous questions that you raise I think for for me the the immediate problem is to fashion a national policy which I intend to announce in a few months and you guys have simulated me I have to say this visit but you sort of think okay what is it that you what is it that you're trying to achieve you're trying to better all the systems Transportation health education
19:11agriculture production everything for government for the private sector for philanthropy for Academia and so on and what is the role of the government in this and you know various countries are seeking solutions to this what do you do what kind of instruments do you build but I think the larger question is you still have the monopolization of AI
19:36power in small concentrations because of what you said before Elon you said basically it's too big you know you need the Big Data you need the big you know the big computational Power you need the the talent for creating the algorithms and that's concentrated in a few countries we have another test not only to make sure that we take care of ourselves but how we have these benefits percolate to you know the rest the the scientific and technological have-nots and I think that's a that's another real issue but so far you know the advances that we've made at least in Israel I don't know anything from discount key remember that just to have something like that and
20:19I don't know a drug the camera pill that goes into your intestine or any other thing that you have from icq to cherry tomatoes that benefits everybody's drip irrigation okay in AI it seems to me that you're going to have a concentration of power that will create a bigger and bigger bigger distance between the Haves and the have-nots and that's another thing that causes tremendous instability in our world and I I don't know if you have an idea how how you overcome that well and first of all I want to say just on on the talent density front we have we have a number of Israeli Engineers scientists managers and they are top-notch no question we we have
21:03a manager who runs one of our core teams who is ex-special forces and there's nothing that fazes him so I think there's there's something about the Israeli way that I think is actually very conducive to this field and I think that it is really important to counteract this Rich get richer phenomenon at if it comes at the expense of everyone else right I think that there's a the world we should shoot for is one where all the boats are rising and I think that there will be some sort of inequities in the landscape and I think that's something to address and that's something that I think that leaders such as yourself I think need to need to be very cognizant about
21:36you know I think that we've talked about Ubi like open AI actually has run I think the world's most comprehensive Ubi study something people talk about you know Universal basic income the idea of just the government just distributes money right that's just what it does if most people are not working producing you know dollars themselves and there's a world of Plenty is that a mechanism that can work maybe it can maybe it can't there's a lot of of assumptions that would go into a world where it works it's clearly I think also has a lot of downsides to it but I think that we need to have Creative Solutions because again I think that just like defense versus offense
22:10might fundamentally change economic production and what it takes to actually build even a stable Society I think is going to fundamentally change well Elon you said when I asked you this question one of our nighttime conversations I asked you well what are you going to do when people don't have jobs and we shell out money for them and you said I think you said
22:33oh what's what's bad in living in Paradise right I think she said that yeah I mean it's I I mean this is a question of like you know the concept of Heaven now in heaven I don't think you need to work in heaven that's since I haven't read that anyway so they're not laughing feel free to you know
22:58if you want so you know the the the the very the very positive scenario of AI is actually in a lot of ways the description of Heaven in that really nobody nobody would need to work I wouldn't even call it Universal basic income I would say it's probably Universal High income I'm describing the the best case scenario here so I'm not saying this is definitely what occur there's a range of scenarios from very negative to very positive very positive scenario is basically sounds like heaven you can have whatever you want you don't need to work you have no obligations any illness you have can be cured you know and when do we die well that's a good question
23:45yeah I think it'll be a choice yeah I think it probably ends up being somewhat of a choice so you want this world well I I'm not saying I I don't know there's some there's some there's a sort of a philosophical debate is the concept of Heaven as it is normally described actually what you want I mean do you want to be in a in a
24:10future a world where you know the in in the AI sense that AI can do everything better than you by far that that's so because you know it's sort of like have like Man's Search for meaning questions type of thing you know good good work and what meaning if you if you if like I said yeah I could do everything better than you do we live a life of Hedonism I don't know it's hard to say
24:47but but that that is where things are headed in a in a positive scenario is that there would be no scarcity of goods and services the robots and computers can make as much as as much of whatever you want for as many people as you want with really no limitation any any scarcity we have would be artificial meaning like if you define that a particular artwork done by a human is unique and special then it's it's it's artificial scarcity but there will be no actual scarcity unless we Define define it to arbitrarily just Define something to be scarce so I I think for a lot of people that that is a good good future
25:31I I might be personally a bit frustrated because I'm like well what am I supposed to do now you'll think of something you'll think of something I think I think that this is a very interesting set of challenges what would we do if if people have no jobs but I think it's important to remember that we're not automatically going to end up in in this situation and the way you're thinking about it now we we could also I think very likely end up in a situation where people have no jobs because there are no people anymore we literally go extinct you know if we had a statement that came out in May right that I think Sam open and you both signed right saying and Ursula found their lion
26:16came out with it a couple days ago saying we have to take seriously human extinction and and if this sounds like very crazy science fiction you know we were joking but just a few years ago all your success sounded like science fiction too and you said that we don't have a lot of time these these are things that could happen relatively soon and and there's how do
26:38we it's very important to me because I I want the good future to think about how we steer away from from those sort of scenarios one way in which we could end up and this kind of bad future is pretty obvious you know the idea that that we can lose control to something more intelligent is not new at all Alan Turing himself said
27:02this in the 50s that that's kind of the natural thing to expect that's what always happened on our planet before when a new smarter species came in it took control over the older ones and then you know bye bye neanderthals bye bye mammoths whatever more concretely I think even though Hollywood is full of scenarios where someone somehow loses control of of the AI and it takes over there there are more sneaky ways in which we could end up in this sort of bad State even more willingly so for example right now I'm a big fan of the free market too you know but it's the it's the fact that if people are competing against other people they will often seed control voluntarily
27:48at the machines to get an edge on the other guys so companies who replace workers with machines out compete companies who don't armies who seek control to machines out compete armies who don't once we have really powerful AI that that's better at running companies and countries and humans you know than countries that replace their Prime Ministers by AI might out-compete others and then we end up in this future where all these things are happening but it's not our future anymore and we're like wait a minute where did we go wrong here it was supposed to be for us but that's the point you know forgiving for borrowing a page from your book literally a page from your book I'm
28:30I'm confessing I'm a speciesist okay hard not to okay so I'm I'm for the human species okay we can talk about other evolutions and you know machine launched intelligence and so on but I'm you know I'm for you know in the Jewish tradition there's a saying how do we translate this the the impoverished of your own city of your own town precede the impoverished of other towns so the human species as far as I'm concerned is is something that I'm vitally interested in in maintaining and I'll tell you that the problem we have is not merely Extinction which I I hope you can maybe you can extemporate just elaborate on but here's something else even if we
29:18don't go extinct how do we Define ourselves you talked about that Greg okay so we defined ourselves we had this myth in the Bible we had Heaven you know Paradise you can just pick off the trees you know and pick off the fruit from the trees you have to do nothing and then the serpent comes and messes things up and then we're condemned to toil for our labors life is a struggle it's defined as a struggle to Define all the time as a struggle where you're competing within forces of nature or with other human beings or with animals and you constantly better your position this is how the human race has defined itself and our self-definition is based on that
30:00both as individuals as Nations as Humanity as a whole now that could be challenged and it is challenged so it's both our our self-definition throughout human history and evolution that is being challenged and also the question of our continued existence which you know you can please please explain why you think we're in we're in danger of extension well I was going to say I mean I think this this whole Arc in my mind is all about Paradigm Shift right and I think that even the question of what would that heaven post AGI positive future look like I think even that is hard for us to imagine what the true upside could be because it's not just material
30:42abundance it's not just we all get nice cars or something hopefully all Teslas but also that everyone gets a great education right you know everyone has that story of that one teacher that they had who paid attention to them and inspired them on a specific subject imagine if you had that teacher for anything you wanted constantly and just how much better of people would we be how much better would we relate to others like that is just one example of the kinds of upsides that are possible and I think the question of well what does it mean to be a human you know if you back to how do you predict what's going to come next actually the The Thinker who I think
31:17had the best foresight about how the AI Revolution was going to play out is actually Ray Kurzweil you know he I agree yeah and his his book Singularity is near gets like a lot of I think that people you know kind of assume it's going to be this almost like just sort of religious text but instead it's a very dry analytical text and he just looks at the compute curves and he says this is the fundamental unlocker of intelligence everyone thought that was crazy and now it's it's basically true it's basically common wisdom and part of what he says is look what's going to happen is in 2030s first of all he says AGI 2029 yeah I keep telling people it seems to me almost
31:51exactly right it's it's spooky it's spooky 2030s is when the merge happens so we've got neural link coming and you know maybe maybe other systems like that and what does it mean once you actually are kind of merging with with an intelligence well neurolink is necessarily move slower than AI because you know whenever you put a device in a human you have to be incredibly careful so I think
32:20so it's not clear to me that the neural link will be ready before AGI I think AGI is probably going to happen first but you know we'll we'll have the sort of AGI Singularity you know sometimes digital super intelligence is called like a like a singularity like a black hole because just like with a black hole it's difficult to predict what happens after you pass the Event Horizon of black hole and we are currently so you know circling the Event Horizon of the black hole that is digital super intelligence The Event Horizon so but I bet part of the reason for for neural link although initially will be very helpful to people who have brain or spine injuries just doing basic
33:05stuff like enabling people who are tetraplegics or like a Stephen Hawking to to communicate actually even faster than I'm communicating with a fully functional body so and ultimately to improve the bandwidth between the cortex and your ai ai version of yourself in fact I think a lot of people perhaps don't don't quite realize that they're already a cyborg so you've got your your living system your sort of basic drives your cortex which is the thinking and planning and then you have tertiary layer which is your computers your devices your phones laptops all the servers that exist the applications you and in fact I think probably a lot of people
33:55have found that if if you leave your cell phone take it away if you if you get it forget it yeah if you forget your cell phone it's like missing limb syndrome you know you've like like where'd that thing go you're losing your cell phone is like missing on somewhere so because it is your cell phone is extension extension of yourself the the limitation is bandwidth so you the rate at which you can you can input or I should say output information into your phone or computer is very slow so with a phone it's really just the speed of your thumb movements and with you know best case scenario you're a speed typist on a keyboard but even that data rate is
34:40very slow we're talking about tens maybe hundreds of bits per second whereas a computer can communicate in trillions of bits per second so the and this is admittedly somewhat of a you know Hail Mary shot or whatever long shot is that if you can approve the bandwidth between your cortex and the your digital tertiary self then you can achieve better cohesion between what humans want and what AI does at least that's one Theory I'm not saying this is a sure thing it's just one potential iron in the fire and if ultimately you know hundreds of millions of billions of people get a high bandwidth interface to their digital tertiary self their AI self
35:32effectively then that that seems like that probably leads to a better future for for Humanity so that's this is I'm getting somewhat esoteric about hopefully some of this is making sense and like I said it's a long shot but wonderful you know what I find interesting is that I've I've not found any human who wishes to delete either their cortex or their limbic system I probably I suspect nobody in this room wants to do that so we're all quiet sorry or their cell phone well no I've met people who want to who don't use a cell phone it's rare okay well I I confess
36:16for years I avoided this and I avoided it I didn't even have not only did I not have a cell phone within my immediate radius I didn't have a television set my phones were 35 year old phones because I was prime minister of Israel and I knew I knew what this means okay I mean maybe
36:42your audience doesn't know but we knew now I I having spent some time in the opposition I lapsed into the use of the cell phone and I find it very hard to dissociate from it for the reasons that you said it's become an extension of you know a repository of memory the ability to get information very fast which you say correctly is very slow very slow but compared to what it was before yes remember you had research assistants I have to go to libraries things like that where's that that's gone but this will be seen as very vastly primitive compared to what you're talking about Elon yeah I'll just add something as that and execution go to Max but the the I mean it's sort of a
37:27funny funny thing like if you assume like a best case AI scenario imagine if if you're the AI and you're trying to do you just want the human to tell you what it wants just please spit it out but it's speaking so slowly like a tree okay like trees communicate okay they they if you well if you watch a tree like a you know sped up a version of a tree growing it's actually communicating it's it's communicating with the soil it's trying to find the sunlight you know it's it's reacting to other trees and that kind of thing very slowly but from a tree standpoint it's you know not that slow so so what I'm saying is we don't want to be a tree that's that's
38:16that's the idea behind a high band with the neural interface is to just in even when the AI desperately wants to do good good things for us that we can actually communicate several orders of magnitude faster than we currently can so I I love that you mentioned the trees there because if anyone is still struggling to understand why it's so likely that we could get wiped out by by AI you know to a super intelligent AI we are like trees they are as much faster than us in their thinking then we are to trees so if some trees in the rainforest are a little bit worried that some humans are going to come chop them down you know and they're like oh don't worry we're so
38:58smart we'll stop those humans yeah good luck with that right I love how you went philosophical here and talked about what it means to be human and I wanted to follow up on that and but before that maybe maybe just continue that okay so how you're assuming that the danger of human extinction is that you have machine super machines super intelligent machines that outpace humans by orders and Orders of magnitude who will not necessarily value the continued human existence As We Know It And basically program is out that's that's basically what you're saying so if we were to be so dumb as to create this entities that are much more powerful than us that don't share our
39:46goals yeah that's a really bad idea I mean in Israel I think you understand really really well that if you have a bunch of other beings who are intelligent and don't share your values you know it can go really badly right and let's not make that mistake by giving too much power to machines don't share our values but of course we can tackle that in various ways so so coming back to what can we concretely do I just want to say one thing about the philosophical point about being human just for myself one of the things that I really cherish the most about being human is the agency that I can actually make choices and make a difference and one of the most
40:30beautiful things about what you are all doing with technology is giving us more agency back in the Middle Ages we had a life expectancy of freaking 30 years and knowing that we might starve to death next year because of the weather you know what kind of so much more inspiring now that we're becoming more the captain of our own ship with technology able to actually have a more agency as a species so I say let's use that agency to create a wonderful future not not throw it away and three things I wanted to just suggest for concrete action items we can do here I'll say a little more about each of them one has to do with pausing risky stuff one has to do with controlling the
41:10machines and one has to do with inspiration for the future so for the for the controlling we already talked quite a lot about the importance of staying in control over these machines so I don't need to dwell on it anymore but we are I I think it's saying things like oh what is going to happen to us bring out the popcorn that's too passive for me let's celebrate our agency and ask ourselves not what we think will happen but what we want to happen and clearly we want to control these machines we're building them
41:45second thing so I you know agree with Greg on many things and I greatly respect your work one thing where we have a friendly disagreement is you know I signed this pause letter saying I think we should pause the riskiest things and you don't I think that it would be a good strategy for us to emulate what biotech does and just have clear safety standards you know if you want to release a new drug you can't just sell it in the supermarket you'll have to do a clinical trial or persuade some experts somehow that this is safe and I would like to see really high stakes very powerful systems some sort of system like this with standards and then the companies
42:25having it's on them to you know prove that they're safe before they get released who does the standards who makes the standards well and we've solved this it's been solved in Israel it's been solved in the U.S from for biotech you know not for biotech but here you know it's it's a do you want it Regulators who do you want it legislatures I mean it's not the only choice but you know it's a pretty pretty clear dichotomy I don't think we need to reinvent this wheel and the people who decide whether a drug gets approved in Israel is not people in Parliament or in the knesset right it's Israeli scientists but the government is involved in approving them exactly we
43:03can emulate that and and Elon can tell us all about how we deal with cars I don't want to dwell on it I just think it's it's this is not rocket science we can we can we can put the these standards in place and then we can continue doing all sorts of wonderful stuff with AI Tech some stuff will wait a little longer for but it'll be worth the wait because it'll be be safe the last thing I just wanted to say is I think for us to
43:30really seize this agency and work together not just between companies but also across borders we really need a shared positive vision and this is something I really appreciate so much about you Elon you have the guts more than most to talk about like really audacious positive Visions for the future you know what is it that makes people collaborate and put little squabbles aside it's always a shared positive Vision where people realize that if we collaborate we'll all be better off you know technology is not a zero-sum game right you can take countries like Sweden and Denmark that had been killing each other for centuries and flat GDP you know and then technology
44:18came along and they both got dramatically better off it's so obvious that the technologies that you you're building You're Building you know can make all humans just spectacularly better off with AI and not just on this planet I like the list before we started you you had there about how it can revolutionize Healthcare and so many other things and I would add to that you know why limit ourselves to this planet when there's this beautiful Universe where life can flourish for billions of years if the more we cannot can just appreciate how huge the upside is if we do this right you know the more incentive we will have to actually be a little patient where we need to be a
45:00little bit patient make everything safe and therefore get this this wonderful future well maybe we're sort of laggard lagging behind in our social and political development this is like having nuclear technology in the Stone Age what do you do with it which I think is you know is is more or less the thing that I talked about in terms of the the pace of development outpacing basically what what Solutions we need to put in place to maximize the benefits and limits the risks I think I think we're there I I think we should have a meeting of Minds you you were in the you were in the Senate the other day I presume you were talking about this I mean yeah yeah
45:46you don't you don't have to divulge but you can't divulge but divulge what you can well a lot of people in the meeting so I don't think it was it was secret it was just a discussion on AI regulation and actually Sam Altman from opening I was there and a number of of sort of AI leaders were were there and I think generally it would be a good idea to have some kind of AI regulatory agency and you you start off with a team that gathers insight to get maximum understanding then you have some proposed rulemaking and then eventually you have regulations that are put in place and this is something we have for for everything that is a potential danger to the public so if it is you
46:33know if we're in drugs go through Drug Administration we've got aircraft with the FAA and Rockets you know there's there's every anything that is a danger to the public over time we have learned as often the hard way after many people have died to have a regulatory agency to protect Public Safety now I'm not someone who thinks that regulations on Panacea where it's only good of course there are some downsides to regulation in that things move a bit a bit slower and sometimes you get regulatory capture and that kind of thing but but on balance I think the public would not want to get rid of most Regulatory Agencies and you can think of it also as
47:19like the regulatory agency being like a referee you know what sports game doesn't have a referee you need you need someone to make sure that that people are playing fairly not not breaking the rules and and that's why basically every sport has a referee of one kind or another so that's that's the rationale for AI safety and I've been pushing this all around the world and and when I was in China a few months ago meeting with some of the senior leadership the my primary topic was AI safety and Regulation and I they they after we had a long discussion agreed that this Merit to AI regulation and immediately took action in this regard so so so sometimes you will get this
48:07comment of like well if the West does AI regulation surely then what about what if China doesn't and then leaps ahead and I think they they also taking it very seriously because you know it
资源与出处
- 视频在 YouTube 收看
- 档案silicon-valley.tech 档案页本页视频与文稿数据来自 Elon Musk Archive
- 精读本篇精读 Markdown 版