打一行字,你从大脑传到机器上的信息量,大概每秒 10 比特。
乐观一点,用拇指打字或者开口说话,能到每秒 100 比特。同一时刻,你面前那台笔记本和网络之间的传输速率是千兆比特、太比特的量级。中间差着七八个数量级。
马斯克在 Neuralink 2023 年的 Show and Tell 发布会上,把这个数字放在了整场演讲的起点。他要回答的问题是:如果有一天真的出现了比所有人类加起来还聪明的数字智能,人类靠什么跟它沟通,又靠什么在它的世界里保留一点位置。
他的答案听起来有点绕:先去解决脖子以上那个几瓦功耗的生物处理器和外部世界之间的带宽问题。
为什么造火箭的人要钻进脑子里
Neuralink 的长期目标被马斯克称为“全脑接口”(whole brain interface):一个通用的输入输出设备,长期看能和大脑的任何一个部分对话,短期看能接入某个特定脑区,解决一批让人失去基本生活能力的问题。
他承认这个长期目标听起来很玄,但那确实是这家公司成立的第一动机。
如果数字超级智能比任何人类都聪明得多,我们怎么在物种层面降低这个风险?就算是一个非常非常友善的 AI,我们怎么跟着一起走,怎么参与进去?(马斯克)
他的判断是,人类跟 AI 之间最大的限制落在带宽上,算力和算法都排在后面。你思考的速度被卡在几十比特每秒,而机器之间的对话可以用千兆、太比特来衡量。这个落差如果一直存在,人类在这场关系里就永远是个反应很慢的旁观者。
所以 Neuralink 要做的事情,从最务实的角度看,是把一个瘫痪的人重新接回数字世界;从最长远的角度看,是给人类整体修一条更宽的信道。
人类早就在事实上和机器合体了
马斯克在台上讲了一个很多人有共鸣的细节:你把手机落在家里,出门会下意识地拍口袋,那种感觉像少了一条肢体。
按他的说法,手机、电脑早就是我们自己的延伸。真正的问题在于这条连接太窄,尤其是从人往机器这个方向。你戳屏幕的速度、说话的语速,都被几十年前的物理条件锁死了。
按这个逻辑继续推,脑机接口要做的其实是一件事:把这根管子加粗。它的意义不止是让残疾人重新能打字,还在于让人类整体的输出速率往上跳一个量级。
从原型到产品,难一百到一千倍
发布会的前半段,马斯克花了不少时间讲一件听上去不那么性感的事:把东西做出来和把东西量产,难度完全不在一个量级。
我常说原型容易,量产难,大概难一百到一千倍。要让它安全、可靠、在各种情况下都能工作、价格可负担、还能大规模生产,这极其困难。(马斯克)
他引用了那句老话,百分之一的灵感加百分之九十九的汗水,然后自己补了一句:也许其实是百分之九十九点九。他用登月举例:想到要去月球很容易,真的去很难。
这个铺垫本身没什么情绪,作用是给后面的技术细节做背景。因为接下来登场的每一项汇报,讲的都差不多是同一件事:怎么把一个实验室里的原型,变成一个能反复生产、反复测试、反复维修的工业产品。
关于人体试验,他给的时间点是:大部分文件已经提交给 FDA,如果顺利,大概六个月后可以把设备植入第一个人类。
他还特意讲了一个容易被忽略的原则:在把设备放进动物之前,他们要做大量的台架测试、加速老化测试,还有一个用橡胶做的“假脑”模拟器,用来模拟大脑的质地。他的原话是,动物植入必须是验证性的,不能是探索性的。也就是说,只有在台架上已经把能想到的都试过了,才会去动一只羊、一头猪或者一只猴子。
已经会玩 Pong 和打字的猴子
现场放了两段猴子视频。
第一段是 18 个月前的 Pager,用意识玩 Pong。马斯克强调了一个很多人没注意的点:你根本看不见植入体。它被做到和头骨被取下的那一块厚度一致,装上去之后外观上完全正常,就像把一块头骨换成了一块智能手表。
他顺带讲了一个细节:猴子本来不会用摇杆玩 Pong。团队先教会 Pager 用摇杆玩,再把摇杆拿走,换成神经信号。那段视频后来在网上被反复传播,很少人知道前面的训练过程。
第二段是另一只猴子 Sake,用意识在虚拟键盘上移动光标,逐格点到高亮的字母上,拼出一句话。马斯克老实交代了这一步的局限:猴子其实不会拼写,真正要拼什么是人选的,所以这不能过度吹。但他也说清楚了这个能力的意义:一个四肢瘫痪的人,哪怕其他什么都做不了,也能靠这个比有手的人更快地操作手机。
从 Pager 那次演示到现在,实验猴群扩展到了六只,Pager 和 Sake 都换上了新一代植入体。第一台设备在 Pager 身上已经工作了一年半,没有观察到不良反应。这点对他们很重要,因为可更换、可升级是这款产品的基本要求。
我们的第一代量产设备大概相当于 iPhone 1。你肯定不想让一台 iPhone 1 永远留在自己脑子里,而 iPhone 14 已经摆在旁边。(马斯克)
所以设计上必须做到完全可逆:能取出来,能在原位换一个新的,如果哪天它不工作了,也能换。这条要求写在设计指标里,没有商量余地。
还有一段容易被忽略的话:猴子做演示是自愿的。它们自己走到设备前,完成任务,奖励是一杯香蕉奶昔。Sake 没有被绑在椅子上。
前两个目标:恢复视觉,重建运动
Neuralink 在人身上的第一批应用锁定在两个方向。
第一个是恢复视觉。马斯克说了一句比较硬的话:哪怕一个人天生失明、从来没有过视觉体验,他们相信也能让他看见。理由是视觉皮层还在,只要能在那片区域写入信号,大脑就会自己把它解读成图像。
第二个是运动皮层。最初的目标是让几乎完全无法控制肌肉的人,比如处于斯蒂芬·霍金那种状态的人,能用意识操作手机,速度超过有手的人。更进一步的方案是桥接:从运动皮层读取信号,送到脊髓里的另一组 Neuralink 设备,绕开断掉的那一段。
听起来也许像奇迹,但我们确信,对脊髓完全断裂的人恢复全身功能,不存在物理上的障碍。(马斯克)
这句话在现场换来了掌声,后面负责脊髓项目的工程师讲得比这谨慎得多,细节放在后面。
这场发布会真正想说的是招人
马斯克中途把话挑明了:这次更新的主要目的是招聘。
他提到一个常见的误解,很多人觉得自己不懂生物学、不懂大脑,所以不可能去 Neuralink 工作。他的反驳是,把这家公司需要的能力拆开看,很大一部分和做一块智能手表、一部现代手机需要的能力是重叠的:软件、电池、射频、感应充电。
当然也有专属的部分,动物照护、临床、法规事务,还有机器学习。他在这里给了一个挺漂亮的说法:大脑是一个生物神经网络,解读生物神经网络最好的工具,是数字神经网络。
他的原话是,如果你擅长做手表、手机、电脑这类复杂设备,你的能力在这里非常有用。
一枚硬币大小的植入体和给它做手术的机器人
Neuralink 的创始团队成员 DJ Seo 接着讲了产品本身。他 13 岁从韩国搬到美国,需要重新学一门语言,那时候他就在想有没有更高效的沟通方式。后来看《黑客帝国》里 Neo 一秒钟学会功夫,他觉得这件事值得干一辈子。
他给出的产品定义是两条腿走路:一个叫 N1 的植入体,一个叫 R1 的手术机器人。
N1 的大小接近一枚硬币,超过一千个通道,能记录也能刺激。电极做在一种叫“线”(thread)的柔性薄膜阵列上,整个设备完全植入、完全无线,手术后藏在皮肤下面,外观上看不见。电池可以无线充电,用户在家里就能充。
他们在这三个方向上做取舍:安全、可扩展、能触达大脑的各个区域。安全是为了让这项技术能被广泛接受,可扩展是为了规模上来之后成本能降下去,触达更多脑区是为了功能可以不断扩展。
R1 机器人负责植入。那些线只有几个红细胞那么宽,要在一颗一直在动的大脑里可靠地插进去,还要避开血管。DJ 现场做了演示:在大脑模型上,把 64 根线一根根从硅基座上剥离、插进预定位置,全程大约 15 分钟。整场发布会进行到后半段时,机器人已经把 64 根线全部插完,每根 16 个电极,一共 1024 个通道,血管全部避开。
产线后来搬到了奥斯汀的专属工厂。报告里专门点出一个细节:设计工程师同时也要上产线做装配和调试。这大幅缩短了迭代周期。手术室也建了起来,他们在奥斯汀有了自己的场地,这是通往自建 Neuralink 诊所的第一步。
光标速度翻倍了,但还比手慢
负责神经解码的工程师讲了猴子玩游戏的原理:从运动皮层记录神经元活动,训练一个神经网络,从放电模式预测光标速度,然后用这个解码器让猴子只靠想就能移动光标。
2021 年初的水平是准,但慢。一年之后,速度快了将近一倍。他在台上承认,还是比他本人用鼠标慢,所以团队还在想办法。
速度只是一部分,功能是另一部分。几十年的软件生态都是为鼠标和键盘设计的,不可能为了脑控重做一遍。他们的做法是让猴子在各种任务上训练:左键、右键、点击拖拽、光标打字、手写、手势,然后针对每种行为设计解码算法。
这里有个很多人想不到的细节:反馈。你按一个按钮,手指能感觉到它被按下去。脑控用户没有这个触觉。他们的过渡方案是视觉反馈,光标颜色随“按压力度”变化。
打字这块,虚拟键盘已经能用了,但虚拟键盘本身就慢,还挡屏幕。斯坦福之前有研究让人想象手写字母,从神经活动里解码,速度比逐键点选快得多。
Neuralink 用猴子复现了这个思路,遇到的问题是数据量:每个字符都要几百个样本才能训练出来,扩展不了。
他们的解法很巧:不去直接解码字符,先解码手在屏幕上的运动轨迹,再把轨迹交给现成的手写识别模型。这样一来,一个通用的手部轨迹解码器就能支持任何语言的字形。
这意味着你可以写英文、希伯来文、普通话,甚至是猴子语,我们都能知道你想要一根香蕉。(Neuralink 工程师)
今天的信号和昨天不一样
软件工程师 Bliss 讲的是可靠性,这一段是整场发布会里最诚实的技术汇报。
他的目标是让瘫痪用户操作电脑的可靠程度,接近他自己用鼠标键盘的水平,也就是几乎从不出错。
第一层保障是离线仿真。他们做了一个模拟大脑,往服务器机架上的植入体里注入合成神经信号。从植入体的角度看,它就在一颗真脑子里。这套仿真跑在每一次代码提交上,从硬件一路验证到解码器。
仿真还不够。真实世界里,神经信号本身每天都在变。
他放了一张图:同一只猴子、同一个通道,每一天的平均放电率都不一样。如果用某一天的数据训练解码器,第二天接着用,放电率的漂移足以让模型输出产生系统性偏差。表现出来就是光标往右上角走特别费劲,往左下角走很顺。
他们试的方向包括:用很多天的数据一起训练,找跨天稳定的模式;以及在植入体上持续统计神经活动,用最新的估计去预处理数据,再喂给模型。
另一个问题是延迟。从大脑放电到光标动,中间只要有抖动,用户就会控制不住地过冲。
他们做出的一项改进叫相位锁定(phase lock):把每个数据包的发送时刻,精确对齐到蓝牙射频唤醒的那一瞬间。这样从放电到进入神经网络预测的时间被压到最短。他展示了改进前后的延迟分布,平均值大幅下降,方差也降了下来。用户感受到的变化是光标反应变得可预测。
32 毫瓦,和爬上去自己充电的猴子
芯片工程师 Avinash 讲的是那枚定制 ASIC:1024 个独立通道,模拟加数字电路,要测到幅度小于 20 微伏的放电信号,同时塞进一枚硬币大小的壳里。
他重点讲功耗,因为目标很具体:让用户一整天不用充电。
2018 年的方案是把每个通道的每个采样都传出芯片,功耗极高。2020 年他们做了一件关键的事:把放电检测搬进芯片。神经元是靠放电传递信息的,只在检测到放电时才往外传,本身就是一种极其高效的压缩。
两年优化下来,系统总功耗降到 32 毫瓦,电池寿命翻倍。
算法是这样的:先做 500 赫兹到 5 千赫兹的带通滤波去掉带外噪声,再用噪声底估计给每个通道生成自适应阈值,最后由检测模块识别一次放电的三个关键点。识别三个点除了判断有没有放电,还能判断放电的形状,这对区分同一通道旁边的不同神经元很有用。
最新的一个优化利用了一个事实:神经元放电其实很稀疏,检测器大部分时间都在找第一个点,很少需要找后两个点。于是他们把每个通道的状态拆成“热态”和“冷态”,热态每个周期都访问,冷态只在越过阈值后才访问。内存访问量降了 30%,功耗降了 15%。
下一代芯片是 4096 通道,尺寸不变,驱动电压更高,还加了一颗 ARM 核。四颗拼在一起装进一个植入体,总共 16000 个通道,还是一枚硬币那么大。
充电这块由电气工程负责人 Matt 讲。植入体的充电有不少约束:充电范围要大,不能靠磁铁对齐;要能抗干扰,要快,不能让人觉得麻烦。最硬的一条是安全,接触脑组织的那一面,温升不能超过 2 摄氏度。
他们的充电器迭代了三代。2020 年 8 月那头叫 Gertrude 的猪用的还是第一代,一个小圆饼。现在的量产版是一个铝制底座加一个可拆卸的远程线圈,线圈面积是第一代的四倍。
现场放了一段视频:猴舍里嵌了一个线圈,旁边加了个香蕉奶昔泵,猴群学会了自己去充电。Pager 爬上去坐好,充电器自动检测到他的存在,从搜索状态切换到充电状态,屏幕上实时显示功率和电流。
技术上有个挺有意思的问题:高品质因数的线圈在远距离下效率很好,但一靠近植入体,会出现“峰值分裂”,最高效率被推到更高的频率上,超出了法规允许的 ISM 频段。下一代的解法是动态调谐,实时调整收发线圈的谐振频率,在性能恶化前把参数改掉。
第三代充电器已经在做,加了双向近场通信,控制延迟更低,热管理更好,充电更快。
硬脑膜:一毫米的膜,卡了两年
手术工程负责人 Christine 讲流程:定位、切口、开颅、取下硬脑膜、插入电极丝、放入植入体。机器人只做插丝那一段,其余由神经外科医生完成。
她讲了一段私人经历。她上学的时候,父亲得了 ALS,逐渐失去走路、用手、说话的能力。她在网上只能找到零星的案例,某个人有一个定制的辅助机器人,选项少得让人沮丧。而全世界的瘫痪患者有几十万,神经外科医生大概每一百万人里只有十个,培养一个要十年以上,时间贵得吓人。
要让这项手术可及、可负担,就必须回答一个问题:一个医生怎么同时监管很多台手术。
她拿 LASIK 做类比。三十年前激光手术刚出来的时候,机器人只做最核心的那一小步,其余都是医生做。一轮轮迭代下来,机器做的越来越多,医生做的越来越少,现在整个过程只要几分钟。
要让机器人接管更多步骤,最大的障碍是硬脑膜。这层膜很坚韧,不透明,把大脑和颅骨隔开。团队花了两年时间,才把插入这件事做得足够可靠。
他们做过一些原型工具。超声切割刀能切骨不切软组织,但速度一快就发热严重。振荡锯用了专门设计的刀片,能切硬物不切皮肤,简单可靠,可就是切不了任意深度和形状。要在人头上开任意形状的孔,只能用数控铣,那就必须解决“怎么保证每一次都不切过头”的问题。他们用力度反馈和阻抗反馈来控制,确保刀具不会穿进脑子。
机器人这一侧,难的是光学。插一根丝需要三条光路:可见光成像看针和丝,激光干涉测量(OCT)实时追踪大脑的运动,还要有照明。这些东西都得塞进颅骨开口的底部,靠近骨壁的时候空间极其局促。团队的解法是用偏振把三条光路合并成一套光学堆栈。合并之后还带来一个额外能力:实时避开血管。大脑在动,一开始选好的点可能已经不是你要插的地方,机器人能在针下去的过程中识别血管,判断这里能不能插。
下一步要拿掉的是切开硬脑膜这一步。机械工程师 Alex 讲了为什么难:设备装好之后,几个月里所有空腔都会被组织填满。电极丝本身很小,取出来很容易,难的是上面长出来的那层包膜。他们用组织学和显微 CT 研究这层膜,试过很多设计路线,效果最好的是“少侵入”:不切开硬脑膜,保留人体自己的屏障。这同时也让手术更简单、更安全。
硬脑膜不透明,血管看不见。他们的方案是用医用荧光染料显影硬脑膜下面的血管,再把 OCT 用到更深的组织结构。未来结合术前 MRI,可以在不暴露脑表面的情况下精确定位。
40 微米的针要穿过它
针头制造负责人 Sam 讲了一组数字,把难度说得很直观:人的硬脑膜厚度可以超过 1 毫米,而他们的针是 40 微米。如果把针放大到铅笔那么粗,硬脑膜就相当于四英寸厚。
针要解决的问题不止是穿过去。它还得和外面的保护套管一起夹住电极丝,把它从硅基座上剥下来,运到脑表面,插进去之后再松开。针不够锋利,就会一直把柔软的脑表面压出凹陷而不刺穿;悬空长度太长,针会弯。
材料是直径 40 微米的钨丝,掺一点铼增加延展性。他们自己设计了飞秒激光加工设备来切针和套管,精度到亚微米。
量产能力的进步很具体:几个月前,一个熟练工做一根针要 22 分钟,良率 58%。现在 6 分钟,任何人培训几分钟就能做到 91% 良率。一键完成切割和测量,数据自动上传,机器人用这根针时会读取它的实际尺寸。
老设计能穿过硬脑膜,但保护电极丝不够。新设计在台架上能穿过九层硬脑膜、总厚度 3 毫米,远超人体可能遇到的情况。
他们还遇到过一个很隐蔽的失效模式:有段时间针能稳定穿过硬脑膜了,但把样品放进显微 CT 一看,夹持力太大,把电极丝往表面下面拽了一点点。这个问题对周围材料的性质极其敏感,同样一套工艺,换一种模拟材料,可能一次都不发生,也可能每次都发生。这就是他们为什么要花大力气做仿真材料。
合成一个大脑
微纳加工研发负责人 Leslie 讲的是“大脑替身”(brain proxy)。逻辑很直白:直接从生物体上学东西太慢,那就做能模拟生物环境的合成材料,先把台架上能学的都学完。
难度在于植入环境有很多解剖层,每层性质都不同,而且随着时间推移,植入位置会愈合,新组织会长出来填满空隙。再加上心跳和头部运动带来的位移,问题变得更复杂。
他们的做法是用生物学的反馈来设计材料:做组织的力学表征,分析电极丝和组织界面的相互作用,很多表征甚至是在手术过程中完成的,方法是对手术机器人做改造,让它同时充当一台灵敏的表征工具。
第一代替身非常简陋,琼脂加一张石蜡膜,放在盘子里。就是靠它,机器人插丝这件事被反复练熟了。现在的版本是复合水凝胶,模量更接近真实人脑,还加了硬脑膜替身和可注射的软组织替身,用来做模拟拔出的测试。
他们的愿望清单很长:带软组织、颅骨、皮肤甚至全身的手术替身;能模拟运动、血管和电生理活动的脑替身;用于生物相容性和电刺激测试的生物替身。已经在做的工作包括实验室培养的脑类器官。
目标很清楚:在台架上多学一点,在动物身上就少用一点。最终的理想是完全替代。
给盲人开灯
视觉项目负责人 Dan 原来是做视觉神经科学的。他加入的原因是,在这款设备上看到了给因为眼外伤或疾病失明的人恢复视觉的可能。
设备适合这件事有几个原因。每个通道都能刺激,也就是说能绕过眼睛,直接在大脑里生成图像。通道数可以做得很多,通道越多,图像的分辨率越高。机器人能把电极插得很深,而人的视觉皮层正好藏在大脑内侧面一条叫距状沟的褶皱里。
他放了一张 MRI,把距状沟标成红色。那里有一张视觉世界的地图,每侧面积大约一张信用卡,展开铺平之后图像是倒过来的,而且严重变形:视野中心、也就是注视点附近被极大放大。林肯那张照片里,他的右眼在画面上很小,在视觉皮层里却占了将近四分之一的面积。
过去半个世纪,视觉神经科学积累了大量的知识,大部分来自对猕猴单个细胞的记录。一个核心发现是:视觉皮层里每个细胞只代表视野里极小的一块区域,你的整个知觉是无数个微小感受野拼成的马赛克。
他们在两只猕猴 Cade 和 Dash 的视觉皮层里植入了设备,用随机棋盘格电影加反向相关的方法,一次把上千个电极的感受野全部测出来。结果能看到大量感受野集中在注视点附近,也有一些散在周边,来自埋得更深的记录点。
接下来是刺激。他们先训练猴子看向屏幕上的真实光点,再插入电刺激。猴子看到光幻视,会朝那个方向做眼跳。屏幕上什么都没有,它也没有别的线索,但还是会看过去。通过改变电流,可以测出这个光点多大、多明显。
现场放了录像,先是四分之一速度,然后是实时。猴子做任务做得飞快,做完看起来就不想干了。
一个光幻视就是一个像素。用 1000 个电极能拼出很粗糙的图像。下一代 16000 个电极,两侧都装就是 32000 个光点。
我们的目标,是给一个在黑暗里活了几十年的人把灯打开。(Dan)
脊髓:断掉的那一段可以接回来
下一代项目负责人 Joey 讲了脊髓这一侧。
先讲一点神经解剖:运动的意图在运动皮层产生,沿着长神经纤维经脊髓下行,这些叫上运动神经元。在脊髓里它们和另一个运动神经元形成突触,后者把指令送到肌肉。脊髓损伤断掉的就是其中一环。
如果把电极插进脊髓,插到运动池里、紧挨着下运动神经元的位置,就可以直接刺激它们,让肌肉收缩。难点在于脊髓很娇嫩,而且在椎管里移动幅度很大,硬电极容易伤到组织,也容易被组织弄坏。
Neuralink 的电极又小又软,机器人能插得很深。现场展示的靶向视图里,电极横跨好几毫米的脊髓,深入到腹角,紧贴下运动神经元。距离越近,动作越精确。
然后是猪的演示。一头猪在跑步机上走,身上不只有一个 Neuralink 设备:大脑里有一个,脊髓里也有一个。团队实时读取神经数据,解码髋、膝、踝的运动。
他们要的方向是反过来:刺激脊髓,让动作发生。
视频里,猪在地上拱来拱去找吃的,地板上有一块蓝色方框。这是它自己选择的“自愿参与区”,进去就说明它愿意接受刺激,离开就停。
刺激某一个电极,腿抬起;刺激另一个,腿伸直。可以在不同电极上组合出时序模式,也可以做出持续的强力收缩,那种你站起来或者负重时需要的收缩。
闭环是这样的:大脑运动皮层记录放电,无线实时传出来,解码成刺激模式,送到脊髓腹角的对应运动池,激活下运动神经元,肌肉收缩,动作发生。感觉这一侧反过来走:背角记录感觉信号,解码后刺激大脑的感觉区,产生触觉和本体感觉。
运动没有感觉其实很难,你可以回忆一下四肢麻木时动一下是什么感受。(Joey)
两个回路合起来,运动意图从大脑出来驱动身体,身体的感觉再回到大脑。
问答里的几个硬问题
发布会后半段是提问,很多有价值的信息藏在这里。
有人问数据会不会开放。马斯克说会,把数据集放到官网上,任何人都能用。他觉得这对做 AI 研究的人会很有价值,可以在此基础上训练大脑的基础模型。
有人问长期植入之后的瘢痕组织。负责微纳加工的 Zach 回答,他们能做的是把电极做得更小更柔,减小免疫反应。同时也在做生物涂层,一方面减少炎症,一方面让电极丝更滑、更容易取出。
关于信号漂移,有人问电极会不会在脑子里移动。马斯克的回答是,电极位置其实相当稳定,因为设备固定在颅骨上,而电极丝有一段盘绕的松弛结构,会把它限制在原来的位置。Bliss 补充说,放电率的变化不太可能是换了一个神经元,因为即使平均放电率变化很大,放电波形还是稳定的。他也承认这不完全排除那种可能。
关于寿命,工程师 Jeremy 给了一组数据。Pager 有 617 天是有用的脑机接口信号。当前版本的设备已经接近一年。加速老化测试里,上一代设备积累到 8 年的等效时间,当前这一代积累到 4 年,还在继续。他把寿命拆成三个因素:密封、电池和内部电子、电极丝。密封理论上能撑二十年以上,目前还没见过失效。电池按使用强度算,三年时大概能保持 80% 的续航,团队正在做翻倍和四倍。真正的瓶颈在电极丝,他们正在推非晶碳化硅绝缘,认为能撑到五年以上,同时开始试原子层沉积。
有人问蓝牙带宽够不够。他们的回答分几层:现在是压缩后每秒 50 KB 左右,蓝牙链路是 150 KB。真正的解法是把机器学习模型直接跑在植入体上,只往外传解码结果。负责这项工作的工程师说,这相当于让复杂神经网络跑在一个车库门遥控器的算力上,这个工程问题挺好玩。射频这边在看 500 MHz 的超宽带,能到 6 到 10 Mbps,延迟也低得多,另外还在看 W 波段。
马斯克补了一句他自己的观察:控制一台手机真正需要的数据率低得可笑,大概每秒 10 比特。
我们的拇指就是两根移动很慢的肉条,这是个很低的标准。(马斯克)
他还提到一个可能会改变手术形态的方向:如果电极丝足够细、足够准,就不需要切开硬脑膜,直接在硬脑膜上打一堆小孔把丝送进去。
那样恢复时间会快得离谱,理论上整个手术可以像 LASIK 一样只要十分钟。(马斯克)
关于 FDA 关心什么,团队列了几条:设备取出的安全性、植入体的热台架测试、生物相容性和化学表征。他们还特别提到一个容易被忽略的困难:产品的很多部分是全新的,薄膜阵列和手术机器人在现有器械里找不到对应物,所以不能靠文献来支撑安全性和有效性,只能自己做测试,一项一项证明。马斯克在这里给了一句很个人化的判断:如果自己的孩子脖子断了,在这个时间点,他会愿意让他接受这个手术。
有人问有没有和运动障碍群体合作。团队说有患者顾问委员会,成员包括四肢瘫痪的人。有位顾问来办公室聊天,说自己最想用 Neuralink 做的两件事:一是炒股,好赢过他兄弟;二是打射击游戏。讲这件事的工程师说,打动他的是这份寻常。
有人问能不能做得更深,进入与神经精神疾病、记忆相关的深部脑区。马斯克的回答是,颅骨里那部分设备本质上不变,改的是电极丝要更长、机器人的针要更长,理论上可以去任何地方。机器人团队负责人补充,现在的成像只能看到大约一毫米深,再深就要换手段,超声和光声成像都在考虑范围内。还有一个更理想的方向:如果丝细到能直接穿过血管,就不会有出血问题。他对比了现有的深部脑刺激:开一个 14 毫米的骨孔,把一根 2 毫米粗的导线盲插进 6 到 8 厘米深,还有 1% 的概率出现无法控制的出血。他们的针是 40 微米。
有人问单个通道出故障怎么办。回答是,通道数远远超过解码需要的数量,现在 1000 个通道,未来 16000 个,坏几个不影响整体性能。
最后一个问题关于输入带宽。团队说这还是个开放问题,方向包括手写解码、不同的键盘布局、滑动输入,以及接入语言和言语相关的脑区。马斯克补充说,一个通用的输入输出设备,对理解大脑本身的推动会大到难以用语言形容。
我们现在对大脑里发生的事情,很大程度是在猜。一旦有了直接的输入输出,我们学到的东西会多出好几个数量级。(马斯克)
整场听下来,有几处可以单独说说
第一,这次展示的重心已经从“脑机接口很酷”挪到了“我们把它当成一个制造问题在解”。飞秒激光的良率从 58% 到 91%,阻抗扫描从 4 小时到 20 秒,检测设备成本降到五分之一,功耗从全采样降到 32 毫瓦。这些数字放在一起,说明这家公司现在最像的是一家做消费电子的公司,只不过产品要装进颅骨。
第二,最难的部分落在那些没人预料到的地方。卡了他们两年的是硬脑膜这层一毫米厚的膜,以及夹持电极丝时的几十微米偏差。这种事在 PPT 里永远不会出现。
第三,时间表上,团队自己给的说法比马斯克保守。马斯克说六个月可以植入第一个人;工程汇报里反复出现的是“还在研究”“还没解决”“还需要更多测试”。这两套说法并不矛盾,前者讲的是里程碑,后者讲的是这条路还没铺完。
第四,普通人能感知到的第一批改变,大概会落在很朴素的地方:一个瘫痪的人用自己的手以外的方式,把手机用得和有手的人一样快。科幻场景还要往后排。
如果你对这件事有兴趣,下面这几条可能对你有用。
- 如果你做硬件、嵌入式或芯片,重点看他们公开提到的几个方向:超低功耗数模混合设计、无线充电的动态调谐、植入级封装和密封。这些和可穿戴设备高度重合,技能可以直接迁移,去官网的招聘页面对照一下自己的项目经历。
- 如果你做机器学习,值得投入的题目是神经解码里的漂移问题:跨天的信号分布变化、把模型压到植入体上运行、低延迟的实时推理。这几个题目现在就有公开的论文和数据可以做。
- 如果你在学术界,盯住他们承诺开放的猴子神经数据集。这类数据在公开领域一直很稀缺,一旦放出来,做时序建模和表征学习的人会很快用上。
- 如果你关注产业节奏,记住一条判断线:什么时候他们能证明“不切开硬脑膜也能完成植入”,手术的恢复时间和成本会有一个台阶式的变化,那才是大规模推广的起点。
- 如果你是患者或家属,别只看发布会,去看他们临床试验的官方登记信息和入组标准。人体试验的名额、适应症、随访要求都写在里面,比任何二手报道都准。
- 如果你想给孩子或者自己规划方向,这篇访谈里最实在的一句是:这家公司里几乎每一个大学专业都能找到对应的人。把一件事做到足够深,比押注某个热门专业更可靠。
0:03foreign [Music] foreign [Music] foreign [Music] thank you [Music]
2:49points [Music] foreign [Music] [Applause] [Music] [Applause] [Music]
4:07I don't know [Music] foreign [Music] [Applause] [Music] foreign
6:01[Music] foreign [Music] [Applause] foreign [Music] thank you
9:00[Music] foreign [Music] foreign [Music] foreign [Music] foreign [Music]
11:51foreign [Music] foreign [Music] foreign [Music] foreign [Music]
13:56[Applause] [Music] ways [Music] [Applause] [Music] thank you [Music]
15:39foreign [Applause] [Music] foreign [Music] foreign [Music] watch
18:45ES [Music] foreign [Music] foreign [Music] foreign [Music] foreign
21:53[Music] foreign [Music] foreign [Music] foreign [Music] foreign [Music]
24:40foreign [Music] [Applause] [Music] thank you [Music] [Applause] foreign [Music]
26:06thank you [Music] foreign [Applause] [Music] I don't know [Music] [Applause] [Music] [Applause] [Music]
27:18foreign [Music] foreign [Music] foreign [Applause] foreign [Music]
30:39foreign [Music] foreign [Music] thank you foreign [Music] foreign
33:10[Music] foreign [Music] foreign [Music] foreign [Music] foreign [Music]
36:04thank you [Music] foreign [Music] foreign [Music] [Applause] [Music] [Applause] [Music]
38:04I don't know thank you foreign [Music] [Applause] [Music] [Applause]
39:43foreign [Music] foreign [Music] [Applause] thank you [Music]
41:59thank you [Music] foreign [Music] foreign [Music] thank you [Music]
45:14foreign [Music] foreign [Music] foreign [Music] foreign [Music]
47:25foreign [Music] foreign [Music] thank you [Music] I don't know [Music] [Applause] [Music] thank you [Music]
50:30[Applause] [Music] thank you [Music] thank you [Applause] foreign [Music]
53:27thank you [Music] foreign [Music] foreign [Music] foreign [Music]
56:55foreign [Music] foreign [Music] foreign [Music] thank you [Music]
59:15foreign [Music] foreign welcome to the neural link show and tell so we've got an amazing amount of new developments to share with you that I think are incredibly exciting as well as tell you about the future of what we're planning to do here it's now this is meant to be a technical podcast or sort of like a work I'm going to provide an overall summary and then we're going to have a number of members of the the neural link team come in and give a deep technical overview of the various areas so yeah so let me move forward with the the overall summary now some of the things I'm going to say are things you've well if you've been following your link you've already heard before
1:00:55that for for a lot of people out there they've no idea what neurolink does and so I'll be a little bit repetitive of things you may already know but that others do not so the the overall the overarching goal of neuralink is to create a ultimately a whole brain interface so a generalized input output device that in
1:01:24you know in the long term literally could interface with every aspect of your brain and in the short term can ask can interface with any given section of of your brain and and solve a tremendous number of things that that cause debilitating issues for people so you know so our long term is like
1:01:51I mean I'll talk a little bit about a long-term goal it's going to sound a little esoteric but it's the it was actually the sort of my Prime motivation which was you know kind of what what do we do about AI like what do we do about artificial general intelligence if we have digital super intelligence that's
1:02:16they're just much smarter than any human how do we mitigate that risk at a species level how do we mitigate that risk and then even in a benign scenario where the AI is very very benevolent then how do we even go along for the go along for the ride how do we participate and the conclusion I the the thing that
1:02:49the biggest limitation in going along for the ride and in aligning AI I think is the is the the bandwidth the the how quickly you can interact with the computer so we're we are all already cyborgs in a way in that your your phone and your computer are extensions of yourself and if you I'm sure you found like if you leave your phone behind you end up tapping your pockets and and it's like having missing limb syndrome like where you know the phone is it is leaving your phone behind is kind of like a missing limit at this point you're so used to interfacing with it you're so used to being a de facto cyborg
1:03:34But but so what's the limitation on on a on a phone or a laptop limitation is the rate at which you can receive and send information especially the the speed with which you can send information so if you're interacting with a phone it's limited by the speed at which you can move your thumbs or the speed which you can talk into your phone this is an extremely low data rate you know maybe it's like 10 optimistically 100 bits per second but a computer can can communicate at you know gigabits terabits per second so this is the fundamental limitation that I think we need to address to mitigate the long-term risk of artificial intelligence and also just go along for
1:04:24the ride and yeah so everybody likes it that's that that's an esoteric explanation that I think will appeal to a niche audience some of whom may be here but and that's a that's a very difficult problem so even if we do not succeed with that problem I think we we like we are confident at this point that we will succeed at many it's it's solving many brain injury
1:05:00issues spine injury issues along the way so yeah so anyways so actually we have Justin Roiland in the audience says the hi Justin so it's a little Rick and Morty reference here the it was great Rick and Morty episode about intelligence enhancement of your dog and what's the worst that can happen so anyway Rick and Morty I recommend it
1:05:38so for so we want to be able to read the signals from the brain you want to be able to to write the signals you want to be able to ultimately do that for the entire brain and then also extend that to communicating to the rest of your nervous system if there's a if you have a sort of a severed spinal cord or neck
1:06:05so now this is a this video is now 18 months old so this is pager who is playing monkey mind pong so this is a pager has a neural link implant in this video and the thing that's interesting is that you you can't you can't even see the the neural implant so it's that it's we've monitorized the neural implant to the point where it matches the the thickness of the skull that is removed so it's essentially the it's sort of like having an Apple Watch or a Fitbit replacing a piece of skull with like a you know a smart watch for lack of a better analogy so so you can see you can really can it he looks pretty he's normal and I think that's pretty important
1:07:07if you have a neuralink device like I could have a neuralink device implanted right now and you wouldn't you you wouldn't even know I mean hypothetically I may be one of these Demos in fact one of these demos I will so yeah anyway so so here's here's first of all it's kind of wild hey monkeys can play Pong they're like they can't actually pay pong if you give them a joystick so Pedro first learned to pay pong with a joystick so I'm like that was novel it's like I didn't know monkeys could play Punk but they can and then so we first trained pager to play Pong with a joystick then we took the joystick away and have the neural link and now this is he's playing
1:08:05telepath it's a telepathic video games essentially so what we've been doing since then is I we've been on the very difficult Journey from prototype to product and I've often said that prototypes are easy production is hard it's really I'd say a hundred to a thousand times harder to go from go from a prototype to a device that is safe reliable Works under a wide range of circumstances is Affordable and done at scale it's it's insanely difficult
1:08:53and there's an old saying that yeah that it's one percent inspiration 99 perspiration but I think it might be 99 or 99.9 perspiration you know the best example I could give of an idea of being easy for the execution being hard is going to the Moon it's the idea of going to the Moon easy going to the Moon very hard
1:09:16so and we've been working hard to be ready for our first human and obviously we want to be extremely careful and certain that that it will work well before putting a device in a human but we're we're submitted I think most of our paperwork to the FDA and we're we think probably in about six months we should be able to have uphost neural Link in a human so [Music] [Applause]
1:09:53but as I said we we do everything we possibly can to test the devices before not even not even going into a human before even going into an animal so we do bench top testing we do accelerator accelerated life testing we have a fake brain simulator that has the the texture and it's like emulating a brain but it's sort of rubber and so any before we would even think of putting a device in an animal we we do everything we possibly can with rigorous bench up benchtop testing so we're not Cavalier in putting devices into animals we're extremely careful and we always
1:10:39want the device whenever we do the implant if it's in a sheep or a pig or a monkey to be confirmatory not exploratory so that we look we've we've done everything we possibly can with benchtop testing and only then would we consider putting a device in an animal and yeah we'll actually show you a demo later today of in a few hours really of of implanting in a brain
1:11:13proxy and if anyone in the audience wants to volunteer with a robot right there so let's see instance since the page of demo we've expanded to work with a troop of six monkeys we've actually upgraded pager they do varied tasks and we do everything possible to ensure that things are stable and replicable and that things like that the device lasts for a long time without degradation so and what you're seeing there is it looks like the Matrix but that's actually though that's a real output of neural signals so that that's that's not a simulation or just a screensaver or something that those are actual neurons firing that is one of the what one of the
1:12:05readouts looks like and here you can see sake it's one of other monkeys typing on a keyboard now this is telepathic typing so to be clear this is the he's not actually using a keyboard he's moving the cursor with his mind to the highlighted key now technically we can't can't actually spell and
1:12:41so I don't want to oversell this thing because that's that's the next version so the but what's really cool here is is sake of the monkey is moving the mouse cursor using just his mind moving the cursor around to the highlighted key and then spelling out what we what we want everyone to spell but and then so this this is a something that could be used for somebody who's who's say it quadriplegic or tetraplegic human even before we make the the spinal cord stuff work is being able to control
1:13:29a mouse cursor control a phone and we we're confident that you that someone who is has basically no other interface the outside world would be able to control their phone better than someone who has Working Hands so foreign upgradability upgradability is very important because our first production device will be much like an iPhone one and I'm pretty sure you would not want an iPhone one stuck in your head if the iPhone 14 is available so it's gonna be it's be able to demonstrate full reversibility and upgrade abilities so you can remove a device and replace it with the latest version or if it stopped working for any reason replace it it's that's that that's a
1:14:27fundamental requirement for the device at your link and I should say both swaki and Paige were upgraded to our latest latest and greatest implants so that that's been really over a year and a half now that that pagers had put the first implant and then the upgraded implant so this is a very good sign that it lasts for a long time with no observed ill effects
1:14:56I think it's also important to show that sake actually likes doing the demo and it's not like strapped to the chair or anything so it's yeah so the monkeys actually enjoy doing the demos because they and they get the banana smoothie and it's kind of a fun game so I I guess smart termic is like We Care a great deal about animal welfare and and I'm pretty sure like our monkeys are pretty happy you know so as you can see there's a quick decision maker on the fruit front so so for our the the first two applications we're going to aim for in humans are restoring vision and I think this is like notable in that even if someone has never had Vision ever like they were born blind we
1:15:59believe they could they can we can still restore vision so because the visual part of the the visual part of the cortex is still still there so yeah even if they've never seen before we're confident that they could they could see and then the the other application being in the motor cortex where we
1:16:28would initially enable someone who has no ability to almost no ability to operate their their muscles you know sort of like a sort of Stephen Hawking type situation and enable them to operate their phone faster than someone who has Working Hands but then even obviously even better than that would be to bridge the connection so take the out the signals from the
1:16:58motor cortex and let's say somebody's got a broken neck then bridging those signals to neural link devices located in the spinal cord so we're confident there are no there are no physical limitations to enabling full body functionality so I mean as miraculous as it may sound we're confident that it is possible to restore full body functionality to someone who has a severed spinal cord so yeah [Applause]
1:17:43so yeah all right and then I went to emphasize game that the primary purpose of this update is recruiting a lot of times people think that they you know they couldn't really work at neurolink because they don't know anything about biology or how brains work and the thing that we really want to emphasize here is that you don't need to because when you break down the the skills that are needed to make neurolink work it's actually many of the same skills that are required to make a smart watch or
1:18:27modern phone work so it's sort of you know software batteries radios inductive charging and you know as well as things that are specific to to us like animal care and clinical and Regulatory matters obviously machine learning that phrase is used a lot but we also need to interpret the signals from the brain which is a biological neural net and the best thing to interpret a biological neural neural net is a digital neural net so this is if there's one message I want to convey it is that if you have expertise in creating Advanced
1:19:19devices like watches and phones computers then your your capabilities would be of great use in solving these important problems that's that's that's one thing the message don't worry convey so let's see yeah so with that I guess DJ so [Applause] so DJ's was on the founding team of newer link and just made immense contribution to the company as of many of the others who will present but I want just to thank DJ for his immense contribution to neurolink foreign
1:20:25[Applause] thanks Elon when I moved from South Korea at age 13 and needed to learn a new language to communicate I wonder whether there are better and more effective means of communicating my thoughts to the outside world and watching Neo learn Kung Fu and The Matrix I remember thinking wow I want to work on that work on making that possible and today I believe that this is attractable engineering challenge since everything about your intentions your thoughts and your experiences are all in your brain encoded as firing statistics of action potentials if you're able to put electrodes in the right places with the right sensing and stimulation capabilities
1:21:23this and many other applications that Elon talked about are possible and we can help a lot of people I'm incredibly excited to be working on this ambitious yet important mission to make that future a reality here at neuralink and I'm also incredibly honored to be working with some of the brilliant colleagues scientists and Engineers across many engineering disciplines to work on this intersection of biology and Technology you'll hear from several of them today to learn about the breadth of technical challenges challenges we face and our progress in the last year and I think you'll find that for most of these challenges as Elon mentioned you don't need a prior
1:22:15understanding of how the brain works and that a lot of what we do is applying engineering first principles to biology so how do you create a high bandwidth generalized interface to the brain from day one we focus on a set of foundational technologies that are safe scalable and capable of accessing all areas of the brain these three axes safety scalability and access to brain regions really form the basis for how we engineer products here at neurolink safety because we want to make our devices as well as the installation as safe as possible so that we can drive the adoption of this technology and scalability because as we make our devices safer and
1:23:10more useful more people will want it and with scale we also want to make it more affordable and access to brain regions so that we can expand the functionalities of our Technologies so our first steps along these dimensions for our device is what we call the N1 implant it's a size of about a quarter and it has over 1 000 channels that are capable of recording and stimulating it's microfabricated on a flexible thin film arrays that we call threads
1:23:54it's fully implantable and wireless so no wires and after the surgery the implant is under the skin and it is invisible it also has a battery that you can charge wirelessly and you can use it at home so similarly for implanting our device safely into the brain we built a surgical robot that we call the R1 robot
1:24:26it's capable of maneuvering these tiny threads they're only on the order of few red blood cells wide and inserting them reliably into a moving brain while avoiding vasculars it's it's quite good at doing this reliably and in fact because we've never shown an end-to-end insertion of a robot in action we're going to do a live demo of the robot doing surgery and our brain proxy so who wants to see some insertions [Applause]
1:25:05with our patient Alpha who's lying comfortably on the patient bed this is what we call the targeting view so what you're seeing is this is a picture of our brain proxy and the pink represents the cortical surface that we want to insert our electrodes into and the black represents the vascular Shores that we want to avoid and what you're seeing is these hash mark with numbers that represents where we intend to put each of our threads
1:25:41so should we see some insertions so this is another view real quick on the left is the view of the insertion area and on the right what the robot's going to do is it's going to peel the array at the threads one by one from its silicon backing and insert it into the targets that we predetermined in the targeting View so
1:26:12there you go that's the first insertion [Applause] so we're going to see a couple more insertions the whole process of inserting about 64 threads in our first product is going to be around 15 minutes for this robot so yeah there's a second one that went in and we're going to do a third one
1:26:39there you go and then that's going to go in the background and we'll come back to it in the later part of the presentation [Applause] and as Elon mentioned we've been working very hard to go from prototype to Building Product as part of this one of the things that we did is to move our device manufacturing to a dedicated facility in Austin for scale-up manufacturing and what's important to highlight and is evident in this clip is that it's very typical for us to have our Engineers who design also work on the physical manufacturing line to build and debug and this has been extremely extremely critical in reducing our iteration cycle time and we've also scaled up our surgery
1:27:36so we now have a dedicated our own or in fact a double or in Austin and this is just a stepping stone before we eventually build our own neuraling Clinic so with this product N1 and R1 our initial goal is to help people with paralysis from complete spinal cord injury regain their digital Freedom by enabling them to use their devices as good as if not better than they could before the injury and as Elon mentioned over the last year this has been the central focus of the company and we've been working very closely with the FDA to get approval and to launch our first and human clinical trial in the U.S hopefully in the six in next six months [Applause]
1:28:36so hopefully this gives you a good overview of our product for the next hour we're going to go through a deep technical Dives on these topics to tell you about our technical challenges share some of our progress and preview what's coming next so with that over to near from my team who's going to talk to you about neural decoding [Applause] thank you and everyone my name is brand interfaces applications our goal is to enable someone with policies control a computer as well as me or even better would like to provide fast and accurate control with all the functionality of computers that works anytime anywhere so I'm very excited to show you how we
1:29:30are using the N1 device with our software and algorithms to achieve this last year we shared with you a video of page of the monkey controlling computer cursor with his brain so how do we do that just a brief reminder first we record is a neural activity from the motor cortex using the N1 device we have we can recall from over thousands of channels while he's playing with the joystick then we can train a neural net that predicts the cursor velocity from the patterns of his neural activity with this decoder you can then control a cursor just by thinking about it without even moving the joystick it can play with this decoder a variety of games also a grid task whereas moving
1:30:12the white dot towards the yellow targets every time he gets one he receive a drop of his favorite smoothie and he chooses to play this game every day here you can see his performance from early 2021 around the time we released the previous demo it's quite accurate but it's a bit slower than what we would like and cursing control is the foundation for interacting with most Computer Applications so since then we've been working to improve cursor speed and accuracy as you can see it's much much faster almost twice as fast [Applause]
1:30:52however it it's still still a bit slower than what I can do so we are working on creative ways to improve that now speed is not enough you want the full set of functionalities and for decades most software was built for mouse and keyboard control and it doesn't make sense to reinvent this entire ecosystem for brain control at least for now so we are working and we are designing a mouse and keyboard interfaces for the brain the way we do that is by training Pages pager and his friends on a variety of computer tasks and then designing algorithms to predict the behavior here you can see a few example of tasks in different phases of monkey training
1:31:34for example left and right click click and drag cursor typing Spike typing handwriting and even hand gestures now interacting with computer is bi-directional and feedback is very important I like when I click on a button and I can physically feel the button being pressed when a potential N1 user will attempt to click they won't be able to fill it an example of how we are addressing that is by providing a real-time visual feedback that represents the strength of the mural Click by changing the color of the cursor just by typing on a physical keyboard is much faster and easier than typing on an iPad keyboard this will make the band control much faster and easier to use
1:32:20typing one of the most important functionalities so you already sent this message and I want to show you the behind the scene of how this message was created and here you can see again exactly using the virtual keyboard tapping this message this virtual keyboard is similar to the one I use on my phone and with the speed and accuracy that we achieved so far typing on a virtual keyboard is already fast and easy however I never use a virtual keyboard when I type on my keyboard on my computer because it covers my screen and it's also much slower than what I can do with my 10 fingers we can do better for example a group from Stanford ask a person
1:32:59to imagine handwriting I had a imagine handwriting letters then they decoded the letters from Israel activity using this approach they were able to speed up the typing and the typing rate we start this project with our monkeys but of course they don't know how to write so to mimic writing we train Angela one of our favorite monkeys to trace digits on an iPad here you can see him chasing the digit 5 and the digit 2.
1:33:25then we recorded his neural activity with the N1 device but now instead of decoding the cursor velocity we decode in real time the digit that he's tracing understanding we had two main takeaways from this project one that monkeys are awesome and can learn very very complex tasks the second one that although it can increase the typing rate it requires hundreds of examples and samples of each of the digits and the characters we wanted to classify this would not scale the way we are solving that is by interaction instead of decoding directly the digits we first decode the hand trajectory of this on the screen and then when we decode the head trajectory we can use any of the Shelf
1:34:14handwriting classifier to predict the digits and the characters for example classifiers that are trained on an MS data set why it's so important it's important because now we can potentially decode any character in any language with only one neural decoder for hand trajectory it means that you can write in English Hebrew Mandarin or even monkey language and we can understand you wanted a banana so there are many challenges ahead of us to improve functionality and speed and I want to hand it off to Bliss to talk about the third Parts how we are making our brain interfaces work anytime anywhere what's up [Applause] hello everyone my name is bliss and I'm
1:35:04a software engineer here at neuralink when I use my computer my mouse and keyboard where can I intend them to at least like 99.9999 of the time my goal is to enable a user with paralysis to control their computer as reliably as I can here's what we want that experience to feel like in this video you can see saki walking over to his MacBook and choosing to work on his typing task the entire decoding system works out of the box and it feels totally Plug and Play the first step to achieving this kind of high reliability is to test extensively offline a typical flow for using the N1 link is to connect over Bluetooth stream out neural activity from the
1:35:50brain and then use that neural activity to train decoders and do real-time inference we've built a simulation for exactly the sequence but instead of using a monkey with an implant we use a simulated brain that injects synthetic neural activity into an implant sitting in a server rack from the point of view of that implant it's in a real brain this stimulation runs on every code commit to validate that from the hardware all the way up through to the neural decoders our entire stack can achieve state-of-the-art performance however while this kind of simulation is great for integration testing of software and Hardware it's not yet detailed enough to guarantee High
1:36:28reliability in the real world in the real world the underlying signals who are trying to decode actually change day to day in this plot you can see the average firing rate detected on a representative channel of sake's implant each bar represents one day and you can see that each day has a different average firing rate than the previous this presents us with a very interesting problem for how to make our decoders robust day to day it can actually happen that if you train a neural decoder on one day of data and then try to use it on the next the average fire rates can actually shift enough to cause a bias in the output of the model here on the right you can see that this
1:37:06bias is making it hard for the cursor to move to the upper right corner you see it struggling here to make it up to the upper right and then it moves much more effortlessly down to the bottom left we're trying many approaches to mitigate this problem some examples include building models on large data sets of many days of data to try to find patterns of neural activity that are stable across days another approach we're trying is to continuously sample statistics of neural activity on the implant and use the latest estimates to pre-process the data before feeding into the model this is really an active area of research for the team and it's a
1:37:40critical problem to solve if we want to enable someone with paralysis to control their computer as well as I can another big problem we have is to minimize the time it takes for a spike in the brain to impact the movement of the cursor on the screen if you have lag or Jitter in this control Loop the cursor becomes hard to control leading to the kinds of overshoots that you can see here on the right no one big Improvement we've made towards in this direction is called phase lock phase lock aligns the edge of each packet that we send off the implant to the exact moment that the Bluetooth radio is going to wake up this minimizes the time it takes for a spike in the
1:38:21brain to be incorporated into the prediction of our neural network here you can see the latency distribution after phase lock not only has the mean been greatly reduced but the variance has been reduced as well this makes it easier for the user to predict the behavior of their cursor over the last year we've made tremendous improvements to the stability and reliability of our system and we've been able to demonstrate consistent high performance across many sessions and many months however there's still a long road ahead of us before the system will truly feel Plug and Play so if solving the hard problems required to ship this technology is exciting to
1:38:57you you should consider applying to join the team now I'm going to hand it over to abnash to talk about how our custom low power Asic detects spikes in the brain [Applause] hi I'm Avinash one of the engineers on the Asic team we designed the custom neural sensors which include both analog and digital circuitry to record and stimulate across 1024 independent channels we Face challenges across all three major metrics performance power and area not only do we have to fit all 1024 channels into a single quarter sized implant but we also have to measure spiking activity less than 20 microvolts in amplitude and today I'd like to focus on the last challenge I mentioned power
1:39:50consumption is important to us because we want to give future users a full day of use of their implant without any Interruption for charging back in 2018 we were sending every sample from every channel off the device for processing which brings a ton of power in 2020 we brought Spike detection onto the chip as you may know neurons transmit information by firing so simply monitoring for these spikes and only sending these Spike Events off the implant acts as a very efficient form of compression and over the past two years we've continued to make optimizations within the Asic dropping the total system power consumption down to just 32 milliwatts
1:40:31and doubling battery life let's take a look at our on-chip Spike detection algorithm which makes our battery-powered implants possible we first start by applying a 500 Hertz to 5 kilohertz bandpass filter to remove noise that's out of band next we use an estimate of the noise floor to generate an Adaptive threshold per Channel and finally our Spike detector module identifies three key points of a spike identifying three points allows us to detect not just the presence of a spike but the shape of a Spike as well this can be extremely important for distinguishing between multiple neurons adjacent to a single Channel today I like to focus on one of the many
1:41:18optimizations that we've made in our latest chip this one's specifically cutting system Power by 15 percent note that neurons Spike relatively infrequently which means that our Spike detector spends a lot of time searching for the first point of a spike and very little time searching for the other two points of a spike that only occur after the threshold is crossed we can use this characteristic of the input waveform to reduce memory accesses within the chip by 30 percent let's take a look at how that works our Spike detector is implemented as a single functional unit that's shared across all channels with an SRAM to buffer the state of each
1:41:58Channel as a sample comes in its Channel state is read from SRAM an incremental Spike detection step is run and then the updated state is written back to SRAM since this is happening 20 million times per second across the implant each of these axises add up quite quickly in our latest chip we split the state into two parts a hot State and a cold state the hot state is accessed on every cycle while the cold state is only accessed once the threshold is crossed reducing the average axis width and saving power
1:42:37we're also working on a Next Generation stimulation focused chip with 4096 channels still within the footprint of our current chips in addition to increasing the channel count we're also increasing the drive voltage so we can get better activation per Channel and to support this higher Channel count as well as a broad range of future applications that you'll soon hear about we're adding an arm core onto the chip and finally since these chips are the same size as our current chips we can still put four of them together into a single implant for a total of 16 000 channels still within the size of a quarter
1:43:21foreign as you can see we've been working very hard to improve the power consumption within the implant but we've also been working very hard to improve the charging experience of the implant which Matt will talk about but first the robot has just completed inserting all 64 threats so let's take a look
1:43:49this is a view of the insertion site similar to the one that DJ showed you earlier but instead of the targeting reticles if you look closely you can see that all 64 threads each carrying 16 electrodes have been inserted into the brain proxy while avoiding vasculature and all just within the past 20 minutes
1:44:10let's hand it over to Matt now to continue the technical deep dive [Applause] [Music] hi I'm Matt head of brain interfaces electrical engineering our fully implantable N1 device depends on a battery for continuous operation when that battery is running low charging is accomplished through Wireless power transfer however unlike many consumer electronic devices which can simply offer a physical connector charging a fully implantable device poses several unique challenges first the system must operate over a wide charging volume without relying on magnets for perfect alignment the system must be robust to disturbance and complete quickly so as not to be
1:44:58overly burdensome however most important is safety in contact with brain tissue the outer surface of the implant must not rise more than two degrees C in pursuit of these goals our charging system has gone through several engineering iterations the first if you watched our Pig demo in August of 2020 Gertrude was implanted with a version of the N1 charged with our first generation charger this device was implemented in a small Puck package and later separated into a remote coil and battery base this charger was challenging to use however we learned a lot through its implementation our current production charger which charges our current generation of
1:45:42implants is implemented in an aluminum battery base which also includes the drive circuitry a remote coil four times the size of our original device also disconnectable this this remote coil has increased switching frequency driving improved coil coupling this charger is in use today including several applications within our engineering and animal Test Facilities I'd like to show you one of these applications here with a device we call our simple charger and the coil has been embedded into the habitat with the addition of one new outer control Loop plus a banana smoothie pump The Troop has been trained to charge themselves so let's see how pager charges is
1:46:39implant on the right we're streaming real-time diagnostics from pagers N1 when he climbs up and sits below the coil you can see the charger automatically detects his presence and transition from searching to charging we see the regulated power output on a scale of zero to one and the current driven into this battery [Music] [Applause]
1:47:10I mentioned earlier that we improved the coil coupling however the high quality Factor coils exhibit good charging performance over relatively larger distances but as they're brought closer to the implant what you see is a peak splitting effect where the the the best highest efficiency power transfer is pushed up into higher frequencies outside of the ism band required for compliance with regulated radiated emissions in our next Generation charger we address this problem by the introduction of dynamic tuning shown on the right this allows us to in real time adjust the resonant frequency of the transmit and receive coils so that we can change their properties
1:48:00just ahead of degraded performance the electrical engineering team is currently engaged in developing a third generation charger notable improvements include bi-directional near field communication this has allowed us to reduce the control latency and improve the thermal regulation improved thermal regulation results in Faster charge times and now Julian will tell us about how we test the N1 [Applause] thank you thank you very much Matt my name is Julian and I lead the embedded software group on the brain interfaces team so when we started building implants we had a small manufacturing line and to collect data from an implant you would manually walk over with your laptop you
1:48:58would connect and collect the data of Interest but our goal is to make an ultra safe and Ultra reliable implant and so to do this we scaled up the manufacturing line our testing throughput and data collection capabilities so firstly we added a large suite of acceptance tests to the manufacturing line these test the functionality of each component and the final assembly implants coming off the line are then subjected to bench top testing accelerated Lifetime and animal models we then collect data from these implants Round the Clock this data is processed by a series of cloud workers and displayed in an aggregate manner and then finally all of this information
1:49:37feeds back into our design process and empowers our Engineers to answer any question about any implant at any time I'm now going to walk you through different parts of this infrastructure starting off with firmware testing so the implant contains a small microprocessor running firmware to manage a whole bunch of its operations and before we release a firmware update we want to rigorously test it with both unit and hardware and the loop tests also known as hilltests so to do a hill test what you do is you instrument the battery you instrument the power rails the microprocessor and then we connect to each device with a Bluetooth client and then we walk the devices through
1:50:15various scenarios to test things like power consumption real-time performance security systems fault recovery mechanisms a lot of different things in our original implementation of these systems we used off-the-shelf components to start automating tests quickly however these systems were constructed in a relatively Artisan fashion and were very difficult to maintain and this meant that testing quickly became the bottleneck for development so to alleviate this the hardware and software teams developed a new system which integrates all the required components onto a single baseboard we can then put the charger and implant Hardware on individual modules that plug
1:50:50into this baseboard including one board with opposing coils so that we can test charging performance this architecture allows us to rapidly iterate different Hardware prototypes because we can simply drop them into this system and reuse all the testing infrastructure additionally we can host the current and next generation of our mural Asics onto fpgas and plug those into this board as well and that allows us to test a whole extra layer all together so that's how we generated this rather inceptive image here on the right what you're looking at is spiking activity emitted from some of our simulated neural sensors streamed through the entire system over Bluetooth
1:51:27and then displayed on a phone this allows us to test everything in one system from Chip to Cloud this system is one-fifth the cost one-fifth the volume and is very easy to manufacture this allows every developer to have a personal unit on their desk and it also allows us to test to Shard the entire test Suite over a large number of these units mounted into a rack all of this has greatly accelerated our rate of development let's look next at how we monitor the implants Electronics the battery and the enclosure so the implant will periodically capture all of its Vital Signs and commit those to flash and then upon next connection with one of our recording stations it
1:52:09will stream that data off so for instance if we look at humidity we can get an understanding of the Integrity of the implant's enclosure and by looking at battery voltage and power measurements we can gauge battery health all of this is done automatically without any intervention giving us 24 7 visibility into the quality of every single device additionally we can use this infrastructure to request High Fidelity information on demand so that we can investigate different anomalous situations so for instance in this particular scenario we were trying to track down the source of some spurious spikes that we were observing on different channels and so we requested
1:52:44role wave samples directly from those channels capturing good quality neural signals requires intact low impedance electrodes and so this is also something we monitor very closely with dedicated circuitry on the neural sensor so how do we do this we do this by first using an onboard DAC to play a test tone on a single Channel and then we record using our adcs simultaneously we record the response signal on both that that channel and physically adjacent channels not only can we measure the impedance of every channel with this but we can also map different physical phenomena to different characteristic signatures so for instance an open channel will appear
1:53:29as a very large response on the channel and shorter channels will appear as a large response on neighboring channels by looking at the purity of the signal coming back we can also validate that the analog front end of the neural sensor itself is operational in our original implementation of doing these impedance scans it took four hours to get through all 1000 channels but by paralyzing the tests down sampling filtering and then reducing the amount of information we have to stream off the device by moving a lot of the calculation to the firmware side we're now able to scan all 1000 channels in just 20 seconds this means that we can run impedance on
1:54:08every implant every day and then our internal dashboards can play back a history of this impedance so that we can get a really good quantitative insight into that interface between biology and electronics now that you have an idea about how we test and monitor our implants I'm going to hand it off to Josh who's going to tell you about how we get feedback even faster by accelerating our implants to failure
1:54:38thank you hello my name is Joshua Hess and I'm an engineer on the brain interfaces team we are responsible for the implant system design as well as many of the manufacturing and testing tools Julian just talked to you a little bit about some of the ways in which we test our implant Electronics hardware and software but what about the entire system as it relates to longevity in tissue one of the ways we've addressed this is with the development of our in-house accelerated lifetime testing system the system allows us to expedite and capture long duration implant failure modes at scale to rapidly increase our pace of iteration even better the system also
1:55:21significantly reduces the amount of tests which require animal models both for Implant prototypes and of course longevity testing so how does the system work on a very basic level it comes down to three things first we want to mimic the internal chemistry of tissue next we want to accelerate these chemical interactions as well as diffusion with our implant materials and finally we want to aggressively cycle the internal Electronics of our implant with these things primarily the first two we have achieved a conservative 4X acceleration Factor by the Arrhenius relationship in other words every day our implants spend in our accelerated system is
1:56:02equivalent to at least four days spent in Vivo historically one of our greatest challenges has been the battle against moisture Ingress into our implants so we continuously monitor the internal humidity to watch for abnormal rise here in white you can see some internal humidity data from implants in some of our animals for the duration of over one year as you can see our internal humidity sensing is so sensitive it can even attack to the very small and slow humidity rise just from diffusion through our implant materials now in blue you can see that same internal humidity data but from devices in our accelerated system now if we adjust this data for
1:56:47acceleration Factor you can begin to see not only the agreement in this data but also just how far into the future the data extends now in red you can see a device which has failed in our accelerated system this device showed an abnormal increase in humidity over the duration of many months before implant electronic failures occurred so how do we build the system
1:57:15well we started building the first system prototype just after the kova shutdown had begun in early 2020. so we had to get a little creative as you can see our first system prototype was a little Scrappy and operated out of one of our Apartments as indicated by the carpeting although Scrappy the system allowed us the fastest path to start testing our devices tuning Our working fluid chemistry and checking our constraints we also immediately started root causing observed failures in early implant prototypes fed that information into the next prototype designs and literally rinsed and repeated over the duration of just a few months the system was built out totally custom
1:57:57and highly iterated with two system versions and countless minder iterations leading us to our currently operated third generation system which achieves high density testing with automatic in vessel charging as well as automatic data collection the system also features an implant sled assembly which accepts brain proxy material such that the implant can be installed and inserted by the surgical robot just like you saw a few minutes ago we also integrated the system into a high density rack mount form factor along with the centralized fluid management system both for chemical uniformity across vessels and also reduce operational maintenance the system has been in operation for the
1:58:39last year and a half and has had its fair share of challenges since the system itself is undergoing the same accelerated abuse as the implants within it it has been extremely challenging to design build and maintain a system of the scale while keeping it robust even against itself so what comes next well we've started work on our fourth generation system and have totally redesigned it from the ground up to be a hot swappable single implant per vessel design partly inspired by high density compute servers with this new system we will achieve a whole new level of density robustness and scale we also intend to have many of these systems operational in the pursuit of
1:59:23capturing even the lowest frequency Edge case failure modes with this we will have thousands of implants testing in pursuit of these goals we've already started work building out the system but there is still a lot left to do there are also many exciting challenges ahead of us such as introducing mechanical stressing brain proxy micro motion and Ethan replicating tissue growth around the threads for more complete and representative accelerated testing so now that you've heard some of the ways in which we rigorously test our implant designs before production for surgery Christine is now going to take you through a detailed look at our surgical process
2:00:10[Applause] I'm Christine leader of the surgery engineering team to get an N1 device it's essentially these steps targeting and the incision drill the craniectomy remove the tough outer meningeal layer called the dura then insert the thin flexible threads of electrodes place the implant into the hole we created and then that's it you've got an implant Under the Skin look ma no wires just kidding I mean seriously no wires but I don't actually have one the surgical robot does the thread insertion part of the surgery this is because it would be very difficult to do manually imagine taking a hair from your head and trying to stick it into a Jello covered by Saran
2:01:05Wrap and doing this at a precise depth and position and doing this 64 times within a reasonable amount of time and a neurosurgeon would probably not like it very much if we asked them to do this for the surgery so we have the robot that you saw doing its tiny dance I sort of wanted to call it Tiny Dancer but it's called R1 which is also great
2:01:26the rest of the surgery is done by the neurosurgeon in order for us to make a accessible and affordable procedure we need to revisit this I'll tell you why when I was in school my dad lost the ability to walk and to use his arms and even to speak he was diagnosed with ALS we would look on the internet and you could see maybe one person here or there who had some cool custom robotic assistive device but it was deeply frustrating how limited were the options available to him and there's hundreds of thousands of people with paresis not even counting people with other conditions that our device might be able to help meanwhile there's not that many
2:02:12neurosurgeons maybe about 10 per million people and it takes about a decade or more to train a neurosurgeon and already generally very busy and as you can imagine the time is very expensive so in order for us to do the most good and have an affordable and accessible procedure we need to figure out how one neurosurgeon could oversee many procedures at the same time this might sound sort of crazy but probably so did laser eye surgery before Lasik made it normal lasik's been around for about 30 years and counting in the beginning the laser robot did just the most fundamental core part that it had to do and the surgeon did the rest and over the iterations the surgeon
2:02:59has to do less and less and the laser robot does most of it and it's a highly compelling procedure it takes just a handful of minutes and often gives life-changing results since I joined in 2017 we've also done a handful of iterations to optimize the threat insertions of the robot one of the challenges that we've had to face has to do with the optim mechanical Packaging so as you can see here there's about three primary Optical paths that are really valuable for a staff reliable threat insertions one is the visible Imaging of the needle inserting a thread and then another is the laser interferometry system called OCT Optical coherence tomography that gives us the
2:03:40precise position of the brain while it's moving in real time and then also we have to provide lighting and illumination to see what's going on in the visible visible light camera and doing all this where the needle is at the bottom of the craniectomy especially when it's close to the skull wall can be pretty difficult to fit everything and be able to see it so the way that the team solved this is by putting all three of these Optical paths into one Optical stack using Photon magic or polarization whatever you want to call it and that enables us to do vessel avoidance in real time so as I mentioned the brain is moving and where we place Targets in the beginning may not be
2:04:18where you want to insert at the moment the needle is going down there so the robot can actually detect the vessels and then determine if we're going to insert onto a vessel or not if it's safe to insert and then that way we can avoid inserting onto major vessels and that brings us to the robot that we have here today there's still a lot for us to do to get to that procedure where we reduce the role of the neurosurgeon and make it affordable and accessible the primary the two elements of the surgery that demand the most skills from the neurosurgeon are the craniectomy and the directomy Alex and Sam are going to tell you a bit more about how we think we can get rid
2:04:57of the directomy step so that leaves the craniectomy in neurosurgery if your craniectomy is small enough you can use a standard tool called a perforator which makes quick work of this job but for a larger craniectomy the surgeon has to rely on their skill in order to accommodate the variability Patient to Patient in skull thickness skull hardness even within the same patient in the same craniectomy you can have different skull thicknesses for example in addition if we can make something that has a very high Precision craniectomy we can open the design space for future ways of mounting the implant to the skull so I'll show you a few of our prototypes
2:05:35ultrasonic Cutters like what's on the screen and oscillating Cutters have the benefit of not cutting soft tissue you can cut the bone and not the brain but however as you can see here our ultrasonic cutter prototype created quite a bit of heat to cut at the rate that we wanted so onto the oscillating saw here we designed a blade to minimize cut time and also conducted sound and also Heating and as you can see you can cut through hard things like bone but not soft things like skin it's simple and it's it works however if you wanted to cut an arbitrary depth or arbitrary shape the oscillating saw just won't cut it
2:06:21I was afraid no one would get it if you guys are smart so there's a time-tested solution for drilling arbitrary shapes which is a CNC drill the challenge with us doing this on a person is that we need to make sure it cuts reliably every single time doesn't cut too deep and a few ways that we're using feedback to you know make sure we don't cut through the brain our force feedback and also impedance and if I could get a volunteer no it's just kidding maybe next time but yeah so this is a some insight into some of the things we're working on to make an accessible and affordable procedure and now Alex is going to tell you a bit about our next Generation
2:07:06developments thanks Christine I'm Alex I'm a mechanical engineer here on the robotics team now that we've covered the technology and surgical process for a current device we'd like to cover some of our next Generation development projects I and the next couple speakers would like to talk about one of those projects which is enabling device upgradeability
2:07:34you've gotten to hear about the advancements we've made over the past year we've improved implant robustness battery and charging performance Bluetooth usability realistically every new device version is going to be significantly better it'll be more functional it'll last longer we need to keep this new technology accessible for our early adopters this means that we need a solution to make device upgrade or replacement just as easy as it is to initially install as many medical device companies have found this is a challenging problem the body's healing response doesn't make this easy so this isn't solved yet but we've made significant progress
2:08:16towards enabling this that we'd like to cover today now we'll have to start with some background as to what makes device upgrade challenging and we'll start with the anatomy Under the Skin you have the skull below that the dura a tough membrane that separates the bone from the brain and between the dura and the Brain you have the pier arachnoid complex a fluid-filled suspension for the brain to install the device the surgeon removes a disc of skull and Dura to expose the brain's surface the device then replaces the remove material the challenge is here at this interface over months all empty volume is filled by tissue encapsulating the device and
2:09:02the threads the device would be trivially easy to remove because of the threads small size they would slip right out of the brain it's the tissue layer that forms above the surface that makes a removable challenging we built tools in-house to study this response and characterize it such as histology and micro CT in these images you can see that layer of tissue that has formed above the surface encapsulating the threads and adhering to the surrounding tissue we've explored many different avenues for Designing around this healing process and finding a solution to make device upgrade seamless our best successes have come from making the procedure less invasive
2:09:47instead of directly exposing the brain's surface we instead keep the dura in place maintaining the body's natural protective barrier this prevents encapsulation of the brain's surface and really this is actually a huge win for making the surgery simpler and safer as Christine alluded to however this doesn't come for free the dura is a very tough opaque membrane as you can see in these sem images it's composed of a dense network of collagen fibers these offer an array of technical challenges for inserting our electrodes one of those challenges is Imaging through the dura as you can see on the left our current custom Optical systems offer pretty incredible capabilities for
2:10:33Imaging the exposed brain surface however as you can see on the right once the dirt is in place you can't see the dense vasculature at the brain surface the dirt is in the way there's simply too much attenuation to solve this problem we're developing a new Optical system that uses medical standard fluorescent dye to image vessels underneath the tissue here you can see that die perfusing through the vessels highlighting them there's still a lot of work engineering work to go to prove accuracy and repeatability of this system but once that's done this will allow us to Target and avoid blood vessels underneath the dura we're also exploring applying our laser
2:11:17Imaging system to deeper tissue structures in the bottom left you can see a section of the tissue layers underneath the dura this image is compiled from multiple volumes from our Optical coherence tomography system you can see the collage of those volumes above in the future these new systems when combined with correlation to pre-op Imaging such as MRI will enable precise targeting without directly exposing the brain surface now Imaging isn't the only challenge that comes with the tough Dural Anatomy now I'd like to hand it over to Sam to talk about some of the challenges of inserting our electrodes through this membrane thank you
2:12:04thanks Alex hey I'm Sam and I lead the needle manufacturing and design team so as Alex mentioned the same properties of the dura that make it a good protector of the brain also make it really difficult for us to insert the threads into in humans the dirt can be over a millimeter in thickness which doesn't sound like a lot but compared to our 40 Micron needles it actually is a lot for example if you scaled up the needles to the size of a pencil the dura would scale to over four inches in thickness take a look at how far you have to zoom in to even see it by the time the features of the needle come into frame you can see individual red blood cells in the same frame
2:12:43this is this is just wait this is a real-life sem image of our latest design on the left there you can see the end of the thread in the middle is the needle and on the light is actually a piece of my hair so yeah it's extremely small and besides being really small there's a lot of other challenges associated with designing this One Challenge is that we have to use the needle and the protective cannula that it sits in to grab onto the thread and to hold it while we peel it from this protective silicon backing and then we have to keep holding it while we bring it over to the surface and then release it from the cannula during insertions another challenge is that the brain is
2:13:23really soft beneath the tough Dura and so if the needle isn't sharp enough it'll just keep dimpling the surface without puncturing and if this free length gets too long it can actually just Buckle the needle like this another challenge is that we don't just have to get the needle through we have to get the thread through as well so we really have to focus on optimizing the combined profile of the needle and thread together these are just some of the challenges associated with designing something like this problem has been improving on our speed of iteration but let's look at how we make these things in the first place so we start with a length of 40 Micron wire made out
2:13:59of tungsten and alloyed with a little bit of rainium for added ductility we designed this femto second laser melon house to cut the features of the needling cannula and it can do this with sub Micron precision we spent a lot of time this year turning this thing from a science project into an industrial system just a couple months ago skilled operator 22 minutes to make a needle and even a skilled operator could only get about 58 yield today that same process takes just six minutes and anyone can get 91 yield with just a few minutes of training with only one click the mill cuts and measures the needle in cannula and uploads the measurements to our limb
2:14:39system so that the robots can use the exact dimensions for each needle that it uses now this is all for our current design though and we've had a couple years to optimize the manufacturing process of it the current design has served us well so far but it doesn't quite protect the thread well enough to get through the tough Dura so like I said we had to come up with something new and we needed to be able to iterate on designs quickly unsurprisingly there's no page in machinery's handbook for this kind of thing so we dug into the science of femtosecond laser ablation and figured out a workflow that allows us to use our laser Mount much like a CNC mill
2:15:15this allows us to iterate several times this allows us to iterate in under an hour for new designs allowing several iterations per day when we're really on a roll as a result the latest design seen on the right can actually insert through nine layers of direct a totaling three millimeters on the bench top this is far more than we could ever expect in a human with significant margin [Applause]
2:15:47the needle isn't the only part of the puzzle though as you can imagine all these designs here work with different threads so we need a way to iterate on that as well and we do this by having our microfabrication process here in-house this summer we completely rebuilt our clean room in about nine weeks which among other things greatly reduced particulate counts which allows yield and throughput to greatly increase this combined with all the other great improvements the microfib team has made allows us to iterate on new designs in just a matter of days the last piece of the puzzle though is testing we can come up with as many new designs as we want but unless we have a way to
2:16:23actually test them in the right conditions we won't know what to tweak or even worse we'll spend time optimizing for the wrong things take this failure mode for example a few months ago we got to the point where we could pretty reliably insert through the dura but when we took the proxies and put them in our micro CT Imaging we realized that our hold on the end of the threads was actually too strong and we were pulling them out just a little bit underneath the surface by the time we solved the problem we realized that this issue was very sensitive to the properties of the surrounding material or tissue we could make a proxy where this never happens
2:16:56and we can make another proxy where this happened every single time and this highlights why it's crucial that we spend time making our benchtop tests Mash tissue as accurately as possible I'm going to pass it off to Leslie now who's going to talk about how we've been doing that thanks [Applause]
2:17:17hi I'm Leslie and I lead microfabrication r d and part of what we're interested in is understanding the biological environment our implant and threads experience once they're fully installed in the body learning directly from biology though is inherently slow so in order to move fast we're developing synthetic materials that mimic the biological environment this allows us to learn as much as we can on benchtop and start taking steps away from the industry standard of animal testing developing accurate proxies though is challenging the implant environment is made up of many anatomical layers that all have unique properties and as time goes on and the implant site
2:17:59heals new tissue forms filling any available space in addition to that motion related to cardiovascular activity and head movement introduce added complexity so to start addressing some of these challenges we're engineering materials using feedback from biology this may involve mechanical characterization of tissue or analysis of interactions at thread tissue interfaces much of this characterization is even done during surgery itself by using custom hardware and software that modifies our surgical robot to double up as a sensitive characterization tool we then use the data collected and feed it back into optimizing our materials so that they behave mechanically chemically
2:18:49and as shown here structurally just like biology we've come a long way from our humble first brain proxy shown here sitting on a plate and consisting of agar and a paraffin sheet and while simple it allowed us to perfect robot insertions through countless bench shop tests today our proxy is slightly more complex where we've upgraded to a composite hydrogel based brain proxy that better mimics the modulus of real human brain we've also Incorporated a duraproxy and the developed an injectable soft tissue proxy that so far has allowed us to perform benchtop mock explant testing we have a super long wish list for our proxy of the future but some of those items include
2:19:40surgery proxy with integrated soft tissue brain bone skin or even a whole body a brain proxy that simulates motion vasculature and electrophysiological activity and a biological proxy to test biocompatibility and electrical stimulation there's a ton of ongoing work getting us closer to our proxy of the future including work on lab-grown cerebral organoids as shown here and all of this will get us closer to a future where we learn more and iterate faster on benchtop and reduce our Reliance on animal models or even when they replace them completely and with that I'll hand it over to Dan who will be presenting a very exciting Next Generation application thank you
2:20:27[Applause] thank you Leslie my name's Dan and I came to work at neurolink after following a career in visual Neuroscience research I was inspired to join this company because I saw in our device the potential to restore Vision to people rendered Blind by eye injury or disease there are a number of particular characteristics of our device that make it uniquely suited to this application firstly as well as being able to record from every channel we can stimulate neural activity in the brain by injecting current through every channel this is important because it allows us to bypass the eye and generate a visual image in the brain directly secondly our device can have an enormous
2:21:21number of electrodes for a visual prosthesis this is important because the more electrodes you can have the higher density of an image you can create in the brain thirdly thanks to our robot we can insert these electrodes deeply into the brain now this is an important thing for a visual prosthesis because the human visual cortex is buried deeply in a fold in the medial face of the brain called the calcarine sulcus in this image I've highlighted the calcarine Cyrus in red in an MRI it contains a map of the visual World visual field it's about surface area equal to a credit card on each side and if you unfold it and flatten it you see that the image is inverted it's upside
2:22:12down but more interestingly it's mag it's distorted so that the central part of the visual field the fixation point is greatly magnified so for example if you look at this image of Lincoln if you look directly into his right eye everything to the left of that fixation point is directed to your right visual cortex and everything to the to the right goes to your left visual cortex his eye even though it's very small in the image is magnified in the brain to occupy nearly a quarter of the surface area of the visual cortex
2:22:49over the last half century visual neuroscientists have developed a profound understanding of visual processing in the brain what's Driven most of this research is recording from single cells in the cortex usually of macaque monkeys one of the seminal discoveries was that every cell in the visual cortex represents only a tiny part of the visual field your perception is made up of a mosaic of tiny receptive Fields each belonging to a single cell in your visual cortex so if you record from one of these cells in a monkey say in this location you can find a very tiny region of the screen where a light stimulus will cause modulation of that neuron
2:23:36another location in visual cortex will have a location Elsewhere on the screen in this case in the lower visual field these regions are called receptive fields we've inserted our device into the visual cortex of two rhesus monkeys whose names are code and dash that means we can record activity from their visual cortex generated by their not their normal home environment as they roam around but as we all know monkeys love banana smoothie that means we can easily teach them to fixate points on a screen and reward them we can reward them very precisely because we can track the location of their eye using an infrared camera one of the things this allows us to do
2:24:26is to plot the receptive fields for every neuron that we can record with a single device now we do this by showing the animal a movie of random checkerboards whilst you fixate steadily on the screen then we take only the frames of the movie that generated a response in the cell and averaged them all together this is a technique known as reverse correlation is generally used quite widely in visual Neuroscience for this purpose and this is an example of a receptive field plotted with this technique the central cross is the fixation point and you can see the little red and blue regions of excitatory and inhibitory receptive field these regions give cortical cells
2:25:09some of their characteristic properties so we can record all receptive fields from all the electrodes at the same time and if we take all these receptive fields and accumulate them together overlap them and place them on a on a computer monitor for scale at a typical viewing distance you begin to get an idea of how much the visual field we can cover with this preliminary device many of the receptive fields are close to the phobia so close to the fixation point and that's partly due to the magnification I talked about of the phobia but there's also a scattering of fields in the periphery these are from recording sites deeper in the brain in the calcium sulcus
2:25:58so far I've only talked about recording information from the cortex but to produce a visual prosthesis we need to stimulate so if we stimulated the cells whose receptive fields are in this location we would produce a perception of a flash in that location that only the monkey can see how do we know that the monkey sees it how do we know what it looks like well unfortunately we can't ask them what they see but we can train them to tell us something about that phosphine we start by training the monkey to fixate a central point on the screen like this white dot and we start by presenting real visual stimuli on the screen and rewarding the monkey for
2:26:44making eye movements toward those stimuli so here we flash a white dot and the monkey makes an eye movement towards it symbolized by the Green Arrow we then choose another random location and reward the monkey for making an eye movement towards it once he's got good at this task we can begin to interleave these real stimuli with electrical stimulation of electrodes and produce a phosphine a monkey sees the Flash and naturally makes us occurred towards it this tells us not only where in the visual field The Flash occurred but we can also change the current that we inject in that electrode to see how often he makes that's a card unnoticeable or how big perhaps the
2:27:27stimulation the phosphine is that we're producing let's look at code performing this task I want to show you first at one quarter speed there's a visual Flash and he makes an eye movement towards it we the monkey can only see what is white on this screen you can't see his own eye movement and he can't certainly can't see when we stimulate but here we stimulate and he makes the same circad to the same location because we stimulated the same electrode nothing appears on the screen at that time and he has no other cue to make that eye movement let me show you this in real time you can see monkey monkeys like to work very quickly and when we stimulate
2:28:12he makes that's a card in real time [Applause] and looks like he's had enough so what I've shown you is a way to produce a phosphine in the visual field this is not something new in visual Neuroscience but if you think about that phosphine as a single Pixel in a visual image all we need to do is scale up and produce a great many more pixels and have them covering the visual field this is a schematic of what a visual prosthesis using our end device my N1 device might look like a camera the output from a camera would be processed by an iPhone for example which would then stream the data to the device and the image would be converted into a pattern of stimulation of the electrodes
2:29:11into visual cortex with a thousand electrodes we might be able to produce an image resembling something that you see there on the right but as Avinash told you our next generation of the device will have 16 000 electrodes if you put a device on both sides of your visual cortex that would give you 32 000 points of light to make an image in someone who's blind our goal will be to turn the lights on for someone who's spent decades living in the dark [Applause] foreign
2:29:52[Applause] who's now going to talk about another very exciting application of our devices thank you Dan so my name is Joey I'm an aero engineer and I'm the head for the next gen team at nerlink so for persons with spinal cord injury the connection between the brain and the body is severed the brain continues functioning normally but it's unable to communicate with the outside world you've already heard about how we can use the N1 link as a communication prosthesis to help someone with spinal cord injury control a computer or a phone but it can also be used to reanimate the body let me show you how first a little neuroanatomy movement intentions arise in motor
2:30:40cortex and are sent down long nerve fibers through the spinal cord these are upper motor neurons in the spinal cord they synapse that is make a connection with another motor neuron a lower motor neuron which sends these movement intentions to the muscles which contract and in turn you have movement while of course there are many other circuits involved in voluntary movement you can think about the spinal cord as many pairs of these two connections and in spinal cord injury one of these connections is severed unable to make the muscles contract let's Zoom a little bit further so here you can see on the left across a cross section of the spinal cord with a
2:31:21fiber coming down schematically this travels through the white matter tracks this is the upper motor neuron and then it synapses within this butterfly shaped region of gray matter in what's known as a motor pool in the motor pool the lower motor neuron descends out the ventral roots to the muscles which contract and then the sensory consequences of those movements for example the touch of your hand against an object return to the spinal cord through the dorsal roots and Ascend the spinal cord up into the sensory regions of the brain again in spinal cord injury this connection is severed if we could place electrodes into the spinal cord say in a motor pool adjacent
2:32:01to lower motor neurons we could stimulate those neurons activating them and in turn causing the muscle to contract and movement to occur but this is very hard to do the spinal cord is quite delicate and it moves significantly within the Bony spinal canal this could cause damage to the electrode it could cause damage to tissue or both but our electrodes are small and flexible and our robot is able to insert them deep into tissue perhaps all the way down into the ventral Horn of the spinal cord and so we have done just that here you can see A View From the R1 robot and it's a targeting View and we've placed electrodes across many millimeters of the spinal cord and the
2:32:44R1 robot is able to insert those electrodes deep into the ventral horn into motor pools in very close proximity to lower motor neurons this is important because it allows them to have a localized connection to those neurons and activate very precise movements now to track movement it's very common to use motion capture markers like you might see in the production of a movie these can be placed with a light adhesive and you can see me placing these on my hand we're going to use these markers to let us zoom in on movement in the next couple of slides okay so here's a pig walking on a treadmill and you may have seen something like this before in a previous
2:33:25knurling presentation but unlike before this pig has more than one neuralink device there's a device in the brain but there's also one in the spinal cord and we can stream neural data from this device these devices in real time and use them to do things like decode the movement of the joints of the pig so here you can see on the left a Time series of the hip knee and ankle and we're decoding those those movements so this is super cool but that's actually not what we want to do we want to go in the other direction we would like to stimulate the spinal cord and cause movement to occur okay so let's do that so here's a pig a happy and healthy Pig
2:34:01doing what pigs like to do which is root around for food and snacks and as you'll see on the floor there's a blue square this is a voluntary engagement Zone where the pig places itself indicating that it's comfortable to receive stimulation when it's in the zone we stimulate and if the pig leaves the stone we'll stop stimulating and as before you can see we're able to track the position of the joints and also stream neural data as well okay so let's stimulate an electrode so here's one electrode on one thread that when we stimulate causes a flexion movement of the leg so on the left you can see the movement of the joints and you can also see the time series of the
2:34:39stimulation pattern in yellow so the leg is moving up here's another electrode which when we stimulate causes an extensor movement this is actually a little harder to see because the leg is straightening and the hips are shifting but if you look carefully you can see how this is the leg is moving we can stimulate on a great variety of threads and produce different movements and actually sequence them spatial temporarily to provide patterns so on the left you can see a Time series of different stimulation on different electrodes you can see the movements of the joints and on the right we're zooming in on muscle activity that gives us an idea of the kind of strength and
2:35:15power and specificity of those movements as well so in addition to doing sequences we can also achieve sustained movement these are powerful muscle contractions of the sort that you might need for standing or other load-bearing activities and are really crucial for interacting through the world okay so stimulating the spinal cord is only one piece of the story you also have to get like command signals for the stimulation of the spinal cord unfortunately we have a way to do that we have the N1 link that you've already heard about placed in motor cortex how would that work so we place threads in motor cortex and record spikes these spikes would be
2:35:52wirelessly transmitted in real time and decoded into patterns of stimulation stimulation would then be delivered to the ventral Horn of the spinal cord to the appropriate motor pool for the muscles that we like to activate we then stimulate activate those lower motor neurons which causes the muscles to contract and movement to occur now of course movement without sensation is actually kind of difficult just think about what it would be like to try to move your limbs if they're numb but we can also get sensory information as well so the sensory consequences of your movement can be recorded in the dorsal Horn of the spinal cord in the form of spikes for example here a feather
2:36:28touching the hand these spikes can in turn be decoded in real time sent to patterns of stimulation to either the same and one device in the brain or perhaps a different one in a sensory area stimulation of that part of the brain would cause percepts of touch and proprioception closing the loop so putting those two Loops together we have motor intentions decoded from the brain used to stimulate the spinal cord causing movement and then the sensory consequences of those actions being recorded in the spinal cord to stimulate the brain causing perception now we have a lot of work to do to achieve this full vision but I hope you can see how the pieces are all there to
2:37:06achieve this and if you find this Prospect as exciting to you as it is to me I hope you'll consider joining us here at neurolink
2:37:25thank you [Applause] thank you [Music] so anybody has questions in the audience thank you this is obviously amazing and has clear therapeutic potential it would also be great for the scientific Neuroscience Community to access some of these tools do you have any plans to make these available to neuroscientists
2:38:15yes yes we do so no it's a great question is I think there's probably a lot that can be figured out if we provide the surgical robot and devices to Neuroscience research Departments of universities and hospitals so I think at the point of which we have we need to be in production with the machines and obviously have the FDA approvals but I think it would make a lot of sense to provide this to research universities and hospitals
2:38:51follow-up questions the question is of the data that we about the data sets that you've collected are there any that you plan to open source for the scientific community yeah I think that would be that would be fine I think yeah sure absolutely because I think it could be really interesting for people working in AI research to build upon that and Mill and build Foundation models for the brain yeah that's it's a good point yeah like actually no problem with just publishing it on our website you can use it if you want looking forward great
2:39:57so thank you for the very wonderful presentation so I have one question so as we know for implantable Electro either for stimulation or recording after we implant the electrode Scar Tissue will grow around the electrode and especially for recording the signal we get will become smaller and smaller after long-term implant how do you solve this issue
2:40:28so for context I'm Zach I leave the microfabrication team on brain interfaces I don't think we can solve it specifically but one thing one advantage we have is about the flexibility and the small size of our threads to try to limit that scar tissue and that damage and some future work that we have started working on that we'll continue working on is pushing the size of the threads down just to try to limit the immune response and really limit that scar tissue growth I actually have a wonderful love so do you think it will be helpful to actually load some drug on on the surface of your electrode or some other way well I think like maybe the just the
2:41:11question is like what what sort of signal degradation have we seen over time and you know basically just does it does it work a year later does it work two years later it does so yeah yeah so that's a good point so in terms of thread longevity specifically really the gold standard that we can use to assess is the data we have from our animal participants and so for that I'm not sure if it was mentioned before but the longest data we have right now is for an animal participant who has 600 600 days with useful functioning channels where we were doing something useful with the the signals for BCI and then with the newest version of our device we have sort of
2:41:56a collection of participants who are at or near one year of data and completely useful functioning BCI from that as well thank you if I may add one more thing so you mentioned potentially having drugs to kind of reduce inflammation so one of the things that we are actually actively working on is having some sort of biological coding to either reduce inflammation or make them slippery so you know you mentioned you heard from the presentation that one of the challenges that we have is removing the threats from these neo-membrane tissue that are formed after implantation so there are programs like that where we're really looking at kind of incorporating
2:42:36some of the learnings from biology and these Coatings into our thread so that we can hopefully reduce inflammation as well as make it easier to extract we're also continuing to reduce the size of the the electrode so as when the electrode gets really small there's the sort of inflammation response Scar Tissue becomes minuscule so it's like a very a very tiny electrode the body basically ignores
2:43:06this is really impressive congrats to the whole team so as as you of course know one of the problems with current electrodes is they're rigid and they move around so you have these neural nonstationarities and I think many of us had hoped that with these very thin threads they would maybe move more with the brain and you wouldn't see that but from the data you showed over many hundreds of days there was a lot of variability so can you speak to how much do they move and do you have any idea of like why does it move can you stop it from moving is it how stable are the signals hour to hour and day to day
2:43:40hi I'm Bliss I'm one of the leads of the software groups in the brand interfaces team in the particular plot you're mentioning before what we were showing was the average firing rate recorded per day on a particular Channel it's as you well know pretty complicated to understand if you're recording from the exact same neuron day after day after day it could be for example that you're actually picking up a different neuron day to day and that's why you get the change in firing rate we don't think this is at least the majority cause of the situation here the reason is if you look at sort of the spike shapes day to day even when the average firing rate is shifting a lot
2:44:10you still see sort of stable Spike shapes that's obviously not a fully bulletproof story but at least gives some confidence that it's not actually different neurons are picking up however there's still is very much a chance that that could be the case in at least some part of the of the robustness now stationary story yeah cool thanks yep thanks question yeah to be clear that like the Electoral position is actually fairly stable because you've got these very tiny basically very tiny wires with with that that and then there's some play in the like you've got you've got the device attached to the skull ritually but then you've got this this long so
2:44:49tiny wire with kind of a a coiled section so it's a it does tend to basically stay in the same place it's on narlings Twitter so we'll be interleaving some of those supe wants to ask what could knurling help people with that most people don't realize well I mean so once you're in there you know
2:45:17there's a lot you could do so you know it can obviously measure temperature so you could do very early detection of a fever you could not measure pressure I think you probably detect that at the very early the very beginnings of a stroke because you can see sort of like electrical signals starting to go sort of paleware so there's actually probably a lot of just General Health monitoring that you could do once you're in there you know and and with with very simple sensors
2:46:01hi you guys all did a great job of distilling a lot of complex engineering and Science and making it wonderfully clear so great job I wanted to ask a little bit about the stimulation I guess for the phosphines and for the the evoked movement how are you thinking is this more like local stimulation is it is it juxta cellular are you steering current around how many cells are you activating how much current are you using I'm just curious what the scale of this is and whether you have a lot of precision or a lot of you know pretty profound behavioral effects too hi yeah I'm Dan and how many cells you stimulate with a single electrode is dependent on the
2:46:41impedance of the electrode size of the conductive pad how much current you deliver the frequency all these factors so there's a great deal of variability that we can use to customize the the shape of a phosphine or the or the shape necessarily but maybe the the intensity of the phosphine we think with our current electrodes at least in code back of the envelope calculation would be something like about a 50 to 100 Micron diameter sphere of cells are being stimulated in a visual system the smaller that sphere the smaller and more specific you can make a particular phosphine basically the smaller the pixel in in the image you can produce so there's
2:47:27plenty of scope for customization of that there's actually also it's possible to get to a much higher like effective pixel count by controlling the the the field electric field between the electrodes so it's not necessarily it's not a one-to-one relationship you could actually dynamically adjust the the field and simulate Farm have a have a very high neuron to electrode ratio so try to like could you get like you know maybe 10 to 100 to 1 potentially so it's a megapixel type time basically
2:48:11can you see normally but I think people would want to know that I think that is one of the one of the possible outcomes pylon this is amazing oh can you talk about the longevity of the implant itself also how would the material of the implant would react with the brain tissue or density of the bone or bone structure thank you
2:48:32yeah happy to talk about this I'm Jeremy an engineer on the brand interfaces team and I think it's good to start with data so like Zach mentioned we have an implant that was you know a monkey who's performing BCI for 617 days that was pager before being upgraded to the latest device for our current version of the device it's lasted for almost a year and then for our accelerated lifetime tester that Josh kind of talked about we have data from our implants from the previous version eight years of accelerated time and from the current version four years of accelerated time and Counting so that's kind of starting with the data those devices are still
2:49:03lasting and still going theoretically there are kind of three fundamental factors that contribute to the longevity of the device one is going to be the seal the Hermetic enclosure of the device two is going to be the battery and internal electronics and then three is going to be the threads that Zach talked about a little bit and the channels being able to functionally record signals from the brain the seal we think will far Outlast the other two in terms of the bottlenecks so the seal just theoretically I think Josh mentioned that it is a thermoplastic polymer material so there's going to be a a very small amount of moisture that diffuses through it over time and we
2:49:37think that that will last you know 20 plus years easily in terms of just that property and like I said we have not seen our seals fail with our current version of the device yet so we haven't really pushed the limits here for the battery and internal Electronics that's really based on usage and how much runtime you want and we are working currently on getting data to project out even farther but right now we believe that we can you know achieve 80 run time at the three year time point which would be about you know three and a half hours for a four hour run time but we're like Avinash mentioned we're we're doubling that very soon and quadrupling is what
2:50:10we have plans to do that so the internal Electronics really aren't the bottleneck either and so really we're attacking the threads themselves and longevity of those channels that Zach Zach can kind of talk about some of the improvements that we're doing to increase that longevity cool thanks yeah so as sort of mentioned before we don't necessarily have an end point as Jeremy said for the testing of the threads that being said we are focusing on longevity because we think this is an important issue to solve so one thing that we're doing in parallel with the current device is aggressively pursuing silic amorphous silicon carbide insulation of the threads which we believe will take
2:50:48us well beyond five years of longevity but of course still to be tested and in parallel with that we're just starting to look at Atomic layer deposition which we think could even push longevity of the threads much further and deposit very thin layers to keep the flexibility of the threads and that Advantage there so along with that we're also of course having to design and validate very robust bench top testing to model really in Vivo conditions and look at Channel degradation so that's what we're looking at for longevity of the threads and then I think you asked about biochem pump and I think for biocomp essentially all the materials we're using right now
2:51:31I can say or at least biostable and we send out testing for biocompatibility very often and essentially what we're doing is we're using in many cases known materials from literature that academic Labs have already started to look at and sort of jumping on that and using that as a starting point thanks for answering that
2:51:52cool so we have another question from Twitter this is from David and he asked the team what are the biggest lessons you learned since the previous presentation it's been about two years I'm sure there was a lot of engineering done so yeah anyone want to answer what we learned in the last two years so one thing that we've learned in the last couple years is just how much the brain moves on the human scale compared to you know when you start small when you make brain proxies and a lot of research starts with rodents the brain does not move that much then you get a human and the Brain can move like hundreds of microns or more and when our threads and
2:52:33needles are so small that motion when you zoom in looks like a mile I think to add to that one thing is how I guess Dynamic the implant environment actually is so we've talked about like when those implant site heals SCAR or new tissue might grow and fill in the space and that'll affect like how our threads might interact in that space so that's why we've emphasized so heavily the importance of Designing accurate proxies so instead of having to wait months for an implant site to heal you can hopefully learn that information in hours I'm Alex on the robotics team I think one of the things we've definitely learned within the engineering teams is
2:53:20the importance of really continuous validation and testing where we're building say motion systems that are precise to single digit microns we need validation and test systems that we trust even more than that to prove that they work reliably and putting just as much focus into those validation and test systems and designing those alongside our products I think is one thing we've definitely learned
2:53:48I think and I don't think that we learn I think as part of BCI or the band control and algorithm is that again building a prototype and making it work with only one monkey one pager was a great maybe a success but also relatively easy to making it work every day for all the other monkeys so actually making it a product is something that it's not easy but we are learning how to do it
2:54:17I mean I've learned that the brain is really squishy like way squishier than you think it's not like you know cauliflower or broccoli or something like that it's more like water balloon and then it's moving in your skull like a lot so got a squish your water balloon in a coconut is maybe a good way to think of it hello given Bluetooth bandwidth limitations have you considered other Technologies for wireless communication foreign hey yeah I can take the first part of this question and then I'll let Matt to the second part of it it's a great question especially as you think about how to increase and scale the number of channels that we want to record from this becomes
2:55:07increasingly a bottleneck for the kind of work that we want to do we're thinking about this in a couple ways one is just directly improving the underlying radio interfaces and I'll let Matt talk about that in a second the other way we're thinking about this is how can you be more efficient with the data you send off the implant and I think the first version that is compression so just taking your data looking at the characteristics of it find out a way to represent it more efficiently and just send off that compressed string so for reference right now our Bluetooth bandwidth is around 150 kilobytes per second the compressed stream of data that we
2:55:37sent off the implant is around 50 kilobytes per second so we're doing fairly well there so far but when you start thinking about 16 000 Channel devices that won't get you all the way there so some other things that can help on the compression slider to actually just send out the output of the machine learning model rather than the input required to actually run it so one thing we've been trying in the background here is called decodon head which is essentially taking the machine learning models right now we're running on MacBooks that our monkeys are gaming on and moving those to actually run on the implant and this is like a super cool engineering problem if you want to
2:56:08talk about how to make complex neural networks run on what is the equivalent of a garage door opener come talk to me it's fun yeah so that's another way to solve this problem is basically do the computationally Intensive work to just get the raw signal that you actually care to use to control something and then send that thing out of the implant on the radio side I'll hand it over to Matt yes so to answer your question we are looking at other radio Technologies one in particular is 500 megahertz pan with ultra wide band at a couple different frequencies so this has an advantage in terms of the bit rate that you can achieve it's on the order of six to eight to ten
2:56:48megabit there's also a latency Improvement that's quite substantial and there's also a another wireless technology that we're looking at at w band hi thank you all for really clear and compelling presentations something that struck me in one of the earliest talks I think it might have been DJs was this vision for the ability to acquire new complex skills via these bcis like the ability to perform Kung Fu and that reflects the fact that the brain is fundamentally a learning machine and yet the many of the Technical Solutions presented later framed were framed in such a way as to try to correct for the way the brain changes over time over longer time skills drift
2:57:44over the course of days or the way that the tissue might heal over time I was curious what your vision collectively was for developing out this technology that interfaces with a fundamentally plastic system that changes in complex ways over a variety of time skills days months years
2:58:08yeah yeah it's a tough question yeah I think it would be kind of maybe bi-directional learning in some way in the sometimes cars that we will might fix our algorithms and we prefer to have like more stable kind of performance but of course if the over time the person in the brain will learn how to use better to be safe we'll need to update our models so they will be kind of in an Interactive kind of a relationship in some way
2:58:39to learn even new tasks this probably yeah will be something we'll over time will need to learn what the person kind of learn how to interact with the computer and then build the appropriate interface the ux and also on the UI and build algorithms that will help him to control what we want just one thing I'd like to add on to what near said yeah it actually is an advantage in some ways that the brain is plastic and learns and that can help us because we actually have to do less work in the the human in the loop we'll actually learn how to use our device better but one of the advantages of our particular approach and device is that we are trying to do an extremely high
2:59:18Channel count device so we can you know uniformly distribute electrodes over a functional region and then it doesn't matter so much whether things move or shift over time we can offload that to software and so we can build algorithms that change over time as well and so both those things are actually I think advantages to our particular approach
2:59:36we have another question from Twitter Juan wants to know what career path do you suggest for somebody that is just getting out of high school if they want to work at neuralink in the future well it's really any of the skills that we described so I mean we're developing new Chips you this
2:59:56material science you know there's software obviously animal care it's a really all the things that were listed in the in the neuralink slash careers okay that'd be a good guide yeah I'm actually very fond of saying you know when you flip through any college like booklets and look through all the majors I think you can point to every single one of those majors and there's someone at this company who either is an expert or you know have majored in that so it really is truly truly multi-disciplinary Endeavor and I think you know just focus on whatever your you know passionate about or whatever you're talented at and then just you know pursue that as deeply as you can
3:00:46and then there's definitely going to be a place for you in you'll end up building your own interfaces hey we got to see the monkeys doing telepathy but could you say a little bit more about the animal behavioral training kind of their lives and day-to-day processes sure I'm Autumn I am head of Research Services which includes our Animal Care Program and as an animal welfare scientist this is a topic that I'm deeply interested in so our training
3:01:20program is outfitted mostly with behavior analysts who help us think about how to remove any of the potential aversives or frustrations from our training we think about conditioning as the primary which includes positive reinforcement as the primary way to train let's see what else can I share with you
3:01:56yeah yeah I mean that may not be part of the behavioral training itself but we think of Animal Welfare assessment in the framework of the three R's which is refinement replacement and reduction and so when we think about refinement behavioral training does apply in in that way and where we want to remove specifically in research restraint is one of the things we make a very top goal to remove so you saw a lot of videos today where animals were walking up to their stations because we worked really hard to remove any requirement to restrain the animal
3:02:41anything else yeah I can just yeah well just on top of the last point you said just as an engineer here one of the things that is really inspiring and really cool about this place is that we do get to work on a lot of technological innovations that directly translate to Greater independence with animals when they're engaging in these tasks so as you saw you know monkeys charge just by voluntarily walking up to a branch they play games in their home habitat with a laptop computer voluntarily and the fully implantable fully wireless device the inductive charger all these things enable that kind of experience and so this is one of the very cool Parts about
3:03:12working here is we do get to innovate on things like that definitely helps to work with a group of Engineers who can like really make cool stuff for for monkeys to be able to do easier behavioral training yeah so I guess to answer the previous question about what you can study to be part of neuraling I guess monkey engineering you can add to add to that Monkey Business hello my question is on upgrade upgradability which you guys mentioned quite a bit so in that procedure I imagine there's some kind of expand procedure and then you're going to putting a new set of implants so could you talk about the damage possible if any tissue damage from the x-band
3:04:02procedure how long you have to wait do you imply in the same areas and what's your like brain scanning for the implant procedure in terms of upgrading it I don't know how many questions I can ask so I can start to speak to some of those so I I work a lot on upgradability and those explant processes and designing those to be better the the goal that we're working towards is that as I mentioned in the presentation it's really just as easy to upgrade an implant as it is to initially install we didn't we didn't show many of those explant examples today but we've come pretty close to just popping out an implant and reinstalling another one in the exact
3:04:58same location definitely definitely the goal it we are installing the implant in primary motor cortex which is a valuable area for interacting with a device like this and so we the goal is to implant in the same location maybe if you expand out to other applications then you'd be interested in moving somewhere else but we definitely want to be able to insert into the same area in terms of damage the I think that the damage that we care most about is damage within the brain and what we've found and we we talked about that that challenge of the tissue layer on top of the brain and where I think we're well on our way towards figuring that out but because of the thread small size
3:05:43the the sort of Scar capsule within the brain is so minimal that they are actually removed quite easily and so we see useful signals even on the second or third time that you've placed an implant and I think some of our BCI folks will probably speak to that we we do have monkey participants working with their second devices and really
3:06:11making use of those sorry so one two questions one was somebody that asked the question about the brain Bean plastic have you noticed any plasticity from a behavior perspective from any of the monkeys or is it too soon to tell or there haven't been any observations from the monkey Behavior we see that it takes them a while to learn how to of course to train on the task but also when they are implanted and it's relatively quickly for them to ramp up and get to a high performance of brain control with pager for example after a few days he was able to like three days already able to learn very quickly to use the device he was trained on the task of
3:07:05course from this previous implant but with the new one he was after three four days he was able to control to a close to Performance of he he held with the previous implant but have you noticed anything on the adverse side which means the brain has outpaced the neural network that you're running
3:07:26it's hard to say yeah no not really okay so I have a another question which was more about the electrical side so you talked about 1024 channels being recording are you transmitting the raw signal or was it only the three Spike events that you were talking about the low mid and the high or is it the raw entire raw firm waveform that you transmit yeah hi I'm Julian I can speak a bit about this and maybe Avinash wants to contribute but our chips see they're all signals but then when we transmit out typically spikes and we detect those spikes in real time on the chip this massively compresses the data I guess yeah we're making improvements to that but we can request we can
3:08:13request rule samples sometimes we also process particular statistics or other data directly on the chip and then send out the calculated values so there are many ways to sort of play with the data yeah so at least with the current and one system that we have which relies on ble radio there is a bandwidth limitation so you can't actually stream raw data from all 1024 channels but just kind of to give you a little bit of a history of how our compression algorithm the spike detection algorithm was developed we did have sort of a wire system there was a paper that we published with the USB C connector that you know streams all those signals through a hype bandwidth wire connection
3:08:56so we did have kind of those development platforms to be able to see the raw signals and know which set of information that we want to extract that are you know going to fit within the bandwidth of the radio as well as is useful for BCI control and you know also just sending data wirelessly does cause a lot of energy so there's any opportunities we have to reduce that burden you know we try to do basically have all that compression closer to where the electrodes are as possible maybe one thing that isn't obvious is that the the actual bit rate that you need to control a phone or a computer is actually very very low so I think we might have the
3:09:40record for bitrate is that correct we think we do maybe so on the order of 10 bits per second so that's super slow but if you think like if when you're inputting data into a phone like how fast your thumb is moving how many thumbs what's your thumb Taps per second it's pretty pretty low and I mean we're like basically our thumbs are like two slow-moving meat sticks that we you know do this and it's like this is really low it's like a low bar that's what I'm saying so for at least for output it's it's a yeah you get 10 bits per second you're you're holding ass so and that's you don't need your Bluetooth anything for you so you could practically send it out
3:10:25with beeps and Buffs you know so it's not if you go if you're going like a high bandwidth visual now you're you know maybe going to megabit plus but it's it's so it's really well within Bluetooth or wait but anyway it's just we're that is what I'm saying is that's not that's not a constraint that the data rate one of the sort of like maybe notable item which we talked about in the presentation but we we think we can probably solve for doing the implant without cutting the the dura we can just do basically a bunch of holes through the door which is like the door is like the big thick iron D thing that contains the that's up
3:11:10against the skull if you don't pierce the dura you know if you don't cut the door away and instead you have a bunch of tiny holes and insert the electrodes through the tiny holes into the rain and then the recovery time is ridiculously fast you know you're not really losing much in the way of cerebral spinal fluid it's it's you couldn't Theory we I mean this could be like a the whole thing could be a 10 minute operation like LASIK like it's fast it's not like a big laborious thing it's super fast
3:11:50just going back to the long-term use I'm wondering if you have any pathology looking at Scar Tissue from many animals that have had long-term implants and along that lines it seems like there might be a little bit of a gap between use in medical conditions and helping individuals from a safety perspective
3:12:19yeah so I didn't quite catch the last question but I'll heard the first one I'll ask you to repeat the the second one so the first one is do we have pathology from from long-term use animals we absolutely do we don't have any pathology from our monkeys which we upgrade and you know are still going we have other studies that are primarily to determine safety and so we do have histopathologic endpoints that we we determine the scar tissue formation around the threads themselves in the brain is typically negligible like it barely reacts to the threats at all so that that's very promising in terms of the scar tissue formation over the the cortex so this neo-membrane
3:12:53growth that fills in the areas that Elon and Alex were mentioning that we remove with our current operation those we we do you know evaluate that scar tissue but it isn't it doesn't pose a problem in any way it's not a continuous reaction to a foreign body it's just filling in tissue that was removed and if you could repeat the the second question I didn't hear that yeah the second question really following up on that seems like there might be a little bit of a gap in use in healthy individuals from a safety perspective you know I think people mentioned that they might be interested in trying prototypes but just wondering what your perspective is
3:13:29on trying to lower the safety risks yeah it's it's a great question so in terms of really it's about the long-term use of the device so you know we have devices that have been implanted like I said in monkeys where you know for many years where we see no behavioral deficit at all so this first is a question of how you evaluate safety so you have histopathologic endpoints you can evaluate but we're also looking for cognitive deficits or behavioral deficits as well and we don't see any of those in our animals which is you know an important point in terms of the histopathologic endpoints they look really really great the challenge is one of explanating the device which is why
3:14:03we're putting so much effort into the reversibility efforts and our through Dura insertions so when removing the device that's when you potentially could cause damage and so we are doing we have a lot of ongoing studies right now to really minimize the risk of that but we don't think it's a substantial risk with our current approach and like I said pager was upgraded with the previous surgical approach and is doing great so clearly it is you know can be perfectly safe but proving that beyond a shadow of a doubt for humans is something that we're still working to do rigorously to answer your question yeah thank you
3:14:36so thank you for a very deep dive on many of the different aspects of the device and the system it's very impressive to see all the engineering work that's gone into it you just mentioned about bitrate as the prior bit rate holder I can confirm you have indeed shattered my record so congratulations on I think I saw a peak of 7.4 bits per second well done my question is actually around the clinical trials and the FDA to the extent that you can share I gather that that device removal or maybe electrode removal is one of the concerns that the FDA highlighted is there anything else you can tell us about what the FDA was concerned about or had questions about
3:15:18with respect to their ID submission yeah I mean we can probably talk a little bit I mean it's these are really challenges you know that we have broadly so explanation safety proving that right rigorously for humans is something that we it definitely is one Challenge and was something that that the FDA commented on other things that they do ask some really great questions so other things that involve you know things like the thermal bench shop testing of our implants so obviously it's important that our implant doesn't damage the tissue by overheating so having really rigorous and valid bench shop testing for that is very important it's actually
3:15:52Something That We're redesigned to be even more accurate now it's also the case that you know they ask a lot of very hard questions on biocompatibility chemical characterization so we've done very rigorous testing for that but you know they they do ask a lot of questions about getting into the weeds of the data and making sure that there really is no chance for any toxic chemicals or bio incompatible materials to be in the brain so these are all things we're working with you know to to just prove again above and beyond Beyond a shot of a doubt one thing that's maybe worth mentioning here is that it can be difficult to appreciate the novelty of
3:16:24our product so the surgical robot and the thin film array in particular are quite new and unlike existing devices and this means that we can't rely heavily on literature to support the safety and efficacy of the device so we do spend a ton of effort in designing and Performing testing on our devices so that we can rigorously prove the safety of them and we can't rely just on another product or on some paper and that's something that we're not willing to compromise for our first human participant working very hard to do I think if you ask a question like like in my opinion like would I be comfortable in planning this in someone you know one of my kids or something
3:17:00like that if at this point like if if they're in a serious like let's say they if they broke their neck would I feel comfortable right now doing it I would I would say we're at the point where I at least in my opinion it would not be dangerous hi thank you for the presentation so I have a non-technical question are you collaborating with people with motor disabilities and if so have they shared any ideas of applications that they would be excited about
3:17:41production I can take the first part of this I'm not the best person to speak to this to be honest but there is a consumer Advisory Board we have made up of a number of people that have various conditions including tetraplegia and they give advice to us on a number of topics there's just as an antidote someone came to the office maybe six months ago and they were telling me what they most wanted to do with their neuralink device and there were two things that they said one was they wanted to be able to trade stocks day to day to be able to beat their brother and the second one was they wanted to be able to play shooter games so I think what was most shocking to me
3:18:15about that encounter was the normalcy of that and I found that conversation truly inspiring so you know who you are the person who came and talked with me have a great day yeah yeah what something that's we've talked about but it's maybe I should be re-emphasized we are doing we're building up a production system for the devices so we're we're building up bringing out the production line making large numbers of devices we want to make thousands ultimately tens of thousands then millions of devices so the the progress at first particularly as it applies to humans will seem perhaps agonizingly slow but we're doing all of the things necessary to bring it
3:19:03to scale in parallel so Theory it should progress should be exponential okay so thank you that was a very cool presentation so one of the stated goals was recording from everywhere in the brain being able to record from and perturb any location so it seems like currently it's all cortical and I'm curious with the current device is it is there any sort of long-term goal or idea as to extending it into going deeper in the brain I mean for Neuropsychiatric disorders for memory all these things are much deeper several centimeters so I'm wondering what's the time scale if you were to give a very rough estimate of when I can expect to see a neuraling product that
3:19:46goes that deep yeah so I mean the the fundamentals of the device in the skull will stay essentially the same because the as I said earlier the the device in this call is very much like a like a smart watch essentially it's got It's a battery radio inductor charger computer and and then you've got the the little wires and so you need to make the wires longer and you'd have to have a deeper insertion needle for the robot but it's this really is intended to be a generalized i o device so apart from the tiny wires being longer and the surgical robot needing a longer needle in theory you should be able to go anywhere because it seems to me that part of
3:20:40the robot is trying to detect where the blood vessels are and then avoid them correct would that be possible at that scale I mean certainly not just visually but maybe there's some other way of detecting it is that is that like is that a current goal and do you expect that within I suppose the next decade definitely yes I'm Ian I run the Robotics and surgery engineering team here like of the three axes that DJ mentioned one of them is you know hack system Warriors of the brain so the robot team thinks about this a ton in terms of what sensors do you need to essentially go past the surface and so in this case you're right that right now we can really only see down
3:21:17the maximum about a millimeter I think within the team there's questions of what's best to use next but like ultrasound and photo acoustic tomography are two that come to mind as things that can get centimeters deep essentially but it's a super interesting problem you sort of need Deep Imaging and some ability to steer to at least avoid large vasculature deep down
3:21:41yeah or if our if we can make our needles and threads small enough in a way that we can still be precise and accurate at a deep depth then maybe you don't cause a bleed if you hit a vessel yeah I think that's that's really the ideal situation if the threads are really tiny they can actually go through a blood vessel and it's and it's okay if they're tiny enough so so we wouldn't need the blood vessel Imaging in that case I I'm I actually am slightly optimistic that that is achievable and Matt could probably speak more into this but what DBS currently is kind of just like send it yeah yeah the current approach involves a wire that you blindly pass in that's
3:22:28massive compared to our threads orders of magnitude bigger and so that's a low bar for us to clear as well because people don't realize like before the Deep bright simulation just how big the hole is it's a I mean what is it like I mean how basically in current deep brain stimulation how much of a borehole is drilled in the brain yeah you're drilling a 14 millimeter Burr hole and then passing a two millimeter wire you know six centimeters eight centimeters deep into the brain so all blindly hoping that you don't hit a blood vessel telling the patient up front this might be good for you and there's a one percent chance you're gonna your brain is gonna bleed in a way
3:23:13we can't control so that is current technology that is happening right now so doing better than that is we can we can definitely do way better than that there's no problem our needle is 40 microns okay yeah exactly thanks again for the phenomenal presentation I thought it was fascinating how rapidly you could test all of these electrodes but it begs the question about like what your fault tolerance is if you run these Diagnostics and it comes back that you have something that's either shorted or high Z how many of those before you get degraded performance the second question is when you're actually inserting this device we saw examples of the electrode going in and then like
3:23:56looping back on itself but it looked like that was something that was assessed basically by slicing the synthetic material I'm curious what you're doing to validate the insertion of all these electrodes sort of in Vivo how do we know that that's not happening on an actual patient yeah I can enter that SEC the second one so like I mentioned we can so we weren't actually sectioning in that case we have a really cool micro CT so I mean it's essentially like a CT scanner so that's just an intact proxy that we put in this machine and we can you know take a picture all the way through it and like I mentioned before like we can make a proxy where it happens you
3:24:35know that that looping back happens every single time and then we can make one where it never happens and we've pinpointed roughly now where actual tissue Falls in there and so our current plan for you know validating and confirming that is making proxies where it you know happens really easily much worse case than any you know any tissue could possibly be and then designing it such that it never happens in that scenario and that'll give us the doing that enough times and with a weak enough proxy that'll give us the confidence that this isn't actually happening
3:25:07this is the next video Yeah and yeah and this is the next Generation we don't see this problem at all with the the current generation and I'll take a stab at your first question so to clarify you asking what happens if there's a fault on a particular Channel or something yeah that's correct yeah so the nominal scenario is that basically the impedance will stabilize pretty quickly within the brain and even at that level we can record great signals we see lots of spikes and we can use f of BCI because we have so many channels like a thousand now 16 000 later we can actually run our models with far less channels than we actually have so it doesn't matter if like one channel dies
3:25:45here or there like we can still do really good decode I'm not sure if we have official numbers on how many channels we need but it's like we have an order of magnitude more and the more we have about it but we can already do a lot with what we have all right maybe just one or two more questions yeah I have a question about your very very long-term inspiration to have this high bandwidth communication with Advanced AIS so it seems like you know the advanced AI would need to understand the humans most complex thoughts and emotions and that's what neuroscientists are trying to do so do you have any Ambitions to tackle Neuroscience Beyond neuro engineering
3:26:34well I mean I think we're going to make the input output device and the software interface with it and I think probably you know post suggestion earlier we'll try to open source as much as possible so people can take a look at it and I think there will be a lot about those that that build upon the work that we're doing you know the same way that if you make a if you make a microprocessor or CPU or computer that people will write lots of software that runs on that computer so but if you don't have the computer this is the software's mood so we're making the input output device with the computer and and then I think probably there there'll probably be a lot of other organizations
3:27:18companies that that built that built upon that Foundation so yeah I mean one of the things that's I would sometimes wonder is that if you do have a whole brain interface and you can record memories and really getting into Black Mirror stuff here but this could be one of them I I also think it's worth mentioning an important point which is that neural link didn't come out of nothing there's decades and Decades of research in the medical academic field that has really set the foundation for what is possible by putting these electrodes in parts of the brain and being able to read those signals decode it for mapping it to some application and you know being being in Academia
3:28:15before before coming to neurolink you know I do think that there's a lot of opportunities for kind of the field to advance at a much rapid rate by having just better tools for observing the Dynamics that are happening and then engaging with it in a seamless way and I think it was Ian who sort of mentioned that you know it's almost as if like we're kind of building an oscilloscope for the brain which I think is like kind of a beautiful analogy of just giving us a bit more abilities into peering into the Dynamics and using those information learned that to I don't know hopefully understand like what makes us and how the brain works and you know know the
3:28:51whole champagne hello the presentation covered keyboard and handwriting based input methods how do you plan to develop an input model that will achieve much higher bandwidths for complex tasks in humans yeah this is a tough question and we start exploring this with monkeys as you saw we have like a multiple we train many monkeys on very different tasks it's still an open question that we're after I think hopefully once we get to our first participant it will be easier to investigate one of the options we are exploring as we showed is to the code handwriting directly this is a one a work that started Stanford and we are exploring here and trying to expand
3:29:36there's also a different in addition to just decoding different things from the brain we also try to provide the user different maybe user like interfaces for example we show different type of keyboards maybe also swipe and other things that can help increase the communication rate so we are kind of tackling those in two dimensions yeah just one other thing to add in that direction as pointed out by many people here so far this is a general i o system that you can sort of plug and play in different places in the brain there's other areas of the brain that can help increase bandwidth for example language wrist speech centers that can help you much more seamlessly
3:30:14communicate for example text if that's your main thing that you're trying to do yeah just I think just having this General input output device will just so gigantically improve our understanding of the brain is is hard to the words can barely Express like you know right now we're just guessing a lot of what's going on in the brain but if you have direct i o it's not normal guessing of what we would learn about the what we will learn about the brain with such a device in in wide use is absolutely or many orders of magnitude more than we currently understand so I guess on that on that note thank you for coming and thank you for watching online [Applause] [Music]
资源与出处
- 视频在 YouTube 收看
- 档案silicon-valley.tech 档案页本页视频与文稿数据来自 Elon Musk Archive
- 精读本篇精读 Markdown 版