Panel Show Me the Money

0:06All right. Um, so we're gonna get going. Uh, thank you, uh, everyone for coming to our AI show me the money panel. Basically here we're gonna talk about whether AI generates any profits and we have an incredible lineup of speakers of the panelists uh who will share their experiences and help us understand what is the way um to get to like the profitability. I will take some time and

0:32ask everyone to introduce themselves. Hi I'm Dylan uh semi analysis do like AI infrastructure consulting and stuff like that. Um, thank you. Hi, I'm Hung. I'm a VP of technology at Quankcom. Uh, in my previous life, I used to be co-founder CEO at Vaii, which got acquired with Quankcom back in April this year. Hi, I'm Tree. I'm chief scientist at Together AI. I'm also an professor at Princeton. I work on efficient training

1:03and inference. Hey, I'm Rahul. I lead the AI team at RAMP. That's a great intro. Uh hi I'm Swix editor of lindspace and I'm technically member of staff at cognition. Hi well I already introduced myself here. I'm at Mila and University of Montreal. Great efficient. Thank you. So we'll start with an icebreaker. This question is to everyone. Um right now there seems to be two different camps. People who believe that AI businesses are already profitable and some people believe that it's sort of like a crypto bubble. um where a lot of the valuations are kind of inflated. So what's your personal take on it? Go

1:46on. Okay. Um so I I think I think uh AI businesses many of them are already profitable if you squint your eyes, right? And and we're going to squint our eyes pretty hard. Let me walk you through it. Right. So well objectively their inference margins are like 50%. So they are they are making money off the inference. But then the obvious question is well like they spent all this money on training. Um, and so what's interesting is if you take the sum of all the R&D expense of the last year and before and then you compare that to the revenue they have today, it's there's actually a 4x

2:16multiple and that's been the case for open eye for the last three years. So um, open eyes reg routinely has some sort of return on invested capital. Um, and their gross margin is 50% for inference. So actually, you know, if you squint your eyes and and anthropic is a similar sort of breakdown. So if you squint your eyes, they're actually like going to be profitable. But then they keep trying to scale up training by like 10x every year. So then of course on a

2:38financial basis it doesn't look good. But that's because all that 10x of training is is for the next year's model. Right. Right. So um I think you know I think this topic has been discussed at before. Right. And you know there there's a typical questions like it's typical perspective like okay you know um this is not going to be um AI is not going to go away there's it's going to be here in the future and so what does the bubble mean right I think a bubble probably means that okay you know well what something like this is going to burst and what bursting means right bursting

3:13probably means that uh you know people will start running out of money without be able to search for uh the candidate for uh you know uh you a killer app or you know ways to make a lot of money. So, you know, um it's it's it's it's it's um it's not clear, right? Obviously, you know, because of this is this is a game of investment. So, we we we're willing to take a risk and, you know, I think we we we're willing to to be able to uh you know, like Yeah, I think I think a risk is is obvious there. But having said that, I think

3:41efficiency is really the key in lowering this risk. Is that right? Because I mean you get things more efficient and you know, you just get more people people will get more time, right? uh uh with fixed amount of money you'll get more time to get more people to explore and yeah it's more likely to be able to find right uh that kill application before like time and money run out right uh well most AI companies are startup and uh I don't think they aim to be profitable unless you're about to go public which some of them are um then you would worry about your your uh your

4:15bottom line but right now I think they're most mostly optimizing for for revenue the revenue seems to be going up um So I think if they really want to they could be profitable uh a lot of them um but they're not really aiming to be profitable right now. I think uh as humanity we're trying to figure out we're running a massive search on on everything in the entire stacks from hardware to software to inference to applications and there are companies allong all around the search space. Some of them will fail, some of them won't fail, some of them will figure out a viable business model. But

4:48this is why VC exists. And so both those things are true that some companies will work and uh AI is real. And I think everybody probably agrees with that. By the way, we don't have to go everyone reply every question, right? It's like I don't know, not every question. You'll have a special one. Uh quick uh my quick anecdote which I'm not going to add on to the others. uh the thinking game was came out as a documentary which was covering Demis and Deis there's this famous clip where he was in the cab he's like you guys understand I'm trying to solve intelligence and you're worried about

5:19profit like in like in like gap accounting revenue or something like who cares like just fund it yeah um okay Ethan can you quickly introduce yourself and the question we are trying to answer um there are two camps of people one believe that AI businesses are profitable already and There is another the kind of camp that

5:41believes that it's all cryptobubble. It's all going to burst at some point. What do you think? Hey, I'm Ethan. I work in AI. Um, so, [laughter] so, uh, this is not biased answer. Yeah. Um, so I'd say they're kind of doing what Amazon was doing back in the day. Like all Amazon cared about was growth. And like Bezos would have the letters to the shareholders like, "We don't care about profit. All we care about is growth." like they already could do profitable things, but they care more about like winning the, you know, race to scale the farthest. It seems like um another interesting observation is Enthropic says we're

6:17going to break even in 2028. Opening eye says we're going to break even in 2030. And partially it seems those numbers are made up based on like the meter plot like extrapolations. So, uh I guess it's just an interesting coincidence that what they claim like lines up with the meter plot extrapolations for like when you get to

6:35one month uh task length rollouts. I I don't know if anyone's talked about that or just I think that's an interesting thing to to talk about and just note that they seem to be lining up with that. I don't know if it's intentionally or not, but they seem to be lining up with the meter plots. So I guess you kind of change the question from profitable versus

6:57nonprofitable to bubble or no bubble. I guess well uh my take uh it's profitable. I guess depends for whom for some yes for some not. But definitely I I I don't know. I I uh I talked to quite a few people and they were like even like what do you mean by this question? Of course it's profitable but um I I guess it's not universal but from my point of view

7:29uh there are clearly some applications. It also depends. Yeah. So it depends like if it's a purely like research organization that yeah probably it's not kind of bringing money. If it's a very practical application then definitely yes. So I would say okay thank you. So my next question is going to be to Dylan. Um I know that your research the economics of AI infra and uh from training perspective like we're burning 100 hundreds of millions of dollars on building these models that we use in various industries. So from your perspective where in the next three to five years you see the sustainable

8:08profits. Yes. I think I think in the first go around I explained it quite poorly. Um but if you take OpenAI today right their revenue is roughly 20 billion um on a reoccurring basis right now right in the month of December they're going to be roughly 20 billion of revenue they started the year at much lower um but if you take all of the R&D expense this includes training this includes research this includes you know all the R&D expense that open paid out in 2024 and before so all the way from 2018 to 2024 that number is about 5 billion um and so what's interesting is there's a 4x

8:44multiple and if you go back a year on both of these metrics it's actually about roughly a 4x multiple again. So I think that's really interesting way to look at it. Now obviously you know you're kind of saying okay well ignore this year's R&D expense because this year's R&D expense for OpenAI is is is north of 20 right? Um so so obviously there is like a bit of like you know I'm playing some timing game here but what's interesting about that is is you can think of it as like you know the these while AI seems to be moving at some insane pace a lot of these developments

9:13um don't happen that fast right pre-training takes months post training after that takes a bit and then you have all your safety and red teaming and all that and and when you started your pre-training you actually had already generated those ideas before that right so there's there is a long time lag here right um and so in some sense right so far all the capital they invest in ends up being, you know, coming out to revenue, right? Now, maybe not directly, but the models get better, more use cases unlocked. You know, this year, um, you know, we've seen many people soar in revenue, for example, on the coding

9:43side, right? Uh, even in just the last four months, OpenAI has gone from Codeex being a Z business to it's like an $800 million business now, right? That's that's incredible. But a lot of that is based on R&D they were doing last year on testime compute for code, right? Now obviously they've continued to make it and improve it this year but this is this is the sort of core belief is do these things keep scaling as they keep pouring money in. Um, what's interesting is open could cut off all R&D today and they would be profitable, right? They would be an objectively profitable company. $20 billion of revenue, roughly

10:12$10 billion of inference spend. Um, now of course, you know, they're not sitting still. No one else is sitting still. The revenue would go to zero and, you know, probably a year or two because no one uses OpenAI anymore because their models are not improving. Um, and so that's sort of the rat race that everyone's in. Um, which is sort of like, you know, there's this gamesmanship where everyone keeps like having to spend even more, right? like oh Sam's announcing hundred billion so now Mark Zuckerberg needs to announce $200 billion right and and and these things while they are announcements there's physical logistics happening behind this all right um and

10:42as you flow down the you know supply chain whether it's you know openi paying Oracle Oracle playing you know Cruso or Oracle paying Nvidia and you know you go down the chain um there there's a number of players making money right Oracle is objectively profitable right on their AI infrastructure spend same with Microsoft Nvidia is making tons of profit so so yes open AAI is you know burning a lot of VC money. Um but theoretically they are they could be profitable today and and their R&D that they've been spending is turning into a profit and then you know same applies to Enthropic right sure they went from 1 billion to 10

11:13billion revenue this year right on a reoccurring basis. So so as far as like where is this going to be profitable there's there's roughly $200 billion of uh junior software engineer uh which is less than five years of experience uh wages paid throughout the industry. Um, anyone who touches cloud code, codeex, uh, cognitions, Devon or or Windsurf or, uh, cursor, you know, I don't want to kill anyone by not mentioning anything, but like GitHub copilot, right? You know, you got to keep going. Replet, you know, these these all of these companies have more than 100 million ARR, by the way. Um, some of these are startups

11:45that, you know, had zero revenue at the beginning of the year. So, anyways, um, you know, I think I think, you know, there's a huge TAM for the software development landscape. Um, same applies to consumer, right? Consumers are going to be huge. Um, you can already see Google is serving Gemini 3 Pro, which is a greater than trillion parameter model in every search. Are they doing that to burn money and light it on fire? No, they've got like extreme efficiency in serving uh that they've been able to uh get from their amazing infrastructure and they clearly see that the value of delivering this generative search experience is worth it from just the

12:18ads, right? Think about it. Tokens, a million tokens of Deepseek costs like 60 cents, right? Um, the average ad click-through ratio, like once you click through, it's incredibly high, right? The monthly revenue for a suburban uh mom in America for Meta is $250. So, you can put a lot of really expensive tokens into search or into ads. And it makes sense, right? So, it's coming. There are profitable ways for these mega companies to throw this stuff in there. And obviously, there's all these use cases like code. So, I think, you know, no matter what way you look at it, we're we're we're going to be profitable. I

12:50think it makes sense. It's just, you know, we're in the exuberance of let's 10x spend every year. Uh so we can, you know, if that payoff comes, then hey, OpenAI next year is going to have more than $40 billion of revenue, right? And this treadmill kind of continues. So, um it's it's all about, you know, spending big because you're in this rat race against everyone else, but also, hey, are the returns going to be there? Now, now you can obviously look around the landscape and see like Meta's not doing so great with their generative, you know, side. They haven't really made money off of that, although they are

13:17implementing some of that into ads, right? It's sort of like a you know there's people where they're doing well and there's people they're not. Seems like um kind of summarize a lot of return on investment will happen through the ads right that that's what you're saying. Well and then the 200 billion code right $200 billion of junior coding and I think that's like directly addressable

13:36today. It's just penetration. I agree. Um in disagreement I can push back a bit. [laughter] You work for a code company bro. No no no no no. This is pro- aagent lab uh negative model lab. Uh so so the the point being like that that analysis in in in sort of like an individual company perspective makes sense but it may fail in the aggregate right because there's a there's a maximum $200 billion or whatever and like you cannot pour infinite money into like there's like what that you know there like three years ago open was relevant last year anthropic was relevant this year deep mine's relevant like you cannot keep

14:14adding because they cannot all pour the same amount of money in and get the same money some amount of money out there's only a limited bucket to go So, uh, yeah, it's it's it's not only that ROI calculation is independent of like opening and winning, it also has to be in like somewhat of a pole position. And if you're like number three, you you

14:31you get in trouble. So, yeah, you you you can you can argue they are number three and CEX still went from, you know, zero to 800 million in three months, four months. Um, Codex is is is revenue is that high. Yes. Um uh there there's a leak about it. There's a financial model internal. Anyways, um I I I agree that like um I agree that the loser isn't going to make money, but as long as someone is making money, then this is a sustainable

15:00business model. Yeah, you cannot have 20s. Mike m come back. H we just bit short on time. [laughter] No, that will not happen. Okay, H question to you. So your um Genai startup Vin AI was recently acquired by Qualcomm and you've seen the entire M&A process firstand so the question to you um do you feel like the valuation of the current AI businesses is driven by the

15:33real value they produce or the hype? Oh wow. Okay. So um okay so I don't know whether my answer is able to like generalize to you know like other acquisition so on but I can talk about a little bit about uh uh Vini in in a context that actually led to the acquisition. Um so yeah so I so so so vi actually is a startup a generative AI startup based in Vietnam and um I I I started spini around 2019 and um this is the the the time when I decided to uh you know u move from deep mind in modern view uh going back to Vietnam and try to

16:18start AI research lab and and you know like uh how silly that that sounded you know back then. Uh but you know I think that the thing that that's that the two thing that struck me the first thing is that okay uh when I moved back to Vietnam from you know uh Bay Area uh very first question is you know where I could actually uh where I could actually find the talent right u and then luckily you know I solved that problem by running AI residency in uh program in Vietnam you managed to hire a lot of really young smart uh uh you know people freshly graduates from uh technical

16:50university in Vietnam they were they were super super smart and we were publishing like tons of papers in Ire and Ner and so on and making our names known to the whole world and okay so that's the first um the second is that well okay I I I no longer had access to Google compute so then uh [laughter] uh what what do we do right so um well so we had uh about 30 nodes right it is 20 20 before chachi we had over 30 nodes um um but you know that that that was already you know like way way way better

17:20than universities even in the US Right. So um and we were working on deep generative models right from the start because because of you know the deep my influence and whatnot and and when chach came out in in in in 2022 uh very quickly were you trying to build uh models pre-trained from scratch for Vietnamese um and both in you know this sort of like L&M and also uh image generation model diffusion models. Uh but of course you know we have 30 nodes

17:48and you know what is it we could do? Okay. Well, you know, we we found exactly every everybody I found you could only train like seven billion parameter models or less, right? Uh and then uh what what were those models useful for? Well, you can try to get them to run on the phone, which we did, right? And in 2023, we're getting uh L&M 4 billion 7 billion parameter model running on the phone. Uh and we were also, you know, getting this diffusion

18:13model had to be distilled to one step. That's that that's that's not trivial, right? But you had this one step diffusion model also running in real time on the phone right as well right and uh the phone was a Samsung it had Snapdragon inside it was a Quong SOC so you know one thing led to another uh u so I you know I I I think for us uh you know the value for Quang was real right and uh and but you know um and and I think you know we're going through this process and and I I I I didn't see uh in any way sort of like you know valuation

18:41going away right so like just but you know it doesn't generalize to other acquisitions I could you that's That's that's what I that's my answer. Thank you. What do other people think? Is it hype or real value?

18:59[laughter] It's it's clearly a lot of hype, right? There's uh there's 20 labs. As Swick said, there can't be 20 labs. There's 20 labs. Say value is not necessarily what's being optimized for. They're optimizing for how do you maximize the amount of funding you can get because they're like trying to have is the value or the hype because they want to scale farther than everyone else. So it's about how do you get the optimal funding model? Like

19:23they're not optimizing for value. They're optimizing for funding like in for their compute basically and energy. All right. But do we do have Google acquiring Windsor? I I I I forget for billions of dollars. So um yeah uh you know Nome Shazir uh went to Google like they were I as far as I

19:46understand isn't a talent acquisition. It's like they they're not using uh Google is not uh is not licensing uh character AI model as far as I know. Uh so some of it is at talent acquisition and um surprising to me that the the uh the u the price of AI researcher keeps going up. I would have I would have thought

20:08that's surprising though. Yeah. A year ago hard. Yeah. A year ago my my my thought would be well uh where is the value going to be created and it's probably going to be in these products right uh like cloud code is amazing but like the model is smart enough already. We need people with good product taste uh to to make

20:26these products that generate uh value. Um but there's still a bunch of uh these new apps uh popping up and and these researchers are being paid a ridiculous amount. You know, I'm not complaining. Um but uh I a year ago I thought most of the value would be come from coming from product people, but I I was wrong. And and now uh yeah, researchers are still being paid a ridiculous amount. Zak is uh is paying I don't know hundreds of

20:53millions of dollars for for for people. So uh some of it is hype probably some of it is talent acquisition. Yeah I guess it has something to do with the fact that we are doing this closed sourced models. So people don't know what's happening in the big labs and those who have an option to see it they

21:09become high in value. Okay so we will move on to the next uh question. Tree this is for you. Um I know investing into the right hardware is super important and you at together AI is trying to diversify between like not just Nvidia hardware but use other vendors. Um so I guess like my question is um since we see a lot of investment happening in the loop between open AAI and Nvidia what advice do you have for non- Nvidia hardware long term? How can we continue staying like relevant and competitive to

21:47Nvidia? Right? So at at together we're still we're on Nvidia right now and I I won't be able to speak uh about the specifics but I I I'll speak in general about companies for example doing inference. Um so I do see a diversification of inference hardware. You know Cerebras is uh is uh doing some you know Grock um there I I think that the you have to make a bet. So if you're if you're a leader like Nvidia, you don't have to make a specific bet. Basically, you optimize for all of the workloads, right? Um but if you're a a a a startup, you I think you have to make a bet. uh

22:24if you're making new chips, you got to say, well, I bet that oh, it's going to be people are willing to pay for super low latency uh uh uh inference for example, or people are willing to pay for very very high throughput or maybe it's like video generation is going to be the thing that's uh uh that's going to be hot next year, right? So, if you don't make a bet and you're trying to do everything that Nvidia is doing, um it's

22:49going to be really really tough, right? Because Nvidia designed very good chips. But if you if if you if you make a bet and you say, "Well, I'm gonna make a chip that's only for let's say video generation or I'm g I'm going to make a chip that's only for uh real time agents or something, right? Um and there's 20 30% chance of of that turning out to be

23:09true." Um then you would succeed, right? So I I think the only way to compete with with Nvidia is you have to make these risky risky bets. Um um I would suggest I would not suggest you know optimizing for all all of the workloads just just pick one lane. Uh just a comment I mean from [clears throat] what I observe uh there is bunch of new types of hardware popping up at least there are many startups doing that and I think there is quite a lot of promise in focusing on speeding up uh the software layer generation basically using whatever generative AI I understand that of

23:50course it's much harder to automatically generate code for the hardware for which you don't have too many examples of that code. But I think I might be wrong. I think that's where things are currently going. Although it's very challenging. I think if more startups will focus on accelerating uh the software layer generation automatically for new hardware, there will be more and more new hardware popping up that will be

24:17competing with Nvidia hopefully. Flip side is uh there's only one tree down. He's only making flash attention for for Blackwell. So, so where who's going to make the uh hardware software codeesign optimization if tree only works on Nvidia and only you know implements Nvidia specific architecture features for flash attention 4. So I I will say my hope is that the next version will not be written by me. It will be written by a language model. Um so you know this is something that uh we're working on and working on with the students and collaborators and so on. Um yeah I agree. So the so the the biggest

24:54moat is this this uh software moat right um and so if you want to cross that that mo like one thing is you can try to appeal to developers and so on that's been you know some companies have been trying that the other way is you should say how do I design my hardware how do I design my programming abstractions how do I design my compilers and profilers that would make it really really easy for AI to write the code for your accelerator makes sense any other opinions on this topic I think This is super important and for us as cerebra it's also

25:25important to discuss that. Okay, we'll keep going. Uh royal question to you. I know that um at ramp you are using foundational models already pre-trained um and you fine-tune from them or continuously train them which seems way more practical than pre-training from scratch and spending so much money on it. Um however it seems like with um this type of approach um you might let's imagine the world where open AI decisively wins and we don't have open source that can uh compete with that. What's your strategy to kind of avoid getting into this permanent dependency from open AAI business plan

26:06and their releases and their prices? Um so here are some things that are generally true. Um most people in the world aren't using LLMs. Most people work in front of a computer are not using LLMs. Most people who use computers are not using LM to the full potential if progress were to stop today. Uh and so there's plenty of uh models today that we could continue using to to like go after them. And so as said earlier, a lot of the effort and uh that that we're putting uh right now is to build a product, the experience and and understanding uh workflows and in financial uh in the financial

26:38industry. Um and we're really not worried about price gouging AI. I mean, they're a reasonable company. There's plenty of other uh competitors uh very certainly we could spend some effort and uh build or or like fine-tune or do RL on open source models and reach the same level of capability. It would be a bad idea for us to take 150 or 200 use cases and then train individual models and host them separately. Uh so if anything we'll pay way more to open AAI so that we can go go after building the product as opposed to doing something else. Uh so we're we're making the investment not

27:09to um not to go train or even like post-train models at the moment and just solve the other piece which is truly what is rare today and go after those people who aren't using outs as much as they should. But you're still using open source models right that you fine tune from. Uh a little bit. Yeah. But most of our workload actually goes to the three big

27:26labs. Okay. So you're diversifying not just open AI dependency but other places that build models at the moment. Yeah. I mean uh I mean uh we've been using uh LLM's uh ever since like OpenAI released their uh chat uh their completions model. So this was before RHF um and OpenAI would straight up go down for five hours a week. The API would go down because there weren't really any production use cases. And so we've always built our our products in a way that can automatically fall back to uh any labs. And so we're trying to find tokens somewhere. You you have to build your product in a way that's resilient

27:57to uh some of these things. And you can be like, oh, if 5.1 is not down with reasoning level low, it's over, like your product doesn't work, it would shut it down. Uh, and roughly the all the APIs are the same. They all produce tokens. Uh, and so I we should be able to swap things out in and around. The other main thing I I would like to note is, uh, when there's a scenario where the model cannot make a mistake at all,

28:18we probably shouldn't be using an LLM. And so now we're in a scenario where either the model makes a mistake one in a thousand or one in a 100 or one in 10. uh and because of that you have you are forced to build a product experience around it and you handle the errors effectively. Uh and so irrespective of what the error rate is we have to build that experience first. Uh and we'd rather start with something low the higher error rate and then at some point when we solve this experience then we can go after the use case once we are settled on the exact setup exact prompt

28:44exact task because every three days we're swapping prompts we're swapping models we're swapping use cases it's actually a bad idea for us to like start using open open source models. That makes sense. So I guess any other opinions on the topic? I feel like it's Oh, yeah. Go ahead. Uh just a bit of a It's really interesting how similar we're running things. Uh Cognition also uh the way I put it is every inference call site is a model router uh that we build internally. Uh and and so everything has to fall back in the same much the same way. And what was the second one? Uh

29:18well you had you had like a second subsidiary like tolerance to uh yeah tolerance yeah build it in the UX and not not throw a bunch of LMS on top of it to uh do that. These are all like really interesting design principles. I I'm calling this like the sort of group this group of uh companies and tactics as the agent lab uh model as opposed to the model labs. when you behave as an agent lab, you sort of treat the model labs as like utility providers that you just sort of uh manhandle and [laughter] and and uh and play around with. But like I think it's really interesting

29:50because like the margins on the agent lab side might be much higher than the model lab side. Interesting. Anyone want to take the other side? Oh yeah, that would be to die. [laughter] Okay, I guess we can move on to the next question. Um Arena this question is for you. I'll just pass the mic to you. Um so I know that you are researching how we can go towards AGI. However, I feel like uh at Nurips uh the papers that get published that are a bit far from what we actually deploy um in companies and I'm curious when do you believe the scientifically interesting research

30:32become practically useful? Basically, you're referring to the joke that everybody keeps mentioning about companies that Yeah. So, if something doesn't work, you publish, if it does, you don't. Right. [laughter] That got to this point. Uh well, except for academia. Uh but on the other hand, I I think uh most of the things that are practically important now did come well, many of them did come initially from

31:02academia, but later on were developed. So it's not uh but I guess being developed to the point that they are really practically useful probably does happen these days mostly in companies but in a sense it's like um kind of symbiotic relationship so it's not like whatever is created in academia doesn't go anywhere so uh in a sense I'm kind of between um between both being kind of in two startups and at Mila and consulting

31:32in several um kind of other companies. Uh yeah, I kind of see it from both sides. But uh I I think the truth is in the middle. So the breakthroughs do start in academia but be then later on they being actually developed to the point of working these days mostly in industry. So I would I I would say so although I mean yeah transformers came out of the research lab in industry and not in

32:03academia. So it goes both ways. Yeah. bottom line is um I wouldn't write off academia just because they don't have compute and uh many interesting things if that was part of the question to uh like you know people trying to find ways around and getting to supercomputers and so on and then you can do open source and scale and study and so on so forth uh so yeah I think it's a combination of both okay makes sense so you're saying basically it's still important I is relevant to go to Nurips's read the papers um if you want to scale AI models

32:41and want to be efficient about it. Well, yeah, I understand that you cannot read all the new papers, but here is your well chart DPT or whatever you use to summarize things for you, right? I mean, we automating everything, including you reading the papers and me. So, yeah. Yeah, it makes sense to ask your whatever favorite chatbot to help you with that. Yeah, you might find some

33:04interesting ideas. So they are not necessarily all behind the closed walls. Yeah. Any takes. Yeah. So um I would say yeah I I absolutely agree lots of uh the innovations that that define AI right uh right now actually came from academia even the transformer. If you if you look at any of the components of a

33:28transformer um it came from academia. attention um came from Mila and uh you know folks at Stanford were working on it before uh Google put put it in the paper that says attention is all you need uh you know the residual connection was uh you know coming was I guess it was MSR Asia right layer uh from uh Jimmy like the atom optimizer uh Jimmybot right any of these things uh I would say 60 70% of them came from came from academia And now Google did did you know the folks at Google did an amazing thing. They put all of this together in a uh and it works really really well

34:08right. Um so my prediction is like right now we're kind of like at the in the execution sort of exploitation phase. Um and it's it's working well but maybe in a couple years we're going to plateau and then that's where we're going to need big ideas and maybe 50 60% chance these big ideas will come from academia. Yeah. And uh just to uh add to that um yes so most of the current state-of-art AI that kind of solves the problems that academia was trying to solve for many years does come from industry but unfortunately in industry you don't have time for exploration you have to do

34:50exploitation. So if there are any crazy ideas and somebody is thinking about well how do we bitter lesson in transformers you don't have time in industry or startup to focus on that right because it may not work most likely or might not be relevant to the business yeah it might not be relevant to anything I mean and uh yeah potentially it could be but again it's a there are class of startups which are very potentially high risk uh high reward and the high risk may be like extremely high and they may get funded especially well in Bay Area yes in New York probably not I mean there's huge difference between

35:27the east coast and west coast investors but let's not go there uh but yeah so these type of things probably safer to do in academia yeah and uh that is kind of downside of industry because you have to compete you have no choice right in academia you also have to compete but at least there is some space for this type of crazy projects that well as I said high risk

35:56high reward. Yeah. Or you need to have like a great investor into your startup that uh doesn't mind you exploring totally fringe things that potentially may kind of I don't know put transformers out of business or something like that. Thank you. Okay. Uh we're going to go to the next question. weeks. The question is for you. I know you are building a community AI engineer community and you see a lot of innovations on the software level. Uh do you think that as a community we are deliberately optimizing the improvement on the software stack versus sort of like going towards edgi from scientific perspective? What's your take on this? Is that um if that's true,

36:41the follow-up question would be is that because this is where the value is and that's the money.

36:52I think it's pretty deliberate. I think the profit motive is very important especially in the Bay Area. Um and I but I I do think obviously there there there's a lot of uh innovation and advances that are very could be very important in AJI. I just don't see the majority of money flowing there. Uh right now

37:10majority what? Money and money, attention, time, papers, products, whatever. Um yeah, yeah, I would I would say like I like this is the year that Apple intelligence has, you know, shipped broadly to billion devices. Uh this is the year that Gemini Nano shipped to every Chrome browser and none of us in this room use it. Um so yeah, thank you for sharing. Thank you for

37:33sharing. Yeah. Yeah. Yeah, I know. I don't know if that helps. I would say AI engineering more on this software sort of agent building layer um is still very much the domain of large models and um I it it does feel quite niche but I I do think that uh just wait like [laughter] it is always a process of diffusion from like the future is here but not evenly distributed and I I think um people might be too impatient for the thing that they want to happen but you believe that it's mostly optimizing the software right that's what we're because there's a lot more software people than hardware people. The the the

38:09number tree does in the world uh is not high. Um so [laughter] uh so so like yeah that uh people always very pro people like uh people that can sort of um cross the boundaries there. Uh it is just not very common. Hopefully people like are finding it easier to to cross the boundaries because LMS speed up learning so much. Um but there there is definitely a strong software bias in everything that we do and um I I think like in to some extent the hardware people have realized this like this cerebras now having a cloud service rather than just selling chips. I think

38:43thank you for mentioning us. Uh yeah, cognition is giant giant fans as you know. Um and and but like you know finding PMF for cerebras like that uh is a is a is an indicator that software drives so much demand. Um and you you need to make yourself legible to the Bay Area with your API u rather than

39:03like force us to learn your hardware. Makes sense. And also like software is like more tangible thing compared to edgi that's hard even to define. Does anyone have any disagreement with this? Yeah, maybe I maybe not a disagreement. I think you know um the pitch for AGI is is is great. I I my personal bias is I don't buy too much. I'm not super AGI field given the fact I'm not at a at a frontier lab. Um but uh I'm I'm sure I have tons of friends at these fronti labs who actually genuinely believe uh that AGI is within I don't know five years or whatever. Um and you know Sam

39:41Alman will go out and and raise money for for for this. Um but if you watch what what these companies are doing like they're optimizing for profitability. They're trying to go public. They uh they are building products right? uh if you read uh AI it's 2027 which is a scenario where they you know it's getting a fast takeoff and you're you're building AI that will build AI and it get better and better. These labs are mostly not really doing that. If not they if if they they're really trying to go for self-improvement. They wouldn't be releasing really really strong models. They would keep strong models to

40:16themselves so that they can go faster. Um but the fact that they keep releasing better and better models, they're not really keeping the models to themselves. Um I think the at least the the business leaders would uh are also also aware that they're not they're mostly building products. They're not really uh you know AGI is kind of a good thing to to raise

40:35money on. So they they release stuff where they're able to distill it and to be small enough to like go for the general public, but they have bigger ones that think for longer that they keep internally. That that's just all. So like the the fancy like AI 2027 stuff they do keep some of that internally just because they can't distill it to smaller models that they can deploy to

40:54800 million users. Just general comment again whenever somebody mentions AGI uh we have a huge variance in the distribution of definitions of that thing right so I mean maybe we shouldn't go there but technically like if you look at what we have right now and if you showed this to somebody 10 years ago when people worked on out of distribution generalization and would put all kind of tricks and invariant risk minimization and this regularizer that regularizer and um then you show them clip and right the zero shot out of distribution generalization. So by all I don't know metrics of uh researcher I don't know even less than 10 years ago that was

41:45pretty agish but the goalposts are moving right so I guess the question is you can take very very practical definition you say artificial general intelligence which means artificial intelligence that generalizes well out of distribution the question is how far out of distribution that's one way and there is another extreme you can start um kind of giving definitions that are further and further away from outer anyway like consciousness has nothing to do with AGI. So definition is very vague that's why it's kind of I think it's it's hard to debate because people may disagree because they just uh mean different things. So I think by the way

42:29in some practical sense AGI very much within reach. I don't know like about the timelines. Ethan is probably more expert on that. Uh but uh no 10 years for sure. Ethan do you want to comment on this? You need Yeah, I can. I don't I don't have a special comment on this [laughter] or it's like AGI is already here. We just I mean it's just like the meter plot narrative is like the the comment but

43:00most people already know it. I know it's just a point on the scaling loss, right? Yeah. Yeah. And we're just going there. So I don't know like what your personal idea timeline. I just say like something similar to recursive self-improvement before 2030. Okay. And for recursive self-improvement, you need to look at the plot that I was using in my

43:17presentation. Get to one meter plot. Yes. Go ahead. Yeah. So, so maybe I want to add a perspective to this, you know, like I think to me um intelligence is also about efficiency, right? And this is an efficiency workshop. Uh I mean brute force intelligence is maybe not uh not not not ex well, I mean it it does the job, right? um in terms of entirely behavioral uh you know exam and how would you score in terms of just look at entire behavioral system but I think intrinsically um we we haven't scratched the surface on efficiency yet right uh I think the human brain is really really

43:57brilliant at how to do things in efficient way um uh for example today you know if if if you if you give chachi okay you know I have only one cent give you the answer right uh how it's going to do it right so there's a lot a lot of thing about you know how to manage computation and get best right juice uh per what uh per compute and what not. I think that's that's that's that that's really uh is kind of underexplored at the moment. Um and I I I would say that that again you know it's going back to why I think efficiency is really

44:24important. Um it will enable uh the whole new uh class of uh uh applications as well. You know when uh uh you you need real-time system you need to give you a response in uh you know like less than subsecond for example. You just have no time to send your data to cloud and going back. So so then what do you do? You have to run this thing on device and then you need to really really efficient model to do that. uh then uh I'm sure that you know there are many times when in the audience you um you you're about to answer to ask chach a

44:55particular question and oh you know that's too sensitive oh you know that's too personal I don't want to do that I don't want to enter that into charge so you probably want to have you know assistant run entirely on device and again efficiency right so you know like I think this is I think you know going back efficiency uh both is I think is important aspect of intelligence that needs to be studied um and and also it's going to enable whole uh new suite of uh application that's not it's not there

45:20yet. Yeah. Um I I actually disagree. I I I think um let's say like you have a serious health concern and um it's like something that you are really worried about. Um what model would you ask? Would you ask an ondevice efficient model or um or maybe it's somebody you care about? Um I I think it's clearly like you you don't want an efficient model right here. And uh the the the other way to think about it is um if you had a a high form of intelligence and it wasn't efficient, you can just ask it to make it efficient. Ask itself to make itself efficient. So I I I think that there's

45:56like so many goalposts and this is just an unnecessary goalpost in the the whole path in and one way to look at efficiency is like uh you know it's very efficient for the consumer not to spend more money on computer hardware on their phone or laptop that can run LLMs um and instead stream it from the cloud, right? It's very efficient for the consumer to get free tokens from WhatsApp AI or Chad GPT AI or Google uh cloud AI or whatever for free. Um it's it's way more efficient that way. Um and then that's an ad supported business tokens millions of tok cost per million tokens of you know

46:28deepseas 60 cents, right? And that's a frontier that's a frontier model, right? So um you know you go down one rung in size um you know and and you're at single digit cents per million token. It's it's it's incredibly efficient, right? uh and and and on device you don't get batch right uh whereas whereas when you're in the cloud you batch many users together and and that again is more efficiency. So I think I think a lot of times the efficiency comes from

46:53being in the cloud, right? Let's try to be quick. Yeah. So but you know like the fact is that today right you already own your device right your device is there is not doing anything right. So actually you can actually have a lot of free compute. uh that's the first thing. I think the second thing is as I mentioned there are uh applications that's require like sort of like subsecond answer and as I get you know there's just not time to to go anywhere else. I think you know like referring to to to the question you asked if if I have a serious health

47:25problem I'll go to the doctor. Okay. Um since we have about 10 minutes left um I think um we're we're gonna close the panel with um the question that's probably the most relevant to the audience we have. So if you were uh finishing your PhD and wanted to think about what kind of industry you should apply for um in order for your equity to still be worth something in the la next

47:52five years. What advice do you have? how to escape the you don't want to um I think I think as someone who did not get a PhD just to be clear um you're young so take the risk like it's okay if your equity is zero in 5 years because you're still like young whereas if you're old you know that you should be more safe but when you're young you should you know yolo risk it right maybe it'll maybe the equity will be zero maybe it'll be 100x more so you know maybe that that's that's the right way to look at it right um if you want

48:21to go work at an enterprise and have your equity go up barely over the next 5 10 years that's Great. But then you're uh as as as Ethan mentioned, stuck in the permanent underclass. He disagrees. Oh, completely disagree. You don't have five years. No, no, I mean you have you have you have five years till you get to one month on the meter plot, which is like

48:41approximately five [laughter] years. That that's literally approximately five years. So you have five years. Okay. All right. So I I I think uh um you should pay attention to uh something that will be still be right uh important in in five years time and and you should ask yourself uh what what should that be right uh even if uh we're going to like make a lot of advancement in terms of HI and whatnot what is it that's still going to be important right um let me give you some hint I think efficiency is still going to be there and the second one is safety I think the closer you get

49:17to hi the the more important safety is going to be right. Um I don't think anybody knows uh uh what's going to happen in five years, right? Um so uh I would say uh you you might want to hedge. So you could hedge across time. So you could be like, "Hey, I'm gonna uh join a big tech for a couple years and then you know, I'll be I'll be stable and then I'll I'll do a startup." Or you do startup first and then join big tech. Yeah. however, however you want to do it. You can hatch across TAM, you can hatch across, you know, if you have a partner um uh you

49:56know, one of you could go to uh big tech and one of you could do startup. I' I've seen that, right? Um more partners. [laughter] That's an option, too. Actually, diversify partners. Yes. Tell tell your kids that have a PhD. [laughter] Um I think there's a big opportunity to build more more stuff that people use. I think there's still very little penetration of uh of the stuff we already have across the world. Um like software engineering is like slowly getting more and more adoption but most other fields like think about if your tax guy is using LLMs and maybe he should be and maybe uh he'll he'll be

50:36cheaper and he'll get get your stuff back faster. uh maybe your doctor is not using LMS or maybe your doctor misses something serious. Um I hope my doctor's using GPD 5.1 uh deep think or whatever or or Gemini deep think. Um and so there there's a lot of opportunity to build like more vertical tools for for pretty much like the entire world and and very few people and very few very there's very little adoption. We're all in a bubble. We're all in Europe and most people aren't. Uh and so um I I I would try to go to build stuff that uh should

51:05exist. Uh I think I'm the token guy that is supposed to tell you to go work at a coding agent lab. Uh if you look at the open router survey that was out this week, um they they actually had this like chart which is very interesting of like frequency of use case and revenue uh or price of use case and this like the sweet spot is high frequency use case and high revenue and uh programming or coding use cases are something like three orders of magnitude higher than the next next best one. Um so uh it's a soft uh yeah software. Uh so so working you know one reasonable like I don't

51:42know what happens in five years but I know right now uh you know products are going from zero to uh 800 million or 0 to 1 billion in in less than a year and uh it's probably still going to keep going for a good amount of time. My main advice is get as close to the largest training runs as quickly as

51:59possible. Like the the other alternative is if you're trying to like maximize value then maybe do like a cursor type thing because like the people who create the value aren't necessarily the people who capture the value. So it depends on whether you're like maximizing like I'm really influencing the AI research thing or I'm like capturing all the value because they're kind of like different

52:20like things to optimize. Yeah. I mean I I'm having hard time to think about anything that cannot be automated in five years. So yeah. Well, there was a joke that maybe you suggest people study philosophy. [laughter] You could you could take 30 pictures on Instagram. Yeah. Seriously. Yeah. I need to think about what is the slowest scaling law in terms of automation for what kind of skills and suggest that but seriously like with uh computer science programs now in whatever universities it's a very good question like what they are wasting time on right so and that will be applicable to pretty much everything right obviously within five years for sure So I I'm I'm

53:16really having hard time right now. I mean I I don't know. Um so it probably some higher level thinking skills will probably still be relevant but uh what exactly should they study to have like uh because basically okay so if you can wipe code almost everything right uh to what extent you'll still need uh like

53:39deep technical knowledge. Yeah, I know. Right now you do need it because you cannot trust the thing, right? But um boy, it's a tough question. Yeah, and I know that there are some of my PhD students in this room. Sorry guys, [laughter] I guess like can you expand on the skill set then? So if not the industry they should apply it for but what skill set

54:00they should develop to stay. Yeah. So and I'm I'm trying to think like uh what should be the skill set. I mean critical thinking is probably will always be useful but specific skill sets like I don't know like uh what people used to what what they still teaching in the like I don't know software engineering courses at universities it's so like last century I think I got one it's so like people usually say like the the things that come online and get good for AI it's like follows more of x paradox so like the longer they've been around evolutionarily the later they get solved And technically the thing that's been

54:37around even before the Cambridge explosion was agency like long-term agency that lasts for like you know 60 years or whatever. So may it might be just agency is the last thing that like gets fully toppled because that's the oldest thing from like a more of x paradox perspective like even before the it was around it's been around since

54:53even before the camron explosion. Uh yeah so about agency there was a yeah the recent tweet right? So agency is more important than intelligence and yeah I think there is a good point. So I I I guess yeah. So train yourself to have more agency. [snorts] The the interesting thing is like you know if you if you truly believe in AGI then you're going to have extremely you're going to be extremely limited in

55:16where you can put those tokens to AGI. So the people who can figure out with their agency where to direct the tokens to make the most profit is actually the people who escape the permanent underclass not the you know um so I think you know there to to that line right agency is important because you need to find where you can profit maximize with the god tokens that makes sense but like overall it seems like fund study the fundamentals that's the advice arena you're giving right and um get a lot of subscriptions for chpt and stuff to ask follow-up

55:50questions and critical thinking. Critical thinking. Well, yeah, critical thinking and agency. So, basically like thinking came pretty recently from like a from like a Morox paradox perspective. From what? Agency is the thing I can think of that's like the furthest that happened the oldest from a more paradox perspective. Yeah. So, bottom line, it's kind of meta skills. Yeah. Because Well,

56:14does Dylan get what I'm saying? I'll talk about it. [laughter] I'll laugh into the mic. Is my Is my giggle that cute? [laughter] No, I don't have anything to say. I was saying critical thinking probably be toppled way before like excessively long-term agency gets toppled. Cool. And it seems like critical thinking is relevant not just the next five years, but I feel like it has been relevant and it will be relevant for the foreseeable future. I hope um I guess we can close the channel the the panel. It seems like we've been so efficient. Thank you guys for being a part of the conversation. Um and I had a

56:57lot of fun asking your questions. So, thank you for your answers. And uh for everyone in the audience, let's give it a round of applause. [applause]