Alex Wang: Why Data Not Compute is the Bottleneck to Foundation Model Performance | E1164

0:00at its core this AI technology has the potential to be one of the greatest military assets that Humanity has ever seen potentially even more of a military asset than nukes let's say China or Russia had AGI today and the United States didn't I would imagine they would use that to conquer the ccp's system is incredibly good at taking very aggressive centralized action and centralized industrial policy to drive forward critical Industries they have a clear shot at racing forward ready to [Music]

0:44go Alex I'm thrilled that we could do this in person thank you so much for joining me today yeah great to be here now listen it's funny I told you I tweeted before like we should skip the founding stories because there are many many great times you've told it before but I want to dive straight in and I want to ask you the question of when we look at model performance today let's just start high level do you think we're seeing a case of diminishing returns where more compute doesn't lead to better performance yeah I think it's pretty fascinating I mean I think there's been um this especially coming

1:15up now where gbd4 uh you know opening eye has had gbd4 since fall of 2022 uh and since that time frame we haven't yet seen a new base model or new a new model that's you know um jaw droppingly better than gbd4 you know we haven't seen the gbd 4.5 or the gbd 5 or you know the other labs haven't yet come out with models that are you know leagues and leagues better than gp4 despite just like way way more compute expenditure you know since since when cat GPT came out you know you can look at the graph of nvidia's Revenue it just inflect it's just like it just goes straight up after

1:54GPT uh 4 came out and it goes from you know I think the nvidia's data center Revenue were doing uh roughly about 5 billion a quarter and then it shoots up to now it's you know north of 20 billion a quarter so there's been you know tens of billions going to 100 more than a 100 billion of spend on uh high-end Nvidia gpus all in the same time frame we haven't yet seen the big breakthrough since gbd4 which actually that model was came out before this huge inflection in Nvidia um expenditure so uh I think overall it's very it's this interesting thing we're we're seeing investment into

2:32compute go up dramatically go up exponentially right now but we're still I think as a community as an industry kind of waiting for the next great model do you think we've reached this kind of ASM toot of performance where actually we'll see this kind of plateauing in performance while we wait for that and do we think that's like a monthly thing or do we think that's kind of like self-driving remember self-driving we saw kind of the plateau in performance actually for kind of several years and actually it was only recently where we see that inflate again it's kind of this interesting thing so there's there's three ingredients that go into um these

3:03AI models or three pillars so there's compute of course there's data and there's the algorithms and the history of AI is that uh progress comes from sort of you know uh all three of these pillar sort of being built alog together so you know you need you certainly need a lot of computational capability but you need the algorithmic advances algorithmic advances like the transformer originally or rhf or you know whatever future algorithmic advances come um and then you need you need the data pillar to support it as well and I think a lot of the plateau that we've recently seen can almost be explained at a very high level from um

3:39hitting kind of a data wall so gbd4 was was a model basically trained on nearly all of the internet uh and using a huge amount of computational capabilities and a lot of the a lot of I think what the industry has been doing over the past few years is scaling the computation dramatically um but not necessarily by building up the other two pillars um in tandem so there needs to be I think a combination of more algorithmic Improvement but in particular we need to we need to ensure that there's more data to support it when you say data wall what is the data wall and what can we do

4:11to enable our overcoming of it yeah so at a at a super um super high level um I think we've we've used up all the easy data we've used up all of the internet data um you know we the uh the common crawl and and uh newer versions of the common crawl just so we understand easy data is stuff on social media anything not behind pay walls anything that's easy and free to Cru anything that's easy and fore to crawl or stuff that can be torrented there's a lot of you know reports that there's like a lot of torrented data in some of these models or you know basically anything that has

4:45that is sort of like already written down and easy to get from the open internet and then um and so you know the the the first stage of a lot of this AI Improvement has been this these advances in pre-training which is basically training these models to um to be really really good at emulating the internet and right now we're at a point where like these models are are exceptionally good at emulating the internet they're better than any human at at emulating the internet but the problem is uh what when we think of AGI when we think of powerful AI systems we want much more than just emulating the

5:19internet you want AI systems that can do tasks you want AI systems that can solve difficult problems you want AI systems that humans can collaborate with to to solve all their daily problems and this this process of building agents and AI uh AI models that are capable of all these things you know we're not going to get there from internet data and we've already used up all the internet data so what we need is why are we not going to get there from internet data when we think about effective agents and when we think about software doing the work not just selling the tools as I think Sarah

5:49tampel said quite well before why is existing data uh not equipped to do that transition from tools to work yeah so the um the the simple answer is like a lot of the uh a lot of the thought process and a lot of the thinking that humans go through when they are doing more complex tasks that doesn't get written down on the internet so for example if I'm a fraud analyst um inside a a large Bank you know I go through my job is like understanding with based on a set of transactions that seems suspicious um whether or not it's a fraudulent uh it's a fraudulent

6:27transaction and I need to analyze all sorts of different pieces of data and use my deduction and use all my like sort of uh human intelligence to make that decision and that process that I go through it's not like every like I'm writing down step by step like oh I looked at this piece of data and I looked at this piece of data and then based on that I deduce this and I'm not writing all that down on the internet to later be crawled uh by these models so one way to think about is like all of the reasoning and thinking that is powering the economy today none of that

6:58gets written down on the internet and so if you just train on the internet the model has no has no ability to learn from all of that so how do we codify and capture the data that's not codified already as you said there with the fraud analyst the thought process the analysis the discussion that goes on in internal meetings that's not codified in data sets how do we capture that to enable us to do the work our thesis or what what I really believe is what we need from now forward is is Frontier data and we need to we need to basically have uh data abundance of Frontier data we're right

7:29now we're sort of data scarcity mindset or or we hitting a data wall and this Frontier data is exactly what we're talking about Frontier data in my mind is you know complex reasoning chains complex discussion um you know uh reasoning chains of models going agent chains of models going and looking up a piece of data doing some reasoning looking up another piece of data um maybe correcting if it has an error um tool use you know all of the all of the key components that we would think of of of an agent being able to do that all needs to be encapsuled into the Frontier data to power the forward capabilities

8:02of these models how do we capture that data yeah I and I think that it needs to be built through basically a a combination of there's basically three pillars so first is uh there's a lot of this data that's locked up in uh the world's Enterprises today and so um and none of that gets on the internet for very good reasons but just to give a sense of scale you know um JP Morgan's uh proprietary internal data set is 150 pedabytes um the gp4 was trained on an internet data set that was less than one pyte so the amount of data that exists inside large Enterprises is just

8:39absolutely astronomical so there's one process of just sort of mining all this existing Enterprise data for all the goodness that exists within it but you would never get that open sourced would you so this is all proprietary and then delivered custom to that customer exactly this has to be a process of like every Enterprise like I have uh I have a set of very important problems for my Enterprise then I need to go through the process of like basically mining all my existing data and refining all that existing data for use for AI systems to solve my own problems when we think about kind of breakthroughs we said said

9:10about diminishing returns at the beginning you know I spoke to one of the most powerful CTO in the world the other day and they said the real breakthrough in kind of this question of all we reaching diminishing returns is like whether we can really solve reasoning like how do you think about our ability to solve reasoning and that impact of data that you mentioned there in helping us navigate that you know if you look at what these models can do they're very good at reasoning in situations where they've seen a lot of data before you know I think we like to think about these AIS as uh as if they're like

9:40little human intelligences but they're they're very different human intelligence and machine intelligence are very different humans are very good at sort of um humans have a very general form of intelligence they have this great ability you know uh if a human if a if a kid is raised in a very small neighborhood um they can live their whole lives in that small neighborhood and they can go to an entirely different part of the world and they can navigate understand what's going on no AI system today would be able to do that level of sort of uh you know drag and drop in one situation to another situation and

10:12figure out what's going on so um so we have to I think we have to be cognizant that that's a limitation but what that means is that you know for any situation that we want these models to perform well in we need to have data of that situation or that scenario and actually the model will perform really well so there's kind of two ways to think about um resolving the reasoning Gap that exists in these current models one is uh obviously you build some sort of General reasoning capability which would definitely be a big breakthrough for sure the other one is just it's like it's a data problem it's like you need

10:45data for every scenario where you want these models to reason well in you just need to overwhelm them with data in all those scenarios and you're going to get models that can reason really well how do we move from an environment of data scity to abundance when we appreciate the immense amounts of data that say JP Morgan or govern has or any large Enterprise has but also the proprietary nature of that which won't actually go to generalized models which will help the world or Humanity or any of these kind of breakthroughs actually occur to everyone else how do we move from that data scarcity to data abundance is it

11:15synthetic data that we create how do we think about that yeah so I think the second part is to your point um new data that has to be produced yeah and we need uh we need the means of production of New Frontier data to get to get us from you know gp4 to gpd1 and I think this is a very natural you know I think when we think about chips the this is very natural which is oh yeah we need to build more and more Fabs we need to build bigger Fabs we need to build uh we need to like increase the the uh resolution and give lower lower

11:44nanometer Fabs like I think we think about for compute it's very natural for us to think about increasing the means of production but I think we don't think about this with data and I think we need to do something very similar and this process of of producing data you know it's it's a combin it's sort of a hybrid uh human synthetic process and that's really how we think about it which is you need you need algorithms that can do a lot of the heavy lifting in producing uh synthetic data but you need human experts who are going to be able to guide the AI systems and basically um

12:15help provide input as to you know when the AI system gets stuck or when they have a factuality issue or when it's in a situation where it hasn't encountered before I mean one another way to think about is we kind of need the a lot of autonomous vehicle scaleup has been through through uh these uh these Safety Drivers you know you have Safety Drivers inside the car and when the car starts screwing up you have the safety driver disengage and sort of uh take over and you need that kind of setup for these AI systems you need um the AI Sy you need AI models to be generating uh large

12:45amounts of data and then humans who can kind of take over and and nudge the models when necessary to make sure you get really high quality data what does that look like in the structure of organizations today like do we create new roles for these AI uh prompt not prompters but um kind of Savers yeah trainers is one term AI trainers or or uh or contributors is another term I think this process of of contributing data to AI is actually one of the uh highest leverage uh jobs that humans can have and the reason for that is um you know uh if let's say I'm a

13:24mathematician I can either go into my hole and you know do pure math uh and try to do uh try to do pure math research and there's like one that's one trajectory for my life the other trajectory is I use all my skills and talents and intelligence to help make these AI models smarter and even if I make let's say I make GPD 4 just like a little bit smarter on math if I um take that little bit of improvement of the model and I sum that up across all the times that gp4 is going to be called and used across every you know every math

13:57student who's going to use gp4 every company that's going to use gp4 um every developer that's going to use gp4 that's a huge amount of impact and so as a as a human expert you have the ability to have society-wide impact by producing data to help improve these models and I think that's a you know what we see is for you know scientists mathematicians doctors you know uh people who are you know human experts in the world it's an incredibly exciting proposition to be able to you know I can transmit my capabilities intelligence training all of that into a model that's going to be able to have society-wide impact I mean

14:35it's an incredibly exciting proposition how do we think about the structure of data often people talk about the biggest challenge in in kind of you know data governance is actually just like the structure and cleanliness of it when we look at the 150 pedabytes of you know JP Morgan data I have no idea but I presume it's not structured perfectly for a lot of models to ingest efficiently how do we think about the structuring of this huge data set that I'm sure all large Enterprises have and the challenge that that poses there's two parallel efforts one is is mining existing data which um by all means it's going to be a onetime

15:08hit there's going to be a onetime benefit that you get from mining all existing and it could be really meaningful do you think in 5 years time everyone will have mind their largest data sources internally I don't think everyone will but certainly the most sophisticated companies will and then and then we'll be at a point where we still need to make the models better and so it all will at the end of the day it'll all boil down to production and and uh effectively like you know what are the what are the means of forward production um in the same way that you know you need the means of Ford

15:37production for chips and and all the other things that you care about okay so we have that in terms of mining existing you said there was another form so there's data mining uh and then there's there's forward data production um these are the two core um core directions for where we need this data to come from and I think you know kind of taking a a broader step back I think that um a lot of AI progress at this point is fundamentally more data bottlenecked such that I think that like if we were able to produce compute and data in lock step with one another so as Nvidia

16:11continue to manufacture you know uh uh hundreds of billions dollars worth more of chips if we were able to produce a proportional amount of data as we got more and more chips and we were able to produce these two together then we would just get um we would get astronomically more capable models just so I understand so when we think about increasing the supply side of data like what is the literal ways that we can do that well like what comes to my mind is actually Dan soker at like Limitless I don't know if you know down at Limitless he was found of optimizely but he basically has

16:41like this new hardware device which records like every single thing that you say and do and it produces your own personal AI because it has everything that you've ever said in the day that that is a new form of data Creation in my mind how do we increase the supply side of data there's probably two main pieces one is to like like this uh effort from Limitless or other efforts which is basically um much more longitudinal data collection collecting more of what's naturally happening in the world so this is you know there's a bunch of forms of this so one is like in a workplace um I think you're going to

17:13want uh some kind of you know as creepy as it sounds some kind of constant data collection of you know what apps are using what order apps are using you know where do you copy paste one thing to another you have a lot of this with RPA and a lot of UI pass flows so exactly quite used to that yeah yeah yeah so so process mining which is one of the terms um in SAS but basically like the continued um collection of uh of existing Enterprise processes then there's the consumer version of that which looks maybe like kind of what you're referencing or you know maybe

17:43it's with the meta Rayband uh collaboration or you know whatever whatever device ultimately does it but sort of something that collects you know the the longitudinal view of your own life and then there has to be um this sort of uh a real a real in investment towards what I you know kind of um uh human experts collaborating with models to produce Frontier data so both of the things I referred to before both you know enterprise process Mining and um you know for lack of a better term sort of like uh consumer data collection those are all going to produce valuable data sets but they're not going to

18:17produce the data that's actually going to push the models forward because to push the models forward we need you know uh you need really highly complex data that's going to be able to push the frontiers of what the models can do so these are this is where need the agentic behavior this is where you need the complex reasoning chains this is where you need Advanced code data or maybe Advanced physics or biology or chemistry data these are the these are the things that are really needed to push the the boundaries of the models and that's a I think this is a global kind of infrastructure level effort that needs

18:46to happen like I think we need to think about it as you know how do we get the world's experts to collaborate with the models to help produce AI systems that are going to be you know the world's best scientists or the world's best Co or mathematicians when we think about the commoditization of the models that's everyone says that we have how do we think about like proprietary access to these data sources you people have said to me before that like you know I don't mean to throw shade but like open AI models are not necessarily better they've just had better access to data they've bought

19:15more data whatever whatever but data being the central kind of superiority element of why they had better performance in the past like will we see one model get access that others don't how do we think about like fair and Equitable access to data from the models slide well I actually think to your point if you think about the competitive playing field of these different model providers against one another um uh they have you know data is one of the few uh you know pillar there's three pillars right there's algorithms compute and data um and data I think is actually the primary pillar that you could imagine uh

19:52a real comp durable competitive Advantage emerging right so so if you think about where are there Moes in this llm race or where their Moes in this Foundation model game I think data is one of the few areas where you can produce a sustainable Mo because the issue is algorithms that's IP that at some point you know the rest of the industry will learn about um compute is uh you know you can have more comput than other people but other people can just spend more money and buy that compute and data is one of the few areas where you can actually produce a a long-term sustainable competive

20:22Advantage I agree when you look at some of open ai's agreements they obviously partner up with the ft to get access to all of the ft's Historical Library and done quite a few actually with axle Springer I think like that is access that a lot of other models do not have which will make their content Superior in whatever queries they have in that respect yeah exactly and I think this is the it's the start of this um I think this form of thinking of sort of um data as a mode right and so you know these are these you know there's the Ft there's aqual Springer these are the in

20:49these are the first um indications of this but in the future I think you know these labs are going to be thinking a lot about okay what are the what's the data that I'm going to use to differentiate relative to my competitors and how I'm going to produce that data and what is the um what is the long-term drial Advantage created by that and so I think you're going to see you know I actually expect that you know everything we've been talking around data around model commoditization you know we're going to see companies start building uh data strategies that that drive more differentiation in the market over time

21:22and I think it's I think it's pretty um it's pretty exciting I mean another way to think about this is you know right now uh if you you know in San Francisco the big you know researchers and and the big CEOs brag about how many gpus they have you know they're the sort of um the biggest indicator of how serious they are about AI is is how many gpus they have um but I think in the future they're going to brag about you know what data they have access to and what are their you know how much data they're producing and what are their sort of

21:50unique rights to to different data sources and I think that's actually going to be the primary plane of of competition in the future versus just okay Jensen's giving me you know however many hundreds of thousands of gpus given data strategy being a potential element that one could win on and compete on in different ways do you think we will not see the commoditization of these models over time there's two Futures one is that even data strategy becomes something that very quickly um commoditized and and different PE and different Labs sort of uh copy one another or they all they all end up converging to the same direction 100% CU

22:28especially with a lot of the content producers they're not going to do exclusive agreements with one model and not other models yeah so so then I think it comes down to like they like different Labs need to have strategies to produce their unique data sets so let's say I'm uh you know anthropic for example has has has focused a lot on Enterprise use cases and maybe they need to develop a data strategy that enables them to have a very differentiated access to new data to support those Enterprise use cases or maybe um open AI with with chat gpg needs to develop you know a unique data strategy

23:03that lets them leverage the fact that they have all these users and you know all this all this reach but I think that like the the various labs are going to need to lean into uh where they're going to be able to get you know proprietor and differentiated data going into the future do you think we're going to see a reversion back to on Prem I'm I'm jumping around so much but I'm loving this conversation sorry for that but like when we think about the like 150 pedabytes of JP more data I don't know if they're going to be like yeah I'll throw it all in the cloud my most

23:29sensitive data and I think will we see the reversion back to on Prem and models that work on Prem for these large Enterprises I think when we talk to these large Enterprises and and the leaders Within These Enterprises you know they are um they are very quickly realizing this fact that you stumbled on which is that um you know their data their Enterprise data might be their only competitive differentiator in an AI world and so um they extremely extremely cautious about you know if they do a deal where all their data somehow you know a model developer gets access to it or you know they share it in some way

24:08then you know they could be um mortgaging away their entire future and so I think they're they're very very cautious about that and this is actually why I think there's a there's a very um there's a very big uh sort of opportunity for whether it's open source models or or the Llama models or the mistol models or whatnot basically these models that can go on Prem and that enterprises can take and then customize on top of their own data and then never has to go back to a model developer or cloud or anything like that I think that there's a huge um sort of unmet need

24:39there and I think that's actually where most serious Enterprises are going to go towards which is my data I need really really strong guarantees that my data is not going to be used in any way to improve my competitors I think AI services will actually create more Revenue over the next 5 years than AI models we will actually exent it come out with I think it was 2.4 billion dollar in revenue from generative Ai and open a was obiously $2 billion how do you think about I'm just intrigued with scale AI today and working with some of the largest Enterprises a Services component the learning and adoption

25:08curve is challenging for large Enterprises do you see that as a core part of your business in the next few years as we scale the education curve yeah I mean I think that um first of all I think you're right I think that the um there's so much value to be generated from AI for sure but then there's this there's this very natural question of uh where's the value capture going to be right and you know this fascinating thing you know um if you go back and read high output management by Andy Grove there's like there's these chapters around like oh you know for Intel like hey we thought that first we

25:45thought that like this is where the value capture was going to be but then we realized it was going to be in this other part of the stack and so we had to migrate to that part of the stack and we had to migrate again and it's this incredible case study you know I remember reading that and uh I read it maybe a decade ago in a in a different era of tech and I was like this is weird this doesn't feel very relevant and then now in AI you're you're seeing it once again where it's I think you know it's so new and so nent where exactly where

26:08in the stack value will acre feels like it's constantly moving and I agree with you I think that the the models themselves um there's so much competition there um that I don't know if I don't know how much value acrs at at literally the model itself but the everything above the model and everything below the model I I feel very confident there there will be value occurring so for the infrastructure I mean Nvidia is the biggest company built on AI today like they're they're the third most valuable company in the world Nvidia is more valuable you know their market cap is higher than meta and Google and Amazon uh and Saudia ramco I

26:45mean it's it's really stag like Nvidia is an incredible incredible company and that's kind of below the model and then above the model you have all these apps and these services that are going to be built on top of it so I was arguing with someone this morning actually on the way here though and I saying like yes okay we have notion Ai and we have you know box the storage company who are going to implement AI Solutions into their existing storage product so you can extract information better yeah have you seen the numbers Salesforce are now growing at single digits like are now growing at single digits point being

27:16actually the commoditization of these features means it'll be better products for us but I don't know if you'll get value extraction from that in the form of increased pricing how do you feel about that yeah so our thesis on this is um you know there's this uh there was this um article that that flew around the end of software right uh I saw this Chris Pike you're from Chris Pike yeah and it was kind of it was an intentionally provocative point of view I think but uh I do think there is a version of this which is true which is sorry for those that haven't read it

27:47what was the cool premise just so they understand it yeah so so he basically I thought it was like a brilliant comparison but he basically Drew this comparison of um software companies today to Media companies pre- social media the rough comparison was you know uh in the days of you know in the in the uh older days of media you had all these incredible media companies they were there were uh these high-end shops where there were like all these experts producing this this uh this very differentiated content but then it got disrupted by social media and the internet broadly because all of a sudden you just had um you know as the content

28:25creation costs came down or more more the content distribution cost came down dramatically the world of media consumption turned into this like very broad constellation uh where you would consume whatever media was produced by anybody that was interesting to you and it was sort of like much more On Demand versus you know being the sort of Walled Garden of of large media producers and basically this comparison to that's what's about to happen to software which is you know now that you're the Enterprise is live with this Walled Garden of some small number of software providers uh and what's going to happen now with G Ai and all these other Trends

28:59is they're going to have this constellation of all these you know uh different apps and point Solutions and this sort of portal to that constellation of of various uh software providers and this sort of like we're going to move from this current world of like smaller number of of Wal gard and SAS apps to this sort of much more decentralized uh Universe do you agree with that it's intentionally provocative right uh but I think one thing that that is true is I do think that enterprises and and the World At Large are going to demand greater levels of customization and you know they're going to demand um

29:39greater levels of personalization and stuff that is that is really purpose-built for their business like a glove I think that like the beginning of this you know the first tech company that ever uh did something in this direction was paler um and you know they got a bad RP for a for a long long time because you know the pal everyone thought the paler oh they're just a consulting company um but paler point of view which is provocative as well was like no what we're going to do is we're going to go into Enterprises we're going to understand exactly what their problems are and we're going to help

30:11them build the perfect application for them that's built on top that like connects all their data and all that stuff and then um and if we can do that then we're going to build something that's that's far more valuable for them than what any other software provider is going to be able to produce and they did this you know obviously before generative AI before all these you know tools that are going to make this this motion a lot more um feasible but I do think there's an element that this is like the way the world is moving which is now especially that the software production cost and software creation

30:42costs are going down so dramatically we're going to end up moving towards a world where um more and more of the software that enterprises consume are going to be customized and custom built and purpose built for exactly their problems what does that mean in terms of the makeup event engineering teams of large Enterprises do they shrink do they focus on different things do we just have teams of the world's best prompters what does that mean in terms of the changing structure of engineering teams yeah well I think software engineering in general is um is going to change dramatically a lot of what developers spend a lot of time on today they will

31:17not need to spend time on going into the future as the models get better and better at coding but there's certainly big parts of what they do which are irreplaceable and over time I think that the the part in particular that's really um that's very very valuable is this is this sort of like General process of going from what are the what are my customer problems or what are the sort of like problems I need to solve and like translating those into engineering problems and scoped sort of like um you know tickets almost that can be solved by an AI engineer everyone says that we're going to see the end of P pricing

31:56like he said you know Chris had that provocative article but like everyone talks about the end of per seat pricing to what extent do you think we will see the end of per seat pricing in this next wave of software and especially with the data lens where you could see a more consumption based pricing model aligned do you think that truly takes over the reason that Percy pricing um uh doesn't make sense going to the future is that at an Enterprise uh today certainly most of the productive work is done by is done by their employees done by people but in a future where you imagine more and more

32:30of the work is done by AI agents or AI models um then per pricing doesn't really makes sense because you want you know as a as a provider of software provider of of solutions you want to make sure that you're capturing the value that you're providing to the people but also the value that you know your agents or your you know your AI systems are producing and so I think that that shifts a lot of the world towards consumption based pricing um versus per seed one of my biggest worries is obviously we're in London we specialize in in many things here at long lunch breaks and regulation um and

33:04so diminishing on London um but my question to you is I really worry that we're going to see regulatory Provisions which stifle Innovation because of consumer data protection acts and just unnecessary regulation around data access do you think I am justified and how do you navigate the regulatory access to data question it's a really important point and I think that that certainly um what we've seen in the EU uh is is a very restrictive approach to data my personal belief I don't think that uh more permissive uh uh regulations around data are at odds with being a liberal democracy more sort of liberal uh data data access uh

33:52Provisions are are in fact very very compatible with being a liberal democracy and I think that that we as a society need to figure out what the right balance there is and and how we sort of square the circle um but uh but I think I think this is a very important question because it's almost like I think in in the United States there's been a huge amount of of effort and and uh real regulatory effort on terms of how do we ensure that we do not slow down chip production and how do we ensure you know how do we make sure that you know um that we can keep

34:27Manufacturing huge amounts of chips and the US won't be disadvantaged from that perspective we need to take a similar lens to data so how do we from from a policy standpoint both in the US and in the UK frankly um how do we think about ensuring that uh as countries we are not going to we're not holding uh we're not tying one hand behind our backs for future data production for these models do you think the US is currently tying one hand behind its back in terms of that we're definitely not taking a pro dat um regulatory what would a pro dat regulatory St look like I think there's

34:59a few things I think first there's um there's large data sets that are not um do not lend proprietary advantages to specific players that need to be sort of centralized and and made accessible to whole Industries so simple examples um safety data um uh in in let's say Aerospace which is a which is a Hot Topic obviously um but safety D data in Aerospace should be collectively pooled for the purpose of advancing the entire industry forward or the data I mentioned before or the the example I mentioned before fraud uh and compliance in financial services should be pulled together and and should be you know uh

35:41should build forward capabilities um so I think there's like entire IND like industrial sectors where there should be some degree of data pooling to just push forward the overall industry and I think what you need is um in in a lot of consumer facing uh areas you need we need to work through a lot of the existing restrictions to make sure that those don't prevent AI progress so so one great example here is actually uh hipa in healthcare so um and and all the pii and other other uh limitations right now um uh you know Hippa and pi and all the pii regulations will more or less

36:23prevent uh patient data from being used to train um you know AI models but I think we can agree as a as a as a civilization as a human race we really want to learn from all of the existing medical data on how do we cure human diseases going forward and so we need to figure out like how are we going to sort of um uh make it so that like there's very clear nymz Provisions or there's a there's a very clear and obvious way in which you can use existing patient data to improve future health outcomes totally agree I I heard actually that China apparently I can't remember who

36:56said this on the show but said they're like two years behind the US in terms of AI progress I heard that and I thought that is absolute uh and I think when you look at like data provisions and what the Chinese government will be willing to do in terms of data access and data provisions and regulation I I think if they are two years behind that will very quickly catch up how do you see China being two years behind and do you agree with that two years ago there were probably more than two years behind so um when you know when opening ey first produced gp4 in the lab uh you

37:30know China were nowhere near that but just even the past few months uh there's a Chinese company 0101 uh that produced uh a model ye large uh y yi- large that is now the uh one of the best models in the world I think it's just behind so it's behind GPD 40 and uh Gemini Claude 3 opus and it's the next model right behind that in the leaderboards so it's one of the best models in the world um so we've already seen them meaningfully catch up they are like you know Chinese Chinese llm and AI capabilities are I would say right now pretty close to neck and neck with us

38:13capabilities and I think if you plot the path ahead based on everything we've talked about with data I mean I think they're going to they have a clear shot at racing forward and racing ahead of us and I think it it comes down to at its core the the the CCP system is incredibly good at um at taking uh very aggressive centralized action and centralized industrial policy to drive forward critical Industries and what we've seen even in the past few years on or the past few decades frankly on solar um uh how China has or the CCP has been able to make um uh take industrial

38:56policy to the point of like being by and large the world's leader and solar and then most recently EVS um uh and how you know uh the CCP system and approach has been able to create you know very very cheap EVS you're seeing this pattern play out over and over again where um you know the the CCP approach to Industrial policy is not the most Innovative but once an industry has been established and it's about you know turning the crank they are better at turning the crank than any other economy in the world agree with you I saw actually a chart I can't remember who tweeted it yesterday but I think it was

39:31I think it was either Elon or Bill Amman and it was basically showed countries different kind of creation of EV providers and it show like the us and it was like the US I mean without Tessa it would have been in the dumps because it would have only had in General Motors but China was like up and to the right for sure does that worry you it worries me a lot yeah and I think that the um you know one of the elephant in the room topics which I think you know is an AI Community we rarely disc discuss is that um at its core this AI technology uh has

40:03the potential to be one of the greatest military assets that Humanity has ever seen you know if you imagine a let's say you had AGI and you have one country with AGI and another country without AGI you know which one will win in a war well probably the one with AGI is going to like figure out all you know how to produce all the weapons or we'll figure out a brilliant military strategy or we'll be able to hack the other country systems and it is Po potentially one of the greatest military assets that the world's ever seen potentially even more of a military asset than nukes right and

40:34so um and uh if you think about this you know we're in a a geopolitical environment that is uh increasingly tense you know the amount of conflict in the world is is has been Monon increasing the past few decades um the uh you're seeing these these uh multiple wars being fought in the world um and some of them without very clear path to resolution and you have there's uh there are totalitarian leaders right now in the world you know many of them with for whom like you know let's say China or Russia uh had AGI today and the United States didn't um you know I would I would

41:15imagine uh they would use that to to um to conquer that's a really scary outcome for the world r large and I think it's one that you know we the Western world needs to spend a lot of our thought and effort towards preventing that outcome given that concern should we not have closed systems obviously open systems have a lot of benefits but the challenge with open systems is anyone can use them and that means that Russia can use them China can use them and everyone has same levels of access well supposedly so should we not have closed systems with what you just said I think there's a bit

41:50of a of a dichotomy that must emerge there's sort of I think we need to think about the most cutting Edge and the most advanced systems um those we will want to ensure are closed for GE political reasons for military reasons for whatever reasons like as we develop systems that are that are genuinely so so powerful we will'll want to keep those closed that that doesn't preclude us from you know making open less Advanced versions of the technology that frankly just have the ability to produce a lot of economic value um and I think that's where we are with l right now like I don't think

42:28llama um that llama 3 models are so Advanced that you would think like llama 3 in and of itself is not a military asset yet um and I think that there's we're there's clearly a line underneath which I think it's totally fine to have open models um and I think we so that's what we need to be thoughtful about is where that line is and when are we getting close to it before we discuss some kind of company building principles which I do want to touch on 10 years time what does that Foundation model there look like who's independent who's been acquired what does it look like I

43:00think at its core um what we've seen about the foundation model race is that it is uh incredibly incredibly expensive um and it is expensive to the level of you know these malls have gone from costing hundreds of millions to a billion dollars to M maybe multiple billions I think in 10 years time maybe they'll cost tens or hundreds of billions and so there's just not very many entities that have that much discretionary Capital to invest into um these AI models so naturally what will happen over time is AI effort the the the foundation mod efforts will coales around you know uh Nations or the large

43:44tech companies over time and so you know the um basically you will see the most you know all these hyper profitable business models whether that's a nation state or one of the hyperscalers will those will be the only entities that could possibly subsidize or or or underwrite these massive AI programs and so I think in the future it looks like you know it is a Battle of Giants already it looks like a Battle of Giants but but at that point it's even more a Battle of Giants so do you agree with me in saying that you'll see all of the smaller players acquired by the large

44:19Cloud providers your Google your Amazon your Nvidia you you name your large incumbents but especially the large Cloud providers and have them integrated into their existing solution uh yes within maybe an asterisk that there's some of these Partnerships that um I think it'll be interesting to see how they play out you know the the opening ey Microsoft partnership or the anthropic uh Amazon partnership you know I think it'll be one of the most interesting questions of this technology era is how do these Partnerships actually end up playing out long term listen I do want to touch on some company building principles let's start with uh I can't remember the exact you

44:54said on PR it was a brilliant statement this was is it which is the best PR is no PR what did you mean Alex the traditional press industry um is is not particularly conducive to Great companies being built and let me be more specific around that I think that the um a lot of traditional press you know is oriented around generating clicks and so the traditional press engine will it'll Build You Up and it'll generate clicks on the way up and it'll tear you down and generate clicks on the way down and this is in contrast to I think 20 VC and other uh sort of direct Outlets so to

45:30speak where I think there's a um where Founders and companies have a direct channel to get their message out and and explain uh what they're working on well I think the other thing and I I actually it's a little bit unfair I feel for traditional media I don't care about clicks like yeah we have sponsors respectfully if we didn't have them we'd still be doing the show I don't do sensationalist headlines I'm not going to put some glossy thing scale AI predicts military Devastation with this episode cuz I'm not there to just optimize for clicks exactly yeah you're you're there to um to genuinely educate and and

46:06explain what's going on to to your audience and I think there's you know it's almost unfur can you imagine if someone said hey I'm going to do scale AI but I don't care if we lose money You' be like oh how do I compete with that yeah but but I think it's um it's pretty Stark you know I've received more fair treatment testifying in front of Congress than I have from uh various media Outlets over the years and um it's it it it feels like this totally ridiculous statement but I think we're in this perverse state of of a lot of traditional media where um the the

46:41system itself you know because of this sort of like very click oriented approach versus a genuine educational approach you know it it um it almost has no way of being uh you know fully fair to the companies and so I think it's the the imperative is on the companies themselves to properly tell their story through direct channels and through podcasts and through um Avenues where their their message won't be won't be altered I I completely I think this is why found a brand today is more important than ever because if you don't own your means of distribution it will be contorted exactly and I think that's

47:18the kind of a shocking state of the world but I think it's has that changed your strategy then yeah I think I think for us we think a lot about how do we how do we get the direct message out there and how do we develop to your point like what are what are the purest ways that we can transmit and explain what we're doing and this is this is a great example you know you'll ask me a question I will answer exactly how I believe that you know exactly the answer that I think and this will go out to your to your listeners and your viewers

47:44and it's sort of the one of the purest forms of getting the message out there I think one thing that people make a big mistake on there is that them try and build the direct channels for the companies and respectfully people don't follow scale people follow Alex it's much easier to build following with personalities and it is companies I think there's like so few companies that can you know open AI one where like I think open eyes and enity has like a lot of as a lot of meaning as a brand but um very it does but if you look at the amount of times that Sam ultman Trends

48:13versus the amount of time that open AI Trends it is disproportionately higher to Sam ultman people still now more than ever love The Cult of Personality yeah that's a fascinating thing I mean that definitely should be uh and that transcends actually when you look at Lionel Messi Miami when you look at margar Robbie with Barbie the celebritization of individuals in organizations or in movements drives everything that's fascinating I mean it it it it uh it probably speaks to a deep a deep human need to to relate you know I think I think we as people we have a lot of circuitry to understand individuals um and we we have this

48:48ability to understand individuals it's very hard to understand what an organization means there's no there's no intuitive so should Founders give a about traditional PR should they care about getting in the traditional press I would argue we're in an era now where they they shouldn't um and they should think about uh they should think about what is a what is an interesting point of view they can have and what are the what is the most pure way to get that point of view across when do you feel the pastro to tear you down unfairly so I I would say that like uh almost almost precisely we've had this this this uh

49:22this story where you know we had an incredible rise up and an incredible come up maybe when we had our initial we initially became a unicorn back in 2019 um and uh for the few years after that you know it it felt like Smooth Sailing and then starting in about um 2022 right when it was the entire Tech narrative was or the entire media narrative let's say was tearing down tech companies because you know it was I mean in some ways it was very fair you know all of many many tech companies received very high valuations there was uh there was an incredible um amount of excitement in

50:00Tech and then the markets all crashed and that's that in starting in 2022 was when I noticed um for us specifically the tone entirely shifted where the media engine pointed itself towards you know pointing out all the missteps from companies like us or a lot of our peers versus uh versus trying to point you know trying to take a balanced perspective and there were even you know another example of this is so starting in about um 20 20 we we began working with the US Military and the US DOD um and this was you know this is obviously long before the current defense Tech um hype wave and and long before all that

50:38but it was it was driven by an intrinsic belief that we had as that I had and we had as a company that it's important for the United States DOD to have access to uh incredible uh AI technology that was like a fundamentally important thing for the future of the world and um and in the Years after that you know uh I would say by and large the traditional media engine actually tore us down for supporting the US government and supporting the military versus I think taking a broader view that you know maybe this was maybe this was a positive thing actually to to support um support

51:14the US Military and this goes to this almost goes to what I was saying about the um about you know the the the dichotomy and treatment and you know testifying in Congress versus uh versus um with the media you know I test ified before Congress about um ai's use in the military and I would say my the the treatment I got there I think was was a properly broad one it was hey obviously this is powerful technology that we need to be thoughtful of but it is so important that America leads in this and you know thank you for for everything that you're doing that was that I felt

51:46was the response whereas in the media it's this incredibly scornful uh perspective around oh like you know is this a good thing do we trust this company like what is mean I mean it's it's this shocking but I think it goes back to the incentives Drive outcomes and what is the incentives of media versus the incentives of Congress Congress are not there to sell you or clicks they're there to hopefully get to an informed decision on the best outcome exactly yeah on the incentives Drive outcomes I I loved something you also said you said why hiring people who give a is harder than it sounds what do you mean and how do you

52:21think about that when hiring you know it sounds so simple when you when you really boil it down down but if you hire people who you know we say give a internally but who really really care you know they really really care about the the their work product they really really care about the quality of their work they really really really care about the organization they care about making sure that the company has an impact you know they just they just really care and what that means how that manifests is you know they're willing to sweat every single detail and if they get roadblocked or there's like

52:51something in their way they'll spend the EXT they'll go the extra mile to make sure that they get through those things you know that ends up being that's that's how startups work fundamentally is that you have these small teams of people who each care 10 times more than the than the average employee 10 or 100 times more than the average employee inside a big company and so they end up you know you end up just solving so many more problems than um than the big company how many people do you have in scale today we are about 800 people 800 people you are now getting to the kind

53:21of bigger company size it is harder you know the kind of only hire a plus players a players a players by definition a r can you have 800 a players I think the answer is yes and I think that um you know what we say a lot internally is how do we hire the Navy Seals not not the Navy not there's anything wrong with the Navy but how do you have you know a really small Elite group um that is really uh where you're really hiring the cream of the crop and this goes down to process you know for us um at this point in the company still I approve every

53:54hire so I will will look at the I will either directly interview or look at the interview feedback and look at the um understand every single person who we hire to ensure that we're keeping an exceptionally High bar and that way what percent of the time will you go against the recommendation of the team on a new hire I would say maybe on average 25 to 30% like a lot like a lot and I think usually it's due to you know maybe there's a new hiring manager um who you know needs to get calibrated or you know it's it's or it's like an edge case of

54:29various forms but to me the way I think about this is like I as the founder of the company I have seen you know everybody who's come in and I've seen who succeeds and who fails and I've I have like I have as an algorithm almost like develop the most fine grain data set of understanding you know what it looks like for people to be successful at scale and what it looks like to have the Navy Seals versus the Navy and it's my job as a Founder to help ensure that we as an organization um are actually utilizing all this knowledge and utilizing all this learning that's been happening over the

55:02past years in the organization and carrying that forward final one what was your biggest like management or leadership up so like for an example for mine I'm like people act out of fear or Freedom you know when you bring someone in some people act out of like you have to perform you have to perform and other people act out of hey I trust you I respect you do your best work and you just have to identify which Camp someone ones in and then hopefully if their skills are there they should operate to their best I wish I'd known that when I started and I didn't and I

55:33just tried to act out of fear for everyone there what do you know now that you wish You' known and where did you up the biggest one was um was actually you know in the same era of like 2020 2021 was thinking that hypergrowth as a company meant that you had to hyper grow your team and so in those few years we we did what a lot of tech companies did we like doubled tripled the team year on year like the the you know from uh in 20120 we were about 150 people by the end of 2022 uh we were over 700 and it was just this it

56:13was this insane amount of hiring and this incredible amount of of of hypergrowth as a team and what I found out is when you hire that quickly it is impossible to to do what we've just been talking about which is maintain M this High bar and maintaining this this sort of feeling of Excellence within the team did you see the reduction of that bar in real time it felt it was kind of subtle it was something where um you know you you would hire all these people in and then um and you notice it like maybe like the next year or the next six six months later you would notice it slowly

56:47and that the organization you know there were there were challenges that the organization used to be able to to deal with and solve that slowly just um calci ified and we weren't able to we weren't able to get around and so you'll notice you know from the end of 22 you know where I said we were 700 people to now we're 800 people the team has mostly kept the same size and I think what we've really thought about is like how do we you know how do we move towards but the company the revenue of the company's grown dramatically it's funny companies have like brand inflection

57:17points they go hot they go cold they go hot again do you know what I mean and it feels like from the outside scales hot again do you know what I mean well that's uh I didn't mean that like to be super nice or or or not nice I didn't mean it when saying you're cold but it's just weird how brands have moments of heat and not heat this is a fascinating thing actually from um you know I I actually asked uh Patrick hon this question as well and stripe obviously is an incredible company that has uh that for a lot of their lifetime I think has

57:49been one of the iconic um Silicon Valley companies and I asked him whether or not he thought that the fact that they were such an iconic company um was beneficial in all the hiring they did and he made an interesting point which was that um and hopefully he's okay with me sharing this but like he made an interesting point that actually you know the um the best people they hired he thinks would have been people who would have joined whether or not they were the hottest company in Silicon Valley um you know it was the it was the sort of like off the beaten path people

58:25who um who are actually the the best hires they could have gotten and A lot of the people who joined because they were the hottest company in Silicon Valley you know for one reason or another weren't necessarily the the the most valuable employees and so there's this there's this element where I think the common belief and the common narrative is like you know you want to be the hottest company so you can attract the best talent so you can hyper grow so you can then go keep growing and I think that's often so so difficult and it's much more about like how do you develop an ecosystem of talent that is

58:55that is like very self-preserving it keeps a very high bar and always seeks out and searches for the best people and then independent of whether the company's Hot or Not hot because you will have to your point you'll have moments where you're hot moments where you're not hot moments where hot not hot and you need that Talent ecosystem to be self-preserving independent of that um to to to drive the best out I also think it depends on function like when you look at a lot of go to market functions like traditionally for sales they do like concentrate towards hotter Brands and actually if you can get a

59:26concentration of incredible sales people especially as you ban geographies I think of like open AI is kind of good to Market team in London unbelievably good one of the best in London and it's because they have an amazing brand do you see what I mean so it depends like how close you are to the nucleus and what function you're in yeah I think that's I think that's right yeah yeah but then if you look at if you look at a lot of the core technical development at open AI a lot of that is still driven 100% ear the valley and all as close as could be well not even that it's it's

59:52these people who have been at opening icense before they became the hotest company ever and so it's it's one of these things where I think there's you know another company um that I think experienced us or went through this um is uh is Airbnb Brian chesky right and I think that you know he's talked about this publicly after the pandemic he all of a sudden realized like hey I have to he had to kind of like rebuild the entire company and he massively Shrunk the team he invested a lot more into Talent density um he and he sort of built it and then he built the team to

1:00:24to remain small and I think that are even now uh the most or one of the most profitable companies per head uh in in all of Tech and that's because of this sort of this realization that he had that you know he didn't need to keep growing that team to see the financial Gams or the financial output Leen I want to do a quick fire so I'm going to say a short statement you give me your immediate thoughts does that sound okay yes let's do it okay so what have you changed your mind on most in the last 12 months I think it's actually everything about this hypergrowth stuff that we've

1:00:52been talking about and it's really around divorcing team hypergrowth from company hypergrowth and and extra investing into quality and and Excellence what's the biggest misconception you hear most often about AI I think the biggest one today is that uh all that's between us and ai's compute and uh and I think we need data to get there too tell me you can have any board member in the world who you don't currently have you have an amazing board but who you don't currently have who would you choose as your next Bard member obviously I don't think this is practical but I do think sa nadela has has been one of the most brilliant

1:01:35business strategists of the modern era I think what what he has accomplished at uh at Microsoft is just staggering and uh and I think any board would be very lucky to have him unfair one for me to ask but I actually like it which is what question are you not asked or are you never asked that you feel you should be the interesting one is like how my perspective on AI has changed in the successive eras and I mention this because um you know I started the company in 2016 uh we the first three years of the company were just full focused on autonomous driving and autonomous

1:02:14vehicles and so we started working on AI and then in 2019 we actually started working on generative AI we start working with open AI on gbt2 and so we we are one of the few AI compan that I think has seen multiple eras of the technology and has seen the sort of um at least the first boom and bus cycle with with uh with autonomous vehicles and I I think it's a it's an interesting one which is like what's the same in in these successive areas and what's different how's your view changed are you most excited now I'm quite excited um but I think there's also there's also reason reasons

1:02:55to be cous and I think that one of the things that happened in the autonomous vehicle craze is that there were a lot of promises that were being made that were divorced from the technical reality and so a lot of the prominent auton vehicle companies a lot of the prominent um organizations were making Bolder and Bolder promises to be able to raise money um and those were you know at first they weren't super divorced but over time they became more and more divorced from the tech realities and that resulted in this very painful trough where it sort of the the promises uh weren't met and so and

1:03:35so it felt like the entire industry was falling apart and actually at the end of the day you know now we have Whos driving ground San Francisco we have L you know perfectly proper L4 autonomous vehicles driving around um Tesla autopilot has gone really good and so if we had more measured promises along the way I think now we would feel amazing about autonomous vehicles whereas instead we went through this huge up this big down and maybe it's sort of like on the upswing again I think this is one of the big concerns I have about generative AI which is uh I hope not but the same thing might

1:04:11happen again which is that we have we have these really big promises that are starting to that get made um about the technology that get divorced from technical reality and then that creates the sort of Gap that is bound to cause a hangle over Will trump win an ultimate one um I still think it's a tossup actually I think that the um this is you know us elections are so strange to think about because it always gets decided by the swing States and frankly I don't trust anyone on the coasts to have and I'm one of these people on the coast I live in San Francisco to have

1:04:51any fine grained understanding of how the swing States will play out and so I have no idea I don't think anybody should listen to anybody who lives on the coast to understand what's going to happen and I think it uh it always boils down to the swing States final one for you my friend what is scale in 10 years time you know hopefully doing something very similar to what we're doing now which is continuing to to be the data Foundry for AI and serve the data pillar for uh for AI progress you know one thing I think a lot about is would you like to go public

1:05:23uh for sure yeah well one thing I think a lot about is how do you solve problemss that will never go out of style but like would you like to be the CEO of a public company uh do you know what I mean like you look at the CIS you're like I don't know why you would if you a stripe well there there's clear benefits to being a public company for sure but I think uh stripe is a is an incredible company and that they they can be incredibly profitable and um and uh so they can they can accomplish all their core financial goals without need

1:05:52to go public listen Alex I loved having you on the show thank you so much much for joining me as I said it's so much nicer to do this in person I'm sorry for the many kind of Meandering pivots and turns but this was fantastic yeah this was a lot of fun