No Priors Ep. 65 | With Scale AI CEO Alexandr Wang

0:05hi listeners and welcome to no priors today I'm excited to welcome Alex Wang who started scale AI as a 19-year-old College Dropout scale has since become a juggernaut in the AI industry modern AI is powered by three pillars compute data and algorithms while research labs are working on algorithms and AI chip companies are working on the compute pillar scale is the data Foundry serving almost every major llm effort including open AI meta and Microsoft this is a really special episode for me given Alex started scale in my house in 2016 and the company has come so far Alex welcome I'm so happy to be talking to you today thanks for

0:49having me know new go for uh for quite some time so excited to be on the Pod why don't we start at the beginning just for a broader audience talk a little bit about the founding story of scale right before scale I was studying Ai and machine learning at MIT and this was the year when Deep Mind came out with alphago uh where Google released tensorflow so sort of the maybe the beginning of the deep learning hype wave or hype cycle and uh I remember I was at College I was trying to to use uh neural networks I was trying to train you know image recognition uh neural networks and

1:21the thing I realized very quickly is that uh these models were very much so just a product of their data and you I sort of played this forward and thought through it and you know these models or AI in general is the product of you know three fundamental pillars there's the algorithms uh the compute and the computation power that goes into them and the data um and at that time it was clear you know there were uh there were companies working on the algorithms uh Labs like openai or Google's Labs or you know a number of of AI research efforts uh there were Nvidia was already a very

1:55clear leader in building compute for these AI systems um but there was nobody focused on data and uh it was really clear that over the long Arc of this technology data was only going to become more and more important and so in 2016 uh dropped out of MIT did YC and really started scale to solve the data Piller of the AI ecosystem and be the organization that was going to solve all the hard problems associated with how do you actually um produce and create enough data to fuel this ecosystem and really this was the start of scale as the the data Foundry for AI it's incredible uh foresight

2:34because you describe it as like the beginning of the deep learning hype cycle I don't think most people noticed that a hype cycle was yet going on um and so uh I just distinctly remember uh you know you working through a number of early use cases you know building this company in my house at the time and discovering I think far before anybody else noticed that um the AV companies were spending all of theiry money on data um how did you think about like talk a little bit about how the business has evolved since then because it's certainly not just that use case today AI is an interesting technology

3:08because it is at the at the core mathematical level such a general purpose technology it could be you know it's basically um functions that can approximate nearly any function including like intelligence uh and and so it can be applied in a very wide bread of use cases and I think one of the challenges in building in AI over the past you know we've been at it for eight years now um has really been uh what are the applications that are gaining traction and how do you build the right infrastructure to fuel those applications so um as an infrastructure provider you know we we provide the data

3:43Foundry for all these AI applications we our burden is to be thinking ahead as to where are the Breakthrough use cases in AI going to be and how do we basically lay down the tracks before the sort of you know freight train of AI comes rolling through um we you know when we got started in 2016 this was uh the very beginning of the autonomous vehicle uh sort of cycle uh it was I think right when we were doing YC was when Cruise got acquired um and it was sort of the beginning of you know the the sort of uh the wave of autonomous driving being one

4:17of the key Tech Trends and um and I think that you know we followed the early startup advice you have to focus uh early on as a company and so we we built the very first data engine that supported um sensor fuse data so support a combination of 2D data plus 3D data so Liars plus cameras that were built on onto the onto the vehicles um and then that very quickly became an industry standard across uh cross all the players uh you know working with folks like General Motors and Toyota and stantis and many others in the first few years the company were just focused on

4:49autonomous driving and a handful of other robotics use cases but that was the that was sort of the The Prime Time AI use case and then starting in about 2019 20 um it was an interesting moment where it was actually pretty unclear where the future of you know AI use cases where AI applications were going to come and this is obviously pre- language model pregenerative Ai and and it was uh a period of high uncertainty so we uh we then started focusing on government applications that was one of the areas where it was clear that there was High applicability um and it was one of the areas that was becoming more and

5:26more important um globally so we built the very first data engines to support government data um this will support uh mostly geospatial and satellite and over other overhead imagery this ended up fueling the first um AI program of record for the US dood um and and was sort of the the the start of our government business and that technology ended up being critical years later um in the in the Ukraine conflict and then also around that time was when we started working on generative AI so we partnered with open AI at that time to do the the very first experiments on rhf on top of gpd2 um this was like the the

6:05primordial days of of rhf and the models back then were uh really rudimentary like they didn't they truly it did not seem like anything to us but we were just like you know opening eye they're a bunch of smart people we should work with them we should partner with them and so um we we partnered with a team that that originally invented rhf and then we basically continued innovating with them from 2019 onwards but we didn't think that much about the the underlying technological Trend um you know they integrated this uh all of this technology into gbd3 with the there was a paper instruct GPT which is kind of

6:40the precursor to chat GPT that we worked with them on and then ultimately you know in 2022 Dolly 2 and and chat GPT rolled around and we ended up focusing a lot of our effort as a company into how do we fuel the data for generative AI how do we be the data Foundry for generative AI um and uh today you know fast forward to today uh our data Foundry fuels basically every major large language model in the industry work with open AI meta Microsoft U many of the other players partner with them very closely in fueling their AI development and in that time frame the

7:16Ambitions of AI have just you know um totally exploded I mean we've gone from you know gbd3 I think was it was a landmark model but it it was you know there was a modesty to gbd3 at the time and now you know we're looking at building you know agents and very complex reasoning capabilities multimodality multilinguality I mean the the infrastructure that we have to build to support all the directions that developers want to take this technology has been really staggering and and quite uh quite incredible yeah you've basically W uh surfed multiple waves of AI and one of the big shifts that's happening right now is there's other

7:52types of parties that are starting to engage with this technology so you're obviously now working with a lot of the technology Giants with government um with Automotive companies it seems like there's emergence now of Enterprise customers and a platform for that there's emergence of sovereign AI how are you engaging with these other massive use cases that are coming now on the generative AI side it's quite an exciting time because I think for the first time in a in maybe the entire history of ai ai truly feels like a general purpose technology which can be applied in you know a very large number of business use cases I contrast this to

8:24you know the autonomous vehicle era where it really felt like we were building a very specific use case that be very very valuable now its general purpose can be it can uh be encompassed across the the broadspan and as we think about what are the infrastructure requirements to support this broad industry and what is the what is the broad Arc of the technology um it's really one where think we think how do we Empower data abundance right um there's a there's this question that comes up a lot you know are we going to run out of tokens uh and uh and what happens when we do and I think that

8:56that's a choice I think we as an industry can either choose data abundance or data scarcity um and we view our role and our job in the ecosystem to be to build data abundance um the key to the scaling of these large language models and the you know these these uh uh language models in general is the ability to scale data and I think that one of the fundamental bottlenecks to you know what's what's in the way of us getting from gp4 to gpd1 is you know data abundance are we going to have the data to actually get there and um our goal is you know how do we how do we

9:32ensure that we have enough tokens to do that and we've sort of as a as a community we have we've had easy data which is all the data on the internet um and we've kind of exhausted all the easy data and now it's about you know forward data production uh that has high supervisory signal that is basically very valuable and we think about this as uh you know Frontier data production and the kinds of data that are really relevant and valuable to the models today there's a um you know the quality requirements have just increased dramatically it's not anymore the case that these models can learn that much

10:04more from you know various comments on Reddit or whatnot they need uh they need truly Frontier data and what does this look like this is you know reasoning chain of thoughts from the world's experts or from mathematicians or physicists or biologists or chemists or lawyers or doctors um this is Agent workflow data of agents in Enterprise use cases or in consumer use cases or uh even coding agents and and other agents like that this is multilingual data so data that encompasses the full span of you know the the many many languages that are spoken in the world this includes uh all the multimodal data to your point like you know how do we

10:44integrate video data uh audio data um you know uh start including more of the esoteric data types that exist within Enterprises and exist within a lot of industrial use cases into these models there's this very uh large mandate I think for for our industry to actually figure out what is the means of production by which we're actually going to be able to generate and produce more tokens um to fuel the future of this industry and I think there's there's a few sources um or there's a few answers to this so the first is uh we need um we need the best and brightest Minds in the

11:19world to be contributing data I think it's um one of the things I think is actually quite interesting about this technology is um you know very smart humans so phds or doctors or lawyers or exp persons in all these various Fields um actually have a can have an extremely high impact into the future of this technology by producing data that ultimately feeds into the algorithms um if you think about it's actually their work is one of the ways that they can have um a very scaled Society level impact you know there's there's an argument that you can make that um uh producing high quality data for AI

11:52systems is is near infinite impact because you know even if you improve the model just a little bit if you were to integrate that over all of the future invocations of that model that's like a ridiculous amount of impact so I think that's something that's quite exciting it's kind of interesting because Google's original Mission was to organize the world's information and make it universally accessible and useful and um they would go and they would um scan in books right from library archives and they were trying to find different ways to collect all the world's information and effectively that's what you folks are doing or helping others do right you're

12:25effectively saying where is all the expert knowledge and how do we translate that into d that can then be used by machines so that people can ultimately use that information and that's super exciting it's exciting to the contributors who are in our Network as well because I think you know that there's there's obviously um a monetary component and they're excited to to do this work but there's a there's a very meaningful motivation which is how do I leverage my expert knowledge and expert insight and use that to fuel this entire AI movement which I think is um is like a deep you know that's kind of like the

12:58deepest scient ific motivation which is how do I use my knowledge and capability and intelligence to fuel humanity and progress and knowledge um going into the future I think it's a somewhat undervalued thing where um it's going to age me but like there was a decade or so where like the biggest thing happening in technology was uh digitization of different processes and I think there's actually some belief that like oh that's happened right like you know interactions are digital and like information is captured in relational database systems on you know customers and employees or whatever but one of the big discoveries as a investor in this

13:35field over the last five years has been like the data is not actually captured yeah for almost any use case you might imagine um for AI right because I have multiple companies and I'm sure a lot does too and you and your personal investing where you know the first six months of the company is a question of where are we going to get this data you go to many of the incumbent um software and services vendors and despite having done this task you know for years they have not actually captured the information you'd want to teach a model yeah um and like that you know that

14:07knowledge capture era I think is happening in scill is a really important part to make a dune 2 analogy I mean I think it really is sp you know data production is very similar to spice production it is the it will be the the lifeblood of all the future of these AI systems um and you know so so I think best and brightest people is one key Source um proprietary data is definitely a very important source as well you know uh crazy stat but JP Morgan's proprietary data set is 150 pedabytes of data um gp4 is trained on less than one pyte of of data so there's clearly so

14:40much data that exists within Enterprises and and uh governments that is proprietary data that can be used for training um incredibly powerful AI systems and then I think there's this key question of what's the what's the future of synthetic data and how synthetic data needs to emerge and and our perspective is that the critical thing is is what we call hybrid human AI synthetic data so how can you build hybrid human AI systems such that uh AI are doing a lot of the heavy lifting but human experts and people you know the basically best and brightest the the smartest people the sort of best at reasoning can contribute all of their

15:19insight and capability to ensure that you produce data that's of extremely high quality of High Fidelity to ultimately fuel the the future of these models I want to pull this right a little bit because something you and I were talking about both in the context of data collection and evals it's like what do you do when the models are actually quite good right better than humans on many measured Dimensions um and so like can you talk about that um from both the data and perhaps um you know we should talk about evaluation as well I mean I think philosophically the question is not is a model better than a

15:49human unassisted from from a model the question is is a human plus a model together when people produce better output than a model alone and I think that'll be the case for a very very very long time that that humans are still you know human intelligence is complimentary to machine intelligence that we're building and they're going to be able to combine to build you know to do things that are uh strictly better than what the models are going to be able to do on their own I have this optimism lot and I had a debate at one point that was um challenging for me uh philosophically

16:20about whether or not Centaur play or like machine and human intelligence were complimentary my simple case for this is when we look at the machine intelligence like the models that are produced you know we always you know you you see things that are really weird you know there's like the the rot 13 versus rot 8 thing for example where the models know how to do R 13 they don't know how to do rot 8 um there's the reversal curse you know there's all these these artifacts that indicate somehow that it is not like human intelligence or not like biological intelligence and I think that's a that's the bull case for

16:50Humanity which is that you know there are certain qualities and attributes of human intelligence which are somehow distinct from the very separate and very different process by which we're training these algorithms um and so then I think you know what what does this look like in practice it's you know if a model produces an answer or response how can a human critique that response to improve it how can a human expert you know highlight whether there's factuality errors or whether there's reasoning errors to improve the quality of it how can the human Aid in guiding the model over like a long period of time to produce

17:23reasoning chains that are very um that are very uh correct and deep and and are able to drive you know the capability of these models forward and so I think there's a lot that goes into this is what we spend all of our time thinking about what is the human Expert Plus uh plus model teaming that's going to help us keep pushing the boundary of what the models are capable of doing how long do you think human expertise continues to play a role in that so if I look at certain models Med Palm 2 would be a good example where Google released a model where they showed that um the

17:52model output was better than the average physician you could still get better output from a cardiologist but if you just ask to GP a Cardiology question the model would do better as ranked by physician experts um so it showed that already for certain types of capabilities the model provided better insights or output than people who were trained to do some some aspects of that how far do you think that goes in terms of or when do you think human expertise no longer is additive to these models is that never is it three years from now I'm sort of curious of the time frame I think it's never because I think that um

18:26you know the the key quality of human uh intelligence or biological intelligence is this ability to reason and optimize over very long time Horizons so and this is biological right because our our goals as as biological entities is to optimize over you know our lifetimes optimize for reproduction Etc so we have the ability as as human intelligences to produce long-term goals continue optimizing adjusting um and reasoning over very long very long time Horizons you know current models don't have this capability because the models are trained on these like little nuggets of human intelligence so they're they're tra you know they're very good at like almost like a a uh like a shot glass

19:09full of human intelligence but they're very bad at continuing that intelligence over a long time period or long time Horizon and so this this fundamental quality of biological intelligence I think is something that will only be taught to the model over time through you know uh through a direct transfer via data um to to fuel these models you don't think there's a like a um architectural breakthrough in planning that solves it I think there will be architectural breakthroughs that improve performance dramatically but I think if you think about it inherently like these models are not trained to optimize over longtime Horizons in any way and we

19:45don't have the environments to be able to to get them to optimize for these like you know Amorphis goals over longtime Horizons so I think this is a somewhat um fundamental uh limitation before we like talk about you know some of the cool releases you have coming out and what's next for scale um maybe we can uh zoom out and just congratulate you on the fundraise that you guys just did um a billion dollars at almost 14 billion in valuation um with uh you know really interesting investors AMD Cisco meta I want to hear a little bit about the strategics our mission is to serve the entire AI ecosystem and the broader

20:22AI industry you know we're an infrastructure provider that's our role is to be um as much as possible supporting the entire industry to flourish uh as as much as possible and we thought an important part of that was um how can we uh be an important part of the ecosystem and build as much ecosystem around this data Foundry which is going to fuel the future of the industry um as much as possible which is one of the reasons why we wanted to bring along a other infrastructure providers like Intel and AMD and folks who are also laying the groundwork for the future of the technology um but also

20:55you know key players in the industry like uh like meta um folks like Cisco as well you know our view is that ultimately there's there's the stack that we think about the tech there's the infrastructure there's the technology and there's the application and our goal as much as possible is how do we um how do we leverage this this data capability this data Foundry to empower every layer of that stack as much as possible um and and build an a broader industry um Viewpoint around what's needed for the future of data I mean I think that this is an exciting moment for us I mean we

21:28see our role uh you know going back to the framing of what's holding us back from gbd1 what's what's in the way from gbd4 to gb10 um we want to be investing into actually enabling that pretty incredible uh technology journey and you know there's there's tens of billions maybe hundreds of billions of dollars investment going into the compute side of this equation and one of the reasons why we thought it was important to uh to raise the money and continue investing is you know there's real investment that's going to have to be made into the data production to get us there with great power comes great responsibility

22:02um if uh you know if these AI systems are what we think they are in terms of societal impact like trust in those systems is a crucial question like how do you guys think about this as part of your work at scale a lot of what we think about is how do we utilize how does the data Foundry um enhance the entire AI life cycle right and that life cycle goes from you know a ensuring that there's data abundance as well as data quality going into the systems but also being able to measure the AI systems which builds confidence in in Ai and also enables F further development and

22:34further adoption of the technology and this is this is the fundamental Loop that I think every AI company goes through you know they they get a bunch of data or they generate a bunch of data they train their models they evaluate those systems and they sort of you know uh go again in the loop and so evaluation and measurement of the AI systems is a critical component of the life cycle but also a critical component I think of of society being build trust in these systems you know how are governments going to know that these AI systems are are safe and secure and fit for uh you know broader adoption within

23:07their countries how do How are um Enterprises going to know that when they deploy an AI agent or an AI system that it's actually going to be good for the consumers and it's not going to create greater risk for them how do um how are Labs going to be able to consistently measure what are the intelligences of my of the AI systems that we build and how are we going to you know how do they make sure they continue to develop responsibly as a result can you give our listeners a little bit of intuition for like what makes eals hard one of the hard things that you know because we're

23:34building systems that we're trying to approximate and and build human intelligence um grading one of these AI systems is is not something that's very easy to do automatically and it's it's sort of like um you know you have to kind of build IQ tests for these models which in of itself is a very fraught philosophical question is like how do you measure the intelligence of a system and this is there's very practical problems as well so most of the benchmarks that we as a community look at for the academic benchmarks yeah the academic benchmarks that are what the industry used to measure the performance of these algorithms are Frau with issues

24:10many of the models are overfit on these benchmarks they're sort of in the training data sets of these models um and so you guys just did some interesting research here publish them yep so we one of the things we did is we published GSM 1K which was a uh a heldout eval so we basically produced a new um evaluation of the math cap abilities of models that there's no way it would ever exist in the in the training data set to really see how much of the per how were the performance of the models um uh what the reported performance of the model capability versus the actual capability and what

24:42you notice is some of the models perform really well but some of them perform much worse than the reported performance and so this whole question of how we as a site are actually going to measure these models is is a really tough one and our answer is we have to leverage the same human experts and the kind of the best and brightest Minds to do expert evaluations on top of these models to understand you know where are they powerful where are they weak and and what's the sort of um what are the sort of risks associated with these with these models so you know one of the

25:11things that um we're very you know we're going to uh we're very passionate about is there needs to be sort of public visibility and transparency into the performance of these models so there need to be leaderboards there need to be evaluations that are public that demonstrate uh in a in a very rigorous scientific way what the performance of the these models are and then we need to build the platforms and capabilities for governments Enterprises labs to be able to do constant evaluation on top of these models to ensure that we're always developing the technology in a safe way and we're always deploying it in a safe

25:41way um so this is something that we think is you know just in the same way that our role as an infrastructure provider is to support the data needs for the entire ecosystem we think that building this layer of confidence in the systems through accurate measurement is going to be fundamental to the further op option or for their development of Technology you want to talk about state of AI at the application ler because you have a uh Viewpoint into that that very few people do you know after gbd4 launched there was sort of this this frenzy of of sort of an application build out and um and I think that there

26:17was you know there were all these like agent companies excitement around agents there was all these like you know a lot of applications that were buil out and I actually think it's it's an interesting um moment in the in the in the the life cycle of AI which is that you know gp4 I think was as a model was a little early of a technology for us to have this entire hype wave around and I think we you know the community very quickly discovered all the limitations of gbd4 but you know we all know gbd4 is not the terminal model that that AI that we are

26:47going to be using there are better models on the way and so I think there was a there's an element by which you know it's sort of a classic hype cycle gbd4 came out lots of hypeon building um applications around gbd4 um but it was it was probably a few Generations too early of a model to for the thousand flowers to bloom and so I think in the coming models we're going to see um we're going to this this sort of like trough of disillusionment I think we're going to come out of because the next the future models are going to be so much more powerful and you're actually

27:15going to have all of the fundamental capabilities you need to build agents or all sorts of incredible things on top of it and we think what what we're very passionate about is how do we Empower um application Builders so whether that be Enterprises or governments or startups to have um to build self-improvement into the applications that they build so uh what we see from the large Labs um like openi and others is that self-improvement comes from data fly wheels so how do you have a flywheel by which you're constantly you know getting new data that improves the your model you're constantly evaluating that system to understand where there's weaknesses

27:53and you're you're sort of like continually hydrating this this workflow um we think that fundamentally every Enterprise or government or startup um is going to need to build applications that that have the self-improvement loop and cycle and it's very hard to build and so you know we built this product our gen platform to really to really build you know lay the ground work and the platform to enable the entire ecosystem to be able to build these self improvement Loops into their into their products um as well as possible I I was just curious I mean one thing related to that is you mentioned that for example

28:26JP Morgan has 150 pedabytes some data relative you know that's 150 times what um some early GPT models trained on how do you work with Enterprises around those Loops or what are the types of customer needs that you're seeing right now or application areas one of the things that every you know all the model developers understand well but the Enterprise understand super well is that um you know not all data is created equal and high quality data or Frontier data is is can be you know 10,000 times more valuable than just any run-of-the-mill data uh within an Enterprise and so a lot of the challenge in you know or a lot of the problems

29:02that we solve with Enterprises are how do you go from this Giant mountain of data that sort of is like all over this truly all over the place um and distributed everywhere within the Enterprise to what are the how do you compress that down and filter it down to the high quality data that you can actually use to you know fine-tune or train um or continue to enhance these models to actually drive differentiated performance I think one of the things that's interesting is that there's some papers out of meta which basically shows that uh actually narrowing the amount of data that you use creates better models

29:35um so the output is better the models are smaller which means they're cheaper to run they're faster to run and so to your point it's really interesting because a lot of people are sitting on these massive data sets and they think all that data is really important and it sounds like you're really really working with Enterprises to sort of narrow that down into what's the data that actually improve the model it's information uh Theory question in some sense um what are some of the launches that are coming coming from scale now you know we're building evaluations uh for the ecosystem so one is that we're going to

30:02launch um uh these private heldout evaluations and have leaderboards associated with these evals um for the leading llms uh in the ecosystem and we're going to rerun this contest uh periodically so every few months we're going to do a new set of held out evals to basically consistently Benchmark and monitor the performance of of our models and continue adding more domain so we're going to start with areas like math and coding instruction following um adversar capabilities and then we're going to over time continue you know increasing the number of of areas that we test these models on we think about kind of like a like an Olympics for llms but

30:38instead of every four years it'll be every few months um uh so that's one thing we're quite excited about um and then uh we have an exciting launch coming with uh some of uh with our government uh customers so uh one of the things that uh we see with with in the government space is they're trying to use um llms and they're trying to use these capabilities is there's actually a lot of uh there's a lot of cases where um even the current agentic capabilities of the models can be extremely valuable to the to the government um and it's often in in pretty boring use cases like

31:11writing reports or filling out forms or pulling information one place to another but it's well within the capabilities of these models um and so we're going to be we're excited about launching uh some agentic features for our for our government customers in uh with our Donovan product these are applications you bu build yourselves or an application building framework so for our government customers we basically build a like a AI staff officer so it's a it's a full application but it integrates with whatever model our customers think is appropriate for their use case and do you think scale will invest in that for Enterprise applications in the future our view for

31:46Enterprises is is fundamentally like how do we how do we for the applications that enterprises are going to build how do we help them build uh self-improvement into those products so we think about much more at the platform level for Enterprises does the new open AI or Google uh release uh change your point of view on anything fundamentally multimodality um you know the applicability of voice agents Etc you know I think you tweeted about this but one one very interesting element is uh the the direction that we're going in terms of consumer focus and it's it's fascinating I mean I think multimodality well taking a step back first off I

32:26think it points to where there's where there's still huge data needs so multimodality as an entire space is one where um for the same reasons that we've like exhausted a lot the internet data there's a lot of scarcity for good multim modal data that can Empower these personal agents and these personal use cases so I think there's um you know as we want to keep improving these systems and improving these personal agent use cases there's you know we think about this a lot what are the data needs that are actually um that are going to be required to actually fuel that um I think the other thing that's fascinating

32:57is is um the convergence actually so both Labs have been working um independently on on various Technologies and you know Astra which was Google's uh major sort of uh hubcap release as well as 40 you know they're both shockingly similar um and uh sort of uh you know demonstrations of the technology and so there's I think that was that was very fascinating the lab are sort of converging on the same end use cases or the same Visionary use cases for the technology I think there's two reads of that one is like there is an obvious technical Next Step here um and very smart people have independently arrived

33:34and the other is like competitive intelligence is pretty good yeah I think both are probably true I think both are true it's funny because when I used to work on products at Google um we'd spend two years working on something and then the week of launch somebody else would come out with something and we launch it and then people would claim that we copied them and so I do think a lot of this stuff just happens to be in some cases just where the whole industry is heading and it's kind of people are aware that multimodality is one of the really big areas um and a lot of these things are years

33:59of work going into it so it's kind of interesting to watch it as an external Observer yeah yeah I mean this is also not a like a a training run that is a onewe copy effort right and well and then I think the last thing that is um that you know I've been thinking a lot about is like when are we going to get smarter models so you know we got multimodality capability that's exciting it's more of a lateral expansion of the models and the the industry needs smarter models we need gp5 or we need g92 or whatever that those models are going to be um and so to me it was you

34:31know uh I was somewhat disappointed because I just want much smarter models that are going to enable kind of as we mentioned before you know way more applications to be built on top of them the year is long end of here um okay so quickfire and Al on chime in if you um have ones here uh um something you believe about AI that other people don't my biggest belief here is that the the path to AGI is uh is one that looks a lot more like curing cancer than uh than developing a vaccine and what I mean by that is I think that the the path to

35:06build AI is going to be um in in you know you're going to have to solve a bunch of small problems that where you don't get that much positive leverage between um solving one problem to solving the next problem and there's just sort of you know it's like curing cancer which is you have to then zoom into each individual cancer and solve them independently and eventually over a multi- deade time frame we're going to look back can realize that we've we've you know built hii we've cured cancer but the the path to get there will be this like you know quite plotting Road of of solving individual capabilities

35:37and building individual sort of um data fly wheels to support this end mission whereas I think a lot of people in Industry paint the path to AGI as like you know eventually we'll just Boop we'll get there we'll like you know we'll like uh we'll solve it uh in one Fell Swoop and um I think this has a lot of implications for how you actually think about you know the technology Arc and and and how the how Society is going to have to deal with it I think it's actually pretty bullish case for society adapting the technology because I think it's going to be you know consistent

36:08slow progress for quite some time and Society will have time to fully sort of uh uh acclimate to the technology that develops when you say solve like a problem at a time right if we just like pull away from the analogy a little bit should uh should I think of that as um generality of multi-step reasoning is really hard as you know Monte col research is not the answer that people think it might be um we're just going to run into scaling walls like what sort of what are the dimensions of like solving multiple problems I think the main thing fundamentally is I think there's there's

36:40very limited generality that we get from these models um and even for multimodality for example uh my understanding there's no positive transfer from learning in one modality to other modality so like training off of a bunch of video doesn't really help you that much with your text problems and vice versa and so I think what this means is like each um sort of uh each niche of capabilities or each area of capability is going to require separate flywheels data flywheels to be able to to push through and drive performance you don't yet believe Nvidia as basis for World model that helps I think reason I think it's

37:13great narrative I don't think there's strong scientific evidence of that yet maybe there will be eventually um but I think that this is the uh I think the base case let's say is one where you know there's not that much generalization coming out of the models and so we actually just need to slowly solve lots and lots of little problems to ultimately result in AGI one last question for for you is like you know leader of scale a scaling organization like what are you thinking about as a as a CEO and this we almost sound cliche but just how early we are in this in this technology I mean I think that

37:46there's um you know it's it's strange because on the one hand it feels like we're so late because the tech Giants are investing so much and there's a bajillion launches all the time and uh there's you know uh there's there's all sorts of investment into the space but markets look crowded in the obvious use cases yeah exactly markets look super crowded but I think fundamentally we're still super early because the technology is you know one 100th or one 1,000th of its future capability and as we as a community and as an industry and as a society ride that wave it's just going to be you know there's so many more

38:20chapters of the book and so as a you know you think about any organization what we think about a lot is is nimbleness like how do we ensure that as this technology continues to develop that we're able to continue um adapting alongside uh the development of the technology great that's a great place to end thanks so much for joining us today yeah thanks Alex thank you find us on Twitter at no prior pod subscribe to our YouTube channel if you want to see our faces follow the show on Apple podcast Spotify or wherever you listen that way you get a new episode every week and sign up for emails or find transcripts

38:54for every episode at no- pri.com