Stop building AI wrappers. Start builing agent harnesses.

0:00Today we're talking with John Nurona, co-founder and chief product officer of Gamma, and we're going to talk about agent harnesses, all the opportunities around it, vertical agent harnesses. Uh, I can't wait to dive in, and I want to give a special shout out and thank you to Anthropic, the team behind Claude, for sponsoring today's episode.

0:26John, thank you so much for coming on the podcast. By the end of this episode, what are people going to learn? Thanks for having me, Greg. I'm really excited to be here today. We're going to dive deep on what it means to actually build an agent in practice, including demystifying what is an agent harness and what that looks like for us at Gamma, which is a creative agent harness, an agent for visual

0:45communication. And not just you're you're being humble. Gamma is I mean it's it's grown extremely quick. I think it's hit 100 million ARR. You've just relaunched the platform. Huge huge release. And there's a lot you're learning. And what I hope to get out of this podcast is really just little insights around opportunities in agent harnesses, what you're learning in real time, um, and other opportunities that exist, startup ideas and stuff like that. So, John, can you commit that by the end of this episode I'm going to learn some of those things? I'm in. Let's do it. Let's make

1:19it happen. So, I want to start off by like back here you can see what is an agent harness because I think a lot of people, you know, scroll X, they see agent harness, but they don't really understand what it is. So, can you clearly explain what it is? Absolutely. Let's break it down into the two words, agent and harness. And I'll use maybe Gamma's journey to illustrate both. So, for everyone's context, Gamma is a tool primarily for making presentations, also documents, websites,

1:45social graphics, that sort of thing. We've actually been around for several years, long before agents were ever a thing. We started the company in 2020, really took off in 2023, and early on we were just making these one-off LLM calls to do things like write a single slide or make the outline for a deck when we very manually strung them together into these preset workflows to do things like

2:04generate a presentation with 10 slides. The big revolution that's happened in the last year or two is this agent revolution where the most powerful AI products now are not stringing together these individual LLM calls by hand, but they're actually having one higher level loop of an LLM run that will do some kind of work, inspect it, use tools, and then turn that into something over a whole series of executed calls. And so we actually went through this whole journey over the course of this year of

2:31rebuilding our product to be an agent. And building your product to be an agent involves building a harness. So the harness is all the tools that agent uses to execute things in its environment, inspect its own work, and deliver something finished to the user, including soliciting from the user what their actual preferences are. So probably the most famous agent harness is something like cloud code. Cloud code is a harness for programming. So cloud code teaches the agent how to go through all the files on your system and see what's there. teaches it how to like run tests um and even prompts it with certain uh rules like go into plan mode

3:06in certain cases or run these tests after completing execution or do this stuff on GitHub. In our case, Gamma is not a coding agent. It's a creative agent. So, it needs to do things like understand your art style and creative preferences, help you craft a story, and we care a lot about visual design. We're doing things like generating images and laying them out. So, we need to know how

3:26they look and make sure they look right. So, our harness takes all of those things and turns them into tools that an agent can use. And I can talk through what that look like. Yeah, I mean, please do. And I think uh also I'm I'm curious if you agree or disagree with me. I've been saying agent harnesses are the new GPT rappers or new rappers, right? Um is that is that just a good tweet or is it uh is there some

3:49truth to it? I think there's truth to it. Although I always uh disputed this concept with the GPT wrapper originally because it sort of implies that 90% of the work is the LLM and then there's just this thin layer on top which is the wrapping back and forth. I think maybe what harnesses illustrate is that actually the wrapping is most of the work. Um that you actually have all these LLMs which are quite similar in their core capabilities. Not to say they don't have some variation but generally all the magic of turning them into a product is building the harness around them. And even the model companies like OpenAI and

4:18Anthropic are putting so much of their work into harness development as well. So they are themselves doing wrapping of these things to achieve different capabilities. Could you walk me through what happens like in 30 seconds after someone types a prompt on gamma? Like how many models does it touch like realistically you know because I think a lot of people are are are not sure h how that how that

4:40actually works. Let me walk you through it. Let's go through a sample prop. So very typical use case for gamma is someone will paste in a bunch of messy notes or a document or PDF and say help me turn this into a 20 slide presentation. So the moment you put that prompt in um a whole bunch of

4:53things actually happen under the hood. The first thing is we actually spin up an entire computer in the cloud. This is what they call an agent sandbox. This is what the opening eye ones broke out of famously in the last few months. So we actually spin up this entire sandbox where the agent gets all these files representing the user's prompt, the file they brought in, also other context about them. Um, we then prompt the agent to go look through all that file and their prompt and run certain tasks. And for us, actually one of the first tasks is usually to figure out what other information we need from the user. Um,

5:22and so typically we will actually pop up a form and ask a few questions. Uh, questions like who's this actually for? You know, is this like an educational presentation for school? Is this for uh adults? Is this for your work thing? We will actually call out often to different connectors. So we've built integrations into tools like notion or slack or linear where we can actually pull in all this relevant context. Just already in that first maybe 10 seconds we've now made calls using probably three or four different models including expensive and cheap ones. We'll talk about actually how we manage costs as well just to sort of source information

5:55and ask a few questions. We've rendered a UI around it um with some generative UI to actually show you what those choices are. Then we're going to actually write an outline for your deck. That's usually several more LLM calls and we will style a template for you. So we have a library of templates with different visual styles. We're going to use vision models, try to figure out which one's a good fit for you. Um we're going to do some cool search embedding things to sort of look at the space of thousands of templates and find the right one and even customize it if you said, "Yeah, but I want it red or blue

6:24or whatever it is." Then we're going to once again show that to you and let you edit it and inspect it. We haven't even started making the deck yet. Uh, and already we've probably made something like 30 or 40 different LLM calls to maybe four or five different models doing different tasks. Uh, now you're going to hit that generate button and we are going to build up a presentation for you before your eyes. And there's actually a lot of pieces involved in doing that. We have to write the words on the slide. We have to lay out all the things. Um, in our case, we generate a

6:53lot of imagery as well, whether decorative or more substantive. We're often making diagrams or infographics or charts. So actually all of those are now more calls that are going out to different models um often in parallel. There's a huge performance consideration because if we did these one by one, it's going to be quite slow and you're going to basically leave before we ever finish. So we do a lot of work to make it all pretty speedy. And then in the space of about a minute, you're going to actually see that whole presentation stream in front of you and generate. And now we've made probably close to 100

7:23different model calls. And I would say probably 10 to 15 different AI models have been used somewhere along the way. And we're still not done because you probably won't stick with the first draft. Um, in fact, we don't stick with the first draft. After we generate that first presentation, we're now going to have a AI vision model look at every single slide and look for problems that it can find. Misaligned text. The

7:45person's head got cut off in the image. Uh, there's weird contrast. Something's not padded right. It's actually amazing how many things can go wrong in something like a simple presentation that a human eye can detect right away. Um, or sometimes a human eye can't detect because it takes a designer's eye. So, not everyone can, but we can train models that can catch that. So, we go through this whole round of visual QA. We actually will do multiple rounds of fixing things based on what we find in that QA and then redo them. Um, and then after all those rounds of fixing, we will finally serve it up to you. And

8:14now at this point, we've probably used 20 different models making like maybe 200 different calls along the way to deliver what looks to you like one finished output. So what you've just described to me is like a workflow in a particular niche. In this case, it's, you know, creative output. Totally. You started with presentations. You've now moved, you know, it's essentially a

8:38creative suite. Yep. you know, where do you see opportunities to create agent harnesses outside? You know, if you were starting a company uh tomorrow, um where do you see an opportunity to create a vertical agent harness? What's what's an example of a niche that you know you see underexplored? That's an interesting one. I mean, you know, just leaning on my experience, I think where we see a lot of value of making our own distinct agent harness is when that problem looks very different from just plain old coding. In our case, what makes it different is that it's very subjective. There's a lot of judgment calls to make along the way

9:12where two different people will disagree about what good looks like, which means you want to have the human in the loop. And also where the person using it actually cares about the specifics of the output. And so I would think about maybe those two traits as something that go into like, wow, these are a good case for making your own agent harness. So just thinking aloud, some examples are like I know a lot of people are making games with uh cloud code and I'm really bullish on this idea of like the personal video game where you almost like vibe code it yourself. Um coding agents are already really good at making

9:44video games, but I think they're often not good at actually sourcing the players opinions on where it should go and where it should evolve. And if you think about, you know, things like Roblox or Minecraft, there's all these things where people are actually shaping the world for themselves. And usually it's like nontechnical people who are not coders themselves. I wonder if there's a harness to be built there where in fact the whole harness runs in

10:05like a 3D world or something. I think people should really think broadly and try to you know in general my philosophy of business is try to be different. Try to do something that looks nothing like what everyone else is making. And for us that's been a challenge because we try to be different but then others you know copy us or do the same thing. And we have to keep looking for more and more different. I think that's, you know, it's not even just for you guys. It's anyone who's

10:27building anything in software today. Yeah. As soon as you catch lightning in a bottle. Totally. Uh people are copying you. But I think the thing that people I think the reality is they're copying where you were six, you know, 3 months ago, 6 months ago, right? So you you as the founder have to be like, okay, where do I what's my strong point of view? What's my thesis on where the world is going? And how do I essentially create like a a product studio where um you know every two weeks you know I'm building a feature 0ero to one exploring a ter you know uh a new piece of this

11:07puzzle. I think that's totally right and I think for anyone thinking about building their own agent harness the really important question to ask is what's my edge going to be? And there's a couple different types of edge you can have. One type of edge is what you just said which is you have a distinct vision of the world and you are just moving faster towards it and it's not the one that others are focused on. Um I think another edge is

11:28you have some kind of proprietary data. You've got some information that other people don't maybe because you already have a successful platform or something. I think a totally different edge too is that it's not just software because software is getting very crowded and it's a scary market to be in but you know there's there's hardware there's like physical services there's all sorts of real life businesses to be made that incorporate AI in interesting ways how how do you you know physical services like can you elaborate on on what you mean yeah I mean to be clear I'm a software guy so I don't know anything about this

11:58but if I were not a software guy I'd be thinking a lot about how can I integrate AI into some kind of real life experience that people are having where there's actual some kind of human exchange of value whether it's like you know a person giving you a massage or a restaurant that's serving you a meal or whatever it is I think there are so many times of like real businesses that are not quite reinvented with AI yet um I don't know that they all need their own harness although perhaps some of them do like another category where I'm seeing a lot of harness development going on is

12:30like legal tech for example I suspect medical tech is like a big one that has a lot of interesting like data blocking and regulatory stuff going on. Yeah, I think legal was one of those where it was like obviously this makes sense. Um accounting obviously makes sense. I think uh the thing I would be thinking about is like would I be building a legal tech a AI

12:52agent harness today? Probably not. Yeah. Um just because it feels so crowded. Totally. So, I like when you know when you're talking about service businesses, even boring businesses in general, like agent harnesses for HVAC companies or, you know, um if someone wants to build an agent harness startup, h um in a niche, how do they approach uh designing designing it in the sense like you need the key insight, you need the niche? One really simple answer about what an agent harness is is an agent harness is just a bunch of tools that an agent uses to complete a task. And so I think the question to ask is what are the unique

13:32tools that either I already have or I can build that are not already out there. And ideally I think as a founder you should always ask the question of what am I uniquely suited to do based on my life experience where I am in the world the people I know that sort of thing. Um

13:48and h how do you think about AI smell? Yeah, totally. Uh, you know, because I think a good a good agent harness is going to kill that smell as much as possible. And I know you have a strong opinion on that. I do. It's I have been using this term AI smell um all year. I've been talking about deodorizing, spraying it off, all that sort of thing. Maybe for people don't just don't know the term. AI smell is what we use to refer to this fact that when people make work with AI, you can often tell they used AI. I think it shows up most in writing where when

14:18someone writes you an email maybe with claude you're just like you don't normally use that word and that's not what you sound like. Um and and I notic it a lot with AI images now where I just like walk around in San Francisco and every cafe every restaurant has a poster where I can just smell like that's nano banana that's GPT image I do this stuff all day so I can like spot them a mile away and um I think more and more people are starting to recognize that AI smell as it diffuses just through the economy and normal people. So for us at Gamma, it was actually a huge goal of this

14:47launch. We had kind of been a victim of our own success as one of the early AI tools where when nobody used Gamma, if you used it, your stuff would look really cool and stand out. But once everybody started using it, everything started to look like Gamma and people didn't want to be recognized as using AI. And so kind of our whole goal of this launch we just pulled off was we need to rapidly expand our sort of uh creative landscape. Um, the truth is there's still going to be AI smells, but there can be a thousand AI smells instead of one monolithic one. And we

15:16can help people jump to different little pockets. And we needed, this is another form of a harness, by the way. We needed to create our own automated system for constantly cycling through different creative looks. We actually use the analogy of fashion. Fashion has a smell, too. You're like, I know that H&M shirt you're wearing. Like, I have the same one. Uh, and you know, with fashion, there's kind of this dynamic where people are cycling through different looks. You know, right now we're into like really big wide jeans, but soon it's going to be skinny jeans again. And so we realized we had to actually ride that cycle and give people the tools to

15:48adapt to it. So is there an eval for this looks generic? You know, uh, eval is such an interesting topic because I think eval work really well for things that are objective. Actually, break, if you don't mind, break down what an eval is. Yeah, totally. So um an eval just means a you could think of it almost like a software test. It's a hard-coded question that you ask and a way of judging it. So a simple example in our world of gamma is we often want the agent to ask questions when you do something ambiguous. So we'll give it nel like um when the user says make a

16:21deck about tigers the agent should ask something like is this for kids or for adults because there's very different kinds of tiger presentations you can make about that. Um, and so we typically use evals to test out new prompts or new models. And usually you score them and you can put a bunch of evals together into a benchmark where you say, "Oh, it turns out model B scores 3% higher than model A." Every time the labs come out with a new model, they're like, "Oh, Opus 7 crushes like these 15 evals." The problem for us in a very creative domain is that eval subjective tasteoriented things like,

16:56"Does this look cool?" and even more so very personal ones like is this me? Does this sound like something I would say? There's no one objective eval for this. And so, you know, we've tried to create some evals that are proxies for this. Like, does it follow instructions? Does it, for example, if you provide your

17:12brand guideline, does it stick to it? That's an objective question that we do test ourselves on. But we've never limited ourselves to just eval. And I think this is actually interesting insight too of how you can compete in this world. Ultimately, what founders can bring to their domain is taste. they have their own sense of what looks good. They're actually talking to their users and those users have one. And the less verifiable a domain you're in, like say fashion, the more I think there's room for these unique perspectives. I think when it comes to clothing design, there should be 100 different harnesses. I think you know Nike should have a

17:43different one than like Adidas because they have a different perspective on taste. Also, for us, because we have a super diverse user base who are all different, we actually rely a lot more on AB testing. We will actually run two models or two prompts at very big scale and measure across that wide user base what's actually performing better because that captures subjectivity and what's what's the metric that

18:03matters most there. You know, we have a bunch of different ones. To give you a few examples, we look at things like um after you generate, how much time do you spend editing what you made? Do you end up sharing it or exporting it? Um are there signals that show you're investing time? We also have a premium product, so people will use our product for free and then if they like it, they'll upgrade to paying. That's kind of my favorite

18:24signal because it's hard to fake. Generally, people will only pay if they're really seeing value out of a tool. So, we use that one as like a northstar. Sometimes we'll even look at like downstream retention, like are people still coming back to the product a week later or a month later. It's obviously slower to measure, but it's

18:38also one of those hard to fake signals. The chart that's going viral this week is I think an A16C chart that says I think only 2% of of people are spending on AI tools. What's your opinion on that? Yeah, it doesn't surprise me. I think in general subscriptions are always a bit of a niche thing towards people that are doing some kind of valuable work. I

19:01don't think consumers like subscription. I, as consumers, don't like subscribing for things. You know, I'm still always looking to jump on somebody's Netflix account or whatever it is. Um, no, I will not share my password with you. Are you sure? What if I explain what an agent harness is? U, but I think it's different for work. I think for work, it's a totally different story where, you know, work subscriptions are what has driven essentially the entire SAS boom the last like 10 to 15 years in software industry and those things will keep being successful. And I think that same chart says that when you actually zoom in to

19:29professionals, the numbers are rising like crazy in terms of who's paying for AI. So that's certainly our main focus and I think most people's main focus in AI is how to find those those proumers, the professional consumers and how to break into B2B where there's a totally different kind of spending going on. If someone wanted to build the gamma for X, what would what would you pick? Like if someone listening to this is like, wow, like you know, I'm inspired by what Gamma is building. Yeah. Um, but I I think that like that, you know, basically what you just described, but like for a different niche,

20:04anything coming to mind? I It's such a personal thing because if you know, if it was my thing, I'd be building it already. I think you need to do the thing that I don't know anything about, whether it's like Pokemon cards or fashion or sneakers or whatever it is, like find your domain where you know what's going on and build it. and and you personally, is there like a harness that you wish existed that doesn't exist that you'd use tomorrow if it if it came

20:28out? You know, um I'm really intrigued by this whole sort of like personal agent space. And I haven't gotten to use all the latest ones, but what I feel uh nobody's totally cracked is like work collaboration. I think if I could build a harness for anything, it would be Slack. Um and I'm a little disappointed that Slack themselves have not built the

20:45harness for Slack. What would that look like in your mind? So, there's just this absolute flood of communication for anybody whose company uses Slack or Microsoft Teams, of course. Um, there's so many messages going through. Um, you're constantly trying to stay on top of like, well, which ones do I actually have to do something about? Um, I've seen them nibbling around the edges of this of like you can sum summarize 30 Slack messages into one, but I want something that just like runs it all truly for me and it learns me. It learns how I would reply before I reply. I want to be able to open any Slack thread

21:17someone's pinged me on and have the first draft drafted for me. And it learns you not by answering like a quiz or onboarding it by actually like your action by observing me. Yeah, by observing I have sent approximately a billion Slack messages. That should be all you need to know how I'm going to reply,

21:32right? Even in emails doesn't feel totally cracked. I've seen Gmail has the little things that are like, oh, you know, like, yes, I'd love to go to the dentist on Thursday, which is not how I'd ever reply in my life. But uh I think there is so much more to just like represent me in a professional setting authentically to me. Like maybe that's what I'm really craving and it's a little bit of what Gamma does is I want

21:52an agent that can learn my voice. And of course that's hard because it's personal and this is why the agents haven't done it. It's really hard to make agents personal. So you know at Gamma you know you're you're building something on top of other people's models. Mhm. Um what's your advice for founders building

22:11on other people's models in general? I think there's no shame in building other people's models. We you know we have done this from the very start often bouncing between anthropic open AI Google models and so much of the discourse has always been like no you got to train your own stuff you got to

22:26fine-tune it because you need the moat. I have always felt that that's um unrealistic for early stage startups in particular. I mean, first of all, the money involved in training your own models is really hard, but also the models out there keep on getting better. Uh, I rarely would want to be competing against the models that are out there. I think though it does change as you get further along in the journey because you start to amass real usage. And I mean, let's be honest, just amassing real usage is the main battle people should be focusing on. By all means, use the models that are out there to get that

22:55usage in the first place. Once you do, though, you can start to do really interesting stuff. We've actually had great success fine-tuning models to bring our cost down and in some cases even to start to do more personalization for people. We also have started to identify specialized tasks that you know the big models are bad at. Like I mentioned earlier, this whole process of

23:12QAing a slide and telling if it's good. We've actually not had great results using any of the models out there for it. So we are actually doing some of our own research to build models for those specialized tasks. I think people should absolutely aspire to that, but by all means use the great models that are out there. What about what about uh thought you know current what what are your current thoughts on frontier models or cloud models versus

23:34open-source local? Yeah. Um I think you know cloud models are underrated. Again I think open source gets a lot of hype and excitement but follow the money. It's all going to the uh closed source models because they're really darn good. So much work has been done for you. And the great thing for all of us is there is intense competition in the world of closed models. I think in a world where you know like Anthropic had just purely run away with it, this would be actually a different story because we wouldn't begin to choose between like great open AAI models, great Gemini models, but I feel like all these companies are just a

24:05few months behind each other and for us in the application layer that's the best thing ever. It keeps the heat on them to keep doing better and better. You've once said that uh you you were surprised with how shallow your AI stack was. Is that still true today? You know, I would say that our AI stack is broad but shallow. What I mean by that is we actually use a ton of different models, but we often use them in fairly simple ways. Like we get so far with prompt engineering, with providing and maybe your listeners have heard the term context engineering increasingly of just like providing the

24:39right context to the models. I really view that as kind of the whole game here of just and it comes back to harness building. Can you get the right context from the user? Can you express the AI's output to the user in the right way? And can you distill all that into things that make the LLM's job really easy? When you make the LM's job really easy, you can use any offtheshelf model and

24:57get good results. There's a big uh debate prompts or fine-tuning. Yeah. You have a a strong opinion there. Yeah. I think for us it's 98% prompts is where we get the value out of cuz prompting is easy and it's portable. So like and by the way, nobody's writing prompts by hand anymore. Like Cloud's writing your prompt for you and so it's really easy. Um, and so I think most problems can be solved with a prompt, including breaking up a prompt into different pieces. And fine-tuning to me is more of an optimization on top. When you want to do things like control costs or ek out that extra 2 or 3% of quality

25:31when a new model drops, how quickly are you implementing it or are you kind of like waiting and seeing or h how do you think about it? Oh no, usually uh within the hour. If it's uh if if it's a model that I have reason to believe is going to be good. So, you know, not every model is interesting, but I would say for us, like any model from Anthropic or OpenAI is interesting, generally if they're dropping it. And I think I mentioned we have all these different jobs in our harness. Some of those jobs we use a like, you know, fable or astro level model for. Some of

26:01those jobs we use a little haiku gemini flash level model for. Some we use a little baby GPT luno model for. And so, anytime any model comes in this class that can help us fill out that sort of paro frontier, I want to try it and see how it does. And generally because our use case is so subjective, yes, we'll run some evals, but I just want to run it on a sample of users. And we're very lucky to have a big user base where I can start a test in an hour and get data

26:242 hours later on how it's doing. Um, so one of the things I learned cuz I'm a co-founder of a design agency called LCA and we actually worked on a project together at Gamma and one of the things I learned working with you guys was you guys run a lot of experiments. Yeah. I'm curious about um experiment you maybe one experiment that the results surprised you um because you guys are just it feels like you guys are running a lot a lot at

26:51a time. We run a huge number of experiments at any given time we probably have 10 or 20 running um and we're cycling through many of the experiments last less than a day. So walk me through like the process of an experiment because I think it's really really interesting. Well, I'll walk you through one that I just started in the Uber ride over to talk to you. I just fired off eight different experiments because uh Claude Haiku 5.5 came out yesterday. We were really excited to get to use it because, you know, I mentioned that we have a lot of these tasks that don't need that like

27:21super fable level model. They're quick and cheap like picking a template from a library or sometimes like writing an outline based on something. And you know, we have been looking forward to new small models we could use to test this. So, uh, this morning I fired off eight different tests for it and right after we get off this podcast, I'm going to check the results and probably cancel four of them and see how they did. Um, and we're testing it for things like making micro edits on a slide or, you know, like revising an outline, whatever

27:47it is. Terms of the kinds of things we learn from these tests, um, often the great thing that we learn is that we can use a model that is half the cost of another model and get the exact same results. But sometimes we actually get uh the opposite finding. We find like wow actually the big beefy expensive model

28:05is necessary to achieve a certain task. Um sometimes we even discover things like good friction like we actually just ran an experiment a few days ago of a fast mode of our product. Um and fast mode just means skip through some of those steps that I told you about earlier like showing you an outline of your presentation or asking you questions up front. And the hypothesis was of course if you remove friction on the process then people will get through it faster and get to that sort of magical value moment more quickly. I was I was really rooting for this test. I wanted to see it win but it did not win

28:36for us actually. Um it turns out those speed bumps we had along the way like seeing your outline and reviewing it, you know, choosing uh the kind of highle arc of your story, answering those questions led to much higher satisfaction ratings. even though it was slower, the people that got through felt more invested in their output and did more and they actually converted to paying at a higher rate. Um, but I think what's also cool is a test is never just one answer or another. You can like dig a little bit deeper. So, we actually shipped a version of fast mode for brand new users on mobile because, you know,

29:08on mobile you don't want to be typing a bunch of answers to a question. I just right before this I posted an ex article about marketing engineers. M um my thesis is that uh marketing's changed. If you think about the Mad Men, Madmen era, it was like, you know, you did one big commercial a year and in the ' 50s and ' 60s and ' 70s. Then it became um in the '9s and 2000s, the digital marketer, maybe a few more uh experiments. Then we had um the growth hacker era, a few more experiments per week. Um I think in 2022 I would say AI assisted marketer uh more experiments

29:47and you know I think we're now entering the age of the marketing engineer where you're you're you know as a marketer you're running dozens of experiments per day. How are you able to do that? You're using AI agents to go and do that for you. I'm curious your P POV on that. Do you think that marketing has changed and like where do you think it where do you see it going? It's funny because before I worked at GM, I worked at a company called Optimizely and this is where my love of AB testing came from. We were AB testing as a service and a lot of our

30:16vision was about personalization. This idea that you know as it gets easier to change your website, we are going to create a hundred little micro variations of it and everyone will get served a slightly different version and that will optimize conversions in all these interesting ways. Surprisingly to me that vision I would say largely has not come true out on the internet. Um, I would say websites are still mostly one-sizefits-all. Uh, so are mobile apps by and large. Um, except for these like specialized areas which are like hyperpersonalized and succeeding. When I say specialized areas, I mean things like your Tik Tok feed, your Instagram reels. Like we know that companies are

30:53spending billions of dollars to ek out every last, you know, dollar from you in those surfaces. I heard that McDonald's is testing personalized pricing. You hear about this? Wow, that sounds like a backlash waiting to happen. That's why I saw it. Yeah. Yeah. But I think maybe that McDonald's example illustrates why this isn't happening. First of all, I think there's a real societal allergy to this kind of thing in the same way that if you found out that we both bought the same airline ticket, but they charged me five bucks more cuz I was in San

31:19Francisco. Well, I think in general, like even surge pricing, like I feel like when I I had surge pricing last night uh coming home from dinner and I was like I was like h you know, like it just it's not a good feeling. Yeah. Yeah. And you know, I actually think in this age of hyper efficient, you know, like performance marketing, actually what's working is companies that really doggedly invest in their brand. This like Apple mindset of like just protect the brand at all costs. And so I think that trend is going to counter some of what you're describing of everybody just going wild with their

31:51marketing agents. That said, I still buy the thesis that like the marketing engineer is rising. And I just love that term marketing engineer because it sounds like a contradiction, you know, much like GTM engineer sounded like a contradiction. These are supposed to be different departments, but I have loved seeing how everybody's skill sets are just blending. Uh, we're seeing this with Gamma actually where people are more and more using Gamma not to just make one presentation here or there, but creating these really cool automations where they're doing things like every time a new deal opportunity is created in Salesforce, let's automatically create a pitch deck based on the call

32:25notes that have happened. How do you set that up? We've actually been building kind of connectors where you can automate more of it. Um, so you can just like plug into Salesforce, but I would say there's still a fair amount of plumbing going on where people are using maybe a tool like NAN or Zappia or something to trigger it all together and wire it. Um, that is a good example of a thing where it's still the domain of specialists. I would say there's like these really smart people or agencies that know how to help a business like wire things up. I think there is still a business opportunity in

32:54taking these practices that are done by the top 1 or 2% of companies who are really savvy about engineering it and just turning it into a super easy thing that everyone does. Another example of this to go back to the personal agents is like what's happening now with tools like Muse and Instinct. You could do some amazing stuff with OpenClaw or Hermes if you could get your Linux box set up in the right place and avoid getting hacked. Personally, I skipped out on that whole trend. I was like, I'm a busy dude. I don't have time to set up my open claw. I'm going to wait a few

33:21months for them to just make it easy for me. And now I got Muse Instinct blowing up my phone because they made it easy for me. Right. So you don't have a Mac Mini running. Uh I never got my Mac Mini. I'm ashamed to say as an AI founder. You know, going back to marketing engineering in your mind, what is a top

33:381% marketing engineer look like? Um well, one thing that comes to mind is they're really a marketing engineer data scientist. So they are like ultra data driven and grounded in what's performing. And this is something that's gotten easier than ever. Like absolutely Claude is my data scientist. And so like I would say now everybody on our team is constantly armed with like very fine grain analysis of like failure modes and opportunities. They're looking at traffic on our site. They're finding gaps. there, you know, like I look at our, you know, the person we have on our team who's closest to a marketing engineer and he is just constantly

34:14thinking of new ways to do SEO ad creative at just an unbelievable scale for one person. You know, they're like they they've gone from being like individual creatives to people who are running this army of agents. Um, and they are just in this highle supervisory role. Yeah. I mean, it's basically a department. It's department. Yeah. Yeah. It's like we're getting to the place that if you're a top 1% marketing engineer, your surface area of what you're working on isn't just

34:42uh well, the surface area is huge. You're looking at SEO, GEO, you're looking at uh social content, you're looking at paid ads, you're looking at um pretty much like all creative brand, you know. Absolutely. It's I think that's what's interesting to me because I think that the people that figure out how to become a top 1% marketing engineer have an unfair advantage around

35:08getting their product seen. Well, and the other 98% are quickly becoming dinosaurs if you can't keep up with the people in the market that are doing this. I've kind of had this idea that all of us over the last maybe like year or two, at least all of us who are using these tools kind of got this rapid series of promotions. We went from individual contributor to manager to director to like VP. We now have these like layers and layers of agents under us in a pyramid. And it turns out that job's actually kind of hard. Like um uh there's a reason that like few people do

35:39it because you are sort of constantly barged by information. You're having to make so many decisions. Uh you kind of are like overwhelmed and confused by all of it. You need tools to even make sense of it all. You also need people to help you make sense of it all. you kind of need your your assistant and you need your chief of staff and you need all of these things and so all of us are having to rapidly upskill to essentially like how to manage a thousand people and building that skill is one of the most important things we can all do. Also I think building tools to help the average

36:10person become a like VP of their own life is still underexplored. Going back to my Slack idea, I also think uh you know experiments in general, it feels like building a startup today is or building a business in general is it's like you have is more it feels like almost a little more science than art.

36:31Yeah. Now in the sense like you're the ones who are going to win are the ones that have the experimentation muscle and just trying a bunch of different things. That's why like I love talking to you because it it feels like that experimentation muscle is I mean that's your like your thing. Um what's one thing at Optimizely that you learned that every AI founder should should you know would be good for

36:59them to know? That's a great one. You know, one thing that comes to mind for me, I think from years of like helping other companies do experimentation was that so much of it just comes to like how many atbats you have. Um, uh, it's all just sort of a numbers game, just like sales, just like dating, just like kind of anything else in life. You got to really try a lot of different stuff versus expecting that you'll run one big, you know, test that magically unlocks your business. Um, I like how you called it a muscle because I do think it is. First of all, because

37:31it's trainable, you know, yeah, maybe some people are born with more muscle than others, but if you put in the work, that muscle is going to grow. I also think like your first experiment is not going to work. Totally. Just like if you're trying to get fit and you're trying to build muscle, you have to just show up to the gym every day and you have to put in the work. Absolutely. And then two months later, then then you might see a six-pack, right? But you can't expect it on day one. I think uh that's the hard part about being uh an experimentter is like there's like a

38:01trough of sorrow that you have to go through. Um and I think what a lot of people end up doing is they're like, you know what, I'm just going to go with my gut. Absolutely. Um and by the way, the same is true of things like posting on social media, writing a blog, right? Sending cold email outbound. There's a trough of sorrow in all of it. And I think all success in life comes from swimming through some kind of trough of

38:24sorrow. What's uh what's something that's underhyped right now that you're like, you know what, people are just not talking enough about this? I was talking with um a friend of mine about voice AI. Um that felt like underhyped. Um right now people are talking about personal agents a lot. Anything that comes to mind? Maybe it's a lame answer, but I actually

38:50think AI image models are underhyped. It's a lame answer because we've been talking about image models for years. Ever since like stable diffusion came out in 2022, people have been making like cool stuff. But I think um ironically as they got good, people stopped talking about them. As they stopped messing up in so many ways and just became sort of like part of the fabric, we sort of like forgot that we have this unbelievable technology that can make anything. And I think if you look at the way people are using like chatgbt images and a few other things, it's kind of just the same three or four

39:23use cases over and over. You know, they're they're making like maybe the the menu for their small business. They're making like a logo for their dad's side project. There's like a handful of things they're doing, but these models are capable of so much. And we have a really cool vantage point to this at Gamma. We didn't talk as much about images, but we use probably like 10 different image models as well for different tasks. And the pace of innovation there has been so much and just this sort of like unexplored capabilities. For us, it's things like making beautiful diagrams or

39:56infographics to go in a presentation. Making cool little textures that can be the background of a presentation, making custom icons, like all these things are possible now and most people just don't realize it's there. And I think you can pay attention to almost like any domain that's creative and find ways that you could be using these things. And especially the average person just has no idea. You know, even one that I think is like probably many AI native people are doing but most people don't realize you can do is using AI image models for like clothing and like home decor. You know, like you you've had a new couch

40:30you want to buy on Amazon. You can actually chat GPT to say what would that look like in my house? um and then say like go find me a few others and visualize those two. Like so many of these things are just waiting to be more deeply productized. Do you think there's an opportunity to build vertical products there like just focused on you know interior design for example um or do you think the the big model companies are just going to own that use case so it's not

40:56even worth trying? I think there is so much room for these vertical use cases and it's all about understanding the specific needs of the people that do them. Like let's just take this furniture one for example. I actually just bought a couch recently. So I Congratulations. Thank you. It's really comfy. And I went through this and it was so interesting um like trying to do this and it turns out the AI image models are amazing at showing me what the couch will look like but they're not actually showing me exactly how it's going to measure, right? Right? And there's a huge difference between, let's say, a 68

41:29couch and a 58 in couch if your walls a certain size. So like, could you actually show me the units in the display? That's a great example of a harness, right? The models are actually capable a lot of these things. One way they're capable is you could imagine doing a thing where you render the couch in my room, but then you use some good oldfashioned programming to draw a line across in pixels and measure how long it is and see if it actually came out at the right size and prompt the model to tweak it to get exactly right. I don't see the labs doing that anytime soon. or

41:55prompt. Hey, like this couch isn't the right couch for you. Yeah, totally. But here's another couch uh from Restoration Hardware that I think would be perfect for these reasons. Totally. And dude, what I learned about couch shopping is there's hundreds of websites that sell couches and you'll have to click through a lot of little buttons on all of them. I I ended up on Google Shopping. It was not a pleasant experience. Maybe somebody's already built a great product for this and I

42:20missed it, but I haven't seen it yet. I think uh there's been examples. I think Levels IO has like a Yeah, he has like an interior I think it's called. Yeah. Yeah. But I think there's still room. You know what I mean? Like I think uh especially as the models are getting better, the image models are getting better. Um the the concept of a harness is getting more and more mature. There's just there's more here. I think uh I think it's safe to say that the also the consumer experience today around couch shopping

42:51or furniture shopping isn't optimized. Totally. Um and what's cool about that specific use case is it's high ticket. Totally. Um so uh and there's margin, right? Like if if for example like I don't think it's crazy to do a deal with Restoration Hardware where they're like we'll give you 5% commission if you recommend sell more couches. Yeah, absolutely. I think the other version of this when you think about like what will survive competition with model companies um and we're certainly facing this question constantly at gamma. There is the pure consumer version which to use the couch analogy is like help me buy a couch. Um but then there's the like proumer

43:31version which is um hey I am an interior designer. Maybe I'm a part-time interior designer but I also have like two kids at home and that's what I do but I I you know I I do it by the hour. Those people need a tool and they will seek out the best and if it helps them do their job better, they will happily pay a subscription for it. Then there's also the true pros. There's restoration hardware itself has people who work there. What tool are they using? I bet it's not just chat GPT to do all of this. I think there is actually an opportunity to build companies for

43:59those. And I do suspect that highest tier of professional the the bigger companies is where you can really build something durable. Yeah. And I I I feel like it's underexplored. Yeah. you know, cuz the narrative I think is there's there's not that much opportunity um creating vertical vertical SAS or vertical AI products. Yeah. Um which is unfortunate because I think that um there is and it's a big world out there. You know, that's maybe one other thing we've learned from the Gamma journey. Um our user base is very broads. If you just think about how many people on earth make presentations, it's a lot of different kinds of people. Um, for us,

44:37uh, less than a quarter of our revenue is in the United States. Um, it's spread across many different job roles from like consultants to teachers to marketers to sales people to lawyers. I didn't know lawyers make presentations. Turns out they do. Um, and you know, the more you can tap into little niches, you know, like in many ways a big company is just a thousand little niches piled on top of each other. So if you can find something that cuts across and you don't have to start broad, you can start with just one but then look for the adjacencies you can tackle 100%. And you actually don't know where

45:07you're where it's going to lead, right? Totally. Like you didn't realize like hey lawyers are using are creating presentations and then you end up who knows you end up creating something specific for lawyers. Absolutely. And then all of a sudden that becomes a huge part of your business. Yeah, totally. And that's why I'm always like get

45:26started like put something out there. Put something out there. You got to just find out. That goes back to experimentation. Experimentation doesn't just mean run an AB test with like 10,000 users in each arm. It usually just means put up a landing page, get a product someone can try, which has never been easier. If you I know you're not leaving Gamma, but like let's say you were to leave Gamma tomorrow. What what's something what what's a company you would start to

45:48build? Oh, that's a good question, man. You know, it's I I just love circling this space of like enabling people's creativity. I think there's other kinds of like creativity that like intrigue me though like um as crowded as the space is I'm so intrigued by just like vibe coding as I practice and just the way I had this experience maybe as somebody who knows a lot of other tech people and lives in San Francisco earlier this year where every time I would catch up with a friend I hadn't seen in 3 or 4 months I'd be like what are you up to and they'd be like check out my app I made

46:21it sort of took the world by storm I want to know maybe what's the next vibe coding or what's the next frontier here of vibe coding as people can make anything um where will that take them and so like within that I'm I'm certainly intrigued by video models finally getting good I'm intrigued by voice models getting good like what's the version of uh some kind of thing you could call and it would do something for you games are so interesting to me I just think like the video game industry is right for some kind of disruption but

46:52I haven't cracked it yet. I don't know. I think on the video side it does feel like we're hitting like the vibe coding moment for video. Yeah. Uh Opus 5.5 has is obviously they make some cool stuff. Make some cool stuff. The Yeah. So I think the question is in a world where anyone can create video, what are the types of videos they can create? And then I'm also interested. It's like as a founder I'm like okay in a world where anyone can create video, what are some startup

47:18ideas that I can create? Totally on top of that. And that goes back to agent harnesses. Yeah. Absolutely. Everything goes back to agent. Everything goes back to the harness, baby. Um, dude, thank you so much for coming on the podcast. Is there anything you want to leave people with? One thing. Uh, I would just always encourage people who are thinking of starting a business to know that it is a long and sometimes lonely journey. And because of that, you really have to love the thing you're working on. Um, one of my kind of rules for myself before we started Gamma and it would stick around if I ever did

47:51another startup is it has to be something that I could dog food that I could be a user of and that I could believe in. And to be honest, that actually rules out a whole lot of boring businesses, but it was good for my soul. I just needed to have something that I wanted to spend hours every day in. And so I always encourage people to dig deep and think about like what is your passion and the thing that um only you know the thing that you want to spend 8 hours a day on that when you look around your friends don't want to spend 8 hours a day on and

48:20zero in on that as your north star. I think that's the important point too around like the thing that you want to spend 8 hours a day on that other people don't. Totally. Cuz like you might be really into sports or something but a lot of other people are into sports and that might be very competitive. I think like if you're if you're choosing a startup idea or choosing a business, I always say like

48:41uh marry the niche, date the product. I love that. Like fall in love with who you're trying to serve. Yeah. Yeah. And because we live in an age where you, you know, you're going to be iterating on the product so many times and it's going to look so different, but if you love the the niche, um, and that's why I always encourage people to, uh, build media, build a following in that niche, cuz then you can sell them a bunch of

49:06different products. You have built an audience. I think that's totally right. You know, Jeff Bezos has a very similar quote where he says, "Be stubborn on the vision, flexible in the details." Well, I'm glad I'm in good company with Jeff Bezos. You sure are. Thank you. I appreciate you and thanks for having me. This is great.

49:19Great time.