Sub Club Podcast: How Apple’s App Tracking Transparency Affects Developers

0:00the more signal you give to the algorithm uh the better the algorithm performs right uh you know in the pre-att world if you give more purchases the better the algorithm performs obviously the algorithm would take purchases from you and everybody else in the world and it would just do better now obviously it's just taking your trials and doing much much better hey you're listening to the sub club podcast a show dedicated to the best practices for building and growing subscription app businesses we'll share insider secrets from the top subscription apps on the app stores let's get into the show welcome to the sub club podcast i'm your

0:40host david bernard and with me as always jacob biting hello jacob hi david here with you as always our guest today is shamanth brow founder and ceo at rocketship hq host of the podcast mobile user acquisition show and lead instructor at the workshop series mobile growth lab shimant's company rocketship hq is a boutique growth marketing agency with eight figures in managed spend prior to founding rocketship hq chamonte laid growth marketing leading to three exits hey shamath welcome to the podcast honored to be here thank you for having me david and jacob yeah so um i wanted to start with a little bit of of a history lesson you've been in in mobile

1:26advertising and working on mobile apps for uh since very early and so could you take just a couple of minutes and step us through the history of kind of what led us to today with app tracking transparency uh and all the different ups and downs and and changes that have happened over the past decade yeah there's been a lot of ups and downs as you said but i think that i see two overarching trends but for folks who want to go into the weeds i would actually recommend two podcast episodes one with mine with you david uh it is called a brief history of app store monetization

2:09you provide a very great perspective into how the app store itself has changed over the years uh the other one was uh an interview i did with david phillips i just titled a brief history of device identifiers that you know we're all about brief histories over here but uh i think pertinent to what we're talking about att and how that essentially disrupted growth marketing today there have been two forces that have led us to this point uh which over the last decade or so right and i think it's important to know and understand both of these just to know how we got here and why it's important right because

2:58this att just did not happen overnight edt prepared with signs for a decade and uh you know i think obviously a lot of this is evident in retrospect uh but i think it's people to know and understand what those breadcrumbs were trend number one was that there has been increasing accumulation of purchase data by platforms over the last decade you know i remember you know david as you pointed out i am a really old person who was around when google advertising just took off uh with all this grey hair uh but you know when i started that we were doing cpc buys ppm buys i started doing mobile advertising

3:45before facebook even had mobile ads mobile app ads right and there was no conversion tracking uh you know in fact i yeah there was like no conversion tracking if you you would buy installs and you're like oh we bought so many stuff we got so many purchases then we we are profitable and we spent like millions on games at the time and some of the roles had it right so it certainly the level of sophistication that emerged in mobile advertising i don't think we could have posted in 2012 2013 2014 but one of like i said from the tpc buys gradually they were cpi buys as ad networks that now are billion

4:29dollar companies iron source and applause and they were tiny ad networks at the time a lot of others basically fell by the wayside uh you know they were like we have enough confidence to be able to build for install rather than just a click upper impression just because we have that kind of data that kind of confidence the next step really was facebook's aeo or purchase optimization this is 2016 right which is which seems so recent and it's so staggering to think that we could not optimize for purchases like athletes today six six years ago uh it's crazy and that was just the biggest game changer and i still remember

5:11having a lot of skepticism that this would even work and i'm like how how are they going to find out who's going to purchase they've never done it nobody's done it but clearly somebody could do it in this facebook right so they had the purchase of data and i think only subsequent to that point of time i think it became evident to me myself that as to why facebook was so successful because they because basically of the idsa right idfa and google id uh they had that idf uh with print from their family of assets certainly on facebook audience networks so they were able to predict

5:49with easy accuracy who the purchasers were obviously they took it a step further with a robot optimization right so and obviously the more data facebook's uh uh sdk got the better it got at predicting who the purchasers were obviously the uh the more data the pixels on the web got the bet the better the accuracy of the sdk became and the other way around right because they had you know if you made a purchase on a beauty website you would make a purchase on a on an e-commerce app so they put all of that data together right so obviously google had a very very similar trajectory i

6:26don't want to go too much into the reads but over the last decade increasing amount of data got accumulated by facebook by google folks like app login and i and source all the ad networks everybody got increasing amount of data about user's purchase of it obviously they just weren't doing this in isolation apple why isn't up to this uh you know google well google had a conflict because they were also making money off of this they are also making money off of this now so they have been less active and pushing back but you know apple said uh-huh these you know you know apples again not to go too much into the weeds

7:04but it's part corporate strategy to but for apple to say aha we have private reminders but it's also very very much about profits for them uh they are promoting their own ad network now i'll be i i don't know i don't have a lot of confidence to how they will do anyway right so apple said you know look we had this udid which is non-resettable let's space it out let's have an idfa which is resettable which turned out wasn't too much of an improvement then they said oh let's make this uh idf is zero but idf is zero which means if a user goes on and turns off the lat that idsa

7:45becomes zero and advertisers could not cannot target people shockingly enough prior to ids zero which was i think 2016 they still had lot but if a user turned off light it just meant advertisers please don't crack the fuse it was almost like a request it was a non-enforceable request basically that these don't track me but obviously nobody cares right anyway so even lab zero was a very very telltale sign that this is where att is where apple is headed and if you had to look on the web uh safari had intelligent tracking prevention they have obviously been much more active on the web in terms of

8:29privacy crackdowns uh mozilla had what what's called etp i think it's called efficient tracking prevention i forget what it's called but then chrome of course said hey we're going to deprecate cell body cookies they've extended the deadline but there's been a lot going on in the direction of privacy right uh and this that has happened very very much in parallel with increasing accumulation of data by platforms and to some extent you know it hasn't been surprising for anyone who's followed the breadcrumbs that's gone to zero and 2018 apple said kids apps will not allow tracking that was almost like a trial balloon for hg and of course 20 20 80 hit

9:11it was not unsurprising i would say that uh it came to be just because of everything over the last decade that i just described yeah that's a that's a really great way to summarize it is those two parallel courses with it's like in the shadows there was like more and more and more more data accumulation feeding all of this but simultaneously there was more and more and more awareness of privacy concerns and what that data was being used for and that you know it does seem like the press you know had a big influence in this i mean when because the new york times and wall street journal both had

9:49big posts in like 2017 2018 um where they showed you know how you could track individual users and when they're going to you know a certain medical clinic or yeah there was another set of stories around u.s service members who were being tracked by fitness apps and revealing uh basic uh um we call unknown previously unknown uh military installations and things like that so yeah yeah there was a lot going on that has led us to this point so so now apple has has dropped the bomb after acting transparency you you're you're you're not allowed to track unless you first prompt and you know we could we could talk an

10:35hour on all the different motivations and the um and even the way they deliver it you know the way they request the prompt is is and the wording of the prompt has has even drawn controversy but let's not get into that i still every time it comes up i still don't know what to click but let's talk about the real world impacts because i think there's been a lot of ink spilled and a lot of discussions around um around those other things but but what i want to hear from you as someone who manages a ton of span and works in the industry and uh and has to deal with this day in day

11:14out um let's talk through the real real world impacts of of of how this is impacting the apps that you work with and what you've seen kind of in the broader industry i thought it was interesting before we jumped on the on the and started recording you actually said uh you were expecting a crazier summer so let's just start with that so you're not quite seeing the disruption you initially expected is that oh am i overreading that i i don't want to be grandstanding here but i certainly was prepared for part far worse uh right and i don't want to jinx this but certainly that couple of

11:53advertisers we're working with that are doing all right that are actually growing but i'll talk about the mechanics that may have contributed to that further on right uh but i certainly was prepared for far far far worse i would say yeah so so what are you seeing i mean um and then one of the things you you've mentioned before is that you are seeing some shift to android tell me about that uh shift to android spend and is that in certain categories across the board i i think it's across the board i think it's much more so in gaming and if you look at a bunch of mmp

12:28reports and estimates they peg the shift to about ios spend dropped to about 30 to 40 percent uh i think that sounds like a realistic range obviously there's some verticals that i hit much much harder uh right and yeah um but definitely i think there's a lot of spence shifting to android i would attribute some of that to the fact that tracking is broken but performance isn't you know oh uh to see this uh right and it's like a number of folks we work with and we also advise and i've just talked to you they're like oh my god my cpa is reported by facebook is

13:13terrible because facebook start tracking everything and then when we look at their blended numbers which is basically the money they make and the trials they get and the subscriptions they get or the questions they get it's all right you know it's which is exactly what i mean by not being as creepy bad as i expected you know if you look at the itunes dashboards you're like you think all right you you know you didn't just go crashing down which is what i was afraid would happen right right that has not happened so no but what is real and true is like i said tracking is broken

13:52even if performance is not right and tracking's just broken just because apples have a concept of privacy threshold which basically means if a campaign does not have a minimum number of installs or purchases apple is going to show zero in stock zero purchases they report all installed but they report very few purchases what that means is if you are let's just say a casual game or a social casino app that has a cost per purchaser of 150 to 200 which is not uncommon for these apps then each campaign if you're running 500 a day per campaign you get two purchases of three purchases per campaign which just gets obfuscated by the

14:39privacy threshold which means if you run a 500 already campaign you're probably getting purchases you're just not seeing them uh which is better than a world where i'm not going to purchase it so yeah are we are we back to the old days of um of half your advertising budget is working you just don't know which half yeah yeah yeah yeah yeah it's very much true yeah i was gonna ask so the pullback on the spend like is that do you know where that's coming from along the chain is that is that companies not being sure anymore and pulling back is it is it agencies is it all long because at some

15:17point somebody has to because i it makes sense that like well one we don't know how effective all this stuff was to begin with right so just losing the tracking doesn't necessarily mean it's less effective it just means we don't know um and so it seems a little foolhardy to just dial back right um yeah you know especially if your business relies on it but it seems like that's what most at least some percentage of companies have done they've they've pulled back just because they're not sure yeah i think i will also say a lot of companies that have pulled back have had relatively strong android products

15:50uh the couple of companies that i know that are doing very well on ios now they actually don't have very strong android products and they were like we don't have a choice right we don't have a choice we'll do what it takes uh i i obviously yeah i don't necessarily need to apply that strong guys will be rather short performance but having a fallback means all right we're going to take ios a little bit easy we'll transfer into android while we figure out what is going on did you get to your question i think a lot of that's coming from companies uh especially larger established companies that have had

16:25bi teams and reporting systems and dashboards that report on d7 pro at the creative level are like we just don't have that in our report anymore and they're spending too much to be confident in just yolo yeah yeah yeah yeah are you seeing any specific trends on on cpms and cross petrol or anything like that as far as with the drop in spend on ios and the increase on android has some of the performance on ios not degrading been more to do with market dynamics changing versus it actually just working as efficiently yeah uh you know i i try not to look at cpms just because cpms are very

17:14contingent on the kind of optimization you have like you know in april if you had value optimization you would be paying cpm that went through the roof and your cpms on audio network would be higher than instagram and facebook and the cpm is a little bit of a fuzzy metric what i like to look at is really the cpa which is a cost per trial which also has become fuzzy because of the privacy threshold so now it's blended cost per trial which right steady now back to your question to your underlying question about do i attribute that to the underlying market dynamics definitely i think that the fact that

17:51there's less of competition i do think has contributed to uh the seat blended cpa being steady for folks that have continued to do ios definitely i do think that uh the lesser competition has played into that yeah that makes sense let's talk a little bit because this is kind of our wheelhouse at uh rubycat obviously um nice shirt by the way jacob this is the original first ever revenue cat t-shirt nice how how are subscription apps being impacted in in what you've seen and then how is that different from your games and other categories that you're working with yeah uh i would say subscription apps are hit much much less hard than

18:38a lot of games again uh i just qualify i don't want to sound like i'm grandstanding because this is not like a party or fiesta yet but i think they're doing better than folks who are really clueless i don't want to be clueless but folks should just are struggling you know i talked about you know let's just say a hypothetical casual game or a social casino app that has a cost per page user of 150 they get killed by the privacy threshold a subscription app is less impacted by that uh just because uh you know again your cost per trial which is the primary metric nearly every subscription app optimizes

19:24for is generally under fifty dollars which means for the same five hundred dollar budget you're getting uh you get uh you're getting ten purchases so you're less susceptible to that privacy threshold right and the other factor that makes the whole 80d tank a lot easier for subscription apps is that nearly every subscription app i know has 90 plus of their free trials happen within the first 24 hours of install what that means with in the att paradigm is a lot lot nearly all of that signal gets captured by the att algorithm by by sql network algorithm uh because scad network works of a system of timers right uh

20:17immediately after install a timer start and after 24 hours the timer resets if your and then another resettable timer starts and if no event has happened in that second timer all of the events that have happened first get sent back to the uh get sent back to apple no other events get sent further and i'm probably definitely grossly simplifying with some of this uh and i have a youtube video that goes into all of this into the v6 people can check that out but my point being the fact that all of the tru nearly all the crimes happen within the first 24 hours make it relatively easier for

21:01subscription apps to have the signal be captured by sql network that's another reason why subscription apps do quite well uh and obviously you know the most signal you give to the algorithm uh the better the algorithm performs right uh you know in the pre-att world if you give more purchases the better the algorithm performs obviously the algorithm would take purchases from you and everybody else in the world and it would just do better now obviously it's just taking your trials and doing much much better and another trend we've seen is that web-based flows work a lot better for subscription apps than for games again there are challenges and execution

21:46but certainly one of the things they've seen that a lot of the relatively most successful apps are doing in the post http world is adopt web-based lows uh right yeah so i i would say those are some of the factors that i can contribute to subscription apps being better off than games in the post att work uh again not to grandstand not to declare this a victory yet but i think that they're much much better off yeah there's there's still also just the dynamic with consumable games like i don't know what retention curves really look like and stuff like this but with subscriptions you know your acquisitions you're making

22:24today only affect you know a chunk of your revenue in the very short term versus and you have this like recurring user base versus consumables if your new users dry up really fast like you're suddenly you've suddenly lost a lot of uh yeah a lot of your business model doesn't work as well right so yeah um but wow that's incredible that the so on those cpi's uh for like social casinos or whatever which i imagine is just a high spend category highly competitive space so if they don't have like value attribution what's actually driving the cpi so high like how do they know like what users to

23:00spend that much money on is it just is it just i guess click base like they still like they can they can proxy and know like people that click on those are part of that high value group or or what what what keeps the what keeps the targeting good enough so that you know because you can imagine if everything was perfectly anonymous all cpi's clicks would be the same right across all apps yeah yeah yeah at this point i don't i would not say we have a perfect answer for us with high cpa i think the best we have right now is treat facebook reported metrics of platform

23:36reported metrics as directional which means your cpa today would not be comparable to your pre att cpa but because it's going to be very very high just because of the privacy threshold that i just described that if you're getting maybe 500 cp on this and 700 on that then you just in for campaign is better than campaign fees but you're not inferring that your that is your actual cost of acquisition so you're taking the cpa as a relative measure and i think that's true for the casual game as it is for subscription apps you're treating the cpa as a relative measure between campaign and campaign not so much as an

24:16absolute measure of unit economics i think that's a great point to transition into what's actually working right now so we're talking about some of the impacts but um i mean what you're kind of hinting at is something that you've mentioned before is that the the best source of truth now is not these highly specific return on ad spend calculation but actually using blended metrics so tell me a little bit about how how you approach thinking about blended metrics as a source of truth versus you know the past you know five or six years where it's way more focused on um very detailed return on ad spend and

25:01again to our earlier point even if that return on ad spend calculation wasn't actually as accurate as it seemed you know you were at least able to calculate it more accurately now it's like kind of everything's out the window so how are you approaching blended spend or blended metrics uh to measure these things yeah i will add the caveat that the blending metrics isn't like modern or new right it's what old school offline advertising worked with right they were like oh this is how much i spent this is how much i made that's how they measured everything before the internet and even with the internet like

25:41a couple of companies we worked with even pre-att worked with blended metrics because they're like look we know that a portion of our paid installs drive organics we have a very very clear correlation between updated organic so we've been we would be leaving money on the table if all we took into account with paid users and not organics so you know people we work with have definitely done that and they're companies that have done it just to pursue top-line growth they're like look we need to grow as aggressively as possible and the way to do that is to take blended metrics to justify the growth rather than to shackle ourselves

26:20by they take uh just page robot without your question how how practically how do we sort of look at this it just basically means we'll instead of saying uh this campaign gave us uh a d7 return on ad spend of 20 or d7 cost per trial of 30 you're basically saying your overall marketing spent gave you a a 20 cost per trial across paid and organic and social and everything that you're spending on right obviously that means a lot of people tend to be uncomfortable with that because they're like oh you know if i hadn't spent on marketing i would have still gotten 200 trials a

27:10day uh and i'm giving credit to marketing for that and you know i don't have a direct answer to that but i think that our answer really is would you want to be would you want a model that's helpful in helping you grow or would you want a model that's accurate but is it helping you grow and i'm not going to clean i have an answer to that one right but uh yeah so you're basically looking at your total number of trials and your totals spent uh and obviously this calculation becomes heavier if you have multiple channels if you're running google facebook map and multiple ad networks then you're

27:51like oh you know my one of the ad networks probably performed badly but my total blend has not changed all that much because my other channel was like ten percent design stuff so there are challenges especially at large levels to spend but it is a very solid source of truth especially for smaller advertisers who may be on a handful of channels because it comes out of your ideas it's what you see in the bank and um you also mentioned that um sending people you mentioned webflows are working really well and i assume what you meant by that is sending people from an ad into onto the web instead of onto the app

28:36store which is it's really fascinating to me on multiple levels because you know the app stores have always been this black box where you you put a certain number of of clicks into it and then you know you see the end result but you don't see any of the steps in between i mean you have some basic metrics with app store analytics and stuff but with the web i would imagine that that gives you a more direct trackable um link from somebody who sees an ad to then actually kind of what they're doing on your website so but then ultimately i've talked to a lot of developers who talk about how on the

29:20web their conversions are actually quite a bit lower than in the app because apple's made it so easy to use in-app purchase so but it seems like maybe there is somewhat of a balance there is that maybe you lose fewer people from having to jump through those hoops of installing the app before they even get to the onboarding before they can be shown um you know the value proposition and then being you know shown a subscription page or whatever it is what have you seen working in regard to web flows and then and specifically for for subscription apps yeah yeah yeah like i said uh i've certainly seen a lot of

29:57success for subscription apps that have adopted web flows uh and a couple of apps i've worked with they've seen month-on-month growth through atg again nothing has a typical uh result but it definitely does happen but i think a couple of elements you know uh i think it's what's most important is to make sure that low is right and there's a couple of possible flows uh uh i think it's important to think to which one is right really let's figure out what's right right i think one flow could be shown that a user goes to a landing page which is basically like a b2b or a b2c landing

30:38page on the web or what you would get for a web product and you have a link that accept the call to call to action on the landing pages go to the app store so a lot of the experience purchase and trial experience happens on the app store the web page i actually does the job of selling the user on the product and it's my hypothesis that this actually works well because the webpage can do a much much better job of selling than the app store can uh while we're still making it clear that this is an app uh and while the actual conversion happens within the app itself

31:17but uh somewhat similar uh a different flow that's very comparable would be take a laser to an ad take a user from an ad to a landing page where users have to input their phone number which again get a user to make it makes it clear to the user that this is an app this is a mobile experience a user gets a text message and user signs up for it uh yeah and and when they click on the text message they get to go to the app store and download the app right uh again another model could be a user clicks on an ad goes to an article or a

31:54content page which is what you would see if you had a taboola or not brain ad again article or a content page leads to an app store and i think the last and i think the more complex sort of flow is just to have on board them on the web uh basically take them to a web page and they on board on the web and uh to hopefully also actually make the purchase on the web in which case you save the 30 this is to be honest the hardest and the most resource-intensive and really it's my recommendation that you pursue that path after you pursue one of the other

32:35paths that i recommended because you don't want to invest a ton of engineering and development time in building a web onboarding flow if you don't even know that the web flow is going to work for you uh so i would recommend just testing the web landing pages first and then using the web onboarding so but uh yeah i think those are some of the most important models that we see work something else i think that's also very very critical is i think a lot of people when they look at a lot of advertisers i know that have started on the web for the first time were like oh let's

33:11put together this nice landing page that looks like our home page uh on our website and just put it out there that doesn't work very well unless you're being very intentional about what value propositions to touch on the actual layout of your landing page and we have a structure that we use now which is like your most important uh page point and value proposition goes to the header then there's social proof then your most important emotional benefits then the teachers i think the most successful advertisers we work with are very very intentional about what that web page is looking like and they also test rigorously uh i think

33:54that all of those elements are very very critical to making the web flow work yeah that's really smart and i hadn't thought of it quite that way about how you know and as i was talking with the app store being the black box is you you're just sending somebody hoping they look at the screenshots hoping the icon resonates with them hoping the title and subtitle are meaningful but when you send them to the web it's not just about you know pushing them right to subscribe on the web but it's actually just having a better opportunity to communicate the value prop so that by the time they get to the app store they

34:27are they have a much higher uh incentive they have a higher um they're just more likely to actually take action by the time they do get to the app store so that makes a lot of sense tells you a lot about the uh quality of like the app store as a sales pitch right yeah i guess when you're like looking at a you know you're trying to differentiate right and and there's only so much you can communicate in a block of text and then a bunch of screenshots right you've seen so much data shoved into the screenshots on amazon right they're not screen shots right they're like a slide deck yeah little

35:00billboards yeah yeah yeah yeah i also think another reason why the app store works so worked so well pre-att was facebook would just show ads to users to install other subscription apps so if you send them directly to the astro they're almost pre-qualified which is great anymore so i think the webs level the field uh a lot yeah it's it's it's a tough um skill set though for a lot of developers because like they don't often have web experience internally and i think i think i hear so much like people get so obsessed about the 30 um and they want to jump straight to that last one you mentioned about

35:44building a whole online purchasing thing which like you know stripes pretty easy to use like it's it's you know it's not that much more work than building a landing page but you have to remember um okay management so now you gotta have a link where somebody can go and cancel that thing now you also have to worry about taxes stripe doesn't like to collect uh tax information for you already you have to you know then synchronize that with your back end and you know if you're using revenue casinos with us or whatever but you got to manage all that too it adds a lot of complexity um for 30 right and when

36:15you're just trying to you know all of these things can find incremental growth yeah but like as you're saying it's important to put them in the right order or you can end up wasting a lot of energy and money yeah yeah yeah well i did want to to move on to the um the future so we we've kind of gotten through the first couple of months of this uh this rough patch and are into this um new era of of mobile advertising are there any things that you're seeing that are especially promising that you think are the future like is the future everything we've been discussing so far

36:57of just 50 of your advertising works and 50 doesn't and you're never going to know which or do you or are there some technologies coming online or some approaches that that are just going to take time to kind of work out yeah i think there's going to be some changes i don't know if these are going to be earth shattering uh in terms of changing what's necessary normal with atp i think the most promising though i would say uh ios 15 custom product pages uh basically you know it solves the problem with gk viewer and during the just you have one small slide deck and everybody sees the same thing and astro

37:39does a terrible job of telling a user on what the product is basically with the custom product features you can have up to 35 versions of your app store uh which means you're like if you're a you know wellness app uh if let's just say you're like a meditation app that has uh meditations for sleep or anxiety and how to meditate you can have a separate landing page so to speak on the app store for sleep anxiety meditation right uh and you can send you get a unique url for each of these so presumably you can have an ad for sleep going to an app store for each sleep as

38:23for anxiety going to an app store for anxiety i think that can help i just don't think it's going to have too much on the measurement front obviously the actual execution is still unclear the announcements out so i think definitely one of the bigger changes i would say that's coming with rs15 the other other one would just be that uh advertisers are going to be receiving postbacks which is huge uh at least an ensuring accuracy of the advertising data so far what's completely bonkers right now is networks like facebook snap everybody gets your post back from s-cad network but you as advertiser you as an advertiser

39:08do not which means you basically take the word for it uh i do know for a fact that tick tock has actually changed values i don't want to call it malicious because the conversion value was null and they changed it to zero uh the problem is that null and zero have very very different meaning zero means install and null means an install happen and there's just no value so i know that they did that change i don't have their continuity to do it so uh but my point is and google explicitly says we are going to use model conversions which basically take our word for it

39:47facebook says campaign data is accurate everything underneath is modeled which means take our word for it with which all of this is because the post pack goes to the networks but not the advertiser with ios 15 if the post back goes to the advertiser you can at the very least verify they're telling the truth with bonkers considering i think david you imagine uh until all this time you you just have to take the platform's word for it even pre-att uh right uh i think that's going to be that's going to be a big big change even though a lot of that's going to happen under the hood and i say advertisers for

40:30the vast majority of advertisers the mmp is going to do the receiving uh of the uh postpartum but i think that's going to be a big deal but i think those are some of the big changes the custom product pages and the postdoctoral advertisers but really in terms of the future in many ways i do think it's going to be

40:48back to 2013 or 2014. i think i talked about how we bought a number of installs and we hope that in a certain percentage well we knew that certain percentage of them would convert to purchases that would be for subscription so there weren't really trials but i think it's going to be a very similar world we're going to be you're going to have to be more comfortable making decisions based off of incomplete data but i do see that in the future one of the things i've been hearing a lot about since since apple announced att last year is incrementality testing so systematically turning on and off you

41:26know so if you're advertising i mean obviously this would be a tool for for larger apps but if you're advertising across facebook google snap tick tock and you know other mobile dsps you know systematically moving spend around and then measuring the difference or even turning spend off in certain channels and increasing spend in other channels um have you seen that work um and are you um excited about the potential of having tools in this space or or do you think incrementality testing is a bit over hyped so i think with any recommendation like incrementality i think the one caveat that a lot of people miss is

42:10that it's useful for a very very tiny fraction of advertisers david like you said if you are on like all cell solubility networks multiple dsps multiple ad networks influencers tv yes absolutely you know you should use incrementality because there's just no way you're going to find out if this is going to work an incrementality and medium x modeling you want to use both of them and to make that work but i would say the kinds of advertisers who need something like this are a very tiny fraction so the vast majority of advertisers even the advertisers who are on four to five channels even advertisers will spend low six

42:50figures in uh monthly spend i don't think uh incrementality or testing is going to be very helpful or actionable just because it's hard uh you know it's it just becomes imprecise at small volumes of data you need a critical mass of data for your analysis to be useful uh right and i think it's a very similar thing with vdmx models right you need enormous enormous budgets for those to be useful and helpful so i i do think these are great but i think the fact that they're not an antidote to all of the havoc that att has uh brought us about they're applicable to a tiny flavor of

43:38the advertisers out there yeah that makes sense and then if if you're only advertising on google or only advertising on facebook or only advertising on the two of them i mean they're they're essentially doing some level of incrementality testing for you right they're they're measuring the performance of this campaign against that campaign and they're ramping up depending on the results they they seem to be seeing so that some of that's kind of already covered if you're using those platforms uh as your primary sources um another thing i i wanted to get your thoughts on was experimentation with other forms of advertising you're you're very focused you know currently on on

44:19um you know paid user acquisition and i don't think that's going away and i think for you know for a lot of apps that is going to be the the the best most reliable way to continue scaling even without accurate measurement but have you seen any other uh pushes with any of your customers um to work on on on different styles of advertising different uh approaches to marketing that are being successful and do you see they're kind of more incentive to try more things these days you know i spoke about web and i think there's definitely much much stronger interest in web campaigns than even six months ago

44:58uh at larger budgets there's definitely stronger interest in influencers but i would again uh much like with the incrementality i would say if you're on smaller budget i do not recommend experimenting i do not recommend diversifying certainly at larger budgets and i would also say folks that are that were spending in the tens of millions in budget folks like that would have been or even like millions a month uh they certainly found the larger studios they have already been on influencers they've already been advertising on tv but none of this would be new to them yeah so i don't know if there's anything radically new that we see yeah and then that kind

45:43of gets back to the the good old tried and true just got to build a good product and work on your monetization and kind of get back to the basics of product as well i think sometimes these these overly complex overly targeted systems especially for people who make software can tend to be busy boxes right they can tend to be they can tend to be things that like con attract our attention and and because they they seem very like you know oh we can get it right and really make it scale and some people have right it's possible but 80 20 i think for a lot of people out

46:19there like just just just focus on the fundamentals and you can go pretty far and then as time comes you can layer in the more you know yeah yeah complexity and precision yeah yeah and like i said at a certain level of scale tv influences all of this becomes much much more meaningful uh and like i said that's certainly more meaningful already but yeah you don't need too much that's fancy well i think that's a great place to wrap up uh it was great chatting with you and and uh a lot of insight there on what's working and and what how to think about things in this this new world of mobile

46:54marketing um you know as we wrap up is there any uh last thoughts uh we're gonna include links to uh uh where people can find you on on on the web and to rocket ship hq and whatnot uh anything else you wanna share oh no as well like i said i think this this was not as bad as it as we thought it would be uh certainly there's ways to mitigate the worst case scenarios uh some of which hopefully i've been able to share uh so uh hopefully they'll come out on the other side of all of this without too much craziness i think people are

47:36gonna keep using apps that's my that's my prediction and i think people are going to keep advertising apps yeah yeah yeah yeah it's the how that's going to have to change and it has to change dramatically and there's no getting around that well it was great chatting with you and uh we'll talk again soon absolutely later yeah thank you bye to make sure you never miss an episode subscribe to the show and your favorite podcast player thanks so much for listening until next time [Music]