2 OpenAI tools combine to make Jarvis

0:00OpenAI just released real-time voice directly inside Codex, and this is the closest thing I've seen to Jarvis. And this is actually just a voice [music] agent that connects to all of my tools, can do basically all of my work, build and edit applications, [music] and fully control my computer. And arguably, the coolest part about this new feature is you can use real-time voice on the ChatGPT app to control Codex [music] on your computer. And I'm going to show you

0:26exactly how to do that. Uh, no, I'm going to show you. In fact, multiple people showing you how to do it, all the different features, and use cases to show you how powerful it is. And it's actually more powerful than what he just said. You can get it to control different apps remotely from

0:39your phone. You're going to see it all. Let's get right into it, cuz boy, I've been watching and using so much of what I learned. Let's get into the first one. Presented by Zapier, the AI automation company. Here is a specific use case. Email in the Codex app or ChatGPT app, I'll use the names interchangeably,

0:55within their browser. Hey, buddy. I need you to look at my drafts in my email. I I created one a little bit ago. Uh, sent to Emily. I need you to pull that up in the Codex browser, please. Sure. I'll find that Emily draft and pull it up for you.

1:14Takes a little while. There it is. All right. So, thank you. Um, you pulled it up for me. Thanks. I'm going to go ahead and send that. Another thing Okay. That's really helpful, and I find that it's most powerful when you're voice to control the Codex ChatGPT app and the browser that is built inside of it. But, I don't [snorts] love the browser inside of it because I don't have all the plugins and a couple of other things are missing for me. Still,

1:40very good use, and uh, let's keep going. I want you to see this. This is uh, Greg Brockman, OpenAI's president, talking about how much audio and how much uh, voice matters right now. Look. Is the goal, though, that we are sort of talking to our computers, and we don't really need to type anymore. I think that we will shift [snorts] to that as by far the vast majority of what

2:03we do. I will say for me, the vast majority of my typing now is done with voice and it feels weird and slow to to type in. I'm now using my voice to control apps more and more. It is not the vast majority for me. I would say it's about 20%, maybe 10% and I'll give you some use cases as we go along here. But what I'd like to know from you is, I know some of you are super smart, far further ahead even than I am. Tell me in the chat how much voice are you using and where do

2:28you see this going? Okay. Um, let's go how do I access we can access this with a couple of caveats, so make sure to stick with me. First of all, if we come to chat.openai.com, we have the voice mode right here that we could activate, but this does come with some limitations. And we'll get into limitations, but I should say if you this is basic for you, I've got all the chapter markings below

2:46so you can skip ahead. that I'm going to talk about here in a second. Now the next place to use it and I highly suggest this over chat.openai.com is the chat.openai.com desktop. That is inside of the desktop app. On top of that, we can also access this on our mobile app on our phone, which comes with some pretty cool capabilities, which I'm going to show you later on in this video. Now let's quickly talk about the difference of using chat.openai.com versus the desktop. So for using this on our phone or the web application, it does not have the ability to run different tasks for us and use our actual screen, pull

3:17information from our computer, etc. Think of this like using normal chat.openai.com mode, you could talk back and forth with it and the model sounds really good, but it doesn't have the full capability. Whereas in the desktop app, we can run tasks and specific agents and basically do everything that I'm going to All right, let's get into it. Now that you see where it is. Oh, by the way, my sponsor is Zapier MCP. I like Zapier whenever I connect any agent to my email, to my notion, to my more sensitive data, I can use Zapier MCP to select and again, it's just checkboxes where I could check and say I want it to

3:47have access to read my email, to draft my email, to maybe add labels to it, but not to hit send on it. And I have access to over 9,000 apps. To be honest with you, I only connect like 11 apps total to it, and then give those 11 apps to an agent or maybe eight, and then give those eight

4:03apps to Hermes agent or whatever it is. But it gives me the safety that I want and the power that I want all my tools to have. It's one of many tools from Zapier. If you use my Frankly, I don't have a URL. Just trust. If you use it, let them know that you heard about them

4:16from me. zapier.com/mcp. Try it out for free. Let's keep going. All right. Now, I want you to see multiple actions together using voice. One thing that's really cool about this is I can close the side panel, and I can just enter this full screen mode. And are are you still there, Codex? Yep, I'm right here. Very cool. Um can you just actually turn

4:39all of these into Don't use any names. Just create a new Google Doc for these, and open it up in a new tab in the browser. And And um just so I get a good summary of the issues going on uh with our product. Sure, I'll take care of that. Did you notice a little jump cut? It's

4:58because this takes time. All right. And I love that I can just see this right here. It's creating file, thinking, getting document, and I don't even need to have it open. I can just kind of have this real-time voice just kind of like hovering over everything. Again, it's vamp in cuz it takes time.

5:17to get used to, but I think I will. And there it is. So, you worked in one app, had something else done in the background, and a lot of this background app stuff I can't really show you. And most people Not most. I haven't seen a good demo, and I've watched now dozens of videos on this. You should just know you can ask it to do things in the background. It doesn't have to be stuff

5:37that you see up front. Okay. So far we looked at it in the app itself, like in the Codex ChatGPT app taking action. Let's break free of that uh with Brock. Now, one thing want to mention that's pretty cool is you can see this little chat GPT voice blob and we can basically put this anywhere on our computer. So, let's say I At the moment I will say I can move it anywhere on my computer, but it you can't properly move it from screen to

6:00screen. I'm editing this video right here for example. You could still see chat GPT voice mode and I could just basically come here and say, "Hey, I want you to look at my screen. What does it look like I'm working on right now? I need help with this." Okay, I'll take a look. I'm reading what's on screen. Did you see that? It took a screenshot of it and that's the difference when you're working with apps outside of the

6:19Codex app. It can't see it in real time. It takes a screenshot when it needs it. I know. You're actively editing a roughly 7-minute project in CapCut with the clips basic settings open. Yeah, could you actually break down how I can apply filter? I can't figure out how to do it. I'll check that real quick. In CapCut, click filters in the left sidebar, then choose one and drag it onto your clip in the timeline. To apply it across the whole video, drop it onto the main video track then adjust its intensity in the

6:47right panel. helpful. Really helpful, very practical, any app. It's more pow- it's more powerful when you tell it to do stuff, but sometimes like this you actually want to know how how um how to do it yourself. It at times requires permissions which drives me crazy and um it can be a little slow, but it's great for just doing it on a separate monitor and then

7:07moving on to something else. Okay, I love this guy. This guy uh Harrison Painter. He's got like an 8-9 minute video and almost the whole time he's got his hands up here like and don't shoot me to show you that he is not using the mouse and tricking you and doing it himself. It is Codex doing it. Uh but here, I think this is helpful

7:25because launching well here. Perfect. Now, I want you to go into the bookmarks, find the socials folder and open LinkedIn. Sure, I'll get that open. Hands up in the Chrome is open and I see the bookmarks bar. I'm finding that folder now.

7:52[snorts]

7:57You see it take a screenshot? All set? LinkedIn's open from your social All right, and then he has it write a LinkedIn post for him and create an image and attach the image and I've got a link to him and everyone else if you want to watch more of them below. Um if I showed this to some of my friends and if you showed it some of your friends are going to say, "Well, can I just go and click into the browser and the bookmark and isn't that faster than waiting on this?" Yeah, that's not the point. The point is that if it could do it and it could do it in the background,

8:23you could string multiple actions together, have them done in the background and in a moment you're going to see that you don't even need to be on your computer to do it. And let's keep going. Posting a tweet with something else stuff. So, if I log into a different account, as you can see, we're going to see if we can actually just like do something useful like posting a tweet

8:40for us. So, let's try this right now. Can you post a tweet about the power of ChatGPT voice for me, please? Checking that. I'll draft something tight and show it to you for approval before posting. Something like ChatGPT voice is like having a collaborator. So, just um just post it. Just go on to X

9:02and write it right away. I've drafted it for the AI Playbook account. Say publish and I'll post it. Publish. Posting it now. Perfect. Done. Perfect. It's submitted and Can you just open up as well? Opening it now. It's open in Chrome. So, I can actually post tweets straight away as you can see here. So, it posted his tweet, pulled it up, control my browser whilst I was talking to you. The only thing that I don't like about it is there's still a little bit of a delay, but I can understand why, especially if It is a delay. I'm going to keep on

9:34moving here. Um again, it will work outside of your computer and it will even access files use case number two, I'm going to use ChatGPT voice mode in order to find a specific file that's living on my computer. So, I generated this a couple of days ago and let's say we don't want to have to go and like try to search for this all across our computer. Instead, let's offload this task to ChatGPT in order to do it. So, as you can see right here, it's living on my desktop, this co-work setup teaching HTML pack that I have open right here. So, I'm just going to come over to ChatGPT and have it find

10:03this for me. Hey, can you go on my computer and I want you to find the ChatGPT co-work setup HTML that you generated for me a couple of days ago. Got it. I'll look for that file and confirm what I find. Also, while you're while you're doing that, what was the last task that we

10:18were running inside of ChatGPT? I'll check that, too. Now, as you can see, it's running and doing multiple tasks at the exact same time, which is the power of using the new ChatGPT voice mode and it's pretty cool when you have multiple different tasks you have it running at once. I found and opened the simple cloud co-work setup teaching pack on your

10:36desktop. It was created July 17th. Yeah, this is exactly task This is exactly what I was wanting. Thank you. Great. Glad it matches what you needed. Now, what was that task you said that you found like the last task you ran for me? Let me pull up the exact last task for you. It was the voice mode update chat where you asked if the new voice mode had launched recently and then to see

10:57your most recent Gmail message. Perfect. I also like that example because you can see that he was interrupting a few times and it felt natural. It didn't feel like you were stopping and having to start the conversation from scratch. Now, this is I think what makes this really exciting that you can do it remotely. In fact, I was in Iceland recently and I wanted um a change done to my Hermes agent desktop app. I couldn't get to do it myself. I have remote desktop connection using Jump and it still wasn't working for me and I said, "You know what? I'm going to have Codex do it for me. I'll tell Codex

11:29what I want, which is to layer in these different models." Anyway, it doesn't matter. And it did it, and I could just speak into it and ask it. Here's how that works. Chat GPT desktop app. Then at the very bottom left where you see your account, you're going to click on this, and then click on settings right here. From here, underneath coding, you want to click on connections. And then we're now going to see this right here where we can control our Mac from our phone. So, now we're going to click this add button. Now we're going to see this QR It just occurred to me, you could

11:58probably use Now, he he got rid of that QR code, but it would be pretty funny if you just used that QR code to control uh Brock's computer, don't you think? Let's keep going. code right here. I'm going to pull up my phone And when you do it, you just use the standard QR reader on your phone. phone and then scan this. And then now inside of the Chat GPT app on our phone, what we're going to do is we're going to come over to the left-hand side, swipe to the left, and then you're going to see this new remote button right here with this little green icon. I'm going

12:22to click on this, and then now you can see the different projects that we have that we've ran in the Okay, what I hadn't seen anyone talk about but help me was I could actually connect multiple computers. If you look under the word remote on his screen, there's a name of his computer, which is

12:38MacBook Pro 5 local. You could switch it and have access to controlling one computer then another and another still, and that again helped me because in Iceland I had one computer that was running on my desktop here in the US in the great state of Texas, and another one that was in my backpack that I never actually took out. But if I wanted to, I could control them both, and now it's really helpful that I'm back. Uh is there anything else that I want to show? No, that's it. Let's go on with this next one. Okay, [snorts]

13:04let's talk about usage limits. terms of usage, the $20 plan, the and the plan that is called plus, um has a 5-hour window of actual usage of 15 to 30 minutes, okay? Um the Pro 5X, the $100 a month, uh has a 5- hour window of about you can use it for about 1 hour to 2 and 1/2 hours. And if you have the $200 plan, plan, you have unlimited voice access. But All right, I was surprised that there were limits. I think he's the only one who showed that. Um, one thing that I tried to do was keep it running the whole time and I told it only start

13:50activating when I say, "Okay, Codex." or "Okay, ChatGPT." Whatever the phrase was. And it was very good about letting me dictate and ignoring it, letting me talk to other people and ignoring it. But every once in a while it would just barge into the conversation, which was frustrating. I'm telling you this because I would like the idea of just keeping it active the whole day and only invoking it when I'm ready for it. Um, okay. Let's go on. Oh, and uh, from my research, I should say these are not hard limits. There's something about OpenAI where they they like to be a lot more flexible than Anthropic, for

14:23example. Next. [snorts] Limits and safeguards. Workspace. You can't swap my model from inside this live conversation. That's set by the task configuration. And I also can't silently take over apps, send messages, spend money, change permissions, or access private data unless you ask. For longer research or building docs or code, I'll often spend Wait, so really tap into that like, um, you can't send messages unless I ask. What does

14:50that mean? It means I can draft a text, email, Slack message, or social post for you, but I won't actually send anything unless you explicitly tell me to. Mhm. No matter what. Okay, so it'll always make sure, no matter what. Like there's no way I can create a skill which bypasses that. Checking that.

15:12No. Right. A skill won't override those safeguards. And that seems like kind of a goofy question. It's already doing so much, why would you want it to do more? And wouldn't you want the safeguards? And I have to tell you the more I use it, the more I feel like, "Ugh, I've got to give you more permission." It feels too amateurish and basic when you're asking it to just connect the iPad that's sitting over here to this computer so I can use it as a second monitor, and it keeps coming back every time I do it day after day and saying, "Can I get this permission?" I just want to give it

15:41permission. All right, next. Let's look at voices. This might be simple, but I think it's important. And that sort of thing. The other thing you can do [snorts] is you can go over to the voice section inside your settings here, and you can change the voice. So, you can switch which microphone you're going to use. Then we can move the pet around, or we can even hide this if you don't want to see the sticky pet with the details of your voice. Um you also get these sticky notes where like it shows you, okay, here's what it's doing and here's what it's working on. So, you can click back

16:12to that and just go back to that whenever you want to. And then also, if we go to our settings inside the voice section here, you can see that we've got voice, and then we can select between multiple different voices here. Hey, how's it going? Hey there. I thought Hey, I'm I'm going to pause it. One thing that he did that I thought was interesting was Julian said, "Can you talk Think what it was was like 20% faster?" And ChatGPT

16:34said, "Yeah, I can. I'll do it." And then it turned out it didn't do it. It promised, but it didn't. I like that I get to pick my voice. I think this is really important. As you can see, I'm I grew up in [clears throat] New York. I have a like a fast-paced everything. I I want more fast-paced. I like that I can pick the kind of personality I want. I wish I could pick a faster pace. I don't need this slowness. Okay, let's move on to the next. And um this one is building an iOS app by voice. This is what we're going to close

17:00with because I think this shows a lot of the the strength of what you can do. And by the way, if you like this, I'd love it if you subscribe because I've got more videos like this. If you don't, I would love your feedback. The negativity stuff is helpful for me cuz that's how I improve. And of course, subscribe if you want to see more, and hit the like button. And I say all that? Let's move

17:17on. Hey, um can you use the build iOS app skill and um I want to create an iOS app. Um can you please start create uh yeah, I want you to create a Swift app that is like an exact replica of the Apple notes app and I want you to run it in the in-app simulator. Okay, in the actual video, which of course I'll link below, he says that and then he also says and build this other app and another app at the same time. Um and then he comes back a a while back later and shows the result. I just want

17:45to jump to the result for you. And the simulator preview is open. Okay, yeah, can you show me just go ahead and use the app for a bit. Test out the features on this. I'll do that now.

18:00It's wild seeing something else control a mouse. You know what? Um let's continue to make some styling adjustments. I actually want the default to be light mode. Change it to light mode and um please make the bottom tab like the liquid glass tab at the bottom higher quality, please. Got it. Making those updates now. Wow, this is just truly insane. Thank I think this is fantastic. I think it's incredibly helpful. If you got anything of value out of it, let me know and if you like this style where I show you the best of multiple videos on a topic, I've got one right here for you to watch

18:38next. I'll see you in that one with uh with a buddy of mine. We're both recording it.