How to Run Your Own Local AI Model with Hermes

0:00Is running local agents and local AI models actually feasible? You've probably seen a lot about frontier models, Opus 5.5, Fable, GPT-6 Astra, and I personally tested some of those myself. It is mind-boggling to see what those models are becoming capable to do. But in this video, I want to try and test something different. I want to share this with you guys. What kind of work can we do by combining Hermes agent

0:22and a open-source local-run AI model? I'll also show how to download and install Hermes agent and your first local AI model using the simplest way possible. You don't need technical knowledge at all to follow me in this video. So, if you're interested in this kind of content, smash the like button, subscribe to our channel, and let's get started. Starting the process from scratch, and that includes this brand new machine you guys are seeing right now. If you've been following my content here, everything I recorded and shared with you guys was using my MacBook Pro with 8 GB of unified memory. And a long

0:53time ago, that used to be something. Nowadays, with AI and especially for running local models, this AI gigabytes of memory means absolutely nothing. So, because I study AI a lot, because I have an AI agency, and I create automation for businesses, and because I wanted to be on top of everything related to AI, I decided to buy a brand new machine. And after a long research, this is the machine I chose, the Mac Studio with the brand new M5 Ultra chip, 96 GB of memory. This will allow me to share with you guys some cool experiments showcasing what local AI models are capable of doing. But don't worry, you

1:29don't have to spend a lot of money and buy this crazy machine to follow me on this video. You don't even have to worry about what kind of AI model your machine is capable of running. All the heavy lifting is done by Hermes agent. Hermes will read your machine and choose the perfect model your computer is capable of running. So, the first thing you need to do is open your browser. Then, you search for Hermes Desktop. Usually, the first link is the one you have to click on, and this will open hermesdesktop.download page. In this first button right here, you actually have your system automatically identified by the website. So, you just

2:04need to click for download. If you don't see your system correctly selected here, you can just scroll the page down and here it is. You can download it for Mac, Windows, or Linux. Now, you just have to install Hermes. You need to come to apps and click on Hermes. Then, you click install Hermes. This is the wizard. It will install, download everything your computer needs to run Hermes efficiently. The installation process will do everything on its own. It should take around 10, maybe 15 minutes depending on your machine. And then, once this is completed, we can finally open the desktop app and install a local AI model. Hermes is installed, and as

2:40you guys can see with this brand new version they just released, Hermes itself takes us through a sort of setup process. This process will ask you for, you know, preferences. And once you're done with it, you can come down to this welcome option. Click on it, and look at this. Hermes read my machine already, and it told me that Qwen 3.8 Flash Next fits on it. So, I can pretty much just click on setup. Hermes will do everything for me, download the model, load the model in my memory, the machine memory actually, and make it available for me to start using AI local models in this machine

3:18paying zero for tokens. Obviously, you can connect your ChatGPT or Claude's subscription to Hermes agents, but this is not the purpose of this video. So, I'm just going to click setup, and then I'll click setup for me. All right, Hermes is downloading now Qwen 3.8 Flash Next. It's not a small model, as you guys can see, almost 92 GB of, you know, total size. While the process is running, let's understand how we are able to run Qwen 3.8 Flash Next, which by the way has 180 billion parameters. It's a pretty big model. In theory, I would not be able to run this model on this Mac Studio because the Mac

3:58has 96 GB. I showed you guys in the beginning of the video. But, we can do that because this model is a mixture of experts model. If you don't know what that means, I'll explain right now to you. A mixture of expert model means that the model doesn't need all like the 100% of its parameters to function. It can load, you know, small parts of the those parameters depending on the question, on the prompt you sent. This way you can run larger models on your machine even though the total number of parameters the model has is superior to the available memory you have on your machine. All right, guys. We have

4:38finished the process and now Qwen 3.8 Flash next is ready and available for use with Hermes agent. So, let's start testing. And for this first test, I want to ask Hermes to create a model calculator. If you don't know what that means, we're going to pretty much replicate what Hermes did. Let's create a calculator to tell us based upon our machine, our hardware configuration, what kind of model can we run. This is the prompt. Jarvis, can you please build an AI model calculator that tells me what open AI model my machine is capable of running. I basically want to enter any machine info model specs and the

5:20calculator should tell me what AI model it is able to run locally. Build it in a single HTML file. As you guys can see, Qwen 3.8 Flash next is here locally and it has the effort level set to medium. Let's shoot this prompt and see how the model does. And by the way, if this is the first time you're running local AI models, the model first takes longer to respond because it has to be loaded in your machine's memory. Then it needs to process the system prompt, the prompt you sent, so it's able to deliver on your request. So, the first message, the first prompt you sent to the model, it

5:59takes longer to There you go, it started replying now. So, it takes longer to do all of those things to start, you know, actually delivering and replying to your request. In our case, it took 1 minute and 11 seconds. It replied with building now a single self-contained HTML calculator with a real quantization memory engine behind it. This first wait I just told you is in the first prompt only, the first message you sent. After this one, the model will reply faster because it cached everything it read before. So, the second, third, and, you know, subsequent responses will arrive faster. Let's wait for Hermes with quant

6:413.8 flash next to finish the process. Once we have the final result, I'll tell you guys how long it took, and we're going to check the results, the quality of the output together. In case you guys are curious to know how much the model is taken from the Mac memory, this is the memory pressure graphic. I don't know if you can see it. I'll zoom it if

7:02needed, but the memory used is at 92 GB. Meaning, we have pretty much the Mac Studio M5 Ultra with 96 GB of unified memory using it at its full capacity to run quant 3.8 flash next. So, in this case, I think Hermes might have overestimated, you know, with the quant 3.8 flash next. On the day-to-day, I'll probably install quant 3.8 27B, which is a smaller model, fits perfectly, it's very capable, and will not take most of the Mac's memory on, you know, automations and daily tasks. All right, guys, we have the first result. Let's check out together. As you guys can see visually, we have a decent job. I would

7:48say a good job, right? There is nothing broken. Obviously, we could improve a lot, but let's take the first result as the final result. If you more advanced on running local AI models and you're curious to know what engine Hermes uses under the hood, you can just hover your mouse here beside the microphone icon and you will see llama CPP is the engine Hermes is using in this case. It's a well-known engine largely used to do

8:16model inference and also fine-tuning. But for Apple's computers, MLX is the best option. It's more optimized for the Apple's architecture. Anyway, guys, let's test. We're going to select Apple silicon in this case. Let's just do a quick test, right? There is no M5. Okay, so let's select M4 Max and let I'll I'll tell you this this already gives me the impression that we are not going to have very much updated results. But anyways, let's let's see if that's the case. No GPU. I'll do 64K for the context length and let's calculate. We do have the results right here. A few options to choose from. The best comfortable pick

9:04is QN 3 32B Q6. Okay, we do have it at nine tokens per second, which makes the use unbearable, I would say. But let's compare with a live online calculator. And here he is. I opened this calculator, filled out, you know, the same information. According to these results, we have a bunch of other options to choose, even the more updated

9:29versions of QN 3.6, 3.3.5. We have a QN 3 30B, but let's see if we can find the exact same model. And here he is, quant 3 32B. We have the same thing, same quantization. And in this case, the tokens per second is even lower, 5.9 tokens per second. Hermes and the calculator we created with it uh gives us nine tokens per second, which is kind of close, right? So, as you guys can see, it does work, right? But we would be needing to improve a little bit more, do some more prompting here and there to adjust and have this calculator actually working and giving us better

10:09numbers. Let's go to the second and more complex test. I'm going to ask Hermes, using the same model, to create a 3D first-person shooter and deliver on a single uh HTML file. This is much more complex than the previous test. So, let's see how it does. As you guys can see, again, the model is being loaded into memory. It will read my prompt, system prompts, and whatnot, and then we'll [music] start creating the result for us. Same as uh we did with the first one. I'll get back with you guys when I have the final results. I'll tell you how long it took for Hermes with this

10:45local model to create the the result, and we we're going to check and test together to see what kind of 3D first-person shooter this model with Hermes is able to deliver on. All right, guys, the model is started working. It is thinking right now. It's been thinking for 4 minutes or so. And in terms of memory, we're going to check how much is consuming. It It is green this time, but it's pretty close to the the like the limit, right? Almost 88 89

11:14GB. It's definitely capable of fitting and running this model, but I won't choose this one for my daily automations, uh which, by the way, I will set up all of them, and then I will share with you guys in future videos. So, if you want to see these videos, please let me know in the comment section below. Hermes

11:32just finished, guys. The result is here. Let's check it out together. The game is called Neon Breach. It's a 3D FPS game. It took about 25 minutes to complete this one. So, let's see the results. This is the entry screen and we do have the controls uh laid out for us. This is pretty much standard for all FPS games. So, we just going to jump right into it and see how we do. Uh we do have like this platform here and a mini-map. Oh, we do have this mini-map right here. I can't point it right now, but you know, on the top right corner there is a cool mini-map. These are the

12:12enemies. So, as we approach, I think uh we can uh kill them. Okay. They They are Oh, oh, oh, oh. Oh, oh, my god. An ambush. [laughter] I got ambushed. Like the enemy wasn't moving, but [laughter] as soon as I got there, they squared me in. Here we go again. Like give me like, you know, cut me some slacks because

12:41Oh, gosh. Because Oh, life. Good. Because I have no mouse pad, right? And it's been too long since I've played games. I'm going to die here. Wow. I think I'm going to get ambushed. Oh, gosh. [music] Come on. Like the the the aim like the mouse is not moving 360, you know? It It moves like kind of See, come on. I can't move past like, I don't know, 270° or

13:18whatever.

13:25Come on. Move. Less enemy. Let's see if we can finish this phase.

13:41We cleared the wave the first wave. Oh gosh.

13:52Now, if I go there, I'm cooked for sure because they ambush me. All right, let's try to kill this one first.

14:06Move. This couple of tasks were just to give you an idea what's possible to do with local AI models. Obviously, we can, you know, explore a lot more and I'm planning to implement Hermes agent to perform various tasks inside my agency. If you want me to share the results of these tasks and experiments that Hermes will take on from now on, let me know in the comments section below. I do believe we are walking to a world where everybody's going to run local AI models as the models, the engines, and also the hardware is improving. Uh we'll have AI on our watches, phones, computers, but we are quite not there yet. Obviously,

14:51the advantages of running local models are immense. The two main ones is privacy, you know, all of the information you're sharing here with your local model running on your machine is not being sent elsewhere. So, it gives you the control of the information. You can share sensitive data without freaking out. And the second biggest advantage, in my opinion, is cost control because you don't depend on AI companies to dictate how much you'll pay monthly for your automations or your AI agents. As for timing, I would say 95, maybe 99% of the people in this planet would not benefit now from, you know, investing in local hardware,

15:32expensive the way it is [music] right now. So, I would definitely advise on, you know, waiting for maybe 1 2 more years and see how the things change, right? But, if you do run business that needs privacy, you definitely need to invest in a machine to run your personal, you know, your local AI model to comply with, you know, regulations and so on. All right, guys. If you enjoyed the video, click the like button, subscribe to our channel, and

15:56I'll catch you guys in this next one.

16:04[music]