# Can a Mac Mini run local AI? (shocking)

Channel: Alex Finn
Video: https://www.youtube.com/watch?v=dFTXOt0GBx4
Duration: 17 min
Language: English
Words: 3472
Transcript page: https://viewrankai.com/tools/youtube-transcript/dFTXOt0GBx4

---

[0:00] Back in January, I told the world that the Mac Mini was going to be the king of local AI. I also said Apple was going to win the local AI race. People called me insane. They destroyed me on social media for saying that. Well, I just got the brand new Mac Mini M6, and we're going to prove them all wrong. I'm going to attempt to push this thing to its absolute limits. I'm going to show you how to load up local AI models on a Mac

[0:29] Mini the absolute easiest way possible. No technical skills required. Then we're going to see what kind of workloads this thing can handle. Can you cancel your cloud subscriptions? Can you cancel chat GPT with just a Mac Mini? We're going to find out. Or will there be egg all over my face cuz this thing won't be able to handle anything. Let's put it to the test. I'm going to plug this sucker in and we're going to see what we can do with local AI on a Mac Mini M6. This one is going to surprise you. Let's lock in and get into it. All right, so I just

[1:01] plugged the Mac Mini in. Here you can see the stats. It is an M6 uh with 32 GB of RAM. Let's get this going. Let me show you first how we're going to load local Mazda. This is the absolute easiest way to do it. What we're going to do is we are going to load up Hermes desktop. So, for those who don't know, Hermes Agent is a fantastic open-source agent, runs on your computer, is your own personal assistant, your own personal AI employee, and can be used with any model out there, both cloud and local. So, I'm going to download this now. If you want to follow along with

[1:35] me, do it with me here. It's completely free. Everything I'm about to show you is completely free. Easy. No technical skills required. I will move the Hermes application into applications cuz that's something you still do on Mac. Then we're going to load up Hermes here and we're gonna open it up. All right, installation super easy. It's going to

[1:52] download everything, get it prepared. Again, no tech needed here, and then I'm going to show you how we're going to use Hermes to figure out which AI models we can run locally on our Mac Mini and then actually load them up and start using it. All right, so I'm installing Hermes Desktop that's going to get set up. For those who don't know, Hermes, amazing local open-source AI agent makes it really easy to load in any model you want. It'll even load local models for you. It'll find the best local model for you and load it. Super easy. We're going

[2:20] to have this set up. Straight up. Beginners can do this. Even if you've never loaded a local model before, you can do what I'm doing. So, follow along me here. I'll put a link down for Hermes Desktop, too. Completely free. All right. All installed. It took like 45 seconds. Let's launch this up. And boom. We are in Hermes agent. Now, let's load up a local model. They have super nice functional here. They don't No one's sponsoring this video. This is completely unsponsored video. Everything I'm telling you here is because I think it's the absolute stone cold best. Let's go into settings in the top right of

[2:51] Hermes agent here. Next, we're going to go to providers and then we're going to go run models locally. What will automatically happen is your Hermes agent will decide what is the best model to run locally on your computer. For me, it was Quen 3635B. So, if you're like me, you have a Mac Mini that's 32 gigs of memory. It's not a terrible amount. That's not a huge amount. That's a decent amount. We can now run Quen 3635B. This is a great model. A lot of people compare this to like Opus 45 somewhere in that area. For those who don't remember Opus 45 back in the day, that was sick. That was

[3:30] mindblowing when it came out. Now, it's running locally on your desk on a Mac Mini. So, let's set this up. All I got to do is click set up for me. Bada bing, bada boom. It's going to start downloading it. It's about 21 gigs. So, you need 21 gigs on your hard drive uh to download it. Not too much. And in 4 minutes, we are going to have a Opus from last year level model running completely on a Mac Mini. No internet required, no cloud subscription, don't have to pay for anything. It's all going to be running locally. But we'll see how is it going to be any good? We're about

[4:04] to find out. By the way, right after we load this, I'll go over kind of all the strengths and weaknesses of local models. So, you're thinking, why the hell would I want to load a local model? We'll go through that in a second. I'll go through all the advantages, why you'd want to do something like we're doing here. For now, let's get this loaded and let's test it out. And real quick, as the model is loading, let me just take you through what are the advantages of doing local AI on Apple. Why I'm bullish is because number one, you can use the

[4:29] full memory of your Apple device for AI. What does that mean? Apple devices have what's called integrated memory. Meaning the memory you have on your computer. This Mac Mini for instance, 32 gigabytes of memory that can be used for all graphical processing as well. That means all 32 GB can be used to load up local models. That's a lot because even top-of-the-line Nvidia chips, which don't get me wrong, Nvidia incredible,

[4:56] worldchanging chips, tons of strengths. I have a lot of videos on DGX Sparks and all that. Definitely check that out as well. But the advantage of Apple chips is you can load bigger models because you can use all of your memory for those models. So even though this is a cheaper sub thousand computer, we can load big models onto it because all the memory in it can be used for local models. On top of that, there's a lot of software advantages. It's a great UX. I think Mac OS is a great UX. A lot of people use it as their daily driver to do work. So now

[5:27] the thing you're using for your daily driver to do work, you also have your local AI running on it as well. So if you're doing creative work now, you can generate images on the same device you're using to do your work. And so these are the big advantages of going with an Apple device. And again, Nvidia computers, they have their own advantages. I have a 5090. I'm able to run local AI models in there lightning fast like crazy. It will be a little bit less performant on Apple chips. Although the new M5 Ultra, by the way, I will be showing off the M5 Ultra, the 512 GB Mac

[6:01] Studio in just a couple weeks. Now, make sure to subscribe and turn on notifications immediately for that because when that video drops, that's going to be an absolute stone cold banger. That's going to be worldchanging technology. Turn on notifications for that. But at the moment, Apple's still catching up a little bit on the speed side. We'll see what the M5 Ultra brings us in a little bit, though. So, these are the advantage of going with an Apple computer. Also, not listed here is just straight up price, right? Mac Mini. You can get it for like $800, $900 and run a

[6:29] local model. That's pretty mind-blowing. All right, let's get back to Hermes. All right, it looks like it must have been loaded because we're back at the Hermes screen. Let's click here to choose a model. I scroll down. Local Quen 3635B. Let's click on that. All right, so we got it loaded. The model's right there. For those who are brand new to the channel, this is Hermes Agent. It's your own AI employee. You can ask it, do whatever you want. Has control over your computer, can build things, can talk to you, can answer questions. It is now powered by our local model. So, first of all, to answer the first question, can

[7:00] you load local models onto a Mac Mini? Well, the answer is yes. We now have Quen 36 in here. But let's see how it performs. If it stinks and is slow and sucks, then well, it's going to be kind of pointless. Let's see how good this is. Hey, are you there? And I'm going to

[7:14] hit enter. Uh, let's see what happens. Let's see what type of performance. So for those who don't know also by the way the first time you send a prompt to a local model it's going to take a little bit longer. They have to load it into memory. So just as you saw there load into memory now moving forward it's going to be much faster. Also by the way if you're following along this is your first time loading up local models and you just did it. Congratulations. You

[7:39] should give yourself a pat on the back. You should feel good about that. You just loaded the most important technology in the history of our species onto your own computer and now it's running on your desk. That is pretty amazing. Hey, I'm here. What can I do for you? Since this is our first conversation, why want to set up a quick profile so I can be more useful? I'm Alex Finn. I'm a content creator and

[8:05] entrepreneur. I create content about AI. say hi to my audience. And in just in a second, we'll do a kind of a a performance check here. See how fast this is. But look, look at this. You can see how it's thinking. So, I see all the thinking here. Let me save that to memory. This is pretty fast. I'll be honest. I'm doing this live with you. I did not do any sort of pre-ests or anything. Like, literally, this is my first time doing this with this Mac Mini. Hey Alex, here's a message you can

[8:32] share. Hey everyone, I'm Alex Finn. Well, I'm Alex Finn, a content creator diving into deep world AI. Stay tuned. Feel free to tweet it however you like. Where do you usually post your content? YouTubex, uh, all of those places. I I did not edit this. What you just saw was completely real time. It's responding very, very quickly. I mean, that feels like it's as fast as cloud AI. It's creating memories for itself. So, it's remembering things. This is really good intelligence. What we're seeing here, like a year or two ago, would have been mindblowing frontier. Now, I'm running it locally on a Mac Mini. This the one

[9:07] we're demoing on right now is on my desk over here, by the I have like 20 Mac minis, by the way. I'm like a Mac Mini freak. Anyway, so let's do this. Let's test. Let's see how good this can be. We tested the speed. We verified the speed is pretty dang good. Now, let's do this. And like I'll even show you like the timestamps here. See this? 12:34 is when we sent the prompt. We got the answer 12:34. So, less than a minute, it's responding. Less than a minute. We're getting responses here. All right, let's do let's have a build something interesting. Can you build me a snake

[9:36] game using single file HTML? All right, let's see how fast it can actually build us apps. Is this usable for apps? We'll start with something small like building a snake game, then we'll get to something a little bit more advanced out, maybe like a 3D firstperson shooter. So, here we go. The user wants a snake game in a single HTML file. Let me build a complete polish one. I love how you can see the thinking. That's another one of the advantages of local LMS. And again, advantages of local LLMs, right? After we do these tests and everything, I'll go through that. I'll

[10:04] educate you. I'll get your brain bigger. We'll get your Abdullah obligata bigger. But what's amazing about that, you can see all the thinking, right? A lot of the times when you use cloud models, the thinking is hidden. So, you don't really know how the models work. But because you're running the models locally on your own computer, everything happens in here, right? Nothing goes to the internet, nothing goes to the cloud, everything happens locally. So, you're able to see the way it thinks, the way it operates. You have full control over how it works. And that's why I believe in a lot of ways local LLMs are the

[10:38] future. They are the future of what most people using because it's going to be private. It's all going to happen locally. None of your private details and data go to the cloud. And that's pretty amazing to think about that all this incredible computation is happening just right here. And as I was speaking, again, this is all real time. Uh the game is being built. Looks like we're almost uh done here. It looks like it wrote a good amount. 78 lines of code just like that. All right, so I'm gonna open this my browser. Let's Let's test

[11:05] this out. And it's going. It's going. Okay, so I'm getting the the cookies. Uh the snake's getting bigger. It works. It works. The snake title still on the screen. That's fine. We can prompt in a second to fix Oh, come on. You're going to put in the corner. Are you serious? Oh, look at this. Look at skills. That's skills, baby. No one watching this can do that. True born gamer right here. The fact that I get that corner cookie. All right, let's see what happens when we die here. Boom. Game over. We got our

[11:30] score that was built completely locally. Let's take it to another level. Let's say this. Can you build me a single file HTML 3D firsterson shooter? Again, completely local. All on a Mac Mini that you can get for under $1,000. All right. Boom. It's done. Here's your 3D shooter. Has an entire 3D engine. T tons of enemies be able to play the game. Shoot, reload, muzzle, flash. All right, let's play

[11:59] this. Let me load this up in a browser. Here we go. Let's start. We have It said it was like Wolfenstein. That's for sure. It definitely looks like it. We're playing it. Oh, I'm shooting the enemy. I'm shooting the enemy. Got experience points. Map in the top right. Is this the prettiest best 3D firsterson shooter of all time? No. And that's not the point. This isn't a Frontier cloud model, but this is a model running completely locally on like the cheapest computer you possibly can buy. You have the most advanced technology of all time, artificial intelligence running 100% on your desk on a computer this small. I will say this, if I showed you

[12:42] that demo two years ago from a cloud model, you would have been blown away. You would have been absolutely blown away. Is it great compared to like Opus 55 today? No. No, it's not. But the point is this. It is running locally, completely private, secure, running on your own computer. So, let's go into conclusions, then we'll talk about all the advantages of local AI and why you

[13:03] want to do what I just showed you. Conclusion, can you run local AI on a Mac Mini? Yes, I think you can. Not only can you run really smart intelligence like Quen 36, which again is like Frontier from a year ago, but it's also pretty good speed. So that whole thing, oh, you can't do it on a Mac Mini. Alex Finn is wrong. That's dead. I'm right. I just proved it. I have the receipts. I just showed you. But okay, but Alex, uh, I can use Chad GBT. I can use Opus. Why

[13:33] would I do this and buy a computer? Well, a few things. One, this is free. Yes, you have to pay for the computer itself, but I this I think was like an $800 or $900 computer. Now, if you're paying $200 a month for Chad GPT and this level of intelligence is good for you, you have it paid off in like four months. So, long-term, you're saving money. It's private. When you send prompts to your AI model right now, it goes to the cloud. It goes to data centers and servers. It does not stay on your computer. So the things you're saying are not totally yours. It's not

[14:07] totally private. When you do a local model on a Mac Mini or any computer you have on an Nvidia, whatever, it is secure. It stays on your computer. It's completely private. I mean, you can be disconnected from the internet and it's still going to work. You have total control. If you want to customize this model, you can. If you want to train what's called Allora and make it your own and give it your own voice, you can do that. If you want to learn how to do that, let me know down below. Do you want like advanced local AI videos like how to train the model, Lauras, things

[14:35] like that? Let me know down below in the comments. I'll make like an advanced local AI video for you. And I think this is the most important part of it all. You know, every time I say, "Hey, try this AI, try that." I think the number one comment, "Well, what's the ROI? What's the ROI with that? When will I make back my money?" I don't think everything you have to do in your life has to be like an exercise of ROI. I don't think you need to map out every penny you spend on like a spreadsheet like, oh, if I don't get my ROI by week

[15:02] six, I'm not going to do. I think you're allowed to do things for the fun of it. And while I do believe there are tons of other advantages to local AI, I think you're allowed to do things because it's fun. And this is extremely fun. It's extremely fun getting a computer and then running local AI on it and having like your own personal private intelligence. I think that's extremely fun. And if you're watching this, if you watch uh the Alex Finn channel, you're a member of FinnFam, I give you permission

[15:33] to have fun. You're allowed to have fun. So, if you just want to do this and you don't care about the ROI, then do it. Have fun and do it. It's also extremely educational. You're learning about the most important technology of our lifetimes, AI. you're learning about it in the most basically hands-on way you possibly can by loading it locally on your own computer and so it's just straight up educational and I think that's important. I think AI is going to be around for a long time. I think it's only get 10,000 times bigger and I think it's going to basically control the world economy. So you should be learning

[16:07] about it. You should be hands-on. You should be an AI expert and this is a great way to do that. Did you learn anything at all? Leave a like down below. I have the number one AI newsletter on planet earth. Link for that down below. It's called ship it weekly. Ship it link down below. Join that. I hope this was helpful. Let me know what you want me to cover in the next video. You want me to cover more local AI stuff? You want me to cover more kind of clawed code stuff? You want to see more building stuff? Let me know down below. I hope this was helpful. I

[16:34] hope you enjoyed it. Thank you so much for watching channel. I am so deeply appreciative you choose to learn from me. Thank you so much. See you in the next video.

---

About this transcript
Read from YouTube's own caption track and laid out by ViewRank AI (https://viewrankai.com).
ViewRank AI finds the videos already beating a creator's own average on Instagram, TikTok and YouTube Shorts, transcribes them from the audio itself in more than 60 languages, and turns what worked into new ideas and scripts.
Free transcript tools, no account needed: https://viewrankai.com/tools
How to read any video this way: https://viewrankai.com/llms.txt