The Most Terrifying Week in AI History...

0:00Everyone is talking about Black Wednesday, because the internet seems to be breaking in real time. And I'm going to be honest with you. This is the scariest week I've seen in AI. PHP, which runs about 70% of the web, had a bug so bad the FBI and the NSA said patch your application or take it offline. And at the same time, hackers were stealing API keys and reselling them on Chinese black markets. This used to take an entire team of hackers. Now it takes one guy, a cheap AI model, and about 50 bucks. So today, I brought on Ange Nanda, one of the best developers I

0:37know, to break down everything that's going on. Okay, Ange, my first question to you is what the hell is going on? There There's a lot going on. I mean, this is the craziest week in cyber for ever since You remember CrowdStrike going down like a year ago? Like this is probably the worst level of incidents that we've ever seen. You know, I think the way to quantify it is that before AI was a thing, all these cloud companies like AWS, Google, Microsoft Azure, they were like touting themselves in having five nines of availability, which means 99.999% percent uptime. That means they only go down for like 2 hours a year. For

1:19context, no one has got even two nines of availability this year. So that's how bad it's gotten. It's actually gotten three orders of magnitude worse than it used to ever be. There were two main things that happened. One is that we found a bunch of bugs in some core technologies that power over 90% of the internet, which is in PHP and in Node.js, as well as GitLab, which hosts the code for almost all enterprises and the second largest Git provider other than GitHub. And then we saw a whole host of attacks that I've actually seen up close and personal first hand, where we saw these cybercriminals essentially

1:59tried hack infrastructure providers like railway and open code to steal AI credits from them. And they wanted to steal these credits to to sell them. So, we can talk more about that later, but that's basically what happened. Okay, so this is let's let's break this down. So, can you talk about like what the specific bug is or bugs are, why it's happening now, and like what models people are using to like exploit these

2:23vulnerabilities? Yeah, that's a great question. So, PHP basically runs most of the internet, and it might not be even if you're coding your website in PHP. No one really uses that anymore. For context, the last last big website that was made in PHP was Facebook, and that was started like over 20 years ago. Ever since then, people stopped using it, but it still powers a lot of the internet because a lot of the core services that power the newer technologies is built on PHP. So, a lot of coding is just like built on the old stuff, and PHP is like one of those like foundational level things. And the bug

2:55was essentially that any hacker, if they could exploit the bug, was able to get root access to all the files on your server. So, it's like really really bad. So, they could like get access to everything. to an entire website. They could read it. They could read everything. They could get all your back-end data, which is like all your secure data that's not supposed to be publicly available. You know, most people were like prevented from this being an issue because, you know, they are using another service provider that automatically upgraded, but some of the older websites are like the ones that really suffered from here. Like, if you

3:32ever had a WordPress blog, you know, get it checked out cuz it's probably probably gone at this point. Yeah, that is concerning, right? Millions of people have used WordPress. Okay, okay. So, okay, so there's this vulnerability, and I guess we don't need to dive too deep into like actually how that came to be. I guess I guess the next question would be I think that NSA or and the FBI basically said you need to patch this. So, there was this vulnerability, and it was up to the people who built these applications on PHP to to like patch them. Like, what is patching that mean? Like, can you

4:07describe what that actually means? Yeah, the actual fix is really simple. You just upgrade your server. So, you take it down for like 5 minutes and you upgrade it. Like, it's really simple. But, you know, these are a bunch of like really old servers most likely. That's why the NSA and FBI released this security warning. And they were like, if you can't take this if you can't upgrade your server like right now, just take it down for a few hours. Cuz that's how bad the exploit was. Remember, this exploit gives any attacker full access to

4:35everything on your server. And how does AI play a role? How does AI play a role in this? Like, is it is it because Okay, yeah, go ahead. So, 15 days of these attacks, we got a new model from our favorite people across the pond, Deep Seek. So, they created Deep Seek version 4.1 flash, which is a extremely cheap model and it's really good at coding. Uh much better than anything in the past and it's really fast. So, this combination of attributes for this model made it so that now it makes economical sense for attackers to basically go and tell their model, "Hey, uh why don't you like go

5:15find every website on the internet and try to hack it?" Basically. Okay. So, so so basically like it is cheaper and faster than ever to go look for these exploits. Where in the past it would require a human um and that's also a niche skill, right? Like Yeah. not everyone would know how to go look for exploits and so that's why it was done by like, you know, at least in my imagination by like hooded criminals that are, you know, in some warehouse somewhere. But now, a 16-year-old Billy Bob in whatever country can just basically ask Deep Seek V4 flash to go look for these vulnerabilities and now

5:49it's now it's like exponential. I'm seeing charts um I'm seeing charts of people, I forget what they're called, but like showing how it's like exponentially worse like over the last 3 days. Like, the amount of these vulnerabilities found are going up exponentially. Um I think it's all based on like the deep seek models because even though AI's gotten good enough to hack maybe like 6 months ago or even like 12 months ago, it was just not economical or it was not allowed because of the guardrails of the US models. So, if you know Anthropic, they really care about security, guardrails, that kind of stuff. So, they made it

6:28really hard for you to use their models in a way that they didn't want you to use them. But, now because deep seek version 4.1 flash came out and it's super cheap. It's like 15 cents per million tokens, which is I think two orders of magnitude cheaper than Sonnet and maybe like four orders of

6:46magnitude cheaper than, you know, Opus. It's just and cents per million tokens out? Yeah. Yeah. Opus is what? Like $50? Yeah, $50. 35, something like that. Yeah. Yeah, yeah, yeah. So, it's like two orders like 100x, you know, cheaper. And also, they're really easy to jailbreak. So, I actually don't think that these Chinese labs are like evil, you know, they're not like, "Ooh, we're going to release this model to like make it so that we will have overthrown US." I think China is fiercely competitive. If you ever saw Travis Kalanick from Uber talk about how China was competitive, like you you've seen that, right? Like, what did he say

7:25about China? He was just like, you know, they just ruthlessly copy everything. So, China is at the point where like their tech industry is looking at America and they're like, "Wow, we we're lagging behind. We just want to copy everything." And they're not doing it with malintent, but they don't have the same level of care about security. So, I actually spoke to the CEO of Xiaomi, which is one of the biggest publicly traded technology companies in China, which is, you know, they're kind of

7:54like, let's say, the Google of China. And he said that he wants US model companies to help them figure out security and like tell them what to do cuz they don't know where to even start Yeah, guardrails on these models. And the US model companies don't want to talk to the Chinese model companies cuz they're basically stealing their uh

8:15models uh and we can get into that. Yeah. So, this is actually the biggest reason why these hacks are happening cuz it's a vicious loop. So, for example, these hackers are exploiters in China will go hack Railway or Open Code and they'll they won't even care about the user credentials, you know? Some of these hacks like they'd have access to your whole bank account or Google. Maybe not bank account, but like your Google

8:40information and your like personal data. They like ignore that and they'll just go to your API keys because these hackers can essentially steal the API keys from companies like Railway and Open Code, like developers who are developing on those platforms. Then they can get um inference to the cloud models for really, really cheap because China's restricted. Remember, China doesn't have access to Anthropic models. It's actually pretty hard for them to get access to it. So, then these hackers take those models and then they sell it to other people in China through like their service. So, there's actually like dozens of these websites that you can find online where I'm not going to tell

9:15you which ones, but like you can buy Opus at like 90% discount to normal Opus pricing because people are literally stealing Opus API keys and like reselling these tokens. And then the labs just buy these uh inference from these providers and then they distill their own models like DeepSeek on the output of the cloud models and then those models are then used to hack. So, then the hacks become even cheaper cuz the models become better, they become smarter, and DeepSeek is like trying to race to like the bottom in terms of price. And so, it's like this vicious cycle where like API keys get stolen, you have distillation, then you get

9:53better Chinese model, and then the hacks get cheaper, so more API keys get stolen, so there's more distillation, so that you get even better [laughter] Chinese model. That's crazy. So, this this whole situation is literally making the Chinese models better. Yes. Yes. The fact that the US is not doing anything or like is so restrictive to China is actually making the Chinese models better. It's like, you know, what Jensen Huang said this on a podcast. He was like, "Because we cut off GPU supply to China, they developed their own chips to run their own models." So, like all these Chinese models run their own chips because, you know, how do you make

10:31diamonds? Pressure. Pressure creates diamonds. China is just in the pressure zone right now. So, they have to It's like do or die for them. Right. And they're they're raising a billions of dollars, so it's it's it's becoming even more crucial that they're competitive. So, yeah, this is not looking This is very This is a very interesting situation. So, I guess let me summarize what we've talked about so far, and then I kind of want to move to more practical topics like tactically what do individuals do and what should businesses do as a result of this? But just to kind of summarize. So, we have this terrible bug that was exposed in

11:04PHP and some other technologies, and you know, people aren't necessarily using PHP as much as they are using other services that have been using PHP for a very long time. So, it's a very foundational layer of kind of the the infrastructure stack that we use for the internet, and it's starting to feel flimsy and people are getting worried. And now, because AI models like Deep Seek 4.1 Flash is getting faster and cheaper, people are using it to look for vulnerabilities, and it's creating this feedback loop whereby as they are using these vulnerabilities to steal API keys, then there are these sites which basically have like thousands of API

11:45keys, and then they just drain them all, and you're you're able to use these at a discount, which allows them to gather more data, China to gather more data from Frontier Labs, which then they can use to make their models better. So, we have this vicious loop, and obviously that sounds scary. That sounds scary, and so now let's let's think about like what we can actually do to prevent bad things to happen. So, I guess to start, can you just outline some of the bad things that could in theory happen to me

12:12or my business because of this. if you're running a business and you use AI, which is probably most of you watching this, and you use API keys and you don't have limits on it, theoretically, a hacker could get your API key and just spend it up. We've had that happen to Vibe Code and you you've seen that first hand, right? We've lost, I think, you know, luckily, we've never been hacked and no user data was lost, but like we burned like, you know, $100,000 um because of these exploiters who basically were able to find a bunch of free credits and then just like run up our API key bill, which is probably the

12:48biggest thing that can happen is is like they just want your API keys. That's it. And it's actually funny because Yeah, go ahead. No, no, I think Yeah, I mean, I think it's easier said than done like at a personal level or if you run a smaller business, you can set API key limits. But obviously, if you're a company that has thousands of users, that you could spend hundreds of thousands of dollars a month on API keys, it's probably harder to set limits. So, I guess Yeah, like what what do you recommend doing in regards to like API keys getting

13:20drained? I I think the most [snorts] ironic thing is that we can't even use the US models to protect us from this incident. So, like Open Code was getting hacked and they were getting their API keys stolen and they were trying to use GPT-6 Astra being like, "Hey, like someone's hacking us. Can you look into what's going on?" And GPT-6 Astra was like, "Cybersecurity wording, we're not allowed to look into this because this might be considered

13:47malicious." that's interesting. Okay, so Oh, because because Okay, okay, this is really interesting. So, like I remember reading about this where like, you know, when Fable 5. Uh God, I'm just really struggling with Fable 5 came out. Yeah. Um they weren't allowing anyone to do any cyber requests, right? That's how they were famously rerouting you silently to like a worse Claude model, like Claude Opus 4.7 or whatever at the time. And so, cuz the way that you would get it to do nefarious things is trick it into saying that you're like stress

14:18testing your infrastructure. Exactly. Yeah. Yeah, but and but but if something actually goes wrong, you can't even get Claude models to do anything because they're like, "Oh, no, they're trying to to pawn us. They're trying to like jailbreak the model to get us to do bad things." And so now like yeah, what did

14:34what did open code have to do then? So they had to use the same open source models that were used to hack them. [laughter] That's crazy. So the [clears throat] most ironic thing is we have to use DeepSeek 4.1 flash to prevent us from DeepSeek trying to hack us. Well, I mean maybe you could use like Kimmy K3 or maybe I don't even know what the stronger the stronger models open models, but that's that sounds really scary. And I guess like is is there some sort of program where OpenAI and Anthropic could go to like trustworthy companies to give them access to these like better models for these tasks or

15:08what? Yeah. Um yeah, OpenAI has a Daybreak Daybreak program. That's what they call them. And then I think um Anthropic calls it like cyber uh protection program or something. They have like cyber protection programs where you can go apply and then they'll like like allow your account to access models that It's the same model, but like it has less uh guard guardrails. So for example, we applied to Anthropic's program, but honestly, these programs are probably have a really long backlog right now. And we haven't gotten access yet. And so it takes some time to like figure this out. So actually we're having to use the open source models to

15:47prevent us. So it's like, you know, we It's like the US invented like guns and we're fighting against people who have swords, but we're not allowed to use the guns in battle cuz our leaders are like, "Hey, these guns are too powerful. We need to slow down the development of guns. We make use." And so we're stuck using swords to fight against swords even though what we we have we have

16:09guns. That's kind of like that. yeah. But but but for somehow they're stealing our guns and learning how to make better swords. [laughter] Somehow. Something like that. Yeah, sure, sure, sure, sure. But yeah, you can apply to them. I'm sure they'll like figure out how to get more companies on onto these programs now that this has become a big issue. I think another big tip for businesses is to use like the one password service accounts, which basically have pointers towards API keys. So then you wouldn't store an API key like openly in your server. You would just store store a pointer to it. And so then it'd be they

16:44it it doesn't like prevent the hacks, but it makes it easy to like restrict them in the future essentially. Um so I recommend using one thing. Okay, and so it's not just API keys that are at risk, right? Like the also your data, right? Your your your usernames and passwords and emails and other data has already been compromised and sold overseas I'm told already. I guess A, how do you know if you've been

17:07compromised if there is a way to know? And then B, like what should I do just as a user of the internet to make sure that my data is safe? Yeah, I think number one, go to haveibeenpwned.com. So uh you can go and put your email in [snorts] there and your password and you can see if it's in any of these like dark web public data sets. The hackers usually, the ones who hack you, don't really care about your data. They just sell them to other people who then do things with that data. Um but if you go to haveibeenpwned.com, you can put in your emails and your passwords and you

17:39can see And is it secure? Like is it secure? It's been around for like 20 years. Like it's I think created by like some pretty um great security engineers. So I I do recommend it. You can put like the official link. But yeah, it's it's been around for a while. And I've I've used it many times and I've seen, you know, when I've been pwned and that kind of stuff. And I think one password actually like sponsors it. So it's Okay, so it's a legit thing. It's it's

18:05It's a legit site. It's haveibeenpwned.com. You you might understand why I would think it's not insecure, you know, it's a funny name. Um Yeah, yeah. Okay, so you can go to this and obviously if you've been pwned then you should probably um reset your um passwords and such. Um another thing, I got my Twitter hacked I think 6 months

18:23ago. I think you remember that. And that was because I had a authorized apps that should not have been authorized. So this is another thing on Twitter, highly recommend looking into it, especially if if you ever click links in an email, like you may have and and somehow they got you to like authorize your Twitter account, you may have someone who just has access at any time to make changes to your account. So the first thing you need to do is like go to authorized accounts on Twitter, Instagram, things like that, so no one has access to your Twitter. I saw a guy, and he's like an artist who got hacked

18:55yesterday. Um but regardless, you should look at that. And then also, you should make sure that you have like two-factor authentication on. Like, do you have any recommendations on how to do that? Or like what best like should you use the like Google Authenticator app or whatever? I I recommend using one password, I think. If you have your passwords in there or and your two-factor codes, that's great. If you use iPhones and Apple devices, you can also use Apple

19:19passwords. I think that's also awesome. But yeah, you should ideally sign in to any services using your Google and Apple account as well, cuz they have the best security. So like even if somehow someone gets your password, and then they figure out your two-factor authentication, but it's coming from like China, like they're going to shut down your account, restrict it. Like they they know the consumer patterns, and they have like these really big machine learning models basically running 24/7 trying to figure out, "Hey, is like any of these accounts vulnerable? Is like do I see any like odd behavior?" And they're like instantly shut your account down and

19:55reach out to you. Sometimes even for like ID verification, which is not ideal, but like, you know, at this point, would you rather lose your data or have Google have a little bit more information about you? I mean, they have everything already, so I think I'd rather trust Google than like any random app developer. Like think about it, right? Like if you go log into an account that's like a new app, and it's been like you put in a username and password, and it's the one you always use. Like if that gets leaked, then they can log into any other sites that also share that same password. But if you

20:26just log in with Google, now you're trusting Google for that authentication for the password, and Google's probably not going to get hacked. So, I recommend signing in with Google and Apple. Um also, other than two-factor authentication, use like passkeys. Passkeys are even more secure than two-factor authentication cuz you need either a physical device to like authenticate yourself, like Touch ID or Face ID, which is even better than than just um an authentication code. And then you can actually use agents to like go scour the internet. So, like if you have Grokbot or ChatGPT dots, you just be like, "Hey, these are my emails. Go to like see if I've been pwned, and go to

21:03like websites that I haven't used in like 6 months, and like delete all of them." And it'll just log into all these websites and like delete all your data, which has like never been possible before. I think that's like one tip that I would really recommend doing. If you haven't used these tools, I think that's

21:16a perfect use case. Use agents to find these things and delete accounts that you don't use. I think that makes sense. Um very interesting. Okay, so I want to talk about building apps. So, obviously, you have a lot of non-technical people now building apps, whether that's mobile apps, web apps, internal tools for your company. I guess what do you think the future of this is? Like what are the precautions people need to take when building apps to like avoid these vulnerabilities, or is this not that big of an issue because most people aren't building apps in PHP? Talk to me a

21:48little bit about that. Yeah, I think the new model they're probably better at security than most humans, right? So, they're using newer technologies, they're not using PHP. They know what to do to protect you against it. Just, you know, don't I think a lot of people who've been pwned, um I see them using Superbase and then like overriding their security rules, which is a very common thing for app developers to do. I just I just say that like, you know, just prompt your AI, be like, "Hey, make my app secure," and it's probably going to do a better job than any like human could do. Like

22:21they're better at us coding. So, and then another thing I recommend is having an always-on agent like Dart or Rockbot or whatever you like, even Cloud Code that's just running in a loop. And it just checks all your billing infrastructure cuz that's the main exploit. Like these people, like they've hacked like open code and they don't care about your user data. They just care about the API keys. So that cuz that directly translates into cash. If they just get your like username and password, now they have to go find which other accounts like are you using the same user like it's a lot more work versus like an API key is like

22:54equivalent to cash. It's basically the most liquid form of hackable item that exists. It's like the the most liquid thing. So, you know, have a always-on agent that like looks at all your transactions and looks at all your data and like looks at all your logging and like just every 30 minutes we is like, is anything bad? If something's bad, fix it and send me a message. That's it. We do that and it's it's prevented us from like at least three of these issues and

23:22like $30,000 in losses at the minimum. And it pays itself back. Like just $200 cloud subscription and you're good to go, you know? Yeah, yeah. Yeah, and this isn't a new thing, right? Like people have been dealing with hacks all year. I mean for the last like 6 to 12 months. They're just it's getting more accessible. And I think Yeah, I guess that brings me to something I was thinking about yesterday. I tweeted, what do you think the last frontier model released to the

23:45public will be? And I was thinking about that and I got actually got really sad because I guess AI for me has always been a thing where you could exist at the frontier just being a citizen, right? You could just Chat GPT was the best model that that came out. I was using the best model when Claude 3.5 came out. You know, the individuals were actually ahead of the government. They were ahead of companies because they were first to adopt and it was the best model on the market. You know, and then I think Fable was kind of the first time where I felt like, oh wow, this is like the frontier

24:16is going to be hidden behind closed doors. And now I just I can't help but get that feeling that that's only going to get worse as especially with the distillations, you know, putting these models out in the world makes these models that can do hacking much better. It seems to me that the models may be behind some closed door thing, and that's kind of sad to me. What do you think about that? Like, do you think we may be at the end of like the publicly accessible frontier, at least in the

24:45near term? I mean I mean I hope not. I feel like that would just make this like process worse. Because if we reach that point, it means that the models have gotten so good where um you know, the moment and it's just a matter of time till they distill like these restricted models, right? And

25:04the moment they do that, what what then? You know, then it's like like I like open code didn't have had to use open source models to protect themselves. Like, we need to equip um our country and like our developers with the better models, I feel like. And if it's restricted, then only some companies will get access to it. So, I'm like scared. I I think it's like you know, you're racing against the clock, but then you're trying to add these restrictions. I have a hot take here. I really think that the US model companies should really work with the Chinese model companies, and like teach them how to build uh safeguards

25:41and guardrails into their models. So, even if you know, these Chinese model companies get better, and they will, it's just a matter of time. They are already good enough for these exploits, but they will get better. But, if they know how to do the guardrails as good as the US model companies, then maybe these hacks will go down because the hackers won't be able to use the Chinese models to do their dirty work. So, I I think there should be more collaboration across the two major AI powers, which is contrary to what I feel like most people are advising. It's like oh, we got to said than done now. You know what I

26:16mean? Like cuz not only do you need to coordinate with what some people consider an adversary, you also need to compete at the same time with companies inside your own country. And you don't want to give away your best secrets because, you know, these are private companies at the end of the day. And

26:32then China could always defect, too. Like they could take the knowledge from them and then remove the guardrails later because it sounds like they're literally benefiting from there not being guardrails, at least right now. And so maybe there's some sort of law passed that requires any open models that are used to have some sort of certification that there are guardrails on. I don't know how this is going to work. But um

26:56yeah, it's easier said than done. Yeah. No, for sure. I mean, you're you're right. Like that's the risk, right? But um yeah, I'm in the I'm in the camp of So, what do you think? You think we should just restrict China from using all these models and like No. No, I I I have no answer. I have no answer. I I I was just saying it's easier said than done. I I think just because it's hard doesn't mean you

27:16shouldn't do it. If If you were Dario, what would you do right now? Dude, I can't be Dario. Dario thinks [laughter] He thinks that the the He thinks in such a different way than most CEOs. He He thinks in essays. Um and I don't think like that. I I don't know. I To me, you know, I maybe I just think simply. Like I'm just scared that I won't get access to Fable 5.5 or that

27:40it'll never be released. I just want a better model. I just I just want to use the best models and make cool stuff, man. Yeah, this is interesting. This is an interesting situation. Okay, so we have covered the bugs. We have covered the attacks. We've covered the fact that it's cheaper and faster and anyone can kind of participate on it. Are we missing

27:58anything? No, I think that's pretty much it. You know, we've covered like what you should do as a company and a business and is also like as an individual. And I think it's like very easy steps that you can take. Like you can do today. Like we did this at my company and it took like a day. And I'm sure you've done it as well and it's like it doesn't take that long to like protect against these things. But like I used to be the person who's like, "Oh yeah, I'm never going to get hacked. I'm a tech guy." But now I'm like my mentality is

28:27completely shifted. It's like it's no longer about like just knowing like a little bit. You got to have your agents protecting you 24/7. So, like as long as I'm sleeping at night and I got this agent running, you know, making sure everything's okay, that's like my my safety net, which is interesting. It's like before we didn't need that. Now we need these like always-on agents as the only way is like it's like sort of the last line of

28:52defense now. Right. I think this is a good emergency podcast. Yeah, where can people find you? Yeah, um I just started making YouTube videos. You can search my full name Ansh Nanda on YouTube. Um feel free to subscribe to me. Um and hopefully me and Raleigh can do this again, but yeah, we We'll do a positive topic next time. We'll talk about fun tech. Maybe we'll do like a review on Opus or Fable 5.5 when it gets released in 2028 after they implement guardrails. [laughter]

29:21All right. Uh thanks for for coming on. I hope you guys gathered a lot of useful info from this. Just do the things. Make sure you're secure. It's getting scary out there, and don't search for vulnerabilities and hack people's API keys. All right. See you guys.