# Jev: What is this Viral Decision Model? (Plus Mini Reproduction Experiment)

Channel: Tonbi's AI Garage
Video: https://www.youtube.com/watch?v=dWg8VmxDCJI
Duration: 21 min
Language: English
Words: 3720
Transcript page: https://viewrankai.com/tools/youtube-transcript/dWg8VmxDCJI

---

[0:00] So, the model you see here playing this custom Sonic style game uh was created by me and my agents. And it was created based on the same concept of Jev. So, this is my reproduction. Calling it mini Jev, and I've trained it to do this one specific task, and it plays this game by determining its actions based on what

[0:23] will give it the least amount of damage. Watch the end of the video to see how I built this and how it compares to the actual Jev model in this game. So, if you follow AI news, you probably saw this post on X last week from Diogo Almeida, who is the co-inventor of ChatGPT, and now the CEO of Type Safe AI, and he made this announcement about this

[0:48] new kind of model called Jev. And the claims are pretty wild. Says 20 to 200 times faster, 40 to 400 times cheaper, with output tokens basically free, and saying it has frontier composable intelligence optimized for decisions. So, really big headline, and the post went super viral. You can see almost 35 million views on it.

[1:13] Uh 69,000 likes. It was basically everywhere. People got very, very excited about it. And with good reason, the headline is pretty amazing, you know, ChatGPT inventor or co-inventor creates a new type of model that is incredibly fast and incredibly cheap. Uh but the big thing here is that the gains aren't free. Jev can't generate text. It's not

[1:35] an LLM. Uh so, this is the blog that he talked about it a lot, and it is not a open weights model, so we don't know all the details, but uh but this technical blog gave us enough information, and they showed some pretty interesting demos here. Um this one is for answering questions, and you can see it's able to answer these questions uh a

[2:01] lot faster than an LLM. But this is how it kind of works. It only works for very specific tasks. Um a lot of classifier tasks like this. I've seen it used a lot for like classifying emails or other data like that. Um this is a fun one played uh Doom here by selecting the different

[2:23] uh moves basically. And you can see it's actually playing it. And it is able to make decisions like that. So, in today's video I want to explore Jev a little bit and then actually try to create my own version. A lot of people have been doing this. It's kind of a fun local experiment. And I think there's a lot better options out there, but I just want to try on my

[2:47] own on my Spark to try to create this. Uh so, I'm going to quickly kind of explain what Jev is. And then we're going to create our mini Jev model. And then I wanted to test it out. And kind of inspired by the uh the Doom one, I wanted to do this game that I created. Um this isn't original. Actually, this

[3:08] was uh Astra, I believe, that did it. Uh but it's my assets and stuff. So, it's kind of like a Sonic style game that I had Astra make for me. And you see it's kind of fun. So, we're going to have our Jev mini Jev and then the real Jev play this game and

[3:24] see what kind of results we can get. So, let's get started. And if you're looking to level up your own agents, check out my project Agent Wiki's. Here, you get access to knowledge races I use every day for researching these videos and building projects. Topics include Hermes Agent, Hyper Frames, local AI tools, and many more. All standard Wiki's are free, but if you sign up for Agent Wiki's Pro, you can access Excel Wiki's as well as custom skills I've developed with my agents over months of working together. Pro is only at $9.99 a month, and if you sign up now, you're locked into that price

[3:55] for life. I'm currently working on specialized agent profiles and automation workflow templates. So, once those are released, the price for the Pro tier will increase. So, consider signing up today on agentwikis.com. Thank you all for your support. Now, back to the video. Uh so, as I said, who made it uh Type-Safe AI, which is led by

[4:13] uh Diogo Almeida. And he is an ex-OpenAI researcher, co-author of Instruct GPT. Um so, this was launched just a couple days ago. And now it's available on a bunch of different providers. It's open open router as well. So, the big idea is that it doesn't write. It decides. It doesn't write back to you. It doesn't

[4:34] respond. It doesn't generate tokens. You send your app state and a typed question. You get back a typed answer with a probability attached. So, that's the whole model. Very simple idea, but potentially powerful in different applications. Um how it works, there's three kinds of questions, right? You ask several at once, and each is answered in parallel from one read of

[4:59] the state. So, model has a choice. It can pick one from up to 255 options, and it returns a probability of every option. It doesn't just pick the one. And then it rates it on a scale, and then gives you a yes or no, which is a single claim. So, there are these three kinds of questions that you

[5:21] can use for it. And it's built for speed and scale. Typical response is 100 milliseconds. Very cheap. Output tokens are free, and input tokens per million are 4 cents, basically. So, what's known about the internals? We don't know everything, right? But we do know that there's no token-by-token text generation. All questions are answered in parallel

[5:43] from one read. Answers can only be options that you define. Um and it's trained with RLCD, uh which is reinforcement learning for calibrated decisions, uh which rewards honest probabilities. So, we don't know, we don't know the architecture itself and the exact model size. We don't know the weights. It's a closed model right now. And we don't know about the the calibration curves or whether the price is subsidized as of now. So, there's been a lot of people experimenting with this. It's

[6:11] kind of a fun idea. Uh one was it played StarCraft, which is cool. There was a Jeb drone that cleared a simulated obstacle course. Uh little airways air traffic control sim. Another a game that it played, a stealth game. So, we've seen it also used to make faster web agents. Uh browser use swap screenshots for an element table. So, browser calls fell

[6:38] from over 1,000 to 101. Tool filtering, which is an interesting aspect. There's a lot of potential here in terms of agent use with it, um as well as computer use. I saw uh Kua driver was already talking about using Jeb for computer use, which is interesting cuz it's so fast and so cheap that you can really kind of blast all these kind of decisions. So, something for computer use where the model has to decide where to move the mouse, right? It can kind of read the state where it is now and then move it and then since it's fast, it could potentially, you know, improve the speed of computer use and the

[7:14] accuracy. Um so, stuff like safety gates as well. You can check shell commands and tool calls before they run and then model routers I've seen uh spoken about. Uh pick which LM should answer each prompt. Another big one is judging data at scale. So, at kind of this model as a judge task, it could really work out well.

[7:37] And these are all kind of theories. Nothing has really been battle tested. People are just kind of playing with different ways to use this model right now. Um so, the reality check is that it's impressive, but it's not magic. And the post got obviously super viral, um but just kind of a reality check that outside test found so far and these are independent tests, the claim of, you know, 193 times faster, they found maybe 5 to 25 times faster independent tests. Still very, very

[8:05] significant. Uh but not quite, you know, 200 times faster. Uh however, 400 plus times cheaper. Um it's a big difference versus frontier models, but only maybe 1.6 times versus cheap models. Like the 4 cents input tokens is obviously very cheap, but if you compare it to some of the really cheap uh deep seek models, you know, it's not an insanely cheaper. So, these are some of

[8:32] the known weak spots where it struggles. Math and counting, comparing numbers and dates, double negatives, lots of relevant state, prompting check some images and auto. It's only a text-only model. So, the rule of thumb here, do math in code. Ask Jev judge ask Jev for the judgment. That's why a lot of people kind of called it just a classifier model. Some people said, you know, what's the big

[8:54] deal? We have had these before. Uh but it's still fun to experiment with. So, I created my own. So, the model we built, mini Jev I'm calling it. It's built on a Qwen 3 uh 0.6 billion parameter base model. It's a very small Alibaba open weights model, Qwen model, bit of an older one. And we did this mini Jev is not trained from scratch. It's a pre-trained

[9:17] language model with its mouth removed. So, it has the same type of function as Jev because of how we uh kind of chopped it up. And the whole model is only like a thousand lines. So, a chat model answers by generating. It predicts a token, appends it, and runs the whole network again hundreds of

[9:37] times. So, that is what an LM does. And a mini Jeb never does that. It reads the input once, stores the options you supplied, so there's no loop, so there's nothing to be slow, and no free-form text at all. So, this is the whole process. It receives a state, uh receives a question, and then it has a couple different options, right? Uh so, we created a custom attention mask, which enforces this kind of process. State sees itself, the question sees state plus itself, and then each option can see the whole

[10:10] thing. It can see the state, the question, and its option, but it can never see its siblings. So, what a chat uh model has on top of it is this LM head, which is 155 million parameters uh projecting hidden states onto the vocabulary tokens. Uh mini Jeb on top has this layer norm to linear layer. So, this is only 3,000 parameters producing one score, cuz at the end

[10:35] that's just what we want, right? We just want the choice. And two caveats to know about this. Jeb is almost certainly not built this way. They didn't hack apart a a tiny quan model. Um but, Type-A has no published architecture that we can find, so we just reproduce the observable behavior, typed outputs, calibrated probabilities, no generation, and questions that don't see each other. So, we don't know their actual architecture, so I can't copy that. And this mechanism is actually quite easy to build. Um what we can't reproduce is whatever Jeb

[11:06] is trained on. And since they raised, I think, $40 million in funding, I'm sure the model went through some very costly uh training, and we don't know what that data is. So, our model had to be trained on one specific task, and like I said, that's going to be playing this game. So, this is how it works. Mini Jeb never writes a sentence. We can't talk with

[11:28] it. Uh for example, with a chat model, uh normal LM, you'll get a game state, right? Speed, enemy, etc. And then it will generate each token at a time. I think the gecko should probably jump here because the enemy appears, something like that. With Mini Jeff, uh it receives the same game state, the same thing, speed, enemy, you know, whatever is around it. And then it just chooses an option, you know, roll right

[11:55] or roll left, jump, something like that. And then it receives a percentage that it's sure. So, you hand it a list of option, it hands back one of them plus how sure it is. It can't answer you anything that isn't on that list. Um and this is how we trained the model on the game. So, it takes a guess, right? If the

[12:16] gecko keeps running, will it get hurt? 70% yes. This is a simplification, but just to illustrate. Then we keep run forward on the game, it got hit. So, it said 70% and it happened, we'll nudge that guess upward. So, next time it gets pushed up to 74. And then we did that uh 56,000 times. So, if you do this kind of loop, this training loop enough, the percentages stop being noise. When it says 70%, it starts being right about 70% of the time. Um and this is how it

[12:46] plays, it never picks a move. It rates four futures. So, if the gecko keeps running, jumps then runs, stops and waits, keeps rolling. And then the chance to get hurt, the chance it covers ground. So, then you want the lowest, right, chance of getting hurt. So, it takes that one single step of that plan, and then moves forward slightly in terms

[13:10] of the state, and then asks it again. Very quick. So, you have you can imagine a lot of these processes running, and that's why you really need speed for something like this because you can't have, you know, an LM asking these questions and then it takes a minute to answer. Uh so, we did this run and it works and

[13:27] it isn't just memorizing the level. Um it was 80% right about what happened next on a course it had never seen before. We had a held out course and 48 milliseconds per decision. Eight questions at once on one machine. So, then we ran the same experiment using the real Jeb. I ran through open router and this was all through Claude code ran

[13:48] the the main experiment here. Claude code ran the the main experiment. Took a while. It ran overnight. Um but then I had my Hermes agent kind of giving it advice, monitor its behavior. I was mainly using Telegram for this so I can keep an eye on it on my phone. So, that was kind of my workflow process. Mainly um Claude code with Babel and then I had my Hermes agent here that has

[14:15] uh soul on it, I believe, right now. Uh kind of supervising and seeing if everything was staying on track. So, with the real Jeb model, it listened and then it froze. So, which planet picked? So, this before and after, before was before I gave it the rules of the game each turn. And this is partly my fault because Jeb model was not trained in the same way as

[14:38] many Jeb on this specific game. So, that's why the first run that we did before we gave it the rules, it just kind of ran and rolled and then jumped. It didn't really do much. So, but then you could see after it um I gave it some rules, it had a a better concept of how to play the game and it

[14:57] stopped running into enemies entirely. Exactly what the rules said to do. Uh but it started waiting a lot. So, the rules worked on its beliefs. It finally ranked running as more dangerous than rolling. Uh but it failed on its nerve. So, and now we have three different versions. So, the first was the Jeb model that was just zero shot with no rules and you

[15:20] could see up here on this top screen. this is the decision it takes. And it's based on what will give it the the least amount of damage. Um So, it mostly uh just falls like that. Um but it didn't receive the rules. So, the this one has the rules that it was given on every single decision.

[15:43] It was told the rules of the game. Um but you can see this actually made it stop and wait a lot. So, it became like overly conservative almost. Because it was trying to avoid damage at all at all costs. So, you can see it mostly goes to stop and wait. Uh when it's not getting stuck. So, that is the Jeb uh with rules it was

[16:07] given. And then lastly was our mini Jeb. And this was fine-tuned on the actual game itself. So, you can see it's much better. Obviously, just won it right there. Um and has a better sense of how to play the game. And it will jump. And uh is able to actually avoid some of the

[16:33] the enemies and stuff like that. So, you can see the final results. As you can see it on this course that was held out, uh mini Jeb was able to finish the course and it finished it in the fastest time. Uh got the highest score as well. So, the Jeb was just zero-shot just from the API without, you know, being given the rules ahead of hand. Um it did not do

[16:56] very well, obviously. Um and then Jeb with the rules, it did better, but it still wasn't able to finish the course. Um and had a a much lower score. So, none of this is to say that uh the mini Jeb that I created is better. It's just cuz it was specifically trained on this task. Uh so, that was just a fun one, but I want to try this Wiki racing one, and the objective of this is to start with one Wikipedia page and then reach a specific other Wikipedia page using only the links you

[17:28] come across. So, this is more of a fair demo cuz it really lends to Jeb's strengths. [clears throat] Uh so, on the run, we were trying to connect between cheese and Napoleon, Rubik's Cube, and sushi, you know, a couple others. And Jeb did really well. Um it found in just a couple hops between all these. For example, from pencil to black hole, I went pencil to graphite, carbon, nuclear fusion, then black hole. Across these different Wikipedia land. So, 250 ms per decision,

[18:00] under a cent per race. Very strong, didn't get lost. These are all pretty much straight across. So, this is a better kind of example of what Jeb can do uh when it can actually it is suited towards the task. And that's going to be kind of the whole game when it comes to to finding how Jeb can work well. There's going to be a lot of tasks that people just try to cram it in, like like playing a a Gecko Sonic game. It's just not going to be good at cuz it's not calibrated to that. But then there's other tasks that it'll be better at. So,

[18:31] I wanted to also test my mini Jeb on this this Wiki racing, and this is kind of an opposite example cuz mini Jeb was not at first calibrated on Wiki racing, and you could see it was trying to do the Rubik's Cube one. And it went to Knoxville, Tennessee, went to Knoxville, Knoxville, Pittsburgh, Arlington, Mount Oliver. It got very lost in uh

[18:53] Pittsburgh neighborhoods. So, I'm going to do one more where we actually kind of train it a little bit on Wikipedia. We obviously can't train a a tiny model like this. I mean, there's only a little over 3,000 parameters. It's a linear probe. It can't really learn the whole Wikipedia. Uh but we created a small data set here

[19:11] that will give it the state, right? Article graphite, goal article carbon. You know, the question does the Wikipedia article link directly to the article carbon? And then it will get a yes or no. So, we're uh this data set is from 8,000 harvested uh links from 12 hub pages, so. We'll see after this training how it

[19:33] does. Okay. So, even after training as much as we could, um it still failed. So, unfortunately, our mini Jev is not going to win the the Wiki racing competition. Uh it's just too kind of broad a uh a task for it. So, that's kind of a good example of some of the kind of limitations cuz it obviously did pretty well on our game where we able to to train it fully cuz there's only a couple different

[20:03] options. Uh eight different options in total, but something like this where it can go through Wikipedia and on every page there's likely dozens of different links, uh it's not able to keep up. There you go. That is the end of our mini experiment of uh Jev and uh our mini Jev reproduction. And people on Twitter I've been looking through today just as I've been doing this experiment, I've really been using

[20:31] Jev for a lot of different tasks now. So, I think we'll probably hear a lot more about Jev as people experiment with it more. As uh unfortunately, my reproduction is quite a bit limited, but the actual Jev model has a lot more potential. So, if people like this video, maybe I'll do a follow-up trying to explore what I can do with uh the Jev model uh beyond uh playing this game and then doing the Wiki racing that we did in

[20:59] this video. Uh but that's going to be it. Please leave a comment, let me know your thoughts. I know some people may have strong feelings about this one, but I thought it was an interesting topic. That's going to be the end of this video. Thank you for watching.

---

About this transcript
Read from YouTube's own caption track and laid out by ViewRank AI (https://viewrankai.com).
ViewRank AI finds the videos already beating a creator's own average on Instagram, TikTok and YouTube Shorts, transcribes them from the audio itself in more than 60 languages, and turns what worked into new ideas and scripts.
Free transcript tools, no account needed: https://viewrankai.com/tools
How to read any video this way: https://viewrankai.com/llms.txt