Stop burning your usage limits - effort levels in Claude Code & Codex

0:00Quick video today on effort levels and thinking tokens. Something I just learned actually. So, you might have been using a harness maybe Codex with GPT- 55 or maybe you've been using Claude Code with Opus 48 and you notice that there is an effort level or thinking level. And what this controls is basically the amount of thinking tokens, the amount of tokens in these little grayed out blocks here that the model is using to process a request in the conversation. So, very high effort, the model's going to think a lot more before it responds. Low effort, it'll just respond, it'll do stuff fast. One thing that I just

0:33learned that's actually different between the harnesses is that Claude Code does not cache or causes a cache miss rather when you change the effort level. What does all this mean? I'll put up a graphic. Well, you can think about messages as being this really long stream, like a conversation of text message list that you're having with your friend. And every time you send another message, the model is actually receiving everything, you know, previous in the conversation. But if we change something early on in the conversation, we're going to cause the entire conversation to be resubmitted every time because the providers do this thing called caching. And caching is just

1:10pre-computing or storing these values so that subsequent messages are less expensive. So, when you send a subsequent message, Anthropic has all of this stuff cached and it's only re-computing the the new messages, right? What the issue is is that effort level is defined at the very start of that string. So, changing the effort level on Claude Code causes the entire message list to be re-computed. And so, if you're in here and you have a chat and you have a chat with Opus and you're sending through like dozens and hundreds of messages and then toggling between low, medium, high, etc., you're going to be causing a lot of cache misses and

1:46you're going to be using more of your Anthropic usage limits. This is important because you'll burn through your credits and people should know about this. What can you do about it? Well, nice thing about Conductor is that if we're going from one chat, so I create a workspace, I have a chat to a new chat, we can simply summarize a transcript of the previous chat and start a new chat with a different effort

2:10level. Does this solve the problem? No, but it means that now we don't have to change the effort level as much in this chat because we're in a dedicated workspace for this chat and we can keep our context separate. So, by having separate workspaces and different chats for dedicated effort levels, we don't have to worry about this problem. Now, with Codex, you don't have to worry about this problem at all because Codex um actually defines effort a little bit differently. So, changing the effort level with Codex does not um invalidate that cache and cause cache

2:41misses. So, that's a little bit about effort levels. It's a little bit about the hardnesses, Codex and Claude Code, and how you can kind of manage some of these usage limits intelligently. Don't go toggling effort levels on thousand-message conversations in Claude Code. That's the TLDR for y'all. All right, I'm Matt with Conductor. Until

2:59next time. Peace.