Workflows vs agents in Claude Code - IEEE Workshop

0:00I know everyone is familiar with agents. It's a massive kind of buzzword because everyone talks about I built an agent did it did it did it. There is a very specific definition for what an agent is and like what it is not. Claude code specifically is what is referred to as a harness. And what a harness is is it wraps the underlying LLM, the model itself. So in this case Opus 48, Sonnet, your GPT 56, Gemini 35, etc. etc. And what it does is it basically allows it to loop in pursuit of a goal that you give it at its most simple term. It is different

0:35from the claw.ai desktop app because it is on your system. It interacts with your files. It lives in your environment. Now it's making API calls to their servers, but it interacts directly with your files on your system in ways that the web-based applications do not. What you do is you provide all of the context. You provide the claw.md, you provide the files, you provide the tooling for which tools it should have

1:01access to. You are in control of that. From Anthropic's website, what is an agent? Agents can be defined in several ways, but notice how they break down workflows. Workflows are systems where LLMs and tools are orchestrated through predefined code paths. An agent, on the other hand, are systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks. That is critical because when people talk about building agents, what they're usually talking about a lot of times is they built a workflow because it it is a defined path where there is a input, there is a model, and there is an

1:37output, and there's no deviation off of that defined code path. The difference between that and an and an agent is the agent has autonomy to navigate its environment and dynamically adapt in pursuit of a goal. It is not defined to that fixed pathway. Now, this doesn't mean that workflows aren't useful. In fact, most business applications are workflows. For most business use cases, you should be building workflows inside of your environment because they are deterministic unless you have a model in there. When we're building these out, especially inside of a business environment where we want to keep costs down and we want to maintain performance of the system over time,

2:22everything that we can keep as fixed as possible is better. When you start throwing agents and letting them do whatever they want in your environment, you're continuously bringing new variables and new failure modes. And these are the things that need to be thought about when we're designing these systems, especially inside of an operating environment. The workflow is you script the path, it's fixed, even if you have a model in there. You can have a fixed path with a model in it where it is thinking about what's going on and then gives you the output. The agent decides the path, chooses its tools in a loop, and has bounded autonomy towards a

2:56goal. For the workflow, the path is fixed here. The inputs may be dynamic, but they're relatively fixed, and the outputs are consistent as well. There is limited or no internal judgment inside of that workflow. No one's making a judgment call about what tool it should use or which environment should go in or which database it should go get from. For the agent, the path can't be predetermined. The agent must verify its reality, and the agent has judgment over

3:22how it gets to that outcome. So, this is the more specific definition of an agent that we're going to kind of walk through. So, an agent is an LLM-driven loop that decides its next action, not fixed code, to pursue a goal using tools against the environment with autonomy bounded at a point that I decide, and observed on its trajectory. But, the bounded autonomy part is super important. So, what do I

3:49mean by bounded autonomy? It has autonomy inside of bounds. There is a environment and there's several ways to define what that bound is. But for agents specifically, we want to have very specific bounds over what they have access to, what tools they can use, which environments they can go into because if we don't have control over it, you have cost performance issues, security issues. The agent has a autonomy over how it gets to its end goal. So, at the start of it,

4:29we're starting with an outcome. Everything is outcome focused here. We define the intent ahead of time as the owner, as the operator for what the agent should be pursuing towards. We define what the outcome looks like, what good looks like. So, read, extract, verify, loop. Read the transcript, decide what matters, pull out the obligations and commitments made, check the files, does the promised asset actually exist? Loop until the goal is achieved. When people are talking about guardrails, what they usually mean is in the system prompt, I wrote "Please do not delete my database." Or, "Please do this." Or, "Don't do this." That is not a hard [snorts] bound. You are

5:06asking the model nicely to follow directions. It does not mean that it will do it. And in fact, I have plenty of examples personally of it not doing so. We can't just rely on the model to be both the governor and also the execution because the more rules that you that you provide it inside of the system prompt, the more it has to focus and pay attention to. And that's where you start to see performance

5:31degradation. How do we deal with that? There are many ways in which we can apply guardrails external of the actual model itself. We can have it enforce a specific schema output and then have code determine what it actually produced if that's inside the bounds of what we thought was acceptable, we can have scripts that run again that are

5:52deterministic, that verify its output. And the hard bounds can be enforced inside of the structure of the model itself. So, permission deny and and ask rules. There are the different modes you can go through. The manual, the ask for permissions, the auto mode, the dangerously bypass. That is a hard bound because every time it it is coming back and saying, "I have to have human

6:17input before I am allowed to proceed." Hooks are code that are injected at different parts in the process. That is a hard bound that we can control what those hooks are doing. Sandbox, this is one of the most widely used one. If you don't want the agent to have access to different environments, you put it inside of a sandbox. It literally can

6:35only be do stuff inside of that sandbox. But again, the soft bounds, if we're just relying on the model's judgment to not blow past its guardrails or to do things inside of a certain scope or to not spend so many tokens or not do certain things, we're just asking it nicely. So again, the hard bounds are what's critical as part of the engineering

6:57around these systems. So, how do we bound the agent specifically? There's a couple of different ways. Access, what it can reach. Approval, what needs a human. Budget, how much it consumes. You can have hard limits for how many tokens it's allowed to use as part of an action. Now, I don't necessarily like that. That's kind of fail-safe. Please don't spend $10,000 in tokens, but it's not the first thing that I would reach for when trying to do define a bound because if we hit that limit, but it was whatever, 500 tokens away from getting to an answer, we've stopped the loop and now we have to have someone go in and

7:34re-inject. Policy, deterministic rules inside of the code. Intent, what do we want as the output? And then, this is not really how we bound it, but how we observe it, right? Having some layer of observability on the agent itself agent, so we know when it ran, what the inputs and outputs were, um what it cost, what systems did it access, what tools it used. And the observability is super important because if you want to audit as part of a production environment, you have to have some trail of record of what the agent did. If you're just like, "Yeah, I used Claude code and let it run

8:09in a loop and access all these systems and change all these things." It's like, "Cool, but then your PM comes back and they're like, 'Hey, did you modify this code base?'" I was like, "No, sorry, Claude code did it." And it's like, "What did it change?" It's like, "Uh I don't know." So, again, the observability and the auditable trails of what ran is incredibly important for production environments. Can the agent reach out of its environment on the web and go and access things? And by default, yes, it can. If it shouldn't do that, that's another variable we need to control. When, where, and why should an

8:36agent go out and reach out into a different environment to pull in information or search for something? So, approval. What needs a human, right? Permission modes, plan default accept app bypass, dangerously bypass permissions, ask and deny rules, staging outboxes. We have the ability to control when an agent has to surface a need for a human to approve that next

9:00step. So, policy and finally deterministic rules enforce in code. So, I mentioned hooks, but again, they are code bits that are injected as part of the agent. Uh runs, we can define what that code is, deny and ask rules. Even now, some of the systems that I'm building, I'm using multiple models as part of the output. I might have a Sonic, for example, as part of a system where it has mostly the primary directive for what it's supposed to do, but then it hands off to another model, and that the only job of that model is to ensure that the output is within the appropriate schema or the

9:38appropriate output format, I should say, because I've had now several instances of fail open errors where the model has, for example, publicly disclosed its internal chain of thought reasoning instead of an actual output. That happens because the guardrails that were put in place by myself were just a system prompt. It was, "Hey, please output in this format." And because of the variation in the input and what happened with the model, it accidentally output its chain of thought instead of the actual output. And in some business environments, that's not acceptable. This goes back to how do we intelligently design and engineer these systems knowing what the failure modes

10:23of these models are and knowing that the guardrails are not hard limits when you're just putting it inside of a system prompt. The overview of this is I have a whole bunch of transcripts, call transcripts, and stuff that I process every single day as part of my business. The call transcripts on my system, they always land in the exact same file folder. I have a workflow that goes and sequentially with five different agents passes that same transcript five times and extracts out different data points every single time from that single transcript. What I'm describing there is

10:59not really an agent. That is a workflow. There are several models inside of it that are part of that process that runs, but they are not what I would define, as we defined previously, an agent because there's no bounded autonomy. The file lands in the same place. There is a cron job. It's like, "Hey, check this check when a new file lands." Automatically pulls up the the agents, spins it up, runs all those models in parallel, and then outputs in the appropriate format

11:28in the same place every every time. There is no deviation. There's no model thinking, there's no I'm going to go and check this other place and look for this. It has one job, read this transcript, pull out these elements, put them into a different file. The way that it works is an agent inside of Cloud

11:44Code is literally just a markdown file. There's nothing else weird about it. It is the call triage agent given a call transcript structure what what was committed to, verify each claim against reality, and turn every external follow-up into a stage approval ready file. It decides its own path, no fixed pipeline. I defined here what tools it can use. So, read, grep, drop, write, and bash. So, if I remove any of those tools from the list, it cannot use the tools. It's part of the schema that's sent in the API request. So, again, that's one of the hard bounds that when you're building these agents, you define what tools it

12:18has access to. It has the directive. But, what you'll also notice here is that it has the intent or what the outcome should be. This is hugely important for agent design. What does correct mean? What should the output look like? What is that golden directive that right you are grading yourself on? And that should be defined as clearly as possible with all of your inputs built in. But now, why is this an agent and it's not just a workflow? Other than it literally being called an agent, because it has bounded autonomy inside of its tool list, inside of the environments that has access to, and inside of its

12:55directive to choose its best path, and choose what tools it wants to use in order to get to that end result. One of the things that I like to do when I'm building out something that I know is going to be ran again and again is starting with the most optimal model. Now, it's Fable. Starting with Fable or Opus just for seeing what it takes in order to get to its outcome. And then continuously swapping in different models until we get to a point where we've maintained performance in the evals and benchmarks that we've set while continuously driving that cost as low as possible. You can build this in

13:28as a hook or as a skill that runs, "Hey, what failure points did you run into in this? Because it probably failed on some tool call, and then it had to rethink and then go and do something in a different direction." So, just telling it to basically recursively self-improve over that time period, "Hey, fix your own mistakes. Go back and edit that skill file. Go back and tell me what opportunities there are to improve." And sometimes it's also optimizing around the schema. So, if you have connectors with MCPs, if you know what the schema is or exactly which tools that you need to go and do that thing, don't give it

14:04everything. Give it only what it actually needs. That's a huge waste, too. Giving it six different connectors because all of that gets loaded into the context. The agent has all of that, and it doesn't need it, you're just wasting tokens because it has to read that every single time. Take advantage of caching. That's not a functionality that people really know about if you're in the desktop app or whatever, but if you can optimize around caching, so you don't have to constantly reread things. Check your global versus local because if you have your connectors in your global, those also get loaded into your local as well. What I've done here is

14:36it's supposed to go into the outbox and read the transcript and search for files in the outbox that are part of the action item and see if the file that we talked about is actually in the outbox or not. In the outbox, there's a bunch of random files. As a result, there's going to be a lot of downstream impacts with that, notably increasing cost because I told the agent, "Hey, go look in this folder and find the correct output that's correlated to that what we talked about on the call." If there's seven files or 20 files, it has to go in and read all those files to figure out what

15:15you asked for, if that's true. There's no other way for the agent to know what's in those files or if it's the correct file or not. So, it has to go read them. And if I've given it access and permission to go and do all of those things, it's going to go do them. And it's going to cost a lot of money. So, these are the type of things where when you're thinking about engineering and designing these type of systems, it's not, "Oh, we just tell the agent to be smarter." Or we tell it, "Hey, don't do these things or do these things." Are we engineering the environment around it so

15:43that it's optimized for the agent to interact with so that it achieves its outcome in the shortest time period possible at the lowest cost. What's awesome about LLMs is it will figure it out. It will go read it and go, "Hey, that file actually wasn't there." But, we didn't need it to read all those things. We could have been smarter on the engineering design, system design,

16:04so that it didn't have to do that.