“Let’s build an agent” has become the default answer to almost any GenAI problem. Often it’s the wrong one. An agent decides its own steps, which makes it flexible, but also slower, more expensive and much harder to test than code that follows a plan.
I find it useful to think of GenAI systems as a ladder. Each rung adds capability, and each one adds latency, cost and things that can go wrong. The skill is to climb only as high as the task needs.
Two small projects of mine sit on the first two rungs, so I’ll use them to make the lower rungs concrete, then walk up the rest.
Rung 1: one LLM call, done well
Quippy is a multi-model chat assistant I built during a GenAI boot camp. Under the hood it’s a single chat call per message. What makes it feel like an assistant is everything around that call:
- A persona in the system prompt. Quippy is witty, explains with analogies and admits when it isn’t sure. The default prompt sets that character, and users can override it.
- Memory as conversation history. Each turn sends the system prompt, the full history and the new message. That’s all “memory” is at this rung.
- Provider-agnostic by design. Users pick from Groq-hosted open models (Llama 3.1 8B, Llama 3.3 70B, GPT-OSS) or OpenAI’s GPT-5 family, and can switch without re-entering keys.
The last point taught me the most. Providers look alike until you swap them. GPT-5 models don’t accept a custom temperature and expect max_completion_tokens instead of max_tokens, so the request has to adapt to the model:
if self.model in self.GROQ_MODELS:
params = {"model": self.model, "messages": messages,
"temperature": temperature, "max_tokens": max_tokens}
else: # OpenAI GPT-5 models
params = {"model": self.model, "messages": messages,
"max_completion_tokens": max_tokens}main.py
Clients are also created lazily, only when a chat request arrives, so the app can start and show its settings before any API key is entered.
When this rung is enough: Q&A, drafting, explaining, rewriting. A good system prompt and history cover far more than people expect. Keep the provider swappable from day one; you’ll want to compare models later.
Rung 2: a fixed workflow
The blog generator takes a topic and writes a blog post. Instead of one big prompt, it’s a two-step LangGraph workflow: one node writes the title, the next writes the content.
self.graph.add_node("title_creation", self.blog_node_obj.title_creation)
self.graph.add_node("content_generation", self.blog_node_obj.content_generation)
self.graph.add_edge(START, "title_creation")
self.graph.add_edge("title_creation", "content_generation")
self.graph.add_edge("content_generation", END)src/graphs/graph_builder.py
The steps never change, so there’s nothing to decide. That’s the point. Splitting the job gives each prompt one task, lets you test and improve each step on its own, and opens the door to using a different model per step. A typed state object carries the topic and the post between nodes, and LangGraph Studio shows each step’s input and output when something looks off.
When this rung is enough: whenever you can write the steps down in advance. That covers a surprising share of business processes: extract, then classify, then summarise, then format.
Rungs 3 and 4: workflows that decide
The next two rungs keep a fixed structure but let parts of it make choices. These are the patterns I reach for most often:
- Routing. A classifier step looks at the input and picks a branch. Both of my projects have natural routing hooks. Quippy could send easy questions to a small, fast model and hard ones to a larger one instead of making the user choose. The blog generator’s state already has an unused
current_languagefield, an obvious place for a router that sends posts down a translation branch. - Parallelisation. Independent steps run at the same time and merge at the end, for example writing several sections of a long post at once. It saves time, not tokens.
- Evaluator–optimiser. A reviewer step scores the output against clear criteria and sends feedback to the generator, which tries again, up to a fixed number of rounds. For the blog generator, that could be a check for structure, accuracy and tone before a post is returned.
Each adds a call or a loop, so each adds latency and cost. Routing can misroute, and a reviewer is only as good as its criteria. Add them when there’s a measurable reason.
Rung 5: agents
An agent is a loop: the model looks at the goal, chooses an action (often a tool call), observes the result, and decides what to do next until it’s done. Nobody writes the sequence of steps in advance.
That’s exactly what you need when the path depends on what the system discovers along the way: investigating a support issue across several systems, or analysing a document set where each finding changes what to look at next. It’s overkill when the steps are known.
If you do climb this far, bring the guardrails with you: tightly scoped tools, a maximum number of steps, tracing of every action, and an evaluation set that tests whole runs, not just single answers.
How to choose
A few questions settle most cases:
| Question | If yes |
|---|---|
| Can one well-prompted call do it? | Stay on rung 1 |
| Can you write the steps down in advance? | Use a fixed workflow |
| Do different inputs need different handling? | Add routing |
| Is quality measurable, and worth a retry? | Add an evaluator loop |
| Do the steps depend on what you find along the way? | Now consider an agent |
The takeaway
Agents are the top of the ladder, not the starting point. Most of the value in GenAI systems comes from the lower rungs done well: a clear system prompt, swappable models, small workflow steps that do one thing each, and routing or review loops where they earn their cost. Climb only as high as the task needs, and you’ll ship systems that are cheaper, faster and far easier to trust.
If you’re deciding how much autonomy your own system needs, I’d love to compare notes. Find me on LinkedIn.