How modern LLM agents work
A large language model agent puts a language model in the decide step. The model reads the situation in plain language and chooses the next action. Surrounding code, usually called the harness or runtime, carries that action out. Four components do most of the work.
Tools and function calling
The model cannot touch your systems directly. Developers describe each tool with a name, a purpose and a strict input schema, for example find_open_slots(technician, after, duration_hours). When the model wants to act, it returns a structured call instead of prose. The runtime validates the arguments, makes the real API request, and hands the result back to the model as a new observation.
That separation is the most important safety property an agent has. The model proposes; your code decides whether the proposal runs. Every tool is a door you chose to build, with a lock you control.
# one task, traced step by step
observe email from j.ortiz: "move thursday's install, add a 2nd unit"
think two intents: reschedule + order change
call find_open_slots(technician="any", after="thu", duration_hours=3)
result ["mon 09:00", "tue 13:00"]
call draft_reply(to="j.ortiz", slots=[...], change="add unit")
result draft saved
gate order change alters the invoice total -> needs approval
stop queued for a person; nothing sent yet
Memory: short term, long term and retrieval
Short-term memory is the context window: the conversation, tool results and instructions. It is precise but limited, and paid for again on every step. Long-term memory lives outside the model: a customer record, a notes table, a summary written back after each session.
Retrieval connects the two. Documents are split into passages and indexed, usually as embeddings, and only the passages relevant to the current step are pulled into context. This is retrieval-augmented generation (RAG), and it is how an agent answers from your actual price sheet instead of its general training. Facts the agent must never get wrong belong in a lookup, not in its recollection of a long chat.
Planning
Simple agents take one step at a time: reason, act, observe, repeat, a pattern often called ReAct. Harder tasks do better with plan-then-execute. The model drafts a short plan, the runtime works through it, and the plan is revised when an observation contradicts it. Either way, cap the steps. An unbounded planner is how an agent ends up calling the same search tool forty times in a row.
Guardrails
Guardrails are the code between the model's intentions and the real world:
- An allowlist of tools, each scoped to the minimum data it needs.
- Schema validation on every argument, with rejections fed back to the model.
- Approval gates for money, customer-facing messages, deletions and anything irreversible.
- Budgets per task: maximum steps, tokens, spend and wall-clock time.
- A kill switch that halts the agent and leaves a clean audit log of every action it took.