AI Engineering Agent Architecture Generative Systems · August 2026

Same LLM, Different Harness.
The Model was never the whole product.

“People who are really serious about software should make their own hardware.” — Alan Kay
Pradeep A. Dalvi
Pradeep A. Dalvi
Principal Systems Architect · Distributed Systems & Payments
14 min read

Context / RAM

What the harness loads before each call — files, prior results, history — is the model's entire working memory. Same weights, different context, different answer.

Tools / BUS

The model can only describe an action until something executes it. Tool access is what turns a completion into a file that actually exists on disk.

Orchestration / LOOP

One shot versus repeated observe-and-retry. Whether a failed step gets a second attempt is a harness decision, not a model one.

Verification / COOLING

A generative system's output isn't guaranteed to repeat itself. Something has to re-open the artifact and check it before calling the job done.

01

The Spec Sheet Doesn't Ship a Laptop

Take two laptops off the shelf with the identical CPU — same silicon, same clock speed, same core count, same cache. Benchmark them and you'll get two different numbers. One has fast RAM and a real heatsink; the other has half the memory and a fan that gives up under load. One never throttles. The other hits 95°C in ninety seconds and quietly halves its own clock speed to survive.

Nobody looks at that gap and concludes the CPU lied about its spec sheet. The chip did exactly what it was rated for. What changed was everything sitting around it — the parts that decide whether the chip's raw capability actually reaches the keyboard, or gets throttled, starved, and swallowed before it does.

This is the argument I made to an engineer recently, after suggesting a coding agent over an AI-augmented IDE for generating a non-trivial .xlsx file. His pushback was fair: “ultimately, how would it be different when the underlying LLMs are the same?” It's the right question. It's also the CPU question, asked about a different kind of chip.


02

Deterministic Silicon, Generative Silicon

Before comparing harnesses, it's worth being precise about what kind of chip is inside the box, because it isn't the same kind of chip the CPU analogy started with — and that difference is exactly why the harness has to work so much harder.

A classical CPU executing an ALU instruction is deterministic by construction. Add two registers and you get one answer, every time, on every run, forever — the same inputs producing the same outputs is the entire basis on which software engineering works. It's why a unit test written once stays true. It's why a stack trace points at a real, reproducible line. Correctness is a property you prove once and then trust.

A large language model doesn't work that way, and not just because someone left the temperature dial turned up. Inference means sampling from a probability distribution over the next token. At temperature above zero, the same prompt legitimately produces different completions on different calls — that's not a bug, it's the mechanism. And even at temperature zero, with supposedly deterministic “greedy” decoding, floating-point arithmetic on GPUs is not strictly associative: batching, kernel scheduling, and parallel reduction order can shift rounding just enough to change which token wins a close race. Reproducibility isn't guaranteed by the hardware the way it is for a CPU add instruction — it has to be engineered on top, if you want it at all.

Infographic illustrating how Deterministic Silicon & Generative Silicon work.
Figure 1: Deterministic Silicon, Generative Silicon.
Property
Traditional Software
Generative Model
Same input, same output
Guaranteed by the hardware
Not guaranteed — sampling + floating-point order
Typical failure mode
Crash, exception, stack trace
Confident, plausible-looking wrong answer
How you gain confidence
Unit test once, trust forever
Verify the specific output, every time
What “correct” means
Matches the specification exactly
Is this instance good enough to ship

The consequence: in a deterministic system, verification is a one-time cost — you write the test, it holds. In a generative system, the model's own output is not a fixed, checkable artifact by default. The harness's execution-and-verification layer is the closest thing to a repeatable test a generative system actually has — which is exactly why that layer matters so much more here than it ever did for classical software.


03

The Model Is the CPU

A large language model is, structurally, a very good next-token predictor with a very large amount of world knowledge compressed into its weights. That's the silicon. It is genuinely where most of the “intelligence” lives, in the same sense that the CPU is genuinely where most of a computer's raw compute lives. Two products calling the same model checkpoint are, at that layer, identical. Same weights, same training, same reasoning ceiling.

But a model in isolation cannot open a file. It cannot run Python. It cannot look at the workbook it just described and notice that the totals don't add up. It generates tokens — text that describes an action, or text that is a piece of code — and then stops. What happens to those tokens next is not the model's decision. That decision belongs to whatever is holding the model.

The tell

If two products can be compared by asking “which LLM does it use,” you've already conceded that the LLM is a component, not the product. Nobody buys a laptop by asking which CPU die it uses and stopping there.


04

The Harness Is the Computer

Agent Harness, n. phr.
the system prompt, context-assembly logic, tool definitions, orchestration loop, sampling configuration, memory, and permission boundaries wrapped around a model checkpoint. The model is one input to this system, not the system itself.

Everything wrapped around the model is usually called the agent harness — the system prompt, the tool definitions, the context assembled before each call, the loop that decides what happens after each response, the permissions on what it's allowed to touch. It's the motherboard, the RAM, the cooling, the storage bus. It doesn't make the CPU smarter. It determines how much of the CPU's capability actually turns into useful work.

Agentic GenAI Core Architecture
┌───────────────────────────────────────────────────────────────────┐
│                           USER GOAL                               │
│               "Generate a quarterly sales .xlsx"                  │
└────────────────────────────────┬──────────────────────────────────┘
                                 │
                                 ▼
┌───────────────────────────────────────────────────────────────────┐
│                        CONTEXT ASSEMBLY                           │
│  • Task framing: xlsx skill loaded, openpyxl/xlsx tool available  │
│  • Ephemeral state: uploaded source data (CSV/sheet), if any      │
│  • Episodic memory: prior turns in this task (columns agreed on)  │
│  • Long-term retrieval: user's past formatting preferences        │
└────────────────────────────────┬──────────────────────────────────┘
                                 │
                                 ▼
┌───────────────────────────────────────────────────────────────────┐
│                         LLM REASONER                              │
│        Thought: "Need to read source data, compute totals,        │
│                 then write formatted .xlsx"                       │
└─────────────────┬───────────────────────────────┬─────────────────┘
                  │                               │
         (task not done yet)               (file built & verified)
                  │                               │
                  ▼                               ▼
   ┌─────────────────────────┐      ┌────────────────────────────┐
   │      TOOL EXECUTOR      │      │       FINAL ANSWER         │
   │     Calls: bash / xlsx  │      │  present_files → user gets │
   │     library functions   │      │  the .xlsx download link   │
   └─────────────┬───────────┘      └────────────────────────────┘
                 │
                 ▼
   ┌─────────────────────────────────────────────┐
   │              ENVIRONMENT                    │
   │  • Reads uploaded data file                 │
   │  • Writes .xlsx via xlsx skill/library      │
   │  • Runs a check (open file, validate cells) │
   └──────────────────────┬──────────────────────┘
                          │
                          ▼
   ┌────────────────────────────────────────────┐
   │              OBSERVATION                   │
   │  • "Sheet written, 3 columns, 40 rows"     │
   │  • Reflection: totals row missing → replan │
   └──────────────────────┬─────────────────────┘
                          │
                          │  ↻ loop back — re-assemble context with
                          │    the fresh file state (not a stale log
                          │    entry saying "file was written")
                          ▼
              [ back to CONTEXT ASSEMBLY ]

This is what Alan Kay's line is really getting at, read outside its original hardware context. Being serious about the outcome means owning the peripherals — not just consuming the chip someone else shipped you. A product that only calls a model API and passes the raw completion straight back to the user hasn't built any hardware at all. It's running someone else's reference board with no cooling and no bus, and hoping the die never needs either.

RAM — Context Management

A CPU starved of memory bandwidth spends its cycles waiting, not computing. A model starved of the right context spends its reasoning guessing, not knowing. What a harness loads before each call is the model's effective working memory.

Bus — Tool Access

A CPU with no disk controller can compute beautifully and persist nothing. A model with no file-write tool can reason beautifully about a spreadsheet and produce nothing but a description of one.

Cooling — Verification

A CPU without cooling doesn't stop — it throttles silently and keeps returning output that looks fine. An agent without a verification step doesn't stop either. It keeps producing plausible artifacts, and nothing flags the moment they went wrong.


05

Inside the Harness: What's Actually Running

“Context, tools, loop” is the right shape, but it's worth opening the box, because this is where two products with the same checkpoint diverge in practice.

The model is stateless. The harness is not.

Every single API call to the model is independent — there is no session, no memory, no persistence on the model's side between one request and the next. The entire conversation — system prompt, prior turns, every previous tool result — is re-sent, in full, as one message array, on every call. The model doesn't “remember” using a tool three turns ago; it re-reads the transcript that says it did. Statefulness is not a property of the model. It's a property of whatever is assembling that array before each call — which means two harnesses can hand the identical model two different transcripts of the “same” conversation, and get two different agents out of it.

Inside ReAct (Reason + Act) Loop
┌──────────────────────────────────────────────────────────────┐
│                                                              │
│   1. PERCEIVE / OBSERVE                                      │
│      Current state + previous observation                    │
│                                                              │
│   2. REASON (Thought)                                        │
│      LLM thinks: “What do I know? What should I do next?”    │
│                                                              │
│   3. ACT                                                     │
│      Call a tool / write code / edit file / search, etc.     │
│                                                              │
│   4. OBSERVE                                                 │
│      Get real result from the environment                    │
│                                                              │
│   5. Check: Goal achieved?                                   │
│      • Yes → Final Answer                                    │
│      • No  → Go back to step 1                               │
│                                                              │
└──────────────────────────────────────────────────────────────┘
Tool-calling — the actual request/response cycle
// Simplified structured tool-use loop
messages = [system_prompt, user_request]
tools    = [{name: "read_file", schema: {...}},
             {name: "run_shell", schema: {...}},
             {name: "write_file", schema: {...}}]

loop:
  response = model.call(messages, tools)

  // model emits either plain text, or a structured tool_use block —
  // never actually executes anything itself
  if response.type == "tool_use":
      result = harness.execute(response.tool_name, response.args)
      messages.append(response)                    // the call
      messages.append({role: "tool_result", content: result})  // the outcome
      continue   // re-invoke with the new transcript
  else:
      return response.text   // model signalled it's done

Notice what the model never does anywhere in that loop: touch a file, spawn a process, or know whether its own code ran successfully. It emits a request to do so, formatted against a JSON schema the harness defined. The harness is what turns that request into a real side effect and reports back what actually happened. Swap the harness and keep the model fixed, and the set of tools offered, how failures are reported back, and what triggers another iteration versus a final answer, all change — even though the “brain” making the decisions is byte-identical.

The loop, step by step
1
Assemble context

Harness decides what goes in the array this turn — which files, which prior results, how much history survives compaction once the window fills up.

2
Model reasons and (maybe) calls a tool

Sampling parameters — temperature, top_p, max tokens — are set by the harness, not the model. Same weights, different knobs, different variance run to run.

3
Harness executes, inside a sandbox

Permission boundaries live here: an allow-list of tools, a filesystem jail, a human-approval gate before anything destructive runs.

4
Result appended, loop or return

Whether a failed tool call triggers a retry, a different tool, or a shrug back to the user is harness logic — the model only sees whatever transcript it's handed next.

None of this is exotic — it's the standard shape behind most production coding agents. But it means “same LLM” has already quietly assumed away five decisions: what context reaches it, what tools it's offered, what sampling settings drive its variance, what sandbox constrains its actions, and what happens when a step fails. Every one of those is a harness choice.


06

Where This Shows Up: Building a Spreadsheet

Zoom out and the whole pipeline from request to file looks like this — the LLM is one box out of five, and it never touches the last two:

Infographic illustrating how Agentic GenAI processes the request.
Figure 2: Request to .xlsx — the full pipeline.

The “Agent / IDE layer” box is doing everything discussed in the last three sections — it decides what the LLM sees, what actions it's even allowed to propose, and whether the runtime's output gets checked before it's called done. Swap that one box between two products and the bottom four boxes can behave completely differently, even when the LLM box is byte-identical. Here's what that divergence actually looks like for a financial model built from a folder of CSVs — same weights both times.

Harness A vs. Harness B — same checkpoint, different outcome
// Harness A — no execution tool, no file access
model.generate("Create an Excel workbook with a P&L, revenue and expenses.")
→ returns openpyxl source as text.
→ nothing executes it.
→ nobody checks whether Revenue − Expenses actually equals the EBITDA cell.

// Harness B — shell + file tools + a verify step in the loop
1. read_file(revenue.csv, expenses.csv)
2. write_file(build_model.py)
3. run_shell("python build_model.py")
4. read_file(financial_model.xlsx)        // open what it just built
5. detect: EBITDA cell off by 3,000
6. edit build_model.py, re-run, re-check
7. return the file only once it passes

The model's raw capacity to reason about a P&L statement hasn't changed between A and B. What changed is that B has a bus (it can execute and persist) and B has cooling (it inspects its own output before calling the job done). A never even notices the arithmetic error, because nothing in A's loop ever looks.


07

Cooling Is the Verification Loop

Push the analogy one step further, because it holds up better than it has any right to. A throttling CPU doesn't throw an error. It degrades quietly and keeps returning results — just slower, worse results, with no flag raised. That is precisely the failure mode of an agent with tool access but no verification step: it doesn't crash on a broken workbook. It hands you the broken workbook, confidently, because nothing in its loop was built to notice.

Tool access expands what an agent can attempt. Verification is what tells it — and you — whether the attempt actually worked.

This is also the fullest sense of Alan Kay's line. Owning your hardware was never just about having more peripherals bolted on — it was about controlling the parts that determine whether the system is trustworthy under load. For a generative system, that trust can't come from the model alone, because Section 2 already established that the model's output isn't a fixed thing you get to test once. The verification loop is the harness builder's version of building your own hardware: it's the part nobody else will build for you, and it's the part that decides whether “it has shell access” means anything at all.


08

What to Actually Compare

“Same LLM” answers exactly one question, and it's rarely the one that matters for a checkable artifact like a spreadsheet. When comparing products like Claude Code, Cursor, Copilot, or Codex, the model is the first question — not the only one. The full comparison, at the harness layer:

Layer
CPU Analogy
What It Actually Decides
Model
The die itself
Reasoning ceiling — identical if the weights are identical
Context
RAM & bandwidth
How much and what context actually reaches the model
Retrieval
Cache / prefetch logic
How relevant files and code get selected into that context
Tools
Storage / I/O bus
Shell, filesystem, browser, Git, MCP, APIs — what's reachable at all
Execution
Instruction pipeline
Whether generated code can actually run, not just be shown
Agent loop
Motherboard / scheduler
Whether it can observe → reason → act repeatedly, or gets one shot
Verification
Cooling & sensors
Whether it runs tests or validates artifacts before calling it done
Permissions
Power delivery limits
What it's actually allowed to modify or execute
Memory
Persistent storage
What survives and gets retained across separate tasks
UX
Case / form factor
IDE vs. terminal vs. chat — how you actually drive the loop
Cost / latency
Clock & power draw
Tokens, caching, tool-call count, inference speed
Reliability
MTBF / error recovery
How gracefully the orchestration handles a failed step
Yes, but the LLM is only the reasoning engine. The difference is in context engineering, tool access, execution environment, orchestration, agent loops, verification, and the way the product converts model output into real-world actions.

That's the technically strong response to “but the underlying LLMs are the same” — and it's every layer in the table above, not just the analogy this piece opened with.

Run this before the next “but it's the same model” comes up
  • Can it execute what it generates, or only describe it?
  • Does anything re-open the artifact and check it before calling the task done?
  • What context does it actually receive — the whole picture, or a fragment?
  • Does it get one shot, or a loop it can iterate inside?
  • Are the sampling settings tuned for reliability, or left at defaults built for chat?

Answer those five, and you've compared the computer — not just quoted the CPU's spec sheet.

The die was never the whole machine. It was always the part that's easiest to point at, and the least useful thing to argue about.

And for a system whose own output can't be trusted to repeat itself, the systems you build around it aren't optional scaffolding. It's the only place “correct” gets decided at all.

“Make your own hardware.”

Kay meant the peripherals. For a generative system, the harness is the peripheral that decides whether the output can be trusted.

If you lead engineering teams, working on AI enablement (AI Assisted or AI Driven) of engineering systems at scale, or find yourself thinking about building something using Agentic GenAI — I’d love to connect. Discussing practical use-cases are far more interesting than simple tech-talks on theoretical system.

linkedin.com/in/pradeep  ·  Pune, India