pencil ai

Why agents need a System 1

Most of what an agent decides is quick. We think the machinery should be quick too.

Pencil AI Team · November 2026

Watch an agent work on a coding task and count its decisions. Which tool should it use next? Did the last edit land in the right file? Have the tests settled the matter? Is this shell command safe to run? Most of these are small questions with a short list of possible answers. Only a few steps involve writing anything.

In many agent loops, each of those questions goes to the same large language model that does the writing.1 We say "many" because harnesses differ and some use plain rules for part of this. The pattern we describe is the common one among the open harnesses we studied.[4] The model reads the whole context again, generates a paragraph of reasoning, and eventually produces "yes". This works. It is also slow, it costs tokens, and it gives you no number you can check for how sure the model was.

We think agents need a second, faster mode for these questions. This essay explains what that mode is, how it fits with the model you already use, and what has to be true before you should trust it.

Two kinds of thinking

Daniel Kahneman gave names to two modes of human thought.[1] System 1 is fast and automatic: you flinch when a ball comes at your face. System 2 is slow and deliberate: you work out 17 times 24 on paper.2 Kahneman uses the labels as a way of telling a story about thought, not as a claim about two regions of the brain. We borrow them in the same spirit. Someone who used System 2 for every flinch would not last a day. Someone who used System 1 to draft a contract would be reckless. A healthy mind routes between the two.

Agents today mostly have the second mode only. Every question gets careful thought, including the ones that deserve a reflex. The cost is not only time. Careful thought generated in prose is hard to audit, because the answer is a paragraph and the paragraph can be persuasive without being right.

What a decision model is

A decision model is a small model that does not write. You give it a piece of text and a closed list of questions. It returns a probability for every answer to every question, in a single pass, with no generated words.

"Closed" is the important part.3 Yes or no, pick one of N, or a score on a scale. Open-ended questions such as "how should we fix this?" stay with the big model. The questions look like these: is the task done, continue, stuck, or does it need the user? Does this tool fit the step? Does this claim say more than its source? Is this text trying to give the agent instructions? Each has a short list of answers, and each list includes "unclear".

Two properties follow from the shape of the job. It is fast, because nothing is generated. And its output is a set of numbers, which you can compare with what happened afterwards. A paragraph of reasoning cannot be scored that way. A probability can.

Questions that concern the same piece of text are asked together. If the agent wants to know whether a tool call fits, whether it is aimed at the right file and whether it is risky, those are three answers from one pass over one request.

The ladder

A decision model does not replace the big model. The two sit on a ladder with a rule below them and you above them.

The bottom rung is a deterministic rule. If tests are red, the task is not done, and no model needs to say so. Next comes the decision model, which answers the closed questions rules cannot. Above it is the big model, which handles anything open-ended or anything the decision model is not sure about. The top rung is you.

A judgement stops at the first rung that is sure. If a rung is unsure, the question moves up. It never moves sideways and it never turns into a guess. We call the principle fail safe, not open: when the answer is unclear, the system takes the cautious side, which usually means asking.

Calibrated, not confident

A fast answer is only useful if you know when to believe it. That is a question about calibration. A forecaster who says "70% chance of rain" is calibrated if it rains on about seven of every ten days they say that.4 The idea is old and simple. Take every answer given 90% confidence, count how often it was right, and compare. A calibration plot draws that comparison.

We do not trust a model's own stated confidence. Models are often more sure of themselves than their record justifies. Instead, each kind of decision is calibrated on labelled examples, so a stated 90% means roughly nine in ten. Then each answer gets its own threshold. An answer whose mistake would be harmful, such as approving a risky command, needs a stricter threshold than one whose mistake costs a retry.

There is outside evidence that this approach is workable. The Artificial Analysis Decision Index reports that on classification and tool-calling benchmarks, leading decision models score comparably to or above frontier language models run at low reasoning effort. It also says calibration is still imperfect overall, and best on When2Call.[2] We cite it as evidence about the category. It is not a measurement of our model.

Earning trust

Even a calibrated model should not start in charge. In Pencil, a decision point moves through stages. It begins off, so the harness behaves as it did before. Then it runs in shadow, answering beside the existing logic while changing nothing, so you can compare the two over real work. It goes live only when that comparison supports it. After that, answers keep being checked against outcomes, and a decision point that drifts is demoted automatically.

Every decision is stored as a record: the probabilities, the threshold, which rung decided and which model version answered. That makes runs replayable. If an agent did something surprising last Tuesday, you can step through the decisions that led there.

The harness around the decision model

A decision model is one part. The rest of the harness matters as much, and most of it is plain engineering.

Edits that cannot quietly go wrong

An edit applies only if the text it targets appears exactly once. If the file has changed since the model read it, the write is refused. Pencil hashes the file before and after an edit to catch changes nobody asked for, keeps an undo journal, and can deliver edits as suggestions for you to review.

Checks before opinions

Code checks run before any model is asked to judge a result. A model judge can add a failure to a check but can never clear one that code has found.

Rails in code

Limits on turns, tokens and time, loop detection and caps on malformed replies live in the harness. Errors are typed and tell the model the next step, so a failure becomes an instruction instead of a puzzle.

Small windows, any model

Pencil is built for context windows as small as 16K, which keeps small and local models practical. It uses one OpenAI-compatible client, with per-model quirks in config, and runs 22 checks against an endpoint before using it.

A safety floor

Some rules sit below every setting. Text from the outside world is treated as data, never as instructions. Under the rule of two, untrusted input combined with private data cannot cause an outside effect without a human in the loop.

Why the harness matters at all

The loop around a model carries real weight, and cost is the easy place to see it. One study of agent context found that simply masking old tool observations was as efficient as asking a model to summarize them, and that both more than halved cost compared with a raw agent.[3] Nothing about the model changed. That result is about harnesses in general. It does not measure Pencil.

What we have not shown yet

We are pre-product. We have no benchmark results of our own to publish, and we have not run Pencil on customer work. The claims in this essay about the decision model are claims about design, supported where we say so by outside sources. When we do publish measurements, we will publish the method first: the tasks, the scoring, the model versions and how to rerun it.

If you want to try the harness when it is ready, you can request early access from the home page.

Notes

  1. Daniel Kahneman, Thinking, Fast and Slow, Farrar, Straus and Giroux, 2011. https://en.wikipedia.org/wiki/Thinking,_Fast_and_Slow
  2. Artificial Analysis, "Artificial Analysis Decision Index", 13 October 2026. https://artificialanalysis.ai/articles/artificial-analysis-decision-index
  3. Lindenbauer et al., "The Complexity Trap: Simple Observation Masking Is as Efficient as LLM Summarization for Agent Context Management", arXiv 2508.21433. https://arxiv.org/abs/2508.21433
  4. "World's first": among the agent harnesses we studied in October 2026 (OpenCode, Pi, Hermes Agent, DeepSeek Harness and Codex CLI), some offer typed decision calls, but none runs a calibrated decision model inside the loop by default. Our own survey.

Citation

Pencil AI Team. "Why agents need a System 1." Pencil AI Research, November 2026. https://pencilton.com/research/why-agents-need-a-system-1/

For BibTeX:

@misc{pencil2026system1,
  author       = {{Pencil AI Team}},
  title        = {Why agents need a System 1},
  howpublished = {Pencil AI Research},
  year         = {2026},
  month        = nov,
  url          = {https://pencilton.com/research/why-agents-need-a-system-1/}
}