A Framework for Engineering Variance with AI Agents
The Piston and the Cup Holder
The Piston and the Cup Holder
In mechanical engineering, "tolerance" is the permissible limit of variation in a physical dimension. It is the acknowledgement that the world isn’t perfect, and designs needs to account for that.
A plastic cup holder in a car has high tolerance. If the mold is off by a millimeter, or if the plastic warps slightly in the summer heat, it doesn’t matter. Your drink will still fit. The user experience remains intact.
A piston ring inside the engine block has extremely low tolerance. If it is off by a fraction of a millimeter, the seal fails, the compression drops, the engine seizes. The margin for error is effectively zero.
We can understand this intuitively in the physical world. You wouldn't use cheap plastic for a piston, and you wouldn't precision machine a cup holder out of hardened steel.
In software, we tend to treat our code bases more or less as uniform constructs. We apply the same processes, the same speed expectations, and increasingly, the same AI tools to every part of the system. We are being asked to "vibe code" our way through mission critical code and it’s dangerous.
If we want to use AI to accelerate delivery we need to stop asking "Can AI write this?" and start asking "What is the tolerance of this part of our system?"
I want to propose a way to categorize this into three specific software engineering tolerances.
1. Logical Tolerance
This is the permissible limit of deviation from truth. How "correct" does the output need to be?
High Tolerance: An animation, a prototype script, or some internal tooling. If the animation jiggles a bit or the script needs a retry, the failure state is annoyance.
Medium Tolerance: Search rankings, recommendation feeds, or "eventual consistency" counters (like a view count). If a user has 42 likes but the screen says 41 for a few seconds, or if a search result is ranked #2 instead of #1, the product may be “imperfect”, but it is still viable.
Low Tolerance: Idempotency keys, math, encryption standards, and payment execution. The failure state here isn't annoyance; it’s liability.
Generative AI is a probabilistic tool. It is a variance generator. It excels in High Tolerance zones because "mostly right" is often good enough. In Low Tolerance zones, like the payment path my team manages, correctness is binary. Code that is 99% correct will 100% fail for some uses.
2. Volatility Tolerance
This is the permissible limit of change a system can absorb. How stable is the system throughout change?
High Tolerance: A standalone microservice, a one-off migration script. These can change daily with minimal blast radius.
Medium Tolerance: Feature-level business logic, like a User Profile screen or a Settings page. These evolve regularly based on product requirements, but they rely on stable foundations and can't break every week.
Low Tolerance: Core platform libraries, API schemas, persistent storage, and foundational data models. These evolve slowly as their impact exponentially grows.
AI promises infinite velocity, to push code as fast as it consumes electricity. Not every part of a system can survive infinite velocity. In an area with low Volatility Tolerance, the constraint isn't how fast you can write code, but how safely you can integrate it over time.
3. Cognitive Tolerance
This is the limit of complexity, and the rate of change, a human can verify and internalize in a reasonable time frame.
High Tolerance: Boilerplate, standard patterns, and simple unit tests. The code is obvious. If AI writes it, it can be scanned in seconds and stamped with "LGTM."
Medium Tolerance: Standard data mapping (DTO to Domain), form validation, or ViewModel state management. It requires actual reading and context, but the logic is generally linear and self-contained.
Low Tolerance: Distributed system logic, concurrency handling, and security protocols.
This isn't just about the verification tax: the time it takes to reason about a complex block of code. It is about the erosion of confidence.
Engineers rely on deep mental models to predict how a system will behave. These mental models are built on assumptions and historical understanding. If we are encouraged and AI allows us to change too much code too quickly, our mental models can’t keep up. Even if the new code is "better", if it invalidates our core assumptions faster than we can relearn them, we lose our ability to reason about the system.
In Low Tolerance zones, we need the code to match our mental model. If we lose that alignment, we are effectively shipping instant legacy code, systems that become brittle immediately because we are too afraid to change what we don't fully understand.
The Core Thesis
AI can be a powerful accelerator, but it is introducing a new way of building software where we need to be very aware of these tolerances .
To use it effectively, we have to map our code base against these three axes.
You cannot readily apply high-variance tools to Low Tolerance zones (the Pistons) without risk of seizing the engine. Conversely, you shouldn't leverage your engineers' time hand-crafting the High Tolerance cup holders.
As the industry pushes for more and more AI adoption, these terms allow us to move the conversation from "Can we use AI?" to "What is the cost?" to “How do areas of our code base thrive with AI?”
We can use this framework to explain that while AI might accelerate code generation in some critical paths, it introduces risks to Logical Tolerance that require a disproportionate investment in verification. It gives us the nuance to say, "If we accelerate here, we are trading off Cognitive Tolerance, which means our future maintenance costs will go up."
The goal of this series is to establish Logical Tolerance, Volatility Tolerance, and Cognitive Tolerance as a shared vocabulary for these trade-offs such that they can be used in key engineering decisions.
This was Part One of The Tolerance Trilogy, thanks for reading! Up next, we’ll dive into why AI written code specifically struggles within large software systems and how we can go about using it successfully.
