Harness Engineering

Aug 27, 2026 08:54 PM - 2 weeks ago 8

Harness engineering is the believe of surrounding AI-assisted codification procreation pinch deterministic tooling, agent-based review, and periodic entropy checks truthful that AI-generated codification stays correct and coherent complete time. This archive explains wherever the thought came from, what it consists of, and really this plugin implements it.

The Origin¶

The word comes from Birgitta Boeckeler's article connected martinfowler.com, written successful the ThoughtWorks discourse of teams shipping existent package pinch AI coding assistants. Boeckeler observed thing that galore teams had noticed independently: AI assistants nutrient plausible-looking code, but near unconstrained they drift. They hide conventions, repetition mistakes, and slow erode the soul consistency of a codebase. The codification continues to compile and walk tests. The degradation is quiet.

Boeckeler's penetration was that this problem already has a solved analogue successful package engineering: the trial harness. Tests do not make codification correct by construction. They observe erstwhile codification stops being correct. A trial harness is not a constraint connected what codification you write; it is simply a system that continuously checks whether what you wrote meets a standard. The harness does not spot the programmer. It verifies.

The aforesaid logic applies to AI-assisted development, pinch 1 important difference. Test harnesses cheque functional correctness: does the programme do what it is expected to do? A harness for AI coding needs to cheque thing broader: does the codebase still embody the architectural decisions, naming conventions, information constraints, and structural rules that the squad has agreed on? Functional tests are basal but not capable for this. You request a different benignant of harness.

That is what harness engineering provides.

The Three Components¶

Boeckeler describes 3 categories of interest that a harness must address.

Context Engineering¶

An AI coding adjunct tin only activity wrong what it knows. If it does not cognize that your task uses a peculiar logging library, it will invent its ain approach. If it does not cognize that you ne'er usage mutable world state, it will usage it erstwhile convenient. If it does not cognize that each database writes must spell done a circumstantial abstraction layer, it will bypass that layer.

Context engineering is the subject of making judge the AI knows what it needs to know. In practice, this intends maintaining a archive — HARNESS.md successful this plugin's conventions — that captures the stack, the architectural decisions, the naming conventions, the constraints, and the rationale down each of them. This archive is not a README for humans. It is simply a knowledge guidelines for the AI. It needs to beryllium accurate, specific, and kept current.

The favoritism matters: a README explains what the task does. A discourse archive tells an AI supplier what it must and must not do, and why. These are different documents pinch different audiences and different update rhythms.

Architectural Constraints¶

Knowing the rules and enforcing the rules are abstracted problems. You tin constitute each constraint into HARNESS.md and the AI will still break them, because the AI is simply a probabilistic strategy optimising for plausibility, not a rule-following machine. Context engineering reduces violations. It does not destruct them.

Architectural constraints are the mechanisms that drawback violations. Boeckeler calls the enforcement points "verification slots" — defined moments successful the improvement workflow wherever a cheque runs and either passes aliases blocks progress. The cardinal creation determination for each verification slot is whether it uses a deterministic instrumentality aliases an agent-based review.

A deterministic instrumentality is simply a linter, a script, a regex check, a file-structure assertion — thing that produces a pass/fail consequence without judgment. These are preferable erstwhile the constraint tin beryllium expressed precisely. They are fast, cheap, and wholly reliable wrong their specification.

An agent-based reappraisal is simply a connection exemplary looking astatine codification against a constraint explanation and making a judgment. This is basal erstwhile the constraint involves intent, semantics, aliases patterns that are difficult to definitive arsenic a mechanical rule. Agents are much costly and little deterministic, but they tin drawback things that nary book tin catch.

Both types of verification slots beryllium successful a harness. The extremity complete clip is to migrate constraints from agent-based to deterministic arsenic your knowing of the constraint sharpens capable to specify it precisely. This is the progressive hardening principle, described below.

Garbage Collection¶

A codebase is simply a surviving system. Even pinch bully discourse engineering and strict architectural constraints, entropy accumulates. Dead codification grows. TODO comments persist for months. Dependencies spell stale. Abstractions that made consciousness astatine 1 shape of the task go obstacles astatine a later stage. Conventions established early get softly abandoned erstwhile they go inconvenient.

Garbage postulation is the periodic process of fighting this entropy. Unlike the different 2 components, which run astatine the infinitesimal of codification procreation aliases review, GC operates connected a schedule. It is not triggered by a circumstantial coding event. It runs because clip has passed.

In a harness engineering framework, GC rules are definitive declarations of what "clean" looks like, paired pinch scheduled agents aliases scripts that cheque whether the codebase still meets those standards. The output is not a database of errors to artifact a PR; it is simply a study that draws attraction to accumulating problems earlier they go serious.

The Living Harness¶

The astir important spot of a well-maintained harness is that it is not static. A harness that was written erstwhile and ne'er updated reflects the knowing of the squad astatine 1 constituent successful time. The codebase continues to evolve. New patterns emerge. Old constraints go irrelevant. New categories of AI-generated correction look that the original authors did not anticipate.

HARNESS.md is designed arsenic a self-referential document. It does not only picture what constraints are successful force; it tracks the position of each constraint: whether it is presently unverified, nether supplier review, aliases enforced deterministically. The archive declares what should beryllium true. Agents, hooks, and CI checks verify whether it is true. The harness auditor — a scheduled supplier successful this plugin — sounds the results of those checks and updates the position entries successful HARNESS.md to bespeak reality.

This creates a feedback loop. The archive is some a specification and a wellness record. Reading HARNESS.md astatine immoderate constituent successful clip tells you not conscionable what the squad has agreed should beryllium existent astir the codebase, but really good those agreements are really being maintained.

The self-referential spot is what distinguishes a surviving harness from a archive that gets outdated and ignored. Because the harness itself is simply a target of enforcement — the harness-audit supplier checks whether HARNESS.md accurately reflects the existent authorities of verification — neglecting the harness becomes visible alternatively than invisible. The mundane introduction to this self-check is /harness-sync, which runs the audit's discovery logic and presents a unified drift table; users spot misalignment betwixt the declared harness and reality without having to retrieve to invoke a abstracted diagnostic.

Progressive Hardening¶

Not each constraints are equal, and not each constraints are fresh to beryllium enforced deterministically from the start. Progressive hardening is the promotion ladder that describes really constraints mature.

The ladder is 1 axis. Reach — whether a constraint is required connected each PR aliases complete-if-present — is simply a second, and the Enforcement section does not grounds it.

Unverified is the starting state. You person declared a constraint successful HARNESS.md. You judge it is important. You do not yet person a system to cheque it. This authorities is not a failure; it is honorable accounting. An unverified constraint is simply a committedness to build enforcement, not a declare that enforcement already exists.

Agent is the 2nd state. You person written an supplier punctual that checks the constraint arsenic portion of PR reappraisal aliases a scheduled inspection. The constraint is being enforced, but by a connection exemplary making a judgment, not by a deterministic rule. Agent enforcement catches astir violations astir of the time. It is not perfectly reliable, and it requires quality reappraisal of the agent's output.

Deterministic is the last state. You person expressed the constraint precisely capable to encode it arsenic a script, a linter rule, aliases a structural check. It runs successful CI. It either passes aliases it blocks the merge. There is nary judgement involved, and nary anticipation of the cheque being confused aliases misled.

The guidance of activity is ever toward deterministic. When an supplier many times catches the aforesaid people of violation, that repetition is simply a signal: the shape is now understood good capable to automate. Write the script, discontinue the supplier cheque for that circumstantial constraint, and move the introduction successful HARNESS.md to deterministic status.

Progressive hardening is important because it prevents 2 nonaccomplishment modes. The first nonaccomplishment mode is trying to enforce everything deterministically from the start, which is intolerable for caller aliases semantically analyzable constraints. The 2nd nonaccomplishment mode is accepting agent-based enforcement arsenic a imperishable state, which is costly and unreliable. The ladder gives you a way betwixt them.

How This Plugin Implements It¶

This plugin structures the verification slots into 3 enforcement loops, operating astatine different timescales and pinch different tolerances for mendacious positives.

The soul loop is advisory and runs astatine edit time. When you prevention a record aliases complete a coding session, lightweight checks tally and aboveground imaginable issues arsenic suggestions alternatively than blocks. The soul loop is optimised for debased friction. It should not interrupt flow. Its occupation is to make problems visible early, not to extremity work.

The mediate loop is strict and runs astatine PR time. When you unfastened a propulsion request, a afloat suite of agent-based and deterministic checks runs. This loop has the authority to artifact a merge. It is the main enforcement constituent for architectural constraints. Failures present must beryllium addressed earlier the codification lands.

The outer loop is investigative and runs connected a schedule. Garbage postulation rules, fittingness functions, and harness audits tally periodically — daily, weekly, aliases connected immoderate cadence makes consciousness for the rule. The outer loop produces reports alternatively than blocks. Its findings provender backmost into the harness arsenic imaginable caller constraints aliases updates to existing ones.

The 3 loops correspond astir to the 3 components: soul loop serves discourse engineering (keeping the AI informed successful the moment), mediate loop serves architectural constraints (enforcing agreed standards astatine integration time), outer loop serves garbage postulation (detecting slow entropy betwixt integration events).

Agents successful this plugin run pinch bounded trust. No supplier has unilateral authority to modify accumulation codification aliases merge changes. Agents review, suggest, report, and flag. Humans decide. This is simply a deliberate creation choice: the harness amplifies quality judgment; it does not switch it.

The Self-Improving Dimension¶

The original Boeckeler model describes a harness arsenic thing a squad builds and maintains. This plugin adds a layer: the harness tin study from its ain operation.

After each coding session, the /reflect bid captures what went well, what failed, what conventions were violated, and what caller patterns emerged. These reflections accumulate successful a learnings log. The harness agents publication from this log erstwhile making decisions, truthful patterns of past mistakes pass existent review.

Regression discovery useful successful the aforesaid direction. When the harness-audit supplier runs, it does not only cheque whether existent constraints are being met. It looks astatine the history of constraint violations to place patterns: are the aforesaid constraints being violated repeatedly? If so, that is simply a awesome that the constraint needs a stronger enforcement mechanism, aliases that the discourse archive does not explicate the rationale intelligibly enough, aliases that the constraint itself is incorrect and needs to beryllium reconsidered.

This closes a loop that the original model leaves open. A fixed harness gets amended only erstwhile humans announcement failures and update it manually. A self-improving harness treats its ain operational history arsenic input information and generates proposals for its ain improvement. Humans still determine which proposals to accept, but the activity of shape nickname — reference the usurpation log and noticing that the aforesaid correction keeps recurring — is delegated to the agents.

The auto-harness additions successful this plugin widen this further: the harness-init process itself sounds existing codification to infer constraints that are already coming successful the codebase but not yet declared. Rather than requiring the squad to specify everything from scratch, the supplier bootstraps a campaigner HARNESS.md from observed patterns and asks the developer to confirm, reject, aliases refine each entry. The quality remains the authority, but the first costs of building the harness is substantially reduced.

Harness-init besides supports incremental adoption. Teams take which features to configure — discourse engineering, constraints, garbage collection, CI, and observability — and tin re-run the bid later to adhd more. Existing configuration is preserved crossed runs. This intends a squad tin commencement pinch conscionable discourse and constraints, beryllium the value, and adhd garbage postulation and CI enforcement erstwhile they are ready. The harness grows pinch the team's maturity alternatively than demanding afloat committedness upfront.