Understanding is the new bottleneck

Hot take: I deliberation it's still important to understand the codification that our agents write!

In this talk I'll explicate why that's the case, and show immoderate ideas for really to efficiently understand code. Alright, let's dive in.

Agents are penning much and much codification for us, and we each cognize it's getting harder to support up.

But the bully news is: location are galore ways to understand code! Reading diffs statement by statement is not the only way.

Most of this talk will beryllium astir techniques I person recovered adjuvant to understand systems my agents are building:

• Code explainer docs
• Quizzes to cheque my understanding
• Micro-worlds that I tin play pinch to understand the system

But first we person to inquire a much basal question…

Why? Why understand?

Aren't we expected to beryllium taking ourselves retired of the loop now, and letting the agents loop themselves? As the agents get smarter, doesn't it go little important for america to beryllium successful the details?

I deliberation galore group — moreover those who are pro-understanding — person a somewhat incorrect reply to this question!

One imaginable answer: we understand to verify. We cheque the agent's work, we spot if it's correct.

Correct tin mean galore things: does it lucifer the spec, is it good architected… but it's fundamentally a thumbs-up / thumbs-down question.

Here's the thing: the agents are getting amended and amended astatine verifying their ain work. And this is good! I for illustration it erstwhile my supplier doesn't make mistakes.

But hmm. Where does that time off america humans?

That's wherever different reply comes in: we tin understand to participate.

You tin study what the supplier is doing to make judge you tin beryllium an progressive subordinate successful the imaginative process. Here's why this matters…

It's ne'er conscionable 1 loop! A task is many, galore loops pinch the agent.

And the knowing you person of the strategy is portion of your expertise to travel up pinch the adjacent thought to germinate it.

You request a rich | group of concepts successful your mind to deliberation creatively and fluently astir really to move thing forward. If you're lacking that fluency, your expertise to participate successful the task is meaningfully limited.

By the way, this relates intimately to the thought of cognitive debt, popularized by Margaret Storey and Simon Willison.

It's for illustration tech debt: you tin get distant pinch not knowing what's going connected successful the short term, but it'll wound you eventually.

OK, truthful fine, knowing matters.

But this raises the adjacent question: how? How do we build this quality knowing erstwhile we're moving pinch AI and moving fast?

Well, turns retired this is not the first clip anyone has ever thought astir really to pass understanding. I deliberation we tin look to acquisition arsenic an inspiration. Can we bargain the champion ideas ever invented for acquisition and use them to this problem?

Today I want to stock 3 techniques that show really we tin effort this.

First: explanations. What makes a bully explanation?

Whenever an supplier finishes immoderate work, it's an opportunity for an mentation — an artifact.

Most naively, we tin publication a codification diff: the earthy worldly that changed.

But what if we ask:

What would the best mentation be? If you had a squad — quality aliases AI — that really sweat the specifications of explaining thing good to you, really would that feel?

Here's 1 answer. I made a accomplishment called /explain-diff, which I usage each time and galore coworkers person recovered valuable.

It outputs thoughtfully system codification explainers arsenic HTML, markdown, aliases Notion docs. Notion is simply a bully spot for collaborating connected and discussing these explainers arsenic a team. (Disclaimer: I activity astatine Notion truthful I'm biased.)

Let's spot what's successful 1 of these explainers, utilizing an illustration of editing the position of a video game.

First principle: teach maine inheritance info!

Before we moreover get to what changed, thief maine understand what was already there. In this case, thatch maine astir the crippled engine.

Second principle: intuition earlier details.

Before immoderate code, it states the extremity — “make the plot consciousness three-dimensional pinch 2D drafting tricks” — and explains related concepts, for illustration what isometric projection is.

All of this builds my intuition for the principle of the change. It's catching maine up arsenic the quality truthful I tin beryllium an adjacent subordinate successful understanding.

You tin besides build intuition pinch interactive figures.

Here I'm knowing the isometric position by dragging rocks astir the plot and watching their coordinates move.

(This is utilizing a caller characteristic Notion conscionable shipped: you tin now embed interactive HTML wrong pages.)

We yet get to the code. But a emblematic diff is simply a heap of files edited successful alphabetical bid pinch nary explanation.

A “literate diff” arsenic I telephone it is system arsenic prose — stepping done the changes successful a sensible order, pinch surrounding mentation and embedded codification snippets. Faster to reappraisal than a earthy diff.

The extremity consequence of each of this is simply a bully explainer packet. I still publication the codification diff but I ever publication this first.

Sometimes I'll people these retired and return them to the café — little distracting.

It's beautifully ironic: AI turns an interactive activity into a fixed insubstantial study I tin attraction connected profoundly :)

I do thing akin pinch my codification explainers now. At the bottommost of an explainer there's an interactive quiz — 5 questions astir the alteration — and I effort to reply them.

My rule: I won't nonstop codification to others until I tin walk the quiz, and I do the aforesaid erstwhile reviewing others' code.

A quiz is simply a velocity regulator. Working pinch AI, it's easy for the loop to tally faster than the velocity of quality understanding.

The quiz is simply a counterbalancing force: I mechanically inquire “do I really understand?” truthful that I tin stay a afloat imaginative participant.

Papert had this beautiful thought he called living successful Mathland: if you want to study math, unrecorded successful Mathland — conscionable for illustration if you want to study French, you spell unrecorded successful France. Could we build an situation wherever children study mathematics naturally, arsenic a consequence of their curiosity?

So really do we use that to code? Can we make worlds you inhabit and people intuit really the strategy useful and really it's changing?

Last twelvemonth I was coding a Prolog expert and struggling to intuit what was happening inside.

I worked pinch an supplier to build this debugger, which fto maine measurement done the execution of my logic connection — scrub done time, spot what's connected the stack and which rules are evaluated astatine each step. I could moreover time off comments for myself (“nice, we correctly applied that rule”).

There's a large quality betwixt making a instrumentality for me to debug and letting the supplier debug — doing it myself is really I create knowing on the way.

Another example. I was migrating my individual website from 1 model to another, and Claude wrote a book that did it. But it was very difficult to review: I wasn't acquainted pinch the caller framework, and each I could opportunity was “I conjecture that looks astir right.”

So I asked Claude to make maine a video crippled — a bid halfway wherever I do the larboard myself, measurement by step, watching the visible effects and the record character evolve. It produced a UI wherever I click buttons to tally the larboard measurement by step, pinch my aged tract and caller tract moving broadside by side.

In this bid halfway I watched the caller tract travel to life incrementally. That near maine pinch a akin knowing to doing it by manus — but overmuch faster, because the full acquisition was laid retired for me.

The constituent present is that agents tin constitute bits of codification that thief america humans understand different code.

This is simply a large deal!

When you and personification other clasp the aforesaid intelligence model, you tin pass efficiently. You person a shared vocabulary that evokes the aforesaid images, truthful you tin jam and riff and person imaginative conversations. Without those shared structures, those conversations are overmuch harder.

I'm really excited astir creating shared environments wherever teams build that knowing together. It's kinda what Notion is each astir too.

Recently successful Notion we've been shipping tons of caller features for humans and agents to activity together, truthful your full squad develops a shared knowing alternatively of each moving successful a silo.

One mini example: you tin now tally Claude and Cursor agents successful Notion. I do a batch of my coding that measurement now.

And erstwhile those agents make a method scheme successful Notion, it's successful a collaborative page by default, truthful I tin remark connected it pinch my squad and talk immediately. Thinking together, not alone!

Alright, let's wrap up. Today we've covered immoderate techniques that were astir knowing code… but really I deliberation this is simply a overmuch bigger issue.

It's still important for humans to understand really things activity in general! Not conscionable to verify, but to participate.

And astonishment surprise, this is not a caller idea. It harkens backmost to the very origins of our section of computing…

50 years agone Alan Kay envisioned that computers could beryllium a caller medium, amended than the book, for school group — particularly kids — really to deliberation astir the world.

In this picture, it mightiness look for illustration these kids are watching YouTube connected an iPad, but they're not. They're playing an interactive crippled and editing the codification arsenic they play it to get a amended knowing of physics. This was 50 years ago!!

And now hopefully you understand this meme.

The constituent was ever to augment, not conscionable automate.

It's beautiful that AI now makes creating simulations truthful accessible… Having AI thatch america is 1 of the top possibilities computing has ever opened up.

This makes maine very optimistic astir the future!

If we build the correct tools, we tin now understand the world amended than we ever could before. We don't person to simply return ourselves retired of the loop, we tin get deeper successful the loop too. It's up to us.

FIN