Show HN: LLM Attention Visualization

One absorbing point astir transformer-based ample connection models are that, during the procreation phase, it is capable to tie accusation from immoderate of its erstwhile tokens. But it needs to beryllium selective; if each token affects the procreation equally, it won't beryllium very effective. This process needs a system to determine really overmuch a token affects the adjacent token.

Turns out, we tin visualize this mechanism!

You tin pat aliases hover complete immoderate of the generated tokens to spot the past tokens that affected* the generation.

* "Affected" mightiness not beryllium afloat accurate, arsenic this visualization is highly simplified. It's calculating the attraction weight, scaled by the magnitude of the worth vector, aggregated crossed each attraction heads, and summed crossed each layers. This is past utilized to power the opacity of the erstwhile tokens. The largest values ever person an opacity of 1 and the remainder are interpolated.

A batch of accusation had to beryllium thrown distant to limit the visualization to conscionable 1 numeric worth per past token. Because of that, erstwhile I started implementing this, I really thought it mightiness not beryllium comprehensible. But it really tin nutrient immoderate absorbing patterns!

For example, successful the default "Office Move Summary" prompt, you tin hover complete the matter that are copied verbatim for illustration the reside and dates. You tin past spot the original information guidelines retired rather a bit, because the generated token takes up a batch of the accusation from the root data.

This addresses 1 point that I've antecedently recovered unintuitive astir LLMs. If they activity by predicting the adjacent tokens probabilistically, why are they someway truthful bully astatine copy-pasting stuff? Won't they yet make a correction conscionable by random chance?

But pinch this mechanism, you tin spot that it doesn't foretell the full series from immoderate constricted soul states. Since it has entree to each past tokens, it tin conscionable determine which past tokens to tie from erstwhile copying, and truthful the probability of errors tin beryllium very low. In the "Debugging an Average Function" example, you tin spot that this rather mini exemplary (600 cardinal parameters) tin easy reproduce an full JS usability isolated from for the intended modification. (Although it's not really tin of uncovering the rumor by itself, truthful it needed immoderate hints.)

Another absorbing portion is erstwhile you hover complete the "remain" successful "Existing entree cards and telephone numbers remain" successful the "Office Move Summary" prompt. You tin spot that it draws from "work" successful "Existing worker entree cards will work" and "stay the same" successful "company telephone numbers will stay the same". So it's benignant of combining the accusation from the words successful some phrases, which I find rather cool.

Implementation

The visualization itself is simply a beautiful basal React app utilizing Transformers.js to make the text. But, since we request to propulsion much information retired of the exemplary to visualize it, it can't usage the regular procreation loop. I had to vibe-code the procreation loop successful the app truthful we tin really support way of the values to visualize.

Despite utilizing a smaller exemplary for this, it's still hundreds of megabytes, and waiting for it to download earlier showing thing conscionable won't work. So I pre-generated a bunch of prompts that tin beryllium loaded and viewed instantly.

Another tricky point is that immoderate of the things successful the visualization are not really meant to beryllium read, truthful they're not defined arsenic outputs. I suppose if you're implementing this utilizing Python ML libraries, it would still beryllium easy to entree them. But Transformers.js uses .onnx files that contains the full computation graph. The exemplary loading and computation logic is implemented successful wasm, truthful there's nary easy measurement to entree thing different than the predefined outputs, arsenic acold arsenic I tin tell.

In the end, I utilized a mini book to modify the onnx record conscionable capable to expose those soul values. But that intends I can't conscionable usage the regular .onnx model. Since I want to person a browser-based procreation feature, I person to upload a abstracted instrumented exemplary to my ain Hugging Face repo and constituent the app there.

You tin find the codification successful the GitHub repo.