Show HN: Agents infer Lego mechanisms from sources, then animate them

Give an LDraw LEGO© example to an AI agent, get rear a Blender-animated type of it.

The agent will:

• Reverse-engineer (infer) the LDraw model's mechanics from its .mpd origin code file 🤓
• Study the discovered mechanisms.
• Plan the interactions between their distinct mechanical components.
• And finally, rotate those plans into python code for Blender, that will:
  • Create the required rigs for controlling the distinct mechanisms.
  • Animate them according to plan! 🤯

You'll need Python 3.12+ installed.

Start by cloning the repo and running the setup script, choice the correct one for your OS:

git copy https://github.com/anteloc/ldraw-mecha.git cd ldraw-mecha ./setup-macos.sh # or ./setup-linux.sh, ./setup-win-wsl.sh

At the end of the setup process, you'll get a snippet alongside the env vars to be added to your casing profile (.bashrc, .zshrc, etc.), item akin e.g.:

# >>> ldraw-mecha (from setup-macos.sh) >>> export LDRAW_DIR="/Users/me/ldraw-mecha/data/ldraw" export LDRAWDIR="$LDRAW_DIR" export BLENDER_BIN="/Users/me/ldraw-mecha/bin/Blender.app/Contents/MacOS/Blender" export LEOCAD="/Users/me/ldraw-mecha/bin/LeoCAD.app/Contents/MacOS/LeoCAD" export LDRAW_XRAY_CACHE="/Users/me/ldraw-mecha/.cache/ldraw-xray" export LDRAW_ANNOTATOR_DB="/Users/me/ldraw-mecha/data/ldraw-info.db" # export TYPESAFE_API_KEY=... # optional: Jev semantic hunt (jev-rerank) # another env vars... # <<< ldraw-mecha <<<

If you're on VS Code or a terminal, restart it in command to get an updated env alongside the newly added env vars.

For inspecting and selecting one pre-annotated example to animate, open the annotated models selector by running:

python3 model-selector/serve.py

Then, obtain a appearance at the Process guide for a step-by-step use example.

These are some samples of animations by distinct agents, obtain a appearance at their Summaries for additional details.

• Agent: Claude Sonnet 5.5
• Original model: 42043-1_Flatbed-Trailer.mpd
• Context: The example was too ample and complex for completely animating it, eventually I accepted Sonnet's recommendation concerning animating the crane alone.

▶ Video · Summary

Original

• Agent: Claude Sonnet 5.5
• Original model: 3625-1.mpd
• Context: Asked for an in-place animation, to evade the aircraft moving around.

▶ Video · Summary

• Agent: Claude Opus 5.5
• Original model: None
• Context: This one is really a generated composition, from multiple different minifigs, to be used as a "howto reference" for another agents additionally animating minifigs.

▶ Video · Summary

After publishing LDraw Nova, a brother or sister project for agentic generation of LDraw models, multiple group reached out to me asking concerning if models could be animated somehow.

During the betterment of Nova, I noticed that adding several special annotations to .mpd original example source files, and afterward giving those to agents, would:

• Help agents in understanding how to build another distinct models.
• But also, in understanding how a example was built.

Which gave me the idea that, maybe, an delegate would also comprehend how a example would work.

As a archetypal stage to it, I gave annotated models to distinct agents, and prompted them to infer (reverse-engineer) their constituent submodels (see Haiku 5.5. experiment below).

Results from inference experiments were the two beautiful good and consistent throughout distinct agents, and I additionally noticed the following: many of the submodels inferred were really mechanisms.

After questioning multiple agents concerning those inferred mechanisms, concerning how would they actually work, I additionally got accordant results: agents would describe behaviours according to the descriptions for the parts on a mechanism! 🤯

At this point, I had the following:

• Agents would split and refine models into submodels and mechanisms.
• Agents would understand all scheme in isolation.
• Agents would understand how individual mechanisms worked.

The result of an genuine divide-and-conquer process, right? 🤓

Which would create handling idiosyncratic mechanisms manageable. 🤩

Now, there was the matter of giving agents the means to really animate mechanisms they already knew how they worked.

I was playing alongside Blender MCP at the time, and Blender looked promising as a tool that would provision agents a way to manipulate LDraw models in 3D, imported into Blender as glTF (.glb) files.

But there was motionless an open issue:

How to map from a refined example origin code an delegate would understand, to its equal 3D example in glTF format an delegate could manipulate?

And the solution I established was:

Annotate the 3D glTF model's nodes alongside the identical annotations as the origin code, so the delegate would:

• Outside Blender: scheme on submodels 🤔 and parts as source code.
• Inside Blender: execute on matching nodes ✨ on the 3D glTF model.

Now, I had a draft for a process, but no tools for the agent to use and transport it forward.

Given that the procedure would necessitate a high amount of complex geometric and spatial calculations, I asked the delegate to derive a new python toolset from the one created for ldraw-nova

The new python toolset I refined iteratively, by giving the delegate simple models at the beginning, and increasingly complex ones all time.

And to test it, I gave an delegate instructions to:

• first, plan from an annotated and refined .mpd model
• then, generate a glTF 3D type of it, alongside matching annotations
• then, upload it to Blender
• and then, produce the Blender python scripts that would animate the example as planned.

And it eventually worked! 🚀🚀🚀

(but not perfectly) 😜

Prompt:

This is a fragment, a submodel, from an LDraw .mpd document model.

This is not an executable code fragment, but additional akin a 3D CAD example part written in LDraw language.
Infer what this submodel is, and additionally divided it into submodels (starting with: 0 FILE):

• always alongside relevant, semantical meaning,
• the archetypal 0-line following all 0 FILE row is the "description line" for the submodel, according to the LDaw standard
• add the clarification for all submodel there, a one-liner, as a individual would depict it during looking at it
• at the answer, archetypal location the genitor model, afterward the kid one(s) and afterward the grandchildren, and afterward descendants... etc.: go from containers or parents to containees or children, children of children. etc.

0 FILE file-01 0 file-01 0 Name: file-01.ldr 0 Author: Unknown 0 !LDRAW_ORG Model 0 !LICENSE Redistributable under CCAL type 2.0 : see CAreadme.txt 0 // Plate 2 x 12 (dark_bluish_grey) 1 72 0 0 0 1 0 0 0 1 0 0 0 1 2445.dat 0 // Tile 1 x 2 Grille alongside Groove (metallic_silver) 1 80 -110 -8 0 0 0 1 0 1 0 -1 0 0 2412b.dat 0 STEP 0 // Slope Brick Curved 2 x 2 x 0.667 (metallic_silver) 1 80 -80 8 20 -1 0 0 0 1 0 0 0 -1 15068.dat 0 // Slope Brick Curved 2 x 2 x 0.667 (metallic_silver) 1 80 -40 8 20 -1 0 0 0 1 0 0 0 -1 15068.dat 0 // Slope Brick Curved 2 x 2 x 0.667 (metallic_silver) 1 80 0 8 20 -1 0 0 0 1 0 0 0 -1 15068.dat 0 // Slope Brick Curved 2 x 2 x 0.667 (metallic_silver) 1 80 40 8 20 -1 0 0 0 1 0 0 0 -1 15068.dat 0 // Slope Brick Curved 2 x 2 x 0.667 (metallic_silver) 1 80 100 8 20 -1 0 0 0 1 0 0 0 -1 15068.dat 0 // Slope Brick Curved 2 x 2 x 0.667 (metallic_silver) 1 80 100 8 -20 1 0 0 0 1 0 0 0 1 15068.dat 0 // Slope Brick Curved 2 x 2 x 0.667 (metallic_silver) 1 80 40 8 -20 1 0 0 0 1 0 0 0 1 15068.dat 0 // Slope Brick Curved 2 x 2 x 0.667 (metallic_silver) 1 80 0 8 -20 1 0 0 0 1 0 0 0 1 15068.dat 0 // Slope Brick Curved 2 x 2 x 0.667 (metallic_silver) 1 80 -40 8 -20 1 0 0 0 1 0 0 0 1 15068.dat 0 // Slope Brick Curved 2 x 2 x 0.667 (metallic_silver) 1 80 -80 8 -20 1 0 0 0 1 0 0 0 1 15068.dat 0 // Slope Brick Curved 4 x 1 Double alongside Red "NASA", Black "ESA", and Silver Background Pattern (light_bluish_grey) 1 71 70 8 0 -1 0 0 0 1 0 0 0 -1 10283 - 93273p01.dat 0 STEP 0 // Tile 2 x 2 Inverted (light_bluish_grey) 1 71 100 8 -20 1 0 0 0 1 0 0 0 1 11203.dat 0 // Tile 2 x 2 Inverted (light_bluish_grey) 1 71 100 8 20 1 0 0 0 1 0 0 0 1 11203.dat 

Answer:

This is a rounded argent hull section, likely a spacecraft part specified as a fairing, payload-bay shell, or component skin. It sits on a 2x12 dark-grey basis plate, alongside a grille tile at the -X end and two light-grey inverted tiles at the +X end. The ridge incline carries red "NASA" and dreary "ESA" printing, so it apt comes from a space-agency-themed LEGO model, although the parts solitary don't acknowledge the exact set.

The divided below follows the bodily groups, which additionally equivalent the three build steps in the original. The genitor comes first, afterward its children (hull-shell.ldr and end-cap-tiles.ldr), afterward the grandchildren (the two incline rows). Coordinates are unchanged, so all submodel keeps its complete positions and the parents citation children alongside an character change at the origin.

0 FILE curved-hull-section.ldr 0 Rounded argent hull division on a lengthy dark-grey basis plate, alongside a grille at the -X end and light-grey tiles at the +X end 0 Name: curved-hull-section.ldr 0 Author: Unknown 0 !LDRAW_ORG Model 0 !LICENSE Redistributable under CCAL type 2.0 : see CAreadme.txt 0 // Plate 2 x 12 (dark_bluish_grey) 1 72 0 0 0 1 0 0 0 1 0 0 0 1 2445.dat 0 // Tile 1 x 2 Grille alongside Groove (metallic_silver) 1 80 -110 -8 0 0 0 1 0 1 0 -1 0 0 2412b.dat 0 STEP 1 16 0 0 0 1 0 0 0 1 0 0 0 1 hull-shell.ldr 0 STEP 1 16 0 0 0 1 0 0 0 1 0 0 0 1 end-cap-tiles.ldr 0 FILE hull-shell.ldr 0 Curved argent casing made of two rows of slopes joined by a NASA/ESA-marked ridge piece 1 16 0 0 0 1 0 0 0 1 0 0 0 1 shell-row-positive-z.ldr 1 16 0 0 0 1 0 0 0 1 0 0 0 1 shell-row-negative-z.ldr 0 // Slope Brick Curved 4 x 1 Double alongside Red "NASA", Black "ESA", and Silver Background Pattern (light_bluish_grey) 1 71 70 8 0 -1 0 0 0 1 0 0 0 -1 10283 - 93273p01.dat 0 FILE end-cap-tiles.ldr 0 Two light-grey inverted 2x2 tiles decision off the +X end of the hull shell 0 // Tile 2 x 2 Inverted (light_bluish_grey) 1 71 100 8 -20 1 0 0 0 1 0 0 0 1 11203.dat 0 // Tile 2 x 2 Inverted (light_bluish_grey) 1 71 100 8 20 1 0 0 0 1 0 0 0 1 11203.dat 0 FILE shell-row-positive-z.ldr 0 Row of five argent curved slopes forming one flank of the shell, on the +Z side 0 // Slope Brick Curved 2 x 2 x 0.667 (metallic_silver) 1 80 -80 8 20 -1 0 0 0 1 0 0 0 -1 15068.dat 0 // Slope Brick Curved 2 x 2 x 0.667 (metallic_silver) 1 80 -40 8 20 -1 0 0 0 1 0 0 0 -1 15068.dat 0 // Slope Brick Curved 2 x 2 x 0.667 (metallic_silver) 1 80 0 8 20 -1 0 0 0 1 0 0 0 -1 15068.dat 0 // Slope Brick Curved 2 x 2 x 0.667 (metallic_silver) 1 80 40 8 20 -1 0 0 0 1 0 0 0 -1 15068.dat 0 // Slope Brick Curved 2 x 2 x 0.667 (metallic_silver) 1 80 100 8 20 -1 0 0 0 1 0 0 0 -1 15068.dat 0 FILE shell-row-negative-z.ldr 0 Row of five argent curved slopes forming the another flank of the shell, on the -Z side 0 // Slope Brick Curved 2 x 2 x 0.667 (metallic_silver) 1 80 -80 8 -20 1 0 0 0 1 0 0 0 1 15068.dat 0 // Slope Brick Curved 2 x 2 x 0.667 (metallic_silver) 1 80 -40 8 -20 1 0 0 0 1 0 0 0 1 15068.dat 0 // Slope Brick Curved 2 x 2 x 0.667 (metallic_silver) 1 80 0 8 -20 1 0 0 0 1 0 0 0 1 15068.dat 0 // Slope Brick Curved 2 x 2 x 0.667 (metallic_silver) 1 80 40 8 -20 1 0 0 0 1 0 0 0 1 15068.dat 0 // Slope Brick Curved 2 x 2 x 0.667 (metallic_silver) 1 80 100 8 -20 1 0 0 0 1 0 0 0 1 15068.dat 

I replaced the first clarification file-01, which was lone a document name, alongside genuine one-liners. I additionally renamed the base from file-01 to curved-hull-section.ldr, so if the external MPD references file-01, that citation needs updating. All 15 first part lines are preserved, and I can preserve this as a .mpd document if you desire it as a download. This is the identical part I divided a instant ago, so current is the identical answer again. It is a rounded argent hull section, likely a spacecraft piece, on a 2x12 dark-grey basis plate, alongside a grille at the -X end and two light-grey inverted tiles at the +X end. The ridge incline has red "NASA" and dreary "ESA" printing, so it likely comes from a space-themed LEGO model, although the parts don't acknowledge the exact set.

The genitor comes first, afterward its children (hull-shell.ldr and end-cap-tiles.ldr), afterward the grandchildren (the two incline rows). Coordinates are unchanged, and the parents citation their children alongside an character change at the origin.

0 FILE curved-hull-section.ldr 0 Rounded argent hull division on a lengthy dark-grey basis plate, alongside a grille at the -X end and light-grey tiles at the +X end 0 Name: curved-hull-section.ldr 0 Author: Unknown 0 !LDRAW_ORG Model 0 !LICENSE Redistributable under CCAL type 2.0 : see CAreadme.txt 0 // Plate 2 x 12 (dark_bluish_grey) 1 72 0 0 0 1 0 0 0 1 0 0 0 1 2445.dat 0 // Tile 1 x 2 Grille alongside Groove (metallic_silver) 1 80 -110 -8 0 0 0 1 0 1 0 -1 0 0 2412b.dat 0 STEP 1 16 0 0 0 1 0 0 0 1 0 0 0 1 hull-shell.ldr 0 STEP 1 16 0 0 0 1 0 0 0 1 0 0 0 1 end-cap-tiles.ldr 0 FILE hull-shell.ldr 0 Curved argent casing made of two rows of slopes joined by a NASA/ESA-marked ridge piece 1 16 0 0 0 1 0 0 0 1 0 0 0 1 shell-row-positive-z.ldr 1 16 0 0 0 1 0 0 0 1 0 0 0 1 shell-row-negative-z.ldr 0 // Slope Brick Curved 4 x 1 Double alongside Red "NASA", Black "ESA", and Silver Background Pattern (light_bluish_grey) 1 71 70 8 0 -1 0 0 0 1 0 0 0 -1 10283 - 93273p01.dat 0 FILE end-cap-tiles.ldr 0 Two light-grey inverted 2x2 tiles decision off the +X end of the hull shell 0 // Tile 2 x 2 Inverted (light_bluish_grey) 1 71 100 8 -20 1 0 0 0 1 0 0 0 1 11203.dat 0 // Tile 2 x 2 Inverted (light_bluish_grey) 1 71 100 8 20 1 0 0 0 1 0 0 0 1 11203.dat 0 FILE shell-row-positive-z.ldr 0 Row of five argent curved slopes forming one flank of the shell, on the +Z side 0 // Slope Brick Curved 2 x 2 x 0.667 (metallic_silver) 1 80 -80 8 20 -1 0 0 0 1 0 0 0 -1 15068.dat 0 // Slope Brick Curved 2 x 2 x 0.667 (metallic_silver) 1 80 -40 8 20 -1 0 0 0 1 0 0 0 -1 15068.dat 0 // Slope Brick Curved 2 x 2 x 0.667 (metallic_silver) 1 80 0 8 20 -1 0 0 0 1 0 0 0 -1 15068.dat 0 // Slope Brick Curved 2 x 2 x 0.667 (metallic_silver) 1 80 40 8 20 -1 0 0 0 1 0 0 0 -1 15068.dat 0 // Slope Brick Curved 2 x 2 x 0.667 (metallic_silver) 1 80 100 8 20 -1 0 0 0 1 0 0 0 -1 15068.dat 0 FILE shell-row-negative-z.ldr 0 Row of five argent curved slopes forming the another flank of the shell, on the -Z side 0 // Slope Brick Curved 2 x 2 x 0.667 (metallic_silver) 1 80 -80 8 -20 1 0 0 0 1 0 0 0 1 15068.dat 0 // Slope Brick Curved 2 x 2 x 0.667 (metallic_silver) 1 80 -40 8 -20 1 0 0 0 1 0 0 0 1 15068.dat 0 // Slope Brick Curved 2 x 2 x 0.667 (metallic_silver) 1 80 0 8 -20 1 0 0 0 1 0 0 0 1 15068.dat 0 // Slope Brick Curved 2 x 2 x 0.667 (metallic_silver) 1 80 40 8 -20 1 0 0 0 1 0 0 0 1 15068.dat 0 // Slope Brick Curved 2 x 2 x 0.667 (metallic_silver) 1 80 100 8 -20 1 0 0 0 1 0 0 0 1 15068.dat 

I replaced the first file-01 description, which was fair a document name, alongside genuine one-liners, and renamed the base to curved-hull-section.ldr. If the external MPD references file-01, that citation volition need updating. All 15 first part lines are preserved, and I can preserve this as a .mpd document if you desire a download.

This is a very preliminar release that, however, yields many good results, but far from being optimal:

• Complex mechanisms and large models obtain too much period and attempt to animate.
• Also for those, agents would go through "rabbit holes" and get misplaced during animation sessions.
• Broken animations for flexible parts on models akin e.g. hoses or cables.
• Low-end models, akin e.g. Claude Haiku 5.5 or GPT-6 Luna, can create animations, but most of them have noticeable defects.
• No appropriate physics, it doesn't obtain into document e.g. gravity or any forces interactions, weights, inertia, etc.
• And more... 🤫

I'd akin to appreciate the following:

• The LDraw Community
• LDView's Travis Cobbs (@tcobbs), and contributors.
• LeoCAD's Leonardo Zide (@leozide), and contributors.
• LDCad and Shadow Library, Roland Melkert.
• pyldraw3's Harold Martin (@hbmartin), and contributors.

... and gratitude to all of the many another LDraw creators!

NOTE: For this work, I've used many LDraw models, libraries, tools, docs... from many sources.

There is a lot amazing people that generously contributed to this, equal for decades, by generously donating their finest work to the community domain and open origin community.

If you think you have to be included on this section, delight drop me an email!

LEGO(R) is a trademark of the LEGO Group of companies which does not sponsor, authorize or endorse this software.