A self hosted AI software factory

tl;dr: It worked! From 1 punctual it created a repo, wrote the exertion and tests, sewage CI green, provisioned Postgres and deployed the vanished app down HTTPS without different connection from me.If you conscionable wanna spot the outcome you tin find a demo video astatine the bottom

LLMs sewage nosy again! Maybe they ever were and I was conscionable stuck successful the trough of disillusionment. Lately, whenever I need a small tool, I conscionable build it.

I was successful the gym the different time and wanted a weights tracker. The app I had successful mind was astir arsenic CRUD-y arsenic it gets, but all the app shop versions wanted £12 per month, truthful I conscionable one-shotted 1 pinch Claude. Great fun, but giving an LLM root entree to my instrumentality successful car mode still doesn’t beryllium correct pinch me.

So, the challenge: really tin I create a afloat distant agentic improvement situation wherever we structurally incorporate the LLM rather than conscionable trusting it? I want to springiness it an instruction and person it autonomously move done the whole SDLC:

• Researching the correct stack and packages to use.
• Planning and penning the codification and tests.
• Committing to Git, building and moving a CI pipeline.
• Deploying the activity to a ‘production’ server pinch databases, o11y, and a domain pinch SSL.

All connected my location server, without different unreality infrastructure bill. The only ongoing costs circumstantial to this research is a £20 Codex sub.

The Server(s)

Here they are successful each their glory.

The 1 astatine the bottommost is simply a 2014 dual-core i3 I’ve been moving arsenic a homelab for 5 years. It’s valiantly hosting this blog and astir 45 different Docker containers, from Pi-hole to a afloat Prometheus / Loki / Grafana stack. It besides has port 443 forwarded from my router. I’d beryllium miffed if an LLM collapsed it, truthful that’s not what we’re utilizing today.

The apical 1 is simply a 2021 10th-gen i7 pinch 32GB RAM, bought caller from eBay pinch thing connected it. Perfect.

The Stack

The halfway improvement stack is self-hosted done Coolify. Inference and integrations for illustration Tailscale, Telegram, DNS and ACME still time off the box. You could big conclusion too, but I don’t person the hardware and I’d alternatively OpenAI subsidise my experiments.

Sources

This isn’t a afloat how-to guide. I could astir apt constitute an Ansible one-shot book to group it each up; time off an rumor on the GitHub repo beneath if you’d for illustration one. If you’ve publication this far, though, you tin astir apt fig it out.

Networking

The first guardrail is obvious: it’s connected its ain metal. Hermes could rm -rf / and astatine worst it would costs maine a couple of hours rebuilding it.

The adjacent furniture of bombproofing is the network. My older server has larboard 443 forwarded from the router; this 1 doesn’t. There’s nary outer ingress, cutting retired a immense onslaught aboveground and each the net inheritance radiation from people speculatively probing /wp-admin connected each DNS A grounds I group up.

But, if there’s nary ingress, really do I:

• Get entree to each our cool caller apps connected my phone?
• Generate an SSL cert astatine a vanity URL truthful I tin entree https://cool-new-app.internal.jakeshomelab.me?

I person Tailscale group up pinch my older server arsenic an exit node. When I’m distant from home, selecting it routes my traffic through that server and Pi-hole, which I’m utilizing for civilization DNS. Pi-hole lets you adhd dnsmasq rules for illustration this:

Anything requesting *.internal.jakeshomelab.me now resolves to my caller server, wherever Coolify’s reverse proxy picks it up and serves my shiny caller services.

SSL Certs

With Caddy aliases Traefik and Docker labels, you tin service larboard 3000 connected instrumentality X from https://my-service.internal.jakeshomelab.me. Point an A grounds astatine the server and it’ll interaction Let’s Encrypt, complete an ACME situation and get an SSL cert. I learned this 3 years agone and it still seems for illustration magic.

The problem is the A record. I don’t want to publically subordinate my-service.internal.jakeshomelab.me pinch my IP, whether group tin entree it aliases not. I want an SSL cert for a shade service.

To lick this problem, I turned to DNS-01. I’ll beryllium honorable this is caller to me, but here’s really it works:

• Buy a domain (in this lawsuit from Porkbun).
• Generate Porkbun API keys and adhd them to Coolify’s situation pinch constitute entree to the domain.
• Modify Coolify’s Docker Compose record to usage lego and the Porkbun API:

Then, erstwhile I registry a caller URL, Traefik / Coolify:

• Uses the Porkbun API to create a caller TXT grounds astatine _acme-challenge.my-service.internal.jakeshomelab.me.
• Let’s Encrypt validates the situation and issues a valid SSL cert.
• Traefik deletes it.

That’s it! You now person a valid HTTPS URL, reachable wrong your tailnet, pinch nary nationalist A aliases AAAA grounds pointing to the service. The hostname whitethorn still look successful nationalist certificate-transparency logs, but the work is only reachable from the tailnet.

The champion spot is that Coolify does this connected the fly. Our supplier tin create a work astatine immoderate subdomain and it’ll ✨magically✨ benignant itself out.

So, glue each this together and you get the following:

The aforesaid setup covers the tooling, truthful Coolify, Hermes, Forgejo and Firecrawl each unrecorded connected their ain section subdomains.

Development Stack & MCPs

Now we person an isolated(ish) box, let’s move connected to the tooling. The devices are good known; gluing them together is the fun part.

Forgejo

We request location durable to shop codification and tally CI. I decided not to usage GitHub because:

• Giving the container my GitHub token alternatively undermines the isolation. Also, it’s not self-hosted.
• Its API and CI infinitesimal limits won’t activity astatine the standard of our caller package factory.
• It’s down astir of the clip these days anyway.

Forgejo is simply a awesome self-hosted alternative. The Docker Compose record linked supra sets up Forgejo and its runners; registering yourself and the runner takes a small other work, but it’s good documented.

I’ve besides included a Compose record for syncing projects backmost to GitHub. That puts your GH token successful the environment, but the trade-off is yours to make.

The Forgejo Hermes accomplishment linked supra gives the supplier afloat power of the instance.

Hermes

Hermes is an OpenClaw-style individual adjunct pinch agentic capabilities. I ne'er sewage successful connected the OpenClaw hype, truthful I can’t comparison the two, but Hermes has a fewer features I’ve recovered handy:

• Web UI: A modular ChatGPT-esque interface for moving from my laptop and managing skills.
• Shared filesystem: I’ve mounted its workspace from the Docker big and shared it complete Samba. The supplier and I can use the aforesaid files alternatively of copy-pasting Markdown and codification around.
• Telegram integration: I tin chat to the supplier from my phone. Setup took 2 minutes and required nary login details, which suited the sandbox approach.
• Self-building skills: Hermes tin create and registry its ain skills. I couldn’t find a bully Coolify one, truthful it read the docs, looked astatine the MCP and built one.
• Firecrawl: Self-hosted Firecrawl gives the supplier overmuch nicer entree to SERP information and web scraping astatine scale.

Getting Hermes and Firecrawl group up pinch the correct keys successful the correct places is simply a monolithic symptom successful the arse. I’ve added Coolify-friendly Docker Compose files to the repo linked above.

Coolify

Coolify is the glue holding this together: a self-hosted PaaS built connected Docker and Compose that comes connected successful leaps and bounds pinch each update. If you want Heroku aliases DigitalOcean App Platform niceties connected your ain hardware, I’d highly recommend it.

Some of my favourite features are:

• It’s conscionable Docker nether the hood. Existing deployments mostly work, and if Coolify won’t do thing weird you can docker exec <whatever> from your laptop. Things are only abstracted distant if you want them to be.
• The SSL / routing stack which I’ve gone into successful extent above.
• Coolify ships pinch a bunch of pre-made recipes for each the astir communal apps. Postgres, Redis, Hermes, Forgejo and almost thing other is disposable to deploy pinch a azygous click.
• Postgres backups to S3 are a three-click job, and env vars and personification guidance are built in.
• GitHub webhooks springiness you automatic deploys connected push to main.

Here are a mates of screenshots of my Coolify setup successful action:

What it really did

The demo beneath shows this beautiful well, but the starting weapon was the pursuing prompt:

From there, it conscionable sewage connected pinch it:

• Created a caller Git repo and bootstrapped SvelteKit, Drizzle, Postgres and Tailwind.
• Wrote the app and its tests, committing the activity successful sensible stages.
• Created a CI pipeline.
• Worked done trial failures until CI turned green.
• Containerised the app and its ain Postgres lawsuit pinch Docker Compose.
• Deployed the batch to Coolify astatine its ain URL.

All without a azygous further prompt. No nudging it done grounded tests aliases copying correction messages backmost into the chat. It conscionable kept going until the app was running.

At that constituent I gave it a whirl and deed a CSRF rumor erstwhile submitting data. I sent 1 much prompt; it diagnosed the problem, fixed it, added regression tests and redeployed.

And it worked!

That’s the loop I wanted: prompt, repo, code, tests, CI, deployment, bug fix. It’s not a analyzable app, obviously, but it went from a paragraph to tested, deployed package and handled each the boring bits successful between. That still feels a spot for illustration witchcraft.

Enough of each that, show maine the goods!

I’m nary YouTuber, but present you go:

Thoughts connected isolation and adjacent steps

There is ever a trade-off betwixt afloat agentic improvement and security. This was a reasonably contrived example: the app useful wholly successful isolation. Most useful package talks to different software, which intends handing complete API keys, and each cardinal adds different small spread successful the sandbox.

Even successful this setup, Hermes tin still:

• Nuke the caller server and everything moving connected it.
• Delete repos, databases and deployments.
• Leak aliases maltreatment immoderate credentials I’ve fixed it.
• Burn done conclusion tokens for illustration its end-of-year reappraisal depends connected it.
• Make rando outbound requests and download immoderate rubbish the net hands it.
• Poke thing other connected my web that the firewall allows it to reach.

So no, it isn’t harmless. What I’ve done is make the instrumentality sacrificial and sharply limit really overmuch worldly I attraction about is wrong reach. The nonaccomplishment mode is now “rebuild the eBay container and rotate a fistful of keys”, alternatively than “discover an LLM has enthusiastically reorganised my existent laptop”. That’s amended I think, but it isn’t magic.

The evident adjacent steps are:

• Put the container connected its ain VLAN and explicitly artifact entree to the remainder of my location network.
• Scope each credential arsenic narrowly arsenic the supplier allows, and rotate them regularly.
• Automate backups and make rebuilding the full container a one-shot job.
  • Coolify’s DB backup and my shared Docker constitute mounts should make this comparatively easy.
• Require support earlier it does thing genuinely nationalist aliases difficult to undo.

At immoderate point, though, capable support gates move your magical autonomous package mill backmost into a postulation of forms you have to capable in. Finding the useful constituent betwixt “needs maine each 5 minutes” and “has the motorboat codes” is the next experiment.