Show HN: Janus – Go binary that runs GGUF models via Vulkan on AMD/Intel/Nvidia

Janus is a single Go binary that runs .gguf models on your device (GPU or CPU) and exposes an OpenAI-compatible API affirmative a built-in web UI. No Python, no Docker, no Ollama required — although Ollama is supported as a backend if you prefer.

Use it your way: call it from the command line (curl, PowerShell, scripts), cable it into Cursor / Cline / any OpenAI client, or use the Web UI — identical local models, any workflow fits you. Janus is the runner and router; you choose the forefront end.

Design idea: the example decides what to do; Go runs inference, routes requests, executes tools, and keeps everything local.

• Local inference — llama.cpp via Vulkan (AMD / Intel / NVIDIA) or CPU fallback
• OpenAI-compatible API — /v1/chat/completions, /v1/models, tool listing/calling
• Web UI — Assistant, Chat, Kernel (tool loop), Config, Memory, Skills
• Built-in tools — read/write files, run commands, math, docx in/out, PDF output, OCR (with Tesseract), and more
• Hot-swap models — alter .gguf in the UI without restarting
• Optional auth — Basic Auth for admin endpoints whenever JANUS_AUTH=true

Disk: scheme for the example size (often 2–8 GB per model) affirmative ~50 MB for Janus + llama.dll.

Optional: Tesseract OCR if you desire scanned-document OCR tools.

git copy https://github.com/Vibra-Ingenn/Janus.git cd janus .\build.ps1

build.ps1 downloads pre-built llama.cpp Vulkan DLLs and compiles dist\janus.exe.

If you already have DLLs in lib\windows\:

.\build.ps1 -SkipDownload

Put a .gguf document in the models\ folder. Easiest way — use the included downloader:

go build -o dist\modelget.exe .\cmd\modelget .\dist\modelget.exe -repo meta-llama/Llama-3.2-3B-Instruct -file Llama-3.2-3B-Instruct-Q8_0.gguf -out .\models\

Or download any compatible GGUF from Hugging Face manually.

Edit .env — you must set the example path:

INFERENCE_BACKEND=vulkan JANUS_MODEL_PATH=./models/Llama-3.2-3B-Instruct-Q8_0.gguf JANUS_MAX_TOKENS=4096 JANUS_AUTH=false

Or rebuild + initiate in one step:

Janus opens http://127.0.0.1:8990 in your browser (disable alongside JANUS_NO_BROWSER=1).

First startup loads the example into VRAM — expect 10–60 seconds depending on example size and disk speed.

curl http://127.0.0.1:8990/health

You should see {"status":"ok",...}.

git copy https://github.com/Vibra-Ingenn/Janus.git cd janus go mod tidy go build -o dist/janus ./cmd/janus cp .env.example .env # edit .env — set JANUS_MODEL_PATH and INFERENCE_BACKEND=cpu if no Vulkan ./dist/janus

On Linux you need libllama.so next to the binary or on LD_LIBRARY_PATH. See docs/AIR_GAPPED_INSTALL.md for offline setup.

After .\dist\janus.exe starts, your browser should open http://127.0.0.1:8990 (or open that URL yourself). Wait for the position pill to display your example — archetypal burden can obtain 10–60 seconds.

Assistant — finest for “get this done” tasks:

1. Type what you desire in the big content box
2. Optional: drag a document onto the upload area or click attach
3. Click Get Started
4. Read the result; use Copy or commencement a New Request

Chat — finest for back-and-forth conversation:

1. Open the Chat tab
2. Type a communication and media Enter (Shift+Enter for a new line)
3. Pick a example from the dropdown if you have additional than one

Click ⚙ Advanced in the top nav to reveal:

1. Advanced → Config
2. Choose a example from the catalog (or paste a way to ./models/your-model.gguf)
3. Click Save & Load Model — no restart needed

Focus the terminal anywhere Janus is operating and media Ctrl+C.

More walkthroughs (uploads, troubleshooting, tools): docs/USER_MANUAL.md

Janus is meant to remain out of your way — run prompts from a terminal, a script, or a third-party app by pointing it at Janus akin any another OpenAI endpoint:

curl http://127.0.0.1:8990/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{  "model": "local",  "messages": [{"role": "user", "content": "Hello!"}]  }'

Base URL: http://127.0.0.1:8990/v1
API key: not required whenever JANUS_AUTH=false

Full tool reference: docs/TOOLS_REFERENCE.md
Operator guide: docs/USER_MANUAL.md

If you already use Ollama alternatively of local GGUF:

INFERENCE_BACKEND=ollama OLLAMA_BASE_URL=http://127.0.0.1:11434 OLLAMA_MODEL=mistral:7b

Janus proxies conversation to Ollama; tools and the web UI motionless work.

If you're construction or hacking on Janus, these are the gotchas that burned us repeatedly. None of this is apparent the archetypal time.

If item motionless feels possessed, inspect logs/janus.log in the project directory and the terminal output from startup.

• Check JANUS_MODEL_PATH in .env matches a genuine document under models/
• Use Config → Save & Load Model in the web UI to choice from detected .gguf files
• Paths are related to anywhere you run janus.exe (usually project base or dist/)

Normal — the example loads on archetypal petition or at startup. Smaller quantizations (Q4, Q5) burden faster than Q8.

1. Update GPU drivers
2. Try CPU: INFERENCE_BACKEND=cpu and JANUS_GPU_LAYERS=0
3. Or use Ollama backend (above)

See the pitfalls table — nearly continually a old janus.exe on Windows, or the incorrect harbor (default 8990, not 8080). If another app owns the port, set JANUS_LISTEN_ADDR=127.0.0.1:8991 in .env.

When JANUS_AUTH=true, Janus prints an admin password on archetypal run. Use it for Config/Model admin routes, or set JANUS_ADMIN_PASSWORD in .env before archetypal launch.

• OCR needs Tesseract installed and on PATH
• PDF content extraction in OSS is constricted — use ocr_extract for scans, or nexus plain content / Word files
• Output PDFs via render_pdf activity without additional installs

cmd/janus/ Main server cmd/modelget/ Hugging Face example downloader internal/kernel/ ReAct iteration (model + tools) internal/tools/ Built-in and community tools internal/engine/ llama.cpp Vulkan/CPU backend internal/bridge/ DLL loader models/ Put .gguf records current (not committed) dist/ janus.exe + llama.dll following build docs/ Manuals and references 

Hacking on Janus is welcome. Read Pitfalls (learned the difficult way) archetypal — particularly the Go-on-Windows rule: stop all janus.exe processes before go build, or you’ll think your fix didn’t work.

Typical loop:

Get-Process -Name "janus" -ErrorAction SilentlyContinue | Stop-Process -Force go test ./... go build -o dist\janus.exe .\cmd\janus .\dist\janus.exe

Or fair .\run.ps1 (stop → build → run in one step). For a complete Windows build including llama DLLs, use .\build.ps1.

Contributions greeted — see CONTRIBUTING.md.

Janus is complete on its own — nothing current is missing or locked rearward a paywall. Use it from the CLI, scripts, Cursor, the Web UI, or any OpenAI-compatible client.

If you bask operating local models and desire to see what alternatively is out there, the identical squad makes Vibe Engine PRO — a distinct merchandise focused on recipe-based automation (multi-step workflows, MCP, pipelines that run in Go without calling the example all step). It is one option, not a requirement. You can additionally broaden Janus yourself via the API.

Vibe Engine connects to a local OpenAI-compatible endpoint — Janus plant for that:

Base URL: http://127.0.0.1:8990/v1 API key: (leave blank whenever JANUS_AUTH=false) 

Early subscribers at $8.99/month remain at that charge if the listed cost goes up later.

• adeptuscamini.com · Download · Demo

Take a appearance if it appears interesting. If Janus is all you need, that's fine too.

MIT — see LICENSE.