Show HN: NixOS-DGX-Spark – Nix and NixOS on the DGX Spark

Try DGX Spark playbooks utilizing Nix connected DGX OS, aliases instal NixOS connected your DGX Spark for the afloat Nix experience. The repository provides USB images and a NixOS module pinch settings for DGX Spark systems.

This useful connected the NVIDIA DGX Spark itself and besides connected the Asus Ascent GX10.

See my 5 infinitesimal lightning talk from Planet Nix for an intro: https://youtu.be/AvK_gi_snJE?si=MPKv3iiuS9B5elIE

You tin usage the dev shells and playbooks successful this repo connected NVIDIA DGX OS (Ubuntu) without installing NixOS.

1. Install Nix utilizing the official installer:

   sh <(curl -L https://nixos.org/nix/install) --daemon

   Alternatively, you tin usage the Determinate Nix Installer:

   curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install

2. Enable flakes and nix-command by adding to /etc/nix/nix.conf:

   experimental-features = nix-command flakes

3. Enable the graham33 Cachix cache — spot Caching below.

On non-NixOS systems, Nix-built CUDA applications need nix-gl-host to find the big GPU drivers. The playbook devshells grip this automatically — container-based playbooks don't request it, and Nix-native playbooks wrap their commands with nixglhost truthful nary manual involution is required.

For different devshells (e.g. cuda), nixglhost is disposable and tin beryllium used to prefix commands manually:

Build the USB image:

The image includes 2 kernel options, selectable from the GRUB footwear menu:

• NixOS (default) - NVIDIA's specialised kernel for DGX Spark pinch afloat GPU support and moving Ethernet
• NixOS (standard-kernel) - Standard NixOS 6.17 kernel (Ethernet has problems)

Disable Secure Boot successful the DGX Spark BIOS and footwear from the USB drive.

You tin past travel the installation instructions successful the NixOS manual: https://nixos.org/manual/nixos/stable/#sec-installation-manual

This module provides configurable DGX Spark hardware support pinch options for kernel selection.

The NVIDIA kernel is simply a civilization build optimised for NVIDIA DGX Spark systems. The kernel configuration is generated from NVIDIA's Debian annotations and compared pinch NixOS defaults to nutrient a minimal, maintainable configuration.

The module besides enables the DGX Dashboard web interface at http://localhost:11000, providing GPU telemetry and strategy monitoring.

The kernel configuration is generated from NVIDIA's Debian annotations and stored successful kernel-configs/nvidia-dgx-spark-<version>.nix. This terse configuration only contains options that disagree from NixOS defaults, reducing verbosity by ~82%.

To regenerate the kernel configuration:

This:

1. Fetches the NVIDIA kernel root from GitHub
2. Builds the NixOS baseline kernel config
3. Compares NVIDIA's annotations pinch NixOS defaults
4. Generates a terse config record pinch only the differences

Regeneration is needed when:

• NVIDIA kernel type changes (update kernel-configs/nvidia-kernel-source.nix)
• NixOS common-config changes (nixpkgs update)

You tin besides usage a section kernel root for development:

Other projects tin import this flake and usage the DGX Spark module:

For a complete NixOS configuration template specifically designed for DGX Spark systems, you tin usage the template:

This creates a complete NixOS configuration with:

• flake.nix - Flake configuration that imports the DGX Spark module
• configuration.nix - Main strategy configuration optimised for DGX Spark
• hardware-configuration.nix - Hardware configuration template

After initialising the template, you request to:

1. Generate hardware configuration and update template:

   # Generate hardware config to a impermanent location to get the existent UUIDs sudo nixos-generate-config --root /mnt --dir /tmp/nixos-config # Copy the existent hardware UUIDs and settings from the generated file # Replace the placeholder UUIDs successful hardware-configuration.nix pinch actual # values from /tmp/nixos-config/hardware-configuration.nix

2. Edit configuration.nix to customise:

   • Change hostname from dgx-spark to your preferred name
   • Update username from nixos to your preferred username
   • Add your SSH nationalist keys for distant access
   • Set your timezone and locale preferences
   • Add immoderate further packages you need

3. Deploy the configuration to /etc/nixos:

   # Copy your configuration to /etc/nixos sudo cp -r . /etc/nixos/ # Apply the configuration sudo nixos-rebuild move --flake /etc/nixos#dgx-spark

As noted in #32, only DGX OS tin footwear from the mill firmware. If NixOS can't boot from the mill firmware, you request to update firmware from DGX OS first.

The module enables fwupd for firmware updates. NVIDIA publishes DGX Spark firmware to the Linux Vendor Firmware Service (LVFS). To check for disposable updates:

To instal disposable updates:

This repository includes devshells for NVIDIA DGX Spark playbooks from https://build.nvidia.com/spark:

Playbook Description Type Tested connected NixOS Tested connected DGX OS

ComfyUI	Run ComfyUI pinch Stable Diffusion 1.5 for AI image generation	🟢 Full Nix²	✅	✅
Connect Two Sparks	Connect 2 DGX Spark systems via QSFP	🟢 Full Nix²	✅	☑️¹
DGX Dashboard	Set up DGX Dashboard for strategy monitoring	🟢 Full Nix²	✅	☑️¹
FLUX.1 Dreambooth	FLUX.1 Dreambooth LoRA fine-tuning	🟠 Container³	✅	✅
Multi-Agent Chatbot	Build and deploy a multi-agent chatbot	🟠 Container³	✅	✅
Multi-modal Inference	Run multi-modal conclusion pinch vision-language models	🟠 Container³	✅	✅
NCCL for Two Sparks	Multi-node GPU connection pinch NCCL	🟢 Full Nix²	✅	☑️¹
NVFP4	FP4 exemplary quantisation pinch TensorRT Model Optimizer	🟠 Container³	✅
OpenShell	Secure long-running AI agents pinch OpenShell sandbox	🔵 Nix + Container⁴	✅
PyTorch Fine-tuning Container	Fine-tune models pinch PyTorch connected DGX Spark	🟠 Container³	✅
PyTorch Fine-tuning Nix	Fine-tune models pinch PyTorch (Nix native, nary containers)	🟢 Full Nix²	✅	✅
Speculative Decoding	Speculative decoding for faster inference	🟠 Container³	✅
TRT-LLM	TensorRT-LLM for optimised inference	🟠 Container³	✅	✅
vLLM Container	Run vLLM conclusion server pinch Qwen2.5-Math-1.5B-Instruct model	🟠 Container³	✅	✅
vLLM Nix	Run vLLM conclusion server natively (Nix native, nary containers)	🟢 Full Nix²	✅	✅

¹ Pre-installed connected DGX OS ² Full Nix: Fully reproducible — each limitations installed via Nix ³ Container: Nix provides podman, but containers are pulled/built astatine runtime ⁴ Nix + Container: CLI devices packaged via Nix, but containers are managed by the instrumentality astatine runtime

Flox distributes pre-built CUDA packages for aarch64-linux with NVIDIA's permission. This includes cudatoolkit, nccl, cuDNN, PyTorch, and different CUDA dependencies — dramatically reducing build times.

If you usage the DGX Spark NixOS module, the Flox cache is configured automatically arsenic a substituter.

For standalone Nix (e.g. connected DGX OS), adhd to /etc/nix/nix.conf:

For packages built by this repo (e.g. dgx-dashboard, openshell):

Install cachix first if needed: https://docs.cachix.org/installation

See https://nixos.wiki/wiki/CUDA for wide CUDA caching details.

You tin instal NixOS connected a DGX Spark remotely using nixos-anywhere. This is useful for headless setups wherever you person SSH entree to the target machine.

This partitions the NVMe disk and installs NixOS pinch the DGX Spark module enabled. You whitethorn want to customise nixos-anywhere/configuration.nix (e.g. to add SSH keys aliases alteration the hostname) — clone the repo and constituent --flake at your section checkout instead.

To trial the disk configuration successful a VM without installing:

See the nixos-anywhere documentation for afloat specifications and requirements.

MIT License - spot LICENSE for details.