Installing NixOS on a Proxmox VM using nixos-anywhere
- TL;DR: Fujin Quick Install (Bare Metal)
- TL;DR: Proxmox Quick Install (Remote)
- Table of Contents
- Fujin Installation (Bare Metal)
- Proxmox Installation Methods
- Post-Installation: Secrets Management
- Optional NixOS Modules
- Notes and Configuration Details
- TODOs
- Inspiration
Abstract
This guide documents methods for installing NixOS on a Proxmox virtual machine and a bare-metal machine (Fujin). It covers remote deployment, bare-metal installation with Disko, and building Proxmox image templates. It also covers post-installation steps for secrets management with sops-nix and lists available custom modules.
TL;DR: Fujin Quick Install (Bare Metal)
- Boot Live Environment: Boot into a NixOS installer or another Linux environment (like Izanami).
-
Clone Repo: Clone this repository.
git clone https://github.com/LichHunter/nixos -
Install with Disko: Run the Disko installer script for the minimal configuration.
nix run github:nix-community/disko#disko-install -- --flake .#fujin-minimal --disk main /dev/nvme0n1 - Reboot & Setup: Reboot into the new system. Mount your backup drive, restore your SSH keys, and clone the repository again.
-
Rebuild to Main Config: Use the custom script to switch to the full configuration.
./bin/rebuild.sh boot
TL;DR: Proxmox Quick Install (Remote)
-
Prepare VM: Boot the target Proxmox VM from a NixOS ISO and set a root password:
passwd -
Deploy NixOS: From your workstation, run
nixos-anywhere.nix run github:nix-community/nixos-anywhere -- --flake .#your-machine-name --target-host root@<vm-ip-address> -
Manage Secrets: Get the host's AGE key, add it to
.sops.yaml, and re-encrypt.sops updatekeys secrets/secrets.yaml
Table of Contents TOC
Fujin Installation (Bare Metal)
Boot Drive Installation
- Boot into izanami or another suitable Linux live environment.
-
Clone the repository:
git clone https://github.com/LichHunter/nixos -
Use Disko to install the minimal configuration for Fujin.
nix run github:nix-community/disko#disko-install -- --flake .#fujin-minimal --disk main /dev/nvme0n1 - Reboot the machine.
Minimal System Setup
-
Mount your backup drive:
mkdir /tmp/drive; sudo mount /dev/sda1 /tmp/drive - Copy the latest backup from the drive to your home folder.
- Unarchive the backup to restore essential files, including your SSH keys.
-
Clone your NixOS repository using your SSH key:
git clone git@github.com:LichHunter/nixos -
Upgrade to the main configuration using the provided rebuild script. This script handles the full `nixos-rebuild boot –flake .#fujin` command, including `sudo` and build host settings.
./bin/rebuild.sh boot -
Install Emacs:
git clone --depth 1 https://github.com/doomemacs/doomemacs ~/.config/emacs ~/.config/emacs/bin/doom install -
Copy your custom Doom Emacs configs:
cp -r ~/nixos/machines/fujin/main/doom-configs/* ~/.config/doom/ -
Sync your Doom Emacs configuration:
~/.config/emacs/bin/doom sync
Proxmox Installation Methods
Method 1: Remote Installation with nixos-anywhere
This method involves booting a minimal NixOS ISO on the target VM and then "pushing" the full configuration to it remotely.
Prerequisites
The minimal NixOS installation ISO does not have a default password for the root user. The nixos-anywhere command requires SSH access, which necessitates a password.
- Boot the Proxmox VM using the minimal NixOS installation ISO.
- Open a terminal on the VM's console.
-
Set a password for the
rootuser by running the following command:passwd
Deploying NixOS
With the root password set on the target VM, run nixos-anywhere from your local machine to deploy your NixOS configuration.
nix run github:nix-community/nixos-anywhere -- \
--flake .#susano-minimal \
--target-host root@192.168.1.85
After this step, proceed to the Post-Installation: Secrets Management section.
Method 2: Deployment via Proxmox Image Template
This method involves building a complete Proxmox backup file (.vma.zst) directly with Nix. This image can then be restored in Proxmox to create a new VM or a reusable template. This approach is faster for creating multiple machines.
Step 1: Build the Proxmox Image
Build the image using a dedicated flake output. This will produce a compressed Proxmox backup file in the ./result/ directory.
nix build .#izanami-proxmox
Step 2: Copy Image to Proxmox Host
You must copy the image to the directory Proxmox uses for backups. First, find this location by running the following command on your Proxmox host:
cat /etc/pve/storage.cfg
Look for a storage location (like dir: local) that includes backup in its content list. The path for that storage (e.g., /var/lib/vz) is the destination. Backups are typically stored in a dump subdirectory within that path.
Use scp to copy the generated .vma.zst file to the backup directory.
scp result/vzdump-*.vma.zst root@192.168.1.53:/var/lib/vz/dump/
Step 3: Restore Image from Proxmox UI
- Navigate to your Proxmox web UI.
- Select your backup storage location from the left-hand menu.
- Go to the Backups tab, select the newly uploaded image, and click the Restore button.
- Important: In the restore dialog, ensure the Unique checkbox is enabled. This generates a new MAC address and other unique identifiers for the restored VM.
Step 4: Test and Convert to Template
-
(Recommended) Before creating a template, test the restored VM. Create a full clone of it, start the clone, and verify you can access it as expected (e.g., via SSH with the pre-configured user).
ssh izanami@some_ip - Once confirmed, you can convert the original restored VM into a template for easy reuse. Right-click the VM and select Convert to template.
Post-Installation: Secrets Management
(This section is primarily for Method 1, or for when a new host key needs to be added after using Method 2)
Step 1: Generating the Host AGE Key
After the installation is complete, you will need the host's AGE key to manage secrets with tools like sops-nix.
-
SSH into the newly installed NixOS machine.
ssh root@192.168.1.85 -
Run the following command to convert the host's public SSH key to an AGE key.
nix-shell -p ssh-to-age --run 'cat /etc/ssh/ssh_host_ed25519_key.pub | ssh-to-age' - Copy the output AGE key for the next step.
Step 2: Updating SOPS and Re-encrypting Secrets
The new AGE key must be added to your .sops.yaml file.
- Open the
.sops.yamlfile in the root of your Nix flake. -
Replace the old key for the host with the new key you generated.
keys: - &primary age19wvqtn4ju6k4vs8fxr34unl6xx4cv04jw0lx9ps20xlde927zfssgl4qke - &susano age1vkfq9gpqfpyq3s9e79e6vw8kv9485tzna4fm3dy6p0u9uz9feu8qr9sgcf # <--- REPLACE THIS WITH THE NEW KEY creation_rules: - path_regex: secrets/secrets.yaml$ key_groups: - age: - *primary - *susano -
After saving, run the
updatekeyscommand to re-encrypt the secrets file with the new set of keys.sops updatekeys secrets/secrets.yaml
Optional NixOS Modules
Reverse Proxies
The following modules can be enabled to provide a reverse proxy.
Note: Previously, all reverse proxy modules were considered non-functional. Recent troubleshooting has provided a fix for Traefik, but the other modules may still have issues.
NGINX
The initial switch to this configuration may be slow as it waits for ACME to issue SSL certificates.
dov = {
# Reverse Proxy
reverse-proxy = {
nginx.enable = true;
};
};
Traefik
dov = {
# Reverse Proxy
reverse-proxy = {
traefik.enable = true;
};
};
Troubleshooting Traefik ACME with DuckDNS
- Context: Issues getting an ACME certificate from DuckDNS with Traefik.
- Roadblock: The process was failing, but eventually succeeded.
-
Possible Solutions:
- Setting
disablePropagationCheck = true;for the DNS challenge. - Extending the
delayfor the DNS challenge.
- Setting
- Notes: It's unclear which specific option resolved the issue. The first time Traefik tries to get a certificate it might fail, and a restart of the service is needed. After some time, the Let's Encrypt certificate will be received.
Caddy
dov = {
# Reverse Proxy
reverse-proxy = {
caddy.enable = true;
};
};
File Servers
copyparty
Provides a web-based file manager. For more information, see the official documentation.
dov = {
file-server.copyparty.enable = true;
};
Dependency: For copyparty to function correctly in this configuration, it requires a Samba share to be mounted to the path /MEDIA. Therefore, the samba module must also be enabled:
dov = {
samba.enable = true;
};
Search Engines
searxng
A privacy-respecting metasearch engine. For more information, see the NixOS Wiki page.
dov = {
searxng.enable = true;
};
Notes and Configuration Details
Disko Configuration for Proxmox (MBR Boot)
A critical requirement for ensuring a NixOS VM can boot correctly in Proxmox is the disk partition scheme. Proxmox expects a Master Boot Record (MBR) compatible setup.
When using disko for declarative disk management, you must configure it to create a GPT partition table that includes a special 1M BIOS boot partition (type EF02). This partition is specifically used by GRUB for MBR compatibility.
Here is an example snippet for the disko configuration:
{
disko.devices = {
disk = {
main = {
device = "/dev/sda";
type = "disk";
content = {
type = "gpt";
partitions = {
boot = {
size = "1M";
type = "EF02"; # for grub MBR
};
# ... your other partitions like root, swap, etc.
};
};
};
};
};
}
For a complete example, you can refer to the official disko repository: gpt-bios-compat.nix.
Generating Hardware Configuration
The nixos-anywhere tool can automatically generate a hardware configuration file from the target machine. This is useful for capturing machine-specific settings.
nix run github:nix-community/nixos-anywhere -- \
--flake .#your-flake-output \
--target-host root@192.168.1.85 \
--generate-hardware-config ./hardware-configuration.nix
TODOs
- Update izanagi to include git by default.
- Add NetworkManager to the fujin-minimal configuration.
- Investigate and fix remaining issues with reverse proxy modules (NGINX, Caddy).
- Troubleshoot and fix an issue that occurs when reloading the NixOS configuration remotely, which breaks the SSH pipe and requires entering the root password three times.
- Investigate and resolve the issue where updating a user's password declaratively using a secret managed by
sopsfailed after the initial installation. - Refactor the
diskoconfiguration to make the disk device name (e.g.,/dev/sda) a variable. This will avoid hardcoding the value and make the configuration more portable. - Create a custom ISO image to streamline the installation process, potentially pre-configuring items like the root user to avoid manual console steps.
- Develop an automated installation script to handle the post-install process, such as fetching the AGE key and updating sops, based on this guide.
Inspiration
The configuration and structure of this setup were inspired by the following repository: