#+TITLE: Installing NixOS on a Proxmox VM using nixos-anywhere
#+AUTHOR: Alexander Derevianko
#+DATE: <2025-07-26 Sat>
#+OPTIONS: toc:t num:nil

*Abstract*
This guide documents the process for a minimal installation of NixOS on a Proxmox virtual machine. It leverages the =nixos-anywhere= tool for remote deployment and =disko= for declarative disk partitioning. It also covers the essential post-installation steps for integrating the new host with =sops-nix= for secrets management and lists available custom modules.

* TL;DR: Quick Install Guide
1. *Prepare VM:* Boot the target Proxmox VM from a NixOS ISO and set a root password:
   #+begin_src sh
   passwd
   #+end_src

2. *Deploy NixOS:* From your workstation, run =nixos-anywhere=, pointing to your flake and the VM's IP address.
   #+begin_src sh
   nix run github:nix-community/nixos-anywhere -- \
     --flake .#your-machine-name \
     --target-host root@<vm-ip-address>
   #+end_src

3. *Get Host Key:* After installation, SSH into the new VM and get its host AGE key.
   #+begin_src sh
   ssh root@<vm-ip-address>
   nix-shell -p ssh-to-age --run 'cat /etc/ssh/ssh_host_ed25519_key.pub | ssh-to-age'
   #+end_src

4. *Update Secrets:* On your workstation, add the new AGE key to =.sops.yaml= and re-encrypt secrets.
   #+begin_src sh
   sops updatekeys secrets/secrets.yaml
   #+end_src

* Table of Contents                                                            :TOC:
- [[#tldr-quick-install-guide][TL;DR: Quick Install Guide]]
- [[#prerequisites-on-the-target-vm][Prerequisites on the Target VM]]
- [[#installation-process][Installation Process]]
  - [[#deploying-nixos][Deploying NixOS]]
- [[#post-installation-secrets-management][Post-Installation: Secrets Management]]
  - [[#step-1-generating-the-host-age-key][Step 1: Generating the Host AGE Key]]
  - [[#step-2-updating-sops-and-re-encrypting-secrets][Step 2: Updating SOPS and Re-encrypting Secrets]]
- [[#optional-nixos-modules][Optional NixOS Modules]]
  - [[#reverse-proxies][Reverse Proxies]]
  - [[#file-servers][File Servers]]
- [[#notes-and-configuration-details][Notes and Configuration Details]]
  - [[#disko-configuration-for-proxmox-mbr-boot][Disko Configuration for Proxmox (MBR Boot)]]
  - [[#generating-hardware-configuration][Generating Hardware Configuration]]
- [[#todos][TODOs]]

* Prerequisites on the Target VM
Before attempting to install NixOS with =nixos-anywhere=, you must first perform a critical setup step on the target Proxmox VM.

The minimal NixOS installation ISO does not have a default password for the =root= user. The =nixos-anywhere= command requires SSH access, which necessitates a password.

1. Boot the Proxmox VM using the minimal NixOS installation ISO.
2. Open a terminal on the VM's console.
3. Set a password for the =root= user by running the following command:
   #+begin_src sh
   passwd
   #+end_src
   You will be prompted to enter and confirm a new password.

* Installation Process
** Deploying NixOS
With the root password set on the target VM, you can now run =nixos-anywhere= from your local machine to deploy your NixOS configuration.

The following command uses =nix run= to execute =nixos-anywhere=, pointing it to a specific flake output (=.#susano-minimal=) and the IP address of the target VM.

#+begin_src sh
nix run github:nix-community/nixos-anywhere -- \
  --flake .#susano-minimal \
  --target-host root@192.168.1.85
#+end_src

* Post-Installation: Secrets Management
** Step 1: Generating the Host AGE Key
After the initial installation is complete, you will need its host AGE key to manage secrets with tools like =sops-nix=. This key is derived from the host's SSH key.

1. SSH into the newly installed NixOS machine.
   #+begin_src sh
   ssh root@192.168.1.85
   #+end_src

2. Run the following command. It temporarily installs the =ssh-to-age= utility and pipes the public SSH host key to it, converting it to an AGE public key.
   #+begin_src sh
   nix-shell -p ssh-to-age --run 'cat /etc/ssh/ssh_host_ed25519_key.pub | ssh-to-age'
   #+end_src

3. The command will output the new AGE public key. Copy this key for the next step.

** Step 2: Updating SOPS and Re-encrypting Secrets
The new AGE key must be added to your =.sops.yaml= configuration file. This allows =sops= to encrypt secrets in a way that the new host (=susano=) can decrypt them.

1. Open the =.sops.yaml= file in the root of your Nix flake.
2. Replace the old key for the =susano= host with the new key you generated.

   #+begin_src yaml
   keys:
     - &primary age19wvqtn4ju6k4vs8fxr34unl6xx4cv04jw0lx9ps20xlde927zfssgl4qke
     - &susano age1vkfq9gpqfpyq3s9e79e6vw8kv9485tzna4fm3dy6p0u9uz9feu8qr9sgcf # <--- REPLACE THIS WITH THE NEW KEY
   creation_rules:
     - path_regex: secrets/secrets.yaml$
       key_groups:
         - age:
             - *primary
             - *susano
   #+end_src

3. After saving the updated =.sops.yaml= file, run the =updatekeys= command. This re-encrypts the specified secrets file with the new set of keys defined in =.sots.yaml=. For more information, see the [[https://github.com/getsops/sops?tab=readme-ov-file#281updatekeys-command][official documentation]].
   #+begin_src sh
   sops updatekeys secrets/secrets.yaml
   #+end_src
   Your secrets are now encrypted for both the primary key and the new host's key.

* Optional NixOS Modules
** Reverse Proxies
The following modules can be enabled to provide a reverse proxy.

*Note:* Previously, all reverse proxy modules were considered non-functional. Recent troubleshooting has provided a fix for Traefik, but the other modules may still have issues.

*** NGINX
The initial switch to this configuration may be slow as it waits for ACME to issue SSL certificates.
#+begin_src nix
dov = {
  # Reverse Proxy
  reverse-proxy = {
    nginx.enable = true;
  };
};
#+end_src

*** Traefik
#+begin_src nix
dov = {
  # Reverse Proxy
  reverse-proxy = {
    traefik.enable = true;
  };
};
#+end_src

**** Troubleshooting Traefik ACME with DuckDNS
- *Context:* Issues getting an ACME certificate from DuckDNS with Traefik.
- *Roadblock:* The process was failing, but eventually succeeded.
- *Possible Solutions:*
  - Setting =disablePropagationCheck = true;= for the DNS challenge.
  - Extending the =delay= for the DNS challenge.
- *Notes:* It's unclear which specific option resolved the issue, but one of them, or a combination, allowed the certificate to be obtained. The first time Traefik tries to get a certificate it might fail, and a restart of the service is needed. After some time, the Let's Encrypt certificate will be received.

*** Caddy
#+begin_src nix
dov = {
  # Reverse Proxy
  reverse-proxy = {
    caddy.enable = true;
  };
};
#+end_src

** File Servers
*** copyparty
Provides a web-based file manager.
#+begin_src nix
dov = {
  file-server.copyparty.enable = true;
};
#+end_src

*Dependency:* For =copyparty= to function correctly in this configuration, it requires a Samba share to be mounted to the path =/MEDIA=. Therefore, the =samba= module must also be enabled:
#+begin_src nix
dov = {
  samba.enable = true;
};
#+end_src

* Notes and Configuration Details
** Disko Configuration for Proxmox (MBR Boot)
A critical requirement for ensuring a NixOS VM can boot correctly in Proxmox is the disk partition scheme. Proxmox expects a Master Boot Record (MBR) compatible setup.

When using =disko= for declarative disk management, you must configure it to create a GPT partition table that includes a special 1M BIOS boot partition (type =EF02=). This partition is specifically used by GRUB for MBR compatibility.

Here is an example snippet for the =disko= configuration:

#+begin_src nix
{
  disko.devices = {
    disk = {
      main = {
        device = "/dev/sda";
        type = "disk";
        content = {
          type = "gpt";
          partitions = {
            boot = {
              size = "1M";
              type = "EF02"; # for grub MBR
            };
            # ... your other partitions like root, swap, etc.
          };
        };
      };
    };
  };
}
#+end_src

For a complete example, you can refer to the official =disko= repository: [[https://github.com/nix-community/disko/blob/master/example/gpt-bios-compat.nix][gpt-bios-compat.nix]].

** Generating Hardware Configuration
The =nixos-anywhere= tool can automatically generate a hardware configuration file from the target machine. This is useful for capturing machine-specific settings.

To do this, include the =--generate-hardware-config= flag in your command. The following example shows how to generate the file and save it as =./hardware-configuration.nix= in your local flake directory.

#+begin_src sh
nix run github:nix-community/nixos-anywhere -- \
  --flake .#your-flake-output \
  --target-host root@192.168.1.85 \
  --generate-hardware-config ./hardware-configuration.nix
#+end_src

* TODOs
- [ ] Investigate and fix remaining issues with reverse proxy modules (NGINX, Caddy).
- [ ] Troubleshoot and fix an issue that occurs when reloading the NixOS configuration remotely, which breaks the SSH pipe and requires entering the root password three times.
- [ ] Investigate and resolve the issue where updating a user's password declaratively using a secret managed by =sops= failed after the initial installation.
- [ ] Refactor the =disko= configuration to make the disk device name (e.g., =/dev/sda=) a variable. This will avoid hardcoding the value and make the configuration more portable.
- [ ] Create a custom ISO image to streamline the installation process, potentially pre-configuring items like the root user to avoid manual console steps.
- [ ] Develop an automated installation script to handle the post-install process, such as fetching the AGE key and updating sops, based on [[https://unmovedcentre.com/posts/remote-install-nixos-config/#update-sops-file][this guide]].