Nixos/README.org

Installing NixOS on a Proxmox VM using nixos-anywhere

• TL;DR: Quick Install Guide
• Table of Contents
• Prerequisites on the Target VM
• Installation Process
  • Deploying NixOS
• Post-Installation: Secrets Management
  • Step 1: Generating the Host AGE Key
  • Step 2: Updating SOPS and Re-encrypting Secrets
• Notes and Configuration Details
  • Disko Configuration for Proxmox (MBR Boot)
  • Generating Hardware Configuration
• TODOs

Abstract This guide documents the process for a minimal installation of NixOS on a Proxmox virtual machine. It leverages the nixos-anywhere tool for remote deployment and disko for declarative disk partitioning. It also covers the essential post-installation steps for integrating the new host with sops-nix for secrets management.

TL;DR: Quick Install Guide

1. Prepare VM: Boot the target Proxmox VM from a NixOS ISO and set a root password:

   passwd

2. Deploy NixOS: From your workstation, run nixos-anywhere, pointing to your flake and the VM's IP address.

   nix run github:nix-community/nixos-anywhere -- \
     --flake .#your-machine-name \
     --target-host root@<vm-ip-address>

3. Get Host Key: After installation, SSH into the new VM and get its host AGE key.

   ssh root@<vm-ip-address>
   nix-shell -p ssh-to-age --run 'cat /etc/ssh/ssh_host_ed25519_key.pub | ssh-to-age'

4. Update Secrets: On your workstation, add the new AGE key to .sops.yaml and re-encrypt secrets.

   sops updatekeys secrets/secrets.yaml

Table of Contents   TOC

• TL;DR: Quick Install Guide
• Prerequisites on the Target VM

• Installation Process

  • Deploying NixOS

• Post-Installation: Secrets Management

  • Step 1: Generating the Host AGE Key
  • Step 2: Updating SOPS and Re-encrypting Secrets

• Notes and Configuration Details

  • Disko Configuration for Proxmox (MBR Boot)
  • Generating Hardware Configuration
• TODOs

Prerequisites on the Target VM

Before attempting to install NixOS with nixos-anywhere, you must first perform a critical setup step on the target Proxmox VM.

The minimal NixOS installation ISO does not have a default password for the root user. The nixos-anywhere command requires SSH access, which necessitates a password.

1. Boot the Proxmox VM using the minimal NixOS installation ISO.
2. Open a terminal on the VM's console.

3. Set a password for the root user by running the following command:

   passwd

   You will be prompted to enter and confirm a new password.

Installation Process

Deploying NixOS

With the root password set on the target VM, you can now run nixos-anywhere from your local machine to deploy your NixOS configuration.

The following command uses nix run to execute nixos-anywhere, pointing it to a specific flake output (.#susano-minimal) and the IP address of the target VM.

nix run github:nix-community/nixos-anywhere -- \
  --flake .#susano-minimal \
  --target-host root@192.168.1.85

Post-Installation: Secrets Management

Step 1: Generating the Host AGE Key

After the initial installation is complete, you will need its host AGE key to manage secrets with tools like sops-nix. This key is derived from the host's SSH key.

1. SSH into the newly installed NixOS machine.

   ssh root@192.168.1.85

2. Run the following command. It temporarily installs the ssh-to-age utility and pipes the public SSH host key to it, converting it to an AGE public key.

   nix-shell -p ssh-to-age --run 'cat /etc/ssh/ssh_host_ed25519_key.pub | ssh-to-age'

3. The command will output the new AGE public key. Copy this key for the next step.

Step 2: Updating SOPS and Re-encrypting Secrets

The new AGE key must be added to your .sops.yaml configuration file. This allows sops to encrypt secrets in a way that the new host (susano) can decrypt them.

1. Open the .sops.yaml file in the root of your Nix flake.

2. Replace the old key for the susano host with the new key you generated.

   keys:
     - &primary age19wvqtn4ju6k4vs8fxr34unl6xx4cv04jw0lx9ps20xlde927zfssgl4qke
     - &susano age1vkfq9gpqfpyq3s9e79e6vw8kv9485tzna4fm3dy6p0u9uz9feu8qr9sgcf # <--- REPLACE THIS WITH THE NEW KEY
   creation_rules:
     - path_regex: secrets/secrets.yaml$
       key_groups:
         - age:
             - *primary
             - *susano

3. After saving the updated .sops.yaml file, run the updatekeys command. This re-encrypts the specified secrets file with the new set of keys defined in .sops.yaml. For more information, see the official documentation.

   sops updatekeys secrets/secrets.yaml

   Your secrets are now encrypted for both the primary key and the new host's key.

Notes and Configuration Details

Disko Configuration for Proxmox (MBR Boot)

A critical requirement for ensuring a NixOS VM can boot correctly in Proxmox is the disk partition scheme. Proxmox expects a Master Boot Record (MBR) compatible setup.

When using disko for declarative disk management, you must configure it to create a GPT partition table that includes a special 1M BIOS boot partition (type EF02). This partition is specifically used by GRUB for MBR compatibility.

Here is an example snippet for the disko configuration:

{
  disko.devices = {
    disk = {
      main = {
        device = "/dev/sda";
        type = "disk";
        content = {
          type = "gpt";
          partitions = {
            boot = {
              size = "1M";
              type = "EF02"; # for grub MBR
            };
            # ... your other partitions like root, swap, etc.
          };
        };
      };
    };
  };
}

For a complete example, you can refer to the official disko repository: gpt-bios-compat.nix.

Generating Hardware Configuration

The nixos-anywhere tool can automatically generate a hardware configuration file from the target machine. This is useful for capturing machine-specific settings.

To do this, include the --generate-hardware-config flag in your command. The following example shows how to generate the file and save it as ./hardware-configuration.nix in your local flake directory.

nix run github:nix-community/nixos-anywhere -- \
  --flake .#your-flake-output \
  --target-host root@192.168.1.85 \
  --generate-hardware-config ./hardware-configuration.nix

TODOs

• Refactor the disko configuration to make the disk device name (e.g., /dev/sda) a variable. This will avoid hardcoding the value and make the configuration more portable across different hardware setups.
• Investigate and resolve the issue where updating a user's password declaratively using a secret managed by sops failed after the initial installation.