I decided to change the setup of my ol' homeserver, a Nuc with core i3 3410u.
I have been using this machine for quite a while with an internal ssd and some external hard drive as the main storage, acting like NAS and running an instance of Nextcloud and Emby.
However I got tired of the cables / external stuff and found a nice passively cooled case by akasa. Along with the adapter they provide the old internal mSata connector can be turned into a regular Sata connector.
So I decided to buy a second harddrive to run as raid 1 and put the operating system onto an external 500gb ssd I already have. I have used it before as an external drive to run linux from, but I could use an usb stick for that, if I really ever need to do it again. I just hope that there will be a tiny bit of room left inside the case to put that ssd inside and sacrifice one of the front usbs.
Anyway, the goal is set, let’s get to it.
Whatever, take me to the TR:DL already
Formating the usb ssd
For this task, I used fdisk. There was already an EFI partition, because of the use as a external boot disk, but the current layout wasn’t what I’d like. first we find out what the drive got enumerated as. Find it using
❯ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT loop0 7:0 0 71.3M 1 loop /snap/lxd/16100 loop1 7:1 0 55M 1 loop /snap/core18/1880 loop2 7:2 0 71.3M 1 loop /snap/lxd/16044 loop3 7:3 0 96.5M 1 loop /snap/core/9436 loop5 7:5 0 9.1M 1 loop /snap/canonical-livepatch/95 loop6 7:6 0 55M 1 loop /snap/core18/1754 loop7 7:7 0 97M 1 loop /snap/core/9665 sda 8:0 0 4.6T 0 disk sdb 8:16 0 59.6G 0 disk ├─sdb1 8:17 0 512M 0 part /boot/efi ├─sdb2 8:18 0 50G 0 part / └─sdb3 8:19 0 9.1G 0 part [SWAP] sdc 8:32 0 465.8G 0 disk ├─sdc1 8:33 0 366.2M 0 part /media/efi ├─sdc2 8:34 0 455G 0 part /media/rootfs └─sdc3 8:35 0 10.4G 0 part
In my case, the the external disk is sdc. this terminal output shows the final result.
fdsik /dev/sdc. Useful commands are:
llist partition types, the ones we need are 1 (EFI) 19 (SWAP) and 20 (Linux FS)
ddelete an existing partition
nadd a new partition
wwrite any changes to disk
qquite discarding any changes
Using the tool is pretty straight forward with a lot of onscreen explanations, I won’t go into to much detail.
As I already had a GPT on this harddisk I only used
d a few times to delete the old partitions and then a
w to actually write the changes. After that I used
n two times to create the main rootfs and a generous swap area.
When asked for the size of the partitions, just enter +
+455G and I used the remaining space for swap next.
The final step here was to format the root partitions using a simple
mkfs.btrfs /dev/sdc2. That’s it.
Moving the file system
To achieve this, I will attempt to just copy the existing efi partition and btrfs snapshot of the current root. After that, adjust the UUID in the fstab of the new root. I am not sure if I need to reinstall grub at this point. However, since I don’t modify the existing boot drive, Nothing can go wrong anyway.
as root user just bashing at the keyboard
mkdir /media/boot /media/rootfs mount /dev/sdc1 /media/boot mount /dev/sdc2 /media/rootfs rm -r /media/boot/* rsync -a /boot/ /media/boot/
And to copy the rootfs with will use btrfs send & receive. Preferably In a byobu terminal, as the ssh could drop and we have to start over again.
cd / btrfs sub snap -r . @rootfs btrfs send @rootfs | btrfs rec /media/rootfs
This leaves us with a read only version of the old root fs, but I noticed that my lxd containers didn’t get moved along as those where more btrfs subvolumes. I also noticed that plexmediaserver left a 5gb folder after it’s deinstallation…
❯ btrfs sub list -o /var ID 297 gen 181903 top level 5 path var/lib/lxd/storage-pools/default ID 340 gen 181942 top level 5 path @rootfs
Now turn the rootfs snapshot into a writable subvolume and mount it somewhere for what’s next.
❯ cd /media/rootfs ❯ mv @rootfs @rootfs_ro ❯ btrfs sub snap @rootfs_ro @rootfs Create a snapshot of '@rootfs_ro' in './@rootfs' ❯ ls @rootfs @rootfs_ro ❯ mkdir /media/newroot ❯ mount -o subvol=/@rootfs /dev/sdc2 /media/newroot
Now the tricky part is, btrfs doesn’t recurse all the subvolumes in question, which is annoying. I will attempt to just recreate the emtpy ones, otherwise send them again.
❯ du -hd1 /var/lib/lxd/storage-pools/default 962M /var/lib/lxd/storage-pools/default/containers 0 /var/lib/lxd/storage-pools/default/snapshots 2.6G /var/lib/lxd/storage-pools/default/images 0 /var/lib/lxd/storage-pools/default/custom 3.6G /var/lib/lxd/storage-pools/default ❯ btrfs subvolume list --sort=path / ID 340 gen 181942 top level 5 path @rootfs ID 297 gen 182094 top level 5 path var/lib/lxd/storage-pools/default ID 298 gen 181903 top level 297 path var/lib/lxd/storage-pools/default/containers ID 312 gen 181903 top level 298 path var/lib/lxd/storage-pools/default/containers/utor ID 301 gen 181903 top level 297 path var/lib/lxd/storage-pools/default/custom ID 300 gen 182064 top level 297 path var/lib/lxd/storage-pools/default/images ID 307 gen 162420 top level 300 path var/lib/lxd/storage-pools/default/images/8aaca3494acf ID 338 gen 181905 top level 300 path var/lib/lxd/storage-pools/default/images/baef6615ff69 ID 299 gen 181903 top level 297 path var/lib/lxd/storage-pools/default/snapshots
First create the empty subvolumes
❯ cd /media/newroot/var/lib/lxd/storage-pools ❯ btrfs sub create default Create subvolume './default' ❯ btrfs sub create default/custom Create subvolume 'default/custom' ❯ btrfs sub create default/snapshots Create subvolume 'default/snapshots' ❯ btrfs sub create default/images Create subvolume 'default/images' ❯ btrfs sub create default/containers Create subvolume 'default/containers'
Then send the read-only subvolumes
cd / ❯ btrfs sub list -r . ID 307 gen 162420 top level 300 path var/lib/lxd/storage-pools/default/images/8aaca3494acf ID 338 gen 181905 top level 300 path var/lib/lxd/storage-pools/default/images/baef6615ff69 ID 340 gen 181942 top level 5 path @rootfs ❯ cd var/lib/lxd/storage-pools/default/images ❯ btrfs send /var/lib/lxd/storage-pools/default/images/8aaca3494acf5cf | btrfs rec ./ At subvol /var/lib/lxd/storage-pools/default/images/8aaca3494acf At subvol 8aaca3494acf ❯ btrfs send /var/lib/lxd/storage-pools/default/images/baef6615ff69 | btrfs rec ./ At subvol /var/lib/lxd/storage-pools/default/images/baef6615ff69 At subvol baef6615ff69
Currently I only have a single container. Create a ro snapshot, send it and finally convert it back to a rw subvolume.
❯ btrfs sub snap -r /var/lib/lxd/storage-pools/default/containers/utor /utor_ro Create a readonly snapshot of '/var/lib/lxd/storage-pools/default/containers/utor' in '//utor_ro' ❯ cd /media/newroot/var/lib/lxd/storage-pools/default/containers ❯ btrfs send /utor_ro | btrfs rec . At subvol /utor_ro At subvol utor_ro ❯ btrfs sub snap utor_ro utor Create a snapshot of 'utor_ro' in './utor' ❯ btrfs sub del utor_ro Delete subvolume (no-commit): '/media/newroot/var/lib/lxd/storage-pools/default/containers/utor_ro'
Setting up fstab with the new UUIDs
When starting out running
blkid I noticed my swap partition had no UUID. But that was just because I didn’t create the swap space yet. So no problem.
❯ mkswap /dev/sdc3 Setting up swapspace version 1, size = 10.4 GiB (11169431552 bytes) no label, UUID=6dfa501b-df5b-459c-92a9-0dd04e7407a3 ❯ blkid /dev/sda: UUID="2a94831d-0606-47dc-b212-1d342725c1e4" TYPE="crypto_LUKS" /dev/sdb1: UUID="016E-E748" TYPE="vfat" PARTUUID="381d023c-2a82-4e3e-aa8a-4f375ace8ebd" /dev/sdb2: UUID="042dd718-251d-4727-9820-e6789b89da44" UUID_SUB="51964a56-eb4d-4016-9dd2-7f091ffe62c9" TYPE="btrfs" PARTUUID="fb6469f4-b0d6-457f-9ff1-2186e7149197" /dev/sdb3: UUID="5ed2aed7-ff00-4cc2-a505-c7e413bc58c7" TYPE="swap" PARTUUID="d0bc8d57-82a2-4283-bc9b-9551c7455a99" /dev/sdc1: SEC_TYPE="msdos" UUID="D660-B8B5" TYPE="vfat" PARTLABEL="EFI System Partition" PARTUUID="e2418736-eeed-43d2-81af-7c309befc701" /dev/sdc2: UUID="3376ef79-d15c-4858-adb4-8c47020fdf04" UUID_SUB="ad77a0d2-f268-4c41-b2ce-c346f6726878" TYPE="btrfs" PARTUUID="956c5302-f8ef-3e43-97a5-1b421258e2c3" /dev/sdc3: UUID="6dfa501b-df5b-459c-92a9-0dd04e7407a3" TYPE="swap" PARTUUID="e29e35a5-63b2-b74b-b080-a64b6483f912"
Now edit the values of the newroot fstab with the UUIDs shown here. Also add the subvol option for the rootfs, as after this operation, I will run it as a subvolume, not clutered about under the main volume.
# /etc/fstab: static file system information. # <file system> <mount point> <type> <options> <dump> <pass> # / was on /dev/sda2 during curtin installation /dev/disk/by-uuid/3376ef79-d15c-4858-adb4-8c47020fdf04 / btrfs subvol=/@rootfs,defaults 0 0 /dev/disk/by-uuid/6dfa501b-df5b-459c-92a9-0dd04e7407a3 none swap sw 0 0 # /boot/efi was on /dev/sda1 during curtin installation /dev/disk/by-uuid/D660-B8B5 /boot/efi vfat defaults 0 0 # mount the base filesystem too, so we can take regular snapshots of the root system easily /dev/disk/by-uuid/3376ef79-d15c-4858-adb4-8c47020fdf04 /mnt/btrfs btrfs defaults 0 0 none /home/synsi/ramdisk tmpfs nodev,nosuid,noexec,nodiratime,size=5G 0 0
Trying to boot
Since the setup should be complete, it’s time to head into the bios and try to boot from the usb harddrive. If everything works out, this part should be done quite fast. Fingers crossed
Well, of course it didn’t work. The bios did not recognize the usb-ssd as an uefi bootable drive, only in legacy mode. But it refused to boot from it in that mode as well. I have to properly move the boot drive, I guess.
This guy saved my life, or at least, a few minutes of it.
sudo mount -subvol=/@rootfs /dev/sdb2 /mnt sudo mount /dev/sdb1 /mnt/boot/efi for i in /dev /dev/pts /proc /sys /run; do sudo mount -B $i /mnt$i; done sudo chroot /mnt grub-install /dev/sdb update-grub
Now the usb-ssd appeared properly in the bios and was even set as the default boot option. Everything looks fine. The lxc container also starts without any issue.
❯ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 59.6G 0 disk ├─sda1 8:1 0 512M 0 part ├─sda2 8:2 0 50G 0 part └─sda3 8:3 0 9.1G 0 part sdb 8:16 0 465.8G 0 disk ├─sdb1 8:17 0 366.2M 0 part /boot/efi ├─sdb2 8:18 0 455G 0 part /mnt/btrfs └─sdb3 8:19 0 10.4G 0 part [SWAP] ❯ ls /mnt/btrfs @rootfs @rootfs_ro
- format another drive
- send all btrfs subvolumes
- chroot and reinstall grub