Migrating a virtual machine to a new hostId

This document describes how to migrate the state of a micro VM while changing the hostId value.

This procedure is NOT required if you rename the virtual machine name.

This includes ZFS datasets and sshd state stored under /var/lib/microvms/$HOST_ID/sshd/.

Steps

1. Snapshot the VM state (strongly recommended)

   Ensure you can roll back in case of an error:

   zfs snapshot rpool/<old_id>/state@pre-migration
   zfs snapshot rpool/<old_id>/state-journal@pre-migration
   

2. Stop the VM

   systemctl stop microvm@$VM_NAME
   

3. Destroy auto-created datasets for the new host ID

   If they exist, ensure they are empty or do not contain any data, destroy the empty datasets to avoid conflicts:

   zfs destroy -r rpool/<new_id> || true
   

4. Rename the old host ID datasets to the new host ID

   zfs rename rpool/<old_id> rpool/<new_id>
   

5. Move the sshd state directory

   Ensure that the new host ID directory do not contain valuable private key material:

   # If the target directory already exist, move it and clean it up at the very end.
   mv /var/lib/microvms/<new_id> /var/lib/microvms/unused_<new_id>
   mkdir -p /var/lib/microvms/<new_id> && cp -r /var/lib/microvms/<old_id> /var/lib/microvms/<new_id>
   

6. Start the VM

   systemctl start microvm@$VM_NAME
   

7. Cleanup the pre-migration snapshots once data is confirmed

   zfs destroy rpool/<new_id>/state@pre-migration
   zfs destroy rpool/<new_id>/state-journal@pre-migration
   

8. Cleanup the remaining sshd state

   Ensure that you confirmed that the rebooted VM is using the intended sshd state.

   rm -r /var/lib/microvms/unused_<new_id>
   rm -r /var/lib/microvms/<old_id>
   

Example

# Old and new host IDs
OLD_ID=9080c1b6
NEW_ID=abcd1234

# Snapshot the old datasets.
zfs snapshot rpool/$OLD_ID/state@pre-migration
zfs snapshot rpool/$OLD_ID/state-journal@pre-migration

# Stop the VM
systemctl stop microvm@vm-example

# Destroy new host datasets (if pre-created)
# Confirm that the new ID dataset do not contain important data.
zfs destroy -r rpool/$NEW_ID || true

# Rename datasets
zfs rename rpool/$OLD_ID rpool/$NEW_ID

# Replace sshd state directory, confirm that the $NEW_ID directory do not
# contain important data.
mv /var/lib/microvms/$NEW_ID /var/lib/microvms/unused_$NEWID
cp -r /var/lib/microvms/$OLD_ID /var/lib/microvms/$NEW_ID

# Restart VM
systemctl start microvm@vm-example

# Cleanup the snapshots once the data is confirmed.
zfs destroy rpool/<new_id>/state@pre-migration
zfs destroy rpool/<new_id>/state-journal@pre-migration

# Cleanup sshd state.
rm -r /var/lib/microvms/unused_$NEWID
rm -r /var/lib/microvms/$OLD_ID

Restoring the old dataset

If something goes wrong during migration, you can roll back to the original state using the snapshots created earlier.

Steps

1. Stop the VM (if running)

   systemctl stop microvm@$VM_NAME
   

2. Rollback to the pre-migration snapshot

   zfs rollback rpool/<new_id>/state@pre-migration
   zfs rollback rpool/<new_id>/state-journal@pre-migration
   

3. Restore the sshd state directory

   For the old host ID, a copy still remains at the old ID address:

   mv /var/lib/microvms/<new_id> /var/lib/microvms/unused2_<new_id> # skip if you are sure you do not need the current sshd state.
   cp -r /var/lib/microvms/<old_id> /var/lib/microvms/<new_id>
   

   For the new host ID, a copy still remains at the unused ID address:

   mv /var/lib/microvms/<new_id> /var/lib/microvms/unused2_<new_id> # skip if you are sure you do not need the current sshd state.
   cp -r /var/lib/microvms/unused_<new_id> /var/lib/microvms/<new_id>
   

4. Start the VM

   systemctl start microvm@$VM_NAME