Linux

Bootloader Fix on NVMe Drive

Bootloader Fix on NVMe Drive

One of my dedicated servers on OVH didn’t get back online after a reboot, so I checked via KVM and found that it was stuck at GRUB 2 prompt. To solve the problem, I changed netboot to rescue mode from OVH control panel, and with the rescue mode SSH credentials emailed to me, performed the following tasks to fix it.

First, I checked the partition table:

[email protected]:~# fdisk -l

Device             Start       End   Sectors   Size Type
/dev/nvme0n1p1      2048   1048575   1046528   511M EFI System
/dev/nvme0n1p2   1048576   2095103   1046528   511M Linux filesystem
/dev/nvme0n1p3   2095104 104493055 102397952  48.8G Linux filesystem
/dev/nvme0n1p4 104493056 878036991 773543936 368.9G Linux filesystem
/dev/nvme0n1p5 878036992 879083519   1046528   511M Linux swap

Disk /dev/nvme1n1: 419.2 GiB, 450098159616 bytes, 879097968 sectors

So it means:

/dev/nvme0n1p1 is /boot/efi partition
/dev/nvme0n1p2 is /boot partition
/dev/nvme0n1p3 is / partition

Now it’s time to mount them accordingly:

[email protected]:~# mount /dev/nvme0n1p3 /mnt/
[email protected]:~# mount /dev/nvme0n1p2 /mnt/boot/
[email protected]:~# mount /dev/nvme0n1p1 /mnt/boot/efi
[email protected]:~# mount --bind /dev /mnt/dev
[email protected]:~# mount --bind /proc /mnt/proc
[email protected]:~# mount --bind /sys /mnt/sys

Once done, I changed root directory to mounted environment to fix the bootloader:

[email protected]:~# chroot /mnt
[[email protected] /]# grub2-install --efi-directory=/boot/efi
Installing for x86_64-efi platform.
Installation finished. No error reported.
[[email protected] /]# grub2-mkconfig -o /boot/grub2/grub.cfg
Generating grub configuration file ...
Found linux image: /boot/bzImage-4.9.78-xxxx-std-ipv6-64
Found initrd image: /initramfs-4.9.78-xxxx-std-ipv6-64.img
Found linux image: /boot/vmlinuz-0-rescue-1e98a4cc1263468b9f9a12e365316b95
Found initrd image: /boot/initramfs-0-rescue-1e98a4cc1263468b9f9a12e365316b95.img
done
[[email protected] /]# exit

After exiting chroot, I unmounted the partitions and rebooted the server:

[email protected]:~# umount /mnt/dev
[email protected]:~# umount /mnt/proc
[email protected]:~# umount /mnt/sys
[email protected]:~# umount /mnt/boot/efi
[email protected]:~# umount /mnt/boot
[email protected]:~# umount /mnt/
[email protected]:~# reboot

Broadcast message from [email protected] on pts/0 (Sun 2020-07-12 01:13:12 CEST):

The system is going down for reboot NOW!

Now that the bootloader has been recovered, I boot back to normal mode from hard disk instead of rescue mode, and in a couple of minutes the server is back online!

Leave a Reply