Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

[Solved] New kernel not booting after upgrade from Bullseye to Bookworm

Linux Kernel, Network, and Services configuration.
Post Reply
Message
Author
User avatar
Praxis
Posts: 102
Joined: 2006-12-19 00:00
Has thanked: 3 times
Been thanked: 1 time

[Solved] New kernel not booting after upgrade from Bullseye to Bookworm

#1 Post by Praxis »

Most of my systems are based on LUKS-encrypted LVM, using the automatic partitioning scheme that was in the installer from the Jessie_ - Official Snapshot amd64 kde-CD Binary-1 20141883. and I've updated them every time there was a new stable release. Four upgrades went smoothly without any significant issue, but on my primary local file server the new Bookworm linux-image-6.1.0-9-amd64 kernel drops me into busybox, though my last Bullseye kernel (linux-image-5.10.0-23-amd64) boots normally to Bookworm.
bootissue.jpg
Sorry, I have two monitors, vertical and horizontal, and only the vertical one lights up until later in the boot process, so the picture doesn't capture too much of the dialog, but from what I can glean the new kernel is not properly initiating my LVM which is at /dev/mapper/hostname--vg-root.

The screen message says:

Code: Select all

Gave up waiting for root file system device. Common problems:
- Boot args (cat /proc/cmdline)
-Check rootdelay= (did the system wait long enough>)
-Missing modules (cat /proc/modules; ls /dev)
I believe the syslog had something like this:

Code: Select all

Begin: Mounting root file system ... Begin: Running /scripts/local-top ...   Volume group "liva-vg" not found
  Cannot process volume group liva-vg
  Volume group "liva-vg" not found
  Cannot process volume group liva-vg
Couldn't find device with uuid Eyx2dv-R4Qo-6oEL-UyVf-vY87-BSJ0-CrcGZi.
  Couldn't find device with uuid Eyx2dv-R4Qo-6oEL-UyVf-vY87-BSJ0-CrcGZi.
  Refusing activation of partial LV liva-vg/freespace.  Use '--activationmode partial' to override.
See 'systemctl status lvm2-pvscan@8:67.service' for details.
My booted system (with the Bullseye kernel) does not show anything when catting that line.
Unit lvm2-pvscan@8:67.servic.service could not be found.

I also got a line like this:
2023-06-13T16:49:55.822032-07:00 ubu udisksd[869]: Failed to load the 'mdraid' libblockdev plugin

The /var/log/bootlog file has a lot of entries like this:

0;1;31mFAILED[0m] Failed to start [0;1;39mLVM event activation on device 254:0

Can anyone belt me with a clue stick?

Edit: I booted back to the new kernel and played around with busybox, but didn't see any way to see the log output and save it to a file, particularly cranking my neck 90 degrees to read off of my vertical monitor. Nothing in kern.log or kern.log.1 caught my eye in the fully booted system using the Bullseye kernel. Ditto dmesg. Time to walk the pooch I'm dog sitting for a neighbor before I get mauled, but I'll check in later.
Last edited by Praxis on 2023-06-17 02:29, edited 1 time in total.
Top
vng2022
Posts: 48
Joined: 2023-06-10 10:33
Has thanked: 1 time
Been thanked: 4 times

Re: New kernel not booting after upgrade from Bullseye to Bookworm

#2 Post by vng2022 »

You can boot with a previous kernel, so run

Code: Select all

# update-initramfs -u -k all
Top
User avatar
Praxis
Posts: 102
Joined: 2006-12-19 00:00
Has thanked: 3 times
Been thanked: 1 time

Re: New kernel not booting after upgrade from Bullseye to Bookworm

#3 Post by Praxis »

That was an excellent suggestion, vng2022. I even thought of it myself after I posted. However after I tried it now both kernels dump me into busybox. Scheis..., er, drat!

I'm going to try and chroot into the system, and maybe reinstall mdadm.
Top
User avatar
Praxis
Posts: 102
Joined: 2006-12-19 00:00
Has thanked: 3 times
Been thanked: 1 time

Re: New kernel not booting after upgrade from Bullseye to Bookworm

#4 Post by Praxis »

OK, I guess I figured this out.

I booted a live Ubuntu install I have on an USB SSD on the machine in question & decrypted the Debian / partition.

cryptsetup luksOpen UUID=3624536 sda5_crypt

But when I did a "ls" of /dev/mapper the Debian LVM root partition was not there. So I did a 'vgscan' and got:

Code: Select all

vgscan

  WARNING: Couldn't find device with uuid Eyx2dv-R4Qo-6oEL-UyVf-vY87-BSJ0-CrcGZi.
  WARNING: VG liva-vg is missing PV Eyx2dv-R4Qo-6oEL-UyVf-vY87-BSJ0-CrcGZi (last written to /dev/mapper/freespace).
  Found volume group "liva-vg" using metadata type lvm2
  Found volume group "vgkubuntu" using metadata type lvm2

  vgchange -ay

  WARNING: Couldn't find device with uuid Eyx2dv-R4Qo-6oEL-UyVf-vY87-BSJ0-CrcGZi.
  WARNING: VG liva-vg is missing PV Eyx2dv-R4Qo-6oEL-UyVf-vY87-BSJ0-CrcGZi (last written to /dev/mapper/freespace).
  Refusing activation of partial LV liva-vg/freespace.  Use '--activationmode partial' to override.
  2 logical volume(s) in volume group "liva-vg" now active
  2 logical volume(s) in volume group "vgkubuntu" now active
So I don't know why the upgrade is having trouble finding the freespace VG.

I also did a 'lvscan' which gave me:

Code: Select all

  WARNING: Couldn't find device with uuid Eyx2dv-R4Qo-6oEL-UyVf-vY87-BSJ0-CrcGZi.
  WARNING: VG liva-vg is missing PV Eyx2dv-R4Qo-6oEL-UyVf-vY87-BSJ0-CrcGZi (last written to /dev/mapper/freespace).
    ACTIVE            '/dev/liva-vg/root' [216.35 GiB] inherit
  ACTIVE            '/dev/liva-vg/swap_1' [<6.98 GiB] inherit
  inactive          '/dev/liva-vg/freespace' [9.25 GiB] inherit
I mounted the now active Debian root partition & the /boot partitions & chrooted to my Debian installation.

Code: Select all

    mount /dev/mapper/liva--vg-root /mnt/dev

    mount LABEL=booty /mnt/dev/boot

    for i in /dev /dev/pts /proc /sys /run; do sudo mount -B $i /mnt/dev$i; done

    sudo chroot /mnt/dev
I also unlocked the LUKS encrypted partition that has 'freespace':

cryptsetup luksOpen UUID=b379bcf9-25d7-4d23-be5d-688758e3e91a freespace

In any case, reinstalling lvm2 & mdadm & running an additional "update-initramfs -u -k all" continued to give me errors related to the "freespace" LVM member (even though it had worked before the upgrade and the encrypted 'freespace' LVM partition was being properly decrypted by my /etc/crypttab).

So I just removed the 'freespace' LV, & PV and rebooted to the new Bookworm kernel.

Code: Select all

vgdisplay -v

lvremove /dev/liva-vg/freespace
  WARNING: Couldn't find device with uuid Eyx2dv-R4Qo-6oEL-UyVf-vY87-BSJ0-CrcGZi.
  WARNING: VG liva-vg is missing PV Eyx2dv-R4Qo-6oEL-UyVf-vY87-BSJ0-CrcGZi (last written to /dev/mapper/freespace).
  WARNING: Couldn't find device with uuid Eyx2dv-R4Qo-6oEL-UyVf-vY87-BSJ0-CrcGZi.
  WARNING: Couldn't find device with uuid Eyx2dv-R4Qo-6oEL-UyVf-vY87-BSJ0-CrcGZi.
  Logical volume "freespace" successfully removed.

vgreduce  liva-vg /dev/mapper/freespace
  Removed "/dev/mapper/freespace" from volume group "liva-vg"
Top
Post Reply

Return to “System and Network configuration”