Configuring GRUB to boot Jussi's kernel - 6.6.50-jl+

andybob · September 21, 2024, 9:49pm

After installing Jussi’s kernel 6.6.50-jl+ (Noble Numbat) I found that I was still booting into 6.8.0-45-generic. I understand GRUB defaults to the highest kernel version number and Jussi’s kernel now has a lower version number than the generic Noble Numbat kernel.

I eventually solved this by configuring GRUB to boot into the last saved OS and then changing the saved default variable to Jussi’s kernel using the menu number. I set out the the commands below (mistakes, blind alleys and red herrings edited out to protect the guilty):

# FIrst, save a copy of your current GRUB file just in case ...
cp /etc/default/grub /etc/default/oldgrub

# Open grub for editing.
sudo nano /etc/default/grub

# Edit the GRUB_DEFAULT item and add a GRUB_SAVEDEFAULT item as follows:
GRUB_DEFAULT=saved
GRUB_SAVEDEFAULT=true
# Exit nano saving to grub.

# Update GRUB
sudo update-grub

# Check your GRUB menu items
grep menu /boot/grub/grub.cfg | awk -F"'" '{ print $2 }'

# Note the location of the item 'Ubuntu, with Linux 6.6.50-jl+'.
# On my system this was under the second main menu item being 'Advanced options for Ubuntu' 
# and was the third sub-menu item. Now the tricky part for young players is
# that GRUB starts counting menu items from 0. So the syntax to set the current
# saved default using menu items for me was "1>2" (where > means sub-menu).
sudo grub-set-default "1>2"

# Check that the current saved default variable has been correctly set.
cat /boot/grub/grubenv

# Reboot the system.
sudo reboot now

jussi_laako · September 21, 2024, 10:28pm

One way that also works that I’ve been using is to edit /etc/default/grub and have one line there:

GRUB_DEFAULT="1>2"

Then run “sudo update-grub”. And that should do it in most usual cases.

Yours is more fool proof though as you get to verify the menu index that should be used.

IgorSki · September 22, 2024, 9:04am

Wow, @andybob , well spotted! And thank you for sharing your script. I did not realise, but true - on the latest reboots my system was picking up the “generic” kernel version.

For those who are interested, I also found this knowlege-base entry discussing how to generally load different versions of kernel on your system:

Now, an interesting part…

I’m religiosly installing JL+ kernels. To be specific - I enjoy the sound of EC-super/light 512+fs modulators @1024. And with my latest set-ups I was able to achive this performance with latest JL+ (albeit without DAC Correction @1024)

However, it turns out that the magic performance (ASDM7ECv3 @1024 with DAC Correction and Room Correction) that I have just shared in another thread was actually happening under generic kernel version ! Which HQP Filter are you using? [2024] - #425 by IgorSki

Tried to play around and replicate the behaviour, yep - I can see that ASDM7ECv3 @1024 with DAC Correction is not possible with JL+ kernel, dropping out pretty much every 5-10 sec. Switching back to generic kernell - I have this performance back again.

Also, validating via iOS HQP client - all correct, I can see that DAC Correction is enabled and Matrix Profile is my simple FIR filters correcting bass response beneath 500Hz…

Bug or Feature?

jussi_laako · September 22, 2024, 10:22am

Stock Ubuntu kernel is 6.8 series short term kernel release. While mine is 6.6 series long term kernel release. See here for list of long term releases. When new kernel release comes out, the older short term releases are not maintained (updated) anymore. Latest kernel at the moment is 6.11, so 6.8 is already way out of maintenance, and thus Canonical needs to do all the maintenance work for their kernel, including backporting bug fixes. While the 6.6 is maintained until December 2026. Switching to a new kernel release can be fairly notable amount of work and may also need some updates/changes to other OS components. This is the reason I rather stick to the long term kernels. Debian is doing the same too. I switch to a new long term kernel when some kernel release is designated such “golden kernel”.

Now, the 6.8 may have some features that make things faster for you. Or the performance overhead of low latency kernel makes enough difference. If you use a NAA, the low latency behaviour is much less important than if you use local output. Because the primary buffer and responsiveness requirement is where the DAC output is. Low latency comes with CPU load overhead, realtime kernels even more so.

Then there are driver differences, for example DSD1024 output from stock Ubuntu kernel doesn’t work to my DACs. And to make USB input work on UP Gateway my custom kernel is also necessary, since stock Ubuntu kernel doesn’t include the drivers necessary to do USB input side.

Also some things don’t work correctly with the stock Ubuntu kernel, due to the way it is configured, but whether that matters for your case varies. (Including HQPlayer OS builds don’t work with the stock kernel due to the way it is configured). But you may be be perfectly happy with the stock kernel especially if you use NAA output, so no need to worry about it if you get better performance that way!

IgorSki · September 22, 2024, 11:11am

Brilliant, thank you for taking time to explain @jussi_laako !

It is a NAA set up, indeed. I will keep this unexpected artefact for now! The real beauty is that I can easily flip back and forth if I wish to get back to EC-super/light 512+fs @1024 ! This is for 13900KS, and BTW on this system @512x48 JL+ CAN DO EVERYTHING (I have tried so far…)

PS: in a meanwhile my 14900KS blues seems to soon come to a resolution, Intel is sending a replacement. Hopefully this time not chipped. Will see what that one can do…

andybob · September 23, 2024, 9:32am

Also an NAA setup.
Running on an i9-13900KF with an RTX 3060 GPU.
Using a Spring I (May arriving shortly) that maxes at DSD 512 atm. I’ve been flipping between kernels but haven’t noticed any combinations where the kernel makes the difference between running without dropouts or not. I’ll report here if I do.

nquery · September 23, 2024, 3:41pm

@IgorSki just to clarify, you are saying that you get slightly better performance with linux-image-6.8.0-40-generic vs JL+ 6.6.50 or linux-image-6.8.0-44-lowlatency vs JL+ 6.6.50? I assume it’s the former but just double-checking. I use a pretty standard NAA setup so will try the generic kernel to see if I get better “throughput”.

Chris_Shorter · September 24, 2024, 6:44am

I have had similar experiences at 512, and a more modest server, with the latest generic kernel vs standard low latency. With low latency, I could only use 7EC-light but can now also use 7ECV3 and 7EC-super with generic and DAC correction active. All the 512+fs modulators are still beyond reach. I was hoping I could introduce x48 but that still lies beyond reach.

Chris

(AMD Ryzen 9 7950X with 64GB but no GPU)

Chunhao_Lee · September 24, 2024, 8:57am

Interesting discussions. I’d love to share the boot parameters for my HQPe server. Although it’s for my 13ch DSD256 immersive / multichannel system but it still can do 2ch DSD512 upsampling.
Before the parameters, here’s the some screenshot when running:
192KHz source / sinc-MGa / ASDM7EC-super 512+fs → DSD512x48

With PEQ text and DAC corrections. Here’s the AMD 7950x CPU loading:

sinc-MGa took some space of my 4090:

Here’s the policies of my HQPe server:

Run realtime kernel
Disable power management (idle control is removed by using CONFIG_CPU_IDLE=n)
Disable hyperthreading (AMD term is SMT) and hyper boost (AMD term is CPB)
Disable all possible causes of OS jitter / latency (NUMA, IRQs, timer migration, watchdog… etc etc)
Trust TSC and set it as primary clock source

The 2nd and 3rd policies need to be disabled in BIOS. It’s easy and won’t hurt your system.

The kernel I’m using is local build, Jussi’s kernel 6.6.50-jl with PREEMPT_RT patch.
Image 2024-9-24 at 16.56

Here’s my boot parameters (very long):

GRUB_CMDLINE_LINUX_DEFAULT="kthread_cpus=0-2 irqaffinity=0-2 nohz=on nohz_full=3-15 isolcpus=managed_irq,domain,3-15 mitigations=off rcu_nocb_poll rcu_nocbs=3-15 skew_tick=1 clocksource=tsc tsc=reliable nohpet nmi_watchdog=0 nosoftlockup default_hugepagesz=2MB hugepagesz=1G hugepages=1 hugepagesz=2M hugepages=512 mce=off numa_balancing=disable amd_pstate=disable processor.max_cstate=0 cpufreq.off=1 cpuidle.off=1"

CPU 0~2 set for house keeping, Kthread and IRQs handling. CPU 3~15 set completely isolated and only for hqplayerd.

HQPlayer’s CPU masking I set:

HQPLAYER_RESERVED_CORES="0000000000000111"

All hqplayerd’s intensive tasks will go CPU 3~15.

Here’s some OS jitter / latency info from RTLA tool:
Average timer latency looks ok to me (<5us).

CPU 3~15 are completely isolated from OS noise for hqplayerd.

Also free from HW noise.

Such realtime kernel tuning really helps my immersive / MCH DSD256 system and not too shabby for high rate DSD 2ch.

– Edited –
Forgot to mention there’s two CLIs are mandatory for realtime kernel:

echo -1 > /proc/sys/kernel/sched_rt_runtime_us
echo 0 > /proc/sys/kernel/timer_migration

– Edited #2 –
Such tuning must turn off IRQ balance (affect after reboot):

systemctl disable irqbalance

IgorSki · September 27, 2024, 4:09am

Hi @nquery , in my case “generic” is slightly better than “JL+” for the specific use case, that is NAA architecture and 7ECv3 modulator SDM@1024 ( @1024x48 drops out)