[elrepo] kernel-lt and microcode updates
Orion Poplawski
orion at nwra.com
Mon Jul 6 19:54:26 EDT 2020
On 7/1/20 1:38 PM, Phil Perry wrote:
> On 01/07/2020 19:18, John Pilkington wrote:
>> On 01/07/2020 18:01, Phil Perry wrote:
>>> On 01/07/2020 17:49, Orion Poplawski wrote:
>>>>>
>>>>> I just re-installed kernel-lt on my laptop:
>>>>>
>>>>> # grep -i microcode dmesg-3.10.0-1127.13.1.el7.x86_64
>>>>> [ 0.000000] microcode: microcode updated early to revision 0xd6,
>>>>> date = 2020-04-27
>>>>> [ 0.021352] SRBDS: Mitigation: Microcode
>>>>> [ 1.542686] microcode: sig=0x806ea, pf=0x80, revision=0xd6
>>>>> [ 1.542918] microcode: Microcode Update Driver: v2.01
>>>>> <tigran at aivazian.fsnet.co.uk>, Peter Oruba
>>>>>
>>>>> # grep -i microcode dmesg-4.4.228-2.el7.elrepo.x86_64
>>>>> [ 0.035312] SRBDS: Vulnerable: No microcode
>>>>> [ 0.992458] microcode: CPU0 sig=0x806ea, pf=0x80, revision=0xb4
>>>>> [ 0.992471] microcode: CPU1 sig=0x806ea, pf=0x80, revision=0xb4
>>>>> [ 0.992491] microcode: CPU2 sig=0x806ea, pf=0x80, revision=0xb4
>>>>> [ 0.992511] microcode: CPU3 sig=0x806ea, pf=0x80, revision=0xb4
>>>>> [ 0.992530] microcode: CPU4 sig=0x806ea, pf=0x80, revision=0xb4
>>>>> [ 0.992550] microcode: CPU5 sig=0x806ea, pf=0x80, revision=0xb4
>>>>> [ 0.992569] microcode: CPU6 sig=0x806ea, pf=0x80, revision=0xb4
>>>>> [ 0.992588] microcode: CPU7 sig=0x806ea, pf=0x80, revision=0xb4
>>>>> [ 0.992673] microcode: Microcode Update Driver: v2.01
>>>>> <tigran at aivazian.fsnet.co.uk>, Peter Oruba
>>>>>
>>>>> and /proc/cpuinfo indicates that microcode is not updated
>>>>>
>>>>> Orion
>>>>
>>>> turning on dracut logging it apprears that:
>>>>
>>>> I: microcode_ctl: kernel version "4.4.228-2.el7.elrepo.x86_64"
>>>> failed early load check for "intel", skipping
>>>>
>>>> I: *** Generating early-microcode cpio image contents ***
>>>> I: *** No early-microcode cpio image needed ***
>>>>
>>>> that check is done in /usr/libexec/microcode_ctl/check_caveats
>>>>
>>>> that is for early loading. not sure why the microcode.service also
>>>> seems to fail to load the firmware.
>>>>
>>>>
>>>
>>> Interesting. Any idea why early updating appears to work on the
>>> distro kernel, but not for kernel-lt? I wonder what check it is failing.
>>>
>>> I will try to run some more tests on my hardware to find out if it's
>>> specific to you/your hardware.
>>
>> For info:
>>
>> [john at HP_Box ~]$ uname -r
>> 4.4.228-2.el7.elrepo.x86_64
>> [john at HP_Box ~]$ dmesg | grep -i microcode
>> [ 0.008291] MDS: Vulnerable: Clear CPU buffers attempted, no microcode
>> [ 1.551836] microcode: CPU0 sig=0x1067a, pf=0x1, revision=0xa0e
>> [ 1.551843] microcode: CPU1 sig=0x1067a, pf=0x1, revision=0xa0e
>> [ 1.551888] microcode: Microcode Update Driver: v2.01
>> <tigran at aivazian.fsnet.co.uk>, Peter Oruba
>> [john at HP_Box ~]$
>>
>> John P
>>
>>
>
> And I see the same thing on my el8 box with kernel-ml. Can't even
> manually load the new firmware with:
>
> echo 1 > /sys/devices/system/cpu/microcode/reload
I think there is still something missing for this manual load to work.
It looks like new EL kernel installs end up creating:
/lib/firmware/$(uname -r)/intel-ucode/ with links to the firmware in the
microcode_ctl package. This isn't happening when installing
kernel-lt-4.4.229-1.el7.elrepo.x86_64. This is where the reload driver
looks for microcode.
However, if I re-install an EL kernel I end up with
/lib/firmware/$(uname -r) directories for *ALL* installed kernels
including kernel-lt.
>
> I did find the answer in /var/log/messages though.
>
> Jul 1 20:02:56 sm7047at DISCLAIMER[66335]: This kernel doesn't handle
> early microcode load properly (it tries to load
> Jul 1 20:02:56 sm7047at DISCLAIMER[66335]: microcode even in
> virtualised environment, which may lead to a panic on some
> Jul 1 20:02:56 sm7047at DISCLAIMER[66335]: hypervisors), thus the
> microcode files have not been added to the initramfs
> Jul 1 20:02:56 sm7047at DISCLAIMER[66335]: image. Please update your
> kernel to one of the following:
> Jul 1 20:02:56 sm7047at DISCLAIMER[66335]: RHEL 7.5:
> kernel-3.10.0-862.14.1 or newer;
> Jul 1 20:02:56 sm7047at DISCLAIMER[66335]: RHEL 7.4:
> kernel-3.10.0-693.38.1 or newer;
> Jul 1 20:02:56 sm7047at DISCLAIMER[66335]: RHEL 7.3:
> kernel-3.10.0-514.57.1 or newer;
> Jul 1 20:02:56 sm7047at DISCLAIMER[66335]: RHEL 7.2:
> kernel-3.10.0-327.73.1 or newer.
> Jul 1 20:02:56 sm7047at DISCLAIMER[66335]: Please refer to
> /usr/share/doc/microcode_ctl/caveats/intel_readme
> Jul 1 20:02:56 sm7047at DISCLAIMER[66335]: and
> /usr/share/doc/microcode_ctl/README.caveats for details.
> Jul 1 20:02:56 sm7047at DISCLAIMER[66335]: MDS-related microcode update
> for Intel Sandy Bridge-EP (family 6, model 45,
> Jul 1 20:02:56 sm7047at DISCLAIMER[66335]: stepping 7; CPUID 0x206d7)
> CPUs is disabled.
> Jul 1 20:02:56 sm7047at DISCLAIMER[66335]: Please refer to
> /usr/share/doc/microcode_ctl/caveats/06-2d-07_readme
> Jul 1 20:02:56 sm7047at DISCLAIMER[66335]: and
> /usr/share/doc/microcode_ctl/README.caveats for details.
> Jul 1 20:02:56 sm7047at journal: This kernel doesn't handle early
> microcode load properly (it tries to load#012microcode even in
> virtualised environment, which may lead to a panic on
> some#012hypervisors), thus the microcode files have not been added to
> the initramfs#012image. Please update your kernel to one of the
> following:#012 RHEL 7.5: kernel-3.10.0-862.14.1 or newer;#012 RHEL
> 7.4: kernel-3.10.0-693.38.1 or newer;#012 RHEL 7.3:
> kernel-3.10.0-514.57.1 or newer;#012 RHEL 7.2: kernel-3.10.0-327.73.1
> or newer.#012Please refer to
> /usr/share/doc/microcode_ctl/caveats/intel_readme#012and
> /usr/share/doc/microcode_ctl/README.caveats for details.
> Jul 1 20:02:56 sm7047at journal: MDS-related microcode update for Intel
> Sandy Bridge-EP (family 6, model 45,#012stepping 7; CPUID 0x206d7) CPUs
> is disabled.#012Please refer to
> /usr/share/doc/microcode_ctl/caveats/06-2d-07_readme#012and
> /usr/share/doc/microcode_ctl/README.caveats for details.
>
> Reinstalling microcode_ctl will generate those messages for you.
>
> Reading the docs (/usr/share/doc/microcode_ctl/caveats/intel_readme)
> tells us how to force the firmware update:
>
> If you want to enforce early load of microcode for a specific kernel,
> please
> create "force-early-intel" file inside /lib/firmware/<kernel_version>
> directory
> and run dracut -f --kver "<kernel_version>":
>
> touch /lib/firmware/3.10.0-862.9.1/force-early-intel
> dracut -f --kver 3.10.0-862.9.1
>
> After performing the above (you will need to append the .arch) I see the
> microcode firmware is now included in the initramfs:
>
> # lsinitrd -k $(uname -r) |grep -i microcode
> drwxr-xr-x 2 root root 0 Feb 28 13:30 kernel/x86/microcode
> -rw-r--r-- 1 root root 19456 Feb 28 13:30
> kernel/x86/microcode/GenuineIntel.bin
> microcode_ctl-fw_dir_override
>
> and the microcode is loaded upon reboot:
>
> [root at sm7047at ~]# dmesg | grep -i microcode
> [ 0.000000] microcode: microcode updated early to revision 0x714,
> date = 2018-05-08
> [ 0.103355] MDS: Vulnerable: Clear CPU buffers attempted, no microcode
> [ 1.724141] microcode: sig=0x206d7, pf=0x1, revision=0x714
> [ 1.724498] microcode: Microcode Update Driver: v2.2.
> [ 7.479059] microcode: updated to revision 0x71a, date = 2020-03-24
> [ 7.479151] x86/CPU: CPU features have changed after loading
> microcode, but might not take effect.
> [ 7.479153] microcode: Reload completed, microcode revision: 0x71a
>
>
> So for kernel-lt and kernel-ml packages, please read the documentation
> and either force loading on a kernel-by-kernel basis, or globally
> depending upon your preference.
>
> Phil
Thanks for that - very informative. Of course the fun part of all this
is that the commit that prevents the early loading of firmware on
virtualized hosts (a15a753539eca8ba243d576f02e7ca9c4b7d7042) is present
in 4.10-rc1 on. So not safe for the current -lt kernel, but should be
okay for kernel-ml. The caveats system seems like it should detect
kernels >= 4.10.0 as safe though.
Orion
--
Orion Poplawski
Manager of NWRA Technical Systems 720-772-5637
NWRA, Boulder/CoRA Office FAX: 303-415-9702
3380 Mitchell Lane orion at nwra.com
Boulder, CO 80301 https://www.nwra.com/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3799 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.elrepo.org/pipermail/elrepo/attachments/20200706/79327ea2/attachment.p7s>
More information about the elrepo
mailing list