Matthew Garrett ([personal profile] mjg59) wrote2013-02-08 22:41
Entry tags:

Samsung laptop bug is not Linux specific

I bricked a Samsung laptop today. Unlike most of the reported cases of Samsung laptops refusing to boot, I never booted Linux on it - all experimentation was performed under Windows. It seems that the bug we've been seeing is simultaneously simpler in some ways and more complicated in others than we'd previously realised.

So, some background. The original belief was that the samsung-laptop driver was doing something that caused the system to stop working. This driver was coded to a Samsung specification in order to support certain laptop features that weren't accessible via any standardised mechanism. It works by searching a specific area of memory for a Samsung-specific signature. If it finds it, it follows a pointer to a table that contains various magic values that need to be written in order to trigger some system management code that actually performs the requested change. This is unusual in this day and age, but not unique. The problem is that the magic signature is still present on UEFI systems, but attempting to use the data contained in the table causes problems.

We're not quite sure what those problems are yet. Originally we assumed that the magic values we wrote were causing the problem, so the samsung-laptop driver was patched to disable it on UEFI systems. Unfortunately, this doesn't actually fix the problem - it just avoids the easiest way of triggering it. It turns out that it wasn't the writes that caused the problem, it was what happened next. Performing the writes triggered a hardware error of some description. The Linux kernel caught and logged this. In the old days, people would often never see these logs - the system would then be frozen and it would be impossible to access the hard drive, so they never got written to disk. There's code in the kernel to make this easier on UEFI systems. Whenever a severe error is encountered, the kernel copies recent messages to the UEFI variable storage space. They're then available to userspace after a reboot, allowing more accurate diagnostics of what caused the crash.

That crash dump takes about 10K of UEFI storage space. Microsoft require that Windows 8 systems have at least 64K of storage space available. We only keep one crash dump - if the system crashes again it'll simply overwrite the existing one rather than creating another. This is all completely compatible with the UEFI specification, and Apple actually do something very similar on their hardware. Unfortunately, it turns out that some Samsung laptops will fail to boot if too much of the variable storage space is used. We don't know what "too much" is yet, but writing a bunch of variables from Windows is enough to trigger it. I put some sample code here - it writes out 36 variables each containing a kilobyte of random data. I ran this as an administrator under Windows and then rebooted the system. It never came back.

This is pretty obviously a firmware bug. Writing UEFI variables is expressly permitted by the specification, and there should never be a situation in which an OS can fill the variable store in such a way that the firmware refuses to boot the system. We've seen similar bugs in Intel's reference code in the past, but they were all fixed early last year. For now the safest thing to do is not to use UEFI on any Samsung laptops. Unfortunately, if you're using Windows, that'll require you to reinstall it from scratch.

[personal profile] idupree 2013-02-09 06:00 (UTC)(link)
Does replacing the laptop's hard disk allow it to boot? (I assume you tested and it doesn't help— I'm just thinking of my old MacBook that was never bricked, but had difficulty booting with certain bit patterns on the internal disk.)

(Anonymous) 2013-02-09 08:56 (UTC)(link)
No, this breaks the firmware, and replacing or removing the disk wouldn't help.

UEFI data in NAND Flash on motherboard

(Anonymous) 2013-02-09 09:32 (UTC)(link)
The storage space in question is in a NAND flash chip on the motherboard. Swapping out a drive changes nothing.

CMOS Battery?

(Anonymous) 2013-02-09 11:11 (UTC)(link)
Hi Matt!

Great writeup as always. Can you comment on the rumors going around that removing the CMOS NVRAM battery will make the board bootable again? Obviously that would mean taking apart the laptop to get at the motherboard, which voids the warranty. But for testing the fix, a developer might be willing to give up their warranty to iterate faster.

https://bugs.launchpad.net/ubuntu-cdimage/+bug/1040557/comments/23 on https://bugs.launchpad.net/ubuntu-cdimage/+bug/1040557 is one example of this.

UEFI to BIOS

(Anonymous) 2013-02-09 11:56 (UTC)(link)
I'm on Linux. How do I do this without reinstalling?

Re: UEFI to BIOS

(Anonymous) - 2013-02-13 09:49 (UTC) - Expand

Re: UEFI to BIOS

(Anonymous) 2013-02-09 16:30 (UTC)(link)
Look into your UEFI for an option called "Compatibility Support Module" or "CSM".

Re: UEFI to BIOS

(Anonymous) - 2013-02-09 16:31 (UTC) - Expand

Re: UEFI to BIOS

(Anonymous) - 2014-01-10 03:18 (UTC) - Expand

Seriously, why?

(Anonymous) 2013-02-09 13:02 (UTC)(link)
Why are people buying Samsung when they could have Apple hardware? Do they not realize that (if you must) you can run Windows on a Mac?

Seriously. If it's price you are worried about, I'll gladly sell you a brick of wood for a VERY good price if you are willing to believe that it's an equivalent piece of hardware and the only difference is price. You get what you pay for, people.

OK was that predictable enough? Well I'm sorry about that. But come on, when are people going to learn?

Re: Seriously, why?

(Anonymous) 2013-02-09 13:44 (UTC)(link)
I'm actually running linux (Mageia2 to be specific) on a macbook pro. I buyed it to try triple boot, but soon discovered that windows runs better virtualized, and osx is not worth the pain on free software.

Re: Seriously, why?

(Anonymous) - 2013-02-09 13:50 (UTC) - Expand

Re: Seriously, why?

(Anonymous) - 2013-02-09 15:24 (UTC) - Expand

Re: Seriously, why?

(Anonymous) - 2013-02-09 22:26 (UTC) - Expand

Re: Seriously, why?

(Anonymous) - 2013-02-10 00:12 (UTC) - Expand

Re: Seriously, why?

(Anonymous) - 2013-02-09 21:57 (UTC) - Expand

Re: Seriously, why?

(Anonymous) - 2013-02-09 22:29 (UTC) - Expand

Re: Seriously, why?

(Anonymous) - 2013-02-09 23:59 (UTC) - Expand

Re: Seriously, why?

(Anonymous) - 2013-02-12 20:05 (UTC) - Expand

Re: Seriously, why?

(Anonymous) - 2013-02-18 06:49 (UTC) - Expand

(Anonymous) 2013-02-09 15:16 (UTC)(link)
This doesn't make the bug any less disastrous, but at least story titles will now shift from "Samsung laptops bricked by Linux" to "Samsung laptops bricked by buggy UEFI".

Regarding Windows, I recently discovered that you can actually migrate a Windows 8 install from MBR+BIOS to GPT+UEFI. It's not straightforward, but it's possible (using bootrec and bcdboot). I haven't tried the other way around or with Windows 7, but I think that should be doable as well.

(Anonymous) 2013-02-09 15:28 (UTC)(link)
You can use Paragon's Migrate to UEFI to shift a Windows install from MBR+BIOS to GPT+UEFI (doesn't officially support Windows 8, and Windows 8 will block its launcher from running, but you can just run explauncher.exe it installs as Administrator, and it'll work - tested on my own install).

That doesn´t change anything

(Anonymous) - 2013-02-10 23:54 (UTC) - Expand

Fix for windows 8

(Anonymous) - 2013-05-23 03:11 (UTC) - Expand

Do you know *what* is broken?

(Anonymous) 2013-02-09 17:38 (UTC)(link)
Matthew, I had a similar issue some time ago regarding notebooks that got "bricked" by a bug in the proprietary nVidia driver. The issue that time was that the driver somehow managed to overwrite the EDID data of the display with garbage. So the driver failed to load and on the next start the BIOS failed to load as well because the video BIOS couldn't identify the display and stopped at that problem. So it would be interesting to know what really gets broken when that bug occurs. Does Samsung know what is dead and how to fix it yet? In my case the bug could be fixed by removing the display and booting with an external monitor. Then reconnect the display and flash the correct data back to the display. Dunno if something like that would help here. Samsung should probably know. The question is if they tell you?

QueryVariableInfo()?

(Anonymous) 2013-02-09 21:51 (UTC)(link)
Hi Matt, is there any way to run RuntimeServices->QueryVariableInfo() in Windows to at least be safer with this? I tried looking through the Windows docs and found nothing.

Re: QueryVariableInfo()?

(Anonymous) - 2013-02-10 07:36 (UTC) - Expand

geez what a surprise........

(Anonymous) 2013-02-10 04:12 (UTC)(link)
NOT

How to prevent bricking laptop?

(Anonymous) 2013-02-10 07:53 (UTC)(link)
I have a Samsung laptop Serie9 which is vulnerable to this bug I believe (NP900X3C-A05NL). I have not bricked my laptop yet.

Is there any setting to prevent running into this bug accidentally?

I precise that I run Linux (xubuntu-12.10) exclusively (Windows8 has been wiped out) and UEFI is disabled in BIOS.

Re: How to prevent bricking laptop?

(Anonymous) 2013-02-12 14:51 (UTC)(link)
According to what Matthew wrote, you should be more or less fine with UEFI disabled in BIOS.

I would do the same, i.e. wipe Windows if I didn't need it sometimes. But I guess there's no way to install Ubuntu using CM mode without removing Windows partitions first that occupy the beginning of the hard drive as it uses the new GTP partition table.

36?

(Anonymous) 2013-02-10 14:18 (UTC)(link)
Hi Matt,
thanks for the insights! One small and insignificant question: how do you figure that your code writes 36 variables? The maximum of the loop is 48 and I don't see how you could notice from the error code that it stopped in iteration 36.

Have a great day!

Bios mode may also have problems

(Anonymous) 2013-02-10 16:37 (UTC)(link)
I bought my samsung NP530U3B last July and never booted it in UEFI mode, only Bios mode. Installed Linux next to the pre-installed Windows 7 and after 3 days tun into a problem. I don't know what caused it, could be i was installing updates in Windows, or that is was cause by Linux (opensuse), but suddenly it would not poweroff anymore nor hibernate/sleep. Sent it for repair by samsung and they replaced the motherboard. SO I suspect something in their firmware that may be triggered either in UEFI or Bios mode can damage this laptop. Since then it has been working ok

(Anonymous) 2013-02-10 21:54 (UTC)(link)
I do not think you bricked it. You might have created a kernel panic that caused the system to halt. Nevertheless, if you can read the logs fully it will point to the sector that caused the issue in most cases you need the source code to remove the abnormality from becoming prevalent in major system releases.

Worst case scenario take legal action against the creator.


Can the system boot or show a bios splash screen upon powering it on?

My Samsung Chronos returns to life

(Anonymous) - 2013-02-14 09:31 (UTC) - Expand

Way to recover

(Anonymous) 2013-02-11 08:21 (UTC)(link)
Did you look at the board if it has the firmware in a socket? If not you may be able to find a connector for SPI. I don't own such da device but there is a list:

http://flashrom.org/FT2232SPI_Programmer

If you need to send it back to Samsung for each test you want to run it would take most likely a bit long.

How does one recover from this?

(Anonymous) 2013-02-24 17:19 (UTC)(link)
I have a Samsung Chronos that just randomly died recently. I had destroyed my Fedora 17 install for the second time, but that's not relevant. I got a Fedora 18 disc to install from. Then, as I was installing it, it yelled about a hardware issue, and went into the world-famous kernel panic. I assumed it was a fluke, and it rebooted again. The same thing happened, and it never came back after this. After it died, the CD was stuck in the drive, so I figured that was the problem, but I'm guessing this is what caused it. I surgically removed the CD, and nothing happened. It doesn't show boot or anything, and I'm unable to use my laptop. Oh, well. At least I now know not to pay for this again. I had some pretty interesting programming on the drive, and I was hoping to recover it, so has anyone found any way to recover from this apart from telling Samsung to fix it? I think I might have voided my warranty by messing with the hardware.

Re: How does one recover from this?

(Anonymous) 2013-06-21 03:13 (UTC)(link)
Pop the hard disk into an external drive enclosure, plug that into a different machine, and read it there. A USB drive bay shouldn't cost you more than about 20 bucks. Easy peasy.

[identity profile] hanishkvc [launchpad.net] 2013-02-26 18:25 (UTC)(link)
I have a query related to using a linux distro on a Samsung laptop with this UEFI bug, while still using Secure boot/UEFI for Linux.

Should the proper solution to using Linux in UEFI/Secure boot mode in such a samsung laptop be to use a Linux kernel with the below Bug fix i.e

commit 266c43c175a51002b04c18a453a39708d1775ced
Author: Satoru Takeuchi
Date: Thu Feb 14 09:12:52 2013 +0900

efi: Clear EFI_RUNTIME_SERVICES rather than EFI_BOOT by "noefi" boot parameter

And in turn pass the noefi boot param to the kernel while booting into Linux.

The reason for my above assumption being that passing noefi to linux kernel as a boot param will I assume disable the use of efi runtime services by the kernel and its modules. And thus in no circumstance (including kernel crash) the Linux kernel will use the efi runtime service to write to the efi storage (I am also assuming that it will not allow any other logic to use efi service, by telling efi that it is relinquishing use of efi runtime service for this instance of the boot) And that is the 100% sure way of ensuring that under linux one cann't trigger this bug in the normal sense (Still is it 100% safe from a security perspective I am not sure if Samsung efi logic doesn't have any loop holes which allows one to call efi services even if one has already relinquished it - I am talking logically here, because I haven't looked into efi in detail so am making some/many assumptions).

So if one wants to dual boot a system with win8 already installed in Secure boot UEFI mode and Linux (in Secureboot/UEFI mode) THEN one should use a distro of linux which is using linux kernels later than Feb 15 with the above mentioned noefi bug fix included and in turn one should boot such a linux distro with noefi boot param to ensure that the Samsung laptops with this efi bug cann't be triggered from Linux during that boot.

Is my above understanding correct.

NOTE: I am not sure the linux kernel handles the transition from efi to no efi runtime mode gracefully if noefi is passed as a argument and the system is already in uefi boot mode. But I am assuming for now that the kernel handles this situation properly as well as that it is required to handle this in a specific manner, which it does. This is my assumption currently because I haven't looked into EFI specs at any level currently.

NOTE: A related query I have posted in the ubuntu launchpad tracked Bug related to this.

Also does anyone know when Samsung will release a fixed efi firmware.

測試

(Anonymous) 2013-02-27 15:11 (UTC)(link)
測試

New Samsung BIOS ver P05ABK

(Anonymous) 2013-03-01 21:49 (UTC)(link)
I was just running Samsung Update on my 900X4C, when it shows me that there is a BIOS update. The new version is P05ABK, i followed the steps (straight forward) and instaled it.
I just cant find release notes regarding the update in samsung site, and this means that i stil dont have the courage to install linux in this machine.

Is there any chance taht this firmware upgrade could correct the error reported?

Re: New Samsung BIOS ver P05ABK

(Anonymous) 2013-03-03 10:22 (UTC)(link)
There is always a chance but I don't trust software that is secretive about what it does. With something as critical as a BIOS you don't want to take chances. I can't even recommend installing the update (as any good systems administrator reviews updates before he applies them).

I just hope they fix things before some scriptkiddy with a grudge on Samsung starts exploiting it.

(Anonymous) 2013-03-03 12:30 (UTC)(link)
After the installation of the new P05ABK Bios update, my system/bios detects only 4 of my 8gb of ram. So I would not recommend to update the BIOS on a 900X4C.

(Anonymous) 2013-03-09 06:22 (UTC)(link)
Wow. Bricked laptop, fauly bios updates... Samsung really knows how to make customers happy.

Thinkoad Bricked... related?

(Anonymous) 2013-03-13 03:31 (UTC)(link)
This might be totally unrelated and coincidental, but a couple days ago my thinkpad was bricked.

I have an Edge E430 (3254-CTO). I was booting Arch on kernel 3.8.2 (CK patchset) when it stalled. This wasn't unusual as there has been major changes in LVM2 with Arch that I have been still trying to figure out. So I manually scanned for the PV when I had a kernel panic.

The kernel panic was really funky looking with very little info having been dumped (maybe 7-10 short jumbled lines). Though this may have been because I was in the initrd still.

After that I was not able to boot whatsoever. No POST or anything. The fan spins like it is going to do something, but nothing after. I even put the optical drive back in so that i could see if it saw that. It spins like it always does when power is applied, but nothing else.

Since you are the only person who seems to be an authority on this particular problem, I am trying to contact you.

Hopefully I will get a reply from you here. I will check periodically. My computer has been sent for repair, but I should contact you at the very least.

Re: Thinkoad Bricked... related?

(Anonymous) 2013-03-13 03:32 (UTC)(link)
Wow auto-correct on my Nexus 7 really sucks sometimes.

Re: Thinkoad Bricked... related?

(Anonymous) - 2013-03-13 14:39 (UTC) - Expand

CSM Ubuntu

(Anonymous) 2013-04-18 23:46 (UTC)(link)
I have a np530u4e-s20de and i need Ubuntu with ROS so my quastion is.

Can i install Ubuntu on it in the CSM Mode - i will format the whole system- . I hope you can help me. Tanks alot.

Greetings

Re: CSM Ubuntu

(Anonymous) 2013-04-26 11:31 (UTC)(link)
Hello Dude,

i write from ths machine ... Yeah you can, but is a little bit difficult. You must formate the whole Plate. Patch the Bootloader (see wiki), create /boot on SDA (HDD)with EXT2 format.

The rest in EXt4 and enjoy Ubuntu. The only problem is, that the linux kernel does not use your ATI graphic Card ... the hardwareID is not in the current Kernel. But the primary Intel is good enough for work and midlevel simulations.

Greetings