mjg59 | Playing with Thunderbolt under Linux on Apple hardware

I've temporarily got hold of a Retina Macbook Pro. Fedora 17 boots fine on it with the following kernel arguments: nointremap drm_kms_helper.poll=0 video=eDP-1:2880x1800@45e, although you'll want an updated install image. The nointremap requirement is fixed by this patch, and the others are being tracked here. The gmux chip that controls the multiple graphics chipsets has changed - patches here. That just leaves Thunderbolt.

Thunderbolt is, to a first approximation, PCIe tunnelled over a Displayport connector. It's a high-bandwidth protocol for connecting external devices. Things are complicated by it also supporting Displayport over the same connection, which I'm sure will be hilarious when we get yet another incompatible display protocol. But anyway. I picked up a Thunderbolt ethernet adapter and started poking.

Booting with the device connected looked promising - an ethernet device appeared. Pulling it out resulted in the pcie hotplug driver noticing that it was missing and cleaning up, although the b44 ethernet driver sat around looking puzzled for a while. So far so good. Unfortunately plugging it in again didn't work, and rebooting without any Thunderbolt devices connected not only did nothing on hotplug, it also meant that the Thunderbolt controller didn't show up in lspci at all.

It turns out that there's several problems at play here. The first is the missing Thunderbolt controller. The problem here is the following code in Apple's ACPI tables:

        Method (RMCR, 0, Serialized)
        {
            If (LNot (OSDW))
            {
                If (LAnd (LEqual (\_SB.PCI0.PEG1.UPSB.DSB1.UPS0.AVND, 0xFFFF), 
                    LEqual (\_SB.PCI0.PEG1.UPSB.DSB2.UPS0.AVND, 0xFFFF)))
                {
                    Store ("RMCR: Disable Link and Power Off Cactus Ridge Chip",
                           Debug)
                    Store (0x01, \_SB.PCI0.PEG1.LDIS)
                    Sleep (0x07D0)
                    Store (0x00, GP23)
                }
            }
        }

This checks if the system claims to be Darwin (the core of OS X). If not, it checks whether two PCIe bridges have been configured. If both are still in an unconfigured state, it cuts the PCIe link to the upstream PCIe link and then sets GP23 to 0. Looking further, GP23 turns out to be a GPIO line. Setting it to 0 appears to cut power to the Thunderbolt controller. In summary, unless your OS claims to be OS X, booting without an attached Thunderbolt device will result in the chipset being powered down so hard that it's entirely invisible to the OS.

Fixing that is as easy as adding Darwin to the list of operating systems Linux claims to be. So, now I could boot without a connected device and still see the controller. Since ACPI seemed to be important here, I started looking at the other ACPI methods associated with the device. The first one that drew attention was Name(_GPE, 0x14). The _GPE method (not to be confused with the _GPE object at the top of the global ACPI namespace) is defined as providing the ACPI general purpose event associated with the device. Unfortunately, it's only defined as doing this for embedded controllers - Apple's use of it on a PCI device is massively non-standard. But, still, easy enough to handle. Intel chipsets map GPE 0x10 to 0x1f to GPIO lines 0-15, so this GPE is associated with GPIO 4. Looking through the ACPI code showed a bunch of other references to GP04, and it turns out that if the chip is powered down (via setting GP23 to 0) then plugging a device in to the port will generate a GPE. This makes sense - a powered down controller isn't going to notice hotplug events itself, so you need some kind of out of band notification mechanism. There's also another ACPI method that flips the polarity of the GPIO line so it doesn't keep generating events for the entire time a device is connected. It didn't take long to come up with a stub driver that would power down the controller if nothing was connected at boot, and then wake it up again on a hotplug event.

Unfortunately that's as far as I've got. I'd been hoping that everything beyond this was just PCIe hotplug, but it seems not. Apple's Thunderbolt driver is rather too large to just be handling ACPI events, and this document on Apple's website makes it pretty clear that there's OS involvement in events and device configuration. Booting with a device connected means that the firmware does the setup, but if you want to support hotplug then the OS needs to know how to do that - and Linux doesn't.

Getting this far involved rather a lot of irritation at Apple for conspiring to do things in a range of non-standard ways, but it turns out that the real villains of the piece are Intel. The Thunderbolt controller in the Apples is an Intel part - the 82524EF, according to Apple. Given Intel's enthusiasm for Linux and their generally high levels of engagement with the Linux development community, it's disappointing[1] to discover that this controller has been shipping for over a year with (a) no Linux driver and (b) no documentation that would let anyone write such a driver. It's not even mentioned on Intel's website. So, thanks Intel. You're awful.

Anyway. This was my attempt to spend a few days doing something more relaxing than secure boot, and all I ended up with was eczema and liver pain. Lesson learned, hardware vendors hate you even more than firmware vendors do.

[1] By which I mean grotesquely infuriating

Flat | Top-Level Comments Only

It has always been confusing to me as to why getting access to information about hardware (and I guess firmware too) is so hard. Surely this information exists, otherwise the engineering people behind the scenes would be working in the dark all the time. It is really that hard to make it accessible? Or is this another one of those counter intuitive benefits "intellectual property" apparently brings to society?

You're thinking like an Open Source developer, by saying "Is there a good reason to keep this secret? If not, it should be shared.". Most tech companies don't think that way; to the extent they think about it at all, they think "Is there a good reason to open this? If not it should be secret.".

Not opening something means not having to review it. This saves time, which means money.

Reviews are obviously required: not just to make sure there is actually no confidential content, but also to make sure there is no copyright problem, no quality problem, etc. Big companies unfortunately cannot afford to "just release" stuff, especially not when their bank account is well in the black. Sorry.

Guess what: the "engineering people behind the scenes" _are_ working in the dark all the time. Speaking as one of them: the proprietary secret-sacred documentation only seldom better then none at all.

They have the support of the manufacturer, though (like e-mails and phones).

You're assuming that not only does the information exist, it exists in a nice usable form that the company could release as "Manual for Programming an 82524EF Controller".

In reality, the information is much more likely to be scattered throughout dozens of individual specification documents that can't be released - not out of malice or desire for control, but simply because nobody has spent the time to gather all that information into something usable by an outsider.

Many times the only documentation available is cornering one of the engineers and grilling them. I've seen cases where the official documentation is a few neatly typed pages, with handwritten corrections by several hands and no explanation whatsoever.

It is just as with our software: Many times there is (next to) no documentation on the inner workings available, developers find writing such boring (and it is suprisingly difficult to explain what you know inside-out to strangers that can't ask back). Sure, you can eventually figure out C code by staring at it long enough, while staring at your average device won't help; but the forces at work are much the same.

I'm one of the engineers that produces the documentation as an architecture spec to the designers. I try to write my documents to be a complete and accurate guide to my blocks. That said, to understand it fully, you need docs to other blocks.

And then the application and marketing teams get involved. They don't want to document everything I've documented (including waveforms and corner conditions), and they want to simplify the overall narrative. That doesn't help.

I'd love to be able to point the end engineers that use our products at my documentation--it's complete, it's thorough, it's well written (so I'm told)--but marketing and management insist we thin things out between what we architect and what we sell.

And since I'm busy working on the next product, it's hard for me to go back and make sure what we share with customers is actually useful after all the deletions. *sigh*

Anyway, that's my $0.02. I'm not sure it's kicked in with the latest MPB Retina Display Macs, but I know it's happened with other products I've been a part of (and are likely part of most of the cell phone calls you make these days..).

Playing with Thunderbolt under Linux on Apple hardware

Corporate IT

Re: Corporate IT

Re: Corporate IT

Re: Corporate IT

Re: Corporate IT

Re: Corporate IT

Re: Corporate IT