Unix sockets, Cygwin, SSH agents, and sadness

[mjg59]

Work involves supporting Windows (there's a lot of specialised hardware design software that's only supported under Windows, so this isn't really avoidable), but also involves git, so I've been working on extending our support for hardware-backed SSH certificates to Windows and trying to glue that into git. In theory this doesn't sound like a hard problem, but in practice oh good heavens.

Git for Windows is built on top of msys2, which in turn is built on top of Cygwin. This is an astonishing artifact that allows you to build roughly unmodified POSIXish code on top of Windows, despite the terrible impedance mismatches inherent in this. One is that until 2017, Windows had no native support for Unix sockets. That's kind of a big deal for compatibility purposes, so Cygwin worked around it. It's, uh, kind of awful. If you're not a Cygwin/msys app but you want to implement a socket they can communicate with, you need to implement this undocumented protocol yourself. This isn't impossible, but ugh.

But going to all this trouble helps you avoid another problem! The Microsoft version of OpenSSH ships an SSH agent that doesn't use Unix sockets, but uses a named pipe instead. So if you want to communicate between Cygwinish OpenSSH (as is shipped with git for Windows) and the SSH agent shipped with Windows, you need something that bridges between those. The state of the art seems to be to use npiperelay with socat, but if you're already writing something that implements the Cygwin socket protocol you can just use npipe to talk to the shipped ssh-agent and then export your own socket interface.

And, amazingly, this all works? I've managed to hack together an SSH agent (using Go's SSH agent implementation) that can satisfy hardware backed queries itself, but forward things on to the Windows agent for compatibility with other tooling. Now I just need to figure out how to plumb it through to WSL. Sigh.

Comments

Sounds fun

[(Anonymous)]

Haha

Empathy

[(Anonymous)]

Attempting to 'unixify' Windows is indeed a task often drought with much pain. MS shower to have recognised this over recent years by investing in better tooling like PowerShell, Windows Terminal and Subsystem for Linux. But it's still non-trivial. For instance, I cannot install WSL2 despite my variation of Windows 10 seemingly being updated to just va tick over the minimum version. Thankfully I don't need it. I feel your pain and acknowledge how ludicrously unnecessary it is.

Great work

[(Anonymous)]

This sort of painful work only gets done occasionally, and is really worth it for the rest of us.

Re: Great work

[(Anonymous)]

this kind of stuff is in fact the "secret sauce" in fortune 100 tech companies' products

[rhialto]

Somehow this whole thing reminds me of ixemul.library on the Amiga. I think part of GeekGadgets / ninemoons / Fred Fish. Somebody mirrored some version of it here: https://github.com/bebbo/ixemul "Essentially it is a BSD Unix kernel running under the Amiga OS.".

[(Anonymous)]

Essentially it is a BSD Unix kernel running under the Amiga OS.

Like User Mode Linux (UML) was basically a Linux kernel running as a Windows process over a decade ago, right?

[sweh]

UML also ran under Linux (Linux under Linux), and this was basically how linode.com started; selling UML based "virtual machines".... 20 years ago!

[rhialto]

Sure. But ixemul was there already in the mid 1990's. It was quite impressive, but not simple... and it allowed to run monstrously big things like gcc much more easily.

Windows SSH HW Certs to WSL

[(Anonymous)]

Cygwin's Putty-derived Xterm-compatible native Windows terminal emulator mintty https://github.com/mintty/mintty includes a --WSL option that allows it to talk to a WSL distro instance using WSLbridge processes running on both ends and a WSLtty "package" that includes base Cygwin, mintty, and wslbridge/2? That may be adaptable to do the "dirty" work for you, or provide a basis for a channel.