My a11y journey
Jun. 20th, 2025 01:11 am![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
mjg5923 years ago I was in a bad place. I'd quit my first attempt at a PhD for various reasons that were, with hindsight, bad, and I was suddenly entirely aimless. I lucked into picking up a sysadmin role back at TCM where I'd spent a summer a year before, but that's not really what I wanted in my life. And then Hanna mentioned that her PhD supervisor was looking for someone familiar with Linux to work on making Dasher, one of the group's research projects, more usable on Linux. I jumped.
The timing was fortuitous. Sun were pumping money and developer effort into accessibility support, and the Inference Group had just received a grant from the Gatsy Foundation that involved working with the ACE Centre to provide additional accessibility support. And I was suddenly hacking on code that was largely ignored by most developers, supporting use cases that were irrelevant to most developers. Being in a relatively green field space sounds refreshing, until you realise that you're catering to actual humans who are potentially going to rely on your software to be able to communicate. That's somewhat focusing.
This was, uh, something of an on the job learning experience. I had to catch up with a lot of new technologies very quickly, but that wasn't the hard bit - what was difficult was realising I had to cater to people who were dealing with use cases that I had no experience of whatsoever. Dasher was extended to allow text entry into applications without needing to cut and paste. We added support for introspection of the current applications UI so menus could be exposed via the Dasher interface, allowing people to fly through menu hierarchies and pop open file dialogs. Text-to-speech was incorporated so people could rapidly enter sentences and have them spoke out loud.
But what sticks with me isn't the tech, or even the opportunities it gave me to meet other people working on the Linux desktop and forge friendships that still exist. It was the cases where I had the opportunity to work with people who could use Dasher as a tool to increase their ability to communicate with the outside world, whose lives were transformed for the better because of what we'd produced. Watching someone use your code and realising that you could write a three line patch that had a significant impact on the speed they could talk to other people is an incomparable experience. It's been decades and in many ways that was the most impact I've ever had as a developer.
I left after a year to work on fruitflies and get my PhD, and my career since then hasn't involved a lot of accessibility work. But it's stuck with me - every improvement in that space is something that has a direct impact on the quality of life of more people than you expect, but is also something that goes almost unrecognised. The people working on accessibility are heroes. They're making all the technology everyone else produces available to people who would otherwise be blocked from it. They deserve recognition, and they deserve a lot more support than they have.
But when we deal with technology, we deal with transitions. A lot of the Linux accessibility support depended on X11 behaviour that is now widely regarded as a set of misfeatures. It's not actually good to be able to inject arbitrary input into an arbitrary window, and it's not good to be able to arbitrarily scrape out its contents. X11 never had a model to permit this for accessibility tooling while blocking it for other code. Wayland does, but suffers from the surrounding infrastructure not being well developed yet. We're seeing that happen now, though - Gnome has been performing a great deal of work in this respect, and KDE is picking that up as well. There isn't a full correspondence between X11-based Linux accessibility support and Wayland, but for many users the Wayland accessibility infrastructure is already better than with X11.
That's going to continue improving, and it'll improve faster with broader support. We've somehow ended up with the bizarre politicisation of Wayland as being some sort of woke thing while X11 represents the Roman Empire or some such bullshit, but the reality is that there is no story for improving accessibility support under X11 and sticking to X11 is going to end up reducing the accessibility of a platform.
When you read anything about Linux accessibility, ask yourself whether you're reading something written by either a user of the accessibility features, or a developer of them. If they're neither, ask yourself why they actually care and what they're doing to make the future better.
The timing was fortuitous. Sun were pumping money and developer effort into accessibility support, and the Inference Group had just received a grant from the Gatsy Foundation that involved working with the ACE Centre to provide additional accessibility support. And I was suddenly hacking on code that was largely ignored by most developers, supporting use cases that were irrelevant to most developers. Being in a relatively green field space sounds refreshing, until you realise that you're catering to actual humans who are potentially going to rely on your software to be able to communicate. That's somewhat focusing.
This was, uh, something of an on the job learning experience. I had to catch up with a lot of new technologies very quickly, but that wasn't the hard bit - what was difficult was realising I had to cater to people who were dealing with use cases that I had no experience of whatsoever. Dasher was extended to allow text entry into applications without needing to cut and paste. We added support for introspection of the current applications UI so menus could be exposed via the Dasher interface, allowing people to fly through menu hierarchies and pop open file dialogs. Text-to-speech was incorporated so people could rapidly enter sentences and have them spoke out loud.
But what sticks with me isn't the tech, or even the opportunities it gave me to meet other people working on the Linux desktop and forge friendships that still exist. It was the cases where I had the opportunity to work with people who could use Dasher as a tool to increase their ability to communicate with the outside world, whose lives were transformed for the better because of what we'd produced. Watching someone use your code and realising that you could write a three line patch that had a significant impact on the speed they could talk to other people is an incomparable experience. It's been decades and in many ways that was the most impact I've ever had as a developer.
I left after a year to work on fruitflies and get my PhD, and my career since then hasn't involved a lot of accessibility work. But it's stuck with me - every improvement in that space is something that has a direct impact on the quality of life of more people than you expect, but is also something that goes almost unrecognised. The people working on accessibility are heroes. They're making all the technology everyone else produces available to people who would otherwise be blocked from it. They deserve recognition, and they deserve a lot more support than they have.
But when we deal with technology, we deal with transitions. A lot of the Linux accessibility support depended on X11 behaviour that is now widely regarded as a set of misfeatures. It's not actually good to be able to inject arbitrary input into an arbitrary window, and it's not good to be able to arbitrarily scrape out its contents. X11 never had a model to permit this for accessibility tooling while blocking it for other code. Wayland does, but suffers from the surrounding infrastructure not being well developed yet. We're seeing that happen now, though - Gnome has been performing a great deal of work in this respect, and KDE is picking that up as well. There isn't a full correspondence between X11-based Linux accessibility support and Wayland, but for many users the Wayland accessibility infrastructure is already better than with X11.
That's going to continue improving, and it'll improve faster with broader support. We've somehow ended up with the bizarre politicisation of Wayland as being some sort of woke thing while X11 represents the Roman Empire or some such bullshit, but the reality is that there is no story for improving accessibility support under X11 and sticking to X11 is going to end up reducing the accessibility of a platform.
When you read anything about Linux accessibility, ask yourself whether you're reading something written by either a user of the accessibility features, or a developer of them. If they're neither, ask yourself why they actually care and what they're doing to make the future better.
no subject
Date: 2025-06-20 11:34 am (UTC)Wayland is a set of protocols. So saying "Wayland doesn't run without crashing" is a bit like saying "POSIX doesn't without crashing", or "I can't connect to TCP/IP today" :-)
Several projects implement the Wayland protocols independently; that includes GNOME (gnome-shell), KDE (kwin), and various others. People usually call something that implements the protocol a "compositor".
Have you tried multiple compositors on your machine? There is very little code shared between different implementations, so if you see multiple different compositors crashing, it's most likely a problem with a particular driver on your machine. Or, if you've only tried one compositor, maybe it's a problem with that implementation.
Sam
no subject
Date: 2025-06-20 11:36 am (UTC)no subject
Date: 2025-06-20 12:06 pm (UTC)When I last went looking for myself, it seemed to me that the compositor landscape consisted of a few things everyone has heard of which are designed for large user bases and aren't very configurable, like gnome-shell, plus dozens of personal projects on Github that seemed to be at about the level of "I thought I'd try writing a compositor for fun, and stopped when it stopped being fun". (Which is a fine thing to do and I'd never discourage it, but that's not the same as wanting to run the result!) Most of the latter seemed to be of the tiling type, presumably on the grounds that those are easier, and make a good Baby's First Compositor project.
In X11, I'm currently a user of the Sawfish window manager, which is programmed in Lisp, and I've heavily customised its window-frame styling, its default placement and sizing policy for new windows, and a large number of custom UI actions on the frames to do various things (like sending the window to the back, or resetting it to a default size I configured for that particular window type). What would you recommend for someone to switch over to?
no subject
Date: 2025-06-20 02:08 pm (UTC)I know of one project that isn't funded but has a team of active volunteers, they make a "configurable" compositor named Sway: https://github.com/swaywm/sway. It's a recreation of the tiling window manager i3, if you ever used that, and I think config is done in the same way. That could the best one to dig into, given you probably don't have infinite time to try out every possible option :-)
There's a second one called Hyprland you may hear about, which is disrecommended for reasons written up here a couple of years ago: https://drewdevault.com/2023/09/17/Hyprland-toxicity.html. Maybe it's improved since then, I don't know.
There's also an entire compositor written in Common Lisp, named Ulubis! But it seems the last commit was 5 years ago. https://github.com/malcolmstill/ulubis
Qtile?
Date: 2025-06-20 05:36 pm (UTC)no subject
Date: 2025-06-20 11:59 am (UTC)You don't just need a set of protocols for applications to ask to do things like opening windows. You also need a large pile of code for putting pixels on the physical screen.
I'm aware that there are multiple Wayland compositors. But I don't think they're so independently written as to have completely separate sets of video card drivers. First, it doesn't stand to reason: surely writing a graphics driver is such hard work that nobody would do it twice if they could avoid it, so in practice there will be one set of drivers that everyone uses. Second, I checked a few compositors with
lddand indeed gnome-shell, kwin and sway seem to be linking in the same set of driver libraries.(Indeed I understand that at the next layer up there's also quite a lot of de-facto shared code between compositors? Maybe not all compositors, but I've definitely heard of the wlroots library, which does a lot of the heavy lifting in a way that many compositors can share.)
I don't remember if I tried all three of those compositors. (My last new machine was a couple of years ago now.) But I certainly tried more than one. So it may not be technically accurate to blame the crash on Wayland, in that it sounds as if I'm blaming it on the protocol, but if the ecosystem of protocol implementations has significant shared code then it's perfectly reasonable to blame a crash on that. And the net effect would still be that when people say "you should run Wayland instead of X11" I have trouble actually doing it.
no subject
Date: 2025-06-20 12:05 pm (UTC)In terms of shared code, I think there are really 3 implementations. GNOME uses Mutter. KDE uses Kwin. And pretty much everyone else uses Wlroots, as far as I know, as it indeed takes a lot of work to implement all the protocols from scratch.
no subject
Date: 2025-06-20 01:04 pm (UTC)no subject
Date: 2025-06-20 02:14 pm (UTC)no subject
Date: 2025-06-20 02:51 pm (UTC)- you only use the most vanilla possible GNOME/Wayland desktop
- people should try different Waylands until they find one that works for them
- there are lots of configurable Waylands
- and, by implication, that every Wayland is its own thing, and that the lack of coordination among them re: accessibility is not a problem, because surely one of them will eventually completely fill each person's use cases
Do you see problems with this set of statements, from the perspective of, say, someone who currently uses just a few accessibility technologies but is aware that in the future, an only partially predictable set of different accessibility technologies will be needed?
I do.
no subject
Date: 2025-06-23 10:24 am (UTC)We live in a time of increasing global inequality. Very few people have the luxury of being able to work on this stuff for free.
Everything else is an implementation detail. Technology doesn't stand still; if nobody's being paid to maintain a codebase, it doesn't matter whether it's implementing X11 or Wayland, things won't work very well. Listen to maintainers and pay them money for the work they're doing.
no subject
Date: 2025-06-23 07:55 pm (UTC)Amongst the design mistakes I would say that Wayland made was to make the compositor do so much of the work, write a reference one, and then say 'but you can write your own'.
It should have been obvious what would happen next: people absolutely would, and even the most popular ones wouldn't be completely compatible with each other.
no subject
Date: 2025-06-27 01:06 am (UTC)From the point of view purely of 'is there an accessibility solution' then yes, once Gnome or KDE have implemented most of the assistive technologies you'd expect under Linux the issue is solved with a small s.
However, that's pushing modern Unix to actually mean 'Linux running GNOME' *and nothing else*. This is a substantially different world than with X, despite its many faults.
Maybe that's actually what some Linux users want, but it starts looking to me remarkably similar to the single compositor and desktop model of Windows and Mac OS and some people will at that point start to wonder 'just why am I bothering with this anyway, when I can't customise *my* computer'
The Wayland architecture, and specifically the people developing for it, is broken. It's driven by large companies, relies on a resource level offered by large companies, and is incredibly Linux centric. The 'reference' Weston compositor for instance only runs on Linux which is not a stable reference in any sense of the phrase I'd consider.
A proper Unix solution would to my mind include a small number of compositor bases (ideally : one) - not linked in source. Any improvement to base compositor functionality would automatically flow down to the part providing the real functionality whether that's a GNOME like behemoth of complexity, or something really bare bones (think cwm for X, labwc is the nearest I've found on Wayland). It would support the smallest and largest developers, and also have to account for non Linux platforms.
This isn't the case currently, any wlroots updates requires the compositor to be updated and relinked, making compositor maintenance a burden for smaller developers. To be very clear : it should be possible to update some base compositor part of Wayland to support a new protocol and then have this available to your main compositor of choice *with no action by the developer*.
I have occasionally had issues with smaller X window managers, usually when installing larger applications that have substandard error handling when calling 'put an icon on the desktop' and falling over on discovering this window manager *doesn't have a desktop*. Most other deficiencies in the window manager can be worked around with additional programs as most functionality exists in X itself, and although you can technically create your own X Server from scratch, practically no-one does, and things such as xorg.conf are a known quantity.
Also, yes, this is written on Wayland, and I still think it's a pain. Especially running a smaller compositor on Not Linux.
no subject
Date: 2025-06-23 03:45 am (UTC)