Ideas on modernizing the open-source desktop

Scott Jenson has been operating on person interfaces (UIs) and person cognition (UX) for many years at Apple, Google, and another companies. Now, he's trying to convince open-source projects to test additional and run the desktop beyond the age-old "windows, icons, menus, pointer" (WIMP) model. At Akademy 2026, KDE's annual developer conference, he shared his complaints and ideas in a conversation aimed at convincing those in attendance to obtain the guide on desktop design.

He introduced himself as person who has been doing UX for a lengthy time, much of that period at Google. He stated that complete that period he had noticed a traditional shift, "because rear in 2005 whenever I joined, we weren't evil; we were really trying to do the right thing". He had worked on the Chrome squad for a while, which he stated was "filled alongside group who wanted nothing but open-source, open-web" projects. Things had changed radically, which led him to depart Google, "but I'm trying to atone for my sins operating for these companies".

This LWN.net subscription-only satisfied has been made accessible to you by an LWN subscriber. To see additional of this content, delight obtain advantage of the following particular offer.

Free test subscription

Try LWN for liberated for 1 month: no payment or credit cardstock required. Activate your test subscription now and see why thousands of readers subscribe to LWN.net.

He now does activity for Mastodon and Home Assistant, "just benevolent of trying to squeeze a small bit additional fine UX scheme into open source". Jenson stated he did not desire to be confrontational, but he idea that open origin needs a lot of help in the area of UX design. That is challenging, since open-source projects are understaffed.

Innovation

He was at Akademy to conversation his ideas since he had stated a talk similar to this at Ubuntu's gathering final year, which proved to be popular. "It got additional than fractional a myriad views on YouTube—which by YouTube standards is trivial, but by Scott standards is enormous." That told him that the topic of desktop UX was item that group wanted to conversation about. This period around, he also wanted to display a few examples of prototypes he had to display his ideas.

He thinks that desktop UX is extremely distinct from another kinds of UX. Jenson related a narrative concerning operating on the Macintosh's Finder application, which is a file manager. The squad "wanted to do ellipses" in document names whenever showing a document in the Finder that had a name too lengthy to display in its entirety. The developers wanted to put the ellipsis at the end of the document name, but he idea that would lose too much information, and suggested that the ellipses have to be in the center of the file names. "It's now considered to be one of those things that's an exciting little attention-to-detail matter the Macintosh does."

Another UX item that he worked on is the way that clicking and dragging files from one opening to another plant on the Mac. A mouse click act has two parts: when the person depresses the clasp (mouse down) and whenever the person releases the button (mouse up). On the Mac, item choice happens on mouse down, but the opening only raises on the mouse up action. In another words, if a person drags an icon from a Finder window into another window, the archetypal opening is not raised.

On most Linux systems, though, if a person is trying to click and drag a document from a window in the backdrop it volition logic the opening to lift as shortly as the person does the "mouse down" action. That complicates trying to click and drag a document from one window to another. That is a essential act that method that users can't move data into applications fairly as effortlessly on Linux as they can on the Mac.

He stated that he had talked concerning that matter two years ago on Mastodon, which raised a reply from a KDE developer who idea it was a fine idea to distinct the mouse downward and mouse up actions whenever dragging data between windows. Then they implemented it a few hours later.

"I forgot who that was. So if anybody current did that", he said, but before he could complete the declaration a sound from location in the area responded, "you're welcome". Jenson yelled "thank you! I've been wanting to encounter you for two years!" That was an example, he said, of how open origin can be awesome: someone can fair say "that's a fine idea, I'll fair do it" and afterward rapidly implement and shove the characteristic to the project.

Stabilized

The item he was really trying to make, though, is that the desktop UX had not really changed in 20 years. There were plentifulness of new ideas and improvements in the 1980s and 1990s, "and afterward it fair stabilized, and not much has really changed". But group are doing much additional alongside their computers today, and he thinks that there are ways for desktop UX to grow.

Much of the UX for Linux systems was initially copied from Windows and the Mac, "KDE equal says it's for Windows users, that it wants to be akin to Windows for users". There's nothing incorrect alongside that, but he felt that the Linux community "in general" had been leaning on Microsoft and Apple to do all of the UX research and testing, "and frankly, [make] the mistakes".

Some power item out that GNOME has conducted user research for several of its scheme changes, and KDE's human-interface guidelines (HIG) were also developed according to research by its contributors. However, it would be fair to say that these are exceptions fairly than the norm; certainly, no one is backing or conducting regular UX scheme investigation and development for the Linux desktop overall.

He stated that it is astute to let Apple and Microsoft do these things archetypal "so we can outline rearward them", but the issue is that "they've fair benevolent of given up to an extent". Apple has been letting its desktop UX, and desktop hardware, "languish for years and years and years" since it had shifted its concentration to the iPhone and iPad devices. "I mean, like, Apple wanted the Mac to die. They wanted the earth to move complete to the iPad. Guess what? It didn't."

Apple has since turned its notice rear to the Mac, but the things they've done since—such as the Liquid Glass scheme language—have not been really famous alongside users. The features that Apple tends to add, he said, are "basically concerning tying it nearer to the iPhone, tying you nearer into their ecosystem". That isn't concerning improving the desktop, it's concerning expanding their moat.

Microsoft "is fair making one error following another". He cited the company's attempts to power users to shop records using its OneDrive cloud-storage service, the privacy-invasive Windows Recall characteristic that drew general criticism, and "ads everywhere" on the Windows desktop. His item was not to attack Apple or Microsoft, although the audience didn't appear to entity to this, it was to item out that the Linux desktop could not follow their example any longer. "The entire Linux community has been fair waiting for Apple and Microsoft, additional or less, to obtain all the risks, and they're not taking any additional risks."

Three pushbacks

Jenson emphasized that he didn't desire to alter everything, but he wanted to see experiments and additional betterment aimed at improving the desktop experience. However, he stated that he gets pushback whenever he tells group that desktop innovation has stopped, and that the Linux desktop "can't fair place and bug fix our way into the future, we have to commencement taking guidance ourselves".

He grouped the pushback into three categories. The archetypal exception he receives is "mobile won; the desktop is old, fair do what mobile does". He dismissed that by saying that mobile has certainly won the person market, social media, and other areas, but "it didn't win productivity". People motionless do their activity on computers, since "desktops are silently awesome; they have keyboards and big screens, and they can do amazing things". However, group have misplaced perspective of that and don't really value how fine desktops are. "I think we should understand why they're good, and afterward what we can do to create them better."

The second exception he hears is that there are lone so many ways that the WIMP model can work. "Just halt trying to fix it; it's done." He disagreed with that as well, and idea that there was additional to be done. The final, particularly strong, exception from the tech community is "don't contact my stuff". People get furious whenever they have worked difficult to get their desktop surroundings fair as they like it and item changes to disrupt that. "And I get that. As I stated before, I don't desire to alter everything." But the earth is changing, "and we need to grow our way gently into several new things."

What is UX?

Thinking concerning anywhere to go next alongside desktop scheme gets to the inquiry "what is UX?" He stated that term was misunderstood and that people, particularly director types, "they think it's fair pixels, but UX is so much additional than pixels". It helps to think of UX as having layers, he said. There is style, which consists of white space, iconography, color schemes, and yes, pixels. But there is additionally structure, he said, which is how the person moves through the application, the navigation model.

Strategy is another part of UX; understanding who the mark person is, and determining what they want. Deciding what is important, and what is not; what will the betterment squad prioritize and what volition it ignore? "The fastest way a UX designer can preserve your squad period is to assistance you say no."

There is additionally a tier he called "stuff": the lower-level restrictions of the underlying innovation that a phase has which power UX. For example, he said, if an use is being created for DOS, "you're not going to compose a nice Macintosh application. The innovation dictates what you can do". The desktop has been evolving for 40 years and has "historical cruft" that carries expected standards.

We fair have a bunch of material that we fair accepted. And I think if we're going to fix the desktop, we have to comprehend this lower-level limitation, and how do we desire to fix that? We fair obtain that mobile and desktop has stuff today, and we fair don't think concerning fixing it.

People forget, he said, that the first Macintosh display was "a whopping 342 pixels high". Naturally, windows overlapped a lot. Since then, "we just assumed that Windows have to overlap". But many group have "giant, giant screens", and the existing window-manager example does not activity so fine for those devices. "I can comprehend why group use tiled opening systems and so forth, because this existing example fair does not activity extremely well."

Understanding the future

Early in the talk, he had referenced a saying by Alan Kay, who had led scheme of the first WIMP interface at Xerox PARC: "Perspective is value 80 IQ points." Jenson now circled rear to Kay: "Who, as you can tell, I really akin a lot. [...] He talks concerning three steps to comprehend stuff". The archetypal step, he said, was to think concerning the present, "what is happening correct now?" The second is "don't ignore the past, cognize what activity has happened before", and the final step is to justify that "your questions are really clear". Too many group hurry to a solution, he said, but they have to be rushing to questions. If the questions are clear, afterward the answers volition be better.

He did not desire to create the conversation concerning AI, he said, but he had to address it. "I have to conversation concerning AI, and it pisses me off since there's so many cool things we could be talking about, and AI fair sucks up all the oxygen, and it's all you can conversation about". But, he conceded, there are many things happening alongside AI on the desktop, albeit in a weird way.

He explained several of the current efforts to merge AI alongside the desktop as "trying to sneak AI in" during keeping the UI exactly the same. Other efforts try to use AI to comprehend what is on the display and use chatbots to engage with the interface, "which I think is benevolent of missing the item here". That is, until final summer whenever Anthropic introduced Claude Code. "It entirely changed things, since was looking downward into your filesystem" and drew environment from the content of a user's files. That allowed it to do additional engaging things and interact with the system.

After mentioning Claude, he paused to say he was lone speaking concerning frontier models since everyone alternatively is discussing them. "I have a lot of personal problems alongside the frontier models. They are ethical and ecological disasters. I am not extremely enthusiastic by them". He stated he was enthusiastic by Apertus, which is billed as a completely open, responsible example for sovereign AI created by the Swiss AI Initiative. "I do think it's possible to conversation concerning ethically trained small tongue models operating locally, and you don't have to market your psyche to the devil."

The curse of straightforward manipulation

The key item he wanted to create was not concerning generative AI uses, but concerning how giving it admission to the filesystem improved its usability. It now had environment and could activity alongside the user's data, but it was operating at the filesystem flat and not using the desktop UX.

He talked concerning the desktop construction blocks, basically the acquainted WIMP model plus the desktop clipboard, as the way that users move data around. Expert users can move data about quickly, he said, but there's a problem: the desktop is stateless. He called this the curse of straightforward manipulation; there is no working memory to the desktop. "If you copy a few too many things to the clipboard, oh sorry, it's gone".

Jenson theorized that the logic there are so many desktop environments, window managers, and Wayland compositors for Linux—as fine as substitute clipboard managers—is since the desktop isn't fairly operating correct for users. He thought it was really concerning operating memory: "How do I oversee my data in specified a way that I can recall it throughout all of these things and really use it." His perspective shifted to asking "what would we do to operating recollection to enhance the desktop UX?"

AI may have stated him the idea, but he added that he wanted to resolve the desktop UX for group first. "If the AI can use it, great. I don't care. That comes later."

Remember your past

There is previous art in trying to create systems that assistance provision users alongside more context and operating recollection connected to their data. He referenced a paper called "Lifestreams: a retention example for individual data". That document made the case that desktop systems are "are ill-equipped to oversee the digital data and events of the representative device user", and introduced the Lifestreams idea as a metaphor for dynamically organizing a user's device workspace.

That document influenced the ill-fated WinFS project from Microsoft, which was an attempt to merge data retention and administration according to relational database systems. It was demonstrated in 2003 and scheduled to container sometime later, but ultimately the project was shelved in 2006. On the free-software flank of the house, there was the semantic desktop project NEPOMUK, which was a investigation project funded by the European Union.

A type of NEPOMUK was included in KDE 4, alongside the idea was that it would authorize applications "to use information from all complete the desktop, the web, another devices, and merge it into one coherent interface". It was afterward removed and replaced alongside the Baloo file-indexing and hunt framework for the Plasma desktop, which has several overlap alongside NEPOMUK's functionality but with a reduced range of features.

Jenson stated that he is concerned that group see the nonaccomplishment of those projects as proof that the ideas were wrong. He asserted that is not the case, "I would say that it is the hardware and systems at the period let the imagination down. I think it's period to rethink these projects" in ray of newer hardware and improved systems.

Prototypes

Having visited the current and the past, as Kay recommended, he had arrive up with a few questions that he idea value exploring connected to desktop UX. The first was "what would happen if we lone designed for ample monitors?" What would it unlock if there were desktop designs that ignored laptops and lone focused on widescreen monitors? Another inquiry he had was how could the desktop grasp a user's intent complete time?

Those questions led to the prototypes that he stated he was a small nervous to show the spectators "because specialized group affection to discover mistakes". The demos are about 29 minutes into the video of Jenson's talk. His slides have not yet been published but he told me at the event that they would be at several point.

The archetypal demo was to display how a desktop power create improved use of a widescreen display. He noted that the center of a widescreen detect is fine for working with an application, but it becomes additional difficult to activity on the sides of the monitor. "It's fine for peripheral vision, but not fine for working".

He demonstrated a desktop layout, a screenshot of which is below, that would put the focus on the opening or windows in the center of the monitor. He had included a grid in the backdrop that tapers off from the center of the display to simulate a widescreen monitor. The idea was that a widescreen-first desktop would emphasize information in the center of the screen, and afterward use an Exposé-like tiling example to arrange the rest of the desktop's windows. This would create the windows that were not in use easier to engage with, he thought, and be a improved example than virtual desktops for managing many of windows. "I'm not against virtual desktops. Virtual desktops are used by extremely organized and intelligent people. I cognize my audience." However, he said, most group can't agreement alongside virtual desktops.

He additionally demonstrated dragging a music-player opening all the way to one flank of the screen, which he called "the stash area". As it moved to the edge, it transformed into a widget alongside fair a perform clasp alternatively of showing the full window. He idea that it would be imaginable to do the things he was demonstrating with Wayland as it is today.

Jenson put up another display of an improved clipboard example that copied ideas from the Obsidian editor's Canvas feature. Canvas allows users to drag and drop images, text, and records to create a ocular layout of information. A person could create a document and drag data from a web browser into the document, autumn a file in from another source, "the idea current is that this is now efficiently a collection of material that I have gathered for this document. And if I near it and then arrive rear tomorrow, it's all motionless there". He added that a local AI would be able to arrive in and arrange the data a person had gathered, "oh, look, you've got a bunch of hotels, let me arrange them for you".

The final idea he discussed was a way of gathering "privacy-preserving data" to assistance provision users a improved visualization of things akin web-browsing history. A web browser volition lone provision the person a perspective of their browsing former that "is not extremely useful" he said, so he tried to collect data "not using AI, just using uncomplicated math" that would use notice signals—such as how long the person was on a web page—to inform a narrative of anywhere they spent their period on the desktop. "I basically came up alongside this spatial associative episodic memory prompt" to arrange data in a helpful, engaging way. He idea that working memory was a fine way to examine new ideas for desktop UX.

So the goal alongside this conversation is to say, let's commencement to obtain this apart. And I really hope that my prototypes logic you guys to arrive to me and say, We forgot concerning this, and let's conversation concerning that. [...] So I fair desire us to try, since no one alternatively is going to try, and I think someone's got to start.

There was period for a sole question. An spectators associate asked how Jenson's model would defend against all of the privacy concerns that Microsoft faced alongside Windows Recall. "How are we going to create certain that this former remains lone accessible by the person and cannot be hacked in akin it could happen alongside Windows Recall?"

Jenson answered that his idea was to collect telemetry data, "not interesting information", which would considerably decrease the attractiveness of the data to would-be attackers. He stated he did not desire to underplay the issue, though, and wanted to build prototypes to see if the idea was fine first, and afterward examine ways to encrypt or defend the data. He accepted that equal this data could be valuable information. He added that the biggest issue alongside Windows Recall was "how stupidly they protected it" and he was certain others could do a lot improved job.

[I would akin to appreciate the Linux Foundation, LWN's journey sponsor, for its assistance alongside my trip to Graz, Austria for Akademy 2026.]

---