Plasma's login experience is an area that we know requires some improvement — it works OK in the basic case, but it's very barebones and doesn't handle anything beyond that.

As a complete desktop experience, it's our job to provide support for the edge cases too.

What we want

• Great out-of-box experience in multi-monitor and high DPI and HDR
• Keyboard layout switching
• Virtual keyboards
• Easy Chinese/Japanese/Korean/Vietnamese (CJK) input
• Display and keyboard brightness control
• Full power management
• Screen readers for blind people (which then means volume control)
• Pairing trusted bluetooth devices
• Login to known Wi-Fi for remote LDAP
• Remote (VNC/RDP) support from startup

A brief history

In Plasma 5, we retired our own bespoke display manager KDM, in favour of SDDM. A display manager started for multiple lightweight Qt Desktops. It was modern at the time making use of new QML for the front and as a big selling point at a time when Plasma was also making use of it.

SDDM's Big Architecture Problem

We ran into a problem, though. SDDM is designed to show a single greeter window, loading arbitrary QML from the specified theme.

Whilst this all sounded great for Plasma, the abstraction is at the wrong level — for our wishlist we need a lot more tight integration from our login screen than just a window showing sessions and users.

With SDDM, power management is reinvented from scratch with bespoke configuration. We can't integrate with Plasma's network management, power management, volume controls, or brightness controls without reinventing them in the desktop-agnostic backend.

SDDM was already having to duplicate too much functionality we have in KDE, which was very frustrating when we're left maintaining it.

The Competition

GNOME's GDM is the gold-standard of display managers, and it achieves this higher level of quality by running half of a Gnome session.

SDDM got closer when it added Wayland support — it had to use a compositor such as KWin. But because the project tries to be agnostic between desktops, it has to support any compositor. There aren't compositor-agnostic ways to do even simple things like set the keyboard layout, so in the end this compositor agnosticism goal simply didn't work.

Theme Problems

A major mistake we made throughout Plasma 5 was conflating "writing UI in a high level scripting language" with "it's themable" as the same thing — they are not. QML does make it easy to modify and iterate without programming skills, but it still contains business logic. It should not be the primary method of customisation that we expose to users.

Ultimately, this was poor for end users. We pushed back on adding support for configurable theme options, because building a theme engine within a theme was wasteful! But often people want to just change a few things. Choosing a theme meant finding a combination that had everything. The store filled up with themes that are 99.9% identical code-wise; most are just wallpaper mods.

It was also poor for theme developers. Not only do they have to modify the visuals, but also re-implement focus handling, accessibility, and the same boring logic again. They can't benefit from widespread testing so regressing these functionalities is common. Reddit is full of screenshots of broken SDDM themes.

Finally, it's poor for us Plasma developers: theme support holds us back from adding new features or tidying code; if we want to add a new feature that in any way affects existing UIs, the situation get very messy very fast. The end result is that features just don't land, and the end user is the one that misses out.

So, what's the plan?

It's worth stressing nothing is official or set in stone yet, whilst it has come up in previous Plasma online meetings and in the 2023 Akademy. I'm posting this whilst starting a more official discussion on the plasma-devel mailing list.

Oliver Beard and I have made a new mutli-process greeter, that uses the same startup mechanism as the desktop session. It doesn't have all the features that we propose at the start of the blog, but an architecture where features and services can be slowly and safely added.

For customisation we intend to expose the same familiar settings that exist in Plasma and bring the design more in-line with the existing screenlocker where we also dropped arbitrary QML years ago. We'll make the wallpaper configurable with any existing wallpaper plugins, and expose the existing plasma theme and colour settings. Syncing will be a case of copying files, not re-inventing things.

The backend

When starting work on this, I tried to explore all alternative backends out there even with fully working implementations, however in practice nothing was maintained that matched our requirements. SDDM has been proven in the real world, so we have taken that and stripped it down to cover what we want moving forwards. I also aim to incubate it into the KDE ecosystem to have full autonomy over the project and merging stuck patches.

Current State

All of this has been implemented as two new repositories. Plasma Login Manager a continuation of SDDM and Plasma Login for front end and KCM (settings) code. These might be merged at some point.

The new code all works, and is at roughly feature parity with what we're replacing. A screenshot looks roughly the same as a stock Plasma SDDM setup. Whilst this is at a state where developers can opt-in, I would not want distros to be packaging things at this point.

Please do reach out if this sounds interesting, either directly or in the Plasma Matrix room - or with merge requests!

About

Home Automation is mostly, as the name implies, about automation.

The machine you're in front of all work-day has an awful lot of useful information that can be useful in your home management.

I have a script lower my blinds if I turn on the camera during the afternoon as otherwise there's an annoying glare. My office lights and monitor both have a redder hue at night, but disabling night-mode on my PC automatically disables the main light performing redshift too. I want my screen to turn off not 10 minutes after activity, which is simultaneously both annoyingly too long and too short, but the moment the motion sensor in my room says I've left.

Home Assistant, the best open source home automation tool out there supports all of this, but it needs some hooks for the PC to tell it about the current state and invoke actions on the PC.

None of the existing solutions worked well for what I wanted. I wrote something for personal use a few years ago that worked great for me, I ran it in the background and didn't plan to take it further.
During the KDE Goals sprint another user suggested creating something very similar (https://phabricator.kde.org/T17435). That goal may not have been chosen, but it did motivate me to tidy up and document what I had and turn it into a more fleshed out project.

The App

I made a small daemon, named Kiot (derived from "KDE Internet of Things") that exposes this information to your Home Automation software.

This is pre-alpha software, there's no GUI configuration it relies on manually setting up a configuration file to point to your Home Automation server. If there's enough traction I might make it more packagable, and create a UI config over the Christmas break.

There's a bunch of sensors and actions shipped, but it contains a faux plugin system to make extension easy along with hooks for user supplied scripts.

• User activity
• Locked state
• Suspend/Power actions
• Camera in use
• Accent Colour
• Arbitrary Scripts
• Shortcuts
• Nightmode Inhibition

Using this from Home Assistant is then as straightforward as possible.

Checking it out

The repository can be found at https://github.com/davidedmundson/kiot which has setup instructions along with developer documentation (https://github.com/davidedmundson/kiot/blob/master/DEVELOPMENT.md) if you want to make any additional plugins.

Whilst commit numbers aren't a perfect measure of anything whatsoever, they do still provide a good indication of whether a project is healthy or faltering. It's useful to keep an eye on them, and the start of the year is a good time to do that.

I like graphs, everyone likes graphs, they need no further introduction.

Frameworks

Plasma

KWin

Conclusions

My main reason for sharing is before I gathered some data I had a negative feeling about the community health, but none of the data backs it up and it was unfounded, it's hard to get a sense of the state of how the year has gone when you're busy living in it.

There was a bit of a boon last year with KF6/ Plasma 6.0 and this year did see a drop relative to that, but it's merely returning to the former average rather than being anything to worry about. The most important graph is how many new authors and contributors we're getting on a year-by-year basis. The number of total and new contributors is relatively consistent.

Things being unchanged and boring doesn't make for a good post title (hence the Youtube-level embellishing) but is a good place for such a large project.

The 3rd graph included is a measure of how many make up the bulk of commits. If this is 1, it means a sole dev is running the project and that's an unhealthy place to be if they ever leave. If it's too high it means there isn't a core team maintaining it. There's no "right" number, but I feel it's a really interesting metric, especially seeing what happened in Kwin.

About the stats

Graphs were generated from the same scripts used by Curl's dashboard (https://curl.se/dashboard.html). It doesn't have all the stats Curl has as each needed some manual poking to make it more generic.

Frameworks and Plasma stats were made by locally git filterbranching all repos into a mega-repo and using that as a base.

Since Plasma 5.18, nearly five years ago, Plasma has shipped with a "telemetry" system. It’s opt-in, allowing users to send a small amount of data back to us.

Was it useful or worth it? It's a question that comes up occasionally, and the answer is mixed. I believe it showed real potential, though the reality of our implementation was somewhat underwhelming and didn't really deliver. There are many lessons learned that are worth sharing with other projects that might face similar endeavours.

The good bits

Where we had data available for topics being discussed it worked. To give two concrete examples from memory:

• A developer claimed, "No one is using a screen smaller than 1024x768," while bumping the minimum size of a window. This was proved wrong; the number of users at 800x600 or even 640x480 is surprisingly high. Still low as an overall percentage, but higher than you would ever intuitively think. Presumably, it's the default for a lot of virtual machines.

• Four years ago, a developer claimed, "No one still uses only OpenGL2; we can change the code to do XYZ." A check of our user base showed it would have affected nearly 5% of our users, so the change was abandoned.

Interestingly, this last topic came up again very recently, as it held back colour management improvements, but in a narrower Wayland-only path and with a fallback. After checking metrics again, the usage was below 1%, so we went ahead with that merge request.

So, are metrics worth it just to stop developers and designers from making nonsense claims out of thin air? Absolutely! 90% of stats are just made up on the spot. Metrics are just as much about preventing changes as it is about sparking changes.

Indirect impact

The other important part is having a more general sense of the landscape. Currently, we have a lot of hard conversations about how quickly we push the move to Wayland. We have voices wanting to maintain support, and we have voices wanting to push quicker. These decisions shouldn't be made just by who can be the loudest. For every individual topic that came up in those discussions, I would always have in mind our current adoption value at the time.

Should we care about Nvidia? Knowing they make up about 25% of our user base makes the decision for us. I ran with an Nvidia card in one machine because of this, implementing Nvidia context loss handling and doing what we could during the Wayland transition
We don't test BSD while developing Plasma, but we also let it hold us back. Should we care about it more or less? My opinion matches exactly what the metrics say.

Some stats and graphs:

Another role of metrics is being a conversation starter—people will fawn over a graph. More topics on Reddit will be about our Wayland usage rather than the topic I'm trying to discuss. I'll focus on Wayland examples beacuse that's a topic close to me.

Wayland adoption over time

I keep tabs on what our metrics show here. We can see the slow increase from under 20% to around 45% over time, showing the progress as both we and the Wayland ecosystem evolved. At Plasma 6, we switched the defaults a small bump in the graph can be seen. but 45% still seemed rather disappointing.

Filtering on just Plasma 6 reveals the true story:

There's still 20% of users switching away, or using a distro with a different default, or having carried over presets, but it's more promising. Interestingly, we can see that the GPU vendor distribution differs between X11 and Wayland.

Problems and lessons learned

Ultimately, despite the positive parts it would be hard to call our telemetry a staggering success. For the handful of examples above, there are a hundreds of cases where we had no data to back anything up. The range of data points was pitiful and it wasn't often used

The viewing tool is really, really important!

Data collection without viewing it is meaningless. As shown above, we often need to drill down and cross-reference filters to extract conclusions.

The original plan was to use the existing UI provided by kuserfeedback, which did not scale at all and quickly fell over. It was designed for high-fidelity data for a small number of users, not what we had.

In a rush, we pivoted to using Grafana because there was already a setup hosted.

It worked—ish, but it’s not designed for this, especially combined with our data structure, which was a manual NoSQL in normal SQL. Every graph needed to be written by hand, and it felt very much like fighting the system rather than working with it. Combined with the limited access permissions granted, it wasn't used by many people.

It being used is the number one indicator of its usefulness!

We need to find a tool specifically designed for visualization and querying datasets (maybe Apache Superset?).

Time-based data just makes noise

Our system sent updates every N days with basically the same data every time. This made writing queries way messier than it should have been. It never added any value; I would always be interested in what the current stats are. As described in the Wayland usage graphs above, if I'm making a Wayland decision, it doesn't matter what most people are using; it matters what people on the latest release are using. We always ended up having to add filters to focus on just the latest version.

The upgrade story needs planning in advance

The amount of data we collected was tiny—some GPU information, screen information, language, and a few other fragments. The plan was to slowly add more and more stats over time, but we hit a wall. Our UX involved the user selecting to enable metrics and it being a fire-and-forget operation.

What do we do when we want to add more data? For example, whether you use an analog or digital clock. We would need to prompt the user and reset their settings in the meantime, which is at odds with it being a setting. The whole thing became such an ordeal that made it not worthwhile.

Wrap up

The project didn't fail, when we had data and it was used it worked, but overall our implementation falls short. I would like to open a discussion at Akademy on how we move forward with our current system potentially starting from scratch treating it more like a survey that we prompt to auto populate and submit each year.

Recent events

A global theme on the kde third party store had an issue where it executed a script that removed user's data. It wasn't intended as malicious, but a mistake in some shell parsing. It was promptly identified and removed, but not before doing some damage to that user.

This has started a lot of discourse around the concept of the store, secuirty and upstream KDE. With the main question how can a theme have access to do this?

To developers it's not a surprise that third party plugins can do this sort of thing. It's as intended. A global theme can ship custom lockscreens, custom applets, custom settings and all of these can run arbitrary bundled code. You can't constrain this without limiting functionality.

To that end there's an explicit warning when downloading plugins.

Expectations

Our primary issue boils down to expectation management and communication.

There are plenty of other ways where users can download and run any other unfettered code from the internet; the Arch user repository (AUR), adding ubuntu PPAs and alike. This isn't a bad thing - it's useful and people do amazing things with this to benefit users.

Nothing on the KDE store happens without explicit user interaction either.

A problem is there's an expectation that because it's programs that it's inherently unsafe and a user needs to trust the source. Our issue is phrases like "global themes" or "Plasma applets" don't always carry this message.

The tech world has changed a lot over the past decade and whilst our code hasn't changed, users expectations have. More and more places provide well kept walled gardens where most actions accessible via the UI are safe-by-default - or at least claim to be!

I've also seen confusion that because a lot of our UI is written in a higher-level language (QML) that's enriched with javascript all browser sandboxing automatically applies. Even though that's not what we claim.

But ultimately if there is a gap in expectations, that's on us to fix.

What can we do better?

In the short term we need to communicate clearly what security expectations Plasma users should have for extensions they download into their desktops. Applets, scripts and services, being programs, are easily recognised as potential risks. It's harder to recognise that Plasma themes, wallpaper plugins and kwin scripts are not just passive artwork and data, but may potentially also include scripts that can have unintended or malicious consequences.

We need to improve the balance of accessing third party content that allows creators to share and have users to get this content easily, with enough speed-bumps and checks that everyone knows what risks are involved.

(UI from Flathub for potentially unsafe content)

Longer term we need to progress on two avenues. We need to make sure we separate the "safe" content, where it is just metadata and content, from the "unsafe" content with scriptable content.

Then we can look at providing curation and auditing as part of the store process in combination with slowly improving sandbox support.

Do I need to do anything as a user?

If you install content from the store, I would advise checking it locally or looking for reviews from trusted sources.

If you're a sys-admin you might want to consider adding the following to block users installing addons with the following kiosk snippet.

/etc/xdg/kdeglobals

[KDE Action Restrictions][$i]
ghns=false

There are release parties around the world, the UK has a party in Cambridge.

Action plan is to go to The Haymakers, High Street, Cambridge and get pizza, drinks and more pizza. at 19:00 Thursday the 29th Feb.

Look for the guy in the blue KDE T-shirt.

Sign up at https://community.kde.org/Promo/Events/Parties/KDE_6th_Megarelease#Cambridge

What the Alpha means

The alpha release primarily focuses on preparing our software for a future release. It involves handling unreleased dependencies, version numbers, co-installation conflicts, and all the relevant bookkeeping work.

This release has been somewhat manic, with issues surfacing up to the last minute. However, that's precisely what this early release is for: resolving these issues now and gathering feedback on packaging to ensure a smoother transition to the beta phase.

Feature Freezes

The complete feature freeze for Plasma is scheduled for the day of the first beta, which is on November 29th. After that, bug fixing will be the sole focus for a period of three months leading up to the final release.

A soft freeze is set for the week before, on the 22nd, to accommodate any significant changes and ensure a seamless beta release.
This is mostly a case of doing a final round of landing straggling merge requests rather than developers starting anything new.

Should I run it?

Plasma 6 is in a pretty good state; I've been using it as a daily driver without issues for months

The alpha release does have known issues, some already fixed, but unreleased with the pending Qt 6.6.1, some our side fixed since alpha tagging, and some we need to follow up, particularly in the more esoteric areas of Plasma.
If you're the sort of user that wants to help out Plasma and are of a skill level where you're happy to log into another desktop session if things are temporarily down.

I would recommend as a user finding a distribution that covers 'git master' builds rather than any snapshot as it can provide a more dynamic list . A list can be found at https://community.kde.org/Plasma/Plasma_6#How_to_use/test_it

Pre-upgrade steps

Please take a backup of ~/.config/plasma* before upgrading. Just in case you need to file bug reports about config migration from 5.

Can I get involved?

Absolutely! It's an exciting time for Plasma and KDE in general. There are numerous tasks you can dive into. Check out our onboarding wiki here: https://community.kde.org/Get_Involved

Every release has a killer feature. Qt 6.6 features the opposite - staying alive. This blog post describes work to make Qt clients more robust and seemlessly migrate between compositors, providing resistance against compositor crashes and more.

Prologue

Right now if you restart pulseaudio your sound might cut out, restart NetworkManager and you lose your wifi, restart an X11 window manager and your decorations disappear.

But within a second it's all back to normal exactly where you left off with everything recovering fine.

This isn't true for display servers. If X11 restarts you're back at the login prompt. All drafts lost, games unsaved, work wasted.

Past

For X11 this was unfixable; clients relied on memory stored by the Xserver, they made synchronous calls that were expected to return values, and multiple clients talked to multple clients.

This was a real problem in my early days of Linux, X11 would lock up frequently enough that most distributions had a shortcut key to restart the server and send you back to the login prompt.

It's less of an issue now as X11 has been in a lengthy period of feature freeze.

Present

Wayland makes it possible to fix all this. Memory allocations are client side, all operations are async, all protocols are designed that the compositor always has complete control.

Yet the current user-facing state is considerably worse:

• Compositors and displays servers are now the same process, doubling the space for errors

• Compositors are typically extensible with 3rd party plugins and scripts

• The wayland security model means the compositor absorbs even more functions from global shortcuts to screencasting and input-method handling

• The wayland ecosystem is not in a period of feature freeze with wayland protocols constantly evolving to cover missing features and new ideas.

  Even if there was a perfect compositor:

• 40% of kwin's crash bug reports are either upstream or downstream causes

• The current compositor developer experience is limited with developers having to relogin and restart their apps and development setup just to test their latest changes.

Plan

The solution for this? Instead of exiting when the compositor closes, simply...don't!

If we could connect to a new compositor we just need to send the right amount of information to bring it in sync and notify the application code of any changes to bring this in sync.

For Qt applications all this information is handled in the backend, in the Wayland Qt Platform Abstraction (QPA).

Qt already has to handle screens and input devices being removed, clipboards being revoked and drag and drops cancelled. Supporting a whole reset isn't introducing any new work, we just have to trigger all of these actions at once, then reconnect to the newly restored compositor and restore our contents.

Applications already have to support all of these events too as well as handle callbacks to redraw buffers. There's no changes needed at an application code level, it's all done as transparently as possible.

Handling OpenGL is a challenge, right now we don't have a way to keep that alive. Fortunately we put in lots of effort previously in supporting GPU resets throughout the Qt stack. The QtWayland backend fakes to the applications that a GPU reset has occured through the Qt abstractions.

For the majority of applications including all QtQuick users this happens automatically and flawlessly, building on the work that we put in previously.

Whilst the overall concepts might sound invasive, the final merge-request to support all of this for all Qt applications was fewer lines than supporting middle-click paste.

Almost no work needs doing on the compositor side. For a compositor there's no difference between a new client, and a client that was running previously reconnecting. The only big change we made within Kwin is having a helper process so the whole process can be seemless and race-free.

Proof

Path Forward

This post is about Qt, but the world is bigger than that. Not only does this technique work here, but we have pending patches for GTK, SDL and even XWayland, with key parts of SDL merged but disabled already.

The challenge for the other toolkits is we can't use the same OpenGL trick as Qt. They either lack GPU reset handling either at a toolkit level or within the applications.

Supporting OpenGL requires new infrastructure. We've tried from the start to get some infrastructure in place to allow this.It was clear that proposing library changes for client reconnection support was an uphill battle whilst being unproven in the wild.

After Qt6 is rolled out to a wider audience and shown to work reliably, we'll refocus efforts on pushing upstream changes to the other platforms.

Potential Perks

This isn't just about crashes. If support becomes mainstream there will be additional opportunities:

• We can easily upgrade to new features without having to log out and back. This is one of the reasons why kwin development is happening at such a rapid pace in recent months. We're not having to test in fake nested sessions or waste time logging in and setting up between changes.

• We can support multihead, running different compositors per group of outputs and move seemlessly between them.

• It's feasible to switch between compositors at runtime. With the application handling the reconnect logic, they can easily handle the case where compositor feature sets vary.

• Checkpoint restore in userspace, being able to suspend your application to disk and then pick up where you left off like nothing happened. It could never work for X applications, but with wayland reconnect support we can make it work.

Participating

More information about the long term goal this can be found at the kwin wiki, or each out to me directly if you want to add support.

Discuss this at discuss.kde.org.

About

What are Input Methods?

Input Methods are a way of allowing third parties programs to manipulate text in text input boxes in applications.

The two traditional uses of this are on-screen keyboards like one has on a mobile phone, and Chinese, Japanese and Korean (CJK) input where a keyboard doesn't map directly to the text we want to see on screen.

It's a lot more than merely injecting keys; the input method needs to have an awareness of the text cursor and the ability to manipulate whole words and sentences at a time. This is extremely important for the CJK way of typing where multiple keyboard keys are used to compose a single character. It's also important for virtual keyboards that have advanced features like auto-completion and spell-checking words.

X vs Wayland

On X11, the standard protocol (XIM) is hardly used, we ended up with every input method creating a plugin for every toolkit. This sufficed, but was not very scalable, doesn't have the best desktop interaction, and is messy to deploy.

The story with wayland is also far from perfect. With the compositor acting as a broker, we have a protocol from the plugins themselves (InputMethod) and protocols to clients (TextInput). Unfortunately there are at least 6 active versions of these protocols, which needs some work.

A recent meeting with other wayland devs has hopefully helped agree on a path forward for the TextInput side, with me taking on a lot of active tasks moving this forward. But we still need to sort out the InputMethod side.

I used Akademy to speak to people using CJK input and got very valuable feedback about the Korean language and some changes needed to make FICTX work properly there.

If you have experience with Chinese or Japanese input on wayland, please do reach out.

A Playground of Input Method Ideas

I don't use CJK keyboard entry nor a virtual keyboard on a daily basis, but there are things I do use that could benefit from the InputMethod tech. Now we have a standard we can look at using this stack for more creative ideas. Doing this now can help work out what the requirements are for the protocols as we stretch it in new directions.

This post describes some ideas of how else we can use Input Methods to improve the Linux desktop experience.

Convenient Clipboard Connections

With a shortcut we can enter clipboard entries, previewing in place. We're able to use the preedit text to display metadata on available entries which won't get committed to provide a streamlined interface, or we could bring up a menu, or both.

Compared to the existing klipper integration we can open at the current carret position, and provide something a bit more streamlined. The preview also helps solve the frustrating problem of pasting only to find you've now got too much or too little whitespace where you hit paste.

Easy Emoji Entry

My attempt at understanding young people. Typing ':' and then a string launches an inbuilt emoji browser based on the typed string.

These days a lot of clients now have in-built support, including all GTK4 text areas, so it might not be needed. Maybe the ship for this has sailed, or there could be still value in centralising it and having a common pattern.

Direct Diacritic Display

Building on an existing merge-request by Janet Blackquill, pressing and holding a key will bring up a prompt with relevant characters based on the held key.

The original version merge request injected itself into the application repurposing a hook meant for themeing, this broke too many clients to be released as part of Plasma and it was unfortunately reverted.

Taking that idea but using input methods should avoid those problems, whilst also solving the issue of key-repeats being sometimes needed.

Timely Translation Tasks

This was solved by having a second text field open when activated where the user can type their native text, so the translations can be shown in the main text area. The current code utilises the command line tool "trans" which in turn makes web requests to proprietory web services, but we could look at other options if we turned this into a product.

Simply Speak

The one I'm most excited about. Working local dictation software that can integrate into every textfield out of the box.

I got best results using whisper-fast with a voice-activity filter on top. This was wrapped into a daemon with a DBus interface we can then use to start and stop on demand. It needs some work to be a shippable product.

Current State

All of the above are fully working examples, but everything was pieced together in a few evenings at Akademy and some time at the airport.

It's not something I would ship to users; the point right now is to be a playground, uncover issues throughout the stack, and try a few different things. That's why we use different techniques for bringing up the input methods and prompts within it rather than trying to be coherent.

That said, I have made an effort to separate the high level application code from the wayland boiler plate so we can quickly iterate to InputMethodV3 to see what's needed, and turn this into a product as we start to come up with a long term direction.

Running it for yourself

The repo exists at https://invent.kde.org/davidedmundson/inputmethod-playground the readme has instructions on how to get set up.

What does it work with?

A form of TextInput is supported in almost all toolkits from Electron, GTK, Qt, SDL and more. This means it's fairly universal across wayland clients.

On the compositor level, InputMethodV1, the only specification in upstream protocols, is supported by only Kwin and Weston, but as noted in the intro this is subject to change in the future.

Future

The playground has exposed several pre-existing deficiencies, especially regarding the placeholder text styling, input panel positioning, key repeats. We can take those forward into fixes that might affect the more important CJK input methods.

As well as being a playground I think there are some ideas worth exploring further, to find a path into the desktop and is yet another reason to be excited about the path to Wayland,

Have more ideas? Let us know in the discussion at http://discuss.kde.org.

This blog introduces a new tool to make it easy to stream wayland windows and screens to existing applications running under Xwayland that don't have native pipewire support in an easy-to-use manner than retains full user control throughout.

Intro

On my Plasma wayland session right now if I use the screen share function in Discord, I'm presented with the following.

It doesn't work

I have no windows listed, even though I clearly have windows open. The full screen shares are blank.
The same will be true in MS Teams, Slack, Zoom as well as any other Xwayland apps that involve screencasts.

Linux enthusiasts - and by reading developer blogs you're definitely in this camp - will understand why this is and probably know some technique to avoid this involving changing my setup or workflow in some way to work round things on a per-app basis.

For our wider userbase this isn't enough. Wayland is a technical detail and we want any switch has to be as transparent as possible for as many apps as possible for all cases - including cases we haven't thought of.

Introducing XwaylandVideoBridge

With our new tool, written by Aleix Pol and myself, running the workflow is as follows: I click on the share button. Immediately I'm presented with an option to choose what windows or screens I wish to make available to X.

This is now selectable in my existing application. The stream continues until I stop sharing within the application.

Left: a prompt to choose windows to stream.
Right: How it appears after selection

Security

The bridge works via the same mechanisms as any "native wayland" video streaming tool would work, through the XDG Desktop Portals. Getting data through PipeWire as requested it through the portal with explicit user consent.
Whilst the bridge does mean that any X application could eavesdrop the shared window the user still remains completely at the helm of which windows are shared to X applications and most importantly when.

We could go the route of trying to be completely seamless to the X client with N fake windows all forwarding content on demand, but I like the demonstration that it's possible to not break existing user applications without having to compromise on our lofty wayland goals.

Performance

Technically there's an overhead, pragmatically it uses no CPU and any latency is negligible compared to the cost of streaming.

Installation

Grab a flatpak from: Our gitlab builder or from source.

Note it requires a non-released libkpipewire, something the flatpak resolves.

Whilst only tested on a Plasma wayland session, this should be usable by any Wayland desktop with a working xdg-desktop-portal, screencasting and standard system tray.
(Edit March 24th: User testing has shown some mixed results on Gnome with colours, and completely not working at all Sway)

Usage

Ensure our proxy is running in the background flatpak run org.kde.xwaylandvideobridge optionally setting it to autostart. After that everything should kick in automatically the next time an Xwayland application starts streaming.

How it works under the hood

The inspiration for this came from the debug tool to show pipewire streams in a window whilst we were working on Plasma remote desktop support. If we force that debug tool to run as an Xwayland client, it becomes visible to other Xwayland chat / streaming programs. We had 90% of the code already.

The only remaining step was some sneaky tricks to hide this X11 window from the user's view making it unfocussable, invisible and out of view. We then added detection for when we're being streamed by using the XRecord extension to monitor all clients redirecting the window we own.

It's an excellent example of X11 allowing you to do really, really stupid things, for novel and creative puposes.

Future Plans

This is only an initial Alpha release. How we take it in the future is not completely decided; it might remain a standalone tool moving to flathub or distros, we might proposed it into Plasma 6 by default. There's a possibility the Linux desktop might be at a point where this is redundant.

There's definitely some minor tweaks still to do on the exact workflow.

Please do give feedback and let us know!

http://disuss.kde.org thread