Help fix up all of System Settings

System Settings is a strange part of Plasma; it composes of umpteen modules gradually written over the last 15 years by a lot of different people.

As each configuration module is generally small and self-contained it doesn't take a lot of continual maintenance so when a developer becomes inactive we tend not to notice.

Unfortunately over time we've seen the number of open tickets on system settings modules increase over time. They get left unsorted as it's "someone elses problem".

This leads to things falling between the cracks, particularly for finding the Plasma 5 / Frameworks regressions as we can't see the real issues through the noise of all the duplicates, outdated bugs and frankly bad ideas.

Whilst I'm doing Plasma this isn't a trend that is going to continue.

I've spent the last week of evenings triaging system settings tickets and fixing up some of the most important bugs.

I need some help on going through the remaining bugs, my personal target is to get the total number of open tickets down to 400 by the end of February.

If you want to help out join us in #kde-quality on Freenode and we can co-ordinate some triage and split out some of fixing, making Plasma a polished experience once again.

A list of bugs split by component can be found here.

How does systemd relate to Plasma?

In some of the comments about the latest Plasma release sometimes someone brings up the topic of systemd. This is normally met with the rebuttal "Why should a desktop environment care about what init sytstem is used?".

It's a very sensible question, but it's also one that's easily answered. I wanted to explain how different parts of systemd relate to Plasma.

The init system

We don't care. It doesn't affect us.

The init system is one part of systemd that doesn't affect us at all, and any other could be used.

Logind

Logind is a tiny daemon that keeps track of seats and sessions on your machine.

In principle it's very simple, every login (including autologin) goes via PAM (Pluggable Authentication Modules) modules; a special PAM module signals to a central daemon that a new session is started and tell it when it stops.

This blog series has the most detail on session tracking and why logind solves problems better.

We need knowledge of all active sessions from within Plasma to be able to offer session switching inside the UI and to warn the user if they attempt to shutdown whilst other sessions are still active.

This class kdisplaymanager in plasma-workspace shows our abstraction layer growing since 2004; it currently has Logind support but as we have been adding to a constantly broken abstraction layer it's used very badly. It has tracking code for over 5 different systems (KDM, old GDM, new GDM (which is now old), consolekit, org.freedesktop.DisplayManager, and now logind) and is probably one of the ugliest pieces of code in Plasma.

Logind does seem to solve these problems better, particularly with regards to multi-seat and the other features it provides.

We need to tidy this area, having an abstraction layer only leaves us with a lowest common denominator, and it isn't pretty.

When people reference standards, someone will cite XKCD, I think more of it like this:

Device Management

Logind allows the session leader to open input devices which are otherwise only available to root.
This allows kwin to monitor key and mouse events whilst blocking other applications from sniffing your keystrokes.

Martin Graesslin's blog explains this in more detail.

Inhibitor Locks

One other feature logind provides that we are making use of is inhibition locks.

Upower will notify us when the system is about to be suspended. Which sounds ideal, we should be able to react to that and perform pre-suspend tasks. Unfortunately it's not very useful as the system is shutdown before it finishes.

When logind is asked to suspend, it will inform all applications which have requested a delay. It will then wait for up to 30s for all apps to respond that they have finished their pre suspend tasks.

This allows us to make sure the lock screen is completely up and running before we suspend the system; so you don't have that jarring and insecure moment where you open your laptop and it is still showing the original contents before the lock screen finally loads.

We even use this feature from Telepathy, the instant messaging system. In order to save bandwidth most IM protocols aren't constantly sending data The problem is when we disconnect it will take a long time for the server to notice and update your status. We have an inhibition delay that gives us a very small window to shout to all the servers "we're going offline now".

Timedated and Friends

Systemd also comes with a set of utilities for performing some system administration utilities, such as changing the hostname, timezone, locale and such.

What makes these interesting is that the components are split into two parts; a small command line client but more importantly a defined DBus interface to a small DBus activated daemon that uses polkit for authorisation and does the actual changes.

This approach is similar to what we do in our date and time config module; we have our GUI and we also have a small helper app that's DBus activated and runs as root which we are shipping ourselves. Our helper app then runs one of 6 different ntp programs; almost all of which are now outdated.

We don't want to be writing code that runs as root and as a user you really don't want us to be doing so.

By using a defined DBus interface we are able to share this code between desktops and share safe audited code for all root access.

Naturally this not only applies for timedated, but also hostnamed, machined and whatever else might come in the future.

Change Monitoring

In addition timedated, the systemd interface for changing the time, emits signals whenever it changes the timezone.

Currently in order to achieve the same effect we have a constantly running daemon that monitors /etc/localtime for file changes and then emits DBus signals for all the other apps.

If we could rely on everyone setting the timezone through timedated we can remove this watch which will free some resources for everyone.

What else could we make use of?

There are a few other places where embracing systemd adds functionality or makes code simpler; our system guard can show which seat and which container an application is running on. I already wrote a patch for this, but got bored #ifdef-ing everything.

Using a common storage for log files means ksystemlog could be restored to a working product again.

User Units

Last but not least we have user units.

Currently init is handled by a rather cumbersome shell script. It's as simple as starting the KDE services in order in a list.

User Units allow Plasma services to use the same features as available to system daemons.

Most obviously we get the faster bootup as we are able to parallelise component startup, but there are more benefits.

Systemd provides services to restart crashing applications with finer control, a watchdog can detect a hung application or if a process gets out of control using too much resources and restart it automatically. Logging will also be greatly improved; faster to run and easier to supply developers with relevant information instead of a giant .xsession-errors file.

A trial has been started here providing user units; which when ready will work their way into the main plasma-workspace.

Longer term, we can investigate replacing parts of kded, the KDE daemon with systemd directly, providing a single common way to manage all services.

Conclusions

Hopefully it clears up what we mean when we talk about systemd and desktop environments, and where we could use different parts of systemd.

It should be apparent that as developers there are parts we want to embrace as it. In many cases it allows us to throw away large amounts of code whilst at the same time providing a better user experience. Adding it as an optional extra defeats the main benefit.

As maintainers we have a duty to balance what will provide the best experience for the majority of our Plasma users without leaving anyone with a broken system. Projects like this bring the interfaces we need to BSD and as it gets more stable we should be able to start distributing features.

Plasmoid Tutorial 1

With Plasma 5.2 out I wanted to update the tutorials on how to write a Plasmoid. Going through all of the steps from hello world, to using Plasma Components to configuration through to differing form factors.

It made sense to publish them as blog posts before I copy them to the wiki.

Behold, the first rather wordy blog post in a series of 7.

Intro

With Plasma5 we have embraced QML as our technology of choice. It is the only method of writing the UI for plasmoids.

Whilst Plasma4 offered a range of language, QML is the only way of interacting with QtQuick, the technology that powers the Plasma Shell. By using this we we get to provide a better developer experience as there is a wealth of existing QML resources. Some of the best links are:

Before you get started with writing plasmoids, it is recommended to read through the basics of these and have a bit of playing to get used to the language.

Writing plasmoids is effectively the same as writing any other QtQuick QML, with a few extensions:

  • We have a specific folder structure with metadata for our plasmashell to be able to load the files.
  • We provide a massive collection of libraries that extend the QtQuick library both with functionality and widgets that blend in with the Plasma theme.
  • Special API exists to interact with the shell, allowing you to save configurations or set default sizes.

In this series of tutorials we'll go through the steps of writing a real plasmoid from scratch, using some of the plasma libraries.

By the end we should have a completely working, deployable RSS reader.

Hello world

Initial Folder Structure

Plasmoids follow the simple KPackage structure. In the top-most folder there should be a file titled metadata.desktop and a single folder called "contents".

Inside the contents folder we place all our QML, assets and other additional files. We split the contents into subdirectories: config, data and ui to make things easier.

In our tutorial we will be making an RSS reader so everything is named appropriately to that.

The basic directory structure should be as follows:

myrssplasmoid/metadata.desktop
myrssplasmoid/contents/ui/main.qml

metadata.desktop

[Desktop Entry]
Name=RSS Plasmoid
Comment=Shows RSS Feeds
Encoding=UTF-8
Icon=applications-rss
ServiceTypes=Plasma/Applet
Type=Service
X-KDE-PluginInfo-Author=David Edmundson
X-KDE-PluginInfo-Email=davidedmundson@kde.org
X-KDE-PluginInfo-Name=org.example.rssplasmoid
X-KDE-PluginInfo-License=LGPL
X-KDE-PluginInfo-Version=1.0
X-KDE-PluginInfo-Website=http://techbase.kde.org
X-Plasma-API=declarativeappletscript
X-Plasma-MainScript=ui/main.qml

Most of the fields here should be fairly self explanatory.
If in doubt, copy this and change the relevant fields.

main.qml

import QtQuick 2.0

Item {
    Text {
        anchors.centerIn: parent
        text: "Hello World!";
    }
}

Providing you have followed the recommended reading this should be fairly self-explanatory, we have a text item in the middle of the plasmoid which will say "Hello World".

Over the next few tutorials this will get somewhat larger, we will also see some of the problems with this example; here translations aren't implemented and the text won't match the Plasma theme colour.

Installation

From the directory above run

plasmapkg2 --install myrssplasmoid

It should now be visible in the plasmashell in the "Add widgets" option along with every other plasmoid.

We can then it to our desktop or panel like any other installed plasmoid.

You will see that that the border and handles are automatically added, and that it is automatically sized to fit the implicit size of the text.

Next

Next tutorial, we will cover getting data from external sources.

Plasma - Calling Qt 5.4 Testers

Plasma 5 pushes QtQuick to the limits. It sounds like a cheesy marketing line, but it's true. Unfortunately this isn't a good thing. Although Plasma 5.1 is somewhat stable we have had some crashers, and whilst we've worked hard to fix the ones that are ours a sizeable number of these were caused by problems deep inside Qt.

Fortunately Qt 5.4 has just been released. It contains countless bug fixes that really improve the situation; several of which were even written by me and the rest of the Plasma crew.

We need people with Qt 5.4 (hey all you Arch people) to help go through all open crash reports and test if they still happen since upgrading.

I don't like closing bugs wtih a vague "it seems to work for me" without getting at least a second opinion, I may be overlook something and it's not fair on the original reporter.

I've added a quick link to the high priority candidates

And feel free to help go through the rest of our list

So far everything is looking very positive towards having a completely rock solid Plasma 5.2 in January; lets make it happen.

HighDPI in KDE Applications

Recently I was asked about the state of High DPI in KDE Applications, by someone with a fancy screen.

In Plasma we have our own solution for solving the HighDPI problem, as we were working with new code where we provide the styling, this is all fairly straightforward. However, this doesn't work for applications which have a lot of existing code we need to bring this feature to.

This upcoming release of Qt (5.4) brings us everything we need to support High DPI in our applications. It's not going to be useful for end users just now, but this is a time where we need each and every developer to start getting interested and making sure their applications are ready.

Support requires at least one line of change in every application.

High DPI in Qt

Normally my application, will look something like this. On my screen this is fine,if you see this on a high DPI screen everything will be tiny.



If we change the DPI used in the fonts we, naturally, get bigger fonts. More usable, but all of the other controls are still hilariously small as they haven't changed. This is our current state.



Device Independent Pixels

Qt borrowed a concept used in Android and iOS of separating the pixel sizes we use in the code, to the actual pixel density on the screen.

From a programming perspective we keep coding like everything is in 96dpi like a normal desktop display, but underneath all co-ordinates and graphics will be doubled or quadrupled to scale up to the device resolution.

This API at a Qt level works per screen; with every screen having it's own independent scaling factor. However, in the X backend for Qt 5.4 it's loaded from an environment variable that applies to all screens. Not ideal, but better than nothing.

This gives a result, that is now usable, but very blocky and pixelated.
(you may need to click the image to really see this)



This blockiness problem is also solved. Whenever we create a pixmap we can make sure we provide graphics that have a higher size in device pixels than their size in user space.

To quote the QPixmap documentation, "For example, painting on a 200x200 image if with a ratio of 2.0 will result in effective (device-independent) painting bounds of 100x100.".

With "pixel" now meaning one of two things can get confusing very quickly.

As you might imagine, all of this wizardry under the scenes means it's moderated likely to break at least some applications. As Qt follows the rule of never breaking existing application it requires each and every application to opt in.

I added a command line flag --highdpi to oxygen-demo, the application in the screenshots, to enable the scaling support and test both side-by-side which finally leaves us with:



There were no additional changes to oxygen demo to make this work yet we see we're using high resolution pixmaps on the icons and on the clear buttons in the text field.

How to make sure your app works with Qt's high DPI

1) Port to Qt5
If you're putting it off porting because you're scared of regressions. The reality is over time you will have more bugs from not porting. Without Qt5 we don't get /any/ scaling. Not even the blocky version.

2) Opt in to the high DPI pixmap support
Add:

app.setAttribute(Qt::AA_UseHighDpiPixmaps, true);

to your main function

3) Port code
If you don't do any low level work with unusual QPainter painting, provide any of your own pixmaps or load textures in QQuickItems, everything will just work.

Most likely your app will have something broken. Useful links are here and here.

4) Test
I only have a poor person's screen, but it's still easy(ish) to test:

  • Change the DPI under system settings -> font -> force font DPI to 192
  • Set the environment variable QT_DEVICE_PIXEL_RATIO=2

On a normal screen everything will appear massive, but hopefully also super smooth.

My new library: Qlogind

What is it?

A high level wrapper round logind.

We are starting to use logind in numerous places; I need it in SDDM to track sessions. we have code in ksmserver to track Lock/Unlock signals, as well as in the kworkspace library to list sessions. It's coming into KWin for device hardware access as well as being used in solid.

The need for a library

Often we can simply automatically generate bindings at compile time using qdbusxml2cpp that turns DBus interface annotation into working code.

Unfortunately that doesn't work here for a few reasons.

qdbusxml2cpp and QDBusAbstractInterface are both really outdated; and don't know how to handle org.freedesktop.DBus.Properties.PropertiesChanged. Logind uses this method as the only way to notify of any changes, and in fact even introduced their own new type of changed signal annotation without these change signals it becomes near impossible to use.

To solve this, and a few other minor issues this repository contains a fork of qdbusxml2xpp and QDBusAbstractInterface with property caching and no blocking methods anywhere.

Ideally I want to push my changes upstream, but I always like to prototype code before committing to new API especially with Qt. Also we're going to need these changes before Qt5.5.

There are some other usages for a library, there are some changes needed to make to the annotation to make it compile (logind has a property called "class", funnily enough the c++ compiler does not like this) as well as code to demarshall complex types.

From personal experience having a good wrapper library can make a lot of development easier as we can write a higher level API on top to ease fetching multiple queries asynchronously.

Example

I've tried to make the code as easy to use in an asynchronous way, without the user having to write chains of lambdas.

Example:

    PendingSession* ps = Session::sessionFromPid(QCoreApplication::applicationPid());
    QObject::connect(ps, &PendingSession::finished, [=](){
        SessionPtr session = ps->interface();
        if (session->active()) {
            qDebug() << "Session " << session->id(); << "is active";
        }
    });

This code internally both a call to Login1.Manager to find the session path for a PID then requesting Properties.GetAll on the Session object in one handy KJob like API.

Where is the code

Code is availabe at git://anongit.kde.org/scratch/davidedmundson/qlogind.git

Current State

It's not quite finished, I only started this 2 working days ago. It needs namespacing, d-pointers and unit tests.

I wanted to write a blog post to get the people who might need to use this to have a look and give some feedback if there's anything missing.

Display Managers In Plasma 5

The last blog posts about KDM/LightDM/SDDM/WhateverDM left things a bit on an exciting cliffhanger so I've been asked a few times what the current state is.

The short summary is we recommend SDDM as the display manager for Plasma 5.

Backstory

KDM was dropped from Plasma 5. KDM includes code from XDM dating back to 1988! It had served it's job well. However, we're now at a point where we need the backend to be Wayland ready and we want to use more modern QML in the front end. When you have to replace both the back and front ends, it's a sign to just start from scratch.

There was some work done 2 years ago into sharing code with LightDM. In the meantime a separate project was started, SDDM which is (yet another) display manager.

Although personally I was very happy with what we had with LightDM it definitely doesn't make sense to split resources, so we focussed everything on SDDM and I have been helping work on that transferring knowledge from my old project.

Plasma Integration

SDDM is itself in Github and usage is shared with other desktop enviroments, in particular Maui. This makes it harder to add Plasma integration in the code itself, but we've managed nonetheless.

The visual design group made some mockups of a new display manager, which became the basis for the implementation of our login screen.

This is shipped with plasma-workspace. It is up to distributions to update /etc/sddm.conf to update the default theme on installation as it's not something we can do from our code.

In addition we also provide a configuration module that fits into SystemSettings. This is now back in the KDE repositories and will be included in Plasma 5.2.

Naturally Plasma can still work with any display manager as half of the point of having a display manager is to allow choosing which session to launch.

KDE Telepathy 0.9.0 Released

Today we released the 0.9 series of KDE Telepathy, a multiprotocol instant messaging client for Plasma.

Amongst the many bugfixes the following features are worth highlighting.

OTR Encryption

One of the most recurring feature requests we've seen over the years was OTR encryption. OTR builds an extra level of encryption on top of the existing protocol embedding all data in the mesasges themselves. This prevents any potential sniffing from the server as well as providing perfect forward secrecy and non-deniable authentication

This summer Marcin Ziemski volunteered to step up to the task and implemented full support for OTR3, including shared secret authentication and key management.

A full video can be seen here

Group Chats

All aspects of group chats recieved a massive overhaul, there is a new join chat dialog, one can stay logged into a room when you close the window and we've redesigned the main chat window to be easier when talking to multiple people.

Video Calls Revisited

Video calls have had both a technical and UI overhaul. Diane Trout ported a lot of QtGstreamer and the CallUI to GStreamer 1.0.

This port allowing us to future proof for the new technologies and have more chance of working calls.

In addition Ekaitz Zárraga has rehauled the UI bringing it into the 21st century which can be seen in the screenshot.

We encourage you to try it out. Depending on your and your partner's setup, you might now have a good video chat experience. For best results use a real jabber server (Not Facebook or Google!) the same client and be sure to have all GStreamer codecs installed.

Speed Improvements and Bug Fixing

Naturally the release brings a slew of bug fixes and improvements not listed here. Specificially the contact list has undergone extensive profiling and optimising since 0.8.

Fundraising

You may notice some of the features were highlighted in the report from our previous sprint report. If you like the new features be sure to contribute to our winter fundraiser so we can keep doing sprints in future.



Is Plasma 5 Ready For "Normal Users"?

I keep seeing the same question on a lot of the social media sites so I want to write a blog post covering everything.

When we released Plasma 5.0 we had gone through a lot of large technical changes throughout the stack. It was expected that there are some regressions, either from bugs or simply some minor components not being priorities as being needed in the first release.

I'm a normal user, is it ready for me?

There is no such thing as a "normal" user. Everyone is different and what is rock solid 100% stable to one person may be very problematic for another. This makes it very hard to give a single answer.

A good example of this was translations. Due to some changes in frameworks when we released Plasma 5.0, we made a very poor job of translation loading and a sizeable amount didn't work properly. Most developers tend to run things in English even when they're across the world and it simply fell through the cracks till it was too late (fixed for 5.1).

If you're a user who uses KDE in English, this naturally isn't a problem that would affect you. If you only speak German this would be a complete showstopper that would make you absolutely hate the 5.0 release.

This same philosophy applies screen hotplugging or specific graphics hardware or RTL layouts. It might be a massive problem, you might find everything works perfectly.

To determine if Plasma 5.1 is "ready for you" the only real option is to try it.

It might seem like it would be a good idea to keep the project in beta until there are no regressions but under open development it simply doesn't work.

We don't get the level of developer and user interest we need to catch and fix all regressions without a release. There's a common mantra "release early, release often" for a reason.

It's also worth remembering that whilst we have some regressions, we also fix a lot of things and it's important that users can chose to reap those benefits.

When will it be in distributions?

Everyone is being very conservative with our 5.x release mostly due to historical reasons of other major releases; Kubuntu in a few weeks will release two ISOs one with classic KDE SC 4, and the other with Plasma 5.

When can I get rid of Qt4?

I've seen a few people suggesting that they can't upgrade as there aren't any applications released on Frameworks 5 yet.

We have put a lot of effort into making sure Qt4 applications fit in natively with Plamsa 5. We make sure system setting syncs as many changes as possible and we have even backported the latest theme so that applications fit in natively. It's near impossible to tell the two apart.

The original plan has always been that we will release applications slowly after the workspace is ported.
Trying to remove Qt4 applications immediately is not the plan, nor is it encouraged by us.

Wrap up

Personally I've been using Plasma 5 in development for 9 months, and I am so accustomed to any remaining issues that I massively prefer it over what we had in 4.x.

From 5.1 I would be comfortable recommending upgrading to users who use KDE casually at home.

Making Plasma better

If you want Plasma to be better as soon as possible remember to join us in #plasma on freenode and contribute.

Also we are starting our year end fundraiser more sprints will result in more fixes, faster.

KDE Telepathy 0.9-beta

We have released a beta of KDE Telepathy 0.9, and libkpeople 0.3.0

Features include:

  • OTR
  • Improved group chats
  • Modernised video chats, now based on GStreamer 1.0
  • Lots of fixes and speed improvements

Tarballs are available here and here.

If you're interested in developing and contributing follow our quick start git installation guide

Please report back any bugs so we can make 0.9.0 a great release.