Kernel Engineering

We look inside the heart of the operating system, and find out why the kernel is hard to optimize.

It’s all too easy to think of Windows or Linux as operating systems that just work when we throw our demands at them. We know that both are designed to handle server and desktop environments, and though logically you’d assume these to be contrary, under the hood the operation of the kernel to support them can be substantially different.

It’s all about latency
If you think about it, for each and every slice of CPU time, your operating system can do only one thing. The magic that is multitasking is the brilliant management and execution of commands from multiple processes to run them, as seamlessly as possible, side-by-side. But when it comes to making sure an operating system performs as required for the two core environments of server and desktop, how the kernel does this makes all the difference.

Microsoft has made lots of noise about the advantages of Vista. When you look under the hood at just the kernel changes you can quickly see how Microsoft has worked hard on the end-user desktop experience, and also the extent to which XP, and Windows versions prior to this, have all been slowly working on the problem. And that it still isn’t completely there yet.

Linux is no different and, in many ways, is worse off. The brain boxes behind Linux kernel development are masters, but for almost two decades now the Linux kernel has been engineered and finely tuned for the market where it first got a foothold: servers. The focus in recent years on Linux for the desktop is great, and much of the work that’s been done has been on the user space (witness Ubuntu and the many changes it makes on the desktop to make Linux accessible). But under the hood, all Linux distributions are still running a kernel that was essentially designed for a server, not a desktop.

This expresses itself in the problems some users face. For example, stuttering audio playback while accessing a disk, or slowed GUI window movement under high processor loads. The problem isn’t that the kernel can’t handle the workload, it’s how the kernel has been designed to handle it.

It’s a complex problem for Windows and Linux. How can you get optimum performance under both these two core environments when, for the most part, they are largely opposed?

There’s a popular maxim that states server and desktop requirements roughly correlate to a sliding scale between throughput and latency. Servers are big on throughput, while desktops are big on (low) latency. And the more you get of one, the less you get of the other. While this is true in principle, the issue can really be broken down to just latency, as lower latencies tend to equate to lower throughput.

For servers, throughput and getting work done is usually more important than latency, as well as ensuring that services (be it file, web, networking and so on) get their fair share of time in the sun. For desktops it’s somewhat the other way around – low latencies mean responding quicker to the user, and giving some tasks higher priority at the expense of others (effectively an ‘unfair’ workload) can mean a more seamless user experience where it counts.

Consequently, there is actually quite a lot that both Vista and Linux do at the kernel level to provide a better experience for the desktop user.

Vista
Microsoft is well aware that the Windows XP, well, experience didn’t turn out to be quite the revolution it promised. In light of this, Vista’s kernel underwent a number of upgrades to help change this, including:

Multimedia class scheduler service: A new service boosts thread priorities of multimedia applications to try and prevent glitches in video and audio playback. Threads are boosted for 80% of the task’s clock rate, then dropped back to ensure other tasks can run.

IO cancellation: Many IO tasks can be prematurely cancelled, and this applies to network connections as well. So no more frozen Explorer when a network device times out.

IO prioritisation: Previously only CPU threads could be prioritised, but now certain IO-intensive jobs (like virus scanning) can be given a lower IO priority. There are five levels from Critical to Very Low. To ensure low priority tasks still get a look in, at least one low priority IO is processed every second.

IO bandwidth reservation: Similarly, some applications such as media streaming can request minimum bandwidth to ensure
interrupt-free IO.

SuperFetch: Preemptively populates memory with data the kernel expects the user to need, thus providing faster response times, and uses IO prioritisation to ensure this doesn’t interrupt the user.

ReadyBoost: You know all about this one. It’s most beneficial for low-memory systems.

ReadyBoot: Analyses the last five boots and devises an optimised startup plan to reduce boot times. A RAM cache is dynamically built and destroyed on demand to facilitate this during boot. The end result is a snappier boot up, personalised to taste.

Parallel sessions: To further improve boot times, startup sessions can be launched in parallel during boot.

Delayed autostart: Some services have delayed start times, such as Windows Update, to allow faster logins. Again, these can take advantage of a low-priority IO assignment.

All this is rather nice if you’re running Vista, and shows just how seriously Microsoft takes the importance of a faster and more responsive desktop experience.

Linux
Linux, as hinted at earlier, has come a long way in terms of user-friendly desktop-focused distributions. Unfortunately, as all of them still run the Linux kernel – which is certainly an efficient kernel – they don’t have heritage in the desktop space. In recent years a stronger focus for the desktop has led to a number of changes. Some of these have been around for a while but have been tweaked, while others – like CFS – are brand new.

Preemption model: The kernel has three default levels: None, Voluntary, and Preemptible. None is ideal for servers and maximises throughput; Voluntary allows low-priority processes to preempt themselves to allow applications to run more smoothly while under load; and Preemptible is similar to Voluntary but with a response time in the milliseconds that’s good for real-time systems.

Timer frequency: The default clock frequency within the kernel. The old 2.4-series kernels defaulted to 100Hz, but the 2.6 series has a wide range of options from 100Hz to 5000Hz+. The faster the timer frequency, the quicker the kernel will respond to interactive events. 100Hz is recommended for servers and 1000Hz is considered ideal for desktops. In typical kernel developer humour, the kernel configuration dialogs note that anything above 1000Hz is ‘Insane’ and anything above 5000Hz ‘Obscene!’

CFQ: The Completely Fair Queuing IO scheduler is a relatively recent development and ensures balanced and fair IO access ideal for desktop systems. By comparison, servers will usually fare better with the Deadline IO scheduler, while the original Anticipatory scheduler sits somewhere in-between.

CFS: The Completely Fair Scheduler is brand-spanking new as of the 2.6.22 kernel and aims to address some of the CPU scheduling issues that can impede desktop performance. Despite its title, the CFS will actually priortise some tasks to ensure a fast and responsive desktop and applications, however it also ensures that programs can never completely starve – just as detrimental to performance.

Re-nicing: Programs have always been able to be re-niced (Read: Setting the ‘niceness’ level, aka thread priority) in Linux, however some recent kernel patches have taken to automatically re-nicing the X processes to prioritise the Linux GUI, further improving desktop responsiveness.

As with Vista, these are just some of the changes under the hood and, especially in the case of CFS, still under development.

Looking ahead
The illusion of multitasking and responsive desktops are just that, an illusion. Even in the age of multi-core CPUs, these rules still apply. But as Linux kernel developers place a stronger emphasis on desktop performance, and Microsoft continues to evolve its focus on the same theme, we will see further changes to the heart of our operating systems to enhance the desktop experience, especially as new technologies (like high-definition media, security, new hardware peripherals and so on) evolve.