Inside the self-tuning “Genetic” Linux

Although it’s been around for a while the genetic library and its associated kernel plugins by Jake Moilanen have again surfaced in some of the more recent cutting-edge kernels.

New kernel features come and go and certainly not all get to stick around in mainline. Genetic isn’t there yet either and it may never be but it’s become increasingly more interesting as its development progresses.

Or maybe I’m just fascinated by the concept of a kernel that can tune itself.

The purpose behind ‘Genetic’ in Moilanen’s own words is “that sys-admins and users never have to explicitly tune their system but have it done autonomically.”

It does this by monitoring the kernel workload and adjusting kernel parameters for (hopefully) optimal performance. It doesn’t just do this via pre-defined tables of variables though there is work being done in this area but by applying genetic evolutionary theory (in the CS sense) to the kernel’s performance adjusting values on the fly and seeing how well they enhance or impact its operation. Values that work are ‘evolved’ further measured again and the process repeats. The idea is that eventually after a certain amount of uptime and/or consistent loads the algorithm has determined the optimal kernel parameters for a given workload.

Why not just tweak these yourself with ‘known good’ values? Because literally the array of kernel tunables combined with the vast array of application loads a kernel can process is limitless. And then naturally the load on the kernel changes and the optimal configuration with it. Of course known good values are close enough which is why the kernel ships with one size fits most values.

But this is the premise behind the genetic libraries — to let the kernel do the work for you. That said nothing in life is free and the overhead of the genetic library and associated plugins can as much they as improve performance also impact it adversely. It’s all dependent on the load.

Currently Moilanen has released the core genetic library and two kernel plugins — one for the Zaphod CPU scheduler and one for the Anticipatory I/O scheduler the default I/O scheduler in the Linux kernel.

So does it work? Well I’ve only just installed a kernel using the genetic anticipatory scheduler (see below) and to be honest I’d be hard pressed to be able to quantify a noticeable improvement — but my gateway box isn’t exactly loaded spending most of its time shuffling packets and serving files. For other results Moilanen has a summary of his own observations on the Genetic library homepage.

Speaking of which one of Moilanen’s planned updates is applying Genetic to the kernel’s packet scheduler. I don’t know why but this sounds fascinating to me — imagine a gateway box (and ultimately built-in Linux based routers) being able to tune kernel (and presumably TCP/IP) parameters on the fly based on Layer 7 classified traffic: different settings for VoIP games with UDP P2P file sharing browsing and so on all dynamically implemented on the fly. Of course it may also make no perceivable difference. But hey it’s there.

I am yet again in awe of the kernel developer community and the innovative and sometimes diverse ways the kernel develops. If you want to experiment the latest version is included in Emission sources 2.6.17-emission8.

Kernel side: the genetic anticipatory I/O scheduler