Category Archives: Linux

SystemTap

Posted by Daniel on July 25, 2009 None comments

SystemTap is the Linux analogy to Solaris DTrace and is similar to the strace command, only much much more powerful. It effectively lets you set breakpoints in the kernel to monitor what your applications are doing. For example if I was worried that some application I’d written was polling way too often, I could ask SystemTap to output the number of times my application calls poll() or select().

To use SystemTap first you write a simple script, or borrow one from someone else. On a Fedora system you’ll fine some sample scripts in /usr/share/doc/systemtap-0.9.8/examples provided you have SystemTap installed. You then run the stap; command. The stap command immediately begins parsing the scipt looking for any tapsets that your script uses and if it does it includes them. It then converts your script into C code and compiles it into a kernel module. This kernel module is inserted into the running kernel and stap attaches to it. The kernel module stays in the kernel until it is cancelled by the user, it reaches an exit function or it encounters too many errors.

While SystemTap can be used to simply dump loads of data about what an application is doing in kernel space that is not its purpose. SystemTap scripts are able to drill down, extract, process and format the data its gathering. For example if you were trying to find out what files a process was writing to your disks could just output every single write call and print it out, or you could keep the statistics and every ten seconds print the top ten files written to. SystemTap is designed to help you filter out all the noise and monitor only what you want to monitor.

The simple way to get started with SystemTap is to download the Beginners guide or the Tutorial. On Fedora systems when you install SystemTap you’ll find the tutorial at /usr/share/doc/systemtap-0.9.8/tutorial.pdf. SystemTap skills are handy for system administrators and developers, so if you fit into those categories I’d highly recommend you check it out.

Random Thought: Where does /dev/zero come from and where does /dev/null go? What happens if you pipe /dev/zero to /dev/null?

The Truth Will Shock and Amaze You!

Posted by Daniel on July 21, 2009 None comments

Pick the correct headline:

  • Meteorologists Determine That Hell Has Frozen Over
  • Scientists Genetically Engineer a Pig Capable of Flight
  • Microsoft Releases 20,000 Lines of GPL code to the Linux Kernel

if you know the subject matter of this blog, or read what category this post is in then you’ve most likely figured it out already. Its true, Microsoft today offered a total of 20,000 lines of code. The code, containing three device drivers related to virtualisation, has been submitted for inclusion into the kernel.

This is hard for some people to understand, indeed some Linux users are so entrenched in their hate of Microsoft that they are calling this as a bad thing. I don’t think people should be dismissing this so quickly. While Microsoft hasn’t been entirely friendly towards Linux in the past, this represent a substantial shift in their behaviour. Its things like this and the Community Promise that gives me hope of a world where Windows and Linux can coexist.

Random Thought: They say: “Linux is only free if your time is worthless.” and I say “Windows is only free if your money is worthless”

Edit (23/07/2009): Infoworld has a good writeup about Red Hat’s reaction to the news.

The Windows Registry vs Gconf

Posted by Daniel on July 18, 2009 1 comments

I often hear people attacking GNOME and Linux because of the claim that they’ve emulated the Windows Registry in Gconf. While this looks to be true at first glance, looking a bit deeper reveals many differences that make the two into completely separate entities. Both GConf and the Registry have their own advantages and drawbacks.

Windows Registry

The Windows Registry was added in Windows 95 to replace .INI files that Windows 3.1 applications would often use. These .INI files would be littered all over the filesystem in whatever location their application decided to put them. .INI files were easy to edit. Requiring only a text editor they could be edited in both DOS and Windows. The Registry was created to keep all the system settings in one place. Before it was released the APIs were made public so that any application could store its data along with all the system data in one common place.

This seemed like a great idea. No longer did you have to go searching to look for the settings for any particular application. Simply open up Registry Editor browse to your applications keys and edit away, unless you accidentally hit some system setting. Because system settings are stored right next to application settings changes to the Windows Registry could render the system unable to boot. This is why you see all those disclaimers on any tutorials involving the Windows Registry.

The Windows Registry is stored in a binary form. This makes it quick to load but limits its flexibility. After a Windows system fails to boot it will ask you if you want to boot with the last known good settings. This basically uses a backup of the Registry made at the last boot-up, where the settings are known to be correct. Not attempt to edit it can be made without a graphical environment and your only option is to roll back to a backup.

GNOME Gconf

On the surface Gconf appears to emulate the Windows Registry. Both gconf-editor and the Windows Registry Editor look very similar, and indeed operate in practically the same way. They’re both a collection of keys and values in a tree form. They also hold some similar settings such as keys to hold the desktop wallpaper location.

Gconf differs significantly to the Windows Registry though. You’ll notice that there are no settings in gconf to configure swapping, tune the filesystems, or configure routing. Gconf doesn’t hold any system settings, it stores only settings related to the desktop environment. You won’t find anything in there to help you configure device drivers or render your system unbootable.

The backing data store for Gconf in XML, which allows editing (though I’ll admit its somewhat difficult) with any text editor, command line or graphical. In the event of you making a change that stops your desktop environment from starting you’ll still be able to boot into a command line mode and attempt to fix the problem. It is also easier to identify when searching for it in a crashed filesystem.

Comparison

Both gconf and windows registry both represent their data in a tree based structure containing keys and values. This is where the similarities end. Gconf does not store system settings, editing it will not destroy your Linux machine. Gconf has a clearly defined scope, and is not meant to encompass every possible setting the computer may need. Gconf is easier to edit and repair, because of its XML format.

I refuse to debate which one is better because they are designed to perform some very different tasks. I do however think that the windows registry would benefit from an XML (or other text backend). Similarly Gconf could benefit from an automatic rollback/checkpointing mechanism.

Edit: Jeff Atwood (one of my favorite blog authors) recently wrote a post about the Windows Registry. Find it at http://www.codinghorror.com/blog/archives/000939.html.

Random Thought: Imagine for a second that transferring zeros over the internet was free and only the ones cost money. Now think of a compression algorithm (or an inflation mechanism rather) that increases the ratio of zeros to ones and makes data cheaper to transfer. Now imagine how you would perform error correction/detection with this protocol.

A Look forward to Fedora 12 (Constantine)

Posted by Daniel on July 11, 2009 1 comments

So, they’ve named Fedora 12 Constantine. Both Constantine and Leonidas are towns in Township in St. Joseph County, Michigan. There is going to be much more to the new Fedora than just a fancy pants new name, so lets look at the accepted features. It needs to be noted that since Fedora has hit the feature freeze stage new features will not be added (except in an exception :P ) but may be dropped.

Better Webcam Support

If you’re like me and have a webcam that sends a jpeg (or even worse, a proprietary compression) stream from the sensor through USB then you’ll like this one. The kernel guys have been adamant that JPEG decompression should not be in the kernel. This makes it hard to present a common interface to userspace, as the stream could come in several different formats. To fix this problem drivers are having their decompression code moved out of the kernel and into a new library, called libv4l.  libv4l presents a common interface for all the webcams it supports to applications, this includes v4l1 and v4l2 webcams. This feature also involves porting applications (like cheese, ekiga and amsn) to this new library.

DebugInfo Filesystem

Currently should you want to debug a package in Fedora you have to install the often large -debuginfo packages. For example the kernel-debuginfo package is usually around 300Mb compressed. When installed this figure can inflate to up to 400Mb. With enough debug packages installed this can quickly fill your system. Even if you install a debuginfo package often the whole package isn’t required, for example installing the kernel debug information just to write SystemTap scripts or to debug a single module. This feature allows debug information to be installed when required and only what is required. This is most useful when users are reporting bugs, as their machines can resolve the addresses in the backtraces on the fly as bugs are reported. This means we get better bug reports, which leads to easier fixes, which leads to more time, which leads to more features and who doesn’t want more features?

DisplayPort Support

DisplayPort is a new type of connector that was designed to replace DVI and VGA. Designed by the people at the Video Electronics Standards Association (VESA), like its predecessors it is royalty royalty free. It is going to be extremely important for laptops, which will most likely switch to it because it uses less power and DVI and LVDS. Adding support for DisplayPort will be more complicated than others but most of the work will be able to be shared (by putting in Xorg or the kernel).

Dracut

Currently when a Red Hat system boots, it loads the initrd, and starts nash. Nash isn’t really a shell its just enough to get load the correct drivers and to finally load init. To rebuild your initrd (to include an extra drive for example) you use a tool called mkinird. These tools are maintained by Red Hat and they have done the bulk of the development work. A better approach would be to have a common set of tools to build Linux ramdisk images that can be used across many distributions. That is basically what Dracut is, and this feature is about moving Fedora to these new tools.

Empathy

GNOME dropped Pidgin and now heralds Empathy as the official instant messenger application. Empathy includes not only instant messaging, but also voice and video chat. It can connect and voice/video chat to Google Talk and other XMPP/Jabber users. Another awesome advantage with Empathy is that it includes an API to allow other applications to check your status online. Empathy will enhance the Linux Desktop experience and provide an instant messaging experience that Pidgin simply can’t.

LibLVM

Should you currently want to write a program that creates, removes or alters in any way the LVM setup of the machine your only option is to exec the command line tools, and parse the output. While this approach does work (anaconda and system-config-lvm currently work this way) it isn’t perfect. What is really needed is somebody to come along and write a library to directly interface with LVM that works in an object oriented fashion. This library is currently being written for Fedora 12 and according to the project pages it is 25% complete.

Multiseat

With the current economy saving money is as important as ever. Multiseat will allow you to plug two keyboards, mice and monitors into the same computer and allow each to work independently. This effectively allows two (or more) users per computer, saving you from buying two PCs. The last time this support was available was Fedora 8 but it was dropped in order to use the new GDM, ConsoleKit and X.

NetworkManager Improvements

When I first used NetworkManager I hated it, and I immediately disabled it and rolled back to the network service. The second time I used it things went a little better, but I still ended up disabling it. In Fedora 10 NetworkManager worked flawlessly for me and I discovered that NetworkManager was previously working as well as it could with a poorly written wireless driver. Yes, NetworkManager has copped a lot of flak recently but its constantly improving and some of its problems are caused by dodgy drivers. This feature (actually two on the Fedora 12 features page) focuses on getting system connections working (the ones in /etc/sysconfig) and IPV6 support.

NFS Client IPV6

At the moment IPV6 roll outs across enterprises are happening, but not at a fast rate. There is a lack of support for many applications and the nfs client is included, yet Solaris has had support for NFS over IPV6 for years. This adds one more application to the slowly growing list of applications that support pure IPV6.

PolicyKit 1.0

PolicyKit is a tool kit designed to allow applications to request escalated privileges, for example the time/date control panel can request access to set the system time or the shutdown dialog can request permissions to log out other users prior to shutdown. Unfortunately the current PolicyKit design is not flexible enough. Fortunately it’s been redesigned and now the new improved PolicyKit is ready to take over. It now includes a plugin API and you can change exactly what admin authentication means.

Systemtap static probes

So if you haven’t heard of SystemTap you soon will, I’ve got a post on it coming up shortly, but for the moment just think Dtrace for Linux. Anyway, SystemTap lets you monitor the performance of your machine and your applications in ways that you define. For example I can monitor how many reads Firefox makes while sitting idle on my system (quite a lot actually) but at the moment I can’t monitor how many hits firefox gets in its cache, or how fast queries to its built in SQLlite database are. If static probes were inserted into Firefox at these particular points then I could, and this feature focuses on getting static probes into postgresql, xorg-x11-server, openjdk, tcl and finally firefox.

XZ RPM Payloads

XZ is the new version of LZMA which offeres better compression at faster rates. This means that RPMs in Fedora will be smaller and will be able to be downloaded quicker. This feature will mean that more RPMS can fit on each DVD (or CD) allowing a bigger distribution on the same media. This feature will also help users on slower connections who are currently using deltarpms as now the deltarpms will be even smaller.

x86 Improvements

The Processor world is constantly moving forward, making new processors and dropping old ones on the floor. Fedora is also moving forward, and its come time to drop support for some older processors so we can optimise for the newer ones. So the proposal is that for Fedora 12 i586 support be dropped and packages be optimised for the i686 architecture. For me personally this means that my extremely old HP Desktop will no longer run Fedora, but at 800MHz and with 256Mb of RAM I should probably be retiring it soon anyway.

Random Thought: Which distro has the highest version number? RHEL is at 5.4, Ubuntu is at 9.04, Fedora is at 11, Gentoo is 2009.0 and Mandriva is at 2009.1. So I guess Mandriva wins :D

Edit (20/7/2009): According to the Fedora 12 Features pages many many new features have been added. A repost will follow in a few weeks.

Edit (6/8/2009): You’ll find the current feature list in my new post titled Fedora 12 (Constantine) Features.

The Obligatory Post on Google Chrome OS

Posted by Daniel on July 9, 2009 2 comments

I’m not excited by Google Chrome OS. And the majority of the reason for that stems from the following paragraph posted on the Official Google Blog:

Speed, simplicity and security are the key aspects of Google Chrome OS. We’re designing the OS to be fast and lightweight, to start up and get you onto the web in a few seconds. The user interface is minimal to stay out of your way, and most of the user experience takes place on the web. And as we did for the Google Chrome browser, we are going back to the basics and completely redesigning the underlying security architecture of the OS so that users don’t have to deal with viruses, malware and security updates. It should just work.

They keep mentioning simplicity, lightweight and ‘back to the basics’. have you ever used an OS designed to be extremely simple? Have you ever tried SplashTop, HyperSpace, Latitude ON or even gOS? They may be fast to start up, but you’ll curse just as much when the one thing you want to do is only possible on the actual OS you have installed, requiring a reboot. My opinion is that these lightweight OSes actually make my life harder. Lets see one use case.

I’m at home and I need to check my email. This can be done in ExpressGate so I boot it up (really fast) open up a browser and browse to my Gmail. So far the experience is relatively smooth, until I discover that somebody has sent me an attachment that I need to see. Of course this requires an application installed in my OS so I have to close my browser and any other sites I had open and wait for a reboot into my OS. This is the inherent problem, if I need to use something that isn’t part of my lightweight distribution then I have to stop everything and wait.

Now I understand Google Chrome OS is going to be the actual OS, except most of the applications will be based in the cloud. This presents another problem, especially on netbooks. Netbooks are portable, and designed to be used in places where there is no internet connection. See the problem? Sure you could use a 3G modem but then you have to fork out a fortune for data (especially excess usage).

Unless Google can find a creative way to solve these problems I don’t think that Chrome OS will be anything exciting. It should be noted though that if I were to pick a company that would be able to take on this challenge I would pick Google. I suppose in the end there is only one way to find out, and that would be to wait until the end product is released.

Random thought: If you use a Live CD of another distribution, but don’t install it, is it still cheating?