Category Archives: System Administration - Page 2

SystemTap

Posted by on July 25, 2009 No comments

SystemTap is the Linux analogy to Solaris DTrace and is similar to the strace command, only much much more powerful. It effectively lets you set breakpoints in the kernel to monitor what your applications are doing. For example if I was worried that some application I’d written was polling way too often, I could ask SystemTap to output the number of times my application calls poll() or select().

To use SystemTap first you write a simple script, or borrow one from someone else. On a Fedora system you’ll fine some sample scripts in /usr/share/doc/systemtap-0.9.8/examples provided you have SystemTap installed. You then run the stap; command. The stap command immediately begins parsing the scipt looking for any tapsets that your script uses and if it does it includes them. It then converts your script into C code and compiles it into a kernel module. This kernel module is inserted into the running kernel and stap attaches to it. The kernel module stays in the kernel until it is cancelled by the user, it reaches an exit function or it encounters too many errors.

While SystemTap can be used to simply dump loads of data about what an application is doing in kernel space that is not its purpose. SystemTap scripts are able to drill down, extract, process and format the data its gathering. For example if you were trying to find out what files a process was writing to your disks could just output every single write call and print it out, or you could keep the statistics and every ten seconds print the top ten files written to. SystemTap is designed to help you filter out all the noise and monitor only what you want to monitor.

The simple way to get started with SystemTap is to download the Beginners guide or the Tutorial. On Fedora systems when you install SystemTap you’ll find the tutorial at /usr/share/doc/systemtap-0.9.8/tutorial.pdf. SystemTap skills are handy for system administrators and developers, so if you fit into those categories I’d highly recommend you check it out.

Random Thought: Where does /dev/zero come from and where does /dev/null go? What happens if you pipe /dev/zero to /dev/null?

Linux ‘top’ Commands

Posted by on July 5, 2009 No comments

As a sysadmin working with Linux PCs I often need real time data on the status of the systems I manage. For example I might need to know what is using up all the bandwidth on an interface, whats taking up  all the memory or why my X displays are running sluggish. The impromptu
standard for naming these commands is to add the ‘top’ suffix. Here is a list of my favorite 8 ‘top’ commands.

top

Top, the grandaddy of all the Linux top commands, is most useful for  monitoring tasks running on your system. On my Fedora system its contained in package procps which on Fedora 11 was 3.2.7. Top has many keybindings to change its behaviour, for example ‘f’ is used to add and remove fields, ‘o’ will help you reorder those fields and the lesser-than and greater-than keys move the search field. You can type ‘h’ for a bigger list.

tload

You caught me! This one doesn’t end with top, but I put it here because on Fedora it comes as part of the procps packages with top, slabtop and others. tload is a good application to have in a small terminal in the background.  It comes packaged along with top. It displays a histogram of the current load for the system. I like to have it running in a transparent terminal that I leave open on my laptop.

htop

An improved, menu driven and colourised version of normal top. Htop allows you to get information on each thread of a program or combine all thread like normal top does. Some would argue that its more powerful, but others simply say its bloated. Whatever you believe, it has some nice features that any sysadmin will appreciate and you’ll soon be wishing htop was avaliable
everywhere.

iftop

top is to cputime what iftop is to your network interfaces. It displays a list of the top servers that are exchanging data over the selected interface. Because of the way it captures packets from the interface it needs root privleges to run.

iotop

iotop displays live system IO statistics. Like top it lists the top applications that are using IO. It can be toggled with the ‘o’ key to only
display programs currently performing IO, which is useful on large servers. You can read more about its keybindings on its manpage.

slabtop

slabtop is especially useful for kernel developers and pedantic system tuners. It displays a summary of all the slab objects allocated in the kernel. I can take options to tell it how to display its information, but only has two keybindings, spacebar is to refresh the screen and ‘q’ is to quit. You can see its options on its manpage.

xrestop

For X developers there is a utility called xrestop. xrestop displays a list of X server resources allocated. It can be useful to see if your application, or your X server is leaking resources. While it only accepts the ‘q’ key to exit it does accept a few options.

powertop

Built by Intel to help tune laptops to get the best performance out of your battery. It shows the percentage time spent in each CPU state and lists the programs and devices that caused the most wake ups from idle mode. Its most useful feature though is that it will analyse your system and give a suggestion on action to be taken to save just that little bit more power.