Tag Archives: HOWTO

Rebooting with ‘The Big Hammer’

Today I had a machine I was working on spit the dummy in a really bad way. It had a tonne of IO errors to its root filesystem and eventually decided to remount it read only. Of course this meant that it was almost entirely wedged. I tried the reboot command, the init command and everything would lockup my terminal. Not having console or physical access to the machine I couldn’t simply hit the power button, so I used the Linux magic commands:

echo 1 > /proc/sys/kernel/sysrq
echo b > /proc/sysrq-trigger

Of course the disk errors meant that it was unable to boot but ‘The Big Hammer’ struck me as something extremely useful.

Using Subversion for Assignments

If you’ve never heard of subversion before then you are in for a pleasant surprise. Subversion is a version control tool, which means it will keep track of several files and all their old versions. Normally subversion is used to help multiple people work together on a single project. It tracks all their changes and combines them all, even flagging when conflicts occur and assists in resolving them. It is also useful when working alone on a school assignment. Here’s a few dot points that capture the essence of why Subversion is useful with assignments:

  • Subversion allows you to work on the same assignment on multiple computers.
  • Subversion can email you with changes you’ve made, allowing to review them.
  • Subversion allows you to show a teacher that you’ve been working on an assignment over the whole time available and not just in the last few days. this gives you greater leverage when asking for an extension.
  • Subversion can help you prove in a disciplinary hearing that you did not plagiarise any code from others showing the natural growth your code had.
  • Subversion can get back that file you just accidentally emptied out of the trash.
  • Subversion can show you all the changes you made between the time you fixed that annoying bug, and now, when you just reintroduced it.

The first step to making an assignment in is to build your repository. If you didn’t do this first that’s okay, you can easily import an existing project into a subversion repository. To create a repository you simply use the ‘svnadmin create’ command. You should then create some folders that should be in every subversion repository (trunk, tags and branches). This next block of commands will show you how to create the initial project. If you’re using these instructions to import an existing project just copy your files into the trunk folder before you run the ‘svn import’ command.

mkdir -p /home/daniel/svn/newproject
svnadmin create /home/daniel/svn/newproject
mkdir -p /tmp/newrepo/{trunk,branches,tags}
svn import /tmp/newrepo file:///home/daniel/svn/newproject -m "Create Initial Structure"
rm -rf /tmp/newproject

The trunk, tags and branches folders aren’t strictly required but can be very useful in certain circumstances. The trunk folder is where you main copy sits, it should be the latest stable version of the software. In an assignment though this is where you will probably be doing all your work, you generally don’t have the need or the time to make and merge branches. Which leads us to branches. Generally you branch software when you are about to make a major change that may break other developers work. You most likely don’t have other developers on your assignment and if you do you’ve probably all decided on what parts you will work on. Finally tags are for labelling certain versions with a specific tag. For example if you have to submit your assignment weekly you could tag each week as you submit, or you could tag as you finish each requirement. To populate these folders you just copy whatever it is you want into them. Subversion will only use a minuscule amount of space as the copy will be stored internally to the repository.

Before you can edit the files in the repository you need to check it out. You can check it out to the same machine, you can use SSH or you could check it out over WebDAV depending how you’ve set it up. The following command checks out the trunk folder into a folder called newproject. This is one of the few times you have to type the full path to the repository. Subversion remembers this for you so that next time you use a subversion command its pre filled.

svn checkout file:///home/daniel/svn/newproject/trunk newproject

What you’ve just checked out is called a ‘working copy’. This is where you make your changes before uploading them again in to the repository. Your working copy also includes copies of the versions you originally checked out so that if you want to revert back to them you can. Because they are stored in the working copy you don’t need access to the repository to revert. To revert back to the version you checked out from the repository you simply run ‘svn revert <filename>’. You can also find the differences between these versions and the current ones by using ‘svn diff <filename>’. The filename is optional and if omitted will print all the changes in the current directories and below.

Part 2 to come…
Random Thought: I’ve just redesigned my website, I’d love to know what my readers think. If you could post your comment on the new design, I’d appreciate it.

Paramaterized Java Classes

One of the biggest features of Java 1.5 was generics. In particular all the collection classes had been extended to use parametrized classes. Normally the collection classes accepted and returned Objects which is the class all other Java classes descend from. Unfortunately this meant that you had to cast everything you got back out of a collection to what you expected it to be. and until you did you would only be able to call methods that were provided by Object. You also had to be ready to catch an exception in case the class could not be cast because it was the wrong object.

Generics and parametrized classes allow Java programmers to place a type on a class and have that type inherited by its methods. For example you can now declare an ArrayList class with a type String. This alters the ArrayList class so that its add method now only accepts objects of type String, the get method now also returns objects of type String. This makes everything type safe which means you don’t have to cast anything and your code won’t compile if you try to put something in the ArrayList that doesn’t match its class.

Java uses parametrized classes to build its collections and you’ll want to use them too if you’re making your own collection class. For example if you were implementing a stack, a queue or a multi-priority FIFO queue are good cases for parametrized classes. Be careful though of the lure parametrized classes can have. They are not a replacement for polymorphism and shouldn’t be used when polymorphism would make more sense. For example if your multi-priority queue gets the priority out of the object itself then you’d need an interface that provides a method to get the priority. Then your class will only be able to accept items that implement that interface, which makes sense in this case as we need to priority to be able to store it.

A parametrized class is really simple to use. Here is an example implementation of a stack collection backed by an ArrayList:

import java.util.ArrayList;
import java.util.Collection;

/**
 * This class acts as a stack. Items can be 'pushed' which adds them to the top
 * of the stack. items can also be 'popped' which removes and returns the top
 * item on the stack and removes it. This means only the most recently added
 * item is available at the current time. To get to older items you need to
 * first remove the others.
 * 
 * Note: Java already has a stack object that should probably be used in
 *       preference to this one. This is only an example implementation.
 * 
 * @author Daniel Hall <daniel@danielhall.me>
 *
 * @param <T> The type of items that can be stored in the Stack.
 */
public class Stack<T> {
	/* Uses the same type as this class to store the items */
	private ArrayList<T> array = new ArrayList<T>();
	
	/**
	 * Creates a Stack containing items already in a collection. The collection
	 * must have the same parameterized type as this class to ensure that we get
	 * the right objects.
	 * @param c The Collection to initialize with
	 */
	public Stack(Collection<T> c) {
		array.addAll(c);
	}
	
	/**
	 * Creates an empty Stack object
	 */
	public Stack() {
		
	}
	
	/**
	 * Adds an item to the top of the stack.
	 * @param item The item which will be added to the top of the stack.
	 */
	public void push(T item) {
		array.add(item);
	}
	
	/**
	 * Removes the first item from the stack
	 * @return The item that was on the top of the stack.
	 */
	public T pop() {
		/* This gets the size so we don't have to do it twice. */
		int count = array.size();
		
		/* If the stack is empty return null, note that the Java implementation
		 * of stack throws an Exception instead.
		 */
		if (count == 0) {
			return null;
		}
		
		/* Remove the last added object (which will have index count - 1) */
		return array.remove(count - 1);
	}
}

Random thought: John Lions wrote a book about the Unix source code, in the seventies, which because it also included some code, was blocked from being published until 1996.

Using Subversion over SSH

Subversion is an amazing tool that you can use to keep track of all the changes you make to a group of files. If you haven’t used it before, or have never heard of ‘version control’ then you should probably read the Subversion Book.

Few people don’t realise that subversion has the ability to connect to a remote repository via SSH. Its extremely simple and can give you all the advantages of storing your important files on a server while still having them readily accessible on your desktop. This means that for example you could have your files (and every old version of your files) stored on a RAID device on a server while working with them locally on your desktop.

To set this up its actually rather simple. First you create your repository and perform the initial import of the files. I usually make it in my home directory as follows:

mkdir -p /home/daniel/svn/newproject
svnadmin create /home/daniel/svn/newproject
mkdir -p /tmp/newrepo/{trunk,branches,tags}
svn import /tmp/newrepo file:///home/daniel/svn/newproject -m "Create Initial Structure"
rm -rf /tmp/newproject

These commands are basically what you’d use to create any subversion repository and people familiar with it require no explanation. Most people probably even have is scripted to make it just that much easier. Here comes the fun part though. Next we (on our local machine) check the files out. To checkout subversion repositories over ssh you simply use the following command:

svn checkout svn+ssh://username@servername/home/daniel/svn/newproject/trunk newproject

All going well you will now see a password prompt and upon successful authentication the files will be checked out. This is all that is required and from now on you can simply use the ordinary svn commands.

Random Thought: … and I says to the kernel developer, I says “git this!”

Oh My God – I broke my LVM

So today I did about the stupidest thing I could have done at the time. I was planning on clearing my USB hard drive so I could start my new backup plan on it. Of course any Linux geek knows the easy way to erase a hard drive is to do a ‘dd if=/dev/zero of=/dev/sdb1’. On almost all my computer there is only one hard drive which maps to /dev/sda. Of course you know exactly where I’m going here don’t you? So this is my home server with two hard drive combines into one volume group. The first hard drive is /dev/sda, the second /dev/sdb and the USB hard drive got mapped to /dev/sdc. So in my case that command obliterated the first 125Mb of my second drive before I noticed.

My machine was still running so I knew I hadn’t wiped anything immediately important. The first thing that I thought of doing was checking what exactly it was that I had wiped and what chance I had of backing up anything before bailing out. Looking at the LVM layout revealed that I’d probably just destroyed the file system I stored my local Fedora repository on, something I could do without. So I umounted it, removed it from /etc/fstab and did a lvremove. This is exactly where I realised the gravity of the situation. LVM was complaining that it couldn’t locate one of the physical volumes. Of course it couldn’t, I’d just blown away all the metadata for it.

Did you know LVM keeps backups of the metadata? Yes, it keeps them in /etc/lvm/backup (for slightly older copies see /etc/lvm/archive) and you can use this to recover the metadata. I thought a good place to do this would be now, before the reboot that could end it all. Try as I might it was refusing to create a volume that already existed and it also complained about the device being in use. I count myself extremely lucky to be able to do what I did next. To me it felt incredible but when you really think about it it makes sense.

I downloaded the Fedora 11 Live CD and burned it to CD. Yep that’s right, while knocking on deaths door my machine managed to launch a torrent client, download a 700Mb ISO and burn it to a CD. After that I backed up the /etc/lvm folder to the USB hard drive that caused this mess. Finally I rebooted into the Live environment. The very next step was to recreate the partition table with fdisk.

Then I recreated the physical volume metadata that was destroyed with the following command:

pvcreate -ff -u DsuvMV-1HVj-SQOU-wZkT-N9M0-LMZd-gPws1U \
 --restorefile /media/usbdisk/lvm/backup/Volgroup00 /dev/sdb1

This forces the creation of a pv with a specific uuid, ignoring any pvs that exist with the same uuid. It also restores the metadata stored in the restorefile. Follow up with this command to restore the full metadata.

vgcfgrestore -f /media/usbdisk/lvm/backup/Volgroup00 -v VolGroup00

Now our LVM metadata is all correct, but at this point we still need to activate the logical volumes.

vgchange -ay

Finally you should fsck your logical volumes to make sure everything is working properly and you don’t get any nasty surprises later. All that is left then is to reboot into your recovered system.

Now thats something they don’t teach you in RHCE!

Random thought: Who needs enemies when I have my own stupidity to contend with?