Tag Archives: Linux

Rebooting with ‘The Big Hammer’

Today I had a machine I was working on spit the dummy in a really bad way. It had a tonne of IO errors to its root filesystem and eventually decided to remount it read only. Of course this meant that it was almost entirely wedged. I tried the reboot command, the init command and everything would lockup my terminal. Not having console or physical access to the machine I couldn’t simply hit the power button, so I used the Linux magic commands:


# echo 1 > /proc/sys/kernel/sysrq
# echo b > /proc/sysrq-trigger

Of course the disk errors meant that it was unable to boot but ‘The Big Hammer’ struck me as something extremely useful.

Using EncFS to encrypt your files

About EncFS

EncFS is an encrypted filesystem based on FUSE. It transparently encrypts files stored in it and places them on another volume. This is in contrast to block level encrypted filesystems which transparently encrypt the data under the filesystem layer as it is being written to disk. Think of EncFS as a bind mount, except that the source for the mount is encrypted and the place it is mounted to is the only place it is available unencrypted.

The main advantage of EncFS filesystems is that when backing up only the files which have changed need to be backed up. This means it works perfectly with tools such as rsnapshot. Another advantage is that the filesystem doesn’t need a block of disk allocated to it and will shrink and expand as the files inside change.

Finally because this is all implemented with FUSE it is all done in userspace. No root access is required (apart from setting FUSE up) to create and alter encfs filesystems.

Setting Up an EncFS Volume

So the first thing you need to do to setup an encfs volume is to install FUSE and EncFS. If you don’t have root access you will have to ask your sysadmin to do this for you, otherwise follow your distribution specific method of installing new packages. On Fedora it is called ‘fuse-encfs’ and on Debian/Ubuntu its called ‘encfs’. On some older systems users wishing to use FUSE may need to be added to the correct group.

First you need to decide where you will put the encfs volume, and where you’ll mount it. I usually put mine in /home/daniel/.crypt and mount it to /home/daniel/crypt. But feel free to name it whetever you want. When you’ve decided run the EncFS with those arguments, for example to use the example I specified it would look like this:

[code]
<daniel@server ~>$ encfs /home/daniel/.crypt /home/daniel/crypt
The directory "/home/daniel/.crypt/" does not exist. Should it be created? (y,n) y
The directory "/home/daniel/crypt" does not exist. Should it be created? (y,n) y
Creating new encrypted volume.
Please choose from one of the following options:
enter "x" for expert configuration mode,
enter "p" for pre-configured paranoia mode,
anything else, or an empty line will select standard mode.
?>

Standard configuration selected.

Configuration finished. The filesystem to be created has
the following properties:
Filesystem cipher: "ssl/aes", version 2:2:1
Filename encoding: "nameio/block", version 3:0:1
Key Size: 192 bits
Block Size: 1024 bytes
Each file contains 8 byte header with unique IV data.
Filenames encoded using IV chaining mode.
File holes passed through to ciphertext.

Now you will need to enter a password for your filesystem.
You will need to remember this password, as there is absolutely
no recovery mechanism. However, the password can be changed
later using encfsctl.

New Encfs Password:
Verify Encfs Password:
[/code]

As you can see the directories don’t need to be created first. There is also a prompt for what security settings you want to use. Hitting enter will give you standard settings, but for something more powerful you should hit ‘p’ then enter. You can now proceed to place files in /home/daniel/crypt and they will be encrypted and placed into /home/daniel/.crypt. If you don’t believe me go ahead and check.

See? I told you so. Now you can unmount it using ‘fusermount -u /home/daniel/crypt’ and mount it again using encfs /home/daniel/.crypt /home/daniel/crypt and typing your password.

Random Thought: When travelling to other countries, local laws may mean that customs can search your laptop, including encrypted filesystems. You may have to reveal your key, or be arrested.

Writing a Daemon in C

What is a Daemon?

A daemon is a program that runs in the background. A daemon will usually be started at system startup and end at system shutdown. The exceptions to this rule are programs like the Bluetooth SDP daemon, which is activated when a new Bluetooth HCI is found,, and ends when it is removed. Daemons run transparently and do not normally interact with the user directly.

Daemons start as ordinary processes but they eventually ‘fork and die’ to start running in the background. Some daemons do only the ‘fork and die’ step but ignore other important steps. Here is a list of what a daemon should do:

  1. Fork to create a child, and exit the parent process.
  2. Change the umask so that we aren’t relying on the one set in the parent.
  3. Open logs to write to in the case of an error.
  4. Create a new session id and detach from the current session.
  5. Change the working directory to somewhere that won’t get unmounted.
  6. Close STDIN, STDOUT and STDERR.

These steps ensure that our association with the calling environment is destroyed and our daemon is now free to run as a completely separate process.

Lastly before writing the daemon you should make sure the code is written securely and in a way that fails gracefully. If your daemon crashes it will not be able to prompt the user about what action to take. The user may not even notice until it is too late.

Forking a child process

In Unix fork() is the only system call with two return values. When you call fork a child process is created which is a near copy of its parent (some things will be different in the child eg. process id). The fork command then returns a 0 in the child and the childs process id in the parent, on failure a -1 is sent to the parent. Generally a program will then check whether it is the child or parent by these return values (just like in movies when a cloned character will check to see if he has a belly button and hence is the original). Here is a snippet of code to do this:
[code lang="c"]
pid_t pid;

/* Clone ourselves to make a child */
pid = fork();

/* If the pid is less than zero,
something went wrong when forking */
if (pid < 0) {
exit(EXIT_FAILURE);
}

/* If the pid we got back was greater
than zero, then the clone was
successful and we are the parent. */
if (pid > 0) {
exit(EXIT_SUCCESS);
}

/* If execution reaches this point we are the child */
[/code]

Changing the umask

Because we are a clone of our parent we’ve inherited its umask. This means the child doesn’t know what permissions files will end up with when it tries to create them. We do this by simply calling umask like this:
[code lang="c"]
/* Set the umask to zero */
umask(0);
[/code]

Open logs to write to

This part can be done in several different ways. You could open text files, log to a database or use syslog. The method I’m going to demonstrate here is to log using syslog. Syslog sends your log messages to a system wide logger, where they can be configured to be written to a file, send to a network server or filtered away entirely.
[code lang="c"]
/* Open a connection to the syslog server */
openlog(argv[0],LOG_NOWAIT|LOG_PID,LOG_USER);

/* Sends a message to the syslog daemon */
syslog(LOG_NOTICE, "Successfully started daemon\n");

/* this is optional and only needs to be done when your daemon exits */
closelog();
[/code]

Create a new session id

Each process on a Unix system is a member of a process group (or session). The id of each group is the process id of its owner. When we forked from our parent earlier we will have inherited its process group, and our process group leader will still be its parent process. We want to create our own process group and become our own process leader otherwise we will look like an orphan. We can do this easily as follows:
[code lang="c"]
pid_t sid;

/* Try to create our own process group */
sid = setsid();
if (sid < 0) {
syslog(LOG_ERR, "Could not create process group\n");
exit(EXIT_FAILURE);
}
[/code]

Changing the working directory

At the moment we have the working directory we inherited from our parent. This working directory could be a network mount, a removable drive or somewhere the administrator may want to unmount at some point. To unmount any of these the system will have to kill any processes still using them, which would be unfortunate for our daemon. For this reason we set our working directory to the root directory, which we are sure will always exist and can’t be unmounted.
[code lang="c"]
/* Change the current working directory */
if ((chdir("/")) < 0) {
syslog(LOG_ERR, "Could not change working directory to /\n");
exit(EXIT_FAILURE);
}
[/code]

Closing the standard file descriptors

A daemon doesn’t interact with the user directly it has no use for STDIN, STDOUT and STDERR and we really have no idea where these are connected or where anything we write to them will end up. As these file descriptors are not required and effectively useless we should close them to save some system resources and prevent any related security problems. We close these descriptors like this:
[code lang="c"]
/* Close the standard file descriptors */
close(STDIN_FILENO);
close(STDOUT_FILENO);
close(STDERR_FILENO);
[/code]

Writing the payload

Now you have a C program that is capable of becoming a daemon, but its a pretty useless daemon if it exits immediately. Payload code is really up to you to design. I’ll offer you a few tips on designing your payload.

  • Put your payload in a loop. Generally in a daemon you want to perform the same action over and over again until you’re killed. If you have to cleanup (such as closing syslog) when the daemon is about to be killed you should add an exit clause that will be activated by a SIGTERM signal handler.
  • Make your code as fast an efficient as possible. This is something you should do with any program, but with daemons it is important that you do not hamper the performance of the rest of the system. This is especially true if you’re going to be running this daemon on desktop systems.
  • Be aware that your code may be preempted very often. As your daemon is going to be running for the amount of time the system is up, it is likely that its execution will be preempted.
  • Be paranoid about security. Daemons are common attack vectors and can be used to gain privileged access to a system. You should consider dropping any privileges that you don’t require.

Conclusion

So if we take all the code I’ve mentioned in this post and put it all together you have a simple daemon. You can download the source from the link here: daemon.c.
If your daemon is only going to be run on Linux and not on a System V style system such as Solaris you can use the daemon function to do a lot of this work for you.

References

Linux Daemon Writing HOWTO in C
Linux Daemon writing in C++

Random Thought: It appears the devil uses a Unix based OS, probably OSX.

Using Subversion for Assignments

If you’ve never heard of subversion before then you are in for a pleasant surprise. Subversion is a version control tool, which means it will keep track of several files and all their old versions. Normally subversion is used to help multiple people work together on a single project. It tracks all their changes and combines them all, even flagging when conflicts occur and assists in resolving them. It is also useful when working alone on a school assignment. Here’s a few dot points that capture the essence of why Subversion is useful with assignments:

  • Subversion allows you to work on the same assignment on multiple computers.
  • Subversion can email you with changes you’ve made, allowing to review them.
  • Subversion allows you to show a teacher that you’ve been working on an assignment over the whole time available and not just in the last few days. this gives you greater leverage when asking for an extension.
  • Subversion can help you prove in a disciplinary hearing that you did not plagiarise any code from others showing the natural growth your code had.
  • Subversion can get back that file you just accidentally emptied out of the trash.
  • Subversion can show you all the changes you made between the time you fixed that annoying bug, and now, when you just reintroduced it.

The first step to making an assignment in is to build your repository. If you didn’t do this first that’s okay, you can easily import an existing project into a subversion repository. To create a repository you simply use the ‘svnadmin create’ command. You should then create some folders that should be in every subversion repository (trunk, tags and branches). This next block of commands will show you how to create the initial project. If you’re using these instructions to import an existing project just copy your files into the trunk folder before you run the ‘svn import’ command.
[code lang="shell"]mkdir -p /home/daniel/svn/newproject
svnadmin create /home/daniel/svn/newproject
mkdir -p /tmp/newrepo/{trunk,branches,tags}
svn import /tmp/newrepo file:///home/daniel/svn/newproject -m "Create Initial Structure"
rm -rf /tmp/newproject[/code]
The trunk, tags and branches folders aren’t strictly required but can be very useful in certain circumstances. The trunk folder is where you main copy sits, it should be the latest stable version of the software. In an assignment though this is where you will probably be doing all your work, you generally don’t have the need or the time to make and merge branches. Which leads us to branches. Generally you branch software when you are about to make a major change that may break other developers work. You most likely don’t have other developers on your assignment and if you do you’ve probably all decided on what parts you will work on. Finally tags are for labelling certain versions with a specific tag. For example if you have to submit your assignment weekly you could tag each week as you submit, or you could tag as you finish each requirement. To populate these folders you just copy whatever it is you want into them. Subversion will only use a minuscule amount of space as the copy will be stored internally to the repository.

Before you can edit the files in the repository you need to check it out. You can check it out to the same machine, you can use SSH or you could check it out over WebDAV depending how you’ve set it up. The following command checks out the trunk folder into a folder called newproject. This is one of the few times you have to type the full path to the repository. Subversion remembers this for you so that next time you use a subversion command its pre filled.
[code lang="shell"]svn checkout file:///home/daniel/svn/newproject/trunk newproject[/code]
What you’ve just checked out is called a ‘working copy’. This is where you make your changes before uploading them again in to the repository. Your working copy also includes copies of the versions you originally checked out so that if you want to revert back to them you can. Because they are stored in the working copy you don’t need access to the repository to revert. To revert back to the version you checked out from the repository you simply run ‘svn revert <filename>’. You can also find the differences between these versions and the current ones by using ‘svn diff <filename>’. The filename is optional and if omitted will print all the changes in the current directories and below.

Part 2 to come…
Random Thought: I’ve just redesigned my website, I’d love to know what my readers think. If you could post your comment on the new design, I’d appreciate it.

Using Subversion over SSH

Subversion is an amazing tool that you can use to keep track of all the changes you make to a group of files. If you haven’t used it before, or have never heard of ‘version control’ then you should probably read the Subversion Book.

Few people don’t realise that subversion has the ability to connect to a remote repository via SSH. Its extremely simple and can give you all the advantages of storing your important files on a server while still having them readily accessible on your desktop. This means that for example you could have your files (and every old version of your files) stored on a RAID device on a server while working with them locally on your desktop.

To set this up its actually rather simple. First you create your repository and perform the initial import of the files. I usually make it in my home directory as follows:
[code lang="shell"]mkdir -p /home/daniel/svn/newproject
svnadmin create /home/daniel/svn/newproject
mkdir -p /tmp/newrepo/{trunk,branches,tags}
svn import /tmp/newrepo file:///home/daniel/svn/newproject -m "Create Initial Structure"
rm -rf /tmp/newproject[/code]
These commands are basically what you’d use to create any subversion repository and people familiar with it require no explanation. Most people probably even have is scripted to make it just that much easier. Here comes the fun part though. Next we (on our local machine) check the files out. To checkout subversion repositories over ssh you simply use the following command:
[code lang="shell"]svn checkout svn+ssh://username@servername/home/daniel/svn/newproject/trunk newproject[/code]
All going well you will now see a password prompt and upon successful authentication the files will be checked out. This is all that is required and from now on you can simply use the ordinary svn commands.

Random Thought: … and I says to the kernel developer, I says “git this!”