Materials

1 USB traffic light hub (search ebay for ‘traffic light hub’)
1 Atmega8u2 breakout board
1 each of 9mm Red Green and Orange LEDs
3 470 Ohm resistors
4 thin patch wires (preferably different colours)
1 USB mini cable
1 Pack of Black Sugru
Solder
Hot Glue
Corrugated Cardboard

Equipment

A Dremel, or similar cutting tool
Hot Glue Gun
Soldering Iron
Wire Strippers
Wire Cutters
Spudger tool, or guitar pick, stanley knife or fingernails
Linux PC (with avr-gcc and dfu-programmer commands installed)

Details

1. Use the spudger to open the traffic light

The weakest part of the traffic light is the stem but it is nearly impossible to open it from there. The approach that I took was to pry the top open a little, then pry the bottom open a little, then carefully pull the two halves apart at the same time. This ensures that you dont snap the plastic holding the stem together. If you don’t have four wires from the materials list, you can strip the USB cable and use those since it contains four differently coloured wires.

2. Gut the traffic light

Remove the USB hub circuit board. You can use this board later in another project or two, should you do you will probably want to cut off the LEDs. In my board the little plastic windows were in the wrong order for Australian traffic lights. Luckily this is easy to fix, simply pop them out and press them back in the correct place. Feel free to put the windows in any order that you wish, just remember to alter the program later before compiling and flashing it.

3. Make room

Its a little hard to fit the Circuitry into the case with the two supports that held the hub in place, additionally you need a little extra space for the thickness of the cardboard. You can get all this by using a Dremel to remove the supports and some of the plastic around the USB sockets. Be careful to not hit the side of the case as I did on my first one as the plastic is thin and easily blemishes the outside. The photo shows one before modification on the left and after a fight with the Dremel on the left.

4. Make the cardboard circuit board

The process of getting a circuit board built takes too long for a quick hack like this and I didn’t have any protoboard lying around. This all means that we get to make a crazy cardboard circuit board. The best thing about cardboard is that you can draw on it as you build it, and you can easily cut it to fit the interior of the device easily. Basically put the LEDs and resistors through the cardboard next to each other. For ease of wiring put align all the LEDs to have their long legs in the same direction. Wire each short leg to a resistor and the free legs of all the resistors to a single black wire. Finally wire each of the long legs of the LEDs to a different coloured wire. Once this is done you should test each lead and LED to make sure it is correctly wired. You can do this by connecting ground to the black lead, and 3-5V to the coloured leads.

5. Connect to the Breakout board

Now we take out the SparkFun Atmega8U2 breakout and solder it to the LEDs on our cardboard board. This is pretty simple, basically you solder the wire from the resistors to the hole labelled GND, the green LED to PB7, the orange LED to PB6 and finally the red LED to PB5. You will likely want to cut the wires so that they reach the board where it will sit, and only have a few extra millimeters. If you leave too much room then you will have issues trying to put the wires inside the case, and of course if you don’t leave enough you wont be able to get the breakout board to sit in the right place.

6. Load the software on the device and test

You can find the code to build the firmware on my GitHub account. Provided that you have avr-gcc and dfu-programmer installed you should be able to simply clone that repository and type ‘make all’ inside it. If that refuses to work for some reason though, I have attached the output of compilation, a hex file (which is uploaded as a txt file to stop wordpress whining). You can download the hex file here: USBTrafficLight.hex. This hex file, or the one that’s output from programming can be programmed onto the device easily. First you put the device into bootloader mode by plugging it into the computer then hit the reset button. Finally you run the following commands:

dfu-programmer at90usb82 erase # Erase MCU
dfu-programmer at90usb82 flash USBTrafficLight.hex # Flash MCU
dfu-programmer at90usb82 reset # Reset MCU

You will have to unplug and plug the device back in to get it out of bootloader mode. Once you have programmed the device you should run a test to make sure it works. You can do this by writing characters to the virtual serial port the device creates. The following commands will do this for you:

echo 'g' > /dev/ttyACM0 # Should be green
echo 'o' > /dev/ttyACM0 # Should be orange
echo 'r' > /dev/ttyACM0 # Should be red

7. Cut a hole in the base

This is the part I’m least happy with. Here you have to cut a hole in the base so that you can plug in a mini usb cable. I generally cut a rectangular hole in the back piece of the traffic light that is about the same size as the USB cable I’m using. This rectangle usually goes about halfway up the base, and goes right to the bottom while being a little bit wider then a USB mini cable. I also cut a small drill hole to allow access to the reset button for loading new firmware. Make sure you test that you can plug in the USB mini cable. Instead of cutting a hole in the base I’ve been thinking about building a USB cable into the device. This is difficult because the USB mini port on the breakout is SMD and the pins are not broken out. If you have any ideas on the best way to do this, please let me know in the comments.

8. Install the cardboard board and the breakout board

The cardboard section should be easy to slide into the back case of the device. Should there be an issue making things fit you can always trim the cardboard, or pad it with paper. You should glue the cardboard into the back through the USB port hole where the hub was. This accomplishes two things, it attaches the cardboard to the back, and it partially fills the holes that we will later fill with Surgu, saving you a little Sugru. If you bought clear LEDs instead of diffused ones you may wish to glue something to diffuse the light to the front (I use baking paper, sometimes two layers). Install the breakout board by putting a little hot glue on the bottom, pushing it into place, plugging in the mini USB cable then finally wiggling it into the perfect position. You’ll also want to tie up your wires using a little electrical tape to make them easier to manage in the last step.

9. Install the front and patch holes

Mould some Sugru into the holes left behind from where the USB hub was, and into the extra space around the USB mini plug. Make sure that you continuously test the USB mini plug to ensure you don’t add too much Sugru, also do not get any Sugru into the plug. You can use soapy water and rubbing to make the Sugru surface smooth if you wish. Then install the front of the case and leave the Sugru to set, which takes about 6-12 hours. Once the Surgu has set the device is ready to use.

Instructions for use

The interface to the device is implemented as a USB to Serial adapter, however since there is not serial interface, and the entire device is self contained we don’t have to implement the entire spec. A USB to Serial device is implemented by sending characters and control messages over the USB bus. However because there is no serial interface we can ignore all the control messages. The firmware above simply grabs the characters from the stream and acts on them. This means that the device will work regardless of the baud rate, stop bits or parity settings. On Linux and OSX this means that all you have to do to control the device is to echo characters to the character device. You can send ‘g’ for green, ‘o’ for orange, ‘r’ for red and any digit from 0-7 for all the possible light configurations. A simple test script for a Linux PC looks like this:

#!/bin/bash

while /bin/true; do
  echo 'g' > /dev/ttyACM0
  sleep 2
  echo 'o' > /dev/ttyACM0
  sleep 2
  echo 'r' > /dev/ttyACM0
  sleep 2
  echo 'a' > /dev/ttyACM0
  sleep 2
done

Tracebacks and coredumps are computer generated, which makes them more accurate then error messages, but harder for humans to understand. Error messages however are put in place by the programmer which means they can occasionally be misleading, inaccurate, ambiguous or just plain wrong. This is not always the programmers fault, sometimes its hard to describe exactly what went wrong. Other times the error describes the situation perfectly, but the sysadmin jumps to a different conclusion based on his circumstances.

Example

Some time ago we had some users complaining about a problem when trying to use X Forwarding via SSH. On this server /home was mounted off a Novell NetWare NFS share. They were getting the following output and were unable to run X11 applications.

xauth: error in locking authority file /home/daniel/.Xauthority

Seeing this error I assumed that something was going wrong with the locking mechanism of NFS. I tried mounting the NFS share with the explicit lock option, but the same error remained. I tried explicitly giving the sync option too, but to no avail. I ended up trying many different NFS options until eventually I gave up and asked the Novell administrators to check their servers. I was convinced that something on their end was causing this locking error.

The Novell administrator responded that they could see nothing wrong on their end. This must mean that something was wrong on our side. I tried restarting the nfsstad and lockd initscripts and the whole machine but once again the same issue persisted. I checked the server using the rpcinfo command, which showed that everything was working fine. I even connected to the daemon using telnet (though I couldn’t talks its language) and confirmed a firewall was not in the way.

I thought that maybe there was something going wrong in the interaction between the client and the server, so I ran a tcpdump to capture all the packets transferred between them. this is where I made a small breakthrough. I found a NFS reply that had returned with SERVFAIL and error code 526. Googling for this error and Netware generally pointed towards a problem with character sets not getting preserved to the Novell server. There was nothing but ordinary characters on the filesystem, so much for that idea.

I wanted to know exactly what was happening when xauth was trying to lock the file, so I did an strace on it. Here are the last few lines (after xauth mmaped its libraries).:

stat("/home/e71377/.Xauthority-c", {st_mode=S_IFREG|0600, st_size=0, ...}) = 0
unlink("/home/e71377/.Xauthority-c")    = 0
unlink("/home/e71377/.Xauthority-l")    = -1 ENOENT (No such file or directory)
open("/home/e71377/.Xauthority-c", O_WRONLY|O_CREAT|O_EXCL, 0600) = 3
close(3)                                = 0
link("/home/e71377/.Xauthority-c", "/home/e71377/.Xauthority-l") = -1 ESERVERFAULT (Unknown error 526)
write(2, "xauth:  error in locking authori"..., 65xauth:  error in locking authority file /home/e71377/.Xauthority
) = 65
exit_group(1)                           = ?

So it appears that this was not a file locking problem at all. xauth was successfully creating the files but it failed when it tried to create a hardlink. Reviewing the code for libXau (AuLock.c) revealed exactly why:

    while (retries > 0) {
        if (creat_fd == -1) {
            creat_fd = open (creat_name, O_WRONLY | O_CREAT | O_EXCL, 0600);
            if (creat_fd == -1) {
                if (errno != EACCES)
                    return LOCK_ERROR;
            } else
                (void) close (creat_fd);
        }
        if (creat_fd != -1) {
#ifndef X_NOT_POSIX
            /* The file system may not support hard links, and pathconf should tell us that. */
            if (1 == pathconf(creat_name, _PC_LINK_MAX)) {
                if (-1 == rename(creat_name, link_name)) {
                    /* Is this good enough?  Perhaps we should retry.  TEST */
                    return LOCK_ERROR;
                } else {
                    return LOCK_SUCCESS;
                }
            } else {
#endif
                if (link (creat_name, link_name) != -1)
                    return LOCK_SUCCESS;
                if (errno == ENOENT) {
                    creat_fd= -1;       /* force re-creat next time around */
                    continue;
                }
                if (errno != EEXIST)
                    return LOCK_ERROR;
#ifndef X_NOT_POSIX
           }
#endif
        }
        (void) sleep ((unsigned) timeout);
        --retries;
    }

xauth isn’t trying to lock the file through flock() or another file locking method, which means that it is not the cause. Instead xauth is creating a file, and then to make sure it is the only program altering .Xauthority it creates a link. If the link succeeds then its the only program, if not then another program has the lock. The problem happens when xauth tries to make the hardlink. Interestingly there is a fallback for non-POSIX systems, but as RHEL is POSIX compatible it is not used.

It appeared that the NFS server did not support hard links. To test this theory I created several files, and attempted to create hard links using ‘cp -l file1 file2′. and they failed in the exact same way. All I had to do now was explain to the Novell Administrator that the problem was not locking, and was in fact that we were mounting a filesystem which did not support hard links on a POSIX compatible system. The Novell share was changed to support hard links (don’t ask me how, I’m not a Novell guy) and everything was working again.

Conclusion

The lesson to take away from here is not that hardlinks are required on POSIX, or that xauth doesn’t use file locking but locks itself via a dance of hardlinks. The lesson here is that you should never trust error messages. Take them as a hint, use them as a starting point but do not take them as law. You need to remember that the error message was written by a human and you may not be interpreting it how it was written.

Google recently released a chunk of code in the form of an Apache module. The idea is that you install it in your Apache server, it sits in between your application and the web browser and modifies the served requests to make the page load faster.
It does this by using combinations of filters, some are well known best practices, others are newer ideas. For example on filter simply minifies your JavaScript while another embeds small images in a page using data-uris. The changes these filters make range from low risk, to high risk. It should be noted that not all the filters will improve the page time some even making pages slower in some cases.

So what’s the issue?

The issue here really isn’t mod_pagespeed, but it’s the way people are viewing it. In my job as a Web Performance Engineer I have had several people recently say to me “let’s put mod_pagespeed on our web server to make it faster”. This is a break from normal attitudes, if someone were to to say “we should put our images into data-uris” then people would question the speed benefit, or the extra load on the server. For some reason when Google implement a page speed module people just assume that it will make their page faster, and that it will work in their environment. The truth is that Google really have no idea what the module will do to your page.

The second issue is that all these tweaks can usually be better implemented at the application level. If you minimize all your JavaScript as part of your build process then the web server will not have to do it for you. The same applies to data-uris. If they are simply part of the page then the browser doesn’t need to read in the extra image, uuencode it, then compress it. All that is quite a lot of work, which only really needs to be done once.

So what should I use mod_pagespeed for then?

You don’t always have access to the application code. If you are using third party software then before mod_pagespeed you may have had no control over the minification of CSS. This is where the module really shines. It gives you a layer between the application code and the web browser where you can apply all sorts of performance tuning.

The other advantage I can see is for looking for the best tunings to apply to your application quickly. You can setup mod_pagespeed and and run experimental tests with the filters on of and with a control to quickly figure out what rules you should apply in your application.


# echo 1 > /proc/sys/kernel/sysrq
# echo b > /proc/sysrq-trigger


Of course the disk errors meant that it was unable to boot but ‘The Big Hammer’ struck me as something extremely useful.

Difficult to learn

This used to be mostly true. Javascript was a poorly documented language, often only documented in tutorial form by w3schools, or technically documented as ECMAScript. The absolute wealth of tutorials and blog posts made the good information few and far between. Largely when looking for information on how to perform a particular function you had to download some sample code and figure out how it was done based on that.

More recently though Javascript has caught the wave that is trying to standardize the web and this has somewhat improved the situation. Browser manufacturers are documenting their Javascript implementations and largely converging on a common standard. Additionally many helper libraries have been introduced to make the task of working on Javascript even easier. Once Javascript may have been difficult to learn, but as of late this is no longer true.

Javascript Documentation:

• Mozilla Developer Center – Javascript
• WebKit DOM Reference
• MSDN – JScript Language Reference
• Opera Libraries

Subtle differences in interpretation

This is is one of the biggest problems you still see in Javascript today. You will often find developers writing functions to simply deal with the differences between browsers, there are even entire libraries dedicate to to abstracting away the differences. If I had a dollar for every implementation of a function to get a XMLHttpRequest object across browsers, I wouldn’t need my job.

Unfortunately it is still however very important to know the differences between implementations of Javascript if you plan on writing anything that will run on more than one browser. These difference may be in the features available in the language, in the Document Object Model or in the way the browser handles CSS. Thankfully many people work on documenting the difference and abstracting around them in libraries.

Javascript Implementations Differences:

• Quirks Mode
• Browser Migration – MDN
• Web Browser CSS Support

Javascript Libraries:

• DOJO ToolKit
• Prototype
• jQuery
• script.aculo.us
• MiniAJAX

Random Thought: If only Facebook didn’t get in the way of Javascript all the time…

To attempt to rectify this SPF was created. To setup SPF you add either a TXT or an SPF record to the DNS zone you will be sending from. This record defines which servers are allowed to send mail that is coming from that domain. So on my domain danielhall.me I could publish an SPF record that says only my mail server is allowed to send mail that ends in @danielhall.me. Any mailservers receiving mail that is from my domain but not coming from an address listed in my SPF record can see that the mail is likely forged and throw it away. SPF works well in most situations but fails at a very common use case. If someone I send mail to tries to forward it to another address using an automatic process (no clicking forward in their client) then the mail will appear to come from my domain when it gets to the user it was forwarded to, however it will have came from the original recipients mailserver.

DKIM solves this problem by giving each sending mailserver a cryptographic key pair. The public keys is then published in a DNS record in that zone and stores the private key somewhere safe on the server. The server then proceeds to sign the headers (especially the From: header) and the body of all outgoing emails. This signature is then attached to the email as an extra header. When the receiving server get the email it gets the signature and uses that along with the list of signed headers to verify the signature against the public key of the signing domain. This means as long as the mail has passed through an authorised mailserver at any point it will be considered valid.

Setting up a DKIM is relatively simple process. You will need access to the  zone records for your domain and access to the configuration of all the mailservers which all mail originating at your domain passes through. You also need to be aware that signing mail makes it slightly more processor intensive to send an email. If you send a large amount of email this difference could be quite significant. If you’re using sendmail you may be able to alleviate it by switching to a less resource hungry MTA like Exim. You should also note that in some configurations DKIM can not be setup. For example if you use masquerading in sendmail DKIM will always fail as sendmail will modify the from header after signing.

Ultimately DKIM is a good move for the internet community at large, especially when combined with SPF. DKIM mail is assured to come from the sender and can be cryptographically proven so. While it does take more take a little more effort to setup and maintain it assures mail from your domain is secure and can be assured to have come from you or your company. Ultimately DKIM can protect your company against phishing attempts and boost your spam scores.

Random thought: What would Email look like if it were designed today?

Wireless networks can either be encrypted or unencrypted but in both these cases only the payload is encrypted. The packet headers which contain information about who the packet is addressed to and who it is from. The reasons for this are similar to why you might write a letter in code, but you would not write the envelope in code. In an unencrypted network the whole packet is sent in clear text including the envelope and contents. The difference between these analogies and how a real network works though is that to read the envelope you need to physically obtain it and there is only one copy. A wireless network broadcasts everything to everyone within 100 meters.

This isn’t really a problem if your network is encrypted as people will not be able to read it easily. If however your network is not encrypted its the equivalent of yelling out everything that you type into and read from your PC. Almost all banking websites will ask your PC to use extra encryption, but many other sites will not. So anyone in a 100 meter range of your computer or access point can watch everything you do on your computer.

What Google were trying to do was get a list of the locations of these access points. So they would have captured the headers of all packets they saw, grabbed the wireless routers address out of it and marked its location on a map. Except according to them they accidentally put code in that captured the whole packet. This meant that for all the unencrypted networks the Google Street View cars drove past they may have captured private information.

There is a class action in Germany against Google for capturing this data, and more can be expected elsewhere soon. Suing Google for this is like walking in to a public place, yelling out a bunch of private information and then suing anyone who happened to be recording at the time, or suing someone for writing down smoke signals you send to someone from the top of a mountain. If your access point is sending data unencrypted then every wireless device within 100 meters cannot help but hear your data, you’re just lucky most will ignore it.

If you really cared about your privacy you would at least make some attempt to restrict others access to your data. Not knowing is much an excuse as not knowing people were recording in that shopping mall. Don’t take your privacy for granted, check whether your network is encrypted, and if you don’t know how, get someone who does. Ignorance is not an excuse! This time it was Google, the next time it could be an identity thief.

Random Thought: If privacy is so important to people at the moment, what’s with all the data on Facebook?

Many sites offer web services, xml data and json encoded data. These can provide almost anything from the weather, to search results, to advanced APIs. To use these services from your site using javascript you’ll have to employ one of the methods below.

Signing Javascript

Firefox allows you to sign your Javascript and place it in a jar file. This will give your code more privileges, You can also request these permissions explicitly without having your code signed, but having a dialog box appear for every AJAX request could get very tiring for the user. Another problem with this approach is that it isn’t documented very well and its Firefox specific. The first link in the references section deals with this method.

Access-Control Headers

This is the w3 approved method of allowing a client from another domain to access your web service. It is a server side method and requires no changes on the client to implement. This is both and advantage and a disadvantage. If you have control over the server then this method is simple, otherwise (for sites such as Yahoo API or other public services) you will not be able to implement this. It should also be noted that this was implemented in Firefox 3.5 so it can’t be used with earlier versions, or other browsers.

To use this method you tell your service to output extra headers that tell the browser whether access was allowed or denied.

Flash Enabled xmlhttprequest

This method involves using an invisible flash player to perform the actual request then handing the result back to the Javascript for processing. Flash still performs permission checking by looking for a /crossdomain.xml file in the root directory of the domain the request is being made to. There are several libraries that implement this approach and a few even implement in a way which is compatible with xmlhttprequest. One downside is this Flash is required, though recently Flash is required for several major sites and most browsers will have it installed.

Add Sites To Trusted Zone

Internet Explorer allows and denies cross-domain based xmlhttprequests based on the security setting. This approach is likely not going to be used on the Internet as it requires user interaction and is Internet Explorer specific. On a corporate Intranet this is slightly less difficult but not by much.

Apache mod_proxy

With this method you use the same server you shared the page from to proxy the requests automatically to the server with the data you’re fetching. For this to work your version of Apache has to be compiled with proxy support or you need to have the mod_proxy dso loaded. This method increases the latency of requests as they must first go via your server. It should also be noted that this cannot be implemented in .htaccess file and must be done in the main configuration.

Manual Proxy

If you don’t have control over your servers configuration then you can mimic the above method by writing a script that forwards the variables required and forwards back the data. This approach can even be more preferable than the above method as it allows you to preprocess the variables and cache the data if required.

References

http://www.mozilla.org/projects/security/components/signed-scripts.html

http://dev.w3.org/2006/waf/access-control/

http://developer.yahoo.com/javascript/howto-proxy.html

https://developer.mozilla.org/En/HTTP_Access_Control

http://ejohn.org/blog/cross-site-xmlhttprequest/

http://ajaxpatterns.org/XMLHttpRequest_Call

http://ajaxpatterns.org/Flash-enabled_XHR

Random Thought: Can you use AJAX to make web applications cleaner?

EncFS is an encrypted filesystem based on FUSE. It transparently encrypts files stored in it and places them on another volume. This is in contrast to block level encrypted filesystems which transparently encrypt the data under the filesystem layer as it is being written to disk. Think of EncFS as a bind mount, except that the source for the mount is encrypted and the place it is mounted to is the only place it is available unencrypted.

The main advantage of EncFS filesystems is that when backing up only the files which have changed need to be backed up. This means it works perfectly with tools such as rsnapshot. Another advantage is that the filesystem doesn’t need a block of disk allocated to it and will shrink and expand as the files inside change.

Finally because this is all implemented with FUSE it is all done in userspace. No root access is required (apart from setting FUSE up) to create and alter encfs filesystems.

Setting Up an EncFS Volume

So the first thing you need to do to setup an encfs volume is to install FUSE and EncFS. If you don’t have root access you will have to ask your sysadmin to do this for you, otherwise follow your distribution specific method of installing new packages. On Fedora it is called ‘fuse-encfs’ and on Debian/Ubuntu its called ‘encfs’. On some older systems users wishing to use FUSE may need to be added to the correct group.

First you need to decide where you will put the encfs volume, and where you’ll mount it. I usually put mine in /home/daniel/.crypt and mount it to /home/daniel/crypt. But feel free to name it whetever you want. When you’ve decided run the EncFS with those arguments, for example to use the example I specified it would look like this:

<daniel@server ~>$ encfs /home/daniel/.crypt /home/daniel/crypt
The directory "/home/daniel/.crypt/" does not exist. Should it be created? (y,n) y
The directory "/home/daniel/crypt" does not exist. Should it be created? (y,n) y
Creating new encrypted volume.
Please choose from one of the following options:
 enter "x" for expert configuration mode,
 enter "p" for pre-configured paranoia mode,
 anything else, or an empty line will select standard mode.
?>

Standard configuration selected.

Configuration finished.  The filesystem to be created has
the following properties:
Filesystem cipher: "ssl/aes", version 2:2:1
Filename encoding: "nameio/block", version 3:0:1
Key Size: 192 bits
Block Size: 1024 bytes
Each file contains 8 byte header with unique IV data.
Filenames encoded using IV chaining mode.
File holes passed through to ciphertext.

Now you will need to enter a password for your filesystem.
You will need to remember this password, as there is absolutely
no recovery mechanism.  However, the password can be changed
later using encfsctl.

New Encfs Password:
Verify Encfs Password:

As you can see the directories don’t need to be created first. There is also a prompt for what security settings you want to use. Hitting enter will give you standard settings, but for something more powerful you should hit ‘p’ then enter. You can now proceed to place files in /home/daniel/crypt and they will be encrypted and placed into /home/daniel/.crypt. If you don’t believe me go ahead and check.

See? I told you so. Now you can unmount it using ‘fusermount -u /home/daniel/crypt’ and mount it again using encfs /home/daniel/.crypt /home/daniel/crypt and typing your password.

Random Thought: When travelling to other countries, local laws may mean that customs can search your laptop, including encrypted filesystems. You may have to reveal your key, or be arrested.

The Internet filter, quite simply put is technically infeasible. The filter will work by directing all requests from Australian users towards a site containing RC content to a filtering device. This device then relays all requests to that site to the actual server, unless a requests is made for a blocked page, which will instead return a page indicating the site is blocked. This is similar to the way the firewall in China and other countries with a national Internet filter. This method is effective in that it is often 100% effective (which means that every page on the blocked list is blocked, with no false positives) when done right. There is a problem however, this method does not scale well. If the government were to block a page on a large site (as was attempted to Wikipedia in the UK) then the filter would not be able to handle the load. Secondly it appears to the administrators of that site that all requests are coming from a few IP adresses. This could cause Wikipedia to eventually block all Australians either because the requests will look similar to a DDOS or because they have no way to distinguish between users and need to prevent abuse. Although the filter may be 100% accurate at blocking web traffic it will not be capable of dealing with many other varieties of Internet data.

The proposed filter will only be capable of filtering standard web traffic from web browsers. The Internet consists of a large number of computers talking in any number of protocols. While web traffic is one of these there are many other ways to exchange information. This filter will not be capable of filtering email, bit torrent, edonkey, gnutella, XMPP, DDC, SSH, VPN, TOR and that is only naming a small portion. Many people caught to have been in possession of child pornography and other illegal content are found to have downloaded it via peer to peer technology. This is because standard web traffic makes it easy to trace and identify the owner, where as peer to peer traffic can be hidden much easier. Secondly web traffic can be ‘tunnelled’ or hidden inside these other protocols and this way completely bypass the filter. This means anyone with sufficient knowledge or five minutes to learn will be able to configure their PC to hide their data amongst an SSH or VPN connection. These technical arguments come from my experience as a systems Administrator, but there are other arguments not so technical.

Steven Conroy has said that the filter will only deal with RC rated content, however there is no transparency about what will be blocked. The government can’t publish a list of sites that are blocked because that will effectively give people looking for this content a list of places to find it. Without knowing what sites are being blocked we won’t know if or when the government decides that they would like to start blocking sites that are debating for or against abortion, euthanasia or any other politically sensitive topic. It may be interesting to know that the definition for RC content includes pages instructing in any crime, which would include euthanasia. A representative for Steven Conroy has specifically stated the filter won’t be filtering pages related to euthanasia but because of this broad definition it could be changed at any time and we wouldn’t know until after the material was blocked.

I am a Unix Systems Administrator, and for the reasons listed above, and more covered better by other bloggers, I am opposed to the filter proposed by Senator Steven Conroy and the Labor government. I urge my readers who are also opposed to the filter to write to your local MP, to Senator Conroy, to Tony Smith (Shadow Minister Minister for Broadband, Communications
and the Digital Economy). If all else fails and the Government does not see sense then use your vote. The filter will not work and will waste taxpayer money that could be used in many better ways.

Random Thought: Will posting instructions about how to bypass the filter be illegal?