Tag Archives: Apache

Simple Configuration Service

The Basics

So I’ve seen a few implementations of ‘config services’ being built in all sorts of {ruby,python,php,java} applications. It really gets on my nerves to see people implementing things that could be much simpler. As Engineers we should really strive to get the most simple product built in a flexible way. So here is my go at implementing a ‘config service’ (basically a key value store optimised for reads). This was originally tweeted in a set of four tweets (1, 2, 3, 4).

Tweet 1

A 'Config Service' that fits in a tweet:
<Location /config>
  DAV svn
  SVNPath /var/svn/config
  SVNAutoversioning on
</Location>

Simply place this in your apache server configuration. You’ll need to create the SVN repo, which can be done with ‘svnadmin create /var/svn/config’ then chmod it to be writiable and readable by apache. What we are doing here is turning on WebDAV (which lets us upload and make directories on the apache server) and turning on SVN backed autoversioning which means that every change we make will be recorded in the SVN repo.

Tweet 2

To get a variable:
curl -s http://config/config/namespace/key

This simply performs a GET request for a key. You can’t do this until you add some keys to the server.

Tweet 3

To change a variable:
echo "value" | curl -T- http://config/config/namespace/key

This creates/modifies a key. You will need to have the namespace created first, which is in the next tweet.

Tweet 4

Make a new namespace:
curl -XMKCOL http://config/config/namespace

This creates a namespace to put keys in.

Extensions

Users

The service above doesn’t support users, so anyone can modify the keys, and the svn logs look like this:

------------------------------------------------------------------------
r5 | (no author) | 2012-10-15 12:36:33 +1100 (Mon, 15 Oct 2012) | 2 lines

Autoversioning commit:  a non-deltaV client made a change to
/namespace/key
------------------------------------------------------------------------

All we need to do is require the user to authenticate when doing PUT requests to the server. You can use this method to restrict certain namespaces too. Generate a htpasswd file for the users and place it in /var/svn/users and change the code to read as follows.

<Location /config>
  DAV svn
  SVNPath /var/svn/config
  SVNAutoversioning on
  AuthName "Config Service"
  AuthType Basic
  AuthUserFile /var/svn/users
  <LimitExcept GET>
    require valid-user
  </Limit>
</Location>

Viewing the logs

Simply do this:

svn checkout http://config/config
cd config
svn log

You can also use this to make bulk/atomic changes in the same way you would make changes to a subversion repository.

mod_pagespeed is not (always) the answer

What is mod_pagespeed

Google recently released a chunk of code in the form of an Apache module. The idea is that you install it in your Apache server, it sits in between your application and the web browser and modifies the served requests to make the page load faster.
It does this by using combinations of filters, some are well known best practices, others are newer ideas. For example on filter simply minifies your JavaScript while another embeds small images in a page using data-uris. The changes these filters make range from low risk, to high risk. It should be noted that not all the filters will improve the page time some even making pages slower in some cases.

So what’s the issue?

The issue here really isn’t mod_pagespeed, but it’s the way people are viewing it. In my job as a Web Performance Engineer I have had several people recently say to me “let’s put mod_pagespeed on our web server to make it faster”. This is a break from normal attitudes, if someone were to to say “we should put our images into data-uris” then people would question the speed benefit, or the extra load on the server. For some reason when Google implement a page speed module people just assume that it will make their page faster, and that it will work in their environment. The truth is that Google really have no idea what the module will do to your page.

The second issue is that all these tweaks can usually be better implemented at the application level. If you minimize all your JavaScript as part of your build process then the web server will not have to do it for you. The same applies to data-uris. If they are simply part of the page then the browser doesn’t need to read in the extra image, uuencode it, then compress it. All that is quite a lot of work, which only really needs to be done once.

So what should I use mod_pagespeed for then?

You don’t always have access to the application code. If you are using third party software then before mod_pagespeed you may have had no control over the minification of CSS. This is where the module really shines. It gives you a layer between the application code and the web browser where you can apply all sorts of performance tuning.

The other advantage I can see is for looking for the best tunings to apply to your application quickly. You can setup mod_pagespeed and and run experimental tests with the filters on of and with a control to quickly figure out what rules you should apply in your application.

Cross-Domain AJAX

When making an xmlhttprequest from a website the browser will restrict you to the site from which the script came. This is a security precaution. If sites were able to tell the browser to make requests from other domains then they would be able to DDOS a site with a users browser. There are legitimate reasons to make requests to other sites though.

Many sites offer web services, xml data and json encoded data. These can provide almost anything from the weather, to search results, to advanced APIs. To use these services from your site using javascript you’ll have to employ one of the methods below.

Signing Javascript

Firefox allows you to sign your Javascript and place it in a jar file. This will give your code more privileges, You can also request these permissions explicitly without having your code signed, but having a dialog box appear for every AJAX request could get very tiring for the user. Another problem with this approach is that it isn’t documented very well and its Firefox specific. The first link in the references section deals with this method.

Access-Control Headers

This is the w3 approved method of allowing a client from another domain to access your web service. It is a server side method and requires no changes on the client to implement. This is both and advantage and a disadvantage. If you have control over the server then this method is simple, otherwise (for sites such as Yahoo API or other public services) you will not be able to implement this. It should also be noted that this was implemented in Firefox 3.5 so it can’t be used with earlier versions, or other browsers.

To use this method you tell your service to output extra headers that tell the browser whether access was allowed or denied.

Flash Enabled xmlhttprequest

This method involves using an invisible flash player to perform the actual request then handing the result back to the Javascript for processing. Flash still performs permission checking by looking for a /crossdomain.xml file in the root directory of the domain the request is being made to. There are several libraries that implement this approach and a few even implement in a way which is compatible with xmlhttprequest. One downside is this Flash is required, though recently Flash is required for several major sites and most browsers will have it installed.

Add Sites To Trusted Zone

Internet Explorer allows and denies cross-domain based xmlhttprequests based on the security setting. This approach is likely not going to be used on the Internet as it requires user interaction and is Internet Explorer specific. On a corporate Intranet this is slightly less difficult but not by much.

Apache mod_proxy

With this method you use the same server you shared the page from to proxy the requests automatically to the server with the data you’re fetching. For this to work your version of Apache has to be compiled with proxy support or you need to have the mod_proxy dso loaded. This method increases the latency of requests as they must first go via your server. It should also be noted that this cannot be implemented in .htaccess file and must be done in the main configuration.

Manual Proxy

If you don’t have control over your servers configuration then you can mimic the above method by writing a script that forwards the variables required and forwards back the data. This approach can even be more preferable than the above method as it allows you to preprocess the variables and cache the data if required.

References

http://www.mozilla.org/projects/security/components/signed-scripts.html

http://dev.w3.org/2006/waf/access-control/

http://developer.yahoo.com/javascript/howto-proxy.html

https://developer.mozilla.org/En/HTTP_Access_Control

http://ejohn.org/blog/cross-site-xmlhttprequest/

http://ajaxpatterns.org/XMLHttpRequest_Call

http://ajaxpatterns.org/Flash-enabled_XHR

Random Thought: Can you use AJAX to make web applications cleaner?