28 Jan 2006

status update: mostly linux geekery

been quite busy recently, with my weird sleeping patterns, distractions and changes in my life -- there's not been too much time.
upgrade to lighttpd

the server that hosts this blog and all my other webby stuff has been up and down. that's because apache has finally decided to gobble up all the memory and turn the server into a memory swapping party. turns out that enabling mod_python and serving 4 django projects off apache wasn't such a good idea. each apache process was taking nearly 17M RSS.

i started migrating all my django projects, which includes my blog, my own personal feedfilter and two private projects, and a trac installation to lighttpd. i chose lighttpd because i've heard alot of good things about it, and it is also written by the same guy who wrote modlogan, a web log analyzer that i used for many years, all the way back in my CSE days (thats 4-5 years ago)!

turns out it was a good choice, lighttpd is not as mature as apache, but it does simple things very well. the server is not multiprocess, so it runs very light and fast, weighing in at 2M RSS total (compare that to nearly 100M apache was consuming with mod_python, mod_php with 5 processes). the flip side is that all your services have to run outside of the server. with 4 django projects and 1 trac project, you're looking at 5 python servers, each with 2 processes, clocking in just under 60M.

nice thing with lighttpd is it is so easy to understand where things are going, not like apache. i know that the request comes in, goes thru a bit of rewriting and out to the python servers using fastcgi. ideally i'd like to get rid of all of apache, but right now it is still the front end to the servers and it proxies the request to the lighttpd. also there are just some rewrite rules that don't work well under lighttpd, plus mod_security is an indispensable spam filter. other than that, there's no reason to continue to stick with apache.

things i learnt while tuning the server

1. mod_python adds about 4-6M to the RSS of an apache process.
2. mod_php adds around 6-7M to the RSS of an apache process.
3. mysql 4.0 leaks memory for some reason, restarting it once in a while helps.
4. postgresql is as stable and lean as a rock. i'm very impressed with it.
5. using django in fastcgi means around 8-10M RSS processes depending on how many databases or complicated your web app is.
6. if you use mod_proxy using ProxyPass, MAKE SURE YOU USE ProxyPassReverse.

final note
the server is much more responsive now, and plus the fine people at bytemark.co.uk which we run our server off has just doubled the bandwidth (luckily, because i ran over last month!) and increased ram and disk space. damn server tuning is addictive.
oh yeah, i have my own fastcgi launcher scripts that launch django and trac projects inspired by the django fastcgi lighttpd howto. if anyone wants them, i might put them online.

You can reply to me about this on Twitter: