21 Feb 2006

apache is dead! long live lighttpd!

after a two day delay over the weekend, as usual the album art widget release unleashes a huge amount of hits to my site from macupdate, apple and some japanese mac sites which i keep on discovering every month. this month it is mac.page.ne.jp, bringing a huge number of hits to my site. it actually has an interesting description of my widget (translated to english via rikai). maybe i should for a japanese translation of the widget some time to show off my bad japanese (どうぞ よろしくおねがいします~~). maybe i'll put in an easter egg in the next version if you have a japanese locale.

oh and talking about my widget, there's a german pdf that has an article about it. interesting because i got some pdf referrerals in my logs which i thought was odd!

anyway, back to the point of the post, the sudden surge of traffic last night killed apache. well, not exactly killed, i use apache as a frontend proxy to a lighttpd/fastcgi backend. and ever since i switched, performance hasn't improved that much, and in some instances, its even worse. i couldn't figure out a reason why, at first i thought it was my python fastcgi server (flup) on the other end. but after some tweaking and tuning, i realised the bottleneck was not there, but in apache. the version of apache i'm using seems to have a problem dealing with alot of proxy requests even with bumping the MaxServers variable quite high.

i had been holding off making the full switch to lighttpd because there are a number of domains hosted on the server i'm using, and some of them make use of features that aren't supported in lighttpd like .htaccess. so what i've done is made lighttpd the frontend server that does its best job serving static files and fastcgi, and relegating apache to a backend server where i proxy requests for domains that need php and .htaccess. with this way, i can slowly ween them off the addictive apache drug onto a more lean setup.

but i'm very glad i've put in the work to do this, now liquidx.net is super responsive compared to what it was like yesterday and the last 6 months.

You can reply to me about this on Twitter: