Your little slug has only 32mb of memory. I know that you want to run a lot of useful applications with your slug then why do not you remove some of the applications already installed in your slug and free more memory. Actually I can feel the difference in my slug after removing the mentioned applications. It takes much less to load my website right now...
First you can think of disabling and removing exim4. (mailserver AFAIK - free 7 mb ram)
update-rc.d -f exim4 remove
apt-get --purge remove exim4-daemon-light exim4-config exim4-base
Also if you do not use nfs service (drive sharing AFAIK) just disable and remove it.
update-rc.d -f nfs-common remove
apt-get --purge remove nfs-common