Bad Behavior is a set of PHP scripts which prevents spambots from accessing your site by analyzing their actual HTTP requests and comparing them to profiles from known spambots. It goes far beyond User-Agent and Referer, however. Bad Behavior is available for several PHP-based software packages, and also can be integrated in seconds into any PHP script.
I’m experimenting with a modified version of this, curious about the impact on traffic.
It could be dramatic. Blocked around thirty spambot requests in first 5 minutes of running. And they didn’t seem to be false positives either.