(I don't want to hear about how crazy I am to want that! :)
Focus-follows-mouse is also known as point-to-focus, pointer focus, and (in some implementations) sloppy focus. [Add other terms that will make this more searchable!] X-mouse
You can do it for Terminal.app by issuing the following command at the command line:
defaults write com.apple.Terminal FocusFollowsMouse -bool true
For X11 apps you can do this:
defaults write com.apple.x11 wm_ffm -bool true
In Snow Leopard, use this instead:
defaults write org.x.X11 wm_ffm -bool true
Apparently there's a program called CodeTek Virtual Desktop that'll emulate it systemwide, but it costs $$ (and they never got a version out for OSX Leopard).