Portsentry is a handy little program that listens to a range of ports and lets you know if anyone tries to connect on them. Since attackers will usually run a variety of probes to determine available services before attempting to compromise a system, this can alert you to potential attacks. Portsenty also supports raw sockets under Linux and so can also detect most stealth attacks (SYN, Xmas Tree, etc.).
The issue is what to do when you get scanned. It entertains me to know a bit more about the situation, so these are a couple of programs I wrote for the situation:
scan.pl: a little perl script to run different utilities like traceroute and whois to provide some information about the hostscan-wrapper.sh: a shell script to convert between the agruments of portsentry and the scanning script. Also it emails the results to an administrator