saving a copy of this here since things on the interweb tend to disappear.
There is some very good info here specifically, I found the following hints helpful:
Start isakmpd with
isakmpd -d. Isakmpd will output things like wrong
file permissions and typos in the configuration file. On connect you
might see things like "NO PROPOSAL CHOOSEN" which can either mean that
your configuration parameters between the client and the server doesn't
match, or that you have typed the wrong pre-shared key.
isakmpd -L' and
tcpdump -avs 1440 -r /var/run/isakmpd.pcap
This one is really nice to check if your configurations between the client and the server match and also to learn howto create isakmpd.conf files for new clients. With
isakmpd -L isakmpd will dump, in tcpdump
format, everything it sends and recieves to /var/run/isakmpd.pcap. You
then check what happened with
tcpdump -avs 1440 -r /var/run/isakmpd.pcap.
Look here for an example output of isakmpd -L and tcpdump.
This output is typically what you want to send to the mailing list when you want help with something if the above doesn't help you.