CLOSED: IPSec VPN stopped working after Amahi reinstall

[silverblaze]

Last summer I initially setup an Amahi server as a XenServer VM and everything worked great, IPsec VPN had no issues. I had a physical machine become available so I decided to try to make my life easier by moving (reinstalling) Amahi to this physical machine. (Not sure if it matters but I had to build this new machine using the Amahi Full instructions, the express install kept failing from missing packages). I was able to get Amahi installed on the new machine, and on my lan everything seems to be running fine (dns and dhcp are working).

I got into my office and tried to use the same VPN configuration I had been using before (I did update to the new random "Secret") and it says it has established the tunnel and gives me the standard Amahi MOTD on the VPN connection. Unfortunately, no traffic is going through the VPN (this is true for both my Windows setup and off of my iPhone). When I try to connect from Windows (Using the Shrew Soft VPN client) I notice the following messages in the /var/log/messages file

Jan 30 13:54:42 localhost racoon: WARNING: Ignored attribute INTERNAL_ADDRESS_EXPIRY
Jan 30 13:54:42 localhost racoon: WARNING: authtype mismatched: my:hmac-sha peer:hmac-md5
Jan 30 13:54:42 localhost racoon: WARNING: trns_id mismatched: my:3DES peer:AES
Jan 30 13:54:42 localhost racoon: WARNING: trns_id mismatched: my:DES peer:AES

from the iPhone all I see is

Jan 30 14:16:16 localhost racoon: [70.196.69.172] ERROR: notification INITIAL-CONTACT received in aggressive exchange.
Jan 30 14:16:17 localhost racoon: WARNING: Ignored attribute INTERNAL_ADDRESS_EXPIRY
Jan 30 14:16:17 localhost racoon: WARNING: Ignored attribute 28683

In both cases I cant get to an internal webserver, even when using the ip address instead of machine name. I know that raccoon handles the IPsec communications, but that is all I know about it. Any suggestions on where to look next?

Note: I turned off the Amahi VM before I ever turned on the fully installed physical Amahi box, and both machines are using the same fixed IP address, so I know my port forwarding is still working and going to the correct box (I even rebooted my physical router to eliminate that).

Any suggestions or pointers will be appreciate (note, I have deleted the network adapter from the vm install so I can look at any configuration on the old one if some wants me to compare before/after configuration files - I didn't see any changes in the main racoon one.

Thanks in advance!

[bigfoot65]

Recommend you check the Amahi wiki for guidance. There are some troubleshooting steps there plus a tutorial on the making changes as well.

[silverblaze]

I did look in there, but I suspect this issue is specific to IPsec since networking and all other behavior from the new Amahi machine seems to be working. I didn't post this as a bug since I suspect it is something in the configuration, but I am a Fedora novice (I mainly use Ubuntu) and for me the IPsec VPN worked out of the box when I had it on the virtual machine so I have no debugging experience with IPsec. (I chose the IPsec VPN option because every time I have looked at OpenVPN I get completely confused on how to configure it and trust that I have actually configured it in a secure manner).

I didn't really see any IPsec specific info on the wiki (aside from references on how to configure it for iOS and to use the Shrew Soft vpn client for Microsoft Windows setups) and from what I can understand of the error messages I posted it seems to be a problem in the encryption handshakes after the initial connect (which the guides I have found don't even discuss, the defaults seem to be supposed to work), and I cant think of why that would have changed between the two installs.

[bigfoot65]

Did you see IPsec VPN wiki page? Also take a look at OpenVPN troubleshooting as it might help.

There is mention of TLS handshake issues.

[silverblaze]

I checked the firewall and racoon config files and everything seemed to be correct, so gave up and I have switched over to the OpenVPN setup instead. Thanks for the attempt to help, I think this is just something deep in the os that I don't have enough knowledge to debug at this point.

Thanks all the same!

[bigfoot65]

Understand. Sorry I could not assist more with a solution. I think you will find the OpenVPN solution works well. I use it for all my devices to include Android with no issues.