[vpn] Checkpoint/Netscreen VPN IKE Error Messages

[dparmer at dsscorp.com]

Hello,

We are having trouble for the past few weeks trying to get a Netscreen 5 to
an NT 4.0 Checkpoint 4.1 SP5 site to site VPN operational.  Generally IKE
Phase 1 completes between the firewalls, but only very infrequently does
IKE Phase 2 compete between the firewalls, according to the Checkpoint and
Netscreen logs.  When Phase 2 does complete, outbound traffic is encrypted
but the return decrypts do not come back.  We have encryption schemes
identical for Phase 1 & Phase 2 between the Checkpoint & Netscreen boxes.
When Phase 2 does not complete, messages in the log viewer include
"Received delete SA from Peer" and  "Received Notification from Peer:
payload malformed", with the source address being the Checkpoint firewall
and the destination being the Netscreen.

Just for kicks, we tried creating a VPN connection to two other Checkpoint
4.1 sites (one had NT 4.0 using 4.1 SP2 and another had W2K using 4.1 SP5)
using the same Netscreen 5 box with identical encryption properties, and
both Phase 1 & Phase 2 became operational, and traffic was being encrypted
and decrypted in both directions.  Thus I eliminated the possibility that
the Netscreen may be the issue.

I then compared a few files on the various firewalls (crypt.def,
objects.C), and could not find anything except cosmetic items that were
different. I also tried the various debugging tools (fw monitor, fw -d d,
FWIKE_DEBUG), and have examined the resultant file output, and was not able
to decipher anything enlightening from these files, although I must admit
that I don't know exactly what kind of packet flow or sequencing I should
be looking for.

Thanks in advance for any assistance.

============================
Dave Parmer
Distributed Systems Services
610-927-2026
dparmer at dsscorp.com



VPN is sponsored by SecurityFocus.com