[VPN] "PAYLOAD_MALFORMED" VPN problem with SonicWall Pro and XAUTH

Dale Shaw DShaw at exceed.com.au
Tue May 27 00:03:51 EDT 2003


Hi,

I am attempting to configure a SonicWall Pro to support remote access
VPN users using a pre-shared key and XAUTH (using RADIUS to a Microsoft
IAS Server). The SonicWall box is already working with just a pre-shared
key -- I am just adding an additional authentication factor.

Here is an excerpt from the log viewer on a VPN client machine:

8<---
10:44:56.129 My Connections\sonic2-client - Initiating IKE Phase 1 (IP
ADDR=203.x.x.34)
10:44:56.129 My Connections\sonic2-client - SENDING>>>> ISAKMP OAK MM
(SA, VID)
10:44:56.479 My Connections\sonic2-client - RECEIVED<<< ISAKMP OAK MM
(SA, VID)
10:44:56.499 My Connections\sonic2-client - Peer is NAT-T capable
10:44:56.499 My Connections\sonic2-client - SENDING>>>> ISAKMP OAK MM
(KE, NON, NAT-D, NAT-D, VID, VID, VID)
10:44:56.960 My Connections\sonic2-client - RECEIVED<<< ISAKMP OAK TRANS
*(Opaque)
10:44:56.960 My Connections\sonic2-client - Received message for
non-active SA
10:44:57.020 My Connections\sonic2-client - RECEIVED<<< ISAKMP OAK MM
(KE, NAT-D, NAT-D, NON, VID, VID, VID)
10:44:57.040 My Connections\sonic2-client - SENDING>>>> ISAKMP OAK MM
*(ID, HASH, NOTIFY:STATUS_INITIAL_CONTACT)
10:44:57.390 My Connections\sonic2-client - RECEIVED<<< ISAKMP OAK TRANS
*(HASH, ATTR)
10:45:03.049 My Connections\sonic2-client - SENDING>>>> ISAKMP OAK TRANS
*(HASH, ATTR)
10:45:03.429 My Connections\sonic2-client - RECEIVED<<< ISAKMP OAK TRANS
*(HASH, ATTR)
10:45:03.429 My Connections\sonic2-client - SENDING>>>> ISAKMP OAK TRANS
*(HASH, ATTR)
10:45:03.429 My Connections\sonic2-client - Initiating IKE Phase 2 with
Client IDs (message id: 32713DEC)
10:45:03.429   Initiator = IP ADDR=y.y.y.214, prot = 0 port = 0
10:45:03.429   Responder = IP SUBNET/MASK=10.0.0.0/255.255.255.0, prot =
0 port = 0
10:45:03.429 My Connections\sonic2-client - SENDING>>>> ISAKMP OAK QM
*(HASH, SA, NON, ID, ID)
10:45:03.830 My Connections\sonic2-client - RECEIVED<<< ISAKMP OAK MM
*(ID, HASH, NOTIFY:STATUS_RESP_LIFETIME)
10:45:03.830 My Connections\sonic2-client - Established IKE SA
10:45:03.830    MY COOKIE b5 f2 7e 43 97 20 ec c7
10:45:03.830    HIS COOKIE 99 b6 fa 4d 4b 2e c b6
10:45:03.850 My Connections\sonic2-client - RECEIVED<<< ISAKMP OAK INFO
(NOTIFY:PAYLOAD_MALFORMED)
10:45:03.850 My Connections\sonic2-client - Received invalid NOTIFY
message (doi = 1, protocol_id = 0)
10:45:03.850 My Connections\sonic2-client - Discarding SA negotiation
10:45:03.850 My Connections\sonic2-client - Deleting IKE SA (IP
ADDR=203.x.x.34)
10:45:03.850    MY COOKIE b5 f2 7e 43 97 20 ec c7
10:45:03.850    HIS COOKIE 99 b6 fa 4d 4b 2e c b6
8<---

I have confirmed that the RADIUS server is configured correctly and
-can- successfully authenticate a user using the SonicWall's web
management interface. I've made sure the IAS server is configured to
accept PAP as an authentication type. I'm fairly certain the problem is
not related to a RADIUS configuration problem between the SonicWall and
the IAS server, in fact when the client throws up the authentication
dialog and credentials are entered, a matching 'success' entry is made
in the IAS and SonicWall log.

Strangely, in testing this configuration the other day, phase 2 did
succeed *once*. Nothing had changed, and after de-activating and
re-activating the security policy, it continued to fail. Very strange -
the only thing I can think of that was different about the time that it
worked was that the client was loading up a heavy web page at the time
and there were a few IKE retransmits as a result (the client is
modem-connected).

Can anyone shed any light on what the "PAYLOAD_MALFORMED" error is
about? I couldn't find any reference to it in SonicWall's support
knowledgebase. The VPN client OS is Windows 2000 Professional. In order
to go back to a working configuration, all I did was disable the
'require XAUTH' option on the SA and changed the SafeNet client back to
Pre-Shared Keys (instead of PSK + XAUTH).

Cheers,
Dale




More information about the VPN mailing list