[osiris] Re: HELP! Hours of session key negotiation failure

Thu Nov 9 21:10:24 EST 2006

Thanks again for your help and suggestions.  I'll respond in the text -
John

On Thu, 2006-11-09 at 18:31 -0500, Justis Peters wrote:
> John,
> 
> John A. Sullivan III wrote:
> > We do actually use very granular and restrictive security.  Promoting,
> > using and developing the ISCS network security management project to
> > create multi-layered, compartmentalized networks to prevent the
> > escalation of privileges in the eventuality of an internal attack
> > (http://iscs.sourceforge.net). However, the monitoring station presents
> > an X.509 cert that gives it unlimited access to the entire security
> > cloud so that it can run both Osiris and Nessus scans.
> >   
> 
> Would there be any value to granting a similarly liberal certificate (on
> a temporary basis) that allows the host in question to have unlimited
> traffic between itself and the osirismd console?  It seems like you've
> pretty much squashed the possibility that this is networking related,
> but it might be a try just to cross that off the list for good.
I would think it would be handled by the connection tracking as related
and established traffic but, it's probably not a bad idea to try.  I
opened up all outbound access for the client but still had the same
results.
> 
> > I did run tcpdump on the client and and management console and could see
> > the packet exchange.  If I telnet on port 2265, it connects but then
> > immediately disconnects.  Other clients stay connected and return some
> > characters.  I assume that is because they have a memory resident key
> > whereas the new client does not.
> 
> This sounds like a very useful lead.  What does the tcpdump look like?
> Have you tried comparing it to a tcpdump between osirismd and one of the
> hosts that actually work?
I put the tcpdump on both sides (the client and the management console).
Each show the same packets so nothing is getting lost.

Here is the osirismd side:
20:53:21.988311 IP 192.168.26.2.4749 > 172.26.204.31.2265: S
2128532705:2128532705(0) win 5840 <mss 1460,sackOK,timestamp 97346274
0,nop,wscale 2>

20:53:22.097691 IP 172.26.204.31.2265 > 192.168.26.2.4749: S
104140346:104140346(0) ack 2128532706 win 5792 <mss
1368,sackOK,timestamp 1588999 97346274,nop,wscale 2>

20:53:22.097733 IP 192.168.26.2.4749 > 172.26.204.31.2265: . ack 1 win
1460 <nop,nop,timestamp 97346285 1588999>

20:53:22.205622 IP 172.26.204.31.2265 > 192.168.26.2.4749: F 1:1(0) ack
1 win 1448 <nop,nop,timestamp 1589011 97346285>

20:53:22.206794 IP 192.168.26.2.4749 > 172.26.204.31.2265: F 1:1(0) ack
2 win 1460 <nop,nop,timestamp 97346296 1589011>

20:53:22.328580 IP 172.26.204.31.2265 > 192.168.26.2.4749: . ack 2 win
1448 <nop,nop,timestamp 1589023 97346296>

Here is the client side:
20:53:19.791579 IP 192.168.26.2.4749 > 172.26.204.31.2265: S
2128532705:2128532705(0) win 5840 <mss 1368,sackOK,timestamp 97346274
0,nop,wscale 2>

20:53:19.804863 IP 172.26.204.31.2265 > 192.168.26.2.4749: S
104140346:104140346(0) ack 2128532706 win 5792 <mss
1460,sackOK,timestamp 1588999 97346274,nop,wscale 2>

20:53:19.905067 IP 192.168.26.2.4749 > 172.26.204.31.2265: . ack 1 win
1460 <nop,nop,timestamp 97346285 1588999>

20:53:19.905732 IP 172.26.204.31.2265 > 192.168.26.2.4749: F 1:1(0) ack
1 win 1448 <nop,nop,timestamp 1589011 97346285>

20:53:20.026878 IP 192.168.26.2.4749 > 172.26.204.31.2265: F 1:1(0) ack
2 win 1460 <nop,nop,timestamp 97346296 1589011>

20:53:20.026917 IP 172.26.204.31.2265 > 192.168.26.2.4749: . ack 2 win
1448 <nop,nop,timestamp 1589023 97346296>

> 
> Maybe someone else on the list (David Vasil?) knows enough about this
> part of the code that you could pretend to be one end or the other of
> the conversation.  I frequently do this to troubleshoot HTTP, SMTP,
> POP3, etc.  In this case, your largest obstacle would be the SSL
> negotatiation.  Maybe you could pair up "stunnel" or something of the
> sort with telnet/netcat/etc in order to get past that point.
That sounds interesting.  It's a bit beyond my skills.
Here is the result of a telnet to a working client on the same xen host
and the same network as the failing client - notice I had to interrupt
the telnet session with ^]:

[root at ltskbunk ~]# telnet nspub1dc1 2265
Trying 172.26.204.5...
Connected to nspub1dc1.atlasgroup.net (172.26.204.5).
Escape character is '^]'.
:!985
32/f��]Z/^]

Here is the telnet response to the failing client:

[root at ltskbunk ~]# telnet mail1dc1 2265
Trying 172.26.204.31...
Connected to mail1dc1.atlasgroup.net (172.26.204.31).
Escape character is '^]'.
Connection closed by foreign host.

Immediate closure.

By the way, here is the tcpdump of that telnet - same pattern as the
osiris connection attempt.  SYN SYN ACK initiated by the manager
followed by FIN FIN ACK initiated by the failed client.  Almost like it
doesn't like the proposal.

20:59:52.278091 IP 192.168.26.2.2386 > 172.26.204.31.2265: S
2555964865:2555964865(0) win 5840 <mss 1460,sackOK,timestamp 97385304
0,nop,wscale 2>

20:59:52.417117 IP 172.26.204.31.2265 > 192.168.26.2.2386: S
540426251:540426251(0) ack 2555964866 win 5792 <mss
1368,sackOK,timestamp 1628029 97385304,nop,wscale 2>

20:59:52.417161 IP 192.168.26.2.2386 > 172.26.204.31.2265: . ack 1 win
1460 <nop,nop,timestamp 97385318 1628029>

20:59:52.558655 IP 172.26.204.31.2265 > 192.168.26.2.2386: F 1:1(0) ack
1 win 1448 <nop,nop,timestamp 1628043 97385318>

20:59:52.558822 IP 192.168.26.2.2386 > 172.26.204.31.2265: F 1:1(0) ack
2 win 1460 <nop,nop,timestamp 97385332 1628043>

20:59:52.718289 IP 172.26.204.31.2265 > 192.168.26.2.2386: . ack 2 win
1448 <nop,nop,timestamp 1628059 97385332>

Here is the trace of the successful telnet:

21:03:03.792508 IP 192.168.26.2.4212 > 172.26.204.5.2265: S
2748088752:2748088752(0) win 5840 <mss 1460,sackOK,timestamp 97404456
0,nop,wscale 2>

21:03:03.917266 IP 172.26.204.5.2265 > 192.168.26.2.4212: S
1156474587:1156474587(0) ack 2748088753 win 5792 <mss
1368,sackOK,timestamp 10082969 97404456,nop,wscale 1>

21:03:03.917299 IP 192.168.26.2.4212 > 172.26.204.5.2265: . ack 1 win
1460 <nop,nop,timestamp 97404468 10082969>

21:03:04.041061 IP 172.26.204.5.2265 > 192.168.26.2.4212: P 1:61(60) ack
1 win 2896 <nop,nop,timestamp 10082981 97404468>

21:03:04.041091 IP 192.168.26.2.4212 > 172.26.204.5.2265: . ack 61 win
1460 <nop,nop,timestamp 97404481 10082981>

SYN SYN ACK PUSH ACK all initiated by the manager.

In the failed case, the client seems to dislike something the manager is
doing.

<snip>
-- 
John A. Sullivan III
Open Source Development Corporation
+1 207-985-7880
jsullivan at opensourcedevel.com

If you would like to participate in the development of an open source
enterprise class network security management system, please visit
http://iscs.sourceforge.net