[OmniOS-discuss] strangeness ssh into omnios from oi_151a9

[Lauri Tirkkonen]

On Thu, Sep 10 2015 11:28:12 +0200, Richard PALO wrote:
> Le 08/09/15 14:12, Dan McDonald a écrit :
> >
> >>On Sep 8, 2015, at 12:32 AM, Richard PALO <richard-S783fYmB3Ccdnm+yROfE0A at public.gmane.org> wrote:
> >>
> >>before the traffic seemingly when things go sour... I notice a Window changed to 1024??
> >
> >Which side is advertising the window change again?  And which side is
> >running -gate from 2ed96329a073f74bd33f766ab982be14f3205bc9 ?
> >
> >This thread has been paged out, so to speak, for long enough.  Can
> >you give me the context of which machine is running what to explain
> >the context of the snoop file?
> >
> >Thanks,
> >Dan
> >
> Just for completeness, same histoire from the OI side, snoop and ssh
> 
> here, 192.168.1.2 is smicro (oi_151a9)
> >e1000g0 192.168.1.1          255.255.255.255          00:12:ef:21:9c:f8
> >e1000g0 192.168.1.2          255.255.255.255 SPLA     00:30:48:f4:33:f0
> and 192.168.1.1 is an Orange Business Services SDSL router.

Are these captures both from the same connection? If so, there is
obviously a middle box modifying the traffic. On *both* ends, it looks
like the other end is sending an empty ACK requesting the window change
to 1024 (packet 41 in snoop.output, with dst 192.168.0.6, and packet 41
in snoop-OI.output, with dst 192.168.1.2). Both of these TCP segments
are missing the required timestamp options.

With the fix for 5850, illumos should never send a segment without
timestamps on a connection which has negotiated timestamps (this one
has, since they are present on previous segments). In addition, as part
of 5850, we follow the RFC recommendation to drop any arriving segments
*without* timestamps on a timestamp-negotiated connection [0]. This is
likely the reason why your use case worked before; the older behavior
was to stop generating timestamps altogether on a connection where any
received segment omits them, but that's the wrong thing to do. There is
a new dtrace probe 'tcp:::droppedtimestamp' which should fire whenever a
segment is dropped by this behavior. You could use that to verify my
speculation, eg.

    # dtrace -n 'tcp:::droppedtimestamp { trace(probefunc); }'

should generate output when the connection hangs (and more information
about the connection is available in (tcp_t*)arg0).

Based on the data you have made available I believe this is an issue
with a middlebox injecting erroneous traffic into the TCP stream for
both peers. This injected segment is ignored by the box with the fix for
5850 appliec, but it causes the older illumos box to stop generation of
timestamps, after which all segments it sends are rejected by the newer
box.

Oh, and in the future, please post snoop capture files (from snoop -o);
it's much easier to find the desired information in those :)

[0]: http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/inet/tcp/tcp_input.c#2878

-- 
Lauri Tirkkonen | lotheac @ IRCnet