[OmniOS-discuss] Low latency networking

Tue Aug 7 16:12:31 EDT 2012

Hi Chris,

On Aug 6, 2012, at 9:07 AM, Chris Nagele wrote:

> Hi Eric,
> 
> Richard had me run:
> 
> truss -dD ping -s 10.135.1.11
> 
> From this result I can't seem to find anything useful. The longest
> time is taken at the top:
> 
> 0.0556  0.0000 write(1, "\n", 1)                               = 1
> 0.9975  0.9419     Received signal #14, SIGALRM, in pollsys() [caught]

It is already over by now. Look for the difference between the first sendto(3, ...) and 
recvmsg(3,...) something like:

 0.0210  0.0013 write(1, " P I N G   1 9 2 . 1 6 8".., 34)      = 34
 0.0212  0.0002 sendto(3, "\b\0F6 ^\r1F\0\0   U1F P".., 64, 32768, 0x0806BD88, 16) = 64
 0.0214  0.0002 sigaction(SIGALRM, 0x08047BA0, 0x08047C20)      = 0
 0.0215  0.0001 lwp_sigmask(SIG_SETMASK, 0x00000000, 0x00000000, 0x00000000, 0x00000000) = 0xFFBFFEFF [0xFFFFFFFF]
 0.0216  0.0001 alarm(1)                                        = 0
 0.0220  0.0004 pollsys(0x08047AA0, 1, 0x00000000, 0x00000000)  = 1
 0.0222  0.0002 recvmsg(3, 0x08047C10, 32768)                   = 84

 -- richard

> 0.9976  0.0001 pollsys(0x08047B50, 1, 0x00000000, 0x00000000)  Err#4 EINTR
> 0.9976  0.0000 lwp_sigmask(SIG_SETMASK, 0x00002000, 0x00000000,
> 0x00000000, 0x00000000) = 0xFFBFFEFF [0xFFFFFFFF]
> 0.9976  0.0000 sendto(3, "\b\0 AA3068A\001 lE51F P".., 64, 32768,
> 0x0807BA90, 16) = 64
> 0.9977  0.0001 alarm(1)                                        = 0
> 0.9977  0.0000 setcontext(0x08047660)
> 0.9978  0.0001 pollsys(0x08047B50, 1, 0x00000000, 0x00000000)  = 1
> 0.9978  0.0000 recvmsg(3, 0x08047CD0, 32768)                   = 84
> 0.9978  0.0000 lwp_sigmask(SIG_SETMASK, 0x00002000, 0x00000000,
> 0x00000000, 0x00000000) = 0xFFBFFEFF [0xFFFFFFFF]
> 0.9978  0.0000 lwp_sigmask(SIG_SETMASK, 0x00000000, 0x00000000,
> 0x00000000, 0x00000000) = 0xFFBFFEFF [0xFFFFFFFF]
> 64 bytes from 10.135.1.11:  0.9979       0.0001 write(1, " 6 4   b y t
> e s   f r o".., 27)      = 27
> icmp_seq=1.  0.9979      0.0000 write(1, " i c m p _ s e q = 1 .  ",
> 12)        = 12
> time=0.225 ms 0.9980     0.0001 write(1, " t i m e = 0 . 2 2 5   m"..,
> 13)      = 13
> 
> Looking at snoop, I see the following:
> 
> 18.34001  10.135.1.10 -> debian-host  length:   98  ICMP Echo request
> (ID: 1780 Sequence number: 5)
> 18.34019  debian-host -> 10.135.1.10  length:   98  ICMP Echo reply
> (ID: 1780 Sequence number: 5)
> 18.34641  debian-host -> *            length:   60  ARP C Who is
> 10.135.1.10, 10.135.1.10 ?
> 18.34643  10.135.1.10 -> debian-host  length:   42  ARP R
> 10.135.1.10, 10.135.1.10 is 90:e2:ba:f:ce:ec
> 19.34000  10.135.1.10 -> debian-host  length:   98  ICMP Echo request
> (ID: 1780 Sequence number: 6)
> 19.34018  debian-host -> 10.135.1.10  length:   98  ICMP Echo reply
> (ID: 1780 Sequence number: 6)
> 20.33998  10.135.1.10 -> debian-host  length:   98  ICMP Echo request
> (ID: 1780 Sequence number: 7)
> 20.34015  debian-host -> 10.135.1.10  length:   98  ICMP Echo reply
> (ID: 1780 Sequence number: 7)
> 
> Which all looks pretty normal as well. I might give superping.d a try
> as well (http://www.c0t0d0s0.org/archives/7479-Dont-misuse-ping!.html).
> 
> Let me know if you have any other ideas.
> 
> Thanks,
> Chris
> 
> 
> On Mon, Aug 6, 2012 at 10:06 AM, Eric Sproul <esproul at omniti.com> wrote:
>> On Sun, Aug 5, 2012 at 8:07 PM, Chris Nagele <nagele at wildbit.com> wrote:
>>> For some reason though, ping times from OmniOS to Debian are still
>>> much slower, above 0.2ms. I know it is nit picking, but we invested in
>>> 10g for lower latency, so I am trying to get the most out of it. From
>>> Debian to OmniOS I get about 0.08ms still.
>> 
>> Hi Chris,
>> It'd be great to see some tcpdumps of this traffic.  The most curious
>> thing to me is that you see differences in RTT between Debian and
>> OmniOS systems just by switching sender and receiver.  Perhaps there's
>> something about the payload of a typical ping packet between the
>> systems.
>> 
>> Eric
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

--
ZFS Performance and Training
Richard.Elling at RichardElling.com
+1-760-896-4422

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://omniosce.org/ml-archive/attachments/20120807/83f85b88/attachment.html>