[OmniOS-discuss] expected memory bandwidth of Supermicro X9DRi-F and X9SRI-F

[Bob Friesenhahn]

On Mon, 8 Jul 2013, Paul B. Henson wrote:

> I have two servers I'm putting together, one using the X9SRI-F and the other 
> the X9DRi-F motherboard. Both have Intel Xeon E5-2620 processors (one for the 
> X9SRI-F and two for the X9DRi-F). Both have Crucial CT2KIT102472BB160B (2 x 
> 8G 1Gx72 DDR3 PC3-12800 Reg/ECC) DIMMs installed
> (6 on X9SRI-F for 48G, 8 on X9DRi-F for 64G). Testing the memory, the X9SRI-F 
> shows bandwidth of about 12GB/s, whereas the X9DRi-F shows only about 8GB/s. 
> This seems considerably lower than the 42GB/s bandwidth the
> E5-2620 is supposed to support, even considering this isn't the most 
> expensive/fasted RAM on the market. I'll also suprised at the 4GB/s 
> difference between the single-proc and dual-proc board.
>
> I was wondering if anybody else was using these boards and if so what memory 
> bandwidth memtest86+ reports, as I've been unable to find a good source of 
> expected bandwidth (versus theoretical maximum bandwidth). I asked supermicro 
> if it was expected to have such a difference in performance between the 
> single processor versus dual processor board, and all they would tell me is 
> "we don't support that memory" <sigh>.

There would be a big difference between peak (carefully multi-threaded 
with non-overlapping access across memory channels and DIMMs) and 
single-threaded memory bandwidth.  More CPUs means more parallel 
channels to the memory.  Each motherboard has an ideal DIMM layout for 
maximum bandwidth for installed memory.  If too few large DIMMs are 
installed then not enough memory I/Os can occur at once.  If too many 
smaller DIMMs are installed, then the memory needs to be clocked 
slower so that errors don't occur.  With many DIMMs, using buffered 
memory can keep the clock speeds up.

Regardless, actual performance depends on how your applications use 
the memory.

Bob
-- 
Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/