[OmniOS-discuss] [smartos-discuss] expected memory bandwidth of Supermicro X9DRi-F and X9SRI-F

[Keith Wesolowski]

On Mon, Jul 08, 2013 at 07:01:13PM -0700, Paul B. Henson wrote:

> I have two servers I'm putting together, one using the X9SRI-F and the 
> other the X9DRi-F motherboard. Both have Intel Xeon E5-2620 processors 
> (one for the X9SRI-F and two for the X9DRi-F). Both have Crucial 
> CT2KIT102472BB160B (2 x 8G 1Gx72 DDR3 PC3-12800 Reg/ECC) DIMMs installed
> (6 on X9SRI-F for 48G, 8 on X9DRi-F for 64G). Testing the memory, the 
> X9SRI-F shows bandwidth of about 12GB/s, whereas the X9DRi-F shows only 
> about 8GB/s. This seems considerably lower than the 42GB/s bandwidth the
> E5-2620 is supposed to support, even considering this isn't the most 
> expensive/fasted RAM on the market. I'll also suprised at the 4GB/s 
> difference between the single-proc and dual-proc board.
> 
> I was wondering if anybody else was using these boards and if so what 
> memory bandwidth memtest86+ reports, as I've been unable to find a good 
> source of expected bandwidth (versus theoretical maximum bandwidth). I 
> asked supermicro if it was expected to have such a difference in 
> performance between the single processor versus dual processor board, 
> and all they would tell me is "we don't support that memory" <sigh>.

Get the documents from Intel describing expected memory bandwidth and
optimal configurations.  There are significant -- nay, dramatic --
differences depending on how you populate your controllers and channels.
It's not surprising that having only one CPU would significantly degrade
performance, since the MCs are part of the CPUs.  There are also
population rules based on DIMM speed, rank count, power levels, and so
on, and if you do not follow them you will have reduced performance.