[OmniOS-discuss] iSCSI poor write performance

[Martin Truhlář]

Hi Hanno

Thank you for your advice, unfortunatelly on dpool is ashift already set to 12 without any impact on performance.

Martin

dpool:
    version: 5000
    name: 'dpool'
    state: 0
    txg: 423442
    pool_guid: 8301756920046328435
    hostid: 390978448
    hostname: 'archnas'
    vdev_children: 6
    vdev_tree:
        type: 'root'
        id: 0
        guid: 8301756920046328435
        create_txg: 4
        children[0]:
            type: 'mirror'
            id: 0
            guid: 1196673952777635344
            metaslab_array: 34
            metaslab_shift: 33
            ashift: 12
            asize: 1000191557632
            is_log: 0
            create_txg: 4
            children[0]:
                type: 'disk'
                id: 0
                guid: 8124964091934866578
                path: '/dev/dsk/c1t50014EE00400FA16d0s0'
                devid: 'id1,sd at n50014ee00400fa16/a'
                phys_path: '/scsi_vhci/disk at g50014ee00400fa16:a'
                whole_disk: 1
                DTL: 490
                create_txg: 4
            children[1]:
                type: 'disk'
                id: 1
                guid: 9348868466535755709
                path: '/dev/dsk/c1t50014EE2B40F14DBd0s0'
                devid: 'id1,sd at n50014ee2b40f14db/a'
                phys_path: '/scsi_vhci/disk at g50014ee2b40f14db:a'
                whole_disk: 1
                DTL: 489
                create_txg: 4
        children[1]:
            type: 'mirror'
            id: 1
            guid: 9943497592636049032
            metaslab_array: 38
            metaslab_shift: 33
            ashift: 12
            asize: 1000191557632
            is_log: 0
            create_txg: 34
            children[0]:
                type: 'disk'
                id: 0
                guid: 2705367364579591435
                path: '/dev/dsk/c1t50014EE05950B131d0s0'
                devid: 'id1,sd at n50014ee05950b131/a'
                phys_path: '/scsi_vhci/disk at g50014ee05950b131:a'
                whole_disk: 1
                DTL: 488
                create_txg: 34
            children[1]:
                type: 'disk'
                id: 1
                guid: 5412107877453931054
                path: '/dev/dsk/c1t50014EE2B5E5A6B8d0s0'
                devid: 'id1,sd at n50014ee2b5e5a6b8/a'
                phys_path: '/scsi_vhci/disk at g50014ee2b5e5a6b8:a'
                whole_disk: 1
                DTL: 487
                create_txg: 34
        children[2]:
            type: 'mirror'
            id: 2
            guid: 4337686502023930092
            whole_disk: 0
            metaslab_array: 40
            metaslab_shift: 33
            ashift: 12
            asize: 1000191557632
            is_log: 0
            create_txg: 65
            children[0]:
                type: 'disk'
                id: 0
                guid: 12065653943105190290
                path: '/dev/dsk/c1t50014EE05958C51Bd0s0'
                devid: 'id1,sd at n50014ee05958c51b/a'
                phys_path: '/scsi_vhci/disk at g50014ee05958c51b:a'
                whole_disk: 1
                DTL: 486
                create_txg: 65
            children[1]:
                type: 'disk'
                id: 1
                guid: 7956964322079560255
                path: '/dev/dsk/c1t50014EE0595617ACd0s0'
                devid: 'id1,sd at n50014ee0595617ac/a'
                phys_path: '/scsi_vhci/disk at g50014ee0595617ac:a'
                whole_disk: 1
                DTL: 482
                create_txg: 65
        children[3]:
            type: 'mirror'
            id: 3
            guid: 13515811785015942389
            metaslab_array: 43
            metaslab_shift: 33
            ashift: 12
            asize: 1000191557632
            is_log: 0
            create_txg: 119
            children[0]:
                type: 'disk'
                id: 0
                guid: 2010958773514461606
                path: '/dev/dsk/c1t50014EE0AEAE7540d0s0'
                devid: 'id1,sd at n50014ee0aeae7540/a'
                phys_path: '/scsi_vhci/disk at g50014ee0aeae7540:a'
                whole_disk: 1
                DTL: 484
                create_txg: 119
            children[1]:
                type: 'disk'
                id: 1
                guid: 6920452460884353416
                path: '/dev/dsk/c1t50014EE0AEAE9B65d0s0'
                devid: 'id1,sd at n50014ee0aeae9b65/a'
                phys_path: '/scsi_vhci/disk at g50014ee0aeae9b65:a'
                whole_disk: 1
                DTL: 491
                create_txg: 119
        children[4]:
            type: 'mirror'
            id: 4
            guid: 13450996153705674574
            metaslab_array: 45
            metaslab_shift: 30
            ashift: 9
            asize: 120020795392
            is_log: 1
            create_txg: 172
            children[0]:
                type: 'disk'
                id: 0
                guid: 642840549260709901
                path: '/dev/dsk/c1t55CD2E404B88ABE1d0s0'
                devid: 'id1,sd at n55cd2e404b88abe1/a'
                phys_path: '/scsi_vhci/disk at g55cd2e404b88abe1:a'
                whole_disk: 1
                DTL: 494
                create_txg: 172
            children[1]:
                type: 'disk'
                id: 1
                guid: 17473204952243782915
                path: '/dev/dsk/c1t55CD2E404B88E4CFd0s0'
                devid: 'id1,sd at n55cd2e404b88e4cf/a'
                phys_path: '/scsi_vhci/disk at g55cd2e404b88e4cf:a'
                whole_disk: 1
                DTL: 493
                create_txg: 172
        children[5]:
            type: 'mirror'
            id: 5
            guid: 6461803899340698053
            metaslab_array: 520
            metaslab_shift: 33
            ashift: 12
            asize: 1000191557632
            is_log: 0
            create_txg: 422833
            children[0]:
                type: 'disk'
                id: 0
                guid: 15790186799979059305
                path: '/dev/dsk/c1t50014EE0AEABB8E7d0s0'
                devid: 'id1,sd at n50014ee0aeabb8e7/a'
                phys_path: '/scsi_vhci/disk at g50014ee0aeabb8e7:a'
                whole_disk: 1
                create_txg: 422833
            children[1]:
                type: 'disk'
                id: 1
                guid: 3033691275784652782
                path: '/dev/dsk/c1t50014EE0AEB44327d0s0'
                devid: 'id1,sd at n50014ee0aeb44327/a'
                phys_path: '/scsi_vhci/disk at g50014ee0aeb44327:a'
                whole_disk: 1
                create_txg: 422833
    features_for_read:
        com.delphix:hole_birth
        com.delphix:embedded_data


-----Original Message-----
From: Hanno Hirschberger [mailto:hannohirschberger at googlemail.com] 
Sent: Wednesday, September 23, 2015 2:43 PM
To: omnios-discuss at lists.omniti.com
Subject: Re: [OmniOS-discuss] iSCSI poor write performance

Hi Martin,

On 23.09.2015 10:51, Martin Truhlář wrote:
> Tests revealed, that problem is somewhere in disk array itself.

are you familiar with the ashift problem on 4k drives? My best guess would be that the 1 TB WD drives are emulating a block size of 512 bytes while using 4k sectors internally. OmniOS is using a ashift value of 9 then to align the data efficiently (on 512 byte sectors!). This slows the whole pool down - I had the same problem before. The ashift value has to be 12 on 4k drives!

Try the command 'zdb' to gather the values for your drives. Look for
'ashift: 9' oder 'ashift: 12'.

Regards,

Hanno
_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss