[OmniOS-discuss] OmniOS based redundant NFS

[sergey ivanov]

Thanks, Stephan!

Please, explain "The reason to use two x two separate servers is, that
the mirrored zpool's vdevs look the same on each NFS head".

I understand that, if I want to have the same zpool based on iscsi
devices, I should not mix local disks with iscsi target disks.

But I think I can have 2 computers, each exporting a set of local
disks as iscsi targets. And to have iscsi initiators on the same
computers importing these targets to build zpools.

Also, looking at sbdadm, I think I can 'create lu /dev/rdsk/c0t0d3s2'.

Ok, I think I would better try it and report how it goes.
-- 
  Regards,
  Sergey.

-- 

Regards,
Sergey Ivanov


On Wed, Sep 27, 2017 at 4:34 PM, Stephan Budach <stephan.budach at jvm.de> wrote:
> Hi Sergey,
>
> ----- Ursprüngliche Mail -----
>> Von: "sergey ivanov" <sergey57 at gmail.com>
>> An: "omnios-discuss" <omnios-discuss at lists.omniti.com>
>> Gesendet: Mittwoch, 27. September 2017 21:31:05
>> Betreff: [OmniOS-discuss] OmniOS based redundant NFS
>>
>> Hi,
>> as end-of-life of r151014 approaches, we are planning upgrade for our
>> NFS servers.
>> I'm thinking about 2 servers providing ISCSI targets, and 2 another
>> OmniOS servers using these ISCSI block devices in mirrored ZPOOL
>> setup. IP address for NFS service can be a floating IP between those
>> 2
>> servers.
>
> This is quite the same setup, as we have it. I am running two omniOS hosts as iSCSI targets and RSF-1 on two other omniOS hosts as NFS heads, where the NFS VIPs are failling over between. This setup has been very stable for quite some time and failovers have been occurring on a couple of occasions.
>
>> I have the following questions:
>> 1. Are there any advantages to have separate ISCSI target servers and
>> NFS servers or I should better combine one ISCSI target and NFS
>> server on each of 2 hosts?
>
> The reason to use two x two seperate servers is, that the mirrored zpool's vdevs look the same on each NFS head.
> This makes for a very straight forward setup, but on the other hand brings some interesting decisions to the table, when it comes to the design of the iSCSI targets.
>
> We all know, that ZFS works best when presented with raw devices and the closest to that would be iSCSI to raw devices/partitions. However, this will leave you to decide how you want to arrange your targets. If you chose to have fewer targets with more LUNs, you will face some interesting challenges when it comes to device failures, which will have you to offline that whole target on your NFS heads, leaving you running with a degraded zpool on your NFS head. I just went through that and I can tell you that you will need to prepare for such a case, and the more drives you are using the more challenging it becomes.
>
>
>> 2. I do not want snapshots, checksums, and other ZFS features for
>> block devices at the level where they are exported as ISCSI targets,
>> -
>> I would prefer these features at the level where these block devices
>> are combined into mirror Zpools. Maybe it's better to have these
>> ISCSI
>> target servers running some not so advanced OS and have 2 Linux
>> boxes?
>
> Me neither, but I chose omniOS as my iSCSI targets nevertheless. I do make heavy use of ZFS' features like snapshots and clones on my NFS heads and I am very comfortable with that. No bells and whistles on my target nodes.
>
>> 3. But if I have SSD for intent log and for cache, - maybe they can
>> improve performance for ZVOLs used as block devices for ISCSI
>> targets?
>>
>
> It will depend on your workload. I do also have some S3700s for ZIL on some of my iSCSI-based ZPOOls.
>
>> Does anybody have experience setting up such redundant NFS servers?
>
> Well… yes, and afaik, there are also some other people around this list, who are using similar setups, short of the iSCS target nodes, providing non-ZFS based LUNs, that seems to be more exotic… ;)
>
>> --
>> Regards,
>> Sergey Ivanov
>
> Cheers,
> Stephan