[OmniOS-discuss] OmniOS based redundant NFS

Sat Sep 30 08:17:11 UTC 2017

I have added a similar solution with full service and storage failover 
of two identical storage servers last year into my napp-it web ui as an 
experimental solution. Main difference is that I do nor use single disks 
as targets but the whole ZFS pools on two servers to mirror them over 
the network and share the resulting network mirror. This will allow a 
head to function with full redundancy if the other server fails. As you 
only need one LUN per server this reduces complexity.  I also switch 
target ip and NFS/SMB service ip for an instant service failover, see my 
solution under http://www.napp-it.org/doc/downloads/z-raid.pdf

Gea
@napp-it.org

Am 30.09.2017 um 08:45 schrieb Stephan Budach:
> Hi Sergey,
>
> ----- Ursprüngliche Mail -----
>> Von: "sergey ivanov" <sergey57 at gmail.com>
>> An: "Stephan Budach" <stephan.budach at jvm.de>
>> CC: "omnios-discuss" <omnios-discuss at lists.omniti.com>
>> Gesendet: Freitag, 29. September 2017 22:30:31
>> Betreff: Re: [OmniOS-discuss] OmniOS based redundant NFS
>>
>> Thanks, Stephan.
>> I did a simple test with creating lu over physical disks for use as
>> ISSI targets, and it worked well. I am going to directly connect 2
>> servers and export their disks as separate I SCSI targets. Or maybe
>> different LUNs in a target. And then on the active server start
>> initiator to get these targets, and combine them into a pool of 2-way
>> mirrors so that it stays degraded but working if one of the servers
>> dies.
> Well, different targets mean, that you will be able to service single disks on one node, without having to degrade the whole zpool, but only the affected vdevs. On the other hand, there is more complexity since, you will have of course quite a big number of iSCSI targets to login to. This may be ok, if the number doesn't get too hight, but going with hundreds of disks, I chose to use fewer targets with more LUNs.
>
> One thing to keep in mind is, that stmfad allows you to create the guuid to your liking. That is that you can freely choose the last 20 bytes to be anything you want. I used that to ascii-code the node name and slot into the guid, such as that it displays on my NFS heads, when running format. This helps a lot in mapping the LUNs to drives.
>
>> So, manual failover for this configuration will be the following. If
>> the server to be disabled is still active, stop NFS, export zpool on
>> it, stop iscsiadm, release shared IP. On the other server: import
>> zpool and start NFS, activate shared IP.
> I am using the sharenfs properties of ZFS, but you will likely have to run zpool export -f <zpool> if you want to fail over the service, since the zpool is still busy. Also, you'd better set zpool failmode to panic instead of wait, such as that an issue triggers a reboot, rather than keeping you NFS head waiting.
>
>> I read once there are some tricks which make clients do not recognize
>> NFS server is changed underneath all mounts, but I never tried it.
> The only issue I came across was, when I deliberatley failed over the NFS service forth and back within the a too short period, which causes the NFSd on the former primary node to re-use the tcp packets numbers, insisting on reusing it's old NFS connections to the clients. I solved that by resetting the NFSd each time a service starts on any NFS head. The currently connected NFS clients are not affected by that and this solved this particular issue for me.
>
> Cheers,
> Stephan
>
>> --
>>    Regards,
>>    Sergey Ivanov.
>>
>> Regards,
>> Sergey Ivanov
>>
>>
>> On Thu, Sep 28, 2017 at 12:49 AM, Stephan Budach
>> <stephan.budach at jvm.de> wrote:
>>> Hi Sergey,
>>>
>>> ----- Ursprüngliche Mail -----
>>>> Von: "sergey ivanov" <sergey57 at gmail.com>
>>>> An: "Stephan Budach" <stephan.budach at jvm.de>
>>>> CC: "omnios-discuss" <omnios-discuss at lists.omniti.com>
>>>> Gesendet: Mittwoch, 27. September 2017 23:15:49
>>>> Betreff: Re: [OmniOS-discuss] OmniOS based redundant NFS
>>>>
>>>> Thanks, Stephan!
>>>>
>>>> Please, explain "The reason to use two x two separate servers is,
>>>> that
>>>> the mirrored zpool's vdevs look the same on each NFS head".
>>>>
>>>> I understand that, if I want to have the same zpool based on iscsi
>>>> devices, I should not mix local disks with iscsi target disks.
>>>>
>>>> But I think I can have 2 computers, each exporting a set of local
>>>> disks as iscsi targets. And to have iscsi initiators on the same
>>>> computers importing these targets to build zpools.
>>>>
>>>> Also, looking at sbdadm, I think I can 'create lu
>>>> /dev/rdsk/c0t0d3s2'.
>>>>
>>>> Ok, I think I would better try it and report how it goes.
>>> Actually, things can become quite complex, I'd like to reduce the
>>> "mental" involvement to the absolute minimum, mainly because we
>>> often faced a situation where something would suddenly break,
>>> which had been running for a long time without problems. This is
>>> when peeple start… well maybe not panicking, but having to recap
>>> what the current setup was like and what they had to do to tackle
>>> this.
>>>
>>> So, uniformity is a great deal of help on such systems - at least
>>> for us. Technically, there is no issue with mixing local and
>>> remote iSCST targets on the same node, which serves as an iSCSI
>>> target and a NFS head.
>>>
>>> Also, if one of the nodes really goes down, you will be loosing
>>> your failover NFS head as well, maybe not a big deal and depending
>>> on your requirements okay. I do have such a setup as well,
>>> although only for an archive ZPOOL, where I can tolerate this
>>> reduced redundancy for the benefit of a more lightweight setup.
>>>
>>> Cheers,
>>> Stephan
>>
>>
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://omniosce.org/ml-archive/attachments/20170930/a93cfa7a/attachment-0001.html>