[OmniOS-discuss] Routing challlenges

[Jim Klimov]

7 апреля 2016 г. 20:50:13 CEST, "Schweiss, Chip" <chip at innovates.com> пишет:
>On Thu, Apr 7, 2016 at 12:51 PM, Michael Talbott <mtalbott at lji.org>
>wrote:
>
>> Oh, I see. Sorry about that, reading it on my phone didn't render
>your
>> diagram properly ;)
>>
>> The reason this is happening is because the omnios box has knowledge
>of
>> both subnets in its routing table and it always takes the shortest
>path to
>> reach an ip destination.
>>
>
>That's definitely the reason, but not correct when stateful firewalls
>are
>involved.
>
>>
>> So you will need to put the "clients" in a unique subnet that always
>> passes through the firewall in both directions (in a subnet that's
>not
>> shared by the omnios machines). Any attempt to add/modify a static
>route to
>> the omnios box to resolve this will likely fail (it'll just move the
>> problem from one network to the other one and cause your "services"
>network
>> to route improperly).
>>
>
>The problem is each person who manages these (there are 4) are also
>clients
>of the services (SMB, NFS).
>
>For management, going through the firewall is fine, it is low volume,
>but
>the services need to be on the same VLAN or else the 1gb firewall will
>choke on the 10gb services.
>
>
>> Either that, or remove the firewall as a hop, set sshd to listen only
>on
>> the management IP, and add a management vlan interface to the clients
>> allowed to connect.
>>
>>
>I've considered this too, but I have some floating IP attached to zfs
>pools
>in an HA cluster that SSH needs to bind to.
>
>Unless I can get the network stack on the management vlan to act
>independently of the other interfaces, it may come to modifying the
>sshd_config and restarting ssh each time a pool is started or stopped
>on a
>host.
>
>-Chip
>
>
>
>> Michael
>>
>>
>> On Apr 7, 2016, at 10:25 AM, Michael Talbott <mtalbott at lji.org>
>wrote:
>>
>> It sounds like you're using the same subnet for management and
>service
>> traffic, that would be the problem causing the split route. Give each
>vlan
>> a unique subnet and traffic should flow correctly.
>>
>> Michael
>> Sent from my iPhone
>>
>> On Apr 7, 2016, at 8:52 AM, Schweiss, Chip <chip at innovates.com>
>wrote:
>>
>> On several of my OmniOS hosts I have a setup a management interface
>for
>> SSH access on an independent VLAN.   There are service vlans attached
>to
>> other nics.
>>
>> The problem I am having is that when on privileged machine on one of
>the
>> vlans also on the service side that has access to the management SSH
>port
>> the TCP SYN comes in the management VLAN but the SYNACK goes out the
>> service VLAN instead of routing back out its connecting port.   This
>causes
>> a split route and the firewall blocks the connection because the
>connection
>> never appears complete.
>>
>> Traffic is flowing like this:
>> client                   firewall                 omnnios
>> 10.28.0.106 ->   10.28.0.254->10.28.125.254  -> 10.28.125.44
>>
>> 10.28.0.106  <--------------------------------- 10.28.0.44
>>
>> How can I cause connections to only communicate on the vlan that the
>> connection is initiated from?
>>
>> I don't want to use the 10.28.0.44 interface because that is a
>virtual IP
>> and will not always be on the same host.
>>
>> -Chip
>>
>>
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>
>>
>>
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>OmniOS-discuss mailing list
>OmniOS-discuss at lists.omniti.com
>http://lists.omniti.com/mailman/listinfo/omnios-discuss

Just in case, remember that with ipfikter on the host you can effect 'source-routing' and assign (at least non-dynamically) what router should be taken for what sort of connections. It works by blocking 'destination-routed' packets if they match your rule, and rewriting them as destined to go out of another NIC and to another gateway.

Example from work (I think I posted it more than once in the past - then if you find those discussions, they might also hold some neat ideas), this is essentially the first rule in /etc/ipf/ipf.conf (before loopbacks, allow-all-for-some and head's for complicated other rule trees):

# enforce that packets coming out of an interface go to the correct subnet
# rhetoric question: does this skip the firewall rules below in the file?
block out quick on vlan186 to vlan81:81.X.Y.1 from 81.X.Y.0/24 to any
block out quick on vlan81 to vlan186:192.168.186.1 from ! 81.X.Y.0/24 to any

Note the "!" negation in the second rule.

HTH,
Jim
--
Typos courtesy of K-9 Mail on my Samsung Android