想要通过 `LXD/LXC` 使用 `OpenVSwitch` 的 `Vlan` 功能

Want to use the `Vlan` feature of `OpenVSwitch` with `LXD/LXC`

我想使用 OpenVSwitchVlan 功能。这是我到目前为止所做的;

hussain@hussain:~$ lxc list
+----------+---------+------+------+------------+-----------+
|   NAME   |  STATE  | IPV4 | IPV6 |    TYPE    | SNAPSHOTS |
+----------+---------+------+------+------------+-----------+
| trusty-1 | STOPPED |      |      | PERSISTENT | 0         |
+----------+---------+------+------+------------+-----------+
| trusty-2 | STOPPED |      |      | PERSISTENT | 0         |
+----------+---------+------+------+------------+-----------+
| trusty-3 | STOPPED |      |      | PERSISTENT | 0         |
+----------+---------+------+------+------------+-----------+

还有这个;

hussain@hussain:~$ sudo ovs-vsctl show
a8498c25-8432-4174-9869-6eae38044cfe
    Bridge "br0"
        Controller ptcp
        Port "vport1"
            Interface "vport1"
        Port "enp1s0"
            Interface "enp1s0"
        Port "br0"
            Interface "br0"
                type: internal
    ovs_version: "2.5.0"

trusty-1 容器中我有这个 ;

root@trusty-1:~# cat /etc/network/interfaces.d/eth0.cfg 
#auto eth0
#iface eth0 inet static
#     address 172.16.3.250
#     netmask 255.255.254.0
#     gateway 172.16.2.1


auto eth0
iface eth0 inet static
        address 172.16.3.250
        network 172.16.0.0
        netmask 255.255.254.0
        gateway 172.16.2.1
        dns-nameservers 172.16.3.199 8.8.8.8
        dns-search google.com
        bridge_ports vport1
        bridge_stp off
        bridge_fd 0
        bridge_maxwait 0

我想做的是 trusty-1 容器使用 vport1 桥接端口,这样我就可以标记它然后对其进行操作对于以后的情况,但实际情况是这样的;

hussain@hussain:~$ sudo ovs-vsctl show
a8498c25-8432-4174-9869-6eae38044cfe
    Bridge "br0"
        Controller ptcp
        Port "vport1"
            Interface "vport1"
        Port vethHBNLTA
            Interface vethHBNLTA
        Port "enp1s0"
            Interface "enp1s0"
        Port "br0"
            Interface "br0"
                type: internal
    ovs_version: "2.5.0"

每次我启动 trusty-1 时都会创建一个临时端口(vethHBNLTA)。

如何确保容器 trusty-1 始终使用 vport1

谢谢。

P.S.: LXC IRC 上一位上帝派来的天使告诉我要做;

$ lxc config edit trusty-1

并添加

  eth0:
    name: eth0
    nictype: physical
    parent: vport1
    type: nic

设备下:

所以我做到了。 但是网络连接中断了。无法再从容器中 ping 主机或 Google。

然后我将容器配置更改为此;

devices:
  eth0:
    name: eth0
    nictype: bridged
    parent: vport1
    type: nic

现在我什至无法启动我的容器。错误日志显示了这一点;

hussain@hussain:~$ lxc info --show-log trusty-1
Name: trusty-1
Architecture: x86_64
Created: 2016/08/10 06:48 UTC
Status: Stopped
Type: persistent
Profiles: default

Log:

            lxc 20160823145623.424 INFO     lxc_start - start.c:lxc_check_inherited:251 - closed inherited fd 3
            lxc 20160823145623.424 INFO     lxc_start - start.c:lxc_check_inherited:251 - closed inherited fd 8
            lxc 20160823145623.426 INFO     lxc_container - lxccontainer.c:do_lxcapi_start:797 - Attempting to set proc title to [lxc monitor] /var/lib/lxd/containers trusty-1
            lxc 20160823145623.427 INFO     lxc_start - start.c:lxc_check_inherited:251 - closed inherited fd 8
            lxc 20160823145623.428 INFO     lxc_lsm - lsm/lsm.c:lsm_init:48 - LSM security driver AppArmor
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .reject_force_umount  # comment this to allow umount -f;  not recommended.
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for reject_force_umount action 0
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:do_resolve_add_rule:216 - Setting seccomp rule to reject force umounts

            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for reject_force_umount action 0
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:do_resolve_add_rule:216 - Setting seccomp rule to reject force umounts

            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .[all].
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .kexec_load errno 1.
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for kexec_load action 327681
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for kexec_load action 327681
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .open_by_handle_at errno 1.
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for open_by_handle_at action 327681
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for open_by_handle_at action 327681
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .init_module errno 1.
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for init_module action 327681
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for init_module action 327681
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .finit_module errno 1.
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for finit_module action 327681
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for finit_module action 327681
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .delete_module errno 1.
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for delete_module action 327681
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for delete_module action 327681
            lxc 20160823145623.428 INFO     lxc_seccomp - seccomp.c:parse_config_v2:456 - Merging in the compat seccomp ctx into the main one
            lxc 20160823145623.428 INFO     lxc_conf - conf.c:run_script_argv:367 - Executing script '/usr/bin/lxd callhook /var/lib/lxd 1 start' for container 'trusty-1', config section 'lxc'
            lxc 20160823145623.429 INFO     lxc_start - start.c:lxc_check_inherited:251 - closed inherited fd 3
            lxc 20160823145623.429 INFO     lxc_start - start.c:lxc_check_inherited:251 - closed inherited fd 8
            lxc 20160823145623.431 INFO     lxc_monitor - monitor.c:lxc_monitor_sock_name:178 - using monitor sock name lxc/d78a9d7e97b4b375//var/lib/lxd/containers
            lxc 20160823145623.451 DEBUG    lxc_start - start.c:setup_signal_fd:289 - sigchild handler set
            lxc 20160823145623.451 DEBUG    lxc_console - console.c:lxc_console_peer_default:469 - no console peer
            lxc 20160823145623.451 INFO     lxc_start - start.c:lxc_init:488 - 'trusty-1' is initialized
            lxc 20160823145623.451 INFO     lxc_confile - confile.c:config_idmap:1500 - read uid map: type u nsid 0 hostid 165536 range 65536
            lxc 20160823145623.451 INFO     lxc_confile - confile.c:config_idmap:1500 - read uid map: type g nsid 0 hostid 165536 range 65536
            lxc 20160823145623.452 DEBUG    lxc_start - start.c:__lxc_start:1326 - Not dropping cap_sys_boot or watching utmp
            lxc 20160823145623.452 INFO     lxc_start - start.c:resolve_clone_flags:1013 - Cloning a new user namespace
            lxc 20160823145623.487 ERROR    lxc_conf - conf.c:instantiate_veth:2595 - failed to attach 'veth1SE4RV' to the bridge 'vport1': Operation not permitted
            lxc 20160823145623.512 ERROR    lxc_conf - conf.c:lxc_create_network:2872 - failed to create netdev
            lxc 20160823145623.512 ERROR    lxc_start - start.c:lxc_spawn:1080 - failed to create the network
            lxc 20160823145623.512 ERROR    lxc_start - start.c:__lxc_start:1353 - failed to spawn 'trusty-1'
            lxc 20160823145623.512 INFO     lxc_conf - conf.c:run_script_argv:367 - Executing script '/usr/share/lxcfs/lxc.reboot.hook' for container 'trusty-1', config section 'lxc'
            lxc 20160823145624.015 INFO     lxc_conf - conf.c:run_script_argv:367 - Executing script '/usr/bin/lxd callhook /var/lib/lxd 1 stop' for container 'trusty-1', config section 'lxc'
            lxc 20160823145624.087 WARN     lxc_commands - commands.c:lxc_cmd_rsp_recv:172 - command get_cgroup failed to receive response
            lxc 20160823145624.087 WARN     lxc_commands - commands.c:lxc_cmd_rsp_recv:172 - command get_cgroup failed to receive response
            lxc 20160823145624.093 INFO     lxc_confile - confile.c:config_idmap:1500 - read uid map: type u nsid 0 hostid 165536 range 65536
            lxc 20160823145624.093 INFO     lxc_confile - confile.c:config_idmap:1500 - read uid map: type g nsid 0 hostid 165536 range 65536
            lxc 20160823145635.567 INFO     lxc_confile - confile.c:config_idmap:1500 - read uid map: type u nsid 0 hostid 165536 range 65536
            lxc 20160823145635.567 INFO     lxc_confile - confile.c:config_idmap:1500 - read uid map: type g nsid 0 hostid 165536 range 65536
            lxc 20160823145635.575 INFO     lxc_confile - confile.c:config_idmap:1500 - read uid map: type u nsid 0 hostid 165536 range 65536
            lxc 20160823145635.575 INFO     lxc_confile - confile.c:config_idmap:1500 - read uid map: type g nsid 0 hostid 165536 range 65536

日志中有一行;

Failed to attach 'veth1SE4RV' to the bridge 'vport1': Operation not permitted

这与某种权限有关吗?

  1. 设置vport1类型为internal,这样,ovs会创建一个虚拟接口vport1。在创建 lxc 容器
  2. 之前,请确保您可以使用 ip link show 看到界面
  3. 将lxc.network.type设置为phys,这样lxc将使用ovs创建的现有接口vport1,而不是创建新接口

所以在经过大量的尝试之后,我以某种方式设法做了我想做的事。对于将来遇到完全相同问题的人,我将此留给您。

要将您的容器与现有接口连接到此:

  1. 在主机上创建一个网桥,我们称之为br0
  2. 在主机上创建一个类型为 internal 的虚拟接口,我们称之为 vport。 这样就可以 ovs-vsctl add-port br0 vport -- set interface vport type=internal
  3. 做一个 $ lxc config edit <containername> 然后在设备下面放这个

    eth0: name: eth0 nictype: physical parent: vport type: nic

这是告诉 LXC 使用 vport 作为其接口的部分。

这就是你要做的所有事情。

虽然容器内的数据包丢弃在这种配置下很大,这让我选择了第二个选项,如下所示。

让你的容器创建自己的虚拟接口

  1. 在主机上创建一个网桥,我们称之为br0
  2. 做一个 $ lxc config edit <containername> 然后在设备下面放这个

    eth0: host_name: vport name: eth0 nictype: bridged parent: br0 type: nic

每次 LXC 重新启动时,它都会创建自己的界面,但每次界面的名称都是 vport

此配置中的数据包丢弃是 none,所以我要这样做。

找到更简单的解决方案:

ovs-vsctl add-br vlan418 vm-bridge 418

vlan418 是要创建的假网桥的名称,vm-bridge 是绑定到 VM 主机的物理 NIC 的主网桥。 418 是 vlan id。

然后在lxd容器配置:

devices:
  eth0:
    name: eth0
    nictype: bridged
    parent: vlan418
    type: nic

就是这样。它完美无瑕。 add-br 创建一个假桥,当 lxd 尝试将端口附加到假桥时,OpenVSwitch 会自动创建所需的端口 vlan418