Does using the vlan option on the physical uplink network work?

https://documentation.ubuntu.com/lxd/en/latest/reference/network_physical/#configuration-options

@boltmark see comment for how we may be able to approach this.

I have got it working as follows:

1. On the localnet OVN logical_switch_port that connects to the uplink provider bridge, we need to set vlan_request to the specified VLAN ID. E.g. ovn-nbctl set logical_switch_port lxd-net3-ls-ext-lsp-provider tag_request=101. This results in egress packets from the OVN router being tagged as they go onto the provider OVS bridge. This change is cluster wide - just set it in the OVN northbound DB.

2. Configure the OVS port that represents the connection to the uplink network parent interface to trunk the specified VLAN. E.g. ovs-vsctl set port lxdovn1b vlan_mode=trunk trunk=101 where lxdovn1b is the end of the veth pair that connects to the OVS provider bridge from lxdbr0 (acting as the uplink). This allows packets from the OVN router to traverse the OVS provider switch to the uplink parent interface whilst still being tagged. This change is per-cluster member as the OVS bridge is local to each cluster member.

3. Finally, and this is only where the uplink network is a Linux native bridge, we need to set the VID of the port that represents the other end of the veth pair that connects between the uplink bridge the OVS provider bridge. E.g. bridge vlan add dev lxdovn1a vid 101 where lxdovn1a is the end of the veth pair that connects to lxdbr0 from the OVS provider bridge. This allows tagged packets into the uplink bridge. We may consider optionally enabling VLAN filtering mode on the uplink bridge, but if its a managed LXD bridge it will already be enabled, and if its an unmanaged bridge we may not want to change the configuration and instead detect if VLAN filtering is not enabled and fail setup. This change is per-cluster member as the uplink bridge is local to each cluster member.

4. Optional for testing: Use a container connected to lxdbr0 (acting as the uplink network) with a NIC device that has the vlan setting set, e.g. lxc config device override c1 eth0 vlan=101, this will enable the tagged packets arriving at lxdbr0 to make it to the c1 container. Because DHCP will not work in this setup, one will need to configure the IPs statically in the same subnet as the lxdbr0 uplink so that the OVN router can reach it. It should then be possible to ping the c1 container's IP from instances connected to the OVN network.

For steps 2 and 3 we will need LXD to notify the other cluster members when the uplink VLAN is changed so that they can reconfigure the local ports.

So in the end it looks like this:

container 1 -> NIC veth port (vlan 101 untagged) -> lxdbr0 -> OVS veth port (vlan 101 tagged) -> OVS provider bridge -> OVN provider port (vlan 101 tagged) -> OVN provider logical switch port (vlan 101 tagged) -> OVN logical router -> OVN logical switch -> container 2

Here's a reproducer for a lab setup in a LXD VM:

lxc launch ubuntu:24.04 v1 --vm -c limits.memory=2GiB -d root,size=20GiB

sudo apt install ovn-host ovn-central
sudo ovs-vsctl set open_vswitch . \
   external_ids:ovn-remote=unix:/var/run/ovn/ovnsb_db.sock \
   external_ids:ovn-encap-type=geneve \
   external_ids:ovn-encap-ip=127.0.0.1

snap install lxd
lxd init --auto
lxc network show lxdbr0

lxc network set lxdbr0 ipv4.dhcp.ranges=10.76.202.10-10.76.202.20 ipv4.ovn.ranges=10.76.202.21-10.76.202.30

lxc network create ovn1 --type=ovn network=lxdbr0
lxc launch images:alpine/3.20 c1-ovn1 -n ovn1
lxc launch images:alpine/3.20 c1-lxdbr0 -n lxdbr0

ovs-vsctl show
be54b5c5-6aa1-4184-b616-106bce2778f8
    Bridge lxdovn1
        Port lxdovn1b
            Interface lxdovn1b
        Port patch-lxd-net2-ls-ext-lsp-provider-to-br-int
            Interface patch-lxd-net2-ls-ext-lsp-provider-to-br-int
                type: patch
                options: {peer=patch-br-int-to-lxd-net2-ls-ext-lsp-provider}
        Port lxdovn1
            Interface lxdovn1
                type: internal
    Bridge br-int
        fail_mode: secure
        datapath_type: system
        Port patch-br-int-to-lxd-net2-ls-ext-lsp-provider
            Interface patch-br-int-to-lxd-net2-ls-ext-lsp-provider
                type: patch
                options: {peer=patch-lxd-net2-ls-ext-lsp-provider-to-br-int}
        Port br-int
            Interface br-int
                type: internal
        Port vethfd24e3a5
            Interface vethfd24e3a5
    ovs_version: "3.3.0"



ovn-nbctl set logical_switch_port lxd-net2-ls-ext-lsp-provider tag_request=101
ovs-vsctl set port lxdovn1b vlan_mode=trunk trunk=101

lxc config device set c1-lxdbr0 eth0 vlan=101

bridge vlan
port              vlan-id  
lxdbr0            1 PVID Egress Untagged
lxdovn1a          1 PVID Egress Untagged
veth44b1a3f9      101 PVID Egress Untagged

bridge vlan add dev lxdovn1a vid 101

Question I have: Should we add a vlan setting to the uplink network or each OVN network?

I believe we can make this simpler as, per my question from 1 year ago, #12234 (comment) it does indeed work with vlan setting on the physical uplink network already, but only when the uplink parent is a standalone network interface and not a bridge.

But we can make a smaller improvement to support uplink parents that are unmanaged bridges too.