[bgp] Optionally configure router-0 as a BGP-EVPN VTEP - #4125
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
PR Summary by QodoOptionally configure router-0 as a BGP-EVPN VTEP
AI Description
Diagram
High-Level Assessment
Files changed (2)
|
Code Review by Qodo
1. VTEP setup lacks diagnostics
|
![qodo-code-review[bot]](https://avatars.githubusercontent.com/in/484649?s=60&v=4){kind=small}
| ip link show vrf-l3vni{{ _vni }} \ | ||
| || ip link add vrf-l3vni{{ _vni }} type vrf table {{ _table }} | ||
| ip link set vrf-l3vni{{ _vni }} up | ||
| ip link show br-l3vni{{ _vni }} \ |
There was a problem hiding this comment.
1. Vtep setup lacks diagnostics 📘 Rule violation ◔ Observability
The VTEP task performs multiple dependent VRF, bridge, and VXLAN operations in one shell invocation without block/rescue/always failure diagnostics. A partial failure is difficult to isolate and can leave router networking incompletely configured.
Agent Prompt
## Issue description
The EVPN VTEP setup combines several dependent netdev operations in one shell task and provides no structured failure diagnostics.
## Issue Context
Use smaller, single-purpose tasks inside `block`/`rescue`/`always`. The `rescue` section should capture relevant link, bridge, VRF, and VXLAN state before stopping execution with `ansible.builtin.fail`.
## Fix Focus Areas
- playbooks/bgp/prepare-bgp-spines-leaves.yaml[291-316]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
| _vni: "{{ router_evpn_l3vni | default(1) }}" | ||
| _vtep_ip: "{{ router_evpn_vtep_ip | default(router_loopback_ip) }}" | ||
| _table: "{{ 1000 + (router_evpn_l3vni | default(1) | int) }}" |
There was a problem hiding this comment.
2. Vtep address remains unassigned 🐞 Bug ≡ Correctness
When router_evpn_vtep_ip differs from router_loopback_ip, the playbook uses it for VXLAN, SNAT, and EVPN advertisement without assigning it to any local interface. The /32 is therefore absent from the default RIB, so it cannot be imported and advertised for the required return path.
Agent Prompt
## Issue description
A custom `router_evpn_vtep_ip` is used throughout the dataplane and FRR configuration but is never assigned locally, preventing its `/32` from being imported and advertised.
## Issue Context
Only `router_loopback_ip` is currently assigned to `lo`. Resolve and validate the effective VTEP address, then ensure its `/32` exists locally before creating the VXLAN device or starting FRR.
## Fix Focus Areas
- playbooks/bgp/prepare-bgp-spines-leaves.yaml[275-314]
- playbooks/bgp/templates/router-frr.conf.j2[73-90]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
| {% if router_evpn_vtep | default(false) %} | ||
| vrf vrf-l3vni{{ router_evpn_l3vni | default(1) }} | ||
| vni {{ router_evpn_l3vni | default(1) }} | ||
| exit-vrf |
There was a problem hiding this comment.
3. Ipv6 renders unusable vtep 🐞 Bug ≡ Correctness
For an IPv6 run, the playbook skips both EVPN netdev creation and SNAT, but the template still emits the L3VNI VRF and IPv4 /32 EVPN configuration whenever router_evpn_vtep is true. FRR is consequently restarted with no corresponding Linux VTEP dataplane, leaving the enabled feature nonfunctional.
Agent Prompt
## Issue description
EVPN dataplane tasks are IPv4-only, but FRR EVPN configuration is rendered for IPv6 runs as well.
## Issue Context
Either guard every template EVPN block with the same IPv4 condition or fail early with a clear assertion when EVPN VTEP mode is requested for IPv6.
## Fix Focus Areas
- playbooks/bgp/prepare-bgp-spines-leaves.yaml[291-314]
- playbooks/bgp/prepare-bgp-spines-leaves.yaml[371-385]
- playbooks/bgp/templates/router-frr.conf.j2[16-90]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
| ip link show vx-0-{{ _vni }} \ | ||
| || ip link add vx-0-{{ _vni }} type vxlan id {{ _vni }} \ | ||
| local {{ _vtep_ip }} dstport 4789 nolearning |
There was a problem hiding this comment.
4. Existing vxlan stays misconfigured 🐞 Bug ☼ Reliability
The existence check skips VXLAN creation without reconciling its local address, VNI, destination port, or learning mode. A rerun after changing the VTEP address—or encountering a same-named device using the incompatible default port—retains the stale dataplane while FRR advertises the new configuration.
Agent Prompt
## Issue description
Existing `vx-0-N` devices bypass configuration of the VTEP address and required UDP port, making playbook reruns non-convergent.
## Issue Context
Ensure existing devices have the requested VNI, local address, `dstport 4789`, and learning mode. Recreate the device safely when immutable attributes differ.
## Fix Focus Areas
- playbooks/bgp/prepare-bgp-spines-leaves.yaml[300-314]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
| iptables -t nat -C POSTROUTING -d {{ _tenant_cidr }} | ||
| -o br-l3vni{{ _vni }} -j SNAT --to-source {{ _vtep_ip }} | ||
| || iptables -t nat -A POSTROUTING -d {{ _tenant_cidr }} | ||
| -o br-l3vni{{ _vni }} -j SNAT --to-source {{ _vtep_ip }} |
There was a problem hiding this comment.
5. Obsolete snat rule wins 🐞 Bug ☼ Reliability
If the configured VTEP address changes, iptables -C misses the new rule and appends it without removing the old rule. The earlier rule still matches the same tenant traffic first and SNATs it to the obsolete address, whose /32 is no longer advertised by the rendered FRR configuration.
Agent Prompt
## Issue description
The check-or-append implementation leaves obsolete SNAT rules ahead of the currently configured rule after variable changes.
## Issue Context
Manage a uniquely identifiable rule or dedicated chain and replace/remove stale entries so exactly one current EVPN SNAT rule applies after every rerun.
## Fix Focus Areas
- playbooks/bgp/prepare-bgp-spines-leaves.yaml[371-386]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
|
Build failed (check pipeline). Post ✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 58m 56s |
The BGP-EVPN dataplane test needs the DC router (router-0) to reach tenant VM IPs advertised via BGP-EVPN Type-5 routes. Until now the router only ran `advertise-all-vni`, so the Type-5 routes propagated through the fabric but router-0 was not a VTEP and could not decapsulate or forward tenant traffic, leaving the private IPs unreachable. Make router-0 a real EVPN VTEP, gated behind `router_evpn_vtep` (default `false`, so existing plain-BGP and non-EVPN jobs are unchanged): - `router-frr.conf.j2`: add the L3VNI VRF (`vrf vrf-l3vniN`/`vni N`), `import vrf` into the default instance so ingress traffic on the default VRF can resolve tenant routes, and a second `router bgp <asn> vrf vrf-l3vniN` instance that imports the VTEP IP from the default VRF and advertises only that `/32` into EVPN. The `route-target` matches the EDPM/OVN side (`<edpm_asn>:<vni>`). - `prepare-bgp-spines-leaves.yaml`: create the VTEP netdevs (`vrf-l3vniN` -> `br-l3vniN` -> `vx-0-N`) before FRR starts so zebra can program the L3VNI. The VXLAN `dstport` is `4789` to match the OVN dataplane on EDPM nodes; the kernel FRR default port is advertisement-only and silently blackholes decapsulated packets. - Add an idempotent SNAT rule that rewrites tenant-bound traffic to the VTEP IP, so the VM replies to an address router-0 advertises into EVPN and no worker subnet needs to be leaked into the tenant VRF. The destination is the whole tempest `project_network_cidr` (`router_tenant_cidr`, default `10.100.0.0/16`) because tempest carves `/28` subnets sequentially out of it -- a single `/24` route flakes once the allocator advances past `10.100.0.255`. The worker-3 node route and the `bgpnet-worker-3` NAD route (`10.100.0.0/16 -> 100.64.10.1`) live in the `architecture` repo and are handled by a separate change. Related-ticket: OSPRH-34785 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Eduardo Olivares <eolivare@redhat.com>
127d5b1 to
a28d2ec
Compare
|
Build failed (check pipeline). Post ✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 13m 35s |
1 participant
The BGP-EVPN dataplane test needs the DC router (router-0) to reach tenant VM IPs advertised via BGP-EVPN Type-5 routes. Until now the router only ran
advertise-all-vni, so the Type-5 routes propagated through the fabric but router-0 was not a VTEP and could not decapsulate or forward tenant traffic, leaving the private IPs unreachable.Make router-0 a real EVPN VTEP, gated behind
router_evpn_vtep(defaultfalse, so existing plain-BGP and non-EVPN jobs are unchanged):router-frr.conf.j2: add the L3VNI VRF (vrf vrf-l3vniN/vni N),import vrfinto the default instance so ingress traffic on the default VRF can resolve tenant routes, and a secondrouter bgp <asn> vrf vrf-l3vniNinstance that imports the VTEP IP from the default VRF and advertises only that/32into EVPN. Theroute-targetmatches the EDPM/OVN side (<edpm_asn>:<vni>).prepare-bgp-spines-leaves.yaml: create the VTEP netdevs (vrf-l3vniN->br-l3vniN->vx-0-N) before FRR starts so zebra can program the L3VNI. The VXLANdstportis4789to match the OVN dataplane on EDPM nodes; the kernel FRR default port is advertisement-only and silently blackholes decapsulated packets.project_network_cidr(router_tenant_cidr, default10.100.0.0/16) because tempest carves/28subnets sequentially out of it -- a single/24route flakes once the allocator advances past10.100.0.255.The worker-3 node route and the
bgpnet-worker-3NAD route (10.100.0.0/16 -> 100.64.10.1) live in thearchitecturerepo and are handled by a separate change.Related-ticket: OSPRH-34785