Alauda Container Platform

Kube-OVN Underlay + MetalLB LoadBalancer Service Configuration

TOC

OverviewPrerequisitesEnvironment RequirementsTraffic FlowConfiguration Steps1. Configure ProviderNetwork with VLAN Sub-interfaces2. Configure Kube-OVN Controller Parameters3. Configure Underlay Subnet External Address Feature4. Create MetalLB External Address Pool5. Create Sample Application and LoadBalancer Service6. Verify Configuration7. Migrating Existing Services

Overview

This solution addresses the integration of MetalLB L2 mode with Kube-OVN Underlay networking. It allows users to utilize Underlay subnet IPs as MetalLB LoadBalancer Service VIPs, directly forwarding traffic to backend business Pods.

Prerequisites

Environment Requirements

  • ACP version: >= 4.2.2
  • IP support: IPv4 only (IPv6 is currently not supported)

Traffic Flow

Traffic Diagram:

Configuration Steps

1. Configure ProviderNetwork with VLAN Sub-interfaces

Important: VLAN sub-interfaces must be used.

Configure Kube-OVN Underlay network to automatically create VLAN sub-interfaces:

apiVersion: kubeovn.io/v1
kind: ProviderNetwork
metadata:
  name: provider
spec:
  defaultInterface: underlay0.341
  autoCreateVlanSubinterfaces: true  # Automatically creates VLAN sub-interfaces (e.g., underlay0.341) if only parent interface (underlay0) exists

---
apiVersion: kubeovn.io/v1
kind: Vlan
metadata:
  name: ovn-vlan
spec:
  id: 0    # Use 0 because autoCreateVlanSubinterfaces creates the VLAN sub-interface (underlay0.341) which handles VLAN tagging, not Kube-OVN directly
  provider: provider
status:
  subnets:
  - ovn-default

⚠️ Warning: When modifying the ProviderNetwork or Vlan resources individually, the Underlay network connectivity will be interrupted. Network connectivity will only be restored after both resources are fully configured and in sync. Plan configuration changes during maintenance windows to minimize service disruption.

2. Configure Kube-OVN Controller Parameters

Configure the Kube-OVN controller with the required parameters for LoadBalancer functionality:

Using Web Console:

  1. Navigate to Administrator > Marketplace > Cluster Plugins, then search for ovn to locate Alauda Container Platform Networking for Kube-OVN
  2. In the plugin row, click the action menu (vertical ⋮) and select Update to open the configuration dialog
  3. Configure the following settings:
    • Skip CT for Dst LPort IPs: No
    • Enable OVN LB Local: Yes

3. Configure Underlay Subnet External Address Feature

Edit the Underlay subnet to reserve an IP range for LoadBalancer usage:

Important: External address pool IPs must be within the Underlay subnet.

Modify the Underlay subnet parameter spec.enableExternalLBAddress: true:

apiVersion: kubeovn.io/v1
kind: Subnet
metadata:
  name: underlay-subnet
spec:
  enableExternalLBAddress: true       # Indicates this subnet has IP range for LB service VIP
  excludeIps:
  - 10.180.208.200..10.180.208.220    # Reserve IP range for external address pool

4. Create MetalLB External Address Pool

# underlay-ippool.yaml
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: acp-underlay-pool
  namespace: metallb-system
spec:
  addresses:
    - 10.180.208.200-10.180.208.220  # Underlay subnet IP range
  avoidBuggyIPs: true
  autoAssign: true
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: acp-underlay-pool
  namespace: metallb-system
spec:
  ipAddressPools:
    - acp-underlay-pool
  nodeSelectors: []

Deploy the address pool:

kubectl apply -f underlay-ippool.yaml

5. Create Sample Application and LoadBalancer Service

# application-with-loadbalancer.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: backend-app
  labels:
    app: backend
spec:
  replicas: 3
  selector:
    matchLabels:
      app: backend
  template:
    metadata:
      labels:
        app: backend
    spec:
      containers:
        - name: backend
          image: nginx:1.25
          ports:
            - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: backend-lb-service
  # To use specific IPPool: add annotation `metallb.io/address-pool: acp-underlay-pool`
  # To use fixed IP: set `spec.loadBalancerIP: 10.180.208.201`
spec:
  type: LoadBalancer
  externalTrafficPolicy: Local  # **IMPORTANT**: Required for preserving source IP and enabling direct Pod routing
  selector:
    app: backend
  ports:
    - port: 80
      targetPort: 80

Deploy the application:

kubectl apply -f application-with-loadbalancer.yaml

6. Verify Configuration

# Check service status
kubectl get svc backend-lb-service -o wide

# Test external access
curl http://10.180.208.200

7. Migrating Existing Services

For existing services using the old address pool (node subnet only), you can migrate them to the new Underlay address pool:

# Add annotation to migrate existing service
kubectl annotate service <existing-service-name> metallb.io/address-pool=acp-underlay-pool --overwrite

# Verify the service has been assigned a new IP from the Underlay pool
kubectl get svc <existing-service-name> -o wide

For new services, add the annotation directly:

apiVersion: v1
kind: Service
metadata:
  name: backend-lb-service
  annotations:
    metallb.io/address-pool: acp-underlay-pool  # Use the Underlay address pool
spec:
  type: LoadBalancer
  externalTrafficPolicy: Local
  selector:
    app: backend
  ports:
    - port: 80
      targetPort: 80
# Check service status
kubectl get svc backend-lb-service -o wide

# Test external access
curl http://10.180.208.200