Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] eip未被正确清理,导致subnet中v4availableIPrange字段一直包含该ip,造成资源浪费 #5022

Open
QEDQCD opened this issue Feb 21, 2025 · 1 comment
Labels
bug Something isn't working eip ipam subnet

Comments

@QEDQCD
Copy link
Contributor

QEDQCD commented Feb 21, 2025

Kube-OVN Version

v1.13.0

Kubernetes Version

Client Version: v1.29.3
Server Version: v1.29.3

Operation-system/Kernel Version

/etc/os-release
"CentOS Stream 9"
uname -r
5.14.0-407.el9.x86_64
sbctl版本
kubectl-ko sbctl --version
ovn-sbctl 24.03.5
Open vSwitch Library 3.3.3
DB Schema 20.33.0
nbctl版本
kubectl-ko nbctl --version
ovn-nbctl 24.03.5
Open vSwitch Library 3.3.3
DB Schema 7.3.0

Description

存在 外网类型子网 subnet1
1 创建eip
kubectl get eip iptableseip-cxxx
NAME IP MAC NAT NATGWDP READY
iptableseip-cxxxj 192.168.1.163 72:cd:a6:87:08:3f fip v-gw-cdd97ce45939c6ba true

2 手动删除eip kubectl edit eip eip-cxxx
去掉finalizers这两行
finalizers:

  • kubeovn.io/kube-ovn-controller

3 子网信息如下
spec:
cidrBlock: 192.168.1.128/26
default: false
enableLb: false
excludeIps:

  • 192.168.1.129..192.168.1.161
  • 192.168.1.173..192.168.1.190
    gateway: 192.168.1.129
    gatewayNode: ""
    natOutgoing: false
    private: false
    protocol: IPv4
    provider: extnet-3xxxxxxxxxxx7.kube-system
    status:
  • v4availableIPrange: 192.168.1.164-192.168.1.166,192.168.1.168-192.168.1.170,192.168.1.172
    v4availableIPs: 8
    v4usingIPrange: 192.168.1.162-192.168.1.163,192.168.1.167,192.168.1.171
    v4usingIPs: 3
    v6availableIPrange: ""
    `

4
v4availableIPs 为 8
v4availableIPrange 可用的ip范围为7
创建vpc-nat-gw pod时以 v4availableIPrange =7 为准,实际环境中有v4availableIPs 8个,有一个ip 192.168.1.163 被浪费

5
执行以下命令都找不到这个被浪费的ip
kubectl get ip |grep 192.168.1 和 kubectl get eip |grep 192.168.1

6
修改subnet excludeIps时
触发subnet IPAM更新,查看ovn-controller得到如下日志
ipam.go:260] already assigned ip 192.168.1.163 to nic iptableseip-cxxx..extnet-346d6500b5094777.kube-system in subnet extnet-346d6500b5094777

7
192.168.1.163被IPAM的nic资源占用,无法清理

Steps To Reproduce

1 创建外网类型子网
2 创建eip
3 手动删除eip

Current Behavior

IPAM的nic资源未被正确清理

Expected Behavior

IPAM的nic资源被清理,或者有无特殊手段能清理该资源

@QEDQCD QEDQCD added the bug Something isn't working label Feb 21, 2025
Copy link

dosubot bot commented Feb 21, 2025

I found a similar closed issue that might be relevant to your problem. The issue was related to IP resources not being properly released, causing residual IP occupation in the subnet. The solution involved ensuring that the IPAM's NIC resource is properly cleaned up when an EIP is manually deleted. This was addressed by a pull request that deletes the IP CRD based on the pod name [1].

To ensure that the IPAM's NIC resource is properly cleaned up when an EIP is manually deleted, you should handle the deletion process correctly in the handleDelOvnEip function. This function includes logic to release the IP address from the IPAM when an EIP is deleted. Here's a relevant code snippet:

func (c *Controller) handleDelOvnEip(key string) error {
    klog.Infof("handle del ovn eip %s", key)
    eip, err := c.ovnEipsLister.Get(key)
    if err != nil {
        if k8serrors.IsNotFound(err) {
            return nil
        }
        klog.Error(err)
        return err
    }

    if eip.Spec.Type == util.OvnEipTypeLSP {
        if err := c.OVNNbClient.DeleteLogicalSwitchPort(eip.Name); err != nil {
            klog.Errorf("failed to delete lsp %s, %v", eip.Name, err)
            return err
        }
    }
    if eip.Spec.Type == util.OvnEipTypeLRP {
        if err := c.OVNNbClient.DeleteLogicalRouterPort(eip.Name); err != nil {
            klog.Errorf("failed to delete lrp %s, %v", eip.Name, err)
            return err
        }
    }

    if err = c.handleDelOvnEipFinalizer(eip); err != nil {
        klog.Errorf("failed to handle remove ovn eip finalizer , %v", err)
        return err
    }
    c.ipam.ReleaseAddressByPod(eip.Name, eip.Spec.ExternalSubnet)
    c.updateSubnetStatusQueue.Add(eip.Spec.ExternalSubnet)
    return nil
}

This code ensures that the NIC resource is properly cleaned up, preventing IP wastage [2][3].

To continue talking to Dosu, mention @dosu.


Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working eip ipam subnet
Projects
None yet
Development

No branches or pull requests

1 participant