ovn-controller stops sending DHCPOFFER

[frct1]

I'm observing for a long time next bug: ovn-controller doesn't send DHCPOFFER but it logs that it does. However this VIF doesn't get it at all.
But once ovn-controller restarted it successfully seems like it actually send DHCPOFFER and immediately receives DHCPACK from VIF.

2025-02-08T14:27:57.462Z|00053|pinctrl(ovn_pinctrl0)|INFO|DHCPOFFER fa:16:3e:80:67:62 13.37.0.5
2025-02-08T14:28:01.746Z|00054|pinctrl(ovn_pinctrl0)|INFO|DHCPOFFER fa:16:3e:80:67:62 13.37.0.5
2025-02-08T14:28:05.582Z|00055|pinctrl(ovn_pinctrl0)|INFO|DHCPOFFER fa:16:3e:80:67:62 13.37.0.5
2025-02-08T14:28:13.053Z|00056|pinctrl(ovn_pinctrl0)|INFO|DHCPOFFER fa:16:3e:80:67:62 13.37.0.5
2025-02-08T14:28:21.972Z|00057|pinctrl(ovn_pinctrl0)|INFO|DHCPOFFER fa:16:3e:84:8e:7f 13.37.0.37
2025-02-08T14:28:25.109Z|00058|pinctrl(ovn_pinctrl0)|INFO|DHCPOFFER fa:16:3e:84:8e:7f 13.37.0.37
2025-02-08T14:28:29.362Z|00059|pinctrl(ovn_pinctrl0)|INFO|DHCPOFFER fa:16:3e:84:8e:7f 13.37.0.37
2025-02-08T14:28:30.003Z|00060|pinctrl(ovn_pinctrl0)|INFO|DHCPOFFER fa:16:3e:80:67:62 13.37.0.5
2025-02-08T14:28:37.820Z|00061|pinctrl(ovn_pinctrl0)|INFO|DHCPOFFER fa:16:3e:84:8e:7f 13.37.0.37
2025-02-08T14:28:38.619Z|00002|fatal_signal(ovn_statctrl2)|WARN|terminating with signal 15 (Terminated)
# ovn_controller is restarted by docker restart ovn_controller
2025-02-08T14:28:50.639Z|00001|vlog|INFO|opened log file /var/log/kolla/openvswitch/ovn-controller.log
2025-02-08T14:28:50.640Z|00002|vif_plug_representor|INFO|unable to set udev receive buffer size: Operation not permitted
2025-02-08T14:28:50.641Z|00003|reconnect|INFO|tcp:127.0.0.1:6640: connecting...
2025-02-08T14:28:50.641Z|00004|reconnect|INFO|tcp:127.0.0.1:6640: connected
2025-02-08T14:28:50.738Z|00005|main|INFO|OVN internal version is : [24.09.0-20.37.0-73.6]
2025-02-08T14:28:50.738Z|00006|main|INFO|OVS IDL reconnected, force recompute.
2025-02-08T14:28:50.738Z|00007|reconnect|INFO|tcp:10.10.9.5:6642: connecting...
2025-02-08T14:28:50.738Z|00008|main|INFO|OVNSB IDL reconnected, force recompute.
2025-02-08T14:28:50.739Z|00009|ovn_util|INFO|statctrl: connecting to switch: "unix:/var/run/openvswitch/br-int.mgmt"
2025-02-08T14:28:50.739Z|00010|rconn|INFO|unix:/var/run/openvswitch/br-int.mgmt: connecting...
2025-02-08T14:28:50.739Z|00011|ovn_util|INFO|pinctrl: connecting to switch: "unix:/var/run/openvswitch/br-int.mgmt"
2025-02-08T14:28:50.739Z|00012|rconn|INFO|unix:/var/run/openvswitch/br-int.mgmt: connecting...
2025-02-08T14:28:50.739Z|00013|reconnect|INFO|tcp:10.10.9.5:6642: connected
2025-02-08T14:28:50.739Z|00001|rconn(ovn_statctrl2)|INFO|unix:/var/run/openvswitch/br-int.mgmt: connected
2025-02-08T14:28:50.743Z|00001|rconn(ovn_pinctrl0)|INFO|unix:/var/run/openvswitch/br-int.mgmt: connected
2025-02-08T14:28:50.756Z|00014|ovn_util|INFO|features: connecting to switch: "unix:/var/run/openvswitch/br-int.mgmt"
2025-02-08T14:28:50.757Z|00015|rconn|INFO|unix:/var/run/openvswitch/br-int.mgmt: connecting...
2025-02-08T14:28:50.757Z|00016|features|INFO|OVS Feature: ct_zero_snat, state: supported
2025-02-08T14:28:50.757Z|00017|features|INFO|OVS Feature: ct_flush, state: supported
2025-02-08T14:28:50.757Z|00018|features|INFO|OVS Feature: dp_hash_l4_sym_support, state: supported
2025-02-08T14:28:50.757Z|00019|main|INFO|OVS feature set changed, force recompute.
2025-02-08T14:28:50.757Z|00020|ovn_util|INFO|ofctrl: connecting to switch: "unix:/var/run/openvswitch/br-int.mgmt"
2025-02-08T14:28:50.757Z|00021|rconn|INFO|unix:/var/run/openvswitch/br-int.mgmt: connecting...
2025-02-08T14:28:50.758Z|00022|rconn|INFO|unix:/var/run/openvswitch/br-int.mgmt: connected
2025-02-08T14:28:50.758Z|00023|main|INFO|OVS OpenFlow connection reconnected,force recompute.
2025-02-08T14:28:50.759Z|00024|rconn|INFO|unix:/var/run/openvswitch/br-int.mgmt: connected
2025-02-08T14:28:50.760Z|00025|features|INFO|OVS Feature: meter_support, state: supported
2025-02-08T14:28:50.760Z|00026|features|INFO|OVS Feature: group_support, state: supported
2025-02-08T14:28:50.760Z|00027|features|INFO|OVS Feature: sample_action_with_registers, state: supported
2025-02-08T14:28:50.760Z|00028|main|INFO|OVS feature set changed, force recompute.
2025-02-08T14:28:51.483Z|00029|binding|INFO|Releasing lport 978a923d-1076-484b-a1d5-ff5827ae69ec from this chassis (sb_readonly=0)
2025-02-08T14:28:53.658Z|00030|poll_loop|INFO|wakeup due to [POLLIN] on fd 30 (10.10.9.3:60902<->10.10.9.5:6642) at lib/stream-fd.c:157 (78% CPU usage)
2025-02-08T14:28:54.726Z|00002|pinctrl(ovn_pinctrl0)|INFO|DHCPOFFER fa:16:3e:84:8e:7f 13.37.0.37
2025-02-08T14:28:59.134Z|00003|pinctrl(ovn_pinctrl0)|INFO|DHCPACK fa:16:3e:84:8e:7f 13.37.0.37
2025-02-08T14:29:01.499Z|00004|pinctrl(ovn_pinctrl0)|INFO|DHCPOFFER fa:16:3e:80:67:62 13.37.0.5
2025-02-08T14:29:03.245Z|00005|pinctrl(ovn_pinctrl0)|INFO|DHCPACK fa:16:3e:84:8e:7f 13.37.0.37
2025-02-08T14:29:03.455Z|00031|memory|INFO|98744 kB peak resident set size after 12.8 seconds
2025-02-08T14:29:03.455Z|00032|memory|INFO|idl-cells-OVN_Southbound:168023 idl-cells-Open_vSwitch:36874 lflow-cache-entries-cache-expr:6494 lflow-cache-entries-cache-matches:1413 lflow-cache-size-KB:20726 local_datapath_usage-KB:1 ofctrl_desired_flow_usage-KB:6342 ofctrl_installed_flow_usage-KB:4695 ofctrl_sb_flow_ref_usage-KB:2355

Any suggestions?

Environment:
OpenStack 2024.2
Kernel 6.8.0-50-generic #51~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC

ovs-vsctl (Open vSwitch) 3.4.0
DB Schema 8.7.0

ovs-ofctl (Open vSwitch) 3.4.0
OpenFlow versions 0x1:0x6

# ovn-nbctl --version
ovn-nbctl 24.09.0
Open vSwitch Library 3.4.0
DB Schema 7.6.0

# ovn-sbctl --version
ovn-sbctl 24.09.0
Open vSwitch Library 3.4.0
DB Schema 20.37.0

[frct1]

@igsilya @dceara
Hi folks, sorry for bothering, could you please take a look by experienced view?
Originally posted to mail too https://mail.openvswitch.org/pipermail/ovs-discuss/2025-February/053448.html