[gnmi] "GRPC error Deadline Exceeded" on subscription streaming sample

[volodymyrsamotiy]

Description
gnmi/test_gnmi_countersdb.py::test_gnmi_counterdb_streaming_sample_02 fails with the error "GRPC error Deadline Exceeded".

There are 2 sub cases in test_gnmi_counterdb_streaming_sample_02 test:

1. Subscribe table
2. Subscribe table key
3. Subscribe table field

1st one and 3rd one are always passing and only 2nd one ("Subscribe table key") is failing:
https://github.com/sonic-net/sonic-mgmt/blob/master/tests/gnmi/test_gnmi_countersdb.py#L157

Please note, there are no errors in syslog.

Steps to reproduce the issue:
Run gnmi/test_gnmi_countersdb.py::test_gnmi_counterdb_streaming_sample_02 test and see it failed

Or run below command manually from ptf docker:

python2 /root/gnxi/gnmi_cli_py/py_gnmicli.py --timeout 30 -t <ip_addr> -p 50052 -xo sonic-db -rcert /root/gnmiCA.pem -pkey /root/gnmiclient.key -cchain /root/gnmiclient.crt --encoding 4 -m subscribe --subscribe_mode 0 --submode 2 --create_connections 1 --interval 0 --update_count 3 --xpath  /COUNTERS_DB/localhost/COUNTERS/oid:0x100000000000d

Describe the results you received:
gnmi/test_gnmi_countersdb.py::test_gnmi_counterdb_streaming_sample_02 test failed with the following error:

E       2025-03-06 16:35:22.395646 response received:
E       sync_response: true
E
E       Received an exception from server side and error message is: '<_MultiThreadedRendezvous of RPC that terminated with:
E               status = StatusCode.DEADLINE_EXCEEDED
E               details = "Deadline Exceeded"
E               debug_error_string = "{"created":"@1741278952.366274673","description":"Error received from peer ipv4:<ip_addr>:50052","file":"src/core/lib/surface/call.cc","file_line":1070,"grpc_message":"Deadline Exceeded","grpc_status":4}"
E       >'.
E       GRPC error
E        Deadline Exceeded

Describe the results you expected:
gnmi/test_gnmi_countersdb.py::test_gnmi_counterdb_streaming_sample_02 test should pass

[dgsudharsan]

@qiluo-msft can you please help investigate?

[arlakshm]

@ganglyu, please help triage this issue