sonic-mgmt-common: port to libyang3

[bradh352]

SONiC Libyang3 upgrade Tracking PR: sonic-net/sonic-buildimage#21679

This PR depends on the other PRs mentioned in the above tracking PR.

The API changes between libyang1 and libyang3 are drastic.

The cvl test cases are currently passing all tests.

This depends on libyang3's devel branch plus this rejected patch:

• validation: Add LYD_VALIDATE_NOEXTDEPS to bypass leafref/when/must CESNET/libyang#2362 -- will never be accepted upstream. libyang maintainer says sonic-yang-mgmt needs to be rearchitected, it is a bad application design to need such a flag.

Or the libyang3 package built via sonic-buildimage which is based on v3.7.8 + all the necessary patches, part of sonic-net/sonic-buildimage#21679

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[bradh352]

@dutta-partha probably going to need your help on this as you are the original author. I've got the port to libyang3 compiling at least, but I'm not entirely sure how to test it or debug it. I'm sure there are issues here.

@ganglyu I've seen you make some commits here too, maybe you can help?

I think out of all the libyang3 porting this is the one that is the most complex and that I know the least about.

[ganglyu]

@bradh352
This pipeline downloads libyang*.deb from the sonic-buildimage.vs master pipeline artifacts. However, since your changes have not been merged into the master branch, the libyang*3.*.deb file is not available.

    - task: DownloadPipelineArtifact@2
      inputs:
        source: specific
        project: build
        pipeline: 142
        artifact: sonic-buildimage.vs
        runVersion: 'latestFromBranch'
        runBranch: 'refs/heads/$(BUILD_BRANCH)'
        patterns: |
            target/debs/bookworm/libyang*.deb
            target/python-wheels/bookworm/sonic_yang_models*.whl
      displayName: "Download sonic buildimage"

[ganglyu]

You can follow the pipeline to setup your environment and run unit test.
https://github.com/sonic-net/sonic-mgmt-common/blob/master/azure-pipelines.yml

[bradh352]

@ganglyu

@bradh352 This pipeline downloads libyang*.deb from the sonic-buildimage.vs master pipeline artifacts. However, since your changes have not been merged into the master branch, the libyang_3._.deb file is not available.

Right, I'm aware of that. This PR is expected to fail in the SONiC CI system (azure pipelines) until the dependent PRs are merged.

I was hoping someone more familiar with sonic-mgmt-common could help with testing and validation of this particular PR. Its by far the most complex port to libyang3 out of all the various sub-projects.

[ganglyu]

@sachinholla would you please help?

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[sachinholla]

Hi @bradh352, all tests are passing on my development vm too.. Changes look good to me. Will ask cvl maintainers to take a look.

Seeing a few compiler warnings though:

In file included from _cgo_export.c:4:
util.go: In function ‘customLogCb’:
util.go:30:65: warning: passing argument 3 of ‘customLogCallback’ discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers]
util.go:26:62: note: expected ‘char *’ but argument is of type ‘const char *’
# github.com/Azure/sonic-mgmt-common/cvl/internal/util
../cvl/internal/util/util.go: In function ‘customLogCb’:
../cvl/internal/util/util.go:30:72: warning: passing argument 3 of ‘customLogCallback’ discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers]
   30 |         customLogCallback(level, (char*)msg, (char*)data_path?data_path:schema_path);
      |                                              ~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~
../cvl/internal/util/util.go:26:62: note: expected ‘char *’ but argument is of type ‘const char *’
   26 | extern void customLogCallback(LY_LOG_LEVEL, char* msg, char* path);
      |                                                        ~~~~~~^~~~

[bradh352]

Hi @bradh352, all tests are passing on my development vm too.. Changes look good to me. Will ask cvl maintainers to take a look.

Seeing a few compiler warnings though:

In file included from _cgo_export.c:4:
util.go: In function ‘customLogCb’:
util.go:30:65: warning: passing argument 3 of ‘customLogCallback’ discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers]
util.go:26:62: note: expected ‘char *’ but argument is of type ‘const char *’
# github.com/Azure/sonic-mgmt-common/cvl/internal/util
../cvl/internal/util/util.go: In function ‘customLogCb’:
../cvl/internal/util/util.go:30:72: warning: passing argument 3 of ‘customLogCallback’ discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers]
   30 |         customLogCallback(level, (char*)msg, (char*)data_path?data_path:schema_path);
      |                                              ~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~
../cvl/internal/util/util.go:26:62: note: expected ‘char *’ but argument is of type ‘const char *’
   26 | extern void customLogCallback(LY_LOG_LEVEL, char* msg, char* path);
      |                                                        ~~~~~~^~~~

These warnings are from the original code and not something newly introduced. Its due to the Golang<->C (cgo) stuff, where Go doesn't know about the const qualifier in C. If you really want me to address it I can probably look into it, here's a reference to the issue:
https://stackoverflow.com/questions/32938558/cgo-cant-find-way-to-use-callbacks-with-const-char-argument

[bradh352]

@sachinholla any other comments on this review? If not, could you approve it?

[faraazbrcm]

@bradh352 Code changes look good to me.
I wanted to ensure SET/GET going fine using gNMI and RESTCONF interfaces. I will run some some tests next week and update you.

[bradh352]

let me know if you need an image to test and for what platform. I've got a tree with all 17 PRs in this series that I test with here: https://github.com/bradh352/sonic-buildimage/commits/bradh352/libyang3/