Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[smartswitch] Add support for ENI Based Forwarding #3398

Merged
merged 14 commits into from
Feb 27, 2025

Conversation

vivekrnv
Copy link
Contributor

@vivekrnv vivekrnv commented Nov 27, 2024

What I did

HLD: sonic-net/SONiC#1842

Requires sonic-net/sonic-swss-common#976

Add DashEniFwdOrch which installs ACL rules to Redirect the DASH packet to corresponding DPU

Why I did it

How I verified it

  1. UT's:
[----------] 9 tests from DashEniFwdOrchTest
[ RUN      ] DashEniFwdOrchTest.TestDpuRegistry
[       OK ] DashEniFwdOrchTest.TestDpuRegistry (0 ms)
[ RUN      ] DashEniFwdOrchTest.LocalNeighbor
[       OK ] DashEniFwdOrchTest.LocalNeighbor (0 ms)
[ RUN      ] DashEniFwdOrchTest.LocalNeighbor_NoVNI
[       OK ] DashEniFwdOrchTest.LocalNeighbor_NoVNI (0 ms)
[ RUN      ] DashEniFwdOrchTest.LocalNeighbor_MacDirection
[       OK ] DashEniFwdOrchTest.LocalNeighbor_MacDirection (0 ms)
[ RUN      ] DashEniFwdOrchTest.LocalNeighbor_Unresolved
[       OK ] DashEniFwdOrchTest.LocalNeighbor_Unresolved (0 ms)
[ RUN      ] DashEniFwdOrchTest.RemoteNeighbor
[       OK ] DashEniFwdOrchTest.RemoteNeighbor (0 ms)
[ RUN      ] DashEniFwdOrchTest.RemoteNeighbor_SwitchToLocal
[       OK ] DashEniFwdOrchTest.RemoteNeighbor_SwitchToLocal (0 ms)
[ RUN      ] DashEniFwdOrchTest.RemoteNeighbor_NoTunnelTerm
[       OK ] DashEniFwdOrchTest.RemoteNeighbor_NoTunnelTerm (0 ms)
[ RUN      ] DashEniFwdOrchTest.TestAclTableConfig
[       OK ] DashEniFwdOrchTest.TestAclTableConfig (0 ms)
[----------] 9 tests from DashEniFwdOrchTest (2 ms total)

[----------] 1 test from MockOrchTest
[ RUN      ] MockOrchTest.EniFwdCtx
[       OK ] MockOrchTest.EniFwdCtx (10 ms)
[----------] 1 test from MockOrchTest (10 ms total)

[----------] 1 test from AclOrchTest
[ RUN      ] AclOrchTest.Match_Inner_Mac
[       OK ] AclOrchTest.Match_Inner_Mac (11 ms)
[----------] 1 test from AclOrchTest (11 ms total)
  1. Manual testing of installing ACL Rules for Local Endpoints:
{
  "DPU_TABLE": {
      "1": {
          "type": "local",
          "state": "up",
          "pa_ipv4": "10.0.0.75",
          "npu_ipv4": "10.1.0.32"
      },
      "2": {
            "type": "local",
            "state": "up",
            "pa_ipv4": "10.0.0.79",
            "npu_ipv4": "10.1.0.32"
      }
  },
  "VIP_TABLE": {
     "10.2.0.1/32" : {}
  }
}

Result:

/* Create */
{
      "DASH_ENI_FORWARD_TABLE:Vnet100:F4:93:9F:EF:C4:7E": {
          "vdpu_ids": "1,2",
          "primary_vdpu": "1",
          "outbound_vni": "4321"
      },
      "OP": "SET"
}

2024 Nov 15 02:31:38.329767 smartswitch NOTICE swss#orchagent: :- initLocalEndpoints: Local DPU endpoint detected 10.0.0.75
2024 Nov 15 02:31:38.329975 smartswitch NOTICE swss#orchagent: :- initLocalEndpoints: Local DPU endpoint detected 10.0.0.77
2024 Nov 15 02:31:38.330287 smartswitch WARNING swss#orchagent: :- findLocalEp: Multiple Local Endpoints for the ENI f4:93:9f:ef:c4:7e found, proceeding with 1
2024 Nov 15 02:31:38.332229 smartswitch NOTICE swss#orchagent: :- doAclTableTypeTask: Created ACL table type ENI_REDIRECT
2024 Nov 15 02:31:38.413127 smartswitch NOTICE swss#orchagent: :- addAclTable: Created ACL table ENI oid:700000000068b
2024 Nov 15 02:31:38.416161 smartswitch NOTICE swss#orchagent: :- addNeighbor: Created neighbor ip 10.0.0.75, b0:cf:0e:0e:35:eb on Ethernet232
2024 Nov 15 02:31:38.420296 smartswitch NOTICE swss#orchagent: :- addNextHop: Created next hop 10.0.0.75 on Ethernet232
2024 Nov 15 02:31:38.420610 smartswitch NOTICE swss#orchagent: :- handleNeighUpdate: Neighbor Update: 10.0.0.75, add: 1
2024 Nov 15 02:31:38.421243 smartswitch NOTICE swss#orchagent: :- getVip: VIP found: 10.2.0.1/32
2024 Nov 15 02:31:38.421641 smartswitch NOTICE swss#orchagent: :- fire: EniFwd ACL Rule Vnet100_F4939FEFC47E_IN installed
2024 Nov 15 02:31:38.422130 smartswitch NOTICE swss#orchagent: :- fire: EniFwd ACL Rule Vnet100_F4939FEFC47E_OUT installed

2024-11-15.00:31:38.328312|DASH_ENI_FORWARD_TABLE:Vnet100:F4:93:9F:EF:C4:7E|SET|outbound_vni:4321|primary_vdpu:1|vdpu_ids:1,2
2024-11-15.00:31:38.330997|ACL_TABLE_TABLE:ENI|SET|PORTS:Ethernet104,Ethernet112,Ethernet120,Ethernet128,Ethernet136,Ethernet144,Ethernet152,Ethernet160,Ethernet168,Ethernet176,Ethernet184,Ethernet192,Ethernet200,Ethernet208,Ethernet216,Ethernet64,Ethernet72,Ethernet80,Ethernet88,Ethernet96,PortChannel1011,PortChannel102,PortChannel105,PortChannel108|STAGE:INGRESS|TYPE:ENI_REDIRECT|POLICY_DESC:Contains Rule for DASH ENI Based Forwarding
2024-11-15.00:31:38.331916|ACL_TABLE_TYPE_TABLE:ENI_REDIRECT|SET|MATCHES:TUNNEL_VNI,DST_IP,INNER_SRC_MAC,INNER_DST_MAC|ACTIONS:REDIRECT_ACTION|BIND_POINTS:PORT,PORTCHANNEL
2024-11-15.00:31:38.413964|NEIGH_TABLE:Ethernet232:10.0.0.75|SET|neigh:b0:cf:0e:0e:35:eb|family:IPv4
2024-11-15.00:31:38.414061|NEIGH_TABLE:Ethernet240:10.0.0.77|SET|neigh:b0:cf:0e:0e:35:e3|family:IPv4
2024-11-15.00:31:38.428735|ACL_RULE_TABLE:ENI:Vnet100_F4939FEFC47E_IN|SET|PRIORITY:9996|DST_IP:10.2.0.1/32|INNER_DST_MAC:f4:93:9f:ef:c4:7e|REDIRECT_ACTION:10.0.0.75
2024-11-15.00:31:38.428818|ACL_RULE_TABLE:ENI:Vnet100_F4939FEFC47E_OUT|SET|PRIORITY:9997|DST_IP:10.2.0.1/32|INNER_SRC_MAC:f4:93:9f:ef:c4:7e|REDIRECT_ACTION:10.0.0.75|TUNNEL_VNI:4321
/* Update */
[
    {
        "DASH_ENI_FORWARD_TABLE:Vnet100:F4:93:9F:EF:C4:7E": {
            "primary_vdpu": "2"
        },
        "OP": "SET"
    }
]

2024 Nov 15 02:33:09.616143 smartswitch NOTICE swss#orchagent: :- processUpdate: Endpoint IP for Rule Vnet100_F4939FEFC47E_IN updated from 10.0.0.75 -> 10.0.0.77
2024 Nov 15 02:33:09.616414 smartswitch NOTICE swss#orchagent: :- fire: EniFwd ACL Rule Vnet100_F4939FEFC47E_IN deleted
2024 Nov 15 02:33:09.616755 smartswitch NOTICE swss#orchagent: :- fire: EniFwd ACL Rule Vnet100_F4939FEFC47E_IN installed
2024 Nov 15 02:33:09.616755 smartswitch NOTICE swss#orchagent: :- processUpdate: Endpoint IP for Rule Vnet100_F4939FEFC47E_OUT updated from 10.0.0.75 -> 10.0.0.77
2024 Nov 15 02:33:09.617020 smartswitch NOTICE swss#orchagent: :- fire: EniFwd ACL Rule Vnet100_F4939FEFC47E_OUT deleted
2024 Nov 15 02:33:09.617440 smartswitch NOTICE swss#orchagent: :- fire: EniFwd ACL Rule Vnet100_F4939FEFC47E_OUT installed
2024 Nov 15 02:33:09.627630 smartswitch NOTICE swss#orchagent: :- add: Successfully deleted ACL rule Vnet100_F4939FEFC47E_IN in table ENI
2024 Nov 15 02:33:09.632306 smartswitch NOTICE swss#orchagent: :- add: Successfully created ACL rule Vnet100_F4939FEFC47E_IN in table ENI
2024 Nov 15 02:33:09.640190 smartswitch NOTICE swss#orchagent: :- add: Successfully deleted ACL rule Vnet100_F4939FEFC47E_OUT in table ENI
2024 Nov 15 02:33:09.644913 smartswitch NOTICE swss#orchagent: :- add: Successfully created ACL rule Vnet100_F4939FEFC47E_OUT in table ENI
2024-11-15.00:33:09.615782|DASH_ENI_FORWARD_TABLE:Vnet100:F4:93:9F:EF:C4:7E|SET|primary_vdpu:2
2024-11-15.00:33:09.618507|ACL_RULE_TABLE:ENI:Vnet100_F4939FEFC47E_IN|SET|PRIORITY:9996|DST_IP:10.2.0.1/32|INNER_DST_MAC:f4:93:9f:ef:c4:7e|REDIRECT_ACTION:10.0.0.77
2024-11-15.00:33:09.618581|ACL_RULE_TABLE:ENI:Vnet100_F4939FEFC47E_OUT|SET|PRIORITY:9997|DST_IP:10.2.0.1/32|INNER_SRC_MAC:f4:93:9f:ef:c4:7e|REDIRECT_ACTION:10.0.0.77|TUNNEL_VNI:4321

Details if related

Signed-off-by: Vivek Reddy <vkarri@nvidia.com>
Signed-off-by: Vivek Reddy <vkarri@nvidia.com>
Signed-off-by: Vivek Reddy <vkarri@nvidia.com>
@mssonicbld
Copy link
Collaborator

/azp run

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@mssonicbld
Copy link
Collaborator

/azp run

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@mssonicbld
Copy link
Collaborator

/azp run

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@mssonicbld
Copy link
Collaborator

/azp run

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@mssonicbld
Copy link
Collaborator

/azp run

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@liat-grozovik
Copy link
Collaborator

@prsunny kindly reminder to review it to avoid additional conflicts

@mssonicbld
Copy link
Collaborator

/azp run

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@vivekrnv
Copy link
Contributor Author

Hi @prsunny, Any ETA to finish review on this PR?

@mssonicbld
Copy link
Collaborator

/azp run

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

Copy link
Collaborator

@prsunny prsunny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall lgtm. Can you please put some example of DB entries for reference? like ENI_FWD_TABLE

@@ -619,6 +623,7 @@ bool AclTableRangeMatch::validateAclRuleMatch(const AclRule& rule) const
return true;
}


Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please remove these extra lines (formatting changes)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated


// TODO: Update in sonic-swss-common schema.h
#define CFG_DPU_TABLE "DPU_TABLE"
#define APP_DASH_ENI_FORWARD_TABLE "DASH_ENI_FORWARD_TABLE"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we move these to swss common since a later change may break the code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed, Will raise a swss-common PR

return update_type;
}

void EniAclRule::fire(EniInfo& eni)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a description of this function and what steps are executed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated

}
}

void DashEniFwdOrch::handleNeighUpdate(const NeighborUpdate& update)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a description. Also is it only for new neighbor add or do you also handle delete neighbor?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated, only neigh update is supported as of now

bool EniInfo::update(const NeighborUpdate& nbr_update)
{
    if (nbr_update.add)
    {
        fireAllRules();
    }
    else
    {
        /* 
           Neighbor Delete handling not supported yet
           When this update comes, ACL rule must be deleted first, followed by the NEIGH object
        */
    }
    return true;
}

Signed-off-by: Vivek Reddy <vkarri@nvidia.com>
@mssonicbld
Copy link
Collaborator

/azp run

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@liat-grozovik
Copy link
Collaborator

@prsunny kindly reminder to approve it

@prsunny
Copy link
Collaborator

prsunny commented Feb 18, 2025

@prsunny kindly reminder to approve it

Hi Liat, we need to get swss-common to approve/merge before this PR

@mssonicbld
Copy link
Collaborator

/azp run

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@vivekrnv
Copy link
Contributor Author

Hi @prsunny, Here is the dependent PR: sonic-net/sonic-swss-common#976

@dgsudharsan dgsudharsan requested a review from prsunny February 20, 2025 15:36
@prsunny
Copy link
Collaborator

prsunny commented Feb 25, 2025

@vivekrnv , swss-common PR is merged. would you update this PR?

@prsunny prsunny requested a review from bingwang-ms as a code owner February 25, 2025 20:31
@mssonicbld
Copy link
Collaborator

/azp run

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

Copy link
Collaborator

@prsunny prsunny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@prsunny prsunny merged commit 8c778bf into sonic-net:master Feb 27, 2025
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants