Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(parachain): create skeleton for bitfield distribution subsystem #4234

Open
17 tasks
Tracked by #3599
haikoschol opened this issue Oct 8, 2024 · 2 comments
Open
17 tasks
Tracked by #3599
Assignees
Labels
C-simple Minor changes changes, no additional research needed. Good first issue/review. S-subsystems-availability issues related to polkadot host availability subsystem functionality. T-implementation this issue/pr is a new feature or functionality.

Comments

@haikoschol
Copy link
Contributor

haikoschol commented Oct 8, 2024

Issue summary

Create skeleton code for the implementation of the Bitfield Distribution subsystem which is described in the Host Implementers' Guide, utilizing the subsystem pattern that has been used in other Gossamer subsystems.

  • Create a new package called bitfielddistribution in dot/parachain/bitfield-distribution
  • Create a struct called BitfieldDistribution and implement the Subsystem interface on it
  • Create struct for overseer message DistributeBitfield
package parachaintypes

import "github.com/ChainSafe/gossamer/lib/common"

type DistributeBitfield struct {
	RelayParent common.Hash
	Bitfield    UncheckedSignedAvailabilityBitfield
}

It might make sense to duplicate the type UncheckedSignedAvailabilityBitfield as CheckedSignedAvailabilityBitfield,
add a method ToChecked() on UncheckedSignedAvailabilityBitfield that performs the validation and use that type in
DistributeBitfield to ensure only valid bitfields are sent to peers.

  • Create empty handler for this message
  • Create empty handlers for
    • validationprotocol.BitfieldDistributionMessage
    • networkbridgevent.PeerConnected
    • networkbridgevent.PeerDisconnected
    • networkbridgevent.NewGossipTopology
    • networkbridgevent.PeerViewChange
    • networkbridgevent.OurViewChange
    • networkbridgevent.UpdatedAuthorityIDs
    • parachaintypes.ActiveLeavesUpdateSignal
  • Register bitfield distribution subsystem with overseer
  • Forward the above messages to the bitfield distribution subsystem from overseer
  • Create struct ProvisionableDataBitfield in dot/parachain/provisioner/messages/messages.go
package provisionermessages

import (
	parachaintypes "github.com/ChainSafe/gossamer/dot/parachain/types"
	"github.com/ChainSafe/gossamer/lib/common"
)

var (
	// ...
	_ Data = (*ProvisionableDataBitfield)(nil)
)

// ...

type ProvisionableDataBitfield struct {
	RelayParent common.Hash
	Bitfield    parachaintypes.UncheckedSignedAvailabilityBitfield
}

func (ProvisionableDataBitfield) IsData() {}

If the type CheckedSignedAvailabilityBitfield mentioned above is created, it should also be used in
ProvisionableDataBitfield.

  • Define data structures for subsystem state (see section below for details)
  • Add tests

Subsystem State

The subsystem should store the view of each peer the subsystem is informed about via the relevant network bridge events. The Parity node also stores the protocol version of the peer. If our implementation only supports version 2 and 3 messages, the subsystem should probably ignore peers that still use version 1 instead.

The subsystem also needs to know the current and previous network grid topologies and the view of the node ("our view"). The grid topology is out of scope for this design document and the subsystem implementation. The bitfielddistribution package should contain an interface that covers the required methods of SessionBoundGridTopologyStorage used in the Parity node.

For each relay parent the subsystem is instructed by the overseer to work on, it needs to maintain the following data:

  • the signing context (retrieved from runtime once)
  • the validator set (retrieved from runtime once)
  • any valid bitfield messages received from a validator (we can probably just store the network messages, since it's v2/v3 only)
  • messages sent to a peer for this relay parent (store the validator ID instead of the message since there can only be one per validator)
  • messages received from a peer for this relay parent (again, store validator ID, this is to avoid sending this peer a message for this validator)

This could be implemented roughly as follows:

type SessionBoundGridTopologyStorage interface {
	// ...
}

type view struct {
	heads []common.Hash
	finalizedBlockNumber uint32
}

// methods on view ...

type perRelayParentData struct {
	sessionIndex parachaintypes.SessionIndex // the required part of the signing context
	validators []parachaintypes.ValidatorID
	onePerValidator map[parachaintypes.ValidatorID]*validationprotocol.BitfieldDistributionMessage
	messageSentToPeer map[PeerID]map[parachaintypes.ValidatorID]struct{}
	messageReceivedFromPeer map[PeerID]map[parachaintypes.ValidatorID]struct{}
}

type BitfieldDistribution struct {
	peerViews map[PeerID]view
	ourView view
	topologies SessionBoundGridTopologyStorage
	perRelayParent map[common.Hash]*perRelayParentData
}

The Parity node also uses a ReputationAggregator. It is up to the implementer of this subsystem to either define an interface analogous to SessionBoundGridTopologyStorage or adjust peer reputation immediately. The subsystem can be modified to use a ReputationAggregator in a follow-up PR, once it has been implemented.

Other information and links

Look at other skeleton PRs to get an idea of what to code and what will become and TODO and a new issue

@haikoschol haikoschol added the S-subsystems-availability issues related to polkadot host availability subsystem functionality. label Oct 8, 2024
@haikoschol haikoschol added C-simple Minor changes changes, no additional research needed. Good first issue/review. T-feat this issue/pr is a new feature or functionality. labels Dec 3, 2024
@haikoschol
Copy link
Contributor Author

@haikoschol Add information about what state the subsystem needs to maintain.

@haikoschol haikoschol added T-implementation this issue/pr is a new feature or functionality. and removed T-feat this issue/pr is a new feature or functionality. labels Dec 17, 2024
@haikoschol haikoschol changed the title feat(dot/parachain): create skeleton for bitfield distribution subsystem feat(parachain): create skeleton for bitfield distribution subsystem Jan 21, 2025
@haikoschol
Copy link
Contributor Author

haikoschol commented Mar 3, 2025

Seems like I misunderstood something about SessionBoundGridTopologyStorage. This should actually be implemented in the subsystem and use/reference SessionGridTopology, which is implemented in #4454. (cc @freddyli7)

update: Actually it might already be implemented and about to be merged. Potentially just needs to be moved to the bitfielddistribution package.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-simple Minor changes changes, no additional research needed. Good first issue/review. S-subsystems-availability issues related to polkadot host availability subsystem functionality. T-implementation this issue/pr is a new feature or functionality.
Projects
None yet
Development

No branches or pull requests

2 participants