Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: refactor stats collection and add more cucumber test #273

Merged
merged 5 commits into from
Mar 12, 2025

Conversation

hansieodendaal
Copy link
Collaborator

@hansieodendaal hansieodendaal commented Mar 10, 2025

Description

  • Added new cucumber tests.
  • Added consistency to connection statistics.

Closes #267

Motivation and Context

See #267

How Has This Been Tested?

Cucumber

What process can a PR reviewer use to test or verify this change?

Code review
Run cucumber:

  • Scenario: New node sync with peers on startup
  • Node will load up blocks from storage on startup

Breaking Changes

  • None
  • Requires data directory on base node to be deleted
  • Requires hard fork
  • Other - Please specify

Copy link

coderabbitai bot commented Mar 10, 2025

Walkthrough

This pull request removes the diagnostic_mode field from the StartArgs structure and eliminates its usage in related functions. It also removes the diagnostic_mode_timer from the Config struct. The PeerStats struct and its associated logic are deleted from the StatsCollector, and the method for updating peer statistics is centralized in a new update_peer_stats method within the PeerStore. Integration tests have been updated to reflect these changes, focusing on node synchronization and block loading during startup.

Changes

File Path(s) Change Summary
integration_tests/src/p2pool_process.rs, p2pool/src/server/config.rs Removed the diagnostic_mode field from StartArgs and the diagnostic_mode_timer field from Config, along with its initialization.
p2pool/src/cli/args.rs Removed the diagnostic_mode field from StartArgs, including its documentation and argument annotations.
p2pool/src/cli/commands/util.rs Removed the diagnostic_mode variable and updated the server function to no longer accept it as a parameter for StatsCollector.
p2pool/src/server/http/stats_collector.rs Removed the PeerStats struct and related logic, including the handle_stat method and CSV reporting functionality. Updated the StatData enum to exclude PeerStats and LocalPeerAddresses.
p2pool/src/server/p2p/network.rs Removed the local_peer_addresses method from the Service struct and simplified peer statistics handling logic.
p2pool/src/server/p2p/peer_store.rs Introduced a new method update_peer_stats to centralize peer statistics updates across various methods.
integration_tests/tests/features/Sync.feature Renamed the scenario "New node sync with peers" to "New node sync with peers on startup" and added a new scenario "Node will load up blocks from storage on startup."
supply-chain/config.toml Removed the [[exemptions.getrandom]] section from the configuration file.
p2pool/src/server/server.rs Removed a call to update_local_peer_addresses during server initialization, which was previously unused.

Suggested reviewers

  • SWvheerden
  • stringhandler

Poem

I’m a hopping rabbit, code in my sight,
Diagnostic carrots glowing so bright.
Peers and nodes in a syncopated dance,
Stats reported with a rhythmic prance.
In the garden of code, I leap with delight—
Celebrating changes that make our system just right!
🥕✨


📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 813142e and b7fe072.

📒 Files selected for processing (5)
  • integration_tests/src/p2pool_process.rs (4 hunks)
  • p2pool/src/cli/args.rs (1 hunks)
  • integration_tests/src/p2pool_process.rs (0 hunks)
  • integration_tests/src/p2pool_process.rs (0 hunks)
  • p2pool/src/cli/args.rs (0 hunks)
🚧 Files skipped from review as they are similar to previous changes (5)
  • p2pool/src/cli/args.rs
  • integration_tests/src/p2pool_process.rs
  • integration_tests/src/p2pool_process.rs
  • p2pool/src/cli/args.rs
  • integration_tests/src/p2pool_process.rs
⏰ Context from checks skipped due to timeout of 90000ms (1)
  • GitHub Check: test (esmeralda)

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@hansieodendaal hansieodendaal marked this pull request as draft March 10, 2025 07:59
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (3)
p2pool/src/server/http/stats_collector.rs (3)

18-25: Consider serialization for PeerStats
If you anticipate additional or more complex reporting, deriving Serialize (or partial equivalents) for PeerStats could simplify expansions to JSON or other formats.


112-112: Consider function refactoring
#[allow(clippy::too_many_lines)] hints that handle_stat or surrounding logic might benefit from modularization for clarity and maintainability.


249-280: Verbose logging
The expanded info! statement is helpful for debugging, but it may produce lengthy logs. If logs become unwieldy, consider condensing or gating them.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 9039b78 and 3469707.

📒 Files selected for processing (6)
  • integration_tests/src/p2pool_process.rs (2 hunks)
  • p2pool/src/cli/args.rs (1 hunks)
  • p2pool/src/cli/commands/util.rs (2 hunks)
  • p2pool/src/server/http/stats_collector.rs (11 hunks)
  • p2pool/src/server/p2p/network.rs (1 hunks)
  • p2pool/src/server/p2p/peer_store.rs (8 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (4)
  • GitHub Check: test (esmeralda)
  • GitHub Check: Cucumber tests / Base Layer
  • GitHub Check: cargo check with stable
  • GitHub Check: ci
🔇 Additional comments (23)
p2pool/src/cli/args.rs (1)

127-129: Good implementation of the diagnostic mode flag

The new diagnostic_mode field is well-defined with a clear documentation comment explaining its purpose. The command-line flag implementation provides flexibility with long form (--diagnostic-mode), short form (-d), and alias (--diag), making it user-friendly.

p2pool/src/server/p2p/peer_store.rs (9)

345-352: Good refactoring to centralize peer statistics collection

The update_peer_stats method nicely consolidates the peer statistics collection logic in one place, simplifying the code and reducing redundancy.


116-117: Appropriate use of refactored method

Good utilization of the new centralized update_peer_stats method when updating the peer lists.


300-301: Appropriate use of refactored method

Consistently calling update_peer_stats after modifying the greylist for seed peers.


319-320: Appropriate use of refactored method

Properly updating peer statistics after modifying the non-squad peers list.


340-343: Good implementation of diagnostic logic

The code properly updates peer statistics after adding a new peer to the whitelist and includes debug logging for new peers.


367-368: Appropriate use of refactored method

Correctly updates peer statistics after clearing the grey list.


377-378: Appropriate use of refactored method

Correctly updates peer statistics after clearing the blacklist.


388-389: Appropriate use of refactored method

Correctly updates peer statistics after moving a peer to the grey list.


405-406: Appropriate use of refactored method

Correctly updates peer statistics after moving a peer to the blacklist.

p2pool/src/server/p2p/network.rs (1)

1117-1130: Enhanced peer statistics collection

Good enhancement to store the seed peer status in a variable and use it for both the disconnection logic and for sending detailed statistics. The send_peer_stats call provides valuable diagnostic information including whether the peer is a seed, the peer ID, addresses, and number of peers added.

p2pool/src/cli/commands/util.rs (1)

182-192: Well-implemented conditional diagnostic data collection

This implementation correctly sets up the diagnostic mode based on the command-line flag. When enabled, it collects the local peer ID and network addresses (both listeners and external) and passes them to the stats collector. The conditional approach ensures that diagnostic data is only collected when explicitly requested.

integration_tests/src/p2pool_process.rs (2)

174-174: Confirm if always-on diagnostic mode is intended
By setting diagnostic_mode to true by default, you may introduce additional overhead in most test scenarios. Consider parameterizing this or verifying that the extra logging is needed in all runs.


383-386: CLI flag injection looks good
Appending --diagnostic-mode when args.diagnostic_mode is true ensures the diagnostic mode is exposed via the command line. Looks appropriate for your use case.

p2pool/src/server/http/stats_collector.rs (9)

4-4: Check concurrency when writing files
Using File and Write is fine, but ensure no other concurrent process attempts to open or write to the same file without coordination, which could cause race conditions or partial overwrites.


7-7: Core dependencies for peer information
taking PeerId and Multiaddr from libp2p is typical for p2p scenarios. No issues spotted.


59-60: Validate reason for storing (PeerId, Vec)
Storing this tuple in diagnostic_mode might be overkill if only the local peer’s addresses are needed. Confirm that both items are always required.


64-68: Constructor signature extension
Adding diagnostic_mode to StatsCollector::new cleanly integrates your optional CSV feature without breaking existing usage.


101-102: Properly initializing new fields
Initializing both diagnostic_mode and peer_stats in the constructor ensures consistent usage within the collector.


164-178: PeerStats match arm
This branch for PeerStats integrates seamlessly into the existing enumeration and updates the internal HashMap. The logic looks correct.


225-239: Diagnostic timer initialization
The conditional 60-second interval is well-handled, ensuring no overhead unless diagnostic_mode is enabled.


281-312: File creation and overwriting
Writing to "peer_connectivity_stats.csv" on each interval could overwrite data. Make sure this behavior is intentional or consider appending with a timestamp-based filename for historical records.


416-422: New PeerStats variant
Adding PeerStats to the StatData enum aligns well with your new diagnostic features, offering a structured way to broadcast peer-centric data.

@hansieodendaal hansieodendaal marked this pull request as ready for review March 10, 2025 11:33
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
p2pool/src/server/http/stats_collector.rs (1)

252-340: Implemented periodic diagnostic reporting to CSV file.

The code creates a timer that periodically writes peer statistics to a CSV file for later analysis. While functional, there are some areas for improvement:

  1. Error handling is minimal - using let _unused = writeln!() ignores write errors
  2. The file is reopened on every timer tick, which could be inefficient
  3. There's no file rotation or size management for long-running instances

Consider enhancing the file I/O with better error handling and potentially implementing file rotation:

- if let Ok(mut file) = File::create("peer_connectivity_stats.csv") {
-     let _unused = writeln!(
-         file,
-         "PeerId: {}, Addresses: {}",
-         peer_id.to_base58(),
-         self.local_peer_addresses.iter().map(|a| a.to_string()).collect::<Vec<String>>().join(",")
-     );
-     // ... more writes ...
-     let _unused = file.flush();
- }
+ let result = File::create("peer_connectivity_stats.csv").and_then(|mut file| {
+     writeln!(
+         file,
+         "PeerId: {}, Addresses: {}",
+         peer_id.to_base58(),
+         self.local_peer_addresses.iter().map(|a| a.to_string()).collect::<Vec<String>>().join(",")
+     )?;
+     // ... more writes ...
+     file.flush()?;
+     Ok(())
+ });
+ if let Err(e) = result {
+     error!(target: LOG_TARGET, "Failed to write diagnostic report: {}", e);
+ }
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 3469707 and 1f4de2b.

📒 Files selected for processing (9)
  • integration_tests/src/p2pool_process.rs (4 hunks)
  • integration_tests/tests/features/Sync.feature (1 hunks)
  • p2pool/src/cli/args.rs (1 hunks)
  • p2pool/src/cli/commands/util.rs (1 hunks)
  • p2pool/src/server/config.rs (2 hunks)
  • p2pool/src/server/http/stats_collector.rs (11 hunks)
  • p2pool/src/server/p2p/network.rs (3 hunks)
  • p2pool/src/server/p2p/peer_store.rs (8 hunks)
  • p2pool/src/server/server.rs (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
  • p2pool/src/cli/args.rs
  • p2pool/src/server/p2p/peer_store.rs
⏰ Context from checks skipped due to timeout of 90000ms (4)
  • GitHub Check: test (esmeralda)
  • GitHub Check: ci
  • GitHub Check: cargo check with stable
  • GitHub Check: Cucumber tests / Base Layer
🔇 Additional comments (20)
p2pool/src/server/config.rs (2)

29-29: Added diagnostic mode timer configuration field.

This field adds support for configuring the interval at which diagnostic data will be collected, allowing customization of the connectivity monitoring process.


51-51: Added reasonable default value for diagnostic_mode_timer.

The default value of 60 seconds provides a good balance between collecting enough diagnostic data and preventing excessive resource usage.

p2pool/src/server/server.rs (1)

120-121: Initializing local peer addresses in stats client.

This change updates the statistics broadcast client with the node's local addresses during initialization, ensuring that connectivity diagnostics have access to address information from the start.

p2pool/src/server/p2p/network.rs (2)

409-413: Added method to aggregate local peer addresses.

This well-implemented method consolidates both external and listener addresses into a single collection, making it easier to track and report node connectivity information.


1100-1108: Improved peer statistics collection.

The code now stores the result of checking if a peer is a seed peer in a variable and reuses it, reducing duplicate computations. It also sends comprehensive peer statistics to the broadcast client, enhancing the connectivity diagnostics feature.

p2pool/src/cli/commands/util.rs (1)

182-193: Implemented conditional diagnostic mode configuration.

The code now properly configures the diagnostic mode based on user arguments. When enabled, it sets up the diagnostic process with the configured timer interval and local peer ID, which will be used for CSV reporting of connectivity statistics.

Worth noting that the implementation elegantly handles both the enabled and disabled states through the Option type.

integration_tests/src/p2pool_process.rs (4)

78-78: Added diagnostic mode timer configuration.

This setting configures how frequently diagnostic data will be collected and written to file (every 10 seconds). This aligns well with the PR objective of collecting connectivity diagnostics.


175-175: Enabled diagnostic mode in integration tests.

Enabling diagnostic mode by default in integration tests is a good approach as it ensures the feature is regularly tested and any regressions would be caught quickly.


384-386: Added command-line handling for the diagnostic mode flag.

This implementation integrates the diagnostic mode flag into the command-line arguments system, following the same pattern as other Boolean flags. The logic is sound and consistent with the rest of the codebase.


496-501: Improved logging during peer connection verification.

The logging interval has been changed from every 10 iterations to every 50, reducing log verbosity. Including the elapsed time in the log message provides valuable context for troubleshooting connectivity issues.

integration_tests/tests/features/Sync.feature (3)

8-8: Clarified scenario title to specify startup sync.

The scenario name now more accurately reflects that it's testing sync behavior specifically during startup. This improves test clarity and documentation.


15-22: Enhanced peer connectivity testing.

Added more test nodes (NODE_C and NODE_D) and explicit verification steps for connectivity between them. This provides more comprehensive test coverage for the peer connection logic, which is particularly valuable given the new diagnostic features.


24-36: Added scenario to test block loading after restart.

This new scenario tests a critical aspect of node behavior: correctly reloading blocks from storage after a restart. This is an excellent addition that ensures data persistence, which becomes even more important when diagnostic data is being collected and analyzed across restarts.

p2pool/src/server/http/stats_collector.rs (7)

19-26: Added PeerStats structure for collecting connectivity diagnostics.

This new structure encapsulates all relevant peer connectivity information including seed peer status, ID, addresses, message counts, and timestamps. The design is clean and well-organized.


60-62: Added diagnostic mode tracking to StatsCollector.

These new fields store the diagnostic mode configuration and collected peer statistics. The implementation allows for optional diagnostic mode operation, making it a non-breaking change.


66-70: Updated constructor to support diagnostic mode.

The constructor now accepts an optional diagnostic mode parameter, which enables the feature without requiring changes to existing code that doesn't use diagnostics. This is a good approach for backward compatibility.


167-205: Added handler for peer statistics.

The implementation correctly handles both new and existing peer statistics, preserving appropriate data when updating existing entries. The logic for updating message counts and timestamps is well-thought-out.


444-454: Added StatData enum variants for peer diagnostics.

These new variants extend the existing event system to support diagnostic data collection. The implementation is consistent with the existing pattern and well-integrated.


479-480: Updated timestamp method for new enum variants.

This method update ensures that the timestamp functionality remains consistent across all event types, including the new diagnostic events. This is proper maintenance of the codebase.


589-611: Added methods to update peer statistics.

These methods provide a clean API for updating local peer addresses and sending peer statistics. The implementation follows the established pattern and maintains consistency.

- Added a command-line option to perform diagnostics upon startup. Various statistics will be
collected as they are encountered and written to a file once populated.
- Added more cucumber tests
@hansieodendaal hansieodendaal changed the title feat: add connectivity diagnostics feat: add connectivity diagnostics and more cucumber test Mar 10, 2025
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🔭 Outside diff range comments (2)
supply-chain/config.toml (2)

368-379: 🛠️ Refactor suggestion

Duplicate Exemption Entries for Clap
Within the configuration, there appear to be multiple exemption entries for clap:
• An unannotated block with version "2.34.0" (lines 368–371)
• An updated block (lines 372–375) with version "3.2.25"
• Another block (lines 376–379) with version "4.5.28"

This redundancy could lead to ambiguity as to which version is being enforced.
Please review and consolidate these entries to ensure consistency.


1072-2611: 🛠️ Refactor suggestion

General Review of the Updated Exemption Entries
This configuration file has been overhauled with a very large number of exemption entries—each specifying a fixed version and the same "safe-to-deploy" criteria. In addition to many individual updates and additions noted above, please review the following points:

Version Changes & Downgrades:
– Notably, the hashbrown exemption (lines 828–831) now pins version "0.14.5" (a downgrade from the previous "0.15.2"). Verify that this change is intentional and supported by security reviews.

Duplicate/Multiple Entries:
– Several libraries (e.g., clap, clap_derive, convert_case) appear more than once with differing version numbers. It is important to consolidate these so the tool using this configuration isn’t confused by conflicting entries.

Consistency Across Exemptions:
– The overall pattern (version and criteria) is consistent; however, given the sheer volume of changes, it is advisable to run an automated check (or review via your dependency management tool) to ensure that every exemption aligns with upstream advisory recommendations and that no inadvertent regressions have been introduced.

Overall, the structured approach is commendable. A careful verification is recommended to ensure that all these version updates and duplicate entries are correct and intentional.

🧹 Nitpick comments (24)
supply-chain/config.toml (24)

92-99: New Exemption Entries for AES-kw and Ahash
Two new exemption blocks have been added: one for aes-kw (version "0.2.1") and one for ahash (version "0.8.11").
Please verify that the chosen versions and the “safe-to-deploy” criteria are correct and intended for your supply‐chain security policy.


144-151: Updated Exemption for Argon2 and Addition for Arraydeque
A new exemption block for argon2 now lists version "0.5.3" and an exemption for arraydeque (version "0.5.1") has been introduced.
Ensure these version updates align with the security requirements and that there is no conflict with any previously established configuration.


252-255: New Exemption for Bitfield
The exemption for bitfield now specifies version "0.14.0".
Please confirm that this update meets your security and deployment needs.


272-275: Addition: Block-padding Exemption
A new exemption for block-padding with version "0.3.3" has been added.
Double-check that this version is the one intended for your project’s dependencies.


276-279: Addition: Blowfish Exemption
An exemption entry for blowfish (version "0.9.1") has been included.
Review for compatibility with any cryptographic modules that use Blowfish so that the safe-to-deploy status is warranted.


296-299: New Exemption: Buffer-redux
The configuration now exempts buffer-redux at version "1.0.2".
Confirm that the pinned version is correct and that no security advisories have been overlooked for this dependency.


304-307: Added Exemption: Bytecount
bytecount now appears with version "0.6.8" under the safe-to-deploy criteria.
Please verify this update against your dependency audit requirements.


316-319: New Exemption: Camellia
A new exemption block for camellia (version "0.1.0") has been added.
Ensure this version is compatible with your cryptographic standards and deployment guidelines.


320-323: Addition: Cast5 Exemption
The exemption for cast5 with version "0.11.1" is now part of the configuration.
No issues are noted, but please confirm that the version is consistent with your overall dependency strategy.


332-335: New Exemption: CFB-mode
A block for cfb-mode (version "0.8.2") is added.
Please check that this version is in line with other cryptographic libraries and that its “safe-to-deploy” status has been validated.


392-395: Exemption for Clap-lex Updated
The exemption for clap_lex is now set to version "0.2.4".
This change appears straightforward but confirm that the version meets future compatibility goals.


400-403: New Exemption: Clipboard-win
The addition of the clipboard-win exemption (version "4.5.0") specifies safe-to-deploy criteria.
Ensure that this matches your integration and does not conflict with any UI or clipboard-management libraries.


420-423: Console-API Exemption Update
The exemption entry for console-api is present with version "0.5.0".
Please double-check that this version is intentionally maintained and harmonized with related packages.


424-427: Console-subscriber Exemption Update
An exemption for console-subscriber with version "0.1.10" is now included.
Confirm that its deployment is consistent with your logging and monitoring strategy.


428-431: New Exemption: const-random
The exemption for const-random has been added with version "0.1.18".
Verify that this version is current and supports secure random value generation.


436-439: New Exemption: const-random-macro
A new exemption block for const-random-macro is now specified with version "0.1.16".
Ensure the update aligns with your compile-time randomness needs and dependency policies.


448-451: First convert_case Exemption Block
An exemption entry for convert_case is introduced with version "0.4.0".
Please review that this version is appropriate within your text-processing context.


464-467: New Exemption: crc24
The exemption for crc24 is added with version "0.1.6".
Please ensure that this version meets your requirements for checksum or error-detection routines.


472-475: Exemption for crossbeam-channel Updated
The exemption for crossbeam-channel now specifies version "0.5.14".
This minor update should be verified against your concurrency and parallelism needs.


488-491: New Exemption: Crossterm
The configuration now exempts crossterm at version "0.28.1".
Confirm that this version is both stable and aligned with your terminal I/O requirements.


492-495: Exemption for Crossterm_winapi Updated
The exemption for crossterm_winapi is updated with version "0.9.1".
Please verify that this library version is compatible with the intended Windows terminal features.


508-511: New Exemption: Cucumber
A new exemption for cucumber has been added (version "0.20.2").
Since Cucumber is used for testing, ensure that its version remains compatible with your Cucumber test suite.


512-515: New Exemption: Cucumber-codegen
The exemption for cucumber-codegen now appears with version "0.20.2".
Please confirm that this code generation tool is fully tested and secure per your CI requirements.


516-519: New Exemption: Cucumber-expressions
An exemption for cucumber-expressions with version "0.3.0" is introduced.
Verify that this meets the pattern-matching needs of your Cucumber tests.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 1a7f365 and f23ce9d.

⛔ Files ignored due to path filters (1)
  • supply-chain/imports.lock is excluded by !**/*.lock
📒 Files selected for processing (1)
  • supply-chain/config.toml (68 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (1)
  • GitHub Check: ci
🔇 Additional comments (1)
supply-chain/config.toml (1)

416-419: Console Exemption Update
The exemption for console now lists version "0.15.10."
This update appears to follow the standard pattern; no issues found.

Copy link
Contributor

@stringhandler stringhandler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I maybe don't understand the purpose here, but it seems like a lot of code to basically count the same data that is already present in the peer_info.log file

@hansieodendaal hansieodendaal marked this pull request as draft March 11, 2025 08:18
@hansieodendaal hansieodendaal changed the title feat: add connectivity diagnostics and more cucumber test feat: improve stats collection and more cucumber test Mar 11, 2025
@hansieodendaal hansieodendaal changed the title feat: improve stats collection and more cucumber test feat: improve stats collection and add more cucumber test Mar 11, 2025
@hansieodendaal hansieodendaal marked this pull request as ready for review March 11, 2025 08:36
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🔭 Outside diff range comments (1)
supply-chain/config.toml (1)

824-831: ⚠️ Potential issue

Duplicate hashbrown entry

There appears to be a duplicate entry for hashbrown with the same version "0.14.5". The second entry (lines 828-831) should likely be removed to avoid confusion.

 [[exemptions.hashbrown]]
 version = "0.14.5"
 criteria = "safe-to-deploy"

-[[exemptions.hashbrown]]
-version = "0.14.5"
-criteria = "safe-to-deploy"
🧹 Nitpick comments (3)
supply-chain/config.toml (1)

144-147: Second entry for argon2

There's another entry for argon2 with version "0.5.3" in addition to the existing one at version "0.4.1" (lines 140-143).

While having multiple versions is sometimes necessary for different component dependencies, consider consolidating to the newer version if possible.

p2pool/src/server/http/stats_collector.rs (2)

167-199: Consider simplifying the peer stats update logic.

The current implementation has nested conditionals for updating existing entries. This could be simplified for better readability while maintaining the same functionality.

-                if let Some(current_entry) = self.peer_stats.get(&peer_id.to_base58()) {
-                    self.peer_stats.insert(peer_id.to_base58(), PeerStats {
-                        peer_is_a_seed_peer,
-                        peer_id,
-                        public_addresses,
-                        number_received: if number_received > 0 {
-                            number_received
-                        } else {
-                            current_entry.number_received
-                        },
-                        timestamp: if number_received > 0 {
-                            timestamp
-                        } else {
-                            current_entry.timestamp
-                        },
-                    });
-                } else {
-                    self.peer_stats.insert(peer_id.to_base58(), PeerStats {
-                        peer_is_a_seed_peer,
-                        peer_id,
-                        public_addresses,
-                        number_received,
-                        timestamp,
-                    });
-                }
+                let entry = self.peer_stats.entry(peer_id.to_base58()).or_insert_with(|| PeerStats {
+                    peer_is_a_seed_peer,
+                    peer_id,
+                    public_addresses: public_addresses.clone(),
+                    number_received: 0,
+                    timestamp: EpochTime::now(),
+                });
+                
+                // Update fields - only update number_received and timestamp if we have a new count
+                entry.peer_is_a_seed_peer = peer_is_a_seed_peer;
+                entry.public_addresses = public_addresses;
+                if number_received > 0 {
+                    entry.number_received = number_received;
+                    entry.timestamp = timestamp;
+                }

298-333: Consider parameterizing the diagnostic file path.

The current implementation writes to "peer_connectivity_stats.csv" in the current directory. Consider making this path configurable or using a specific directory to avoid potential permission issues or file conflicts.

Also, be cautious about the timestamp conversion at line 315. While you have a fallback to i64::MAX if the conversion fails, it might be more intuitive to handle extremely large timestamps differently.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between f23ce9d and 813142e.

⛔ Files ignored due to path filters (2)
  • supply-chain/imports.lock is excluded by !**/*.lock
  • supply-chain/imports.lock is excluded by !**/*.lock
📒 Files selected for processing (15)
  • integration_tests/src/p2pool_process.rs (4 hunks)
  • p2pool/src/cli/commands/util.rs (1 hunks)
  • p2pool/src/server/config.rs (2 hunks)
  • p2pool/src/server/http/stats_collector.rs (11 hunks)
  • p2pool/src/server/p2p/network.rs (3 hunks)
  • p2pool/src/server/server.rs (1 hunks)
  • p2pool/src/server/http/stats_collector.rs (1 hunks)
  • supply-chain/config.toml (68 hunks)
  • integration_tests/src/p2pool_process.rs (0 hunks)
  • p2pool/src/cli/commands/util.rs (1 hunks)
  • p2pool/src/server/config.rs (0 hunks)
  • p2pool/src/server/http/stats_collector.rs (2 hunks)
  • p2pool/src/server/p2p/network.rs (1 hunks)
  • p2pool/src/server/server.rs (0 hunks)
  • supply-chain/config.toml (0 hunks)
🚧 Files skipped from review as they are similar to previous changes (7)
  • p2pool/src/server/config.rs
  • p2pool/src/server/http/stats_collector.rs
  • p2pool/src/server/server.rs
  • integration_tests/src/p2pool_process.rs
  • p2pool/src/cli/commands/util.rs
  • p2pool/src/server/server.rs
  • p2pool/src/cli/commands/util.rs
⏰ Context from checks skipped due to timeout of 90000ms (1)
  • GitHub Check: Cucumber tests / Base Layer
🔇 Additional comments (14)
supply-chain/config.toml (5)

92-95: New security exemptions added for cryptographic libraries

The addition of new exemptions for cryptographic libraries like aes-kw is consistent with improving the codebase's security posture. These exemptions properly follow the established pattern in the file.


384-387: Clap_derive version exemption

The exemption for clap_derive version "3.2.25" has already been noted in a previous review, and an issue has been created to track it (tari-project/tari#6870).


452-455: Multiple convert_case versions

As previously confirmed, the multiple convert_case versions are intentionally kept separate since they're used by different components (tari_common uses v0.6.0 while cucumber v0.20.2 uses v0.4.0 for tests).


508-511: Cucumber test dependencies added

The addition of cucumber-related exemptions (cucumber, cucumber-codegen, cucumber-expressions, and updated gherkin) aligns well with the PR objective of adding more cucumber tests for improved test coverage.

Also applies to: 512-515, 516-519, 792-795


1808-1811: Rust-ini exemption

The AI summary mentions this entry replaced rustc-hash, but both appear to be in the file now (with rustc-hash still present at lines 1812-1814). Verify that both dependencies are needed.

p2pool/src/server/p2p/network.rs (1)

1116-1120: The update to seed peer checking is clean and straightforward.

The direct check if a peer is a seed peer within the if condition is an improvement over potentially storing an intermediate variable. This makes the code more direct and easier to follow.

integration_tests/src/p2pool_process.rs (4)

78-78: New diagnostic_mode_timer configuration looks good.

Adding this configuration parameter with a value of 10 seconds enables regular diagnostic information collection, which will improve debugging capabilities.


175-175: Appropriate enablement of diagnostic mode in StartArgs.

Setting diagnostic_mode to true ensures that the new diagnostic functionality will be active during integration tests, which is valuable for capturing detailed connection statistics.


384-386: Good implementation of command line argument handling for diagnostic mode.

The conditional addition of the --diagnostic-mode flag when args.diagnostic_mode is true ensures proper command-line parameter passing to the underlying process.


496-501: Improved debug logging with less verbosity and better timing information.

Changing the debug log frequency from every 10 to every 50 iterations reduces log noise, and including the elapsed time provides valuable context about how long the connection attempt has been running.

p2pool/src/server/http/stats_collector.rs (4)

20-26: Well-structured PeerStats struct to capture peer connection statistics.

The new PeerStats struct effectively encapsulates all the necessary peer information including seed peer status, peer ID, addresses, message counts, and timestamps.


66-70: Good enhancement to StatsCollector constructor with diagnostic mode support.

The addition of the diagnostic_mode parameter allows for conditional diagnostic data collection, making the feature opt-in rather than always-on.


437-447: Good addition of new StatData variants.

The addition of PeerStats and LocalPeerAddresses variants to the StatData enum provides clear structure for the new types of statistics being collected.


472-473: Properly updated timestamp method to handle new variants.

The timestamp method correctly implements handling for the new StatData variants, ensuring consistent behavior across all variants.

@SWvheerden SWvheerden changed the title feat: improve stats collection and add more cucumber test chore: refactor stats collection and add more cucumber test Mar 12, 2025
@SWvheerden SWvheerden merged commit 48f6340 into tari-project:development Mar 12, 2025
14 checks passed
@hansieodendaal hansieodendaal deleted the ho_diagnostics branch March 12, 2025 06:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Expand p2pool cucumber tests with following scenarios
3 participants