[pull] master from netdata:master

[pull]

See Commits and Changes for more details.

---

Created by pull[bot] (v2.0.0-alpha.1)

Can you help keep this open source service alive? 💖 Please sponsor : )

Summary by Sourcery

This pull request introduces several enhancements and a new feature focused on improving data integrity and expanding the system information available in the daemon status. It adds CRC32 checksums to journal files for enhanced data integrity and enriches the daemon status file with detailed host and OS information.

New Features:

• Adds fields to the daemon status file to include host architecture, virtualization details, container information, kernel version, OS name, OS version, OS family, and OS platform, enhancing system awareness.

Enhancements:

• Improves data integrity for journal files by adding CRC32 checksums to all sections, including file header, extent list, metric list, and page headers.
• Enhances the daemon status file by persisting and reusing values from the previous session, ensuring a more consistent representation of the system's state across restarts.

[sourcery-ai]

Reviewer's Guide by Sourcery

This pull request introduces a new journal file format (v2) with improved data integrity, enhances the daemon status file with more host and OS information, removes the file locking mechanism, and comments out OS-specific file metadata code.

Updated class diagram for journal file v2 format

classDiagram

  class journal_v2_header {
    uint32_t magic
    usec_t start_time_ut
    usec_t end_time_ut
    uint32_t extent_count
    uint32_t extent_offset
    uint32_t metric_count
    uint32_t metric_offset
    uint32_t page_count
    uint32_t page_offset
    uint32_t extent_trailer_offset
    uint32_t metric_trailer_offset
    uint32_t journal_v1_file_size
    uint32_t journal_v2_file_size
    void *data
  }

  class journal_extent_list {
    uint64_t datafile_offset
    uint32_t datafile_size
    uint16_t file_index
    uint8_t pages
  }

  class journal_metric_list {
    nd_uuid_t uuid
    uint32_t entries
    uint32_t page_offset
    uint32_t delta_start_s
    uint32_t delta_end_s
    uint32_t update_every_s
  }

  class journal_page_header {
    uint32_t crc
    uint32_t uuid_offset
    uint32_t entries
    nd_uuid_t uuid
  }

  class journal_page_list {
    uint32_t delta_start_s
    uint32_t delta_end_s
    uint32_t extent_index
    uint32_t update_every_s
    uint16_t page_length
    uint8_t type
  }

  journal_v2_header -- journal_extent_list : contains
  journal_v2_header -- journal_metric_list : contains
  journal_metric_list -- journal_page_header : contains
  journal_page_header -- journal_page_list : contains
  journal_page_list -- journal_extent_list : refers to

  note for journal_v2_header "Journal v2 header structure"
  note for journal_extent_list "Extent section item"
  note for journal_metric_list "Metric section item"
  note for journal_page_header "Page section item header"
  note for journal_page_list "Page section item"

Loading

Updated class diagram for DAEMON_STATUS_FILE

classDiagram
  class DAEMON_STATUS_FILE {
    OS_SYSTEM_MEMORY memory
    OS_SYSTEM_DISK_SPACE var_cache
    const char *architecture
    const char *virtualization
    const char *container
    const char *kernel_version
    const char *os_name
    const char *os_version
    const char *os_id
    const char *os_id_like
    bool read_system_info
  }

  note for DAEMON_STATUS_FILE "Daemon status file structure"

Loading

File-Level Changes

Change	Details	Files
Implements a new journal file format (v2) with enhanced data integrity and organization.	Defines the structure of the journal v2 file format, including header, extent section, metrics section, page data section, and file trailer. Adds CRC32 checksums to all sections for data integrity. Specifies the time representation used in the journal file. Defines structures for the journal v2 header, extent list, metric list, page header, and page list. Adds a macro for journal v2 header padding. Removes the old journal v2 block trailer structure.	src/database/engine/journalfile.h
Enhances daemon status file to include more host and OS information.	Adds architecture, virtualization, and container information to the host object in the daemon status file. Adds kernel version, OS name, OS version, OS family, and OS platform to the OS object in the daemon status file. Updates the daemon status file JSON serialization and deserialization functions to include the new fields. Adds a function to retrieve daemon status fields from system info. Populates daemon status fields from system info if they are not already set.	src/daemon/daemon-status-file.c src/daemon/daemon-status-file.h src/database/rrdhost-system-info.c src/database/rrdhost-system-info.h
Removes file locking mechanism.	Removes the file locking code in netdata_main.	src/daemon/main.c
Comments out OS-specific file metadata code.	Comments out the platform-specific implementations for os_get_file_metadata.	src/libnetdata/os/file_metadata.c

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!
• Generate a plan of action for an issue: Comment @sourcery-ai plan on
  an issue to generate a plan of action for it.

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.