Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Overhaul stats: Segregated metrics for each tracker running on a different socket address #1263

Open
josecelano opened this issue Feb 12, 2025 · 4 comments
Assignees
Labels
- Admin - Enjoyable to Install and Setup our Software Enhancement / Feature Request Something New

Comments

@josecelano
Copy link
Member

josecelano commented Feb 12, 2025

Relates to: #1341 (comment)
Depends on: #1385

While I was reviewing the tracker-core package documentation I've realized we have the tracker_usage_statistics config option in the core configuration:

[core]
tracker_usage_statistics = true

That value enables or disables the statistics.

I've recently moved the statistics out of the tracker core package. Therefore It does not make sense to have that config option there because there is nothing related to statistics tin the tracker-core package now.

That value is used in the app bootstrap:

// HTTP stats
let (http_stats_event_sender, http_stats_repository) =
    http_tracker_core::statistics::setup::factory(configuration.core.tracker_usage_statistics);
let http_stats_event_sender = Arc::new(http_stats_event_sender);
let http_stats_repository = Arc::new(http_stats_repository);

// UDP stats
let (udp_stats_event_sender, udp_stats_repository) =
    udp_tracker_core::statistics::setup::factory(configuration.core.tracker_usage_statistics);
let udp_stats_event_sender = Arc::new(udp_stats_event_sender);
let udp_stats_repository = Arc::new(udp_stats_repository);

The same config option enables stats in both the UDP and HTTP trackers.

The current config omitting irrelevant values:

[core]
tracker_usage_statistics = true

[[udp_trackers]]
bind_address = "0.0.0.0:6969"

  [udp_trackers.cookie_lifetime]
  secs = 120
  nanos = 0

[[http_trackers]]
bind_address = "0.0.0.0:7070"

[http_api]
bind_address = "0.0.0.0:1212"

  [http_api.access_tokens]
  admin = "***"

[health_check_api]
bind_address = "127.0.0.1:1313"

Maybe we should create two new values for UDP and HTTP trackers. However in the current configuration there is not a place to put generic configuration for all HTTP trackers or all UDP trackers.

We can only add a config option per instance.

NOTE: That you can run more than one HTTP and/or UDP trackers.

If we do that the config could be:

[core]
tracker_usage_statistics = true

[[udp_trackers]]
bind_address = "0.0.0.0:6969"
tracker_usage_statistics = true

  [udp_trackers.cookie_lifetime]
  secs = 120
  nanos = 0

[[http_trackers]]
bind_address = "0.0.0.0:7070"
tracker_usage_statistics = true

[http_api]
bind_address = "0.0.0.0:1212"

  [http_api.access_tokens]
  admin = "***"

[health_check_api]
bind_address = "127.0.0.1:1313"

Where you can enable/disable stats for each instance.

cc @da2ce7

@josecelano josecelano added - Admin - Enjoyable to Install and Setup our Software Enhancement / Feature Request Something New labels Feb 12, 2025
@josecelano
Copy link
Member Author

I've just realized all trackers of the same type use the same events sender and metrics repository. I mean, there is only one stats event sender and stats metrics repository service for all HTTP tracker or UDP tracker. That means counters for are for all trackers. There is no way to get metrics for each socket.

If we want to enable/disable stats for all server at the same time we need to introduce new section in the configuration file. That would be global configuration for all UDP or HTTP trackers. For example:

[core]
#tracker_usage_statistics = true <- remove this

[udp_trackers]
tracker_usage_statistics = true <- global config for all UDP trackers

[http_trackers]
tracker_usage_statistics = true <- global config for all HTTP trackers

[[udp_trackers]]
bind_address = "0.0.0.0:6969"
tracker_usage_statistics = true

  [udp_trackers.cookie_lifetime]
  secs = 120
  nanos = 0

[[http_trackers]]
bind_address = "0.0.0.0:7070"
tracker_usage_statistics = true

[http_api]
bind_address = "0.0.0.0:1212"

  [http_api.access_tokens]
  admin = "***"

[health_check_api]
bind_address = "127.0.0.1:1313"

We need to decide if:

  • We want only aggregate statistics per tracker type (HTTP or UDP) or we want also statistics for each socket.
  • If we want to enable/disable statistics for each socket.

Assuming we want both (metrics per socket and aggregate metrics per tracker tpye):

  1. We can enable/disable stats in the configuration at the single service level:
[[http_trackers]]
bind_address = "0.0.0.0:7070"
tracker_usage_statistics = true
  1. We can change repositories to include single service stats.

The current ones would be aggregate metrics. We can get the individualized metrics per sockets by adding new metadata (socket address) to the event. We need to change the event from:

pub enum Event {
    Connect { ip: IpAddr }
    Announce { ip: IpAddr }
    Scrape { ip: IpAddr }
}

to:

pub enum Event {
    Connect { client_ip: IpAddr, server_socket_addr: SocketAddr }
    Announce { client_ip: IpAddr, server_socket_addr: SocketAddr }
    Scrape { client_ip: IpAddr, server_socket_addr: SocketAddr }
}

cc @da2ce7

@josecelano josecelano changed the title Add new config options to enable/disable stats for UDP and HTTP trackers. Overhaul stats events: Add new config options to enable/disable stats for UDP and HTTP trackers. Mar 3, 2025
@josecelano
Copy link
Member Author

In today's meeting @da2ce7 and I agreed on:

  • In the long-term only generate metrics and the lowest granularity level. That means we collect metrics for socket-bound services. For example, a single UDP tracker, a single HTTP tracker. Consumers of those metrics in the future have to build their own aggregate data:
    • Metrics for services using the same IP
    • Metrics for services using the same socket
    • Metrics for services using the same IP version
    • Metrics for all UDP servers (protocol)
    • Metrics for all HTTP servers (protocol)
    • Global metrics
  • We need a stats repository per socket address.
  • You can enable/disable stats for each socket-address service.
  • In the short term (the scope of this issue) we can refactor stats/events but continue exposing only the global metrics (for all trackers).
  • In the configuration the new flag to enable stats would have a true default value.

@josecelano
Copy link
Member Author

Hi @da2ce7 one question, so far we have been using events only to generate the stats. However after our discussion today I think events might be used for different purposes in the future. For example, to decouple some part of the application. I think we need to:

  • Always enable events. You cannot disable them.
  • The config option tracker_usage_statistics only enables the listener that listen to the events to collect metrics but it's does not disable sending the events.

I don't thing it's going to be cause performance issues because we always check if we have to send the event and I was even considering implementing an events sender that does nothing (null object pattern).

I think in general we have to decouple the stats terminology from the events. Right now events are only created for stats, but we should split the statistic modules into two modules:

  • Events modules: event definition and trigger events, ...
  • Statistics modules: event listeners and metrics repositories, ...

What do you thing?

Relates to: #1343

@josecelano josecelano changed the title Overhaul stats events: Add new config options to enable/disable stats for UDP and HTTP trackers. Overhaul stats: Add new config options to enable/disable stats for UDP and HTTP trackers. Mar 12, 2025
@josecelano josecelano self-assigned this Mar 12, 2025
@josecelano josecelano changed the title Overhaul stats: Add new config options to enable/disable stats for UDP and HTTP trackers. Overhaul stats: Segregated metrics for each tracker running on a different socket address Mar 12, 2025
@josecelano
Copy link
Member Author

Since we are going to add the segregated metrics instead of changing the aggregate metrics, we can do the following:

  • Rename metrics as I described here following Prometheus conventions.
  • Add metrics proposed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
- Admin - Enjoyable to Install and Setup our Software Enhancement / Feature Request Something New
Projects
None yet
Development

No branches or pull requests

1 participant