Custom Python runtime metrics dashboard

[xrmx]

Is your feature request related to a problem? Please describe.

We should create our own runtime metrics dashboard in Kibana to have a decent UX out of the box.

One thing we must take into account are metrics vs fork(). On projects that are using processes for concurrency the same service may send different metrics values for each process.
Not sure we have a good solution for handling this in OTel upstream.

[AlexanderWert]

Duplicate of #24

[xrmx]

@AlexanderWert #24 was for system metrics but we later decided to disable them by default. This one was for showing something out of the box on the metrics page.

[AlexanderWert]

sorry, reopened

[xrmx]

After discussion with the APM team it was suggested to show only aggregated data in the dashboard so that multiple processes sending the same metrics won't be a problem

[xrmx]

And before doing a dashboard it would be better to wait for https://github.com/open-telemetry/opentelemetry-python-contrib/pull/2727 (or similar)

[xrmx]

Created open-telemetry/opentelemetry-python-contrib#3250 as a prerequisite for this, we want to track current semconv process.* metrics (while still experimental) and not some very old metrics (process.runtime.*).

[xrmx]

Opened a PR to have the semantic conventions to track CPython garbage collector open-telemetry/semantic-conventions#1931 .

[xrmx]

Kibana PR for adding a custom dashboard for EDOT Python where we don't have the system metrics used by the default one elastic/kibana#213599

[xrmx]

Kibana PR merged and will be available in 8.18, 8.x and 9.0