Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ignore flaky opentelemetry-jaeger tests #1529

Conversation

mattbodd
Copy link
Contributor

@mattbodd mattbodd commented Feb 13, 2024

Fixes #1505
Two tests have been intermittently failing from the opentelemetry-jaeger crate:
exporter::config::collector::http_client::collector_client_tests::test_bring_your_own_client
exporter::config::collector::tests::test_collector_exporter

As the opentelemetry-jaeger crate will be removed in the next major release, these tests can be ignored for the time being to alleviate the issue.

Changes

Ignores tests:
exporter::config::collector::http_client::collector_client_tests::test_bring_your_own_client
exporter::config::collector::tests::test_collector_exporter

Merge requirement checklist

  • CONTRIBUTING guidelines followed
  • Unit tests added/updated (if applicable)
  • Appropriate CHANGELOG.md files updated for non-trivial, user-facing changes
  • Changes in public API reviewed (if applicable)

@mattbodd mattbodd requested a review from a team February 13, 2024 08:56
Copy link

codecov bot commented Feb 13, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (87153d3) 62.8% compared to head (f6df258) 62.5%.

Additional details and impacted files
@@           Coverage Diff           @@
##            main   #1529     +/-   ##
=======================================
- Coverage   62.8%   62.5%   -0.4%     
=======================================
  Files        144     144             
  Lines      19891   19891             
=======================================
- Hits       12502   12433     -69     
- Misses      7389    7458     +69     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@TommyCpp TommyCpp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this works.

The issue here is we didn't isolate the env vars between tests. A more robust fix probably is running jaeger tests in single thread.

Maybe we can ignore the jaeger in coverage tests?

@cijothomas
Copy link
Member

I think this works.

The issue here is we didn't isolate the env vars between tests. A more robust fix probably is running jaeger tests in single thread.

Maybe we can ignore the jaeger in coverage tests?

yes. I think the easiest approach is to just ignore the tests, and do the jaeger release, followed by its code+tests removal.

@mattbodd
Copy link
Contributor Author

Thanks @TommyCpp and @cijothomas for your comments. As Cijo mentioned, since we are deprecating opentelemetry-jaeger I didn't opt to fix these flaky tests and instead just silenced them so CI doesn't fail. Is there anything I should do differently or can this change be merged?

Copy link
Contributor

@TommyCpp TommyCpp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let add a comment that says we ignore this because it's flaky and won't fix because jaeger has been deprecated

@mattbodd
Copy link
Contributor Author

Let add a comment that says we ignore this because it's flaky and won't fix because jaeger has been deprecated

This is a good suggestion. I've left a comment above each ignored test.

@cijothomas cijothomas merged commit 054b0c8 into open-telemetry:main Feb 14, 2024
14 checks passed
@mattbodd mattbodd deleted the ignore_flaky_opentelemetry_jaeger_tests branch February 14, 2024 16:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Flaky test in CI
3 participants