feat(anta): Limit concurrency #680

carl-baillargeon · 2024-05-16T05:04:18Z

Description

This PR improves the test runner by introducing a generator-based approach for managing test coroutines and setting a configurable limit on the number of concurrent tests.

Instead of loading all test coroutines into a list, the runner now uses a generator to yield tests. This approach prevents memory overload and improves performance when dealing with a large number of tests.
A limit on the number of concurrent tests is introduced to avoid overwhelming the runner. This limit is configurable with an environement variable.

Implementation:
The generator yields test coroutines, ensuring that only a limited number of tests are scheduled and run concurrently.
Upon reaching the concurrency limit, the runner waits for some tests to complete before scheduling new ones from the generator.

Fixes: #832

Checklist:

Update FAQ
Update benchmark + unit tests
ci: add codspeed to benchmark ANTA #826
Confirm that the number of open file descriptors never exceed device * max_connections (100 by default)

sonarqubecloud · 2024-06-12T16:43:30Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

github-actions · 2024-07-04T14:03:49Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

anta/runner.py

github-actions · 2024-08-28T18:50:12Z

Conflicts have been resolved. A maintainer will review the pull request shortly.

sonarqubecloud · 2024-08-28T19:07:52Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
84.8% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

github-actions · 2024-09-11T13:47:41Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

github-actions · 2024-10-02T15:05:25Z

Conflicts have been resolved. A maintainer will review the pull request shortly.

codspeed-hq · 2024-10-02T15:15:13Z

CodSpeed Performance Report

Merging #680 will not alter performance

_{Comparing carl-baillargeon:refactor/runner_limit (545324d) with main (9ba0284)}

Summary

✅ 22 untouched benchmarks
🆕 2 new benchmarks

Benchmarks breakdown

	Benchmark	`BASE`	`HEAD`	Change
🆕	`test_setup_tests[1-device]`	N/A	16.2 ms	N/A
🆕	`test_setup_tests[2-devices]`	N/A	16.3 ms	N/A

github-actions · 2024-10-10T18:49:29Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

github-actions · 2024-11-04T21:15:42Z

Conflicts have been resolved. A maintainer will review the pull request shortly.

for more information, see https://pre-commit.ci

anta/runner.py

mtache · 2025-01-21T15:15:29Z

tests/benchmark/test_anta.py

@@ -69,6 +70,7 @@ def test_anta(

    results = session_results[request.node.callspec.id]

+    # TODO: Use AntaRunner in ANTA v2.0.0


I put this TODO everywhere we are using main().

mtache

Default value code could be simplified/cleaned.
Good idea to have this anta.settings module, we should use it for ANTA_DEBUG, in another PR ;)

github-actions · 2025-02-07T13:16:51Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

github-actions · 2025-02-07T13:20:37Z

Conflicts have been resolved. A maintainer will review the pull request shortly.

gmuloc

only the guide

docs/advanced_usages/scaling.md

github-actions · 2025-02-21T08:22:15Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

github-actions · 2025-02-24T12:21:05Z

Conflicts have been resolved. A maintainer will review the pull request shortly.

sonarqubecloud · 2025-02-26T14:08:26Z

Quality Gate passed

Issues
3 New issues
0 Accepted issues

Measures
0 Security Hotspots
93.5% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

github-actions bot assigned carl-baillargeon May 16, 2024

carl-baillargeon force-pushed the refactor/runner_limit branch from bf88158 to d7de654 Compare June 12, 2024 16:43

github-actions bot added the state: conflict label Jul 4, 2024

gmuloc reviewed Aug 1, 2024

View reviewed changes

anta/runner.py Outdated Show resolved Hide resolved

gmuloc reviewed Aug 1, 2024

View reviewed changes

anta/runner.py Outdated Show resolved Hide resolved

carl-baillargeon force-pushed the refactor/runner_limit branch from b623e90 to 4879120 Compare August 28, 2024 18:49

github-actions bot removed the state: conflict label Aug 28, 2024

github-actions bot added the state: conflict label Sep 11, 2024

mtache changed the title ~~refactor(anta): Refactor runner to use a generator with a limit~~ feat(anta): Limit concurrency Oct 2, 2024

feat(anta): limit the number of tests run concurrently

83206e8

mtache force-pushed the refactor/runner_limit branch from 68e259e to 83206e8 Compare October 2, 2024 15:05

github-actions bot removed the state: conflict label Oct 2, 2024

gmuloc and others added 2 commits October 7, 2024 09:53

Merge branch 'main' into refactor/runner_limit

d86f85c

Merge branch 'main' into refactor/runner_limit

6d59aa9

github-actions bot added the state: conflict label Oct 10, 2024

Merge branch 'main' into refactor/runner_limit

d2aa7c7

github-actions bot removed the state: conflict label Nov 4, 2024

gmuloc and others added 2 commits November 4, 2024 23:30

Refactor: Fix github merge conflict resolution

7ac0358

[pre-commit.ci] auto fixes from pre-commit.com hooks

96da6ef

for more information, see https://pre-commit.ci

gmuloc reviewed Nov 4, 2024

View reviewed changes

anta/runner.py Outdated Show resolved Hide resolved

test: Temporarily disable one benchmark

f2f3c6a

mtache reviewed Jan 21, 2025

View reviewed changes

mtache requested changes Jan 21, 2025

View reviewed changes

mtache and others added 2 commits January 21, 2025 16:25

Merge branch 'main' into refactor/runner_limit

610d2a2

Merge branch 'main' into refactor/runner_limit

5226028

carl-baillargeon linked an issue Feb 6, 2025 that may be closed by this pull request

Refactor: ANTA NRFU RunInformation log should be adjusted to be less confusing #832

Open

gmuloc requested a review from mtache February 7, 2025 10:31

Merge branch 'main' into refactor/runner_limit

9395a58

github-actions bot added the state: conflict label Feb 7, 2025

github-actions bot removed the state: conflict label Feb 7, 2025

carl-baillargeon added 2 commits February 7, 2025 14:45

Address review comments

d8ca4ac

Merge branch 'main' into refactor/runner_limit

216598a

carl-baillargeon force-pushed the refactor/runner_limit branch from 925be23 to 216598a Compare February 7, 2025 19:47

carl-baillargeon added 3 commits February 7, 2025 15:14

Fix minor things

17b244a

Address more review comments

dee3bc9

Update doc for inf

9af6327

gmuloc reviewed Feb 10, 2025

View reviewed changes

github-actions bot added the state: conflict label Feb 21, 2025

carl-baillargeon added 3 commits February 22, 2025 13:09

Merge branch 'main' into refactor/runner_limit

e2e21f1

Remove scaling guide

5a4c37a

Update unit tests

e25826e

github-actions bot removed the state: conflict label Feb 24, 2025

carl-baillargeon added 2 commits February 24, 2025 07:44

Fix unit tests for Windows

84fd77a

Update

9750d6c

carl-baillargeon requested a review from gmuloc February 25, 2025 13:27

Merge branch 'main' into refactor/runner_limit

545324d

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(anta): Limit concurrency #680

feat(anta): Limit concurrency #680

carl-baillargeon commented May 16, 2024 •

edited

Loading

sonarqubecloud bot commented Jun 12, 2024

github-actions bot commented Jul 4, 2024

github-actions bot commented Aug 28, 2024

sonarqubecloud bot commented Aug 28, 2024

github-actions bot commented Sep 11, 2024

github-actions bot commented Oct 2, 2024

codspeed-hq bot commented Oct 2, 2024 •

edited

Loading

github-actions bot commented Oct 10, 2024

github-actions bot commented Nov 4, 2024

mtache Jan 21, 2025

carl-baillargeon Jan 22, 2025

mtache left a comment

github-actions bot commented Feb 7, 2025

github-actions bot commented Feb 7, 2025

gmuloc left a comment

github-actions bot commented Feb 21, 2025

github-actions bot commented Feb 24, 2025

sonarqubecloud bot commented Feb 26, 2025

		@@ -69,6 +70,7 @@ def test_anta(

		results = session_results[request.node.callspec.id]

		# TODO: Use AntaRunner in ANTA v2.0.0

feat(anta): Limit concurrency #680

Are you sure you want to change the base?

feat(anta): Limit concurrency #680

Conversation

carl-baillargeon commented May 16, 2024 • edited Loading

Description

Checklist:

sonarqubecloud bot commented Jun 12, 2024

Quality Gate passed

github-actions bot commented Jul 4, 2024

github-actions bot commented Aug 28, 2024

sonarqubecloud bot commented Aug 28, 2024

Quality Gate passed

github-actions bot commented Sep 11, 2024

github-actions bot commented Oct 2, 2024

codspeed-hq bot commented Oct 2, 2024 • edited Loading

CodSpeed Performance Report

Merging #680 will not alter performance

Summary

Benchmarks breakdown

github-actions bot commented Oct 10, 2024

github-actions bot commented Nov 4, 2024

mtache Jan 21, 2025

Choose a reason for hiding this comment

carl-baillargeon Jan 22, 2025

Choose a reason for hiding this comment

mtache left a comment

Choose a reason for hiding this comment

github-actions bot commented Feb 7, 2025

github-actions bot commented Feb 7, 2025

gmuloc left a comment

Choose a reason for hiding this comment

github-actions bot commented Feb 21, 2025

github-actions bot commented Feb 24, 2025

sonarqubecloud bot commented Feb 26, 2025

Quality Gate passed

carl-baillargeon commented May 16, 2024 •

edited

Loading

codspeed-hq bot commented Oct 2, 2024 •

edited

Loading