Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure Transforms and Checks are executed with the correct timeout #5471

Merged
merged 1 commit into from
Jan 21, 2025

Conversation

dgarros
Copy link
Collaborator

@dgarros dgarros commented Jan 15, 2025

Fixes #5456

This PR enforces the timeout for Transforms and Checks leveraging Prefect timeout capabilities.
One good learning from this exercice is that we can setup custom timeout when we are calling a flow or a task directly but it's not possible when using a deployment.

I was able to manually validate that it's working as expected but I couldn't come up with a good way to build a unit tests for timeout.

I also fixed a few minor issues related to flow_run_name and tags and in a few places I introduced Protocols to improve typing

@dgarros dgarros changed the title Ensure Transform are executed with the correct timeout Ensure Transforms are executed with the correct timeout Jan 15, 2025
@github-actions github-actions bot added the group/backend Issue related to the backend (API Server, Git Agent) label Jan 15, 2025
Copy link

codspeed-hq bot commented Jan 15, 2025

CodSpeed Performance Report

Merging #5471 will not alter performance

Comparing dga-20240115-fix-5456 (d3561ec) with stable (5e914b2)

Summary

✅ 10 untouched benchmarks

@dgarros dgarros force-pushed the dga-20240115-fix-5456 branch from f8cb159 to 8ec4c28 Compare January 19, 2025 14:07
@github-actions github-actions bot added the type/documentation Improvements or additions to documentation label Jan 19, 2025
@dgarros dgarros force-pushed the dga-20240115-fix-5456 branch from b05f702 to a71a49f Compare January 20, 2025 07:14
@dgarros dgarros changed the title Ensure Transforms are executed with the correct timeout Ensure Transforms and Checks are executed with the correct timeout Jan 20, 2025
@dgarros dgarros requested a review from a team January 20, 2025 07:25
@dgarros dgarros marked this pull request as ready for review January 20, 2025 07:25
@LucasG0
Copy link
Contributor

LucasG0 commented Jan 20, 2025

I was able to manually validate that it's working as expected but I couldn't come up with a good way to build a unit tests for timeout.

I think one way could be to patch flow.fn attribute of a given flow with an asyncio.sleep(timeout + 1), then call flow.with_options(timeout=timeout)() and expect it to raise a TimeoutError

@@ -264,7 +264,7 @@ async def test_import_all_yaml_files(
assert config_file
await repo.import_jinja2_transforms(branch_name="main", commit=commit, config_file=config_file)

rfiles = await client.all(kind=InfrahubKind.TRANSFORMJINJA2)
rfiles = await client.all(kind=CoreTransformJinja2)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rfiles, oh the memories :)

@@ -5,6 +5,10 @@

class PersonWithCarsTransform(InfrahubTransform):
query = "person_with_cars"
timeout = 2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on #5483 and opsmill/infrahub-sdk-python#55 I think we should look at moving the timeout value away from this file and instead define it in .infrahub.yml instead (so that it's also possible to do this with a Jinja2 Transform). I'm wondering when we make that change what the impact would be with this setting here. Do we need a temporary test that validates that the timeout of this Transform is set to 2 just so that it doesn't just slip through the cracks if we update it later. I.e. I think we'll forget to go back here and remove it and update the .infrahub.yml file for this repo.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good point but not sure if we can easily add this test today, I don't think this repo fixtures is used to test the import of Python Transform, at least I couldn't find it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's imported here:

class TestProposedChangePipelineRepository(TestInfrahubApp):
    @pytest.fixture(scope="class")
    async def initial_dataset(
        self,
        db: InfrahubDatabase,
        initialize_registry: None,
        git_repos_source_dir_module_scope: Path,
        client: InfrahubClient,
        bus_simulator: BusSimulator,
        prefect_test_fixture,
    ) -> None:
        await load_schema(db, schema=CAR_SCHEMA)

        bus_simulator.service.cache = RedisCache()

        john = await Node.init(schema=TestKind.PERSON, db=db)
        await john.new(db=db, name="John", height=175, age=25, description="The famous Joe Doe")
        await john.save(db=db)
        koenigsegg = await Node.init(schema=TestKind.MANUFACTURER, db=db)
        await koenigsegg.new(db=db, name="Koenigsegg")
        await koenigsegg.save(db=db)
        people = await Node.init(schema=InfrahubKind.STANDARDGROUP, db=db)
        await people.new(db=db, name="people", members=[john])
        await people.save(db=db)

        jesko = await Node.init(schema=TestKind.CAR, db=db)
        await jesko.new(
            db=db,
            name="Jesko",
            color="Red",
            description="A limited production mid-engine sports car",
            owner=john,
            manufacturer=koenigsegg,
        )
        await jesko.save(db=db)

        branch1 = await client.branch.create(branch_name="branch1")

        FileRepo(name="car-dealership", sources_directory=git_repos_source_dir_module_scope)
        client_repository = await client.create(
            kind=InfrahubKind.REPOSITORY,
            data={"name": "car-dealership", "location": f"{git_repos_source_dir_module_scope}/car-dealership"},
            branch=branch1.name,
        ) <---
        await client_repository.save()

I'm not sure what happens if we don't change this if something starts failing or if we'd just add 8 seconds to the test time and not realize why?

@dgarros dgarros force-pushed the dga-20240115-fix-5456 branch from a71a49f to d3561ec Compare January 20, 2025 15:25
@dgarros dgarros merged commit ce4c1d5 into stable Jan 21, 2025
36 checks passed
@dgarros dgarros deleted the dga-20240115-fix-5456 branch January 21, 2025 06:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
group/backend Issue related to the backend (API Server, Git Agent) type/documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bug: artifact generation does not respect timeout value set using INFRAHUB_TIMEOUT value
4 participants