Testing multiple zebrads

[ala-mode]

Adding tests that fail with error from #19

Aimed to be fixed with #79

[nachog00]

This is a sequential launch of zebrad's. Notice how there are being awaited.

We need a single test to launch 2+ instances in parallel. In as comparable as possible of a way as how nextest run launches 2 tests and each test launches a zebrad process.

Only then will they be really competing for the cache dir lock.

[ala-mode]

I'm not sure. Are you saying that...

[ala-mode]

This zebrad1 being launched ...

[ala-mode]

awaited here...

[ala-mode]

stops because this zebrad2 launching with the same chain_cache config..

[ala-mode]

and means this test running zebrad1 ..

[ala-mode]

Or this one, is corrupted because the launches are being awaited?

I get the error ..IO error: While lock file... with launch_zebrad_multiple_times_with_cache and either
launch_zebrad_with_cache or launch_zebrad_with_cache_again

though I have not completely isolated the chain_cache dir yet, I think your fix will make all of these pass.

We can also write a test that will spawn multiple tasks, but I think these tests are useful.

[nachog00]

launch_zebrad_multiple_times_with_cache is failing because zebrad1 isn't being closed. If you do close it, it should launch both alright.

The key aspect is that they only raco for the lock if they are both running at the same time.

launch_zebrad_multiple_times_with_cache runs them sequentially, since its awaiting them.

I agree that it's an useful test. Could be made into 2 distinct ones:

• one that does just this and DOESNT stop zebrad1 and it asserts that it fails (without the copy cache fix)
• one that does stop zebrad1 and asserts that both run alright.

BUT this test isn't yet running 2 zebrad instances in parallel like 2 distinct tests each spawning a separate zebrad processes would. That's what we are aiming for.

[ala-mode]

I think, in this test, against current dev, it is running two instances at the same time, with the same cache... please read my comment below.

[ala-mode]

Each cargo nextest run test runs in an (OS) process. So, to make a single test act like two tests that are causing a failure (#19), I tried to replicate that case with launch_zebrad_with_cache_again - essentially, the same test twice.

We could additionally perhaps make new, more robust, longer-running tests similarly.

In the case of a single test: all async tasks are managed by the runtime, tokio's runtime. tokio::spawn makes a lightweight task without a new process. Then if we want, we can bring async tasks back into sync, we can await, either at each launch or we can wait for each task to complete if we do, say, a joinset. I am pretty sure join!() is the same too. This isn't the way it's done in any of the code as far as I see, but I don't think this point should make a difference.

Zebrad::launch itself is an async function which returns returns a Future: (I think it's correct to say 'of type') Result<Self, Launcherror>. If there's another function we want to test against, let's look at that. There are places in the code where the function is impl LocalNet<Zainod, Zebrad> or impl LocalNet<Lightwalletd, Zebrad> , but I believe those async functions also just in turn call to Zebrad::launch

One other thing we could do is to use std::process::Command to launch zebrad in child processes. However then we would have (AFAIK) essentially pushed those into the background, so I think we would have to issue RPC commands or some other kind of signaling. I believe that's beyond the scope for the error we wanted to overcome, and I'm not sure what it would demonstrate. It would be different to have child processes than two separate tests, but maybe it could provide something? I don't see a place in prior testing where this is done.

I'll make another test where both are launched before any await is called... If I can do that, will you show me how that forthcoming test and launch_zebrad_multiple_times_with_cache behave differently?

And then, if we really want to get visibility to confirm some state during testing, we could write some kind of script to inspect and record the process(es) running during tests.

I added a test with 3 async zebrad tasks: launch_zebrad_multiple_times_in_tokio_tasks_with_cache, and both that and the one without tasks launch_zebrad_multiple_times_with_cache fail for me with IO error: While lock file: ...