[qob] Add ability to attach to existing batch

[jigold]

Change Description

This PR allows you to reuse an existing batch when submitting QoB jobs. The goal of this change is to be able to group QoB queries together to allow for easier visibility of how much a pipeline costs as well as make the batches page less cluttered so it's easier to tell what is running.

Security Assessment

• This change has no security impact

Impact Description

This change only affects the client library and doesn't touch the backend code.

(Reviewers: please confirm the security impact before approving)

---

You all changed how the tests are structured. Please rewrite the test if it doesn't make sense given the current infrastructure.

[jigold]

This code was unused.

[ehigham]

hail/hail/python/hail/backend/service_backend.py

Lines 404 to 415 in 2937ed8

	await self._batch.submit(disable_progress_bar=True)
	self._batch_was_submitted = True
	with timings.step("wait driver"):
	try:
	await asyncio.sleep(0.6) # it is not possible for the batch to be finished in less than 600ms
	await self._batch.wait(
	description=name,
	disable_progress_bar=self.disable_progress_bar,
	progress=progress,
	starting_job=j.job_id,
	)

^ these line makes me scared. What happens if this is submitted from an existing batch that waits for this job. Is this effectively a deadlock? I'm guessing yes and the solution is job groups in the batch client...

[jigold]

To my understanding and a quick glance at the code, each new query is in its own job group and the driver waits on jobs within it's child job group.

hail/hail/python/hail/backend/service_backend.py

Line 389 in fc85a38

job_group=self._batch.create_job_group(attributes={'name': name}),

I could be wrong though and it's a legitimate concern.