Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Job execution time on qpu #231

Open
cosenal opened this issue Feb 24, 2025 · 1 comment
Open

Job execution time on qpu #231

cosenal opened this issue Feb 24, 2025 · 1 comment
Assignees

Comments

@cosenal
Copy link
Contributor

cosenal commented Feb 24, 2025

Some benchmark definitions (e.g. CLOPS) are based off execution time of a job on the qpu. For cloud-accessible platforms, it's important that this metrics is exposed via API. For these benchmarks, the clock can't be taken client-side, o/w it would include queueing time and wire time.

@cosenal cosenal self-assigned this Feb 24, 2025
@cosenal cosenal changed the title Job execution time Job execution time on qpu Feb 24, 2025
@cosenal
Copy link
Contributor Author

cosenal commented Feb 25, 2025

Some documentation of execution-time like properties for different providers

IBM

job.metadata()
ExecutionSpans

IonQ

job.metadata()
{..., 'start': 1738019212, 'response': 1738019213, 'execution_time': 823, ...}

AWS

Both job.metadata() and job.result() only contain createdAt and endedAt timestamp, but no "startedAt". Since createdAt includes queueing time, it's not useful for CLOPs-like metrics

job.metadata()
{..., 'createdAt': datetime.datetime(2025, 2, 23, 16, 39, 27, 828000, tzinfo=tzutc()), 'endedAt': datetime.datetime(2025, 2, 23, 16, 39, 32, 50000, tzinfo=tzutc()), ...}

Azure

job.result()
Result(...
creation_time='2025-02-25T15:28:30.30961Z',
  begin_execution_time='2025-02-25T15:28:33.814Z',
  end_execution_time='2025-02-25T15:28:33.829Z',
...
)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant