Added AI Agent Semantic Convention #1739

gyliu513 · 2025-01-13T20:05:39Z

Changes

Please provide a brief description of the changes here.

Note: if the PR is touching an area that is not listed in the existing areas, or the area does not have sufficient domain experts coverage, the PR might be tagged as experts needed and move slowly until experts are identified.

Merge requirement checklist

CONTRIBUTING.md guidelines followed.
Change log entry added, according to the guidelines in When to add a changelog entry.
- If your PR does not need a change log, start the PR title with [chore]
schema-next.yaml updated with changes to existing conventions.

Signed-off-by: Guangya Liu <gyliu@ibm.com>

lmolkova

Great start!

.chloggen/1732.yaml

model/gen-ai/registry.yaml

model/gen-ai/spans.yaml

docs/gen-ai/gen-ai-agent-spans.md

gyliu513

@lmolkova thanks for all of your comments, most of them are addressed now, but I still need to check how to define the tool datastore, let me check more for this.

.chloggen/1732.yaml

model/gen-ai/registry.yaml

model/gen-ai/spans.yaml

docs/gen-ai/gen-ai-agent-spans.md

lmolkova

Thank you, it looks great, left a small comment

model/gen-ai/spans.yaml

model/gen-ai/registry.yaml

model/gen-ai/spans.yaml

docs/attributes-registry/gen-ai.md

docs/gen-ai/gen-ai-agent-spans.md

samuelcolvin

Personally I think this concentrates too much on remote agents, which AFAIK are not widely used.

I suggest create_agent should be removed until there's more evidence of the adoption of the underlying pattern.

docs/attributes-registry/gen-ai.md

docs/gen-ai/gen-ai-agent-spans.md

gyliu513 · 2025-01-28T20:03:12Z

Hey all, thanks for the comments, we have several follow ups for this PR, I will create related PRs soon to fix.

model/gen-ai/spans.yaml

### Motivation and Context  Resolves: #10174 This PR instruments Agent invocation operation with traces and adds an example with tracing enabled using Console and Application Insights. Trace operation and attribute names are based on the work in following PR about adding AI Agent Semantic Convention to OpenTelemetry (currently in-progress): open-telemetry/semantic-conventions#1739. Examples from Application Insights: End-to-end transaction: ![image](https://github.com/user-attachments/assets/80647a3e-d2a4-4c08-8e3f-029727cb91ed) Traces: ![image](https://github.com/user-attachments/assets/9f51c94c-3020-4106-af10-32e2a6cad023) Trace attributes related to Agent: ![image](https://github.com/user-attachments/assets/c178bd84-812a-46cd-a8e1-f507238e7fa1) ### Contribution Checklist  - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄

.chloggen/1732.yaml

### Motivation and Context  Resolves: #10174 This PR instruments Agent invocation operation with traces and adds an example with tracing enabled using Console and Application Insights. Trace operation and attribute names are based on the work in following PR about adding AI Agent Semantic Convention to OpenTelemetry (currently in-progress): open-telemetry/semantic-conventions#1739. Examples from Application Insights: End-to-end transaction: ![image](https://github.com/user-attachments/assets/80647a3e-d2a4-4c08-8e3f-029727cb91ed) Traces: ![image](https://github.com/user-attachments/assets/9f51c94c-3020-4106-af10-32e2a6cad023) Trace attributes related to Agent: ![image](https://github.com/user-attachments/assets/c178bd84-812a-46cd-a8e1-f507238e7fa1) ### Contribution Checklist  - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄

- Generate `semconv/v1.31.0` - Stop generating deprecated metric semconv similar to all other generation - Fix acronyms: - `ReplicationController` - `ResourceQuota` ## [`v1.31.0` semantic convention release notes](https://github.com/open-telemetry/semantic-conventions/releases/tag/v1.31.0): <h3>🛑 Breaking changes 🛑</h3> <ul> <li> <code>code</code>: <code>code.function.name</code> value should contain the fully qualified function name, <code>code.namespace</code> is now deprecated (<a href="open-telemetry/semantic-conventions#1677" data-hovercard-type="issue" data-hovercard-url="/open-telemetry/semantic-conventions/issues/1677/hovercard">#1677</a>) </li> <li> <code>gen-ai</code>: Introduce <code>gen_ai.output.type</code>and deprecate <code>gen_ai.openai.request.response_format</code> (<a href="open-telemetry/semantic-conventions#1757" data-hovercard-type="pull_request" data-hovercard-url="/open-telemetry/semantic-conventions/pull/1757/hovercard">#1757</a>) </li> <li> <code>mobile</code>: Rework <code>device.app.lifecycle</code> mobile event. (<a href="open-telemetry/semantic-conventions#1880" data-hovercard-type="pull_request" data-hovercard-url="/open-telemetry/semantic-conventions/pull/1880/hovercard">#1880</a>) The <code>device.app.lifecycle</code> event has been reworked to use attributes instead of event body fields. The <code>ios.app.state</code> and <code>android.app.state</code> attributes have been reintroduced to the attribute registry. </li> <li> <code>system</code>: Move CPU-related system.cpu.* metrics to CPU namespace (<a href="open-telemetry/semantic-conventions#1873" data-hovercard-type="issue" data-hovercard-url="/open-telemetry/semantic-conventions/issues/1873/hovercard">#1873</a>) </li> <li> <code>k8s</code>: Change k8s.replication_controller metrics to k8s.replicationcontroller (<a href="open-telemetry/semantic-conventions#1848" data-hovercard-type="pull_request" data-hovercard-url="/open-telemetry/semantic-conventions/pull/1848/hovercard">#1848</a>) </li> <li> <code>db</code>: Rename <code>db.system</code> to <code>db.system.name</code> in database metrics, and update the values to be consistent with database spans. (<a href="open-telemetry/semantic-conventions#1581" data-hovercard-type="issue" data-hovercard-url="/open-telemetry/semantic-conventions/issues/1581/hovercard">#1581</a>) </li> <li> <code>session</code>: Move <code>session.id</code> and <code>session.previous_id</code> from body fields to event attributes, and yamlize <code>session.start</code> and <code>session.end</code> events. (<a href="open-telemetry/semantic-conventions#1845" data-hovercard-type="pull_request" data-hovercard-url="/open-telemetry/semantic-conventions/pull/1845/hovercard">#1845</a>) As part of the ongoing migration of event fields from LogRecord body to extended/complex attributes, the <code>session.start</code> and <code>session.end</code> events have been redefined. </li> </ul> <h3>💡 Enhancements 💡</h3> <ul> <li> <code>code</code>: Mark <code>code.*</code> semantic conventions as release candidate (<a href="open-telemetry/semantic-conventions#1377" data-hovercard-type="issue" data-hovercard-url="/open-telemetry/semantic-conventions/issues/1377/hovercard">#1377</a>) </li> <li> <code>gen-ai</code>: Added AI Agent Semantic Convention (<a href="open-telemetry/semantic-conventions#1732" data-hovercard-type="issue" data-hovercard-url="/open-telemetry/semantic-conventions/issues/1732/hovercard">#1732</a>, <a href="open-telemetry/semantic-conventions#1739" data-hovercard-type="pull_request" data-hovercard-url="/open-telemetry/semantic-conventions/pull/1739/hovercard">#1739</a>) </li> <li> <code>db</code>: Add database-specific notes on db.operation.name and db.collection.name for Cassandra, Cosmos DB, HBase, MongoDB, and Redis, covering their batch/bulk terms and lack of cross-table queries. (<a href="open-telemetry/semantic-conventions#1863" data-hovercard-type="pull_request" data-hovercard-url="/open-telemetry/semantic-conventions/pull/1863/hovercard">#1863</a>, <a href="open-telemetry/semantic-conventions#1573" data-hovercard-type="issue" data-hovercard-url="/open-telemetry/semantic-conventions/issues/1573/hovercard">#1573</a>) </li> <li> <code>gen-ai</code>: Adds <code>gen_ai.request.choice.count</code> span attribute (<a href="open-telemetry/semantic-conventions#1888" data-hovercard-type="issue" data-hovercard-url="/open-telemetry/semantic-conventions/issues/1888/hovercard">#1888</a>) Enables recording target number of completions to generate </li> <li> <code>enduser</code>: Undeprecate 'enduser.id' and introduce new attribute <code>enduser.pseudo.id</code> (<a href="open-telemetry/semantic-conventions#1104" data-hovercard-type="issue" data-hovercard-url="/open-telemetry/semantic-conventions/issues/1104/hovercard">#1104</a>) The new attribute <code>enduser.pseudo.id</code> is intended to provide a unique identifier of a pseudonymous enduser. </li> <li> <code>k8s</code>: Add <code>k8s.hpa</code>, <code>k8s.resourcequota</code> and <code>k8s.replicationcontroller</code> attributes and resources (<a href="open-telemetry/semantic-conventions#1656" data-hovercard-type="issue" data-hovercard-url="/open-telemetry/semantic-conventions/issues/1656/hovercard">#1656</a>) </li> <li> <code>k8s</code>: How to populate resource attributes based on attributes, labels and transformation (<a href="open-telemetry/semantic-conventions#236" data-hovercard-type="issue" data-hovercard-url="/open-telemetry/semantic-conventions/issues/236/hovercard">#236</a>) </li> <li> <code>process</code>: Adjust the semantic expectations for <code>process.executable.name</code> (<a href="open-telemetry/semantic-conventions#1736" data-hovercard-type="issue" data-hovercard-url="/open-telemetry/semantic-conventions/issues/1736/hovercard">#1736</a>) </li> <li> <code>otel</code>: Adds SDK self-monitoring metrics for span processing (<a href="open-telemetry/semantic-conventions#1631" data-hovercard-type="pull_request" data-hovercard-url="/open-telemetry/semantic-conventions/pull/1631/hovercard">#1631</a>) </li> <li> <code>cicd</code>: Adds a new attribute <code>cicd.pipeline.run.url.full</code> and corrects the attribute description of <code>cicd.pipeline.task.run.url.full</code> (<a href="open-telemetry/semantic-conventions#1796" data-hovercard-type="issue" data-hovercard-url="/open-telemetry/semantic-conventions/issues/1796/hovercard">#1796</a>) </li> <li> <code>user-agent</code>: Add <code>user_agent.os.name</code> and <code>user_agent.os.version</code> attributes (<a href="open-telemetry/semantic-conventions#1433" data-hovercard-type="issue" data-hovercard-url="/open-telemetry/semantic-conventions/issues/1433/hovercard">#1433</a>) </li> </ul> <h3>🧰 Bug fixes 🧰</h3> <ul> <li><code>process</code>: Fix units of process.open_file_descriptor.count and process.context_switches (<a href="open-telemetry/semantic-conventions#1662" data-hovercard-type="issue" data-hovercard-url="/open-telemetry/semantic-conventions/issues/1662/hovercard">#1662</a>)</li> </ul> --------- Co-authored-by: Robert Pająk <pellared@hotmail.com>

gyliu513 requested review from a team as code owners January 13, 2025 20:05

gyliu513 marked this pull request as draft January 13, 2025 20:06

gyliu513 force-pushed the agent-sc branch 8 times, most recently from e814811 to 7b7efce Compare January 15, 2025 21:06

Added AI Agent Semantic Convention

c7fa834

Signed-off-by: Guangya Liu <gyliu@ibm.com>

gyliu513 force-pushed the agent-sc branch from 7b7efce to c7fa834 Compare January 15, 2025 21:12

Merge branch 'main' into agent-sc

f4c961a

gyliu513 marked this pull request as ready for review January 15, 2025 21:12

lmolkova reviewed Jan 15, 2025

View reviewed changes

gyliu513 commented Jan 16, 2025

View reviewed changes

gyliu513 force-pushed the agent-sc branch 5 times, most recently from 5628dbe to 9904399 Compare January 17, 2025 03:45

lmolkova approved these changes Jan 22, 2025

View reviewed changes

model/gen-ai/spans.yaml Outdated Show resolved Hide resolved

1st round review from @lmolkova

556aa52

gyliu513 commented Jan 22, 2025

View reviewed changes

model/gen-ai/registry.yaml Outdated Show resolved Hide resolved

model/gen-ai/spans.yaml Outdated Show resolved Hide resolved

gyliu513 force-pushed the agent-sc branch from 9904399 to 556aa52 Compare January 22, 2025 16:44

lmolkova added 2 commits January 22, 2025 13:43

Merge branch 'main' into agent-sc

a7809f0

Merge branch 'main' into agent-sc

46cf30c

github-actions bot added the enhancement New feature or request label Jan 23, 2025

Merge branch 'main' into agent-sc

6d2aa19

TaoChenOSU reviewed Jan 23, 2025

View reviewed changes

docs/attributes-registry/gen-ai.md Show resolved Hide resolved

TaoChenOSU reviewed Jan 23, 2025

View reviewed changes

docs/attributes-registry/gen-ai.md Show resolved Hide resolved

TaoChenOSU reviewed Jan 23, 2025

View reviewed changes

docs/attributes-registry/gen-ai.md Show resolved Hide resolved

TaoChenOSU reviewed Jan 23, 2025

View reviewed changes

docs/gen-ai/gen-ai-agent-spans.md Show resolved Hide resolved

samuelcolvin suggested changes Jan 24, 2025

View reviewed changes

docs/attributes-registry/gen-ai.md Outdated Show resolved Hide resolved

docs/attributes-registry/gen-ai.md Show resolved Hide resolved

docs/gen-ai/gen-ai-agent-spans.md Show resolved Hide resolved

lmolkova mentioned this pull request Jan 28, 2025

Added new AWS Attributes for various services #1794

Open

3 tasks

added client-side & agent-side description

b9cc0a2

Merge branch 'main' into agent-sc

6142c54

TaoChenOSU approved these changes Jan 29, 2025

View reviewed changes

lmolkova reviewed Jan 30, 2025

View reviewed changes

model/gen-ai/spans.yaml Outdated Show resolved Hide resolved

remote agent service

5d4b2a8

dmytrostruk mentioned this pull request Feb 4, 2025

.Net: Added traces for Agent invocations microsoft/semantic-kernel#10387

Merged

4 tasks

lmolkova approved these changes Feb 4, 2025

View reviewed changes

.chloggen/1732.yaml Outdated Show resolved Hide resolved

lmolkova added 2 commits February 4, 2025 14:17

Update .chloggen/1732.yaml

919b4e7

Merge branch 'main' into agent-sc

dddd34e

fix ci error for semantic convention

e13d38a

lmolkova merged commit 5da613a into open-telemetry:main Feb 5, 2025
15 checks passed

gyliu513 deleted the agent-sc branch February 5, 2025 01:27

gyliu513 mentioned this pull request Feb 5, 2025

AI Agent Span Semantic Convention #1657

Draft

3 tasks

MrAlias mentioned this pull request Mar 20, 2025

Generate v1.31.0 semconv open-telemetry/opentelemetry-go#6479

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added AI Agent Semantic Convention #1739

Added AI Agent Semantic Convention #1739

gyliu513 commented Jan 13, 2025

lmolkova left a comment

gyliu513 left a comment

lmolkova left a comment

samuelcolvin left a comment

gyliu513 commented Jan 28, 2025

Added AI Agent Semantic Convention #1739

Added AI Agent Semantic Convention #1739

Conversation

gyliu513 commented Jan 13, 2025

Changes

Merge requirement checklist

lmolkova left a comment

Choose a reason for hiding this comment

gyliu513 left a comment

Choose a reason for hiding this comment

lmolkova left a comment

Choose a reason for hiding this comment

samuelcolvin left a comment

Choose a reason for hiding this comment

gyliu513 commented Jan 28, 2025