Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Introduce generic gen_ai.output.type instead of gen_ai.openai.request.response_format #1757

Merged
merged 14 commits into from
Feb 3, 2025
Merged

Introduce generic gen_ai.output.type instead of gen_ai.openai.request.response_format #1757

merged 14 commits into from
Feb 3, 2025

Conversation

lmolkova
Copy link
Contributor

@lmolkova lmolkova commented Jan 17, 2025 •
edited
Loading

Response format today could only work for structured outputs, but it should be useful in multi-modal cases.
In this PR, the proposal is to capture modality in the gen_ai.output.type and have dedicated attributes to describe format for specific types such as gen_ai.output.json.schema.name.

E.g. for structured outputs gen_ai.output.type = json and gen_ai.output.json.schema.name = my_type_name.
For images it could be gen_ai.output.type = image, gen_ai.output.image.format = url.

See Cohere, Azure AI Inference, Vertex AI, Vertex AI docs

Also fixing a tiny bug where we have different id and value for completion token types and other minor nits.

Merge requirement checklist

@lmolkova lmolkova changed the title Rename gen-ai.openai.request.response_format to gen-ai.request.response_format generic Rename gen-ai.openai.request.response_format to gen-ai.request.response_format Jan 17, 2025
@lmolkova lmolkova marked this pull request as ready for review January 17, 2025 17:35
@lmolkova lmolkova requested review from a team as code owners January 17, 2025 17:35
aabmass
aabmass reviewed Jan 17, 2025
View reviewed changes
Copy link
Member

@aabmass aabmass left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Vertex AI/Gemini also supports response schema. I'll take a look and see if it works with this PR

karthikscale3
karthikscale3 approved these changes Jan 17, 2025
View reviewed changes
Copy link
Contributor

@karthikscale3 karthikscale3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

trentm
trentm approved these changes Jan 17, 2025
View reviewed changes
aabmass
aabmass reviewed Jan 17, 2025
View reviewed changes
@aabmass aabmass mentioned this pull request Jan 17, 2025
Open
@lmolkova lmolkova changed the title Rename gen-ai.openai.request.response_format to gen-ai.request.response_format Rename gen_ai.openai.request.response_format to gen_ai.request.response_format Jan 18, 2025
@lmolkova lmolkova mentioned this pull request Jan 18, 2025
Merged
3 tasks
@lmolkova lmolkova force-pushed the gen-ai-response-format-generic branch from 5e574a5 to 07f27d5 Compare January 23, 2025 21:51
@lmolkova lmolkova requested a review from TaoChenOSU January 23, 2025 21:52
trentm
trentm reviewed Jan 24, 2025
View reviewed changes
@lmolkova lmolkova changed the title Rename gen_ai.openai.request.response_format to gen_ai.request.response_format Introduce gen_ai.output.type and gen_ai.output.json.schema.name to record response type and format. Deprecate gen_ai.openai.request.response_format Jan 27, 2025
@lmolkova lmolkova changed the title Introduce gen_ai.output.type and gen_ai.output.json.schema.name to record response type and format. Deprecate gen_ai.openai.request.response_format Introduce gen_ai.output.type and gen_ai.output.json.schema.name, deprecate gen_ai.openai.request.response_format Jan 27, 2025
aabmass
aabmass reviewed Jan 28, 2025
View reviewed changes
drewby
drewby approved these changes Jan 28, 2025
View reviewed changes
Copy link
Member

@drewby drewby left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One fix, and a couple questions. Otherwise LGTM.

aabmass
aabmass approved these changes Jan 29, 2025
View reviewed changes
@lmolkova lmolkova changed the title Introduce gen_ai.output.type and gen_ai.output.json.schema.name, deprecate gen_ai.openai.request.response_format Introduce generic gen_ai.output.type instead of gen_ai.openai.request.response_format Jan 31, 2025
trask
trask approved these changes Feb 1, 2025
View reviewed changes
@lmolkova lmolkova merged commit dcb30bf into open-telemetry:main Feb 3, 2025
15 checks passed
@MrAlias MrAlias mentioned this pull request Mar 20, 2025
Merged
pellared added a commit to open-telemetry/opentelemetry-go that referenced this pull request Mar 26, 2025
@MrAlias @pellared
Generate v1.31.0 semconv (#6479)
95e5bba 
- Generate `semconv/v1.31.0`
- Stop generating deprecated metric semconv similar to all other
generation
- Fix acronyms:
  - `ReplicationController`
  - `ResourceQuota`

## [`v1.31.0` semantic convention release
notes](https://github.com/open-telemetry/semantic-conventions/releases/tag/v1.31.0):

<h3>🛑 Breaking changes 🛑</h3>
<ul>
<li>
<p><code>code</code>: <code>code.function.name</code> value should
contain the fully qualified function name, <code>code.namespace</code>
is now deprecated (<a
href="open-telemetry/semantic-conventions#1677"
data-hovercard-type="issue"
data-hovercard-url="/open-telemetry/semantic-conventions/issues/1677/hovercard">#1677</a>)</p>
</li>
<li>
<p><code>gen-ai</code>: Introduce <code>gen_ai.output.type</code>and
deprecate <code>gen_ai.openai.request.response_format</code> (<a
href="open-telemetry/semantic-conventions#1757"
data-hovercard-type="pull_request"
data-hovercard-url="/open-telemetry/semantic-conventions/pull/1757/hovercard">#1757</a>)</p>
</li>
<li>
<p><code>mobile</code>: Rework <code>device.app.lifecycle</code> mobile
event. (<a
href="open-telemetry/semantic-conventions#1880"
data-hovercard-type="pull_request"
data-hovercard-url="/open-telemetry/semantic-conventions/pull/1880/hovercard">#1880</a>)<br>
The <code>device.app.lifecycle</code> event has been reworked to use
attributes instead<br>
of event body fields. The <code>ios.app.state</code> and
<code>android.app.state</code> attributes<br>
have been reintroduced to the attribute registry.</p>
</li>
<li>
<p><code>system</code>: Move CPU-related system.cpu.* metrics to CPU
namespace (<a
href="open-telemetry/semantic-conventions#1873"
data-hovercard-type="issue"
data-hovercard-url="/open-telemetry/semantic-conventions/issues/1873/hovercard">#1873</a>)</p>
</li>
<li>
<p><code>k8s</code>: Change k8s.replication_controller metrics to
k8s.replicationcontroller (<a
href="open-telemetry/semantic-conventions#1848"
data-hovercard-type="pull_request"
data-hovercard-url="/open-telemetry/semantic-conventions/pull/1848/hovercard">#1848</a>)</p>
</li>
<li>
<p><code>db</code>: Rename <code>db.system</code> to
<code>db.system.name</code> in database metrics, and update the values
to be consistent with database spans. (<a
href="open-telemetry/semantic-conventions#1581"
data-hovercard-type="issue"
data-hovercard-url="/open-telemetry/semantic-conventions/issues/1581/hovercard">#1581</a>)</p>
</li>
<li>
<p><code>session</code>: Move <code>session.id</code> and
<code>session.previous_id</code> from body fields to event attributes,
and yamlize <code>session.start</code> and <code>session.end</code>
events. (<a
href="open-telemetry/semantic-conventions#1845"
data-hovercard-type="pull_request"
data-hovercard-url="/open-telemetry/semantic-conventions/pull/1845/hovercard">#1845</a>)<br>
As part of the ongoing migration of event fields from LogRecord body to
extended/complex attributes, the <code>session.start</code> and
<code>session.end</code> events have been redefined.</p>
</li>
</ul>
<h3>💡 Enhancements 💡</h3>
<ul>
<li>
<p><code>code</code>: Mark <code>code.*</code> semantic conventions as
release candidate (<a
href="open-telemetry/semantic-conventions#1377"
data-hovercard-type="issue"
data-hovercard-url="/open-telemetry/semantic-conventions/issues/1377/hovercard">#1377</a>)</p>
</li>
<li>
<p><code>gen-ai</code>: Added AI Agent Semantic Convention (<a
href="open-telemetry/semantic-conventions#1732"
data-hovercard-type="issue"
data-hovercard-url="/open-telemetry/semantic-conventions/issues/1732/hovercard">#1732</a>,
<a
href="open-telemetry/semantic-conventions#1739"
data-hovercard-type="pull_request"
data-hovercard-url="/open-telemetry/semantic-conventions/pull/1739/hovercard">#1739</a>)</p>
</li>
<li>
<p><code>db</code>: Add database-specific notes on db.operation.name and
db.collection.name for Cassandra, Cosmos DB, HBase, MongoDB, and Redis,
covering their batch/bulk terms and lack of cross-table queries. (<a
href="open-telemetry/semantic-conventions#1863"
data-hovercard-type="pull_request"
data-hovercard-url="/open-telemetry/semantic-conventions/pull/1863/hovercard">#1863</a>,
<a
href="open-telemetry/semantic-conventions#1573"
data-hovercard-type="issue"
data-hovercard-url="/open-telemetry/semantic-conventions/issues/1573/hovercard">#1573</a>)</p>
</li>
<li>
<p><code>gen-ai</code>: Adds <code>gen_ai.request.choice.count</code>
span attribute (<a
href="open-telemetry/semantic-conventions#1888"
data-hovercard-type="issue"
data-hovercard-url="/open-telemetry/semantic-conventions/issues/1888/hovercard">#1888</a>)<br>
Enables recording target number of completions to generate</p>
</li>
<li>
<p><code>enduser</code>: Undeprecate 'enduser.id' and introduce new
attribute <code>enduser.pseudo.id</code> (<a
href="open-telemetry/semantic-conventions#1104"
data-hovercard-type="issue"
data-hovercard-url="/open-telemetry/semantic-conventions/issues/1104/hovercard">#1104</a>)<br>
The new attribute <code>enduser.pseudo.id</code> is intended to provide
a unique identifier of a pseudonymous enduser.</p>
</li>
<li>
<p><code>k8s</code>: Add <code>k8s.hpa</code>,
<code>k8s.resourcequota</code> and
<code>k8s.replicationcontroller</code> attributes and resources (<a
href="open-telemetry/semantic-conventions#1656"
data-hovercard-type="issue"
data-hovercard-url="/open-telemetry/semantic-conventions/issues/1656/hovercard">#1656</a>)</p>
</li>
<li>
<p><code>k8s</code>: How to populate resource attributes based on
attributes, labels and transformation (<a
href="open-telemetry/semantic-conventions#236"
data-hovercard-type="issue"
data-hovercard-url="/open-telemetry/semantic-conventions/issues/236/hovercard">#236</a>)</p>
</li>
<li>
<p><code>process</code>: Adjust the semantic expectations for
<code>process.executable.name</code> (<a
href="open-telemetry/semantic-conventions#1736"
data-hovercard-type="issue"
data-hovercard-url="/open-telemetry/semantic-conventions/issues/1736/hovercard">#1736</a>)</p>
</li>
<li>
<p><code>otel</code>: Adds SDK self-monitoring metrics for span
processing (<a
href="open-telemetry/semantic-conventions#1631"
data-hovercard-type="pull_request"
data-hovercard-url="/open-telemetry/semantic-conventions/pull/1631/hovercard">#1631</a>)</p>
</li>
<li>
<p><code>cicd</code>: Adds a new attribute
<code>cicd.pipeline.run.url.full</code> and corrects the attribute
description of <code>cicd.pipeline.task.run.url.full</code> (<a
href="open-telemetry/semantic-conventions#1796"
data-hovercard-type="issue"
data-hovercard-url="/open-telemetry/semantic-conventions/issues/1796/hovercard">#1796</a>)</p>
</li>
<li>
<p><code>user-agent</code>: Add <code>user_agent.os.name</code> and
<code>user_agent.os.version</code> attributes (<a
href="open-telemetry/semantic-conventions#1433"
data-hovercard-type="issue"
data-hovercard-url="/open-telemetry/semantic-conventions/issues/1433/hovercard">#1433</a>)</p>
</li>
</ul>
<h3>🧰 Bug fixes 🧰</h3>
<ul>
<li><code>process</code>: Fix units of
process.open_file_descriptor.count and process.context_switches (<a
href="open-telemetry/semantic-conventions#1662"
data-hovercard-type="issue"
data-hovercard-url="/open-telemetry/semantic-conventions/issues/1662/hovercard">#1662</a>)</li>
</ul>

---------

Co-authored-by: Robert Pająk <pellared@hotmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@trentm trentm trentm approved these changes

@drewby drewby drewby approved these changes

@trask trask trask approved these changes

@aabmass aabmass aabmass approved these changes

@gyliu513 gyliu513 gyliu513 approved these changes

@karthikscale3 karthikscale3 Awaiting requested review from karthikscale3

@TaoChenOSU TaoChenOSU Awaiting requested review from TaoChenOSU

Assignees
No one assigned
Labels
area:gen-ai breaking
Projects
Semantic Conventions Triage
Archived in project
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
8 participants
@lmolkova @trentm @drewby @trask @aabmass @gyliu513 @TaoChenOSU @karthikscale3