Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add reasoning model semantic conventions. #1965

Open
Cirilla-zmh opened this issue Mar 6, 2025 · 2 comments
Open

Add reasoning model semantic conventions. #1965

Cirilla-zmh opened this issue Mar 6, 2025 · 2 comments

Comments

@Cirilla-zmh
Copy link
Member

Area(s)

area:gen-ai

What's missing?

Source semconv: https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-events/#event-gen_aichoice

Since the release of the DeepSeek R1 model, an increasing number of model providers have started offering reasoner-type LLMs. Some of our users are paying attention to certain new elements emerging in LLM invocations, such as reasoning content.

Of course, for different vendors, there are still many different detailed context. For instance, whether reasoning tokens should be counted separately, or whether the "reasoning effort" attribute should be recorded. I believe it may not be the right time to discuss these topics yet. However, when it comes to reasoning content, the answer is clear — this parameter is highly valuable for aiding in evaluations related to chain-of-thought reasoning.

Reference:

  1. https://api-docs.deepseek.com/guides/reasoning_model
  2. https://platform.openai.com/docs/guides/reasoning
  3. https://docs.dify.ai/learn-more/use-cases/integrate-deepseek-to-build-an-ai-app

Describe the solution you'd like

Add the capture of message.reasoning_content in gen_ai.choice in case of more and more llm sdks/orchestration frameworks are supporting for this field.

@michaelsafyan
Copy link
Contributor

Would this extension be needed if the proposal in #1913 were to be adopted? (Could this be just another kind of "part")?

@Cirilla-zmh
Copy link
Member Author

Would this extension be needed if the proposal in #1913 were to be adopted? (Could this be just another kind of "part")?

Of course! Reasoning contents or another attributes are part of gen_ai.message. It makes sense for me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

No branches or pull requests

2 participants