Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs(weave): Update info hierarchy #3819

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/docs/guides/core-types/evaluations.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Offline Batch Evaluation
# Evaluation Overview

Evaluation-driven development helps you reliably iterate on an application. The `Evaluation` class is designed to assess the performance of a `Model` on a given `Dataset` or set of examples using scoring functions.

Expand Down
2 changes: 1 addition & 1 deletion docs/docs/guides/tracking/redact-pii.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Redact PII from Traces
# Redacting PII

:::important
This feature is only available for Enterprise users, and is only accessible via the Python SDK.
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/guides/tracking/tracing.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import TracingCallsFilterImage from '@site/static/img/screenshots/calls_filter.p
import BasicCallImage from '@site/static/img/screenshots/basic_call.png';


# Calls
# Tracing Basics

<DesktopWindow
images={[
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/tutorial-tracing_2.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

# Track data flows and app metadata
# Tutorial: Track Application Logic

In the [Track LLM inputs & outputs](/quickstart) tutorial, the basics of tracking the inputs and outputs of your LLMs was covered.

Expand Down
2 changes: 1 addition & 1 deletion docs/docs/tutorial-weave_models.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

# App versioning
# Tutorial: App versioning

Tracking the [inputs, outputs, metadata](/quickstart) as well as [data flowing through your app](/tutorial-tracing_2) is critical to understanding the performance of your system. However **versioning your app over time** is also critical to understand how modifications to your code or app attributes change your outputs. Weave's `Model` class is how these changes can be tracked in Weave.

Expand Down
131 changes: 86 additions & 45 deletions docs/sidebars.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,63 +11,119 @@ const CATEGORY_SECTION_HEADER_MIXIN: SidebarItemCategoryBase = {
const sidebars: SidebarsConfig = {
documentationSidebar: [
{
label: "👋 Getting Started",
label: "👋 Introduction",
...CATEGORY_SECTION_HEADER_MIXIN,
items: [
"introduction",
{
type: "doc",
label: "Trace LLMs",
label: "Quickstart: Track LLM Calls",
id: "quickstart",
},
],
},
{
label: "🔄 Iteration",
...CATEGORY_SECTION_HEADER_MIXIN,

items: [
{
type: "doc",
label: "Trace Applications",
label: "Track Application Logic",
id: "tutorial-tracing_2",
},
"tutorial-weave_models",
{
type: "category",
collapsible: true,
collapsed: false,
label: "Tracing & Debugging",
items: [
{
type: "doc",
label: "Tracing Overview",
id: "guides/tracking/tracing",
},
"guides/tracking/costs",
{
type: "doc",
label: "Logging Media",
id: "guides/core-types/media",
},
"guides/tools/playground",
"guides/integrations/index",
{
type: "doc",
label: "Compare Traces",
id: "guides/tools/comparison",
},
]
},
{
type: "category",
collapsible: true,
collapsed: true,
label: "Version Control for Models & Prompts",
items: [
{
type: "doc",
label: "App Versioning",
id: "tutorial-weave_models",
},
"guides/core-types/models",
"guides/core-types/prompts",
"guides/tracking/objects",
"guides/tracking/ops",
]
},
],
},
{
label: "📊 Evaluation",
...CATEGORY_SECTION_HEADER_MIXIN,
items: [
{
type: "doc",
label: "Build an Evaluation",
label: "Build an Evaluation Pipeline",
id: "tutorial-eval",
},
{
type: "doc",
label: "Evaluate a RAG App",
id: "tutorial-rag",
},
{
type: "category",
collapsible: true,
collapsed: false,
label: "Evaluations",
items: [
"guides/core-types/evaluations",
"guides/core-types/datasets",
"guides/evaluation/scorers",
"guides/evaluation/builtin_scorers",
]
},
],
},
{
label: "🤖 Product Walkthrough",
label: "🚀 Productionization",
...CATEGORY_SECTION_HEADER_MIXIN,
items: [
{
type: "category",
collapsible: true,
collapsed: false,
label: "LLM Application Tracing",
link: { type: "doc", id: "guides/tracking/index" },
label: "Collect Feedback & Examples",
items: [
"guides/tracking/tracing",
"guides/tracking/ops",
"guides/tracking/objects",
"guides/tracking/feedback",
"guides/tracking/redact-pii",
"guides/tracking/faqs"
],
]
},
{
type: "category",
collapsible: true,
collapsed: false,
label: "Evaluation",
items: [
"guides/core-types/evaluations",
{
type: "category",
collapsible: true,
collapsed: false,
label: "Online Evaluation",
label: "Online Evaluation",
link: { type: "doc", id: "guides/evaluation/guardrails_and_monitors" },
items: [
{
Expand All @@ -83,30 +139,7 @@ const sidebars: SidebarsConfig = {
autoAddBaseUrl: true,
}
],
},
{
type: "category",
collapsible: true,
collapsed: false,
label: "Scoring",
items: [
"guides/evaluation/scorers",
"guides/evaluation/builtin_scorers",
"guides/evaluation/weave_local_scorers"
],
}
],
},
"guides/core-types/prompts",
"guides/core-types/models",
"guides/core-types/datasets",
"guides/tracking/feedback",
"guides/tracking/costs",
"guides/tools/comparison",
"guides/tools/playground",
"guides/core-types/media",
"guides/core-types/env-vars",
"guides/troubleshooting",
{
type: "category",
collapsible: true,
Expand Down Expand Up @@ -151,7 +184,6 @@ const sidebars: SidebarsConfig = {
collapsed: true,
label: "Frameworks",
items: [
,
"guides/integrations/langchain",
"guides/integrations/llamaindex",
"guides/integrations/dspy",
Expand All @@ -174,6 +206,15 @@ const sidebars: SidebarsConfig = {
}
],
},
{
label: "🛠️ Tools & Resources",
...CATEGORY_SECTION_HEADER_MIXIN,
items: [
"guides/core-types/env-vars",
"guides/troubleshooting",
"guides/tracking/faqs",
],
},
],
// TODO: add the actual ts-sdk sidebar
typescriptSdkSidebar: [
Expand Down