Skip to content

Commit

Permalink
Merge branch 'v1.15' into issue_4487
Browse files Browse the repository at this point in the history
  • Loading branch information
hhunter-ms authored Feb 3, 2025
2 parents 3a923f4 + ee5e9fe commit bf9caca
Show file tree
Hide file tree
Showing 42 changed files with 1,003 additions and 560 deletions.
8 changes: 6 additions & 2 deletions daprdocs/config.toml
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ id = "G-60C6Q1ETC1"
lang = "en"
[[module.mounts]]
source = "../sdkdocs/rust/daprdocs/content/en/rust-sdk-contributing"
target = "content/contributing/sdks-contrib"
target = "content/contributing/sdk-contrib/"
lang = "en"

[[module.mounts]]
Expand Down Expand Up @@ -143,7 +143,11 @@ id = "G-60C6Q1ETC1"
[[module.mounts]]
source = "../translations/docs-zh/translated_content/zh_CN/sdks_js"
target = "content/developing-applications/sdks/js"
lang = "zh-hans"
lang = "zh-hans"
[[module.mounts]]
source = "../translations/docs-zh/translated_content/zh_CN/sdks_rust"
target = "content/developing-applications/sdks/rust"
lang = "zh-hans"
[[module.mounts]]
source = "../translations/docs-zh/translated_content/zh_CN/pluggable-components/dotnet"
target = "content/developing-applications/develop-components/pluggable-components/pluggable-components-sdks/pluggable-components-dotnet"
Expand Down
2 changes: 1 addition & 1 deletion daprdocs/content/en/concepts/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ Dapr exposes its HTTP and gRPC APIs as a sidecar architecture, either as a conta
## Hosting environments

Dapr can be hosted in multiple environments, including:
- Self-hosted on a Windows/Linux/macOS machine for local development
- Self-hosted on a Windows/Linux/macOS machine for local development and in production
- On Kubernetes or clusters of physical or virtual machines in production

### Self-hosted local development
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ This simplifies some choices, but also carries some consideration:

## Actor communication

You can interact with Dapr to invoke the actor method by calling HTTP/gRPC endpoint.
You can interact with Dapr to invoke the actor method by calling the HTTP endpoint.

```bash
POST/GET/PUT/DELETE http://localhost:3500/v1.0/actors/<actorType>/<actorId>/<method/state/timers/reminders>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,10 @@ You can remove the actor reminder by calling
DELETE http://localhost:3500/v1.0/actors/<actorType>/<actorId>/reminders/<name>
```

If an actor reminder is triggered and the app does not return a 2** code to the runtime (for example, because of a connection issue),
actor reminders will be retried up to three times with a backoff interval of one second between each attempt. There may be
additional retries attempted in accordance with any optionally applied [actor resiliency policy]({{< ref "override-default-retries.md" >}}).

Refer [api spec]({{< ref "actors_api.md#invoke-reminder" >}}) for more details.

## Error handling
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,10 @@ description: "Overview of the conversation API building block"
The conversation API is currently in [alpha]({{< ref "certification-lifecycle.md#certification-levels" >}}).
{{% /alert %}}

Using the Dapr conversation API, you can reduce the complexity of interacting with Large Language Models (LLMs) and enable critical performance and security functionality with features like prompt caching and personally identifiable information (PII) data obfuscation.
Dapr's conversation API reduces the complexity of securely and reliably interacting with Large Language Models (LLM) at scale. Whether you're a developer who doesn't have the necessary native SDKs or a polyglot shop who just wants to focus on the prompt aspects of LLM interactions, the conversation API provides one consistent API entry point to talk to underlying LLM providers.

<img src="/images/conversation-overview.png" width=800 alt="Diagram showing the flow of a user's app communicating with Dapr's LLM components.">

Dapr's conversation API reduces the complexity of securely and reliably interacting with Large Language Models (LLM) at scale. Whether you're a developer who doesn't have the necessary native SDKs or a polyglot shop who just wants to focus on the prompt aspects of LLM interactions, the conversation API provides one consistent API entry point to talk to underlying LLM providers.

In additon to enabling critical performance and security functionality (like [prompt caching]({{< ref "#prompt-caching" >}}) and [PII scrubbing]({{< ref "#personally-identifiable-information-pii-obfuscation" >}})), you can also pair the conversation API with Dapr functionalities, like:
- Resiliency circuit breakers and retries to circumvent limit and token errors, or
- Middleware to authenticate requests coming to and from the LLM
Expand Down Expand Up @@ -57,4 +55,4 @@ Want to skip the quickstarts? Not a problem. You can try out the conversation bu
## Next steps

- [How-To: Converse with an LLM using the conversation API]({{< ref howto-conversation-layer.md >}})
- [Conversation API components]({{< ref supported-conversation >}})
- [Conversation API components]({{< ref supported-conversation >}})
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
type: docs
title: "How-To: Schedule and handle triggered jobs"
linkTitle: "How-To: Schedule and handle triggered jobs"
weight: 2000
weight: 5000
description: "Learn how to use the jobs API to schedule and handle triggered jobs"
---

Expand All @@ -20,7 +20,103 @@ When you [run `dapr init` in either self-hosted mode or on Kubernetes]({{< ref i

In your code, set up and schedule jobs within your application.

{{< tabs "Go" >}}
{{< tabs ".NET" "Go" >}}

{{% codetab %}}

<!-- .NET -->

The following .NET SDK code sample schedules the job named `prod-db-backup`. The job data contains information
about the database that you'll be seeking to backup regularly. Over the course of this example, you'll:
- Define types used in the rest of the example
- Register an endpoint during application startup that handles all job trigger invocations on the service
- Register the job with Dapr

In the following example, you'll create records that you'll serialize and register alongside the job so the information
is available when the job is triggered in the future:
- The name of the backup task (`db-backup`)
- The backup task's `Metadata`, including:
- The database name (`DBName`)
- The database location (`BackupLocation`)

Create an ASP.NET Core project and add the latest version of `Dapr.Jobs` from NuGet.

> **Note:** While it's not strictly necessary
for your project to use the `Microsoft.NET.Sdk.Web` SDK to create jobs, as of the time this documentation is authored,
only the service that schedules a job receives trigger invocations for it. As those invocations expect an endpoint
that can handle the job trigger and requires the `Microsoft.NET.Sdk.Web` SDK, it's recommended that you
use an ASP.NET Core project for this purpose.

Start by defining types to persist our backup job data and apply our own JSON property name attributes to the properties
so they're consistent with other language examples.

```cs
//Define the types that we'll represent the job data with
internal sealed record BackupJobData([property: JsonPropertyName("task")] string Task, [property: JsonPropertyName("metadata")] BackupMetadata Metadata);
internal sealed record BackupMetadata([property: JsonPropertyName("DBName")]string DatabaseName, [property: JsonPropertyName("BackupLocation")] string BackupLocation);
```

Next, set up a handler as part of your application setup that will be called anytime a job is triggered on your
application. It's the responsibility of this handler to identify how jobs should be processed based on the job name provided.

This works by registering a handler with ASP.NET Core at `/job/<job-name>`, where `<job-name>` is parameterized and
passed into this handler delegate, meeting Dapr's expectation that an endpoint is available to handle triggered named jobs.

Populate your `Program.cs` file with the following:

```cs
using System.Text;
using System.Text.Json;
using Dapr.Jobs;
using Dapr.Jobs.Extensions;
using Dapr.Jobs.Models;
using Dapr.Jobs.Models.Responses;

var builder = WebApplication.CreateBuilder(args);
builder.Services.AddDaprJobsClient();
var app = builder.Build();

//Registers an endpoint to receive and process triggered jobs
var cancellationTokenSource = new CancellationTokenSource(TimeSpan.FromSeconds(5));
app.MapDaprScheduledJobHandler((string jobName, DaprJobDetails jobDetails, ILogger logger, CancellationToken cancellationToken) => {
logger?.LogInformation("Received trigger invocation for job '{jobName}'", jobName);
switch (jobName)
{
case "prod-db-backup":
// Deserialize the job payload metadata
var jobData = JsonSerializer.Deserialize<BackupJobData>(jobDetails.Payload);

// Process the backup operation - we assume this is implemented elsewhere in your code
await BackupDatabaseAsync(jobData, cancellationToken);
break;
}
}, cancellationTokenSource.Token);

await app.RunAsync();
```

Finally, the job itself needs to be registered with Dapr so it can be triggered at a later point in time. You can do this
by injecting a `DaprJobsClient` into a class and executing as part of an inbound operation to your application, but for
this example's purposes, it'll go at the bottom of the `Program.cs` file you started above. Because you'll be using the
`DaprJobsClient` you registered with dependency injection, start by creating a scope so you can access it.

```cs
//Create a scope so we can access the registered DaprJobsClient
await using scope = app.Services.CreateAsyncScope();
var daprJobsClient = scope.ServiceProvider.GetRequiredService<DaprJobsClient>();

//Create the payload we wish to present alongside our future job triggers
var jobData = new BackupJobData("db-backup", new BackupMetadata("my-prod-db", "/backup-dir"));

//Serialize our payload to UTF-8 bytes
var serializedJobData = JsonSerializer.SerializeToUtf8Bytes(jobData);

//Schedule our backup job to run every minute, but only repeat 10 times
await daprJobsClient.ScheduleJobAsync("prod-db-backup", DaprJobSchedule.FromDuration(TimeSpan.FromMinutes(1)),
serializedJobData, repeats: 10);
```

{{% /codetab %}}

{{% codetab %}}

Expand Down Expand Up @@ -92,66 +188,8 @@ In this example, at trigger time, which is `@every 1s` according to the `Schedul
}
```

At the trigger time, the `prodDBBackupHandler` function is called, executing the desired business logic for this job at trigger time. For example:

#### HTTP

When you create a job using Dapr's Jobs API, Dapr will automatically assume there is an endpoint available at
`/job/<job-name>`. For instance, if you schedule a job named `test`, Dapr expects your application to listen for job
events at `/job/test`. Ensure your application has a handler set up for this endpoint to process the job when it is
triggered. For example:

*Note: The following example is in Go but applies to any programming language.*

```go

func main() {
...
http.HandleFunc("/job/", handleJob)
http.HandleFunc("/job/<job-name>", specificJob)
...
}

func specificJob(w http.ResponseWriter, r *http.Request) {
// Handle specific triggered job
}

func handleJob(w http.ResponseWriter, r *http.Request) {
// Handle the triggered jobs
}
```

#### gRPC

When a job reaches its scheduled trigger time, the triggered job is sent back to the application via the following
callback function:

*Note: The following example is in Go but applies to any programming language with gRPC support.*

```go
import rtv1 "github.com/dapr/dapr/pkg/proto/runtime/v1"
...
func (s *JobService) OnJobEventAlpha1(ctx context.Context, in *rtv1.JobEventRequest) (*rtv1.JobEventResponse, error) {
// Handle the triggered job
}
```

This function processes the triggered jobs within the context of your gRPC server. When you set up the server, ensure that
you register the callback server, which will invoke this function when a job is triggered:

```go
...
js := &JobService{}
rtv1.RegisterAppCallbackAlphaServer(server, js)
```

In this setup, you have full control over how triggered jobs are received and processed, as they are routed directly
through this gRPC method.

#### SDKs

For SDK users, handling triggered jobs is simpler. When a job is triggered, Dapr will automatically route the job to the
event handler you set up during the server initialization. For example, in Go, you'd register the event handler like this:
When a job is triggered, Dapr will automatically route the job to the event handler you set up during the server
initialization. For example, in Go, you'd register the event handler like this:

```go
...
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
---
type: docs
title: "Features and concepts"
linkTitle: "Features and concepts"
weight: 2000
description: "Learn more about the Dapr Jobs features and concepts"
---

Now that you've learned about the [jobs building block]({{< ref jobs-overview.md >}}) at a high level, let's deep dive
into the features and concepts included with Dapr Jobs and the various SDKs. Dapr Jobs:
- Provides a robust and scalable API for scheduling operations to be triggered in the future.
- Exposes several capabilities which are common across all supported languages.



## Job identity

All jobs are registered with a case-sensitive job name. These names are intended to be unique across all services
interfacing with the Dapr runtime. The name is used as an identifier when creating and modifying the job as well as
to indicate which job a triggered invocation is associated with.

Only one job can be associated with a name at any given time. Any attempt to create a new job using the same name
as an existing job will result in an overwrite of this existing job.

## Scheduling Jobs
A job can be scheduled using any of the following mechanisms:
- Intervals using Cron expressions, duration values, or period expressions
- Specific dates and times

For all time-based schedules, if a timestamp is provided with a time zone via the RFC3339 specification, that
time zone is used. When not provided, the time zone used by the server running Dapr is used.
In other words, do **not** assume that times run in UTC time zone, unless otherwise specified when scheduling
the job.

### Schedule using a Cron expression
When scheduling a job to execute on a specific interval using a Cron expression, the expression is written using 6
fields spanning the values specified in the table below:

| seconds | minutes | hours | day of month | month | day of week |
| -- | -- | -- | -- | -- | -- |
| 0-59 | 0-59 | 0-23 | 1-31 | 1-12/jan-dec | 0-6/sun-sat |

#### Example 1
`"0 30 * * * *"` triggers every hour on the half-hour mark.

#### Example 2
`"0 15 3 * * *"` triggers every day at 03:15.

### Schedule using a duration value
You can schedule jobs using [a Go duration string](https://pkg.go.dev/time#ParseDuration), in which
a string consists of a (possibly) signed sequence of decimal numbers, each with an optional fraction and a unit suffix.
Valid time units are `"ns"`, `"us"`, `"ms"`, `"s"`, `"m"`, or `"h"`.

#### Example 1
`"2h45m"` triggers every 2 hours and 45 minutes.

#### Example 2
`"37m25s"` triggers every 37 minutes and 25 seconds.

### Schedule using a period expression
The following period expressions are supported. The "@every" expression also accepts a [Go duration string](https://pkg.go.dev/time#ParseDuration).

| Entry | Description | Equivalent Cron expression |
| -- | -- | -- |
| @every | Run every (e.g. "@every 1h30m") | N/A |
| @yearly (or @annually) | Run once a year, midnight, January 1st | 0 0 0 1 1 * |
| @monthly | Run once a month, midnight, first of month | 0 0 0 1 * * |
| @weekly | Run once a week, midnight on Sunday | 0 0 0 * * 0 |
| @daily or @midnight | Run once a day at midnight | 0 0 0 * * * |
| @hourly | Run once an hour at the beginning of the hour | 0 0 * * * * |

### Schedule using a specific date/time
A job can also be scheduled to run at a particular point in time by providing a date using the
[RFC3339 specification](https://www.rfc-editor.org/rfc/rfc3339).

#### Example 1
`"2025-12-09T16:09:53+00:00"` Indicates that the job should be run on December 9, 2025 at 4:09:53 PM UTC.

## Scheduled triggers
When a scheduled Dapr job is triggered, the runtime sends a message back to the service that scheduled the job using
either the HTTP or gRPC approach, depending on which is registered with Dapr when the service starts.

### gRPC
When a job reaches its scheduled trigger time, the triggered job is sent back to the application via the following
callback function:

> **Note:** The following example is in Go, but applies to any programming language with gRPC support.
```go
import rtv1 "github.com/dapr/dapr/pkg/proto/runtime/v1"
...
func (s *JobService) OnJobEventAlpha1(ctx context.Context, in *rtv1.JobEventRequest) (*rtv1.JobEventResponse, error) {
// Handle the triggered job
}
```

This function processes the triggered jobs within the context of your gRPC server. When you set up the server, ensure that
you register the callback server, which invokes this function when a job is triggered:

```go
...
js := &JobService{}
rtv1.RegisterAppCallbackAlphaServer(server, js)
```

In this setup, you have full control over how triggered jobs are received and processed, as they are routed directly
through this gRPC method.

### HTTP
If a gRPC server isn't registered with Dapr when the application starts up, Dapr instead triggers jobs by making a
POST request to the endpoint `/job/<job-name>`. The body includes the following information about the job:
- `Schedule`: When the job triggers occur
- `RepeatCount`: An optional value indicating how often the job should repeat
- `DueTime`: An optional point in time representing either the one time when the job should execute (if not recurring)
or the not-before time from which the schedule should take effect
- `Ttl`: An optional value indicating when the job should expire
- `Payload`: A collection of bytes containing data originally stored when the job was scheduled

The `DueTime` and `Ttl` fields will reflect an RC3339 timestamp value reflective of the time zone provided when the job was
originally scheduled. If no time zone was provided, these values indicate the time zone used by the server running
Dapr.
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ app.get('/dapr/subscribe', (_req, res) => {
## Retries and dead letter topics

By default, when a dead letter topic is set, any failing message immediately goes to the dead letter topic. As a result it is recommend to always have a retry policy set when using dead letter topics in a subscription.
To enable the retry of a message before sending it to the dead letter topic, apply a [retry resiliency policy]({{< ref "policies.md#retries" >}}) to the pub/sub component.
To enable the retry of a message before sending it to the dead letter topic, apply a [retry resiliency policy]({{< ref "retries-overview.md" >}}) to the pub/sub component.

This example shows how to set a constant retry policy named `pubsubRetry`, with 10 maximum delivery attempts applied every 5 seconds for the `pubsub` pub/sub component.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -309,6 +309,8 @@ context.AddMetadata("dapr-stream", "true");

### Streaming gRPCs and Resiliency

> Currently, resiliency policies are not supported for service invocation via gRPC.
When proxying streaming gRPCs, due to their long-lived nature, [resiliency]({{< ref "resiliency-overview.md" >}}) policies are applied on the "initial handshake" only. As a consequence:

- If the stream is interrupted after the initial handshake, it will not be automatically re-established by Dapr. Your application will be notified that the stream has ended, and will need to recreate it.
Expand Down
Loading

0 comments on commit bf9caca

Please sign in to comment.