Skip to content

Commit 5a3491e

Browse files
avillelaavillela
authoredDec 9, 2024
Update example to support Dynatrace back-end (#13)
* Add Jaeger to test trace visualization * Add dynatrace configs * Cleanup * Cleaning up scripts * Cleanup
1 parent bc014e1 commit 5a3491e

11 files changed

+443
-35
lines changed
 

‎.devcontainer/devcontainer.json

+9-6
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,20 @@
11
{
22
"name": "default",
3-
"image": "mcr.microsoft.com/devcontainers/base:ubuntu",
3+
"image": "mcr.microsoft.com/devcontainers/python:dev-3.11-bullseye",
44
"features": {
55
"ghcr.io/devcontainers/features/docker-in-docker:2": {
66
"version": "latest"
77
},
8-
"ghcr.io/devcontainers/features/python:1": {
9-
"version": "3.12.1",
8+
"ghcr.io/devcontainers/features/go:1": {
9+
"version": "1.23.3",
1010
},
1111
},
1212
"overrideFeatureInstallOrder": [
1313
"ghcr.io/devcontainers/features/python",
1414
"ghcr.io/devcontainers/features/docker-in-docker",
15-
]
16-
}
17-
15+
],
16+
"remoteEnv": {
17+
"PODMAN_USERNS": "keep-id"
18+
},
19+
"containerUser": "vscode"
20+
}

‎README.md

+49-3
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,52 @@ Now you are ready to deploy the Kubernetes resources
6262
./src/scripts/04-deploy-resources.sh
6363
```
6464

65-
### 3b- Kubernetes deployment (ServiceNow Cloud Observability backend)
65+
### 3b- Kubernetes deployment with Dynatrace backend
66+
67+
> 🚨 If you want to send telemetry to [Dynatrace)](https://www.dynatrace.com), you'll need to follow the steps below, and skip [Step 3a](#3a---kubernetes-deployment-collector-stdout-only).
68+
69+
To send telemetry to Dynatrace, you will first need a [Dynatrace free trial account](https://www.dynatrace.com/signup/). You will also need to obtain an [access token](https://docs.dynatrace.com/docs/analyze-explore-automate/dynatrace-for-ai-observability/machine-learning-platforms/bedrock#create-a-dynatrace-token).
70+
71+
We're going to store the access token in a Kubernetes secret, and will map the secret to an environment variabe in the [`OpenTelemetryCollector CR`](https://github.com/avillela/otel-target-allocator-talk/main/src/resources/02-otel-collector-dt.yml#L17-L21).
72+
73+
First, create a secrets file for the Lightstep token.
74+
75+
```bash
76+
tee -a src/resources/00-secret-dt.yaml <<EOF
77+
apiVersion: v1
78+
kind: Secret
79+
metadata:
80+
name: otel-collector-secret
81+
namespace: opentelemetry
82+
data:
83+
DT_TOKEN: <base64-encoded-dynatrace-token>
84+
DT_ENV_ID: <base64-encoded-dynatrace-environment-identifier>
85+
type: "Opaque"
86+
EOF
87+
```
88+
89+
Replace `<base64-encoded-LS-token>` with your own [access token] (https://docs.dynatrace.com/docs/discover-dynatrace/references/dynatrace-api/basics/dynatrace-api-authentication#create-token). Make sure you grant the correct permissions to your token so that you Dynatrace can ingest traces (`opentelemetryTrace.ingest`), logs (`logs.ingest`), and metrics (`metrics.ingest`). See [Dynatrace Token Scopes](https://docs.dynatrace.com/docs/discover-dynatrace/references/dynatrace-api/basics/dynatrace-api-authentication#token-scopes).
90+
91+
Replace `<base64-encoded-dynatrace-environment-identifier>` with your own environment ID. Your Dynatrace tenant will have a URL like this: `https://<dynatrace-environment-identifier>.apps.dynatrace.com`. The part before between `https://` and `apps.dynatrace.com` is what your environment identifier. Make sure you base64 encode that value before adding it to your `secrets` YAML.
92+
93+
Be sure to Base64 encode it like this:
94+
95+
```bash
96+
echo <LS-access-token> | base64
97+
```
98+
99+
Or you can Base64-encode it through [this website](https://www.base64encode.org/).
100+
101+
Finally, deploy the Kubernetes resources:
102+
103+
```bash
104+
# Don't forget to pass the "dt" argument
105+
./src/scripts/04-deploy-resources.sh dt
106+
```
107+
108+
> For more on OTel Collector configuration with Dynatrace, check out [this doc](https://docs.dynatrace.com/docs/ingest-from/opentelemetry/collector/configuration).
109+
110+
### 3c- Kubernetes deployment with ServiceNow Cloud Observability backend
66111

67112
> 🚨 If you want to send telemetry to [ServiceNow Cloud Observability (formerly known as Lightstep)](https://www.servicenow.com/products/observability.html), you'll need to follow the steps below, and skip [Step 3a](#3a---kubernetes-deployment-collector-stdout-only).
68113
@@ -73,7 +118,7 @@ We're going to store the access token in a Kubernetes secret, and will map the s
73118
First, create a secrets file for the Lightstep token.
74119

75120
```bash
76-
tee -a src/resources/00-secret.yaml <<EOF
121+
tee -a src/resources/00-secret-ls.yaml <<EOF
77122
apiVersion: v1
78123
kind: Secret
79124
metadata:
@@ -98,7 +143,8 @@ Or you can Base64-encode it through [this website](https://www.base64encode.org/
98143
Finally, deploy the Kubernetes resources:
99144

100145
```bash
101-
./src/scripts/04-deploy-resources-ls-backend.sh
146+
# Don't forget to pass the "ls" argument
147+
./src/scripts/04-deploy-resources.sh ls
102148
```
103149

104150
## 4- Check logs

‎src/resources/02-otel-collector-dt.yml

+76
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
# API reference https://github.com/open-telemetry/opentelemetry-operator/blob/main/docs/api.md
2+
# Refs for v1beta1 config: https://github.com/open-telemetry/opentelemetry-operator/issues/3011#issuecomment-2154118998
3+
apiVersion: opentelemetry.io/v1beta1
4+
kind: OpenTelemetryCollector
5+
metadata:
6+
name: otelcol
7+
namespace: opentelemetry
8+
spec:
9+
mode: statefulset
10+
# Using an older version of the DT Collector (based on an older Collector contrib version)
11+
# where Python auto-instrumentation and context propagation doesn't break
12+
image: ghcr.io/dynatrace/dynatrace-otel-collector/dynatrace-otel-collector:0.7.0
13+
targetAllocator:
14+
enabled: true
15+
serviceAccount: opentelemetry-targetallocator-sa
16+
prometheusCR:
17+
enabled: true
18+
podMonitorSelector: {}
19+
serviceMonitorSelector: {}
20+
## If uncommented, only service monitors with this label will get picked up
21+
# app: my-app
22+
env:
23+
- name: DT_TOKEN
24+
valueFrom:
25+
secretKeyRef:
26+
key: DT_TOKEN
27+
name: otel-collector-secret
28+
- name: DT_ENV_ID
29+
valueFrom:
30+
secretKeyRef:
31+
key: DT_ENV_ID
32+
name: otel-collector-secret
33+
config:
34+
receivers:
35+
otlp:
36+
protocols:
37+
grpc: {}
38+
http: {}
39+
prometheus:
40+
config:
41+
scrape_configs:
42+
- job_name: 'otel-collector'
43+
scrape_interval: 10s
44+
static_configs:
45+
- targets: [ '0.0.0.0:8888' ]
46+
47+
processors:
48+
cumulativetodelta: {}
49+
batch: {}
50+
51+
exporters:
52+
otlp:
53+
endpoint: 'otel-jaeger-collector:4317'
54+
tls:
55+
insecure: true
56+
otlphttp:
57+
endpoint: "https://${DT_ENV_ID}.live.dynatrace.com/api/v2/otlp"
58+
headers:
59+
Authorization: "Api-Token ${DT_TOKEN}"
60+
debug:
61+
verbosity: detailed
62+
63+
service:
64+
pipelines:
65+
traces:
66+
receivers: [otlp]
67+
processors: [batch]
68+
exporters: [otlp,otlphttp,debug]
69+
metrics:
70+
receivers: [otlp, prometheus]
71+
processors: [cumulativetodelta]
72+
exporters: [otlphttp,debug]
73+
logs:
74+
receivers: [otlp]
75+
processors: [batch]
76+
exporters: [otlphttp,debug]

‎src/resources/02-otel-collector.yml

+10-4
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,8 @@ metadata:
77
namespace: opentelemetry
88
spec:
99
mode: statefulset
10+
# Using an older version of the Collector contrib where Python auto-instrumentation
11+
# and context propagation doesn't break
1012
image: otel/opentelemetry-collector-contrib:0.102.1
1113
targetAllocator:
1214
enabled: true
@@ -35,20 +37,24 @@ spec:
3537
batch: {}
3638

3739
exporters:
38-
logging:
40+
otlp:
41+
endpoint: 'otel-jaeger-collector:4317'
42+
tls:
43+
insecure: true
44+
debug:
3945
verbosity: detailed
4046

4147
service:
4248
pipelines:
4349
traces:
4450
receivers: [otlp]
4551
processors: [batch]
46-
exporters: [logging]
52+
exporters: [otlp,debug]
4753
metrics:
4854
receivers: [otlp, prometheus]
4955
processors: []
50-
exporters: [logging]
56+
exporters: [debug]
5157
logs:
5258
receivers: [otlp]
5359
processors: [batch]
54-
exporters: [logging]
60+
exporters: [debug]

0 commit comments

Comments
 (0)
Please sign in to comment.