Skip to content

Commit 9741e2c

Browse files
authored
Prometheus federation example (signalfx#57)
* Add an example showing how to use the OpenTelemetry collector to collect metrics from a Prometheus server * Fix imports * Remove mention of corda, use app as app name * remove the sleep statement since we rely on the ticker
1 parent f3960d1 commit 9741e2c

File tree

10 files changed

+437
-0
lines changed

10 files changed

+437
-0
lines changed
+22
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
# Prometheus Federation Endpoint Example
2+
3+
This example showcases how the agent works with Splunk Enterprise and an existing Prometheus deployment.
4+
5+
The example runs as a Docker Compose deployment. The collector can be configured to send various metrics to Splunk Enterprise.
6+
7+
Splunk is configured to receive data from the OpenTelemetry Collector using the HTTP Event collector. To learn more about HEC, visit [our guide](https://dev.splunk.com/enterprise/docs/dataapps/httpeventcollector/).
8+
9+
To deploy the example, check out this git repository, open a terminal and in this directory type:
10+
```bash
11+
$> docker-compose up --build
12+
```
13+
14+
Splunk will become available on port 18000. You can login on [http://localhost:18000](http://localhost:18000) with `admin` and `changeme`.
15+
16+
Once logged in, visit the [analytics workspace](http://localhost:18000/en-US/app/search/analytics_workspace) to see which metrics are sent by the OpenTelemetry Collector.
17+
18+
Additionally, you can consult the [Prometheus UI](http://localhost:9090) to see the metric data collected from the sample go program.
19+
20+
# Diagram of the deployment
21+
22+
![Diagram](diagram.png)
8.03 KB
Loading
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
version: "3"
2+
services:
3+
# Sample Go application producing counter metrics.
4+
prom-counter:
5+
container_name: prom-counter
6+
build:
7+
context: prom-counter
8+
restart: always
9+
# The Prometheus server:
10+
prometheus:
11+
image: prom/prometheus
12+
container_name: prometheus
13+
ports:
14+
- 9090:9090
15+
volumes:
16+
- ./prometheus.yml:/etc/prometheus/prometheus.yml
17+
# Splunk Enterprise server:
18+
splunk:
19+
image: splunk/splunk:latest
20+
container_name: splunk
21+
environment:
22+
- SPLUNK_START_ARGS=--accept-license
23+
- SPLUNK_HEC_TOKEN=00000000-0000-0000-0000-0000000000000
24+
- SPLUNK_PASSWORD=changeme
25+
ports:
26+
- 18000:8000
27+
healthcheck:
28+
test: ['CMD', 'curl', '-f', 'http://localhost:8000']
29+
interval: 5s
30+
timeout: 5s
31+
retries: 20
32+
volumes:
33+
- ./splunk.yml:/tmp/defaults/default.yml
34+
- /opt/splunk/var
35+
- /opt/splunk/etc
36+
# OpenTelemetry Collector
37+
otelcollector:
38+
image: quay.io/signalfx/splunk-otel-collector:0.4.0
39+
container_name: otelcollector
40+
command: ["--config=/etc/otel-collector-config.yml", "--log-level=DEBUG"]
41+
volumes:
42+
- ./otel-collector-config.yml:/etc/otel-collector-config.yml
43+
depends_on:
44+
- splunk
45+
- prometheus
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
receivers:
2+
prometheus_simple:
3+
collection_interval: 10s
4+
# the federation endpoint:
5+
# Read more about it here: https://prometheus.io/docs/prometheus/latest/federation/
6+
# You can query the federation with PromQL, encoded as part of the query string.
7+
endpoint: prometheus:9090
8+
metrics_path: /federate
9+
params:
10+
match[]: '{job="counter"}'
11+
otlp:
12+
protocols:
13+
grpc:
14+
15+
exporters:
16+
splunk_hec/metrics:
17+
# Splunk HTTP Event Collector token.
18+
token: "00000000-0000-0000-0000-0000000000000"
19+
# URL to a Splunk instance to send data to.
20+
endpoint: "https://splunk:8088/services/collector"
21+
# Optional Splunk source: https://docs.splunk.com/Splexicon:Source
22+
source: "app:metrics"
23+
# Optional Splunk source type: https://docs.splunk.com/Splexicon:Sourcetype
24+
sourcetype: "prometheus"
25+
# Splunk index, optional name of the Splunk index targeted.
26+
index: "metrics"
27+
# Maximum HTTP connections to use simultaneously when sending data. Defaults to 100.
28+
max_connections: 20
29+
# Whether to disable gzip compression over HTTP. Defaults to false.
30+
disable_compression: false
31+
# HTTP timeout when sending data. Defaults to 10s.
32+
timeout: 10s
33+
# Whether to skip checking the certificate of the HEC endpoint when sending data over HTTPS. Defaults to false.
34+
# For this demo, we use a self-signed certificate on the Splunk docker instance, so this flag is set to true.
35+
insecure_skip_verify: true
36+
37+
processors:
38+
batch:
39+
queued_retry:
40+
41+
extensions:
42+
health_check:
43+
pprof:
44+
endpoint: :1888
45+
zpages:
46+
endpoint: :55679
47+
48+
service:
49+
extensions: [pprof, zpages, health_check]
50+
pipelines:
51+
metrics:
52+
receivers: [prometheus_simple]
53+
processors: [batch, queued_retry]
54+
exporters: [splunk_hec/metrics]
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
FROM golang:1.14-stretch
2+
3+
WORKDIR /go/src/app
4+
5+
COPY go.mod .
6+
COPY main.go .
7+
8+
RUN go get
9+
10+
RUN go build
11+
12+
CMD /go/src/app/app
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
module app
2+
3+
go 1.14
4+
5+
require (
6+
go.opentelemetry.io/otel v0.15.0
7+
go.opentelemetry.io/otel/exporters/metric/prometheus v0.15.0
8+
go.opentelemetry.io/otel/exporters/otlp v0.15.0
9+
go.opentelemetry.io/otel/sdk v0.15.0
10+
go.uber.org/zap v1.16.0
11+
)

0 commit comments

Comments
 (0)