Skip to content

Commit 49ca152

Browse files
kpetremannkpetremann
authored
docs: data aggregation API + version matrix (#12)
1 parent 8e54429 commit 49ca152

File tree

8 files changed

+235
-14
lines changed

8 files changed

+235
-14
lines changed

‎docs/docs/.pages

+1
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ nav:
77
- monitoring.md
88
- configuration-cleaning.md
99
- ...
10+
- version_matrix.md
1011
- FAQ.md
1112
- CMDB
1213
- Data-Aggregation-API

‎docs/docs/Data-Aggregation-API/configuration.md

+140
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,140 @@
1+
# Configuration
2+
3+
## Configuration file
4+
5+
The Data Aggregation API can be configured with flags, environments variables and configuration file.
6+
7+
The precedence order for the different methods is:
8+
9+
!!! info
10+
11+
* flags
12+
* environment variables
13+
* configuration file (settings.yml)
14+
15+
```yaml
16+
Datacenter: "europe"
17+
18+
Log:
19+
Level: "error"
20+
Pretty: true
21+
22+
API:
23+
ListenAddress: "127.0.0.1"
24+
ListenPort: 1234
25+
26+
Authentication:
27+
LDAP:
28+
URL: "ldaps://URL.local"
29+
BindDN: "cn=<user>,OU=<ou>,DC=<local>"
30+
BaseDN: "DC=<local>"
31+
Password: "<some_password>"
32+
WorkersCount: 10
33+
Timeout: 5s
34+
MaxConnectionLifetime: 1m
35+
InsecureSkipVerify: false
36+
37+
NetBox:
38+
URL: "https://netbox.local"
39+
APIKey: "<some_key>"
40+
DatacenterFilterKey: "site"
41+
LimitPerPage: 500
42+
43+
Build:
44+
Interval: "30m"
45+
AllDevicesMustBuild: false
46+
```
47+
48+
## Global settings
49+
50+
| Parameter | Default | Description |
51+
|-----------|---------|-------------|
52+
| Datacenter | `` | Value used to filter devices. The key is defined by `DatacenterFilterKey`. |
53+
54+
## Log settings
55+
56+
All parameters below are in the `Log` section of the configuration.
57+
This section is optional.
58+
59+
| Parameter | Default | Description |
60+
|-----------|---------|-------------|
61+
| Level | `info` | Log level. |
62+
| Pretty | `false` | If enabled: human readable logs (with colors). If disabled: structured logs. |
63+
64+
## API settings
65+
66+
All parameters below are in the `API` section of the configuration.
67+
This section is optional.
68+
69+
| Parameter | Default | Description |
70+
|-----------|---------|-------------|
71+
| ListenAddress | `0.0.0.0` | Listening address of the web API. |
72+
| ListenPort | `8080` | Listening port of the web API. |
73+
74+
## Authentication settings
75+
76+
All parameters below are in the `Authentication`->`LDAP` section of the configuration.
77+
78+
LDAP Authentication is only to authenticate users when they try to query the Web API, i.e. when they want to retrieve the built config.
79+
80+
This section is optional.
81+
82+
| Parameter | Default | Description |
83+
|-----------|---------|-------------|
84+
| InsecureSkipVerify | `false` | Ignore LDAP TLS warnings. |
85+
| URL | | URL of the LDAP server. |
86+
| BindDN | | Bind used to query the LDAP server. |
87+
| Password | | Password to query the LDAP server. |
88+
| BaseDN | | Only users matching the BaseDN are authorized to query the web API. |
89+
| WorkersCount | `10` | Number of workers to authenticate users concurrently. |
90+
| Timeout | `10s` | Time to wait before considering a LDAP request timed out. |
91+
| MaxConnectionLifetime | `1m` | Lifetime of worker connection to LDAP. Useful to re-use existing LDAP connection. |
92+
93+
94+
## NetBox settings
95+
96+
All parameters below are in the `NetBox` section of the configuration.
97+
98+
| Parameter | Default | Description |
99+
|-----------|---------|-------------|
100+
| URL | | NetBox base URL. |
101+
| APIKey | | NetBox API token. |
102+
| DatacenterFilterKey | `site` | Key used to filter devices, using `Datacenter` as a value. |
103+
| LimitPerPage | `100` | Number of elements per page when getting data from NetBox API. |
104+
105+
## Build settings
106+
107+
All parameters below are in the `NetBox` section of the configuration.
108+
This section is optional.
109+
110+
| Parameter | Default | Description |
111+
|-----------|---------|-------------|
112+
| Interval | `1m` | Build interval, e.g.: `10m`, `1h` |
113+
| AllDevicesMustBuild | `false` | The build fails if one device has not been built. |
114+
115+
## Alternative methods
116+
117+
### Environment variables
118+
119+
All settings available in the configuration file can be set as environment variables, but:
120+
121+
* all variables must be prefixed by `DAAPI_`
122+
* uppercase only
123+
* `_` is the level separator
124+
125+
For example, the equivalent of this config file:
126+
127+
``` yaml
128+
Datacenter: "europe"
129+
NetBox:
130+
URL: "https://netbox.local"
131+
APIKey: "<some_key>"
132+
```
133+
134+
is:
135+
136+
```shell
137+
DAAPI_DATACENTER="europe"
138+
DAAPI_NETBOX_URL="https://netbox.local"
139+
DAAPI_NETBOX_APIKEY="<some_key>"
140+
```

‎docs/docs/Data-Aggregation-API/design.md

+30-9
Original file line numberDiff line numberDiff line change
@@ -83,12 +83,33 @@ Thanks to [ygot](https://github.com/openconfig/ygot) and [OpenConfig YANG models
8383

8484
## Reporting
8585

86-
!!! warning
87-
88-
:wrench: This section is under construction.
89-
90-
## Endpoints
91-
92-
!!! warning
93-
94-
:wrench: This section is under construction.
86+
| Endpoint | Description |
87+
|------------|-----------|
88+
| `/v1/report/last` | Last or ongoing build report |
89+
| `/v1/report/last/complete` | Report of the last complete build (whether it failed or not) |
90+
| `/v1/report/last/successful` | Report of the last successful build |
91+
92+
The reports contain:
93+
* The status of the build.
94+
* When it started and finished.
95+
* Statistics like performance.
96+
* Logs.
97+
98+
## OpenConfig
99+
100+
| Endpoint | Description |
101+
|------------|-----------|
102+
| `/v1/devices/[hostname]/openconfig` | Get OpenConfig data for a specific device |
103+
| `/v1/devices/*/openconfig` | Get OpenConfig data for all devices |
104+
| `/v1/devices/[hostname]/afk_enabled` | Indicates if a device has the tag AFK enabled in NetBox |
105+
| `/v1/devices/*/afk_enabled` | Same as above, but for all devices |
106+
107+
Note: `afk_enabled` can be used to enable or disable AFK schedule in Salt via `afk-enabled` tag in NetBox DCIM.
108+
109+
# Other endpoints
110+
111+
| Endpoint | Description |
112+
|------------|-----------|
113+
| `/metrics` | Prometheus metrics |
114+
| `/api/version` | Details about the running version |
115+
| `/api/health` | Dummy endpoint for basic healthcheck of the app |

‎docs/docs/Data-Aggregation-API/installation.md

+38-2
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,41 @@
11
# Installation
22

3-
!!! warning
3+
## Quickstart
44

5-
:wrench: Open Source version not yet ready.
5+
1. Download the latest release archive.
6+
2. Extract the single binary.
7+
3. Configure the Data Aggregation API accordingly.
8+
4. Run the binary.
9+
10+
Example of basic configuration (`settings.yml`):
11+
```yaml
12+
Datacenter: "europe"
13+
Log:
14+
Level: "error"
15+
Pretty: true
16+
17+
API:
18+
ListenAddress: "127.0.0.1"
19+
ListenPort: 1234
20+
21+
Log:
22+
Level: "info"
23+
Pretty: true
24+
25+
NetBox:
26+
URL: "https://netbox.local"
27+
APIKey: "<some_key>"
28+
DatacenterFilterKey: "site"
29+
30+
Build:
31+
Interval: "30m"
32+
```
33+
34+
See [Configuration](./configuration.md) for more details.
35+
36+
37+
## Dependencies
38+
39+
The only dependency is our [Network CMDB](https://github.com/criteo/netbox-network-cmdb) NetBox plugin.
40+
41+
The installation guide of this plugin is available in the CMDB section [here](/docs/docs/CMDB/).

‎docs/docs/SONiC-support/overview.md

+3-1
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44

55
| SONiC version | Support |
66
|---------------|---------|
7-
| 201911 | :white_check_mark: |
7+
| 201911* | :warning: |
88
| 202205 | :white_check_mark: |
99

1010
!!! info "Legend"
@@ -14,6 +14,8 @@
1414
- :warning: deprecated
1515
- :no_entry: not supported
1616

17+
* FRR 7.2.1 in SONiC 201911 has a memory leak which can be triggered by AFK
18+
1719
## Installation
1820

1921
| Step | Description | Guide |

‎docs/docs/index.md

+1-2
Original file line numberDiff line numberDiff line change
@@ -22,8 +22,7 @@ It is based on [NetBox](https://netbox.dev), [OpenConfig](https://www.openconfig
2222
| Repository | Description | Latest commit |
2323
|------------|-------------|---------------|
2424
| [Network CMDB](https://github.com/criteo/netbox-network-cmdb) | Network CMDB plugin for Netbox | ![Last commit](https://img.shields.io/github/last-commit/criteo/netbox-network-cmdb/main) |
25-
| Data aggregation API (coming soon) | Aggregate data from CMDB and convert to OpenConfig | |
26-
| [AFK Salt modules](https://github.com/criteo/afk-saltstack) | Salt modules to apply configuration from OpenConfig data | ![Last commit](https://img.shields.io/github/last-commit/criteo/openconfig-SaltStack/main) |
25+
| [Data aggregation API](https://github.com/criteo/data-aggregation-api) | Aggregate data from CMDB and convert to OpenConfig | ![Last commit](https://img.shields.io/github/last-commit/criteo/data-aggregation-api/main) || [AFK Salt modules](https://github.com/criteo/afk-saltstack) | Salt modules to apply configuration from OpenConfig data | ![Last commit](https://img.shields.io/github/last-commit/criteo/openconfig-SaltStack/main) |
2726
| [SONiC Salt Deployer](https://github.com/criteo/sonic-salt-deployer) | Tool to deploy and configure salt-minion on SONiC devices | ![Last commit](https://img.shields.io/github/last-commit/criteo/sonic-salt-deployer/main) |
2827
| [SONiC SaltStack](https://github.com/criteo/sonic-saltstack) | States/execution modules for SONiC | ![Last commit](https://img.shields.io/github/last-commit/criteo/sonic-SaltStack/main) |
2928
| [SONiC utilities](https://github.com/criteo/criteo-sonic-utilities) | SONiC scripts used by some SONiC SaltStack modules | ![Last commit](https://img.shields.io/github/last-commit/criteo/criteo-sonic-utilities/main) |

‎docs/docs/installation.md

+1
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
| Component | How to install |
44
|-----------|----------------|
55
| NetBox | [https://docs.netbox.dev/en/stable/installation/](https://docs.netbox.dev/en/stable/installation/)
6+
| [Network CMDB](https://github.com/criteo/netbox-network-cmdb) | Network CMDB plugin for Netbox | ![Last commit](https://img.shields.io/github/last-commit/criteo/netbox-network-cmdb/main) |
67
| Data Aggregation API | see our [Installation guide](/Data-Aggregation-API/installation/)
78
| SaltStack | [https://docs.saltproject.io/salt/install-guide/en/latest/](https://docs.saltproject.io/salt/install-guide/en/latest/)
89
| SaltStack for JunOS/EOS | [proxy-minion](https://docs.saltproject.io/en/latest/topics/proxyminion/index.html) with [napalm driver](https://github.com/napalm-automation/napalm-salt)

‎docs/docs/version_matrix.md

+21
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
# Version matrix
2+
3+
## v0.1-alpha
4+
5+
|Component|Version|
6+
|---------|-------|
7+
|Network CMDB||
8+
|Data Aggregration API|v0.0.1-alpha.x|
9+
|AFK SaltStack module||
10+
|SONiC SaltStack modules||
11+
12+
|Dependency|Version|
13+
|----------|-------|
14+
|NetBox|v3.3.10|
15+
|Saltstack|3001.x to 3004.x|
16+
17+
|Supported Network OS|Supported Versions|
18+
|--------------------|------------------|
19+
|SONiC|202205, 202211|
20+
|Juniper JunOS|>16.1 (tested with up to 22.x)|
21+
|Arista EOS|should all work, but tested up to 4.22.x|

0 commit comments

Comments
 (0)