Skip to content

Commit

Permalink
Merge pull request #5682 from Skanetrafiken/67039-federated-identity-…
Browse files Browse the repository at this point in the history
…service-bus

Add configuration switch for Service Bus authentication with Federate…
  • Loading branch information
Bartosz-Kruba authored Feb 22, 2024
2 parents 8ba2fb8 + f8f6dab commit dde7c51
Show file tree
Hide file tree
Showing 21 changed files with 495 additions and 47 deletions.
4 changes: 2 additions & 2 deletions doc-templates/UpdaterConfig.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ GBFS form factors:
## Other updaters in sandboxes

- [Vehicle parking](sandbox/VehicleParking.md)
- [Siri over HTTP](sandbox/SiriUpdater.md)
- [Siri over Azure Message Bus](sandbox/SiriAzureUpdater.md)
- [Siri over HTTP](sandbox/siri/SiriUpdater.md)
- [Siri over Azure Message Bus](sandbox/siri/SiriAzureUpdater.md)
- [VehicleRentalServiceDirectory](sandbox/VehicleRentalServiceDirectory.md)

Original file line number Diff line number Diff line change
Expand Up @@ -8,17 +8,26 @@ IT also OTP to download historical data from en HTTP endpoint on startup.
Skånetrafiken, Sweden
developer.otp@skanetrafiken.se

## Documentation

Documentation available [here](../../examples/skanetrafiken/Readme.md).

## Configuration

To enable the SIRI updater you need to add it to the updaters section of the `router-config.json`.

### Siri Azure ET Updater

<!-- INSERT: siri-azure-et-updater -->

### Siri Azure SX Updater

<!-- INSERT: siri-azure-sx-updater -->

## Changelog
- Added configuration for turning off stop arrival time match feature.
- Added configuration for turning off stop arrival time match feature.
- Initial version (April 2022)
- Minor changes in logging (November 2022)
- Retry fetch from history endpoint if it failed (February 2023)
- Solve a bug in SiriAzureETUpdater and improve error logging (March 2023)

## Documentation

Documentation available [here](../examples/skanetrafiken/Readme.md).

### Configuration

See example configuration in `examples/skanetrafiken/router-config.json`.
- Add support with federated identity authentication (February 2024)
File renamed without changes.
14 changes: 14 additions & 0 deletions docs/RouterConfiguration.md
Original file line number Diff line number Diff line change
Expand Up @@ -817,6 +817,20 @@ Used to group requests when monitoring OTP.
"toDateTime" : "P1D",
"timeout" : 300000
}
},
{
"type" : "siri-azure-et-updater",
"topic" : "some_topic",
"authenticationType" : "SharedAccessKey",
"fullyQualifiedNamespace" : "fully_qualified_namespace",
"servicebus-url" : "service_bus_url",
"feedId" : "feed_id",
"customMidnight" : 4,
"history" : {
"url" : "endpoint_url",
"fromDateTime" : "-P1D",
"timeout" : 300000
}
}
],
"rideHailingServices" : [
Expand Down
4 changes: 2 additions & 2 deletions docs/UpdaterConfig.md
Original file line number Diff line number Diff line change
Expand Up @@ -414,7 +414,7 @@ HTTP headers to add to the request. Any header key, value can be inserted.
## Other updaters in sandboxes

- [Vehicle parking](sandbox/VehicleParking.md)
- [Siri over HTTP](sandbox/SiriUpdater.md)
- [Siri over Azure Message Bus](sandbox/SiriAzureUpdater.md)
- [Siri over HTTP](sandbox/siri/SiriUpdater.md)
- [Siri over Azure Message Bus](sandbox/siri/SiriAzureUpdater.md)
- [VehicleRentalServiceDirectory](sandbox/VehicleRentalServiceDirectory.md)

3 changes: 3 additions & 0 deletions docs/examples/skanetrafiken/Readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,9 @@ id from the message. In case OTP was not able to find corresponding trip additio
performed based on arrival-times/stop-patterns from the ET message. This feature turned off by default but can be
activated by adding *fuzzyTripMatching* property to updater configuration.

### FederatedIdentity
It is also possible to connect to Service Bus through FederatedIdentity. Change **authenticationType** to
**FederatedIdentity** and provide **fullyQualifiedNamespace** in router-config.



Expand Down
3 changes: 2 additions & 1 deletion docs/examples/skanetrafiken/router-config.json
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,8 @@
"type": "siri-azure-sx-updater",
"topic": "<topic_name>",
"feedId": "<feed_id>",
"servicebus-url": "<service_bus_url>",
"authenticationType": "FederatedIdentity",
"fullyQualifiedNamespace": "<fully_qualified_namespace>",
"customMidnight": 4,
"history": {
"url": "<history_endpoint_url>",
Expand Down
197 changes: 197 additions & 0 deletions docs/sandbox/siri/SiriAzureUpdater.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,197 @@
# Siri Azure Updater

It is a sandbox extension developed by Skånetrafiken that allows OTP to fetch Siri ET & SX messages through *Azure Service Bus*.
It also allows for OTP to download historical real-time data from an HTTP endpoint on startup.

## Contact Info

Skånetrafiken, Sweden
developer.otp@skanetrafiken.se

## Documentation

Documentation available [here](../../examples/skanetrafiken/Readme.md).

## Configuration

To enable the SIRI updater you need to add it to the updaters section of the `router-config.json`.

### Siri Azure ET Updater

<!-- siri-azure-et-updater BEGIN -->
<!-- NOTE! This section is auto-generated. Do not change, change doc in code instead. -->

| Config Parameter | Type | Summary | Req./Opt. | Default Value | Since |
|------------------------------------------------------------|:---------:|----------------------------------------------------------------|:----------:|---------------------|:-----:|
| type = "siri-azure-et-updater" | `enum` | The type of the updater. | *Required* | | 1.5 |
| [authenticationType](#u__11__authenticationType) | `enum` | Which authentication type to use | *Optional* | `"sharedaccesskey"` | 2.5 |
| [customMidnight](#u__11__customMidnight) | `integer` | Time on which time breaks into new day. | *Optional* | `0` | 2.2 |
| feedId | `string` | The ID of the feed to apply the updates to. | *Optional* | | 2.2 |
| [fullyQualifiedNamespace](#u__11__fullyQualifiedNamespace) | `string` | Service Bus fully qualified namespace used for authentication. | *Optional* | | 2.5 |
| fuzzyTripMatching | `boolean` | Whether to apply fuzzyTripMatching on the updates | *Optional* | `false` | 2.2 |
| [servicebus-url](#u__11__servicebus_url) | `string` | Service Bus connection used for authentication. | *Optional* | | 2.2 |
| topic | `string` | Service Bus topic to connect to. | *Optional* | | 2.2 |
| history | `object` | Configuration for fetching historical data on startup | *Optional* | | 2.2 |
|    fromDateTime | `string` | Datetime boundary for historical data | *Optional* | `"-P1D"` | 2.2 |
|    timeout | `integer` | Timeout in milliseconds | *Optional* | `300000` | na |
|    url | `string` | Endpoint to fetch from | *Optional* | | na |


##### Parameter details

<h4 id="u__11__authenticationType">authenticationType</h4>

**Since version:** `2.5`**Type:** `enum`**Cardinality:** `Optional`**Default value:** `"sharedaccesskey"`
**Path:** /updaters/[11]
**Enum values:** `sharedaccesskey` | `federatedidentity`

Which authentication type to use

<h4 id="u__11__customMidnight">customMidnight</h4>

**Since version:** `2.2`**Type:** `integer`**Cardinality:** `Optional`**Default value:** `0`
**Path:** /updaters/[11]

Time on which time breaks into new day.

It is common that operating day date breaks a little bit later than midnight so that the switch happens when traffic is at the lowest point. Parameter uses 24-hour format. If the switch happens on 4 am then set this field to 4.

<h4 id="u__11__fullyQualifiedNamespace">fullyQualifiedNamespace</h4>

**Since version:** `2.5`**Type:** `string`**Cardinality:** `Optional`
**Path:** /updaters/[11]

Service Bus fully qualified namespace used for authentication.

Has to be present for authenticationMethod FederatedIdentity.

<h4 id="u__11__servicebus_url">servicebus-url</h4>

**Since version:** `2.2`**Type:** `string`**Cardinality:** `Optional`
**Path:** /updaters/[11]

Service Bus connection used for authentication.

Has to be present for authenticationMethod SharedAccessKey. This should be Primary/Secondary connection string from service bus.



##### Example configuration

```JSON
// router-config.json
{
"updaters" : [
{
"type" : "siri-azure-et-updater",
"topic" : "some_topic",
"authenticationType" : "SharedAccessKey",
"fullyQualifiedNamespace" : "fully_qualified_namespace",
"servicebus-url" : "service_bus_url",
"feedId" : "feed_id",
"customMidnight" : 4,
"history" : {
"url" : "endpoint_url",
"fromDateTime" : "-P1D",
"timeout" : 300000
}
}
]
}
```

<!-- siri-azure-et-updater END -->

### Siri Azure SX Updater

<!-- siri-azure-sx-updater BEGIN -->
<!-- NOTE! This section is auto-generated. Do not change, change doc in code instead. -->

| Config Parameter | Type | Summary | Req./Opt. | Default Value | Since |
|------------------------------------------------------------|:---------:|----------------------------------------------------------------|:----------:|---------------------|:-----:|
| type = "siri-azure-sx-updater" | `enum` | The type of the updater. | *Required* | | 1.5 |
| [authenticationType](#u__10__authenticationType) | `enum` | Which authentication type to use | *Optional* | `"sharedaccesskey"` | 2.5 |
| [customMidnight](#u__10__customMidnight) | `integer` | Time on which time breaks into new day. | *Optional* | `0` | 2.2 |
| feedId | `string` | The ID of the feed to apply the updates to. | *Optional* | | 2.2 |
| [fullyQualifiedNamespace](#u__10__fullyQualifiedNamespace) | `string` | Service Bus fully qualified namespace used for authentication. | *Optional* | | 2.5 |
| fuzzyTripMatching | `boolean` | Whether to apply fuzzyTripMatching on the updates | *Optional* | `false` | 2.2 |
| [servicebus-url](#u__10__servicebus_url) | `string` | Service Bus connection used for authentication. | *Optional* | | 2.2 |
| topic | `string` | Service Bus topic to connect to. | *Optional* | | 2.2 |
| history | `object` | Configuration for fetching historical data on startup | *Optional* | | 2.2 |
|    fromDateTime | `string` | Datetime boundary for historical data. | *Optional* | `"-P1D"` | 2.2 |
|    timeout | `integer` | Timeout in milliseconds | *Optional* | `300000` | na |
|    toDateTime | `string` | Datetime boundary for historical data. | *Optional* | `"P1D"` | 2.2 |
|    url | `string` | Endpoint to fetch from | *Optional* | | na |


##### Parameter details

<h4 id="u__10__authenticationType">authenticationType</h4>

**Since version:** `2.5`**Type:** `enum`**Cardinality:** `Optional`**Default value:** `"sharedaccesskey"`
**Path:** /updaters/[10]
**Enum values:** `sharedaccesskey` | `federatedidentity`

Which authentication type to use

<h4 id="u__10__customMidnight">customMidnight</h4>

**Since version:** `2.2`**Type:** `integer`**Cardinality:** `Optional`**Default value:** `0`
**Path:** /updaters/[10]

Time on which time breaks into new day.

It is common that operating day date breaks a little bit later than midnight so that the switch happens when traffic is at the lowest point. Parameter uses 24-hour format. If the switch happens on 4 am then set this field to 4.

<h4 id="u__10__fullyQualifiedNamespace">fullyQualifiedNamespace</h4>

**Since version:** `2.5`**Type:** `string`**Cardinality:** `Optional`
**Path:** /updaters/[10]

Service Bus fully qualified namespace used for authentication.

Has to be present for authenticationMethod FederatedIdentity.

<h4 id="u__10__servicebus_url">servicebus-url</h4>

**Since version:** `2.2`**Type:** `string`**Cardinality:** `Optional`
**Path:** /updaters/[10]

Service Bus connection used for authentication.

Has to be present for authenticationMethod SharedAccessKey. This should be Primary/Secondary connection string from service bus.



##### Example configuration

```JSON
// router-config.json
{
"updaters" : [
{
"type" : "siri-azure-sx-updater",
"topic" : "some_topic",
"servicebus-url" : "service_bus_url",
"feedId" : "feed_id",
"customMidnight" : 4,
"history" : {
"url" : "endpoint_url",
"fromDateTime" : "-P1D",
"toDateTime" : "P1D",
"timeout" : 300000
}
}
]
}
```

<!-- siri-azure-sx-updater END -->

## Changelog
- Added configuration for turning off stop arrival time match feature.
- Initial version (April 2022)
- Minor changes in logging (November 2022)
- Retry fetch from history endpoint if it failed (February 2023)
- Solve a bug in SiriAzureETUpdater and improve error logging (March 2023)
- Add support with federated identity authentication (February 2024)
File renamed without changes.
4 changes: 2 additions & 2 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -96,8 +96,8 @@ nav:
- Actuator API: 'sandbox/ActuatorAPI.md'
- Direct Transfer Analyzer: 'sandbox/transferanalyzer.md'
- Google Cloud Storage: 'sandbox/GoogleCloudStorage.md'
- SIRI Updaters: 'sandbox/SiriUpdater.md'
- SIRI Updater (Azure): 'sandbox/SiriAzureUpdater.md'
- SIRI Updaters: 'sandbox/siri/SiriUpdater.md'
- SIRI Updater (Azure): 'sandbox/siri/SiriAzureUpdater.md'
- Vehicle Rental Service Directory API support: 'sandbox/VehicleRentalServiceDirectory.md'
- Smoove Bike Rental Updator Support: 'sandbox/SmooveBikeRental.md'
- Mapbox Vector Tiles API: 'sandbox/MapboxVectorTilesApi.md'
Expand Down
11 changes: 6 additions & 5 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -907,17 +907,18 @@
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-core</artifactId>
<version>1.45.0</version>
<version>1.46.0</version>
</dependency>
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-messaging-servicebus</artifactId>
<version>7.14.5</version>
<version>7.15.0</version>
</dependency>
<dependency>
<groupId>com.azure.resourcemanager</groupId>
<artifactId>azure-resourcemanager-servicebus</artifactId>
<version>2.32.0</version>
<groupId>com.azure</groupId>
<artifactId>azure-identity</artifactId>
<version>1.11.2</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>ch.poole</groupId>
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
package org.opentripplanner.ext.siri.updater.azure;

import com.azure.identity.DefaultAzureCredentialBuilder;
import com.azure.messaging.servicebus.ServiceBusClientBuilder;
import com.azure.messaging.servicebus.ServiceBusErrorContext;
import com.azure.messaging.servicebus.ServiceBusException;
Expand Down Expand Up @@ -36,6 +37,8 @@
public abstract class AbstractAzureSiriUpdater implements GraphUpdater {

private final Logger LOG = LoggerFactory.getLogger(getClass());
private final AuthenticationType authenticationType;
private final String fullyQualifiedNamespace;
private final String configRef;
private final String serviceBusUrl;
private final SiriFuzzyTripMatcher fuzzyTripMatcher;
Expand Down Expand Up @@ -63,6 +66,8 @@ public abstract class AbstractAzureSiriUpdater implements GraphUpdater {

public AbstractAzureSiriUpdater(SiriAzureUpdaterParameters config, TransitModel transitModel) {
this.configRef = config.configRef();
this.authenticationType = config.getAuthenticationType();
this.fullyQualifiedNamespace = config.getFullyQualifiedNamespace();
this.serviceBusUrl = config.getServiceBusUrl();
this.topicName = config.getTopicName();
this.dataInitializationUrl = config.getDataInitializationUrl();
Expand Down Expand Up @@ -105,10 +110,17 @@ public void run() {
}

// Client with permissions to create subscription
serviceBusAdmin =
new ServiceBusAdministrationClientBuilder()
.connectionString(serviceBusUrl)
.buildAsyncClient();
if (authenticationType == AuthenticationType.FederatedIdentity) {
serviceBusAdmin =
new ServiceBusAdministrationClientBuilder()
.credential(fullyQualifiedNamespace, new DefaultAzureCredentialBuilder().build())
.buildAsyncClient();
} else if (authenticationType == AuthenticationType.SharedAccessKey) {
serviceBusAdmin =
new ServiceBusAdministrationClientBuilder()
.connectionString(serviceBusUrl)
.buildAsyncClient();
}

// If Idle more then one day, then delete subscription so we don't have old obsolete subscriptions on Azure Service Bus
var options = new CreateSubscriptionOptions();
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
package org.opentripplanner.ext.siri.updater.azure;

public enum AuthenticationType {
SharedAccessKey,
FederatedIdentity,
}
Loading

0 comments on commit dde7c51

Please sign in to comment.