This document aims to answer questions on how to configure external dependencies and which public interfaces are offered by Indexing-Service in a comprehensive way. It is meant to be used for getting an overview and guidance in addition to the official documentation, which is available at the official Indexing-Service Web page (not yet available).How can this software be connected with other software? This is the question this document aims to answer.
ℹ️ Note: This document applies to the Indexing-Service version it is shipped with. If you have a specific version running, please refer to
INTERFACE.mdof this particular release.
- Messaging
- Elasticsearch Messaging is used to inform indexing service about any updates regarding metadata documents. Indexing service will fetch the document, transform it (if mapping available) and send it to elasticsearch for indexing.
External dependencies are third-party services that are required for Indexing-Service to work properly or that can be added optionally to provide additional functionality. Typically, external dependencies require additional software to be installed and configured, before they can be included in the Indexing-Service configuration, which is typically done via the main configuration file application.properties. If you do not want to lose your default settings, we recommend that you make a copy of "application.properties" and move it to the "config" subfolder. Remove all properties you want to keep from the new file. All properties in "config/application.properties" override the settings in "application.properties".
External dependencies are third-party services that are required for Indexing-Service to work properly or that can be added optionally to provide additional functionality. Typically, external dependencies require
additional software to be installed and configured, before they can be included in the Indexing-Service configuration, which is should be done via the configuration file config/application.properties.
A relational database is required by Indexing-Service to store administrative metadata for mapping documents. If not configured properly, Indexing-Service will fail to start.
- H2 In-Memory (driver included, used for testing only, not recommended for production) Example
- H2 File-Based (driver included, used for basic Docker setup or for 'quick and dirty' tests, not recommended for production) Example
- Postgres (driver included, requires a running PostgreSQL server, recommended for production) PostgreSQL, Example
ℹ️ Note: Other relational databases, such as MariaDB, SQLite, or Oracle, may also work, but require additional steps. To allow Indexing-Service to connect, the source code repository must be cloned, an appropriate JDBC driver has to be added to
build.gradleand Indexing-Service has to be compiled. Proper JDBC drivers are typically provided on the database's web page. Afterwards, the database can be configured inconfig/application.propertiessimilar to PostgreSQL but with database-specific property naming. Please refer to the driver documentation for details.
Indexing-Service requires access to the local file system in order to store and manage uploaded mapping documents. Indexing-Service requires access to one folder, which can be located on the local hard drive or mounted via NFS. It's recommended to create a subfolder named 'mapping' in the installation directory.
- see
application.properties
ℹ️ Note: The file path to the folder has to start with three '/'. If you overwrite the default setting we recommend to create or edit 'config/application.properties'. e.g.:
metastore.indexer.mappingsLocation:file:///data/indexing/mappingIf the folder do not exist, it will be created.
AMQP-based messaging is a mandatory feature of Indexing-Service, which allows Indexing-Service to receive messages about creation, modification, and deletion events inside MetaStore related to metadata documents and process them in an asynchronous way.
- RabbitMQ (dependencies included, serves as messaging distributor, requires a running RabbitMQ server)
Indexing-Service requires an elasticsearch instance for indexing received metadata documents. (JSON metadata documents are provided by default. XML documents should be transformed to JSON using GEMMA (currently the only available mapping)) (In future versions mapping-service should be integrated.)
By default, Indexing-Service itself is open for all kinds of operations, i.e., read and write, where write access should be restricted on the user interface level, e.g., by a password-protected area for critical operations. Optionally, authentication and authorization via JSON Web Tokens (JWT) issued by a Keycloak instance, can be configured, which allows a fine-granulated access management on document level.
ℹ️ Note: As Indexing-Service is not planned to be used by humans, no access control is used by default. Therefor the service should not be accessible from outside KIT. Managing mappings should be done by an administrator.
- Setup Indexing-Service Configuration is similar to MetaStore.
Public Interfaces are used to access Indexing-Service in order to obtain its contents, typically this happens via HTTP/REST. Depending on the interface, special clients or protocols must be used to access a specific public interface.
Interfaces which trigger functionality or return information on request.
A REST interface which allows to access the service functionality, like creating/registering, updating and validating metadata documents/schemas on request.
- OpenAPI via running instance.
- Usage with Examples
-/-
A REST interface which allows to access the status of the service:
- hostname:port/contextpath/actuator/info (information about version)
- hostname:port/contextpath/actuator/health (information about (connected) services)