GH-286 : Implement ELK logging infrastructure

[patrickmac3]

Description

For the issue #286 , I integrated the ELK stack for the backend logging infrastructure.

ELK is the following technologies:

• ElasticSearch: stores the logs
• Logstash: ingests, transforms and loads logs into ElasticSearch
• Kibana: Visualization tool that works with ElasticSearch to create interactive dashboard and reports.

Changes

In microservices directory

• added docker-compose.yml file to run the stack locally
• create a logstash directory containing the following:

• config/logstash.yml for defining how Logstash processes the incoming data.
• pipeline/logstash.conf used for specifying various settings related to Logstashes behavior.

In each microservice

• added logging package with the following filter classes:

• EndpointFilter is used to add an endpoint field to the logs processed and visualized using ELK.
• RequestIdFilter is used to add a request_id field to logs to help with tracability within ELK.

• within the resources directories i added logback-spring.xml to configure logging in the console and Logstash

Manual Testing

1. Run the docker containers from the microservices directory using docker-compose up -d. (it could take a few minutes before everything is completely running)
2. Choose any microservice(s) and run it/them (do not forget to build beforehand)
3. Open kibana at http://localhost:5601
4. Inside of kibana, create an Index Pattern by going following these links in the side menu:

• Stack Management > Index Patterns > create index

5. name the index pattern sportahub-logs* and thee existing one should appear (select it)
6. Go to Discovery from the home menu
7. Visualize the logs