Skip to content

Commit fc7f7a7

Browse files
author
Uzziah Eyee
committedApr 30, 2019
Made microservices easier to locally develop.
·
v2.0
1 parent f0b0bb4 commit fc7f7a7

File tree

10 files changed

+497
-31
lines changed

10 files changed

+497
-31
lines changed
 

‎.vscode/launch.json

+20
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
{
2+
// Use IntelliSense to learn about possible attributes.
3+
// Hover to view descriptions of existing attributes.
4+
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
5+
"version": "0.2.0",
6+
"configurations": [
7+
{
8+
"name": "Attach to api-server container",
9+
"type": "node",
10+
"request": "attach",
11+
// should match the exposed debug address specified in docker-compose
12+
"address": "localhost",
13+
"port": 9229,
14+
"protocol": "inspector",
15+
// you typically copy the src folder into a location in docker...specify this mapping.
16+
"localRoot": "${workspaceFolder}/api-server/src",
17+
"remoteRoot": "/usr/src/app"
18+
}
19+
]
20+
}

‎README.md

+64-10
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,52 @@
11
# Horus
22

3-
Monitoring containerized microservices with a centralized logging architecture.
3+
A test project for learning microservices observability through: **centralized logging** and **distributed tracing**.
44

5-
## Dependencies
5+
## Requirements
66

77
- Docker Compose v1.23.2
8+
- OSX or GNU/Linux operating system
89

9-
## Setup
10+
## Development Setup
1011

11-
1. Signup with an ELK SaaS provider like [Logz.io](logz.io) to obtain an authentication token. Then for each microservice, rename `example.env` to `.env` and supply the needed secrets.
12+
1. Clone this repo and open the root folder in an IDE like *Visual Studio Code*.
1213

13-
2. Run the following commands.
14+
2. For each microservice, rename `example.env` to `.env` and supply the needed secrets.
1415

15-
docker-compose build --pull
16-
docker-compose up -d --force-recreate
17-
18-
3. Then log into your ELK SaaS and view your microservices logs.
16+
3. Start all microservices in *development mode*.
17+
18+
docker-compose -f docker-compose.dev.yml \
19+
up -d --build
20+
21+
> Development mode is important for 2 reasons.
22+
> - Changes to a service's source files will automatically restart/rebuild the docker service to show the changes.
23+
> - You can attach a remote debugger from your IDE to the docker process for seemless debugging like placing breakpoints and watching variables.
24+
> - NOTE: When you edit a service's source files, the docker process will restart, severing your remote debugger connection. So, you'd need to reattach the debugger again.
25+
26+
4. Attach the IDE's debugger to the desired service. Follow these steps in Visual Studio Code: *shift+cmd+D > Select a debug configuration > F5*.
27+
> All Visual Studio Code debug configurations are stored in *.vscode/launch.json*. You can modify configs as you see fit.
28+
29+
### Useful dev commands
30+
31+
# list all running services
32+
docker-compose -f docker-compose.dev.yml ps
33+
34+
# stop all services
35+
docker-compose -f docker-compose.dev.yml down
36+
37+
# restart only all [or specific] service
38+
docker-compose -f docker-compose.dev.yml \
39+
up -d --no-deps --build [service-name]
40+
41+
# tail logs from all [or specific] service
42+
docker-compose logs -f [service-name]
43+
44+
## Production Setup
45+
46+
TODO
47+
48+
- Signup with an ELK SaaS provider like [Logz.io](logz.io) to obtain an authentication token. Then for each microservice, rename `example.env` to `.env` and supply the needed secrets.
49+
- Then log into your ELK SaaS and view your microservices logs.
1950

2051
## Project Documentation
2152

@@ -27,6 +58,16 @@ I wrote an accompanying [article](https://hackernoon.com/monitoring-containerize
2758

2859
## Notes
2960

61+
- You can update a single service/container while the entire suite is running using the following command
62+
63+
docker-compose -f docker-compose.dev.yml \
64+
up -d --no-deps --build <service_name>
65+
66+
docker-compose -f docker-compose.prod.yml \
67+
up -d --no-deps --build <service_name>
68+
69+
> `--build` recreates the container from its image/dockerfile and `--no-deps` ensures dependent services aren't affected.
70+
3071
### Docker Networking
3172

3273
By default each containerized process runs in an isolated network namespace. For inter-container communication, place them in the same network namespace...as seen in *docker-compose.yml*.
@@ -36,8 +77,21 @@ By default each containerized process runs in an isolated network namespace. For
3677
1. You can pass secrets for a microservice using the `env_file` attribute in *docker-compose.yml*.
3778
2. Microservices can communicate using their service names if they are in the same docker network.
3879

80+
### How do you debug an app running in a container (i.e breakpoints and watch variables etc.)?
81+
?
82+
- You can tail the logs from all services using `docker-compose [-f <file>] logs -f`.
83+
3984
### Improvement Considerations
4085

4186
1. **Name Duplication:** The value of the `API_SERVER_ADDRESS` variable in *user-simulator/.env* depends on the service name `api-server` specified in *docker-compose.yml*. If we rename the service, we must also change the variable. Is there a way to make this DRY?
4287

43-
2. In the log-shipper container, I had to install a logz.io-specific plugin. Can't this step be eliminated since fluentd is capable of connecting to https endpoints without plugins?
88+
2. In the log-shipper container, I had to install a logz.io-specific plugin. Can't this step be eliminated since fluentd is capable of connecting to https endpoints without plugins?
89+
90+
3. Use sub-second precision for fluentd timestamps (probably best to use nanoseconds.)
91+
92+
## Some References
93+
94+
https://medium.com/lucjuggery/docker-in-development-with-nodemon-d500366e74df
95+
https://blog.risingstack.com/how-to-debug-a-node-js-app-in-a-docker-container/
96+
https://codefresh.io/docker-tutorial/debug_node_in_docker/
97+
https://code.visualstudio.com/docs/editor/debugging

‎api-server/.gitignore

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
11
.env
2-
/node_modules
2+
node_modules
33
npm-debug.log
44
.DS_Store

‎api-server/Dockerfile

+3-1
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,11 @@ WORKDIR /usr/src/app
55
# optimization to only rebuild npm modules iff package[-lock].json changes
66
COPY src/package*.json ./
77

8+
# TODO: only install nodemon if some arg is set to 'dev'
9+
RUN npm install -g nodemon
810
# For production `RUN npm install --only=production`
911
RUN npm install
1012

1113
COPY src .
1214

13-
CMD ["npm", "start"]
15+
CMD ["node", "."]

‎api-server/example.env

+1-1
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
PORT=
1+
# TODO

‎api-server/src/app.js

+8-10
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,17 @@
1-
// const dotenv = require('dotenv')
2-
// if (process.env.NODE_ENV !== 'production') {
3-
// const result = dotenv.config()
4-
// if (result.error) {
5-
// throw result.error
6-
// }
7-
// }
8-
91
const app = require('express')()
102
const uuid = require('uuid/v1')
113

124
app.use('/api/v1/tokens', (req, res) => {
135
console.log(`Handling request for token`)
146
res.json({token: uuid()})
157
})
16-
17-
const server = app.listen(process.env.PORT, () => {
8+
app.use('/api/v1/whereami', (req, res) => {
9+
// TODO: get geolocation info from IP address
10+
// TODO: get weather info from geolocation
11+
// TODO: return everything
12+
let data = {'info': null}
13+
res.json(data)
14+
})
15+
const server = app.listen(process.env.PORT || 3000, () => {
1816
console.log(`Listening on ${ server.address().address }:${ server.address().port }`)
1917
})

0 commit comments

Comments
 (0)