Skip to content

Latest commit

 

History

History
163 lines (130 loc) · 8.9 KB

2016-02-26-bonfire.md

File metadata and controls

163 lines (130 loc) · 8.9 KB
layout title categories author lang
post
How to query logs from command line
Logs
Pierre De Paepe
en

Bonfire is a open source command line interface to query Graylog searches via the REST API. It tries to emulate the feeling of using tail on a local file and add other valuable options.

Requirements

  • Python
  • Pip

Install

$ git clone https://github.com/blue-yonder/bonfire.git
$ cd bonfire
$ pip install --user .

tail -f

$ bonfire -h laas.runabove.com -u ra-logs-12345 --tls -f
Enter password for ra-logs-12345@laas.runabove.com:12900:
Please select a stream to query:
0: Stream 'My stream' (id: 55210a04e4b09e9fa4fa0209)
Enter stream number: [0]:

> /tmp/what.log

$ bonfire -h laas.runabove.com -u ra-logs-12345 --tls -@ "2 minutes ago" "*" -o /tmp/what.log
Enter password for ra-logs-12345@laas.runabove.com:12900:
Please select a stream to query:
0: Stream 'My stream' (id: 55210a04e4b09e9fa4fa0209)
Enter stream number: [0]: 0

Password management

Typing your password can be avoided by using the -k switch that will store your password using the python keyring, this will store your password in your Desktop keyring system: https://bitbucket.org/kang/python-keyring-lib

Config file

To avoid repeating argument on the command line, you can create a config file (./bonfire.cfg or ~/.bonfire.cfg) to store where you want to connect:

[node:default]
host=laas.runabove.com
username=ra-logs-12345
tls=True
port=12900
default_stream=55955321e4b00da05ab5ea3e

You can define other named nodes and call them with the --node switch.

[node:sadev]
host=laas.runabove.com
username=ra-logs-12345
tls=True
port=12900
default_stream=5592a8f6e4b09e9fc5329ba6

You can now call named node:

$ bonfire --node sadev -k
NOTICE  [2015-07-27 12:11:38.13] [2015-07-27 10:11:37.137253Z][9a9b9759-60f5-4609-88c2-e809b5a361a8] search app.preprod.sailabove.io # source:172.17.42.1; facility:gelf-rb; line:; module:
NOTICE  [2015-07-27 12:11:38.14] [2015-07-27 10:11:37.137610Z][9a9b9759-60f5-4609-88c2-e809b5a361a8] bash: dig: command not found # source:172.17.42.1; facility:gelf-rb; line:; module:
NOTICE  [2015-07-27 12:11:38.14] [2015-07-27 10:11:37.173477Z][9a9b9759-60f5-4609-88c2-e809b5a361a8] 2015-07-27 10:11:37 URL:http://proof.ovh.net/ [2272/2272] -> "/dev/null" [1] # source:172.17.42.1; facility:gelf-rb; line:; module:
NOTICE  [2015-07-27 12:11:38.14] [2015-07-27 10:11:37.230451Z][08bce56b-9e85-4daa-9c8d-5b39c0d75d69] nameserver 213.186.33.99 # source:172.17.42.1; facility:gelf-rb; line:; module:
NOTICE  [2015-07-27 12:11:38.14] [2015-07-27 10:11:37.230451Z][08bce56b-9e85-4daa-9c8d-5b39c0d75d69] search app.preprod.sailabove.io # source:172.17.42.1; facility:gelf-rb; line:; module:
NOTICE  [2015-07-27 12:11:38.14] [2015-07-27 10:11:37.230973Z][08bce56b-9e85-4daa-9c8d-5b39c0d75d69] bash: dig: command not found # source:172.17.42.1; facility:gelf-rb; line:; module:
INFO    [2015-07-27 12:11:38.38] becfb0408cde # source:%{host}; facility:gelf-rb; line:; module:
INFO    [2015-07-27 12:11:42.29] 58d24d1cb8e3 # source:%{host}; facility:gelf-rb; line:; module:
INFO    [2015-07-27 12:11:43.38] becfb0408cde # source:%{host}; facility:gelf-rb; line:; module:
WARNING [2015-07-27 12:11:44.66] libceph: mon0 10.99.176.46:6789 socket closed (con state CONNECTING) # source:172.17.42.1; facility:gelf-rb; line:; module:

Stored queries

You can define named queries and call them from the command line:

.bonfire.cfg

[query:libceph]
query=program:kernel AND libceph
limit=20

Call

$ bonfire --node sadev -@ 2015-07-20 -k :libceph
WARNING [2015-07-27 12:13:54.87] libceph: mon0 10.99.176.46:6789 socket closed (con state CONNECTING) # source:172.17.42.1; facility:gelf-rb; line:; module:
WARNING [2015-07-27 12:14:04.89] libceph: mon0 10.99.176.46:6789 socket closed (con state CONNECTING) # source:172.17.42.1; facility:gelf-rb; line:; module:
WARNING [2015-07-27 12:14:14.90] libceph: mon0 10.99.176.46:6789 socket closed (con state CONNECTING) # source:172.17.42.1; facility:gelf-rb; line:; module:
WARNING [2015-07-27 12:14:24.92] libceph: mon1 10.99.180.46:6789 socket closed (con state CONNECTING) # source:172.17.42.1; facility:gelf-rb; line:; module:
WARNING [2015-07-27 12:14:34.93] libceph: mon1 10.99.180.46:6789 socket closed (con state CONNECTING) # source:172.17.42.1; facility:gelf-rb; line:; module:
WARNING [2015-07-27 12:14:44.95] libceph: mon0 10.99.176.46:6789 socket closed (con state CONNECTING) # source:172.17.42.1; facility:gelf-rb; line:; module:
WARNING [2015-07-27 12:14:54.97] libceph: mon0 10.99.176.46:6789 socket closed (con state CONNECTING) # source:172.17.42.1; facility:gelf-rb; line:; module:
WARNING [2015-07-27 12:15:04.98] libceph: mon1 10.99.180.46:6789 socket closed (con state CONNECTING) # source:172.17.42.1; facility:gelf-rb; line:; module:
WARNING [2015-07-27 12:15:15.00] libceph: mon1 10.99.180.46:6789 socket closed (con state CONNECTING) # source:172.17.42.1; facility:gelf-rb; line:; module:
WARNING [2015-07-27 12:15:25.01] libceph: mon0 10.99.176.46:6789 socket closed (con state CONNECTING) # source:172.17.42.1; facility:gelf-rb; line:; module:
WARNING [2015-07-27 12:15:35.03] libceph: mon1 10.99.180.46:6789 socket closed (con state CONNECTING) # source:172.17.42.1; facility:gelf-rb; line:; module:
WARNING [2015-07-27 12:15:45.04] libceph: mon2 10.99.184.46:6789 socket closed (con state CONNECTING) # source:172.17.42.1; facility:gelf-rb; line:; module:
WARNING [2015-07-27 12:15:55.06] libceph: mon0 10.99.176.46:6789 socket closed (con state CONNECTING) # source:172.17.42.1; facility:gelf-rb; line:; module:
WARNING [2015-07-27 12:16:05.08] libceph: mon1 10.99.180.46:6789 socket closed (con state CONNECTING) # source:172.17.42.1; facility:gelf-rb; line:; module:
WARNING [2015-07-27 12:16:15.09] libceph: mon2 10.99.184.46:6789 socket closed (con state CONNECTING) # source:172.17.42.1; facility:gelf-rb; line:; module:
WARNING [2015-07-27 12:16:25.11] libceph: mon0 10.99.176.46:6789 socket closed (con state CONNECTING) # source:172.17.42.1; facility:gelf-rb; line:; module:
WARNING [2015-07-27 12:16:35.12] libceph: mon2 10.99.184.46:6789 socket closed (con state CONNECTING) # source:172.17.42.1; facility:gelf-rb; line:; module:
WARNING [2015-07-27 12:16:45.14] libceph: mon1 10.99.180.46:6789 socket closed (con state CONNECTING) # source:172.17.42.1; facility:gelf-rb; line:; module:
WARNING [2015-07-27 12:16:55.16] libceph: mon0 10.99.176.46:6789 socket closed (con state CONNECTING) # source:172.17.42.1; facility:gelf-rb; line:; module:
WARNING [2015-07-27 12:17:05.17] libceph: mon2 10.99.184.46:6789 socket closed (con state CONNECTING) # source:172.17.42.1; facility:gelf-rb; line:; module:

Parametric queries

You can also define queries with parameters and define this parameter from the command line:

.bonfire.cfg

[query:containerlogs]
query=_exists_:containerlog_message AND container_uuid:"${uuid}"
fields=container_uuid,containerlog_message
limit=10

Call

$ bonfire --node sadev -k -x "uuid=08bce56b-9e85-4daa-9c8d-5b39c0d75d69" :containerlogs
INFO    [2015-07-27 12:22:38.60] container_uuid:08bce56b-9e85-4daa-9c8d-5b39c0d75d69; containerlog_message:2015-07-27 10:22:36 URL:http://proof.ovh.net/ [2272/2272] -> "/dev/null" [1]
INFO    [2015-07-27 12:22:38.61] container_uuid:08bce56b-9e85-4daa-9c8d-5b39c0d75d69; containerlog_message:nameserver   213.186.33.99
INFO    [2015-07-27 12:22:38.61] container_uuid:08bce56b-9e85-4daa-9c8d-5b39c0d75d69; containerlog_message:search   app.preprod.sailabove.io
INFO    [2015-07-27 12:22:38.61] container_uuid:08bce56b-9e85-4daa-9c8d-5b39c0d75d69; containerlog_message:bash: dig: command not found
INFO    [2015-07-27 12:22:38.62] container_uuid:08bce56b-9e85-4daa-9c8d-5b39c0d75d69; containerlog_message:2015-07-27 10:22:37 URL:http://proof.ovh.net/ [2272/2272] -> "/dev/null" [1]
INFO    [2015-07-27 12:22:38.62] container_uuid:08bce56b-9e85-4daa-9c8d-5b39c0d75d69; containerlog_message:nameserver   213.186.33.99
INFO    [2015-07-27 12:22:38.62] container_uuid:08bce56b-9e85-4daa-9c8d-5b39c0d75d69; containerlog_message:search   app.preprod.sailabove.io
INFO    [2015-07-27 12:22:38.62] container_uuid:08bce56b-9e85-4daa-9c8d-5b39c0d75d69; containerlog_message:bash: dig: command not found
INFO    [2015-07-27 12:22:38.63] container_uuid:08bce56b-9e85-4daa-9c8d-5b39c0d75d69; containerlog_message:2015-07-27 10:22:38 URL:http://proof.ovh.net/ [2272/2272] -> "/dev/null" [1]
INFO    [2015-07-27 12:22:38.63] container_uuid:08bce56b-9e85-4daa-9c8d-5b39c0d75d69; containerlog_message:bash: dig: command not found

#Getting Help