4 ELK stack for CLAMP : Logstash is used to retrieve events and notifications from DMaaP and pushes them into Elasticsearch.
5 Kibana is then used to extract statistics.
8 Deployment instructions
9 -----------------------
11 Requirements: docker-compose
13 1. Update configuration in docker-compose file
14 2. `docker-compose up -d elasticsearch logstash kibana`
16 If you encounter connection problems with kibana, first deploy elasticsearch, wait for it to be available, then kibana.
21 backup.py and restore.py scripts are available inside the kibana docker image for saving and restoring the configuration.
22 the default configuration is located in the kibana docker image under the directory "/saved-objects/default/"
26 firts create the directory "/saved-objects/mybackup" if it doesn't exist
27 docker-compose exec kibana backup.py -C /saved-objects/mybackup
30 usage: backup.py [-h] [-v] [-C CONFIGURATION_PATH] [-f] [-H KIBANA_HOST]
32 Description of the script
35 -h, --help show this help message and exit
36 -v, --verbose Use verbose logging
37 -C CONFIGURATION_PATH, --configuration_path CONFIGURATION_PATH
38 Path of the configuration to be backed up.
39 -f, --force If the save folder already exists, overwrite files
40 matching a configuration item that should be written.
41 Files already in the folder that do not match are left
43 -H KIBANA_HOST, --kibana-host KIBANA_HOST
50 to restore the configuration, you previously backed up, use the command below:
51 docker-compose exec kibana restore.py -C /saved-objects/mybackup -f
52 to restore the default confgiuration use the command below:
53 docker-compose exec kibana restore.py -C /saved-objects/default -f
56 usage: restore.py [-h] [-v] [-C CONFIGURATION_PATH] [-H KIBANA_HOST] [-f]
58 Restores the kibana configuration.
61 -h, --help show this help message and exit
62 -v, --verbose Use verbose logging
63 -C CONFIGURATION_PATH, --configuration_path CONFIGURATION_PATH
64 Path of the configuration to be restored.Should
65 contain at least one folder named index-
66 pattern,config,search,visualization or dashboard
67 -H KIBANA_HOST, --kibana-host KIBANA_HOST
69 -f, --force Overwrite configuration if needed.
75 The following tools are available in the 'tools/' folder.
80 Small script ease Elasticsearch /painless/ field development.
81 It reads a json file as a query for Elasticsearch, pushes it on the ES server, and display back the answer in a loop, each time the file is modified.
84 ### DMaaP Service Mocker
86 Script that simulates control loop DMaaP services to provide sample data to logstash through DMaaP.
90 * Add a script that verifies that elasticsearch is available before starting loading the default configuration for kibana