dcaegen2/collectors/hv-ves.git
5 years agoSimplify consul usage 07/82907/3
Filip Krzywka [Thu, 21 Mar 2019 10:12:17 +0000 (11:12 +0100)]
Simplify consul usage

Change-Id: I707b6c52eb356cab471215b534f1143be3e4cbae
Issue-ID: DCAEGEN2-710
Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
5 years agoMake security keys optional with SSL disabled 05/82805/5
Filip Krzywka [Wed, 20 Mar 2019 13:53:57 +0000 (14:53 +0100)]
Make security keys optional with SSL disabled

Change-Id: I0b0dd63cb3609cddf6aa5e63cf0a0f147a0aca91
Issue-ID: DCAEGEN2-1340
Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
5 years agoInclude basic configuration in Docker image 00/82900/2
Piotr Jaszczyk [Thu, 21 Mar 2019 08:10:11 +0000 (09:10 +0100)]
Include basic configuration in Docker image

Makes the collector start out of the box, even without valid CBS
configuration.

Note: the basic configuration will need to be changed later.

Issue-ID: DCAEGEN2-1340
Change-Id: I3c2b502f253677e78a4f4990002e683613b8869e
Signed-off-by: Piotr Jaszczyk <piotr.jaszczyk@nokia.com>
5 years agoRemove environment variables and program arguments 57/82157/10
Jakub Dudycz [Wed, 13 Mar 2019 17:44:31 +0000 (18:44 +0100)]
Remove environment variables and program arguments

- Move all command line program arguments to json file.
- Reorganize configuration classes and the way they are passed through application
- Implement HV VES configuration stream
- Create concrete configuration from partial one
- Modify main HV-VES server starting pipeline

Change-Id: I6cf874b6904ed768e4820b8132f5f760299c929e
Signed-off-by: Jakub Dudycz <jakub.dudycz@nokia.com>
Issue-ID: DCAEGEN2-1340

5 years agoUse Netty HttpServer in XnfApiServer 95/82395/5
Izabela Zawadzka [Fri, 15 Mar 2019 08:43:56 +0000 (09:43 +0100)]
Use Netty HttpServer in XnfApiServer

Change-Id: I86e06bd540c961098ee11af99735a5b35ce760fd
Issue-ID: DCAEGEN2-1325
Signed-off-by: Izabela Zawadzka <izabela.zawadzka@nokia.com>
5 years agoExtract HV-VES configuration module 19/82019/8
Filip Krzywka [Tue, 12 Mar 2019 10:50:37 +0000 (11:50 +0100)]
Extract HV-VES configuration module

To avoid cyclic dependency between "modules trio"
configuration-core-ssl some classes from core.model were also extracted.

Change-Id: Ie11029ae3500964f67f4d72279ddd68cdb2a1f0c
Issue-ID: DCAEGEN2-1332
Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
5 years agoUse proper Consul version in docker-compose.yml 39/82039/1
Piotr Bocheński [Tue, 12 Mar 2019 15:01:12 +0000 (16:01 +0100)]
Use proper Consul version in docker-compose.yml

Change-Id: I90c430c15832ef85cc6ef4e197cae141f246cf7b
Issue-ID: DCAEGEN2-1302
Signed-off-by: Piotr Bocheński <piotr.bochenski@nokia.com>
5 years agoUse netty HttpServer in DcaeAppApiServer 21/81921/6
Izabela Zawadzka [Fri, 8 Mar 2019 08:14:56 +0000 (09:14 +0100)]
Use netty HttpServer in DcaeAppApiServer

Change-Id: I895ad192babd9cc40266d0bec3830fcd4b0e054b
Issue-ID: DCAEGEN2-1325
Signed-off-by: Izabela Zawadzka <izabela.zawadzka@nokia.com>
5 years agoUse CBS by means of SDK in place of Consul 93/79393/12
Filip Krzywka [Thu, 28 Feb 2019 16:33:02 +0000 (17:33 +0100)]
Use CBS by means of SDK in place of Consul

- changed IO creation in main to fix error with too early calling
changeState method on collector HealthState
- increased coverage a little with few tests
- corrected coverage-report pom file to include new modules
- temporarily changed to 1.1.4-SNAPSHOT version of sdk due to need
of new version of CBSLookup

Change-Id: Ic73b46cf881ab4fabf52bef0327b09082aa90dc6
Issue-ID: DCAEGEN2-1302
Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
5 years agoMerge "Assure coverage is checked for all modules"
Piotr Wielebski [Mon, 11 Mar 2019 13:47:27 +0000 (13:47 +0000)]
Merge "Assure coverage is checked for all modules"

5 years agoAssure coverage is checked for all modules 38/81938/3
Piotr Jaszczyk [Mon, 11 Mar 2019 13:16:39 +0000 (14:16 +0100)]
Assure coverage is checked for all modules

Change-Id: Ia9bb6cef9154d64cfa45455e38c9fe7603a64c74
Issue-ID: DCAEGEN2-1328
Signed-off-by: Piotr Jaszczyk <piotr.jaszczyk@nokia.com>
5 years agoAdded JSON config file format utils 09/79309/10
pbiniek [Wed, 27 Feb 2019 22:31:15 +0000 (23:31 +0100)]
Added JSON config file format utils

Change-Id: I97fdd72324495b4c838e44c306cbcacac6b11bc1
Signed-off-by: Pawel Biniek <pawel.biniek@nokia.com>
Issue-ID: DCAEGEN2-1323

5 years agoRemove Guava 19 dependency via protobuf-java-util 16/79716/4
Jakub Dudycz [Tue, 5 Mar 2019 14:49:04 +0000 (15:49 +0100)]
Remove Guava 19 dependency via protobuf-java-util

Change-Id: Ic11cab5bce064817a954a462b3d2eea463198bd8
Signed-off-by: Jakub Dudycz <jakub.dudycz@nokia.com>
Issue-ID: DCAEGEN2-1311

5 years agoExtract module for ArgBasedConfiguration 58/79258/7
Jakub Dudycz [Wed, 27 Feb 2019 11:23:00 +0000 (12:23 +0100)]
Extract module for ArgBasedConfiguration

Change-Id: Ib11700650583378eb347451950a24c5f238382e4
Signed-off-by: Jakub Dudycz <jakub.dudycz@nokia.com>
Issue-ID: DCAEGEN2-1292

5 years agoUse DCAE SDK 1.1.3 (no snapshot) 11/79611/2
Piotr Jaszczyk [Mon, 4 Mar 2019 11:51:49 +0000 (12:51 +0100)]
Use DCAE SDK 1.1.3 (no snapshot)

Issue-ID: DCAEGEN2-1069
Change-Id: I48c1011873d31254df5e99ad970dd0ff18cc4c3f
Signed-off-by: Piotr Jaszczyk <piotr.jaszczyk@nokia.com>
5 years agoFix ssl related bug in xnf simulator 65/79465/5
Jakub Dudycz [Fri, 1 Mar 2019 16:39:09 +0000 (17:39 +0100)]
Fix ssl related bug in xnf simulator

Fix bug when xnf simnulator was using same SecurityKeys object instance for every new VesClient,
which resulted in fault while trying to connect to collector.

With new implementation simulator reuses same HvVesProdcuer from SDK
for every VesEvent request received and creates new Producer for every
WireFrameEvent request. This allows to continue testing cases in which
there is need to assert if connection was dropped from malicious client.

Change-Id: I5f51a58de85cccf7de6ab2392f86259502be31dd
Issue-ID: DCAEGEN2-1291
Signed-off-by: Jakub Dudycz <jakub.dudycz@nokia.com>
Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
5 years agoFix bug when xnf simulator was not sending events 47/79247/2
Jakub Dudycz [Wed, 27 Feb 2019 09:16:01 +0000 (10:16 +0100)]
Fix bug when xnf simulator was not sending events

Change-Id: I53cdf572d2df66d6021651424931a67108c482d3
Signed-off-by: Jakub Dudycz <jakub.dudycz@nokia.com>
Issue-ID: DCAEGEN2-1291

5 years agoUse sdk/hvves-producer in hvves/xnf-simulator 68/78768/3
Jakub Dudycz [Tue, 19 Feb 2019 17:06:33 +0000 (18:06 +0100)]
Use sdk/hvves-producer in hvves/xnf-simulator

Change-Id: I8f493b0edd2cbaef136a22d914ad24198bb63a7f
Signed-off-by: Jakub Dudycz <jakub.dudycz@nokia.com>
Issue-ID: DCAEGEN2-1253

5 years agoUse SDK/SSL in HV-VES 59/78559/5
Piotr Jaszczyk [Fri, 15 Feb 2019 11:59:26 +0000 (12:59 +0100)]
Use SDK/SSL in HV-VES

Issue-ID: DCAEGEN2-1226
Change-Id: I7cfc09001f7315c1b6f4fcf150ad631630c810ef
Signed-off-by: Piotr Jaszczyk <piotr.jaszczyk@nokia.com>
5 years agoMerge "Generate VesEvents in hv-ves/message-generator"
Piotr Jaszczyk [Mon, 18 Feb 2019 13:16:37 +0000 (13:16 +0000)]
Merge "Generate VesEvents in hv-ves/message-generator"

5 years agoGenerate VesEvents in hv-ves/message-generator 52/77952/8
Jakub Dudycz [Mon, 4 Feb 2019 14:20:14 +0000 (15:20 +0100)]
Generate VesEvents in hv-ves/message-generator

- Split message generator on two specialized generators
  for VesEvent and WireFrame related message types
- Refactor whole message-generator module

Change-Id: I1266b549a9a4d27213d03e8921298deab2dacb59
Signed-off-by: Jakub Dudycz <jakub.dudycz@nokia.com>
Issue-ID: DCAEGEN2-1162

5 years agoUse protobuf definitions from SDK 77/78377/4
Filip Krzywka [Wed, 13 Feb 2019 10:09:38 +0000 (11:09 +0100)]
Use protobuf definitions from SDK

Change-Id: I22163f3239769070e1e04c8d0e8f0da179069632
Issue-ID: DCAEGEN2-1218
Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
5 years agoAdd sdk to dependencies of XNF simulator 78/77778/4
kjaniak [Mon, 4 Feb 2019 11:52:21 +0000 (12:52 +0100)]
Add sdk to dependencies of XNF simulator

Change-Id: Ib1e96c71dcddcd5228f8d186a66c924a25e32dec
Issue-ID: DCAEGEN2-1163
Signed-off-by: kjaniak <kornel.janiak@nokia.com>
5 years agoCorrect totalConnections metric 72/76272/7
Filip Krzywka [Thu, 24 Jan 2019 10:21:26 +0000 (11:21 +0100)]
Correct totalConnections metric

In previous implementation performed healthcheck on container
was counted as client connection, because metric was notified
for every TCP channel opened. This was making this metric less useful
(to avoid saying "useless" ;) ).

- refactored NettyTcpServer trying to extract functions with logic
not so strictly related to HV-VES behavior. This also resolves
discussions in https://gerrit.onap.org/r/#/c/76274/
- some renames and methods splitting was made in attempt to make code
more readable
- hv-ves should not count connections from either "127.0.0.1" or
"localhost" to his `totalConnections` metric
- removed redundant logging by adding new methods to Logger

Change-Id: I5f10dac8dac82eafd3b0de92a7ec43f2c23b8c16
Issue-ID: DCAEGEN2-1119
Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
5 years agoChange Health Status when shutting down 81/76181/6
Piotr Jaszczyk [Wed, 23 Jan 2019 08:51:50 +0000 (09:51 +0100)]
Change Health Status when shutting down

Change-Id: I889c934f29d34c1211c6e07f18cb16bbe35d711f
Issue-ID: DCAEGEN2-1065
Signed-off-by: Piotr Jaszczyk <piotr.jaszczyk@nokia.com>
5 years agoMitigation of unexpected hv-ves shutdown 74/76274/4
kjaniak [Thu, 24 Jan 2019 06:29:50 +0000 (07:29 +0100)]
Mitigation of unexpected hv-ves shutdown

Change-Id: I96183e2358ac10457568d223f37d69c61dfd252d
Issue-ID: DCAEGEN2-1117
Signed-off-by: kjaniak <kornel.janiak@nokia.com>
5 years agoAdd grafana to docker-compose 86/76186/1
Filip Krzywka [Wed, 23 Jan 2019 11:04:57 +0000 (12:04 +0100)]
Add grafana to docker-compose

With default HV-VES dashboards included.

Change-Id: I8c15a2d609676218dce2e63d2a0730a359abf0af
Issue-ID: DCAEGEN2-1115
Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
5 years agoMerge "Close KafkaSender when handling SIGINT"
Piotr Wielebski [Tue, 22 Jan 2019 13:59:22 +0000 (13:59 +0000)]
Merge "Close KafkaSender when handling SIGINT"

5 years agoClose KafkaSender when handling SIGINT 16/76116/5
Piotr Jaszczyk [Tue, 22 Jan 2019 10:43:18 +0000 (11:43 +0100)]
Close KafkaSender when handling SIGINT

Closing KafkaSender should result in flushing any pending messages.

Change-Id: Ib251f5ca3527266831189df542784cc17173d8dc
Issue-ID: DCAEGEN2-1065
Signed-off-by: Piotr Jaszczyk <piotr.jaszczyk@nokia.com>
5 years agoEnable percentile histograms in metrics 20/76120/2
Filip Krzywka [Tue, 22 Jan 2019 11:56:53 +0000 (12:56 +0100)]
Enable percentile histograms in metrics

To allow more precise graphing.

Change-Id: I3c498b24c01e599eec316f552e25d06d653a08bb
Issue-ID: DCAEGEN2-1107
Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
5 years agoFix inconsistencies with logging guideline 06/75806/2
Piotr Jaszczyk [Tue, 15 Jan 2019 10:23:35 +0000 (11:23 +0100)]
Fix inconsistencies with logging guideline

Change-Id: I527ba0b3c313d946ae979b599e87bb4267a1793d
Issue-ID: DCAEGEN2-1085
Signed-off-by: Piotr Jaszczyk <piotr.jaszczyk@nokia.com>
5 years agoMerge "Align kafka properties with VESHV_MAX_PAYLOAD_SIZE"
Piotr Jaszczyk [Thu, 10 Jan 2019 10:38:17 +0000 (10:38 +0000)]
Merge "Align kafka properties with VESHV_MAX_PAYLOAD_SIZE"

5 years agoAlign kafka properties with VESHV_MAX_PAYLOAD_SIZE 58/75458/8
kjaniak [Tue, 8 Jan 2019 12:00:37 +0000 (13:00 +0100)]
Align kafka properties with VESHV_MAX_PAYLOAD_SIZE

Change-Id: I5cbfb8a982cd1efbdf58c2c0aed71f064f7b7cb8
Issue-ID: DCAEGEN2-1066
Signed-off-by: kjaniak <kornel.janiak@nokia.com>
5 years agoUpdate development scripts 03/75603/3
Filip Krzywka [Thu, 10 Jan 2019 06:59:56 +0000 (07:59 +0100)]
Update development scripts

This changeset is mostly follow-up after merges in DCAEGEN2-1063

- new options for hv-ves hostname and port in xnf simulations scripts
- new option for disabling SSL in xnf simulations scripts
- fail start-simulation script on not completing all simulations
- use docker-healthchecks to determine xNF simulator readiness in
development scripts
- fix logging in xNF simulator on closing connection
- minor corrections in logs and usage for development scripts

Change-Id: Ic939a139fdaf86ad487e0b31d6abcfc6c8a28885
Issue-ID: DCAEGEN2-1063
Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
5 years agoMerge "Align XNF simulator batch with Netty's queue size"
Piotr Jaszczyk [Tue, 8 Jan 2019 13:33:20 +0000 (13:33 +0000)]
Merge "Align XNF simulator batch with Netty's queue size"

5 years agoMerge "Handle sigterm signal"
Piotr Wielebski [Tue, 8 Jan 2019 12:43:25 +0000 (12:43 +0000)]
Merge "Handle sigterm signal"

5 years agoHandle sigterm signal 45/75445/3
Piotr Jaszczyk [Tue, 8 Jan 2019 10:00:41 +0000 (11:00 +0100)]
Handle sigterm signal

Change-Id: If6f431bfdc42f8d53497078b18813147cad1bad0
Issue-ID: DCAEGEN2-1065
Signed-off-by: Piotr Jaszczyk <piotr.jaszczyk@nokia.com>
5 years agoAlign XNF simulator batch with Netty's queue size 66/75366/2
Jakub Dudycz [Mon, 7 Jan 2019 14:30:34 +0000 (15:30 +0100)]
Align XNF simulator batch with Netty's queue size

This change will lessen the probability of out of memory errors

Change-Id: I96c735f4d7f5cedf3fddd66830e6557ea35ad1ea
Signed-off-by: Jakub Dudycz <jakub.dudycz@nokia.com>
Issue-ID: DCAEGEN2-1062

5 years agoMerge Healthcheck descriptions 31/75431/6
Filip Krzywka [Tue, 8 Jan 2019 06:53:25 +0000 (07:53 +0100)]
Merge Healthcheck descriptions

- start new API server for healthchecks in xNF simulator on port 6063
- changed DCAE App default port to 6064
- switched to InetSocketAddresses usage in components configurations

Change-Id: I398f9ea6e887f78d88286ed717d310d3297b1571
Issue-ID: DCAEGEN2-1063
Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
5 years agoMerge "Change AtLevelLoggers to be initialized by lazy"
Piotr Jaszczyk [Fri, 4 Jan 2019 08:50:59 +0000 (08:50 +0000)]
Merge "Change AtLevelLoggers to be initialized by lazy"

5 years agoChange AtLevelLoggers to be initialized by lazy 18/75218/3
Izabela Zawadzka [Thu, 3 Jan 2019 13:49:54 +0000 (14:49 +0100)]
Change AtLevelLoggers to be initialized by lazy

Change-Id: I3e9565d432c6df8dbc357caa61c10204f69f2e2a
Signed-off-by: Izabela Zawadzka <izabela.zawadzka@nokia.com>
Issue-ID: DCAEGEN2-1053

5 years agoAdd script for sending messages from multiple xNFs 81/74981/8
Filip Krzywka [Tue, 18 Dec 2018 09:10:06 +0000 (10:10 +0100)]
Add script for sending messages from multiple xNFs

Change-Id: I9acd3278929c30a3036f97f403a31a8817723d27
Issue-ID: DCAEGEN2-1027
Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
5 years agoRename of metrics 09/75109/4
kjaniak [Thu, 27 Dec 2018 08:01:47 +0000 (09:01 +0100)]
Rename of metrics

Names of metrics were a bit misleading e.g. total and count addition
to name of metrics. Clean up in unit tests.

Change-Id: I339c8824f31a226bdbe648027b114321e3fd9858
Issue-ID: DCAEGEN2-1046
Signed-off-by: kjaniak <kornel.janiak@nokia.com>
5 years agoMerge "Enable setting log level from command line"
Piotr Jaszczyk [Fri, 21 Dec 2018 11:02:12 +0000 (11:02 +0000)]
Merge "Enable setting log level from command line"

5 years agoEnable setting log level from command line 75/74875/4
Izabela Zawadzka [Wed, 19 Dec 2018 06:25:33 +0000 (07:25 +0100)]
Enable setting log level from command line

Change-Id: I8397e0134d254cd5b6be79ed2b847ce265fc775c
Signed-off-by: Izabela Zawadzka <izabela.zawadzka@nokia.com>
Issue-ID: DCAEGEN2-1045

5 years agoFix misleading logs when obtaining consul config 93/74993/4
Jakub Dudycz [Thu, 20 Dec 2018 15:43:22 +0000 (16:43 +0100)]
Fix misleading logs when obtaining consul config

Change-Id: Idaccc954a0fdc03c6336e6f126f8c7e99010956d
Signed-off-by: Jakub Dudycz <jakub.dudycz@nokia.com>
Issue-ID: DCAEGEN2-1048

5 years agoThere should be one KafkaSender per configuration 69/74969/8
Piotr Jaszczyk [Tue, 18 Dec 2018 14:58:56 +0000 (15:58 +0100)]
There should be one KafkaSender per configuration

We should keep only one instance of KafkaSender per instance. However,
   as the configuration might be changed (Consul update) it cannot be a
   strict singleton. Hence there should be 1to1 relationship beetween
   ConsulConfiguration and KafkaSender.

Change-Id: Ie168028c4427741254b8c2fe316b82cca72d7668
Issue-ID: DCAEGEN2-1047
Signed-off-by: Piotr Jaszczyk <piotr.jaszczyk@nokia.com>
5 years agoAdd metrics for active connections count 22/74722/8
Jakub Dudycz [Mon, 17 Dec 2018 15:03:10 +0000 (16:03 +0100)]
Add metrics for active connections count

* Fix and refactor gauges tests in MicrometerMetricsTests
  as they were not executing
* Fix client disconnection handler in NettyTcpServer
* Add metrics gauge and counters required to measure active connections

Change-Id: I5620d398525c6859679cd5a49dc55a9fefd8b592
Signed-off-by: Jakub Dudycz <jakub.dudycz@nokia.com>
Issue-ID: DCAEGEN2-1041

5 years agoMetric: Message latency 18/74718/5
Piotr Jaszczyk [Mon, 17 Dec 2018 12:22:52 +0000 (13:22 +0100)]
Metric: Message latency

Defined as a difference between now and vesHeader.lastEpochTime.

Change-Id: I4aa97e8efc13cb0039fde38b4fd2aa6411c7b89a
Issue-ID: DCAEGEN2-1036
Signed-off-by: Piotr Jaszczyk <piotr.jaszczyk@nokia.com>
5 years agoAdd metric for rejected clients count 20/74720/5
Filip Krzywka [Mon, 17 Dec 2018 13:25:56 +0000 (14:25 +0100)]
Add metric for rejected clients count

- renamed few counters to be more verbose about what they count
- removed not needed 'total' suffix in metrics name

Change-Id: I6be0201e5f39f1298706c536b12410413d49df19
Issue-ID: DCAEGEN2-1043
Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
5 years agoMetric: Processing time 55/74655/8
Piotr Jaszczyk [Fri, 14 Dec 2018 11:05:47 +0000 (12:05 +0100)]
Metric: Processing time

Add processing time metric measured as difference between "sent to DMaaP" and "WTP decoded" events.

Change-Id: I73bb665145019fcca5ae36e2199ed0e1cc088fdf
Issue-ID: DCAEGEN2-1036
Signed-off-by: Piotr Jaszczyk <piotr.jaszczyk@nokia.com>
5 years agoMerge "Add metrics for dropped messages"
Piotr Jaszczyk [Mon, 17 Dec 2018 12:59:49 +0000 (12:59 +0000)]
Merge "Add metrics for dropped messages"

5 years agoAdd metrics for dropped messages 67/74667/9
Jakub Dudycz [Fri, 14 Dec 2018 14:20:56 +0000 (15:20 +0100)]
Add metrics for dropped messages

Add counters for messages dropped due to validation or undefined routing
Slight refactoring

Change-Id: Ibe4e38445e81babc745d7a7d95356910845293ce
Signed-off-by: Jakub Dudycz <jakub.dudycz@nokia.com>
Issue-ID: DCAEGEN2-1037

5 years agoMerge "Use SHA-256 for detecting changes in Consul config"
Piotr Wielebski [Fri, 14 Dec 2018 11:26:54 +0000 (11:26 +0000)]
Merge "Use SHA-256 for detecting changes in Consul config"

5 years agoCreate component tests skeleton for metrics 48/74648/2
Filip Krzywka [Thu, 13 Dec 2018 10:22:29 +0000 (11:22 +0100)]
Create component tests skeleton for metrics

Change-Id: I5acc919171953a4bb428585a7b8420b3223aa556
Issue-ID: DCAEGEN2-1032
Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
5 years agoUse SHA-256 for detecting changes in Consul config 47/74647/2
Piotr Jaszczyk [Fri, 14 Dec 2018 08:10:36 +0000 (09:10 +0100)]
Use SHA-256 for detecting changes in Consul config

String.hashCode is probably not strong enough and was not designed to be
used in that way.

Change-Id: Iecb75dcdc742d1673b81cbaa213cff69297ff6f4
Issue-ID: DCAEGEN2-1034
Signed-off-by: Piotr Jaszczyk <piotr.jaszczyk@nokia.com>
5 years agoMerge "Add all required and reasonable MDCs"
Piotr Wielebski [Fri, 14 Dec 2018 08:38:03 +0000 (08:38 +0000)]
Merge "Add all required and reasonable MDCs"

5 years agoMerge "Expose Prometheus metrics"
Piotr Wielebski [Fri, 14 Dec 2018 08:37:24 +0000 (08:37 +0000)]
Merge "Expose Prometheus metrics"

5 years agoMerge "Allow to specify JVM options by means of env-var"
Piotr Wielebski [Fri, 14 Dec 2018 08:37:08 +0000 (08:37 +0000)]
Merge "Allow to specify JVM options by means of env-var"

5 years agoAdd all required and reasonable MDCs 03/74603/10
Piotr Jaszczyk [Thu, 13 Dec 2018 08:26:36 +0000 (09:26 +0100)]
Add all required and reasonable MDCs

Change-Id: I34beb32a7c53da97c6945ec8d0022ac37059b7c5
Issue-ID: DCAEGEN2-670
Signed-off-by: Piotr Jaszczyk <piotr.jaszczyk@nokia.com>
5 years agoMerge "Don't escape in component tests"
Piotr Jaszczyk [Fri, 14 Dec 2018 06:29:41 +0000 (06:29 +0000)]
Merge "Don't escape in component tests"

5 years agoExpose Prometheus metrics 62/74562/7
Piotr Jaszczyk [Wed, 12 Dec 2018 08:07:05 +0000 (09:07 +0100)]
Expose Prometheus metrics

Expose both JVM and HV-VES metrics by means of HealthCheckApiServer in
Prometheus format.

Change-Id: I81b74cd945c67569703a4a74f5a2683ede18d64a
Issue-ID: DCAEGEN2-1029
Signed-off-by: Piotr Jaszczyk <piotr.jaszczyk@nokia.com>
5 years agoMerge "Log details about header validation failure"
Piotr Jaszczyk [Thu, 13 Dec 2018 13:20:51 +0000 (13:20 +0000)]
Merge "Log details about header validation failure"

5 years agoLog details about header validation failure 40/74340/6
Izabela Zawadzka [Thu, 6 Dec 2018 10:21:59 +0000 (11:21 +0100)]
Log details about header validation failure

Change-Id: I836dcbee47bc2c2bfecc9486f6f8344a5bd8cd65
Signed-off-by: Izabela Zawadzka <izabela.zawadzka@nokia.com>
Issue-ID: DCAEGEN2-1019

5 years agoDon't escape in component tests 10/74610/1
Filip Krzywka [Thu, 13 Dec 2018 13:03:37 +0000 (14:03 +0100)]
Don't escape in component tests

We want our test output to be readable, so we should not escape
\n or \t in console output.

Change-Id: I4c673760ea0cd55007b47b7490036c0693887eda
Issue-ID: DCAEGEN2-1026
Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
5 years agoMerge "Align logback configuration"
Piotr Jaszczyk [Thu, 13 Dec 2018 09:16:18 +0000 (09:16 +0000)]
Merge "Align logback configuration"

5 years agoAlign logback configuration 60/74560/6
Filip Krzywka [Wed, 12 Dec 2018 07:11:30 +0000 (08:11 +0100)]
Align logback configuration

- followed current logging guidelines in regard of format
- in simulators currently very simple (minimalistic) pattern will
be used as they do not contain any MDCs or Markers
- in main logbacks included three patterns varying on verbosity and
human-readability (for production full logging, ie. ONAP_LOG_PATTERN,
should be used)
- removed logback-test from domain module as it was not enabled anyway
(no dependency on logback) and we should never have need for logging
in this module
- made logback-tests consistent (using simple pattern) except in
component-tests, where it might be useful to have more verbose
output (including MDCs etc.)

Change-Id: Ifd851d12a77d5303b836a25c0e168e18a3a804e9
Issue-ID: DCAEGEN2-1026
Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
5 years agoMerge "Add of adapter fun with param ClientContext"
Piotr Jaszczyk [Thu, 13 Dec 2018 08:26:21 +0000 (08:26 +0000)]
Merge "Add of adapter fun with param ClientContext"

5 years agoAdd of adapter fun with param ClientContext 00/74600/2
kjaniak [Thu, 13 Dec 2018 06:39:51 +0000 (07:39 +0100)]
Add of adapter fun with param ClientContext

Change-Id: Ib9ac6ab16c51ddf40ae849f4ce04ab7e25609f56
Issue-ID: DCAEGEN2-1031
Signed-off-by: kjaniak <kornel.janiak@nokia.com>
5 years agoAllow to specify JVM options by means of env-var 59/74559/2
Piotr Jaszczyk [Wed, 12 Dec 2018 06:48:42 +0000 (07:48 +0100)]
Allow to specify JVM options by means of env-var

Change-Id: I042e022d650a07804096b397c31abc82a9633089
Issue-ID: DCAEGEN2-1025
Signed-off-by: Piotr Jaszczyk <piotr.jaszczyk@nokia.com>
5 years agoAdd helper scripts 61/74561/5
Filip Krzywka [Wed, 12 Dec 2018 07:14:57 +0000 (08:14 +0100)]
Add helper scripts

Change-Id: I8d5b48511e1b7f9cb7d1f65e8605ebe4cf1b17fc
Issue-ID: DCAEGEN2-1027
Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
5 years agoAdd Markers logs 50/74450/4
Filip Krzywka [Mon, 10 Dec 2018 14:18:50 +0000 (15:18 +0100)]
Add Markers logs

Change-Id: I5d3c9e1e4659782719e4847c5bcedbca39c173a0
Issue-ID: DCAEGEN2-669
Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
5 years agoAdd log diagnostic context 92/74392/6
Piotr Jaszczyk [Fri, 7 Dec 2018 13:41:39 +0000 (14:41 +0100)]
Add log diagnostic context

As it's not trivial to use MDC directly from logging framework in
reactive application, we need to do some work manually. The approach
proposed is an explicit MDC handling, which means that context is
kept as an object created after establishing client connection. Next,
new instance of HvVesCollector (and its dependencies) is created. Every
object is propagated with ClientContext so it can use it when calling
logger methods.

In the future ClientContext might be used to support other use-cases,
ie. per-topic access control.

As a by-product I had to refactor our Logger wrapper, too. It already
had too many functions and after adding MDC number would be doubled.

Change-Id: I9c5d3f5e1d1be1db66d28d292eb0e1c38d8d0ffe
Issue-ID: DCAEGEN2-671
Signed-off-by: Piotr Jaszczyk <piotr.jaszczyk@nokia.com>
5 years agoUpdate INFO.yaml 72/74372/1
Jessica Wagantall [Thu, 6 Dec 2018 23:34:01 +0000 (15:34 -0800)]
Update INFO.yaml

Add Piotr Jaszczyk and Piotr Wielebski
Remove Przemyslaw Wasala

Change-Id: I2b8fae7a3f3423dd9877e38d3ddc765d6b343291
Issue-ID: CIMAN-33
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
5 years agoEnhance of logging in test part 24/73924/17
kjaniak [Thu, 29 Nov 2018 12:44:50 +0000 (13:44 +0100)]
Enhance of logging in test part

Change-Id: Ibdec0ac5cead7e46ada4c32983b9ccf962df703b
Issue-ID: DCAEGEN2-1004
Signed-off-by: kjaniak <kornel.janiak@nokia.com>
5 years agoHarmonize logging and add new logs 07/73907/10
Filip Krzywka [Thu, 29 Nov 2018 10:58:40 +0000 (11:58 +0100)]
Harmonize logging and add new logs

- corrected docker-compose consul url

Change-Id: I78df868e0dd51008ef39d01553e6a0a3b8273a54
Issue-ID: DCAEGEN2-1003
Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
5 years agoCustom detekt rule for logger usage check 91/73891/6
Piotr Jaszczyk [Wed, 28 Nov 2018 14:46:50 +0000 (15:46 +0100)]
Custom detekt rule for logger usage check

Check if logger invocations don't use unoptimal invocations, eg.
concatenation `debug("a=" + a)` instead of lambda use `debug {"a=" + a}`

Unfortunately to avoid defining dependencies in many places and having
circural dependencies it was necessarry to reorganize the maven module
structure. The goal was to have `sources` module with production code and
`build` module with build-time tooling (detekt rules among them).

Issue-ID: DCAEGEN2-1002
Change-Id: I36e677b98972aaae6905d722597cbce5e863d201
Signed-off-by: Piotr Jaszczyk <piotr.jaszczyk@nokia.com>
5 years agoBump checkstyle version 68/73668/3
Filip Krzywka [Tue, 27 Nov 2018 12:04:48 +0000 (13:04 +0100)]
Bump checkstyle version

- changed plugin invocation due to backwards incompatible
changes made in detekt's RC9 version
- disabled few rules that we actually don't follow in our code and
we seem to be fine with it
- turned on UnusedPrivateMember as it actually is a code smell/debt
- supressed warnings in where it was plausible (according to common sense)
- fixed warnings wherever possible

Change-Id: I7be97f471ff46786ef1fca3432b759e7820ac681
Issue-ID: DCAEGEN2-996
Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
5 years agoMerge "Remove Consul configuration decoding"
Vijay Venkatesh Kumar [Tue, 27 Nov 2018 23:28:54 +0000 (23:28 +0000)]
Merge "Remove Consul configuration decoding"

5 years agoMerge "Fix tagging for CSITs"
Vijay Venkatesh Kumar [Tue, 27 Nov 2018 23:22:05 +0000 (23:22 +0000)]
Merge "Fix tagging for CSITs"

5 years agoMerge "Add healthcheck script"
Vijay Venkatesh Kumar [Tue, 27 Nov 2018 23:21:22 +0000 (23:21 +0000)]
Merge "Add healthcheck script"

5 years agoFix tagging for CSITs 28/73228/2
Filip Krzywka [Mon, 19 Nov 2018 15:17:05 +0000 (16:17 +0100)]
Fix tagging for CSITs

- latest tag should not be used to prevent confusion with casablanca,
instead 1.1-SNAPSHOT should when referring to latests (Dublin) build
- this allows for less frequent changes in CSIT
repository after code freeze (when changes need to
update service version)

Change-Id: I4c94a5a2e90d0c77d51110ed97c9e28c30130f10
Issue-ID: DCAEGEN2-983
Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
5 years agoRemove Consul configuration decoding 27/68627/4
Filip Krzywka [Mon, 24 Sep 2018 08:42:47 +0000 (10:42 +0200)]
Remove Consul configuration decoding

Change-Id: I36a6aea55507abfb61613806c93257b049d6eea4
Issue-ID: DCAEGEN2-827
Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
5 years agoAdd healthcheck script 63/73363/4
Filip Krzywka [Thu, 22 Nov 2018 10:09:03 +0000 (11:09 +0100)]
Add healthcheck script

To be used in HV-VES deployment template

Change-Id: I87e1d52db2bb530e131837052f55a29adb5d524b
Issue-ID: DCAEGEN2-794
Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
5 years agoFix xNF simulator sslContext 36/73136/2
Filip Krzywka [Tue, 20 Nov 2018 12:58:12 +0000 (13:58 +0100)]
Fix xNF simulator sslContext

If --ssl-disable flag is specified in command line arguments, we shouldn't
put sslContext into TcpClient as this results in IllegalArgument.

Change-Id: I2e2d2ea89736f1e372e1b408c8f3cbe50bf1ecea
Issue-ID: DCAEGEN2-985
Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
5 years agoUpdate proto file to v14 23/73123/2
Jakub Dudycz [Tue, 20 Nov 2018 10:43:23 +0000 (11:43 +0100)]
Update proto file to v14

- update proto file
- update vesEventListenerVersion validation regex
  to match new requirements

Change-Id: Ic0bd855fd03caa3ca3120b2c49d5fa4f53de2540
Signed-off-by: Jakub Dudycz <jakub.dudycz@nokia.com>
Issue-ID: DCAEGEN2-984

5 years agoMerge "Fix Common Event Header fields validation"
Przemyslaw Wasala [Mon, 19 Nov 2018 10:28:28 +0000 (10:28 +0000)]
Merge "Fix Common Event Header fields validation"

5 years agoFix Common Event Header fields validation 75/72775/6
Jakub Dudycz [Thu, 15 Nov 2018 12:29:09 +0000 (13:29 +0100)]
Fix Common Event Header fields validation

- "sequence" is no longer a required parameter,
  since deafult value "0" is acceptable
- "vesEventListenerVersion" has to match the regular expression "7\.[0-9]+\.[0-9]+"

Signed-off-by: Jakub Dudycz <jakub.dudycz@nokia.com>
Issue-ID: DCAEGEN2-976
Change-Id: I2f9fd6f375ccca3255cc9e035918dc37cc97bd6a

5 years agoUpdate project and dependencies 79/72279/5
Piotr Jaszczyk [Wed, 7 Nov 2018 14:08:43 +0000 (15:08 +0100)]
Update project and dependencies

* Changed version from 4.0.0-SNAPSHOT to 1.1.0-SNAPSHOT as per Vijay
  suggestion
* Updated Reactor to BOM Californium-SR2
* Updated mockito-kotlin to 2.0.0
* Introduced some fixes to support OpenJDK 11 compilation

Change-Id: Ib25979ef50c7241a019bf98efd9759e0b8792d58
Issue-ID: DCAEGEN2-961
Signed-off-by: Piotr Jaszczyk <piotr.jaszczyk@nokia.com>
5 years agoHandle stream error early 40/72040/6
Filip Krzywka [Wed, 7 Nov 2018 07:16:09 +0000 (08:16 +0100)]
Handle stream error early

Should fix inconsistent logging due to Reactor Signal sometimes
propagating from WireChunkDecoder stream to VesHvCollector stream as
Signal.CANCEL instead of Signal.ERROR and thus not being handled
correctly.
As a drawback however we will log error twice in case it comes from
WireChunkDecoder as we want to terminate connection in such case and
so we need to propagate error.

In WireChunkDecoder `doOnTerminate` was changed to
`doFinally` as this method handles also cancellation
signals and not only terminal signals.

Also fixed minor checkstyle reported issues.

Change-Id: I6e91d96c5a1a3ecf30603db9a71e032c770d507f
Issue-ID: DCAEGEN2-955
Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
5 years agoMerge "Switch to JRE 11 in Dockerfiles"
Przemyslaw Wasala [Fri, 9 Nov 2018 13:52:17 +0000 (13:52 +0000)]
Merge "Switch to JRE 11 in Dockerfiles"

5 years agoSwitch to JRE 11 in Dockerfiles 95/72295/3
Jakub Dudycz [Fri, 9 Nov 2018 10:30:27 +0000 (11:30 +0100)]
Switch to JRE 11 in Dockerfiles

Change-Id: I3f8cc792677f01b6f4eeb7cc36822b6a9ed482d9
Signed-off-by: Jakub Dudycz <jakub.dudycz@nokia.com>
Issue-ID: DCAEGEN2-969

5 years agoUpdate protobuf version to 3.6.1 88/72088/1
Jakub Dudycz [Wed, 7 Nov 2018 15:36:17 +0000 (16:36 +0100)]
Update protobuf version to 3.6.1

Update protobuf and maven protobuf plugin versions

Change-Id: I12dbc71926d74eb78ef4553e4b7abe72d34274e0
Signed-off-by: Jakub Dudycz <jakub.dudycz@nokia.com>
Issue-ID: DCAEGEN2-962

5 years agoUpgrade Kotlin to 1.3 27/71927/1
Piotr Jaszczyk [Tue, 6 Nov 2018 09:09:32 +0000 (10:09 +0100)]
Upgrade Kotlin to 1.3

Migrate to Kotlin 1.3 with non-experimental coroutines support. Also
bump Arrow-Kt version so it follows 1.3 release.

Change-Id: Ia2f26d0c710b7cfae709e35700b72607aeddaf00
Issue-ID: DCAEGEN2-949
Signed-off-by: Piotr Jaszczyk <piotr.jaszczyk@nokia.com>
5 years agoChange version numbers for Dublin release 48/71848/1
Piotr Jaszczyk [Mon, 5 Nov 2018 14:08:50 +0000 (15:08 +0100)]
Change version numbers for Dublin release

Change-Id: I4e1a53ea95ec8d880ff00aa94d3ba5511c23eadf
Issue-ID: DCAEGEN2-943
Signed-off-by: Piotr Jaszczyk <piotr.jaszczyk@nokia.com>
5 years agoRemove any inconsistencies with specification 96/69896/1 1.0.0 3.0.0-ONAP
Piotr Jaszczyk [Fri, 5 Oct 2018 08:19:57 +0000 (10:19 +0200)]
Remove any inconsistencies with specification

* Fix payloadId field issue (was 1 byte, should use 2 byts)
* Copy final version of protobuf and asn definitions (mostly comments)
* Added links to yet-to-be updated RTD documentation

Change-Id: I69bda676423ad601797d95577ff8af6707cacb0c
Issue-ID: DCAEGEN2-857
Signed-off-by: Piotr Jaszczyk <piotr.jaszczyk@nokia.com>
5 years agoIntroduce configurable payload size limitation 93/69893/1
Piotr Jaszczyk [Fri, 5 Oct 2018 07:02:06 +0000 (09:02 +0200)]
Introduce configurable payload size limitation

Maximum payload size will be configurable (from command line parameter
or environment variable). The default value is same as previous
hardcoded value, ie. 1 MiB = 1024 * 1024 bytes.

Change-Id: Iec83d8295252bac353d3794b13454fdbbc80ecc4
Issue-ID: DCAEGEN2-856
Signed-off-by: Piotr Jaszczyk <piotr.jaszczyk@nokia.com>
5 years agoChange magic byte from 0xFF to 0xAA 89/69789/1
Piotr Jaszczyk [Thu, 4 Oct 2018 06:04:00 +0000 (08:04 +0200)]
Change magic byte from 0xFF to 0xAA

In final protocol specification the value has changed to 0xAA.

Change-Id: Id86c15117732b03bca38c674960d5eba8da88a6e
Issue-ID: DCAEGEN2-854
Signed-off-by: Piotr Jaszczyk <piotr.jaszczyk@nokia.com>
5 years agoChange domain to lower case 20/69720/2
Piotr Jaszczyk [Wed, 3 Oct 2018 08:33:40 +0000 (10:33 +0200)]
Change domain to lower case

Change-Id: I097def2ba720fafe0a1e1a88e87f8c03663627c9
Issue-ID: DCAEGEN2-847
Signed-off-by: Piotr Jaszczyk <piotr.jaszczyk@nokia.com>
5 years agoHV-VES Domain update 68/69568/5
Piotr Jaszczyk [Mon, 1 Oct 2018 05:54:26 +0000 (07:54 +0200)]
HV-VES Domain update

Change domain name. In this project this yields following changes:
* Comment change in VesEvent proto
* Updated measurements proto (kept as documentation only, ie. not used
    in the collector)
* Update name in test scenarios for unit and component scope

Change-Id: I719077b5fcb642c067d25b7aa174a2e629404689
Issue-ID: DCAEGEN2-847
Signed-off-by: Piotr Jaszczyk <piotr.jaszczyk@nokia.com>