dcaegen2/collectors/hv-ves.git
4 years agoAdding commiter rights to project 52/101452/1
kjaniak [Mon, 10 Feb 2020 12:52:49 +0000 (13:52 +0100)]
Adding commiter rights to project

Kornel Janiak added to HV-VES commiters group.

Change-Id: I839d48ab3a0fe6a31a7318b74f7811c576a523c1
Issue-ID: DCAEGEN2-1891
Signed-off-by: kjaniak <kornel.janiak@nokia.com>
4 years agoPatch version increased (1.4.0 -> 1.4.1) 94/100894/2
Joanna Jeremicz [Wed, 29 Jan 2020 11:58:11 +0000 (12:58 +0100)]
Patch version increased (1.4.0 -> 1.4.1)

Issue-ID: DCAEGEN2-1982
Signed-off-by: Joanna Jeremicz <joanna.jeremicz@nokia.com>
Change-Id: I1c26cd3384ac443984cfbfed9b0bed722e1b31eb

4 years agoRelease 1.4.0 version 62/100362/2
Joanna Jeremicz [Thu, 16 Jan 2020 10:38:50 +0000 (11:38 +0100)]
Release 1.4.0 version

Issue-ID: DCAEGEN2-2013
Signed-off-by: Joanna Jeremicz <joanna.jeremicz@nokia.com>
Change-Id: I9702821796954872626161e955f31492602c2946

4 years agoUpdate sdk version to 1.3.4 64/100364/1 1.4.0
Joanna Jeremicz [Thu, 16 Jan 2020 10:48:56 +0000 (11:48 +0100)]
Update sdk version to 1.3.4

Issue-ID: DCAEGEN2-1982
Signed-off-by: Joanna Jeremicz <joanna.jeremicz@nokia.com>
Change-Id: Ida298b7ae31cd3cd3bbf34763260b7de209c19ac

4 years agoChange hv-ves version after last release 10/100310/1
kjaniak [Wed, 15 Jan 2020 10:30:18 +0000 (11:30 +0100)]
Change hv-ves version after last release

Since this version will include java 11 minor version top up.

Change-Id: Ie45f1a6afc33c2199d0a3f4d048420769158a6e6
Issue-ID: DCAEGEN2-1982
Signed-off-by: kjaniak <kornel.janiak@nokia.com>
4 years agoTop up Java Version to 11 29/100129/4
pwielebs [Thu, 9 Jan 2020 13:18:35 +0000 (14:18 +0100)]
Top up Java Version to 11

Signed-off-by: pwielebs <piotr.wielebski@nokia.com>
Issue-ID: DCAEGEN2-1982
Change-Id: I5e4d0cdd11741186909db7a2d3b600202c3c0d63

4 years agoRelease hv-ves 1.3.1 74/99474/3
Joanna Jeremicz [Wed, 11 Dec 2019 09:03:54 +0000 (10:03 +0100)]
Release hv-ves 1.3.1

Issue-ID: DCAEGEN2-1823
Signed-off-by: Joanna Jeremicz <joanna.jeremicz@nokia.com>
Change-Id: Id8607e555a72250de2e699c7204897fc2ee86cb3

4 years agoPrepare release 1.3.1 03/99403/2
Joanna Jeremicz [Tue, 10 Dec 2019 10:12:50 +0000 (11:12 +0100)]
Prepare release 1.3.1

Issue-ID: DCAEGEN2-1823
Signed-off-by: Joanna Jeremicz <joanna.jeremicz@nokia.com>
Change-Id: Ifb02692ac982f28a916fa0e70548216e891216fe

4 years agoSDK version 1.3.2 -> 1.3.3 00/99200/1 1.3.1
Joanna Jeremicz [Thu, 5 Dec 2019 10:03:35 +0000 (11:03 +0100)]
SDK version 1.3.2 -> 1.3.3

Issue-ID: DCAEGEN2-1823
Signed-off-by: Joanna Jeremicz <joanna.jeremicz@nokia.com>
Change-Id: Id9775115edf8ba205789f8abfa328b84e4f5e038

4 years agoUpdate DCAE Committer in INFO.yaml 95/99095/1
JoeOLeary [Tue, 3 Dec 2019 21:05:01 +0000 (21:05 +0000)]
Update DCAE Committer in INFO.yaml

Issue-ID: DCAEGEN2-1891
Signed-off-by: JoeOLeary <joseph.o.leary@est.tech>
Change-Id: I3c79f8af0298ccedb3e06f4acc9e701d88c7a100

4 years agoFix HV-VES static code vulnerabilities 83/98583/6
Joanna Jeremicz [Tue, 19 Nov 2019 06:47:26 +0000 (07:47 +0100)]
Fix HV-VES static code vulnerabilities

- Update kafka-clients version
- Update SDK version
- Update reactor-bom version

Issue-ID: DCAEGEN2-1823
Signed-off-by: Joanna Jeremicz <joanna.jeremicz@nokia.com>
Change-Id: Ib2d49c46e2d1b58d5131b9af6fc27804d443da71

4 years agoMerge "Improve performance test environment rebooting script"
Tony Hansen [Fri, 15 Nov 2019 15:26:43 +0000 (15:26 +0000)]
Merge "Improve performance test environment rebooting script"

4 years agoImprove performance test environment rebooting script 44/98144/4
Remigiusz Janeczek [Thu, 7 Nov 2019 15:17:11 +0000 (16:17 +0100)]
Improve performance test environment rebooting script

Issue-ID: DCAEGEN2-1883
Signed-off-by: Remigiusz Janeczek <remigiusz.janeczek@nokia.com>
Change-Id: I841ee6326467d495dc40686bca0f22800cb4ac61

4 years agoMerge "Revert - Fix memory usage per pod table"
Tony Hansen [Tue, 12 Nov 2019 04:04:43 +0000 (04:04 +0000)]
Merge "Revert - Fix memory usage per pod table"

4 years agoFix for 30s bucket problem in metrics 33/98133/1
kjaniak [Thu, 7 Nov 2019 11:29:12 +0000 (12:29 +0100)]
Fix for 30s bucket problem in metrics

Change-Id: I9a6b362f59a47454cb36d9a54861d1bcc1b934be
Issue-ID: DCAEGEN2-1890
Signed-off-by: kjaniak <kornel.janiak@nokia.com>
4 years agoMerge "Add graph for processing time without routing"
Piotr Wielebski [Mon, 4 Nov 2019 15:03:15 +0000 (15:03 +0000)]
Merge "Add graph for processing time without routing"

4 years agoMerge "Reorganize deletion of completed producers"
Piotr Wielebski [Mon, 4 Nov 2019 15:00:11 +0000 (15:00 +0000)]
Merge "Reorganize deletion of completed producers"

4 years agoAdd graph for processing time without routing 16/97916/1
Remigiusz Janeczek [Mon, 4 Nov 2019 14:50:37 +0000 (15:50 +0100)]
Add graph for processing time without routing

Issue-ID: DCAEGEN2-1858
Signed-off-by: Remigiusz Janeczek <remigiusz.janeczek@nokia.com>
Change-Id: I9a3069e639bd723c9ea02b98de412a41cfb4eb5d

4 years agoMerge "Add metric for processing without routing"
Piotr Wielebski [Mon, 4 Nov 2019 14:48:46 +0000 (14:48 +0000)]
Merge "Add metric for processing without routing"

4 years agoMerge "Add option to set kafka retention time for load tests"
Piotr Wielebski [Mon, 4 Nov 2019 12:43:48 +0000 (12:43 +0000)]
Merge "Add option to set kafka retention time for load tests"

4 years agoMerge "Add test environment rebooting script"
Piotr Wielebski [Mon, 4 Nov 2019 12:38:36 +0000 (12:38 +0000)]
Merge "Add test environment rebooting script"

4 years agoAdd option to set kafka retention time for load tests 38/97738/6
Remigiusz Janeczek [Wed, 30 Oct 2019 13:37:05 +0000 (14:37 +0100)]
Add option to set kafka retention time for load tests

Issue-ID: DCAEGEN2-1783
Signed-off-by: Remigiusz Janeczek <remigiusz.janeczek@nokia.com>
Change-Id: Ib78d018864d9cc220b1f23ab2f3f4340c17cef78

4 years agoAdd test environment rebooting script 56/97556/3
Remigiusz Janeczek [Thu, 24 Oct 2019 14:10:43 +0000 (16:10 +0200)]
Add test environment rebooting script

Issue-ID: DCAEGEN2-1883
Signed-off-by: Remigiusz Janeczek <remigiusz.janeczek@nokia.com>
Change-Id: I4a3c801d4cc20d1db55f491e2f49133da36da05e

4 years agoAdd metric for processing without routing 37/97737/2
kjaniak [Wed, 30 Oct 2019 13:19:27 +0000 (14:19 +0100)]
Add metric for processing without routing

Performance tests need better check of processing time in HV-VES.

Change-Id: I0792c4ac014a7b8907ef314a3fd9981776dc0b35
Issue-ID: DCAEGEN2-1890
Signed-off-by: kjaniak <kornel.janiak@nokia.com>
4 years agodcae committer updates 51/97751/3
Vijay Venkatesh Kumar [Mon, 28 Oct 2019 20:38:22 +0000 (20:38 +0000)]
dcae committer updates

Change-Id: Ia09f96a8b03aa4b7c2c776cbe9a97627042297b2
Signed-off-by: Vijay Venkatesh Kumar <vv770d@att.com>
Issue-ID: DCAEGEN2-1891
Signed-off-by: Vijay Venkatesh Kumar <vv770d@att.com>
4 years agoReorganize deletion of completed producers 54/97554/1
kjaniak [Thu, 24 Oct 2019 14:44:38 +0000 (16:44 +0200)]
Reorganize deletion of completed producers

Load performance test enhancement

Change-Id: I8b87ff7fdaaa17dd69cd5136726faa45f497dc71
Issue-ID: DCAEGEN2-1881
Signed-off-by: kjaniak <kornel.janiak@nokia.com>
4 years agoCleanup in cloud script 45/97245/2
kjaniak [Thu, 17 Oct 2019 11:02:39 +0000 (13:02 +0200)]
Cleanup in cloud script

Change-Id: I8919bbb8403d8d008a678c2310175232eae8df1d
Issue-ID: DCAEGEN2-1783
Signed-off-by: kjaniak <kornel.janiak@nokia.com>
4 years agoMerge "Add load test case to performance script"
Piotr Wielebski [Thu, 24 Oct 2019 09:48:55 +0000 (09:48 +0000)]
Merge "Add load test case to performance script"

4 years agoAdd load test case to performance script 18/97518/1
kjaniak [Thu, 24 Oct 2019 07:31:45 +0000 (09:31 +0200)]
Add load test case to performance script

This is done for cloud script only

Change-Id: If66225ab56d04c4f1a9037eca58f89b0c0be1c1e
Issue-ID: DCAEGEN2-1881
Signed-off-by: kjaniak <kornel.janiak@nokia.com>
4 years agoFix performance test issues 67/97467/4
Remigiusz Janeczek [Wed, 23 Oct 2019 11:36:56 +0000 (13:36 +0200)]
Fix performance test issues

Issue-ID: DCAEGEN2-1783
Signed-off-by: Remigiusz Janeczek <remigiusz.janeczek@nokia.com>
Change-Id: I367d109fc5903cd3fd6dbfeeda77d30d780a8a09

4 years agoMerge "Add setting up consul configuration to cloud performance test shell scripts"
Piotr Wielebski [Mon, 21 Oct 2019 08:29:23 +0000 (08:29 +0000)]
Merge "Add setting up consul configuration to cloud performance test shell scripts"

4 years agoAdd setting up consul configuration to cloud performance test shell scripts 21/97321/2
Remigiusz Janeczek [Fri, 18 Oct 2019 16:11:29 +0000 (18:11 +0200)]
Add setting up consul configuration to cloud performance test shell scripts

Issue-ID: DCAEGEN2-1783
Signed-off-by: Remigiusz Janeczek <remigiusz.janeczek@nokia.com>
Change-Id: I8b1b2c68578e52898af0596a039ecebf764560a0

4 years agoImprove grafana dashboards 18/97318/1
Remigiusz Janeczek [Fri, 18 Oct 2019 14:36:45 +0000 (16:36 +0200)]
Improve grafana dashboards

Issue-ID: DCAEGEN2-1858
Signed-off-by: Remigiusz Janeczek <remigiusz.janeczek@nokia.com>
Change-Id: I6ce040a7e9ad4e70c2f8c1139de66b1145cabe5b

4 years agoRevert - Fix memory usage per pod table 63/97263/1
pwielebs [Thu, 17 Oct 2019 14:26:33 +0000 (16:26 +0200)]
Revert - Fix memory usage per pod table

(local deployment)

Signed-off-by: pwielebs <piotr.wielebski@nokia.com>
Issue-ID: DCAEGEN2-1833
Change-Id: I1119c39ce3b25701a8ea467b90ad052b2c7b0dd0

4 years agoMerge "Verify system_cpu_usage metric and fix memory usage per pod in cloud deployment"
Piotr Wielebski [Thu, 17 Oct 2019 14:10:40 +0000 (14:10 +0000)]
Merge "Verify system_cpu_usage metric and fix memory usage per pod in cloud deployment"

4 years agoVerify system_cpu_usage metric and fix memory usage per pod in cloud deployment 40/97240/5
pawel [Thu, 17 Oct 2019 09:30:46 +0000 (11:30 +0200)]
Verify system_cpu_usage metric and fix memory usage per pod in cloud deployment

Issue-ID: DCAEGEN2-1855
Signed-off-by: pawel <pawel.kasperkiewicz@nokia.com>
Change-Id: I99ffdf23515ebe9c070abc5b1993db0cebaedf80

4 years agoReorganize of Performance tests dashboard 51/97251/1
kjaniak [Thu, 17 Oct 2019 11:58:18 +0000 (13:58 +0200)]
Reorganize of Performance tests dashboard

Change-Id: I4fc7c90034c5c060d50c740eb567c992bf29fe74
Issue-ID: DCAEGEN2-1783
Signed-off-by: kjaniak <kornel.janiak@nokia.com>
4 years agoMove certificates directory creation to dockerfile 01/97201/2
Remigiusz Janeczek [Wed, 16 Oct 2019 15:12:06 +0000 (17:12 +0200)]
Move certificates directory creation to dockerfile

Issue-ID: DCAEGEN2-1783
Signed-off-by: Remigiusz Janeczek <remigiusz.janeczek@nokia.com>
Change-Id: If4d9c0dab255ce4f2024f06da40c724dd98d7e5f

4 years agoFix of hv-ves consumer image version 09/97109/1
kjaniak [Tue, 15 Oct 2019 09:11:25 +0000 (11:11 +0200)]
Fix of hv-ves consumer image version

Changed image version of consumer in performance tests

Change-Id: If83a334defc301ffb89e6596f5f23acac25b0216
Issue-ID: DCAEGEN2-1783
Signed-off-by: kjaniak <kornel.janiak@nokia.com>
4 years agoMerge "Fix memory usage per pod table"
Tony Hansen [Sun, 13 Oct 2019 00:47:10 +0000 (00:47 +0000)]
Merge "Fix memory usage per pod table"

4 years agoMerge "Fix of configmap issue"
Tony Hansen [Sun, 13 Oct 2019 00:41:28 +0000 (00:41 +0000)]
Merge "Fix of configmap issue"

4 years agoFix of configmap issue 37/96937/2
kjaniak [Fri, 11 Oct 2019 13:12:21 +0000 (15:12 +0200)]
Fix of configmap issue

Change-Id: Ia24f86ac0a516dea06b2e4d6c2d0479005864493
Issue-ID: DCAEGEN2-1783
Signed-off-by: kjaniak <kornel.janiak@nokia.com>
4 years agoFix memory usage per pod table 22/96922/1
pwielebs [Fri, 11 Oct 2019 09:19:28 +0000 (11:19 +0200)]
Fix memory usage per pod table

(local deployment)

Signed-off-by: pwielebs <piotr.wielebski@nokia.com>
Issue-ID: DCAEGEN2-1833
Change-Id: I242c3a9bac523997d91c218ec21b05e79006db96

4 years agoAdd SSL support to cloud performance tests 14/96914/1
kjaniak [Fri, 11 Oct 2019 05:23:59 +0000 (07:23 +0200)]
Add SSL support to cloud performance tests

Change-Id: Ie0a6341592efe3bfe3e386c01026600d16110cef
Signed-off-by: kjaniak <kornel.janiak@nokia.com>
Issue-ID: DCAEGEN2-1827

4 years agoAdd graphs for performance monitoring 65/96665/5
kjaniak [Mon, 7 Oct 2019 09:46:48 +0000 (11:46 +0200)]
Add graphs for performance monitoring

Performance tests need graphs representing: total, message queue on HV-VES,
offset per kafka topic partition, incoming data rate. Added deletion of producers at test end.
Gauges for offsets turned to Range vector (distinguishable based on tag).
Since Producer is to fast it was reconfigured to send messages with 1 ms interval.

Change-Id: I09fc1edbcd824294d0bb89acc2ea2306a6df00ca
Issue-ID: DCAEGEN2-1807
Signed-off-by: kjaniak <kornel.janiak@nokia.com>
4 years agoSetup Grafana on Cloud (hv-ves) 44/96544/5
pawel [Thu, 3 Oct 2019 07:27:33 +0000 (09:27 +0200)]
Setup Grafana on Cloud (hv-ves)

Issue-ID: DCAEGEN2-1784
Signed-off-by: pawel <pawel.kasperkiewicz@nokia.com>
Change-Id: I87159f360854dccb56e9393bfbe87f66f71dcb6e

4 years agoAdd partition offset metric to each topic partition 68/96168/3
kjaniak [Tue, 24 Sep 2019 07:46:24 +0000 (09:46 +0200)]
Add partition offset metric to each topic partition

Before this commit offset consumer was able to fetch offset just from one partition.
This commit solve this.

Change-Id: I2c2c300219e43ab422b237094ad775ca8795169e
Issue-ID: DCAEGEN2-1783
Signed-off-by: kjaniak <kornel.janiak@nokia.com>
4 years agorelease 1.3.0 hv-ves 28/95428/1 elalto 5.0.1-ONAP 5.0.2-ONAP
Vijay Venkatesh Kumar [Tue, 10 Sep 2019 11:53:57 +0000 (11:53 +0000)]
release 1.3.0 hv-ves

Change-Id: Iacc2a85513a6de9e1ea32a98acf31c42568f1654
Signed-off-by: Vijay Venkatesh Kumar <vv770d@att.com>
Issue-ID: DCAEGEN2-1669

4 years agoAdd Prometheus deployment for HV-VES performance tests - cloud 45/92245/6
kjaniak [Tue, 30 Jul 2019 07:32:56 +0000 (09:32 +0200)]
Add Prometheus deployment for HV-VES performance tests - cloud

Script for cloud is updated with proper prometheus config and deployment.

Change-Id: Icf2527e76bc2028807653313c8ec978d779c6ec9
Issue-ID: DCAEGEN2-1692
Signed-off-by: kjaniak <kornel.janiak@nokia.com>
4 years agoMerge "Setup grafana for local performance tests"
Piotr Wielebski [Tue, 6 Aug 2019 08:04:16 +0000 (08:04 +0000)]
Merge "Setup grafana for local performance tests"

4 years agofix hv-ves build issue 95/92595/2
Vijay Venkatesh Kumar [Thu, 1 Aug 2019 22:01:51 +0000 (22:01 +0000)]
fix hv-ves build issue

Change-Id: I549bcb1ab068a4983dfcbf8e49a0d4efed46f9cb
Signed-off-by: Vijay Venkatesh Kumar <vv770d@att.com>
Issue-ID: DCAEGEN2-1578

4 years agoremove docker binding to mvn 15/92315/1
Vijay Venkatesh Kumar [Sun, 21 Jul 2019 07:02:01 +0000 (07:02 +0000)]
remove docker binding to mvn

Change-Id: Idd3472fd3c9fb40c6f0312b2ab440126e33f3517
Signed-off-by: Vijay Venkatesh Kumar <vv770d@att.com>
Issue-ID: DCAEGEN2-1578

4 years agoFix build failure in coverage module 19/92019/1
Izabela Zawadzka [Thu, 25 Jul 2019 15:36:42 +0000 (17:36 +0200)]
Fix build failure in coverage module

xpath command had to be replaced as the versions available for CentOs 7 dont support used options

Signed-off-by: Izabela Zawadzka <izabela.zawadzka@nokia.com>
Issue-ID: DCAEGEN2-1687
Change-Id: I0b9ee73982fa9a348a7822d040ca90435fbc264c

4 years agoFix of hv-ves dns configuration 31/91731/1
kjaniak [Fri, 19 Jul 2019 09:46:03 +0000 (11:46 +0200)]
Fix of hv-ves dns configuration

Change-Id: I2f859c5a683fb9de0efd2ba93dcd853d8f70df1d
Issue-ID: DCAEGEN2-1664
Signed-off-by: kjaniak <kornel.janiak@nokia.com>
4 years agoAdding script for Performance tests of HV-VES on cloud 61/91361/4
kjaniak [Fri, 12 Jul 2019 12:29:25 +0000 (14:29 +0200)]
Adding script for Performance tests of HV-VES on cloud

Change-Id: If71b0e8b04ce23a5daafef632788667ccd50c1a1
Issue-ID: DCAEGEN2-1664
Signed-off-by: kjaniak <kornel.janiak@nokia.com>
4 years agoSetup grafana for local performance tests 89/91489/3
Izabela Zawadzka [Tue, 16 Jul 2019 06:12:32 +0000 (08:12 +0200)]
Setup grafana for local performance tests

Signed-off-by: Izabela Zawadzka <izabela.zawadzka@nokia.com>
Issue-ID: DCAEGEN2-1666
Change-Id: I49bfccd30e4089a30a235957d3f1a404bfba5e76

4 years agoUse consumers in main 33/91133/4
Izabela Zawadzka [Wed, 10 Jul 2019 07:29:51 +0000 (09:29 +0200)]
Use consumers in main

It includes --disable-processing flag.
Also fixed some issues with script for local performance test.
Also added KafkaConsumer::poll in OffsetKafka Consumer - without it KafkaConsumer::assignment returns empty set

Signed-off-by: Izabela Zawadzka <izabela.zawadzka@nokia.com>
Issue-ID: DCAEGEN2-1657
Change-Id: I95fadb45f321398346094dfa0c4a6e9da954c186

4 years agoMerge "Measure message travel time"
Piotr Jaszczyk [Mon, 8 Jul 2019 09:10:38 +0000 (09:10 +0000)]
Merge "Measure message travel time"

4 years agoIncrease coverage in new consumer 33/90933/3
Filip Krzywka [Fri, 5 Jul 2019 09:26:24 +0000 (11:26 +0200)]
Increase coverage in new consumer

Change-Id: I58e88182e84a7d807d481e8cbd7196d8b82309ac
Issue-ID: DCAEGEN2-1619
Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
4 years agoMerge "Add performance test producer deployment yaml"
Piotr Jaszczyk [Fri, 5 Jul 2019 09:00:13 +0000 (09:00 +0000)]
Merge "Add performance test producer deployment yaml"

4 years agoAdd performance test producer deployment yaml 89/90889/3
Jakub Dudycz [Wed, 3 Jul 2019 13:17:16 +0000 (15:17 +0200)]
Add performance test producer deployment yaml

- create cloud performance test directory structure
- move consumer deployment file to "tools/performence/cloud" directory

Change-Id: Ia5c6d8ed15a2188ff9306cf788052f432dae1e33
Signed-off-by: Jakub Dudycz <jakub.dudycz@nokia.com>
Issue-ID: DCAEGEN2-1652

4 years agoMeasure message travel time 80/90880/4
Izabela Zawadzka [Thu, 4 Jul 2019 06:59:04 +0000 (08:59 +0200)]
Measure message travel time

Issue-ID: DCAEGEN2-1654
Signed-off-by: Izabela Zawadzka <izabela.zawadzka@nokia.com>
Change-Id: Ifd6597209c5be51d5b4ff5faf7b3b1b1a2871403

4 years agoMerge "Change handling of offset metric for consumer app"
Piotr Jaszczyk [Thu, 4 Jul 2019 05:21:20 +0000 (05:21 +0000)]
Merge "Change handling of offset metric for consumer app"

4 years agoMerge "Add K8S deployment for kafka-consumer"
Piotr Jaszczyk [Wed, 3 Jul 2019 13:11:11 +0000 (13:11 +0000)]
Merge "Add K8S deployment for kafka-consumer"

4 years agoMerge "Create script to run performance test locally"
Piotr Jaszczyk [Wed, 3 Jul 2019 12:39:12 +0000 (12:39 +0000)]
Merge "Create script to run performance test locally"

4 years agoChange handling of offset metric for consumer app 36/90836/1
kjaniak [Wed, 3 Jul 2019 11:31:31 +0000 (13:31 +0200)]
Change handling of offset metric for consumer app

Change-Id: Ia83469783d64d0174624489df671006e72bc8948
Issue-ID: DCAEGEN2-1635
Signed-off-by: kjaniak <kornel.janiak@nokia.com>
4 years agoCreate script to run performance test locally 15/90315/9
Jakub Dudycz [Fri, 21 Jun 2019 15:19:49 +0000 (17:19 +0200)]
Create script to run performance test locally

Change-Id: I6bbf0e9b029aef1558008286f0fbec83b76be44c
Signed-off-by: Jakub Dudycz <jakub.dudycz@nokia.com>
Issue-ID: DCAEGEN2-1641

4 years agoReorganize kafka module 24/90524/11
kjaniak [Wed, 26 Jun 2019 13:57:29 +0000 (15:57 +0200)]
Reorganize kafka module

Change-Id: I2eb7a8a6e92c9d89586b877f4cae438497b62ae2
Issue-ID: DCAEGEN2-1635
Signed-off-by: kjaniak <kornel.janiak@nokia.com>
4 years agoAdd K8S deployment for kafka-consumer 98/90298/2
Filip Krzywka [Fri, 21 Jun 2019 08:37:51 +0000 (10:37 +0200)]
Add K8S deployment for kafka-consumer

Change-Id: Idba5c738dc2062b30c32bb928d701614743f0828
Issue-ID: DCAEGEN2-1619
Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
4 years agoJJB alignment across DCAE components - DCAE HV-VES 79/90579/4
rajendrajaiswal [Thu, 27 Jun 2019 09:58:07 +0000 (10:58 +0100)]
JJB alignment across DCAE components - DCAE HV-VES

Change-Id: I815c32a91a3e710d9076495dbafcba420bde4da5
Issue-ID: DCAEGEN2-1578
Signed-off-by: rajendrajaiswal <rajendra.jaiswal@ericsson.com>
4 years agoImplement kafka consumer metrics 49/90449/5
Filip Krzywka [Tue, 25 Jun 2019 12:39:07 +0000 (14:39 +0200)]
Implement kafka consumer metrics

- bump Micrometer version 1.0.8 -> 1.1.5

Change-Id: I7a00fbf252df0f31f12f8743e8719701bd282ce2
Issue-ID: DCAEGEN2-1626
Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
4 years agoAdd environment configuration to kafka consumer 27/90427/6
Filip Krzywka [Tue, 25 Jun 2019 09:32:36 +0000 (11:32 +0200)]
Add environment configuration to kafka consumer

- HV-VES-specific environment prefix moved inside HvVes modules to
allow simpler no-prefix API for other modules
- created OptionDSL for brevity

Change-Id: I2fabbda1280cc0f913f8a0a04b4a055f39ed1fae
Issue-ID: DCAEGEN2-1626
Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
4 years agoImplement message counting in consumer 56/90156/8
kjaniak [Wed, 19 Jun 2019 06:39:33 +0000 (08:39 +0200)]
Implement message counting in consumer

Issue-ID: DCAEGEN2-1635
Change-Id: I2666de7bad27052d9cefa0f687ad0772d4c9a95d
Signed-off-by: kjaniak <kornel.janiak@nokia.com>
4 years agoMerge "Expose Prometheus metrics in Kafka Consumer"
Piotr Jaszczyk [Tue, 25 Jun 2019 09:33:07 +0000 (09:33 +0000)]
Merge "Expose Prometheus metrics in Kafka Consumer"

4 years agoExpose Prometheus metrics in Kafka Consumer 98/89898/8
Jakub Dudycz [Fri, 14 Jun 2019 13:48:32 +0000 (15:48 +0200)]
Expose Prometheus metrics in Kafka Consumer

Change-Id: I9d0568a5fc296604d7ac7a45b8bbd0289b845938
Signed-off-by: Jakub Dudycz <jakub.dudycz@nokia.com>
Issue-ID: DCAEGEN2-1626

4 years agoFix request interval 05/90005/2
Filip Krzywka [Mon, 17 Jun 2019 11:52:06 +0000 (13:52 +0200)]
Fix request interval

In previous implementation DistinctUntilChangedSubscriber always
requested from upstream 256 events, which resulted in immediate
256 requests to CBS.

Request amount is not configurable in other way than hard-limiting
using `limitRequest`, which limits request amount for single subscriber.
(At least in our pipeline)

To avoid multiple manual subscribes, this commit changed CbsClientAdapter
to use Mono instead of Flux for CbsRequests and repeat this Mono
conditionally. Flux inside of repeatWhen is emitting event after each
onComplete received from upstream Mono and resubscribes to it if condition
is met. This seemed like good place to put our interval mechanism, which
is always-pass condition, but condition resolving blocks for variable
duration.

Change-Id: I04d1e657ec4d82185f6f07422c25c2d2ff23e60d
Issue-ID: DCAEGEN2-1557
Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
4 years agoMerge "Update docker-maven-plugin to latest version"
Piotr Jaszczyk [Fri, 14 Jun 2019 11:03:23 +0000 (11:03 +0000)]
Merge "Update docker-maven-plugin to latest version"

4 years agoCreate Kafka Consumer project stub 17/89817/2
Jakub Dudycz [Thu, 13 Jun 2019 14:51:43 +0000 (16:51 +0200)]
Create Kafka Consumer project stub

Change-Id: Ib45ca6a9ab282dab509e34f9a3d3682373d794d3
Signed-off-by: Jakub Dudycz <jakub.dudycz@nokia.com>
Issue-ID: DCAEGEN2-1619

4 years agoUpdate docker-maven-plugin to latest version 99/89799/1
Jakub Dudycz [Thu, 13 Jun 2019 11:44:31 +0000 (13:44 +0200)]
Update docker-maven-plugin to latest version

Update fabric8io/docker-maven-plugin to version 0.30.0

Change-Id: I528405896443b77e521a783511898298bfd91d2e
Issue-ID: DCAEGEN2-1617
Signed-off-by: Jakub Dudycz <jakub.dudycz@nokia.com>
4 years agoRetry infinitely 38/89738/2
Filip Krzywka [Mon, 10 Jun 2019 07:02:43 +0000 (09:02 +0200)]
Retry infinitely

- changed specification to retry infinitely
- moved MDC to cbsAdapter constructor as in whole module it contains
only local context (instanceID etc.). Also permuted constructor params
to match ConfigurationProviders order
- refactored module tests as ground for future enhancements

Change-Id: Ic074b9c421b60662e5512c55c7b1dfb90ab0d2ea
Issue-ID: DCAEGEN2-1557
Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
4 years agoRun container commands as non-root 01/89501/2
Filip Krzywka [Thu, 6 Jun 2019 09:24:36 +0000 (11:24 +0200)]
Run container commands as non-root

- 1410 uig/gid is arbitrary, note however that any volume mounted into
container should limit permissions to this number
- also reduced firstRequestDelay in configuration provided through
local docker-compose

Change-Id: I77eeebeec5931db5c04f8f6f24d7c6fe7a121015
Issue-ID: DCAEGEN2-1557
Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
4 years agoMerge "Support CBS request interval reconfiguration"
Piotr Jaszczyk [Thu, 6 Jun 2019 05:33:46 +0000 (05:33 +0000)]
Merge "Support CBS request interval reconfiguration"

4 years agoSupport CBS request interval reconfiguration 06/88306/18
kjaniak [Wed, 22 May 2019 20:19:49 +0000 (22:19 +0200)]
Support CBS request interval reconfiguration

Change-Id: Ie8892e33b2f6a58d6076f66e6cc6a2df830dfa48
Issue-ID: DCAEGEN2-1525
Signed-off-by: kjaniak <kornel.janiak@nokia.com>
4 years agoBump Kotlin and Reactor versions 42/88842/2
Piotr Jaszczyk [Thu, 30 May 2019 08:47:53 +0000 (10:47 +0200)]
Bump Kotlin and Reactor versions

Issue-ID: DCAEGEN2-1542
Change-Id: I1b14b29a780deedc9c64bdfbded4b3b05e990279
Signed-off-by: Piotr Jaszczyk <piotr.jaszczyk@nokia.com>
5 years agoEnable Kafka consumer offset committing 38/87438/1
Filip Krzywka [Fri, 10 May 2019 09:53:21 +0000 (11:53 +0200)]
Enable Kafka consumer offset committing

It appears that reactor-kafka is setting auto.commit property to false,
which makes our CSITs fail nondeterministically due to automatic reset of
consumer offset.

By acknowledging manually, we will mark every message for committing,
which will be performed according to ConsumerConfiguration.
This way, Kafka broker should persist consumer offset.

Change-Id: I0c5156ff8df9bb3341e733e50a3c6866fdd94976
Issue-ID: DCAEGEN2-1495
Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
5 years agoUpdate versions for El Alto release 12/87012/3
Piotr Jaszczyk [Mon, 6 May 2019 09:29:45 +0000 (11:29 +0200)]
Update versions for El Alto release

Change-Id: If86dcbf8695cc50a540fe892a4f70b3ad24ac5d1
Issue-ID: DCAEGEN2-1474
Signed-off-by: Piotr Jaszczyk <piotr.jaszczyk@nokia.com>
5 years agoChange of KAFKA_ADVERTISED_LISTENERS 89/86389/1 1.1.0 4.0.0-ONAP
kjaniak [Fri, 26 Apr 2019 11:51:17 +0000 (13:51 +0200)]
Change of KAFKA_ADVERTISED_LISTENERS

Fix for DMaaP app bug

Change-Id: I05312a6b5d1acc1f7894f17d89922c508bbbd41d
Issue-ID: DCAEGEN2-1448
Signed-off-by: kjaniak <kornel.janiak@nokia.com>
5 years agoUse AAF credentials from stream definition 33/86033/7
Piotr Jaszczyk [Tue, 23 Apr 2019 09:11:45 +0000 (11:11 +0200)]
Use AAF credentials from stream definition

Change-Id: I4fc20c116c60f6e7d46215a32c33884cd957e93b
Issue-ID: DCAEGEN2-1448
Signed-off-by: Piotr Jaszczyk <piotr.jaszczyk@nokia.com>
5 years agoUse SASL auth in kafka connections 84/85684/2
Filip Krzywka [Thu, 18 Apr 2019 12:27:05 +0000 (14:27 +0200)]
Use SASL auth in kafka connections

Change-Id: I55a9289901a6a44f3d07a3cf4e5a028399a5d0dc
Issue-ID: DCAEGEN2-1448
Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
5 years agoAdd internal modifiers to comply with new HvVesCustomRule 05/85305/4
Izabela Zawadzka [Mon, 15 Apr 2019 11:03:45 +0000 (13:03 +0200)]
Add internal modifiers to comply with new HvVesCustomRule

Change-Id: I33ad24179d05ded58f6784544be9bbdd56210720
Signed-off-by: Izabela Zawadzka <izabela.zawadzka@nokia.com>
Issue-ID: DCAEGEN2-1438

5 years agoMerge "Create custom rule to report public modifiers in impl"
Piotr Jaszczyk [Mon, 15 Apr 2019 10:53:53 +0000 (10:53 +0000)]
Merge "Create custom rule to report public modifiers in impl"

5 years agoCreate custom rule to report public modifiers in impl 51/85051/5
Izabela Zawadzka [Thu, 11 Apr 2019 12:27:21 +0000 (14:27 +0200)]
Create custom rule to report public modifiers in impl

Change-Id: I383ca27a835943bd2dc2508425264ad7f64c7725
Signed-off-by: Izabela Zawadzka <izabela.zawadzka@nokia.com>
Issue-ID: DCAEGEN2-1430

5 years agoMove cmd ssl logic to xnf-simulator 72/84872/7
Jakub Dudycz [Wed, 10 Apr 2019 13:02:01 +0000 (15:02 +0200)]
Move cmd ssl logic to xnf-simulator

Change-Id: I562acbb0b7522e6cefea2a2dc1a9e292c1227b41
Signed-off-by: Jakub Dudycz <jakub.dudycz@nokia.com>
Issue-ID: DCAEGEN2-565

5 years agoCreation of server module 99/84199/9
kjaniak [Wed, 3 Apr 2019 13:48:28 +0000 (15:48 +0200)]
Creation of server module

Issue-ID: DCAEGEN2-1390
Change-Id: I07410b16ed6566b933d5f1efa35bddb965225794
Signed-off-by: kjaniak <kornel.janiak@nokia.com>
Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
5 years agoMerge "Extract transforming logic from validator"
Piotr Jaszczyk [Fri, 12 Apr 2019 06:38:04 +0000 (06:38 +0000)]
Merge "Extract transforming logic from validator"

5 years agoMerge "Fix xnf-simulator message sending bug"
Piotr Jaszczyk [Fri, 12 Apr 2019 06:20:23 +0000 (06:20 +0000)]
Merge "Fix xnf-simulator message sending bug"

5 years agoExtract transforming logic from validator 69/84869/11
Filip Krzywka [Wed, 10 Apr 2019 09:36:48 +0000 (11:36 +0200)]
Extract transforming logic from validator

Change-Id: Ic019b1796e17d24f14f41a817af6e5ecd8c7244b
Issue-ID: DCAEGEN2-1416
Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
5 years agoFix xnf-simulator message sending bug 50/84850/2
Jakub Dudycz [Wed, 10 Apr 2019 10:51:14 +0000 (12:51 +0200)]
Fix xnf-simulator message sending bug

Fix bug where xnf-simulator was failing to send messages with invalid wire frame.
Generated events couldn't pass HV-VES client validatation, because they had empty payload.

Change-Id: I20fb6254e432de096acfcfb8f0a0a5f912ca08c0
Signed-off-by: Jakub Dudycz <jakub.dudycz@nokia.com>
Issue-ID: DCAEGEN2-1413

5 years agoMerge "Read passwords from files"
Piotr Wielebski [Wed, 10 Apr 2019 11:59:26 +0000 (11:59 +0000)]
Merge "Read passwords from files"

5 years agoRead passwords from files 25/84825/4
Piotr Jaszczyk [Wed, 10 Apr 2019 08:32:00 +0000 (10:32 +0200)]
Read passwords from files

Key- and trust-store passwords should be read from files in order to
work with DCAE tls-init-container.

Change-Id: Ibe454663328268f33f8be25ef9ec129f1ce1d396
Issue-ID: DCAEGEN2-1412
Signed-off-by: Piotr Jaszczyk <piotr.jaszczyk@nokia.com>