dcaegen2/collectors/datafile.git
4 years agoAdd config-volume to DFC for storing know host keys 95/117595/1
Tomasz Pietruszkiewicz [Mon, 8 Feb 2021 14:01:21 +0000 (15:01 +0100)]
Add config-volume to DFC for storing know host keys

Change-Id: If2de5453cbe262f251b77b9f6e25b45e36207ea8
Issue-ID: DCAEGEN2-2549
Signed-off-by: Tomasz Pietruszkiewicz <tomasz.pietruszkiewicz@nokia.com>
4 years agodcae INFO.yaml update 10/116810/1
vv770d [Wed, 13 Jan 2021 21:36:41 +0000 (21:36 +0000)]
dcae INFO.yaml update

Committer changes

Change-Id: If12981641229ed9a72237928e35870b97757400b
Issue-ID: DCAEGEN2-2583
Signed-off-by: vv770d <vv770d@att.com>
4 years agoRelease Datafile 1.5.1 63/116663/2
Krzysztof Gajewski [Thu, 7 Jan 2021 09:45:16 +0000 (10:45 +0100)]
Release Datafile 1.5.1

Issue-ID: DCAEGEN2-2527

Signed-off-by: Krzysztof Gajewski <krzysztof.gajewski@nokia.com>
Change-Id: I3070555a3305d17d7317479e66afd7273583af83

4 years agoUpdate Datafile version to 1.5.1 87/116587/2 1.5.1
Krzysztof Gajewski [Tue, 5 Jan 2021 13:40:29 +0000 (14:40 +0100)]
Update Datafile version to 1.5.1

Issue-ID: DCAEGEN2-2527

Signed-off-by: Krzysztof Gajewski <krzysztof.gajewski@nokia.com>
Change-Id: I271b521521cfd38c0da05ae65d52550908021540

4 years agoAdd HTTP as new protocol to collect files from xNFs 59/116459/6
Krzysztof Gajewski [Tue, 15 Dec 2020 10:19:51 +0000 (11:19 +0100)]
Add HTTP as new protocol to collect files from xNFs
- HTTP basic auth included
- small code refactoring related to the task

Issue-ID: DCAEGEN2-2527

Signed-off-by: Krzysztof Gajewski <krzysztof.gajewski@nokia.com>
Change-Id: I13ec80e996861e14d2c561087c4af3b34d861030

4 years agoAdd initial changelog 97/116397/1
Pawel [Tue, 15 Dec 2020 13:37:24 +0000 (14:37 +0100)]
Add initial changelog

Issue-ID: DCAEGEN2-2557
Signed-off-by: Pawel <pawel.kasperkiewicz@nokia.com>
Change-Id: Ida4e1adf2b8760589a4e2508d8bb68aea452ef85

4 years agoRelease Datafile 1.5.0 86/116186/4
tkogut [Tue, 8 Dec 2020 08:02:53 +0000 (09:02 +0100)]
Release Datafile 1.5.0

- upgrade Reactor and SpringBoot

Issue-ID: DCAEGEN2-2537
Signed-off-by: tkogut <tomasz.kogut@nokia.com>
Change-Id: Ib4e9bf481ef23fa4f7e63ef7efb254558cd466d6

4 years agoUpgrade Reactor and SpringBoot 95/115995/3 1.5.0
tkogut [Wed, 2 Dec 2020 10:53:06 +0000 (11:53 +0100)]
Upgrade Reactor and SpringBoot

Issue-ID: DCAEGEN2-2537
Signed-off-by: tkogut <tomasz.kogut@nokia.com>
Change-Id: Ide560554f11d01737b351d6138430efe6005791d

4 years agoRemove blueprint 33/115033/2
JerzySzachniewicz [Wed, 18 Nov 2020 11:26:48 +0000 (12:26 +0100)]
Remove blueprint

All future blueprint updates will be done under blueprint repo

Issue-ID: DCAEGEN2-2456
Signed-off-by: JerzySzachniewicz <jerzy.szachniewicz@nokia.com>
Change-Id: I388a17c68caf33b96533d8c3463d519846cbe5ca
Signed-off-by: JerzySzachniewicz <jerzy.szachniewicz@nokia.com>
4 years agoAlign component-spec properties 69/113969/1
Tomasz Wrobel [Mon, 19 Oct 2020 06:26:16 +0000 (08:26 +0200)]
Align component-spec properties

Change FTPes certs to PKCS12 (extternal communication).

Issue-ID: DCAEGEN2-1795
Signed-off-by: Tomasz Wrobel <tomasz.wrobel@nokia.com>
Change-Id: I06a729adb32290e2ebd942a37a144ae034a08a42

4 years agoAdd use_external_tls property to component-spec 76/113876/1
Tomasz Wrobel [Wed, 14 Oct 2020 07:28:08 +0000 (09:28 +0200)]
Add use_external_tls property to component-spec

Issue-ID: DCAEGEN2-1795
Signed-off-by: Tomasz Wrobel <tomasz.wrobel@nokia.com>
Change-Id: I71a9538b79deac7e58d1cd8131e623ae3758fdce

4 years agoAlign DFC component spec 99/112799/6
Tomasz Wrobel [Thu, 17 Sep 2020 07:50:06 +0000 (09:50 +0200)]
Align DFC component spec

Issue-ID: DCAEGEN2-2461
Signed-off-by: Tomasz Wrobel <tomasz.wrobel@nokia.com>
Change-Id: Ib09be9a2f21643b8e6de95ecc295d16f7ba7e8cb

4 years agoUpdate datafile component spec 16/113416/1
Remigiusz Janeczek [Thu, 1 Oct 2020 10:12:01 +0000 (12:12 +0200)]
Update datafile component spec

Issue-ID: DCAEGEN2-2461
Signed-off-by: Remigiusz Janeczek <remigiusz.janeczek@nokia.com>
Change-Id: I03a75203ab6870d8799fb498c67abbc226a54fb0

4 years agodcae INFO.yaml update 47/112547/1 7.0.0-ONAP
vv770d [Fri, 11 Sep 2020 17:20:06 +0000 (17:20 +0000)]
dcae INFO.yaml update

Project maturity
Committer changes

Change-Id: Ifb10ebbada39ae0bc80f83b5a584d70caf9364a2
Issue-ID: DCAEGEN2-2425
Signed-off-by: vv770d <vv770d@att.com>
4 years agoFix INFO.yaml files 97/112297/1
Jessica Wagantall [Tue, 8 Sep 2020 17:47:10 +0000 (10:47 -0700)]
Fix INFO.yaml files

Proper syntax for INFO.yaml requires the first
line to be "---" which was previously removed.
Re-adding this missing line.

Issue-ID: CIMAN-387
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Change-Id: I9558792715aeaf57dcc4a56da3df0339459afdfd

4 years agoRelease Datafile 1.4.3 33/111833/1
Remigiusz Janeczek [Fri, 28 Aug 2020 09:28:56 +0000 (11:28 +0200)]
Release Datafile 1.4.3

Issue-ID: DCAEGEN2-2267
Signed-off-by: Remigiusz Janeczek <remigiusz.janeczek@nokia.com>
Change-Id: I9b92ff47474baf5b44526b1f6e804b85c4492cd7

4 years agoMerge "Update dcae SDK from 1.1.6 to 1.4.2" 1.4.3
Vijay Venkatesh Kumar [Fri, 28 Aug 2020 00:40:49 +0000 (00:40 +0000)]
Merge "Update dcae SDK from 1.1.6 to 1.4.2"

4 years agoUpdate dcae SDK from 1.1.6 to 1.4.2 68/111668/6
Remigiusz Janeczek [Tue, 18 Aug 2020 10:48:19 +0000 (12:48 +0200)]
Update dcae SDK from 1.1.6 to 1.4.2

Bump project version from 1.4.2 to 1.4.3
Update deprecated calls to JsonParser
Make logs always go to file and console

Issue-ID: DCAEGEN2-2267
Signed-off-by: Remigiusz Janeczek <remigiusz.janeczek@nokia.com>
Change-Id: Ib8d7f82b3daf03ca327581c9a5dc4f6f27a20141

4 years agoRelease Datafile 1.4.2 78/111678/1
ERIMROB [Tue, 25 Aug 2020 08:29:38 +0000 (09:29 +0100)]
Release Datafile 1.4.2

Issue-ID: DCAEGEN2-2267
Change-Id: I3e41172daff912774ca5c683a6a3a07a43ea6104
Signed-off-by: ERIMROB <robertas.rimkus@est.tech>
4 years agoRemove DFC Vulnerabilities 30/111430/1 1.4.2
ERIMROB [Fri, 14 Aug 2020 12:39:24 +0000 (13:39 +0100)]
Remove DFC Vulnerabilities

Issue-ID: DCAEGEN2-2267
Change-Id: Id9110c8b4b747bba3fcce1c1411a40062aa40740
Signed-off-by: ERIMROB <robertas.rimkus@est.tech>
4 years agoopenJDK image update to 11.0.8 34/110834/2
deen1985 [Mon, 3 Aug 2020 09:06:37 +0000 (11:06 +0200)]
openJDK image update to 11.0.8

Change-Id: I341390e5bde3e894455ac7717990669f4e0f0f94
Signed-off-by: deen1985 <damian.nowak@nokia.com>
Issue-ID: DCAEGEN2-2378
Signed-off-by: deen1985 <damian.nowak@nokia.com>
4 years agoMerge "Removed support for FTPS protocol (from now only FTPeS and sFTP will be suppor...
Tony Hansen [Fri, 31 Jul 2020 12:28:32 +0000 (12:28 +0000)]
Merge "Removed support for FTPS protocol (from now only FTPeS and sFTP will be supported)."

4 years agoRemoved support for FTPS protocol (from now only FTPeS and sFTP will be supported). 77/110477/8
Jan Malkiewicz [Thu, 23 Jul 2020 07:04:34 +0000 (09:04 +0200)]
Removed support for FTPS protocol (from now only FTPeS and sFTP will be supported).

Issue-ID: DCAEGEN2-2367

Signed-off-by: Jan Malkiewicz <jan.malkiewicz@nokia.com>
Change-Id: Ifada2899431c56bdbfcf3aad73fdc9b376ae9c4f

4 years agoRevert java version to 11 90/110690/1
Piotr Marcinkiewicz [Wed, 29 Jul 2020 13:23:46 +0000 (15:23 +0200)]
Revert java version to 11

Issue-ID: DCAEGEN2-2219
Signed-off-by: Piotr Marcinkiewicz <piotr.marcinkiewicz@nokia.com>
Change-Id: I82b8fa0ead6e9576c8953739e6563488d4ac9764

4 years agoMerge "Add log when file is skipped because it was already published"
Henrik Andersson [Wed, 29 Jul 2020 06:08:25 +0000 (06:08 +0000)]
Merge "Add log when file is skipped because it was already published"

4 years agoAdd log when file is skipped because it was already published 88/110488/6
Remigiusz Janeczek [Thu, 23 Jul 2020 11:00:25 +0000 (13:00 +0200)]
Add log when file is skipped because it was already published

Issue-ID: DCAEGEN2-2366
Signed-off-by: Remigiusz Janeczek <remigiusz.janeczek@nokia.com>
Change-Id: I913b887833e412dd9c0093936a9231d37f63e7d1

4 years agoAdd sftp strict host key checking to DFC. 90/110190/28
Jan Malkiewicz [Wed, 15 Jul 2020 13:28:41 +0000 (15:28 +0200)]
Add sftp strict host key checking to DFC.

Issue-ID: DCAEGEN2-2219

Signed-off-by: Jan Malkiewicz <jan.malkiewicz@nokia.com>
Change-Id: Iadf6c6bd743c42ebb3bf9ad8ac443fc0f3f58063

4 years agodcae committer updates in INFO.yaml 07/110507/1
vv770d [Thu, 23 Jul 2020 19:12:48 +0000 (19:12 +0000)]
dcae committer updates in INFO.yaml

Change-Id: If660aaa0c8ea06a4717e8ab8d95db745922c5868
Issue-ID: DCAEGEN2-2365
Signed-off-by: vv770d <vv770d@att.com>
4 years agoUpdate to Java 11 50/110450/4
JoeOLeary [Wed, 22 Jul 2020 13:08:40 +0000 (14:08 +0100)]
Update to Java 11

minor version bump to 1.4.0

Issue-ID: DCAEGEN2-1691
Signed-off-by: JoeOLeary <joseph.o.leary@est.tech>
Change-Id: Iaf9e8fdcf469fe79c63e8e3868b18168a60a7f2c

5 years agoRelease Datafile 1.3.0 57/103257/1 6.0.0-ONAP
JoeOLeary [Fri, 6 Mar 2020 11:52:44 +0000 (11:52 +0000)]
Release Datafile 1.3.0

Signed-off-by: JoeOLeary <joseph.o.leary@est.tech>
Issue-ID: DCAEGEN2-2127
Change-Id: I2f176ee2fe8e25a9c93dfe32bb36615b076d78ec

5 years agofix sonarcloud issues 60/101560/1 1.3.0
Hansen, Tony (th1395) [Tue, 11 Feb 2020 22:05:48 +0000 (22:05 +0000)]
fix sonarcloud issues

Change-Id: I91ca1c35aac746a176c9ab3aab520728edc01bb7
Signed-off-by: Hansen, Tony (th1395) <th1395@att.com>
Issue-ID: DCAEGEN2-2065

5 years agoUpdate DCAE Committer in INFO.yaml 94/99094/2
JoeOLeary [Tue, 3 Dec 2019 21:05:01 +0000 (21:05 +0000)]
Update DCAE Committer in INFO.yaml

Issue-ID: DCAEGEN2-1891
Signed-off-by: JoeOLeary <joseph.o.leary@est.tech>
Change-Id: Iaf1391e62f3c56f6a401d4ec8217bd63bc8f9cc8

5 years agoMerge "DataFileCollector use wrong KeyManagerFactory"
Tony Hansen [Tue, 12 Nov 2019 04:07:18 +0000 (04:07 +0000)]
Merge "DataFileCollector use wrong KeyManagerFactory"

5 years agoRevert "Fix security vulnerabilities" 81/97481/1
Vijay Venkatesh Kumar [Wed, 23 Oct 2019 14:01:27 +0000 (14:01 +0000)]
Revert "Fix security vulnerabilities"

This reverts commit 23b56c6b8fc256e5a070cfefe31c90b1fbe46bce.

Issue-ID: DCAEGEN2-1810
Change-Id: I5d883b17ea7e4de0570a10a5a20971f3a221cb88
Signed-off-by: Vijay Venkatesh Kumar <vv770d@att.com>
5 years agoDataFileCollector use wrong KeyManagerFactory 96/97196/1
burdziak [Wed, 16 Oct 2019 13:21:59 +0000 (15:21 +0200)]
DataFileCollector use wrong KeyManagerFactory

Issue-ID: DCAEGEN2-1854
Signed-off-by: burdziak <olaf.burdziakowski@nokia.com>
Change-Id: I71c7526097014e10d0ef091e38a929b81ba1f627

5 years agoFix security vulnerabilities 98/96798/1
elinuxhenrik [Wed, 9 Oct 2019 14:08:34 +0000 (16:08 +0200)]
Fix security vulnerabilities

Change-Id: Ie9c575a6fe0b8b9eca62a2fdccd49e056b4353b1
Issue-ID: DCAEGEN2-1810
Signed-off-by: elinuxhenrik <henrik.b.andersson@est.tech>
5 years agoUpdate blueprint 13/96513/1
JoeOLeary [Wed, 2 Oct 2019 14:45:27 +0000 (15:45 +0100)]
Update blueprint

* Make topic name, consumer group and consumer id configurable
* Add use existing to DR feed

Issue-ID: DCAEGEN2-1793
Signed-off-by: JoeOLeary <joseph.o.leary@est.tech>
Change-Id: I454acf3390c112be16c1604d8d62098b7f3c45e6

5 years agoPersonalized client certificate authentication via FTPes in 5G Bulk PM usecase doesn... 31/95731/5
burdziak [Mon, 16 Sep 2019 10:56:44 +0000 (12:56 +0200)]
Personalized client certificate authentication via FTPes in 5G Bulk PM usecase doesn't work

Issue-ID: DCAEGEN2-1754
Signed-off-by: burdziak <olaf.burdziakowski@nokia.com>
Change-Id: Idec3efce8412dd175ecf75f55790a24cec651104

5 years agoUpdate keyPasswordPath 71/95571/2
ecaiyanlinux [Thu, 12 Sep 2019 12:23:43 +0000 (14:23 +0200)]
Update keyPasswordPath

Change-Id: I126c018444d235a1972b4b9cecc93dfd2f5cb4c1
Signed-off-by: Martin Yan <martin.c.yan@est.tech>
Issue-ID: DCAEGEN2-1702

5 years agoUpdate Cloudify types file version 57/95557/1
JoeOLeary [Thu, 12 Sep 2019 07:28:22 +0000 (08:28 +0100)]
Update Cloudify types file version

Issue-ID: DCAEGEN2-1686
Change-Id: Id7f5bd56bb8e0f80e8006d84b03ce7323e509a87
Signed-off-by: JoeOLeary <joseph.o.leary@est.tech>
5 years agoupdate Dockerfile & copyrigt statement 60/95360/2
ecaiyanlinux [Tue, 10 Sep 2019 13:14:09 +0000 (15:14 +0200)]
update Dockerfile & copyrigt statement

Issue-ID: DCAEGEN2-1702
Change-Id: If990e180b96f943dd7381a758c11d10e4733706c
Signed-off-by: Martin Yan <martin.c.yan@est.tech>
5 years agoTLS init container usage 32/95132/2
ecaiyanlinux [Fri, 6 Sep 2019 15:24:59 +0000 (15:24 +0000)]
TLS init container usage

Change-Id: Iec51c722f6a3a519fd7eb0f37f8b801eebbc039b
Issue-ID: DCAEGEN2-1702
Signed-off-by: ecaiyanlinux <martin.c.yan@est.tech>
5 years agoAlign spec and blueprint with bp-generator tool 45/92445/3
RehanRaza [Mon, 26 Aug 2019 09:15:24 +0000 (09:15 +0000)]
Align spec and blueprint with bp-generator tool

Change-Id: Ifbafe8be8bcb34417873af3e6bb1c7323f891fe0
Issue-ID: DCAEGEN2-1580
Signed-off-by: RehanRaza <muhammad.rehan.raza@est.tech>
5 years agoMerge "change mr default port to https:3905"
Tony Hansen [Wed, 14 Aug 2019 18:23:10 +0000 (18:23 +0000)]
Merge "change mr default port to https:3905"

5 years agoDFC Housekeeping 15/93215/2
PatrikBuhr [Mon, 12 Aug 2019 12:28:01 +0000 (12:28 +0000)]
DFC Housekeeping

Removed warnings.
Improved code formatting
Removed unused files
Removed datafile_endpoints.json from the container

Change-Id: I7334775be793cd9cb709ca1e031c620c2c1c2b3f
Issue-ID: DCAEGEN2-1645
Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
5 years agodcae-ED rls 1.2.2 dfc 19/92919/1 5.0.1-ONAP
Vijay Venkatesh Kumar [Sat, 3 Aug 2019 14:54:30 +0000 (14:54 +0000)]
dcae-ED rls 1.2.2 dfc

Change-Id: I3fc37cba2b3fcb11a23ca3a9e598241eb0b9e59f
Signed-off-by: Vijay Venkatesh Kumar <vv770d@att.com>
Issue-ID: DCAEGEN2-1669

5 years agobump dfc version to 1.2.2 89/92889/1
Vijay Venkatesh Kumar [Sat, 3 Aug 2019 09:29:27 +0000 (09:29 +0000)]
bump dfc version to 1.2.2

Change-Id: I080c26db2639eff2174d4aac334990af67019f46
Signed-off-by: Vijay Venkatesh Kumar <vv770d@att.com>
Issue-ID: DCAEGEN2-1669

5 years agorelease dfc 1.2.1 13/92613/1
Vijay Venkatesh Kumar [Fri, 2 Aug 2019 00:49:34 +0000 (00:49 +0000)]
release dfc 1.2.1

Change-Id: I5143dd5de0c734c4772bef425b3748194621751d
Signed-off-by: Vijay Venkatesh Kumar <vv770d@att.com>
Issue-ID: DCAEGEN2-1669

5 years agoFix dfc gerrit mvn job 27/92427/1 5.0.0-ONAP
Vijay Venkatesh Kumar [Wed, 31 Jul 2019 22:48:13 +0000 (22:48 +0000)]
Fix dfc gerrit mvn job

Change-Id: Ia160651fd8ab0aead0aa43fd993f8e5dbff94ddc
Signed-off-by: Vijay Venkatesh Kumar <vv770d@att.com>
Issue-ID: DCAEGEN2-1578

5 years agochange mr default port to https:3905 58/92358/1
yanhuanwang [Wed, 31 Jul 2019 12:01:10 +0000 (12:01 +0000)]
change mr default port to https:3905

Change-Id: I3fada4907601e999490d73bbc85e792c538547b2
Issue-ID: DCAEGEN2-1595
Signed-off-by: yanhuanwang <martin.c.yan@est.tech>
5 years agoremove docker bind from mvn 46/92146/1
Vijay Venkatesh Kumar [Sat, 20 Jul 2019 05:46:14 +0000 (05:46 +0000)]
remove docker bind from mvn

Change-Id: I5787075e559bcfb56373de5cfa7e09365f16f8b8
Signed-off-by: Vijay Venkatesh Kumar <vv770d@att.com>
Issue-ID: DCAEGEN2-1690
Issue-ID: DCAEGEN2-1578

6 years agoCbs Client integration 23/89223/14
YongchaoWu [Fri, 12 Jul 2019 09:01:01 +0000 (09:01 +0000)]
Cbs Client integration

Cbs Client is integrated to read configurations from consul

Issue-ID: DCAEGEN2-1595
Change-Id: Idb0ebd34eba077f9c1cb584abab4d8722b56f6c5
Signed-off-by: YongchaoWu <yongchao.wu@est.tech>
6 years agoJJB alignment across DCAE components - DCAE Datafile Collector 61/90661/2
elinuxhenrik [Fri, 28 Jun 2019 12:18:16 +0000 (12:18 +0000)]
JJB alignment across DCAE components - DCAE Datafile Collector

Change-Id: I27dd76a8f1b84a8ab26080381321c509c1117a9b
Issue-ID: DCAEGEN2-1578
Signed-off-by: elinuxhenrik <henrik.b.andersson@est.tech>
6 years agoDFC - housekeeping 75/90575/3
elinuxhenrik [Tue, 2 Jul 2019 13:53:56 +0000 (13:53 +0000)]
DFC - housekeeping

Removed Sonar warnings.
Improved code coverage.
Improved code formatting

Change-Id: I0e8287cecb18a8976635427697dfaeb823da204b
Issue-ID: DCAEGEN2-1645
Signed-off-by: elinuxhenrik <henrik.b.andersson@est.tech>
6 years agoCode formatting with maven 82/89882/1
PatrikBuhr [Fri, 14 Jun 2019 11:42:38 +0000 (11:42 +0000)]
Code formatting with maven

This commit only contains code formastting changes done by command:
mvn formatter:format spotless:apply process-sources

Change-Id: I0fb9d166ad5d9171cdeee9f26b6d353bca74069c
Issue-ID: DCAEGEN2-1538
Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
6 years agoBugfix, improved behaviour for large files 44/89244/5
PatrikBuhr [Fri, 14 Jun 2019 06:38:21 +0000 (06:38 +0000)]
Bugfix, improved behaviour for large files

Previously files was read into a buffer for publishing.
This does not work when files are bigger than the available memory.
After the fix , files are streamed instead.

Implemented a new REST primitive for exposing status and
statistics. To be used for test and trouble shooting.

Change-Id: Iab5a1ee9ffcbf6836fcf709d115bf25ab0391732
Issue-ID: DCAEGEN2-1532
Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
6 years agoDatafile - Improve JUnit tests 13/87213/10
elinuxhenrik [Wed, 5 Jun 2019 08:54:17 +0000 (08:54 +0000)]
Datafile - Improve JUnit tests

Improve JUnit tests for the Datafile Collector.
Add tests of logging.
Remove dependency to old JUnit version.
Improve some log messages.
Improve code coverage.

Change-Id: Ie8204619ab7fe4294c4ac0dd8f030973653ef4a3
Issue-ID: DCAEGEN2-1490
Signed-off-by: elinuxhenrik <henrik.b.andersson@est.tech>
6 years agoBlueprint alignment 44/89144/4
YongchaoWu [Tue, 4 Jun 2019 08:34:39 +0000 (08:34 +0000)]
Blueprint alignment

Blueprint is aligned with the dmaap topic configuration

Change-Id: I4303724eed21047b87a8b7438970625b08811d22
Issue-ID: DCAEGEN2-1145
Signed-off-by: YongchaoWu <yongchao.wu@est.tech>
6 years agoMerge "Fix a small bug for swagger-ui"
Henrik Andersson [Tue, 4 Jun 2019 06:59:40 +0000 (06:59 +0000)]
Merge "Fix a small bug for swagger-ui"

6 years agoMerge "Bugfixes, Generalizing Data File Collection to handle any type of file"
Henrik Andersson [Tue, 4 Jun 2019 06:58:01 +0000 (06:58 +0000)]
Merge "Bugfixes, Generalizing Data File Collection to handle any type of file"

6 years agoMerge "DFC, Purging the cache of published files occasionally fails"
Henrik Andersson [Tue, 4 Jun 2019 06:57:26 +0000 (06:57 +0000)]
Merge "DFC, Purging the cache of published files occasionally fails"

6 years agoMerge "Code formatting with maven"
Henrik Andersson [Tue, 4 Jun 2019 06:56:42 +0000 (06:56 +0000)]
Merge "Code formatting with maven"

6 years agoBugfixes, Generalizing Data File Collection to handle any type of file 26/89126/1
PatrikBuhr [Mon, 3 Jun 2019 08:38:10 +0000 (08:38 +0000)]
Bugfixes, Generalizing Data File Collection to handle any type of file

- When a change ID was not configured, the task counter was not decreased.
The result was that the DFC stopped polling,

- When the check if a file is a already published fails
(most likely to a problem is the DR), the DFC will try
to publish it (instead of just ingnoring it).

Change-Id: If9f5b962210f809d5d2ae0aa60d3a7f99099c058
Issue-ID: DCAEGEN2-1532
Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
6 years agoDFC, Purging the cache of published files occasionally fails 20/89120/1
PatrikBuhr [Mon, 3 Jun 2019 06:59:50 +0000 (06:59 +0000)]
DFC, Purging the cache of published files occasionally fails

Solved a threading issue.

Change-Id: I1272cadd5d8bc8bb8ffc4c3d33c9b5c706470864
Issue-ID: DCAEGEN2-1588
Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
6 years agoGeneralizing Data File Collection to handle any type of file 21/88521/9
PatrikBuhr [Wed, 29 May 2019 09:39:53 +0000 (09:39 +0000)]
Generalizing Data File Collection to handle any type of file

Extension of the DFC to be able to handle any file types types, which are published on different DR feeds.
This association between file type and DR feed is defined by configuration.
The file type is defined by the changeIdentifier in the fileReady VES message reported from the PNF.

The creation of DR feeds and configuration will be done by the DMAAP plugin, but that is
not tested yet.

Change-Id: I13b36acd926a6941ee733e6b37922049fb54a5d9
Issue-ID: DCAEGEN2-1532
Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
6 years agoCode formatting with maven 39/88739/1
PatrikBuhr [Wed, 29 May 2019 08:31:52 +0000 (08:31 +0000)]
Code formatting with maven

mvn formatter:format spotless:apply process-sources

The follwowing settings are updated from the onap origin:

<setting id="org.eclipse.jdt.core.formatter.comment.line_length" value="120"/>
<setting id="org.eclipse.jdt.core.formatter.lineSplit" value="120"/>

Change-Id: If9c368edc36399e8662aea20c4fbff53756d367a
Issue-ID: DCAEGEN2-1538
Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
6 years agoFix a small bug for swagger-ui 03/88603/1
yanhuanwang [Tue, 28 May 2019 08:15:18 +0000 (08:15 +0000)]
Fix a small bug for swagger-ui

swagger-ui and actuator-ui are missing.
This commit is to fix this bug.

Change-Id: I237ad1739dd23b7316a3efecb40a9bd78247ff33
Issue-ID: DCAEGEN2-1535
Signed-off-by: yanhuanwang <martin.yan.seu@gmail.com>
6 years agoMerge "Fix for robustness"
Henrik Andersson [Wed, 22 May 2019 10:15:51 +0000 (10:15 +0000)]
Merge "Fix for robustness"

6 years agoFix, skip FTP retry in certain cases 45/87945/2
PatrikBuhr [Fri, 17 May 2019 06:38:19 +0000 (06:38 +0000)]
Fix, skip FTP retry in certain cases

In certain conditions there is no reason to retry fetching files.
For instance when the file is removed in the PNF or when
the password/certificate is wrong.

When the DFC is started there are always
queued VES events that referes to removed files which in turn results
in that the DFC will retry fetching these files in vain.
The DFC house keeps its number of concurrents tasks to not exeed quotas
for memory,open file descriptors etc.
As more threads are occupied with retrying, the fewer threads can do
their intended work, which decreases the throughput.

Testing has showed that already when the number of PNFs are 10, the throughput
is radically decreased (and the problem is then escalating).

Change-Id: I4833724a3ef3509025f4a0a438c6c8025932b0f4
Issue-ID: DCAEGEN2-1508
Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
6 years agoFix for robustness 44/87944/1
PatrikBuhr [Fri, 17 May 2019 06:19:25 +0000 (06:19 +0000)]
Fix for robustness

In the long time stability test it has showed that when there are an exessive amount of FileReady events,  the DFC will fetch these from the MR and build up an internal queue.

This has the effect that the DFC will consume too much memory. Itmight also affect load balancing (in case of several DFC instances) so that one might consume all events and the others get nothing.

The DFC should not fetch new FileReady events when it fully loaded with work.

Change-Id: I58665edd678d2f1c8a32d0e56455228b522aab40
Issue-ID: DCAEGEN2-1509
Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
6 years agoBugfix: add MDC context to http redirect logs 30/87530/4
RehanRaza [Mon, 13 May 2019 14:02:39 +0000 (14:02 +0000)]
Bugfix: add MDC context to http redirect logs

Change-Id: Icdf00480eed43536a8b4693cbb208560f0881c95
Issue-ID: DCAEGEN2-1462
Signed-off-by: RehanRaza <muhammad.rehan.raza@est.tech>
6 years agoMerge "Clean up pom files"
Vijay Venkatesh Kumar [Fri, 10 May 2019 16:37:40 +0000 (16:37 +0000)]
Merge "Clean up pom files"

6 years agoCode improvement, FilePublishInformation serializing 49/87149/2
PatrikBuhr [Wed, 8 May 2019 07:18:24 +0000 (07:18 +0000)]
Code improvement, FilePublishInformation serializing

Updated serializer of FilePublishInformation so that only elements that
should be part of publishing is included.

Change-Id: If5632af8bf9fd26f8bc96b25d8bfbccf63dfec2f
Issue-ID: DCAEGEN2-1488
Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
6 years agoClean up pom files 53/87153/3
elinuxhenrik [Wed, 17 Apr 2019 09:07:48 +0000 (11:07 +0200)]
Clean up pom files

Change-Id: Ifede82e15ebc38af92ade8259a6e1f641946b815
Issue-ID: DCAEGEN2-1487
Signed-off-by: elinuxhenrik <henrik.b.andersson@est.tech>
6 years agoMerge "Bugfix, trace context"
Henrik Andersson [Tue, 7 May 2019 11:27:11 +0000 (11:27 +0000)]
Merge "Bugfix, trace context"

6 years agoOwnership fix in config directory 14/86814/3
TamasBakai [Sun, 5 May 2019 08:32:53 +0000 (08:32 +0000)]
Ownership fix in config directory

Change-Id: Iec3ca75c42642da27c52bccd6d60811c434f778a
Issue-ID: DCAEGEN2-1466
Signed-off-by: TamasBakai <tamas.bakai@est.tech>
6 years agoBugfix, trace context 88/86788/1
PatrikBuhr [Thu, 2 May 2019 07:17:11 +0000 (07:17 +0000)]
Bugfix, trace context

When retrying of file publish, the RequestID in the trace
context is incorect. The context must be expicitly updated
since retrying is executed in a separate thread.

Change-Id: I1c5dc0ba0e53c1219e23f93d8c5b420996a16c93
Issue-ID: DCAEGEN2-1462
Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
6 years agoExtend Swagger for actuator end-points also 49/86249/1 1.1.1 1.1.2
TamasBakai [Thu, 25 Apr 2019 08:47:13 +0000 (08:47 +0000)]
Extend Swagger for actuator end-points also

Change-Id: Ia036f7d79d25ceb2f1e465d79f17db1acbabbfbf
Issue-ID: DCAEGEN2-1443
Signed-off-by: TamasBakai <tamas.bakai@est.tech>
6 years agoBugfix, serializing in DataRouterPublisher 76/85476/1
PatrikBuhr [Tue, 16 Apr 2019 14:15:12 +0000 (14:15 +0000)]
Bugfix, serializing in DataRouterPublisher

This fixes a bug introduced in c7b0c7014688c447d411f791a094440403e65228.
The new property must be removed before publishing to data router.
The consequense of the bug is that the DR rejects the publishing.

Change-Id: Ib42cbed4decdba4c13678cea47dff35ca77679a5
Issue-ID: DCAEGEN2-1305
Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
6 years agoImproved logging 07/85207/1
PatrikBuhr [Fri, 12 Apr 2019 11:57:47 +0000 (11:57 +0000)]
Improved logging

File name is used in Requext_ID to make it
more easy to trouble shoot.

Change-Id: Ied98766f1a177a9bda35ec5892a60d06619ff3a7
Issue-ID: DCAEGEN2-1305
Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
6 years agoFixed build problems 47/85147/4
PatrikBuhr [Fri, 12 Apr 2019 08:24:36 +0000 (08:24 +0000)]
Fixed build problems

Fixed a build problem cause by a merge
error from previous commits.

Fixes warnings from maven compile

Change-Id: I1ce88b2686b3be5ecd17113cbfde3e65297c27c6
Issue-ID: DCAEGEN2-1118
Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
6 years agoMerge "Robustness issue"
Vijay Venkatesh Kumar [Thu, 11 Apr 2019 20:53:30 +0000 (20:53 +0000)]
Merge "Robustness issue"

6 years agoMerge "Enabling Spring boot actuator"
Vijay Venkatesh Kumar [Thu, 11 Apr 2019 15:47:00 +0000 (15:47 +0000)]
Merge "Enabling Spring boot actuator"

6 years agoRobustness issue 26/85026/2
PatrikBuhr [Thu, 11 Apr 2019 15:36:48 +0000 (15:36 +0000)]
Robustness issue

Limiting the number of paralell threads when
retying of ftp or publish.
Previously there was no upper limit on how many paralel
threads that could be started by retry attempts.
For instance, a worker thread with 100 new files
would start 100 new threads when the ftt server
was unavailable.

Change-Id: Ia8792f03ea55c0c467ef248ddc9d59187c06a946
Issue-ID: DCAEGEN2-1118
Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
6 years agoEnabling Spring boot actuator 59/85059/2
PatrikBuhr [Thu, 11 Apr 2019 15:27:07 +0000 (15:27 +0000)]
Enabling Spring boot actuator

This makes it possible by a REST API:
- change logging levels
- get the application log
- get build info for the DFC
- get various metrixes for the execution
  such as heap consumption, number of threads etc.

Change-Id: I570bc0db6a9b9977ecfd83389b463fe652e4ba16
Issue-ID: DCAEGEN2-1428
Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
6 years agoExtend heartbeat to also include som health status 08/84508/7
elinuxhenrik [Thu, 11 Apr 2019 07:04:31 +0000 (07:04 +0000)]
Extend heartbeat to also include som health status

Add info about the internal status of DFC to the healtCheck message.

Change-Id: Ic2ad3e5b97c42d6affe1ac4e1d7cf2274027f881
Issue-ID: DCAEGEN2-1403
Signed-off-by: elinuxhenrik <henrik.b.andersson@est.tech>
6 years agoHousekeeping 95/84095/7
elinuxhenrik [Tue, 26 Mar 2019 09:22:23 +0000 (10:22 +0100)]
Housekeeping

No functional changes made in this commit.

Removed CheckStyle and Sonar warnings.
Formatted code.
Renamed methods and classes for better understanding.
Removed unnecessary classes.
Moved all code to single project.

Change-Id: Ie3feb6c6a985e94a382812aa083dcf57bc46c7b3
Issue-ID: DCAEGEN2-1367
Signed-off-by: elinuxhenrik <henrik.b.andersson@est.tech>
6 years agoUpdating certificates 02/79902/5
TamasBakai [Wed, 10 Apr 2019 10:02:21 +0000 (10:02 +0000)]
Updating certificates

Issue-ID: DCAEGEN2-1300
Change-Id: I9ce457b69e5866e77a465fb040d196b8905ca274
Signed-off-by: TamasBakai <tamas.bakai@est.tech>
6 years agoImproved logging 46/83946/4
PatrikBuhr [Thu, 4 Apr 2019 14:41:34 +0000 (14:41 +0000)]
Improved logging

Fixed problem with startup which lead
to that the REST API was not working running
locally.

Fixed problem with DmaapProducerHttpClient which would use
no timeout, which can lead to infinitly haninging threads.
A long timeout is used instead.

Change-Id: I28469b1b3aaad0dab4cf247bb8af968e71a60133
Issue-ID: DCAEGEN2-1305
Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
6 years agoChange file name sent to DataRouter 22/83622/2
elinuxhenrik [Fri, 29 Mar 2019 09:05:28 +0000 (09:05 +0000)]
Change file name sent to DataRouter

In the previous commit for this issue, the file name sent to DataRouter
was changed. This turned out to be wrong, so in this commit it is
changed back again.

Change-Id: I9b0d726f406758846fd256626e73edf59d710c8b
Issue-ID: DCAEGEN2-1256
Signed-off-by: elinuxhenrik <henrik.b.andersson@est.tech>
6 years agoMerge "Fix DataFile - Align Dmaap Topic configuration"
Vijay Venkatesh Kumar [Thu, 28 Mar 2019 02:58:11 +0000 (02:58 +0000)]
Merge "Fix DataFile - Align Dmaap Topic configuration"

6 years agoupdate INFO.yaml 96/83396/2
Vijay Venkatesh Kumar [Tue, 26 Mar 2019 13:14:54 +0000 (13:14 +0000)]
update INFO.yaml

Change-Id: I147b698c40bb80a09c885c16d08137dcb4a0cf69
Signed-off-by: Vijay Venkatesh Kumar <vv770d@att.com>
Issue-ID: DCAEGEN2-1270
Signed-off-by: Vijay Venkatesh Kumar <vv770d@att.com>
6 years agoLocal filename updated, stability issues 97/82597/6
PatrikBuhr [Fri, 22 Mar 2019 14:29:29 +0000 (14:29 +0000)]
Local filename updated, stability issues

The local filename is changed so it contains PNF name
instead of the PNF IP address.

The paralellism is restricted to 100 worker threads in order to
solve problems with too many open file descriptors and out of memory.

Logging is improved.

Change-Id: I24ce2e23020cc253a3c7bebac1ab5cf703b5b144
Issue-ID: DCAEGEN2-1118
Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
6 years agoAdd check to DataRouter if file has been published 29/78929/14
maximesson [Thu, 21 Mar 2019 15:58:55 +0000 (15:58 +0000)]
Add check to DataRouter if file has been published

For each file in the FileReady message that DFC does not know if it has been
published yet, it should ask DataRouter if it has been published already to
avoid downloading and publishing a file more than once.

Change-Id: I18117a6e968ec929aa255052a4c44f890a8ed39d
Issue-ID: DCAEGEN2-1256
Signed-off-by: maximesson <maxime.bonneau@est.tech>
6 years agoFix DataFile - Align Dmaap Topic configuration 33/82433/2
YongchaoWu [Mon, 18 Mar 2019 14:17:02 +0000 (14:17 +0000)]
Fix DataFile - Align Dmaap Topic configuration

Remove additional configuration in Blueprints
Remove additional configruation in component
specs

Change-Id: I0d546227af0be485f2ffed27a93c701a7654ff8e
Issue-ID: DCAEGEN2-1145
Signed-off-by: YongchaoWu <yongchao.wu@est.tech>
6 years agoMerge "Thread safety issues"
Tony Hansen [Mon, 18 Mar 2019 13:45:01 +0000 (13:45 +0000)]
Merge "Thread safety issues"

6 years agoDataFile - Align Dmaap Topic configuration 83/82083/6
YongchaoWu [Thu, 14 Mar 2019 15:14:55 +0000 (15:14 +0000)]
DataFile - Align Dmaap Topic configuration

Blueprints alignment
Configbinding service support
Modify the policy model complaint to R4 model expected by Policy

Change-Id: Ic53f4ab2a5867e5cdf0fc735be180715077154ee
Issue-ID: DCAEGEN2-1145
Signed-off-by: YongchaoWu <yongchao.wu@est.tech>
6 years agoThread safety issues 33/81833/4
PatrikBuhr [Thu, 14 Mar 2019 14:27:30 +0000 (14:27 +0000)]
Thread safety issues

The TrustManager is now loaded and initialized once
in a thread safe way (instead of each time it is used).

Removed some unneeded wrappers.

Using AutoCloseable for FTP clients to make
sure they are closed in case of exceptions.

Made AppConfig thread safe.

Change-Id: Ia6a2c8a76bf960013180fdd7c53ae0ff17b26505
Issue-ID: DCAEGEN2-1118
Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
6 years agoMerge "DFC logging according to ONAP specification"
Vijay Venkatesh Kumar [Tue, 12 Mar 2019 14:57:26 +0000 (14:57 +0000)]
Merge "DFC logging according to ONAP specification"

6 years agoMerge "Update Readme.md"
Vijay Venkatesh Kumar [Mon, 11 Mar 2019 14:29:12 +0000 (14:29 +0000)]
Merge "Update Readme.md"