cps.git
5 months agoRefactor: Improve code quality and compliance with SonarQube standards 57/139257/2
sourabh_sourabh [Wed, 23 Oct 2024 14:12:41 +0000 (15:12 +0100)]
Refactor: Improve code quality and compliance with SonarQube standards

- Consolidated multiple `if` statements to enhance readability and maintainability.
- Replaced traditional `instanceof` checks followed by casting with the new pattern matching syntax for improved type safety and clarity.
- Removed unnecessary casts and streamlined code in `addYangLeafList` and `addDataNodeFromNormalizedNode` methods.

Issue-ID: CPS-1957
Change-Id: I46696dc184b6b3c3d0fe3a6bd12267a4e0d76193
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
5 months agoResize cps expectation 59/139259/1
halil.cakal [Wed, 23 Oct 2024 14:51:56 +0000 (15:51 +0100)]
Resize cps expectation

- set expectation to 70 (cmhanldes/second) for discovery case

Issue-ID: CPS-2350

Change-Id: I3d0f9f5e20de87f79ebbe569c7905f4eefed35e8
Signed-off-by: halil.cakal <halil.cakal@est.tech>
5 months agoMerge "Enable 2 minute initial module processing delay"
Toine Siebelink [Mon, 21 Oct 2024 10:40:18 +0000 (10:40 +0000)]
Merge "Enable 2 minute initial module processing delay"

5 months agoEnable 2 minute initial module processing delay 30/139230/1
danielhanrahan [Fri, 18 Oct 2024 11:38:12 +0000 (12:38 +0100)]
Enable 2 minute initial module processing delay

Issue-ID: CPS-2403
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I614313461bd36ca351a8d42d9683b88460090b50

5 months agoUpdate SpotBugs dependency 29/139229/1
danielhanrahan [Fri, 18 Oct 2024 10:19:04 +0000 (11:19 +0100)]
Update SpotBugs dependency

- update spotbugs to 4.8.6
- remove old findbugs dependency (spotbugs replaces findbugs)
- remove an unused SupressFBWarnings

Issue-ID: CPS-2458
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: If9d5571ae4293a2653a117a988414af2452647c0

5 months agoRelease Stage 7 for 3.5.4 26/139226/5
seanbeirne [Thu, 17 Oct 2024 15:49:30 +0000 (16:49 +0100)]
Release Stage 7 for 3.5.4

Issue-ID: CPS-2457
Change-Id: Id9a267c587f16f3fb5edb558b33cd0003864ed93
Signed-off-by: seanbeirne <sean.beirne@est.tech>
5 months agoRelease Stage 6 for 3.5.4 24/139224/2
seanbeirne [Thu, 17 Oct 2024 15:37:00 +0000 (16:37 +0100)]
Release Stage 6 for 3.5.4

Issue-ID: CPS-2457
Change-Id: I9aa7265d8179c55a71a7722e3ef42c44b0f8dbfe
Signed-off-by: seanbeirne <sean.beirne@est.tech>
5 months agoRelease Stage 5 for 3.5.4 23/139223/2
seanbeirne [Thu, 17 Oct 2024 15:30:38 +0000 (16:30 +0100)]
Release Stage 5 for 3.5.4

Issue-ID: CPS-2457
Change-Id: I8bc9b2588539b64a6c8de936b6df97111665f1ed
Signed-off-by: seanbeirne <sean.beirne@est.tech>
5 months agoRelease notes and documentation updates for 3.5.4 22/139222/2
danielhanrahan [Thu, 17 Oct 2024 13:55:49 +0000 (14:55 +0100)]
Release notes and documentation updates for 3.5.4

- Release notes update
- Update openapi info.version to 3.5.4
- Update scheduled-tasks docs to reflect changes in retry mechanism
- Update deployment docs to update configurable timer parameters
- Remove unused locked-module-sync.sleep-time-ms

Issue-ID: CPS-2457
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ieb559bbfe348848c4b8669410861edfeeb6c9bf3

5 months agoRefactor and Optimize ModuleSyncWatchdog for Improved Lock Handling and Queue Management 08/139208/4
sourabh_sourabh [Thu, 17 Oct 2024 07:22:56 +0000 (08:22 +0100)]
Refactor and Optimize ModuleSyncWatchdog for Improved Lock Handling and Queue Management

- Enhanced logging for populateWorkQueueIfNeeded and resetPreviouslyLockedCmHandles
methods, improving clarity and error handling.
- Improved readability and maintainability of the locking mechanism with
  Hazelcast's FencedLock.
- Optimized error handling in catch blocks, logging detailed exception messages and stack
traces for better troubleshooting.
- Refined lock acquisition and release flow, with clear log messages for both successful
and failed lock operations, ensuring safe handling of Hazelcast distributed locks.

Issue-ID: CPS-2403
Change-Id: Ie089f36a817d4965782235b51ee987ef054516b1
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
5 months agoMerge "Update documentation related to hazelcast"
Toine Siebelink [Wed, 16 Oct 2024 16:07:56 +0000 (16:07 +0000)]
Merge "Update documentation related to hazelcast"

5 months agoUpdate documentation related to hazelcast 95/139195/3
mpriyank [Mon, 14 Oct 2024 12:44:31 +0000 (13:44 +0100)]
Update documentation related to hazelcast

- removed old entries of data structures which are removed now.
- updated the documentation to correctly reflect our infrastructure
  startegy related to hazelcast.
- fixed few NB comments on the merged patch.

Issue-ID: CPS-2408
Change-Id: Id7ce4ac695ad94742490ac5d2be89cd23ffda3e5
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
5 months agoMerge "Add missing duplication test in integration test suite"
Sourabh Sourabh [Wed, 16 Oct 2024 12:33:02 +0000 (12:33 +0000)]
Merge "Add missing duplication test in integration test suite"

5 months agoMerge "Added additional logging"
Daniel Hanrahan [Wed, 16 Oct 2024 09:21:31 +0000 (09:21 +0000)]
Merge "Added additional logging"

5 months agoAdd missing duplication test in integration test suite 03/139203/1
ToineSiebelink [Tue, 15 Oct 2024 17:01:30 +0000 (18:01 +0100)]
Add missing duplication test in integration test suite

- Added test for  ModuleSerive.createSchemaSetFromModules()
- Unintendedly Updated one swagger file in docs because
1) DT team forgot to update the relevant file
2) Gers committed pom update to automatically update swagger files
3) I ran 'mvn install'

Issue-ID: CPS-2399
Change-Id: I33261c32bbca58ff21aff3cde9588617199f104c
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
5 months agoAdded additional logging 00/139200/5
sourabh_sourabh [Tue, 15 Oct 2024 11:11:16 +0000 (12:11 +0100)]
Added additional logging

- Added logging for non existing xpath while updating data notes and its
  decendants.

Issue-ID: CPS-2403
Change-Id: I1ecaf2aed77aec8d266f6a758fe80ee96717d9c7
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
5 months agoMerge "add documentaiton for cps-and-ncmp deployment"
Priyank Maheshwari [Tue, 15 Oct 2024 10:20:31 +0000 (10:20 +0000)]
Merge "add documentaiton for cps-and-ncmp deployment"

5 months agoadd documentaiton for cps-and-ncmp deployment 81/139181/5
halil.cakal [Wed, 9 Oct 2024 16:33:19 +0000 (17:33 +0100)]
add documentaiton for cps-and-ncmp deployment

- the optimum percentage of the JVM allocation documented
- remove anchorDataCache from the documentation
  since no more used

Issue-ID: CPS-2180

Change-Id: Ie1b8c43136a38f8da5a865493bed1df030a497d6
Signed-off-by: halil.cakal <halil.cakal@est.tech>
5 months agoMerge "Copy Swagger file to docs folder"
Toine Siebelink [Mon, 14 Oct 2024 12:13:35 +0000 (12:13 +0000)]
Merge "Copy Swagger file to docs folder"

5 months agoMerge "use one hazelcast instance per jvm"
Lee Anjella Macabuhay [Thu, 10 Oct 2024 12:51:40 +0000 (12:51 +0000)]
Merge "use one hazelcast instance per jvm"

5 months agouse one hazelcast instance per jvm 30/139130/5
mpriyank [Tue, 10 Sep 2024 15:12:28 +0000 (16:12 +0100)]
use one hazelcast instance per jvm

- updated all the cache(distributed datastructures) to use the same instance config now.
- there will be just one instance per JVM now
- Better cleanup of hz instance in the testware
- Refactored the testware to verify the configs
- Expected Impact on lowering the memory usage, less number of TCP
  communications between members
- NOTE: we need to do a full regression as the changes impacts all the
  use cases which involves cache

Issue-ID: CPS-2408
Change-Id: I7564992a9990f44ef3defb4f50cb7d094cad7b92
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
5 months agoCopy Swagger file to docs folder 82/139182/1
egernug [Thu, 10 Oct 2024 09:11:27 +0000 (10:11 +0100)]
Copy Swagger file to docs folder

To avoid manual copy/paste errors updates to docs folder for RTD are generated and copied when compiling CPS

Issue-ID: CPS-2332

Change-Id: I1a64f7cc74a82341403f8125adc5216ef046429c
Signed-off-by: egernug <gerard.nugent@est.tech>
5 months agoMerge "Autodiscover cps-and-ncmp containers when load balancing"
Daniel Hanrahan [Thu, 10 Oct 2024 09:53:47 +0000 (09:53 +0000)]
Merge "Autodiscover cps-and-ncmp containers when load balancing"

5 months agoMerge "Added architecture tests to validate ONLY NCMP REST/Service dependencies"
Sourabh Sourabh [Wed, 9 Oct 2024 15:09:11 +0000 (15:09 +0000)]
Merge "Added architecture tests to validate ONLY NCMP REST/Service dependencies"

5 months agoAdded architecture tests to validate ONLY NCMP REST/Service dependencies 03/139003/17
leventecsanyi [Mon, 23 Sep 2024 14:01:03 +0000 (16:01 +0200)]
Added architecture tests to validate ONLY NCMP REST/Service dependencies

  - removed old tests and added new test cases for the NCMP controller and service layers
  - other dependencies will be added in later commits

Issue-ID: CPS-2422
Change-Id: Iabc4b53e9414477cf3d0b2e1e5a03ba8b4995169
Signed-off-by: leventecsanyi <levente.csanyi@est.tech>
5 months agoMerge "Added K6 script test to check a throughput of CPS kafka consumer"
Daniel Hanrahan [Wed, 9 Oct 2024 11:22:46 +0000 (11:22 +0000)]
Merge "Added K6 script test to check a throughput of CPS kafka consumer"

5 months agoMerge "Implementation of Data validation feature in Create a Node API"
Toine Siebelink [Wed, 9 Oct 2024 09:37:42 +0000 (09:37 +0000)]
Merge "Implementation of Data validation feature in Create a Node API"

5 months agoAutodiscover cps-and-ncmp containers when load balancing 72/139172/4
halil.cakal [Mon, 7 Oct 2024 15:48:05 +0000 (16:48 +0100)]
Autodiscover cps-and-ncmp containers when load balancing

- remove hard-coded container names from load balancer(nginx) and
  prometheus

Issue-ID: CPS-2437

Change-Id: Ia64b949c2eb3e4d67e123b046e66d5bfdda2c801
Signed-off-by: halil.cakal <halil.cakal@est.tech>
5 months agoFine tune JVM parameters for cps-and-ncmp 97/138997/6
halil.cakal [Mon, 23 Sep 2024 10:05:16 +0000 (11:05 +0100)]
Fine tune JVM parameters for cps-and-ncmp

- limit the total memory with 2G
- set heap memory 75% of the total RAM

Issue-ID: CPS-2180

Change-Id: Ia8a6c2bd40866052caba4f332ac563e1ed829212
Signed-off-by: halil.cakal <halil.cakal@est.tech>
5 months agoImplementation of Data validation feature in Create a Node API 76/138676/28
Arpit Singh [Fri, 9 Aug 2024 06:53:49 +0000 (12:23 +0530)]
Implementation of Data validation feature in Create a Node API

Added support to validate JSON/XML data without the need of persisting
it in the databse.
 - added "dryRunInQuery" flag as a new query parameter
 - added new method as part of CpsDataService layer to perform data
   validation
 - added new method in yang parser "validateData" to validate
   data without persisting it

Issue-ID: CPS-2361
Change-Id: I43dd33cc6120576d0fac606d5c4b0168d107311d
Signed-off-by: Arpit Singh <as00745003@techmahindra.com>
5 months agoMerge "Policy Executor: handle errors, part 2 (fighting between IntelliJ and Checksty...
Sourabh Sourabh [Tue, 8 Oct 2024 15:51:01 +0000 (15:51 +0000)]
Merge "Policy Executor: handle errors, part 2 (fighting between IntelliJ and Checkstyle best practices)"

5 months agoMerge "Re-adjust cps expectations"
Lee Anjella Macabuhay [Mon, 7 Oct 2024 14:08:34 +0000 (14:08 +0000)]
Merge "Re-adjust cps expectations"

5 months agoMerge "RTD update for delta API 2"
Priyank Maheshwari [Mon, 7 Oct 2024 08:42:08 +0000 (08:42 +0000)]
Merge "RTD update for delta API 2"

6 months agoBump CPS to 3.5.4-SNAPSHOT 22/139122/1
mpriyank [Fri, 4 Oct 2024 15:59:40 +0000 (16:59 +0100)]
Bump CPS to 3.5.4-SNAPSHOT

- bump cps to 3.5.4-SNAPSHOT
- updated release notes

Issue-ID: CPS-2434
Change-Id: I3d5e6f37711bb29b61b598cc5af57ecfdc35cbba
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
6 months ago[CPS] Step-3: OSLO release changes 21/139121/1
sourabh_sourabh [Fri, 4 Oct 2024 15:23:06 +0000 (16:23 +0100)]
[CPS] Step-3: OSLO release changes

- cps maven stage docker yaml is created for release 3.5.3

Issue-ID:CPS-2434

Change-Id: I4048b9f54b9c0b0eb433fef9d9c403d54c39bc35
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
6 months ago[CPS] Step-2: OSLO release changes 20/139120/2
sourabh_sourabh [Fri, 4 Oct 2024 15:09:08 +0000 (16:09 +0100)]
[CPS] Step-2: OSLO release changes

- cps maven stage master yaml is created for release 3.5.3

Issue-ID:CPS-2434
Change-Id: I5353eb64252b3e071b5203d931f0f646ef290c13
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
6 months ago[CPS] OSLO release changes 18/139118/3
sourabh_sourabh [Fri, 4 Oct 2024 13:35:32 +0000 (14:35 +0100)]
[CPS] OSLO release changes

- Updated open api for cps, ncmp and policy apis.
- Updated release notes with bus and features.

Issue-ID: CPS-2434
Change-Id: Ib4d80f3939b6e267f228177c817e43832a9384db
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
6 months agoMerge "[BUG] Fix memory leak related to using arrays in Hibernate"
Lee Anjella Macabuhay [Fri, 4 Oct 2024 12:11:17 +0000 (12:11 +0000)]
Merge "[BUG] Fix memory leak related to using arrays in Hibernate"

6 months agoRevert "[1/2] Move Swagger API to docs folder" 35/139135/2
Lee Anjella Macabuhay [Thu, 3 Oct 2024 13:47:14 +0000 (13:47 +0000)]
Revert "[1/2] Move Swagger API to docs folder"

This reverts commit b76392e2d1629d4eb67b10c450cdd954ef678966.

Reason for revert: Release blocker

Change-Id: Ic477dbaaad34992c8f440981d92f12b977cdf96e
Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech>
6 months agoRTD update for delta API 2 65/137565/25
Arpit Singh [Mon, 6 May 2024 14:42:52 +0000 (20:12 +0530)]
RTD update for delta API 2

 - Documentation update for Delta Feature and Delta API 1
 - Added documentation for API 2: Delta between anchor
   and JSON payload

Issue-ID: CPS-2212
Signed-off-by: Arpit Singh <as00745003@techmahindra.com>
Change-Id: Id74cd930ce48e5cb414aa62c5381b79675346a37

6 months ago[BUG] Fix memory leak related to using arrays in Hibernate 13/139113/5
danielhanrahan [Wed, 2 Oct 2024 20:31:04 +0000 (21:31 +0100)]
[BUG] Fix memory leak related to using arrays in Hibernate

The use of arrays like String[] instead of Collection<String> in
JpaRepository methods is causing a memory leak, most likely due to a
bug in the hypersistence-utils dependency which provides the feature.
Note code using arrays in JDBC (via jdbcTemplate) is not affected.

This patch removes most uses of arrays in Java, except one needed for
FragmentPrefetchRepository using JDBC setArray(), which is safe.

Issue-ID: CPS-2430
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I94f8c3d4c8c32ebe0978c08d3226a196a95bf3b9

6 months agoRevert "Fix inputSpec for openapi code gen" 34/139134/3
Lee Anjella Macabuhay [Thu, 3 Oct 2024 13:46:59 +0000 (13:46 +0000)]
Revert "Fix inputSpec for openapi code gen"

This reverts commit 4baa75b06dcfb9365a8ea5cde03d77d7f21fa01c.

Reason for revert: Release blocker

Change-Id: I6156461de64dcec41e3f84cbd0cf90f63938bea6
Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech>
6 months agoRevert "CI: Add test (silent) checkov scan as part of the verify process" 16/139116/1
Jessica Wagantall [Thu, 3 Oct 2024 16:08:35 +0000 (09:08 -0700)]
Revert "CI: Add test (silent) checkov scan as part of the verify process"

This reverts commit c8e2b3383830395bb4bc37371a9c6119a316cb53.

Issue-ID: CIMAN-33
Change-Id: I5120097ad05394e3667a868c4b7edd44ef1aa070
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
6 months agoPolicy Executor: handle errors, part 2 34/139034/10
ToineSiebelink [Tue, 1 Oct 2024 17:40:39 +0000 (18:40 +0100)]
Policy Executor: handle errors, part 2
(fighting between IntelliJ and Checkstyle best practices)

- non-2xx responses are processed using web client exceptions
- handle unknown host exception
- upgraded spotbugs (checkstyle and related mvn plugin)
- fixed some small spotbugs due to upgrade
- added commented instructions in docker compose to enable debugging
- added some environment variables for policy executor configuration
- extract out Sleeper in stub service to achieve 100% coverage
- added cause to Policy Executor exceptions where applicable
- ignored (new) spotbug rule about catch NPE because of issue in 3pp
- ignored (new) spotbug rule about \n in string due to multiline string block

Issue-ID: CPS-2412
Change-Id: I6835a73320c436cbeea12cc7a06f15899eec7bf1
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
6 months agoAdded K6 script test to check a throughput of CPS kafka consumer 40/139040/4
sourabh_sourabh [Mon, 30 Sep 2024 10:13:30 +0000 (11:13 +0100)]
Added K6 script test to check a throughput of CPS kafka consumer

- Added new K6 script to produce cm avc enents. (It is not a part
  regular automated tests)
- Added a json resource (avc event)

Issue-ID: CPS-2329
Change-Id: I6446bc120382257c5039df17fd34b84b7c6f6550
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
6 months agoRe-adjust cps expectations 38/139038/3
halil.cakal [Fri, 27 Sep 2024 13:43:00 +0000 (14:43 +0100)]
Re-adjust cps expectations

- aligning cps expectation for new plots

Issue-ID: CPS-2350

Change-Id: I5b300c4f5432bf2bdfe8a8b0d52b2ce60bd338d6
Signed-off-by: halil.cakal <halil.cakal@est.tech>
6 months agoUpdate JIRA and Wiki links host name from wiki.onap.org to lf-onap.atlassian.net 95/139095/10
sourabh_sourabh [Wed, 2 Oct 2024 15:37:47 +0000 (16:37 +0100)]
Update JIRA and Wiki links host name from wiki.onap.org to lf-onap.atlassian.net

Issue-ID: CPS-2432
Change-Id: I53f380a4fe0968d1ec9f1393651c7c43f03d0d95
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
6 months agoMerge "BugFix: 500 Server error when upgrade and service restart with data in DB"
Sourabh Sourabh [Mon, 30 Sep 2024 11:10:19 +0000 (11:10 +0000)]
Merge "BugFix: 500 Server error when upgrade and service restart with data in DB"

6 months agoFix inputSpec for openapi code gen 37/139037/1
egernug [Fri, 27 Sep 2024 12:33:24 +0000 (13:33 +0100)]
Fix inputSpec for openapi code gen

Existing inputSpec only accessible from cps root. Changing path to be accessible elsewhere

Issue-ID: CPS-2332

Change-Id: I94ada93ea3c88d2fdca84d49c21e1ffc2923f4b2
Signed-off-by: egernug <gerard.nugent@est.tech>
6 months agoMerge "[1/2] Move Swagger API to docs folder"
Toine Siebelink [Thu, 26 Sep 2024 08:59:12 +0000 (08:59 +0000)]
Merge "[1/2] Move Swagger API to docs folder"

6 months agoCI: Add test (silent) checkov scan as part of the verify process 33/139033/1
Jessica Wagantall [Wed, 25 Sep 2024 20:17:55 +0000 (13:17 -0700)]
CI: Add test (silent) checkov scan as part of the verify process

Issue-ID: CIMAN-33
Change-Id: I70b171824acb913f67fd28c119bfbcbc1cc1e470
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
6 months agoBugFix: 500 Server error when upgrade and service restart with data in DB 32/139032/1
halil.cakal [Wed, 25 Sep 2024 10:07:01 +0000 (11:07 +0100)]
BugFix: 500 Server error when upgrade and service restart with data in DB

- return NONE for get effective trust level api if the trustlevel caches
  empty (restart case)

Issue-ID: CPS-2409

Change-Id: I72a755ca6ba5d4a78f5458a235c0d1f43aaa8d53
Signed-off-by: halil.cakal <halil.cakal@est.tech>
6 months agoMerge "[k6] CM handles searches using different filters"
Priyank Maheshwari [Wed, 25 Sep 2024 09:26:32 +0000 (09:26 +0000)]
Merge "[k6] CM handles searches using different filters"

6 months agoPolicy Executor: handle errors 82/138982/6
ToineSiebelink [Thu, 19 Sep 2024 16:23:58 +0000 (17:23 +0100)]
Policy Executor: handle errors

- configurable default answer
- apply default answer upon non 2xx response
- delayed default webclient read timeout
- add custom timeout method with original read timeout in seconds
- apply default answer upon timeout
- add integration test with short timeout error scenario

Issue-ID: CPS-2412
Change-Id: I62527a27e426c2f01fda2182ebd2513242c29ac1
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
6 months agoRemove Hazelcast cache for prefix resolver (CPS-2417 #2) 85/138985/3
danielhanrahan [Tue, 17 Sep 2024 16:48:22 +0000 (17:48 +0100)]
Remove Hazelcast cache for prefix resolver (CPS-2417 #2)

This patch removes the AnchorDataCache from CPS, which is used for
prefix resolution in get/query operations.

As such, Hazelcast is no longer a dependency of CPS, only NCMP.

- Changed PrefixResolver to be more efficient.
- Removed AnchorDataCache and associated classes.
- Moved HazelcastCacheConfig to NCMP.
- Removed Hazelcast dependency from cps-service/pom.xml

This shows good performance improvements in some APIs such as v2 GET
which is nearly 2x faster (also 5x faster including base patch).

Issue-ID: CPS-2417
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I24768469f24e90b70f7a6187faa4f5b3d75777d2

6 months agoFix intermittent error in integration test 09/139009/1
ToineSiebelink [Tue, 24 Sep 2024 17:16:31 +0000 (18:16 +0100)]
Fix intermittent error in integration test

- make collection verification order independent

Issue-ID: CPS-475
Change-Id: Ib070fd5360463c93ba6193a84a1dfcd11cd802fa
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
6 months agoMerge "Reduce anchor lookups related to PrefixResolver (CPS-2417 #1)"
Toine Siebelink [Tue, 24 Sep 2024 16:23:40 +0000 (16:23 +0000)]
Merge "Reduce anchor lookups related to PrefixResolver (CPS-2417 #1)"

6 months agoMerge "Revert "Correctly report trust level if DMI is down""
Daniel Hanrahan [Tue, 24 Sep 2024 16:01:48 +0000 (16:01 +0000)]
Merge "Revert "Correctly report trust level if DMI is down""

6 months ago[k6] CM handles searches using different filters 96/138996/2
danielhanrahan [Sat, 21 Sep 2024 15:47:43 +0000 (16:47 +0100)]
[k6] CM handles searches using different filters

Each 5 VUs for searches uses a different filter:
no filter, module, property, cps path, and trust level.
This is needed since each search may have different
performance. This also helps reduce load versus doing
a combined module and property search all the time.

Issue-ID: CPS-2349
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I33a75260f97697d2061d10b80ba7eeb2f6c346f5

6 months agoReduce anchor lookups related to PrefixResolver (CPS-2417 #1) 84/138984/2
danielhanrahan [Tue, 17 Sep 2024 19:50:29 +0000 (20:50 +0100)]
Reduce anchor lookups related to PrefixResolver (CPS-2417 #1)

Instead of looking up same Anchor many times inside a for-loop,
do it once outside the loop.

This greatly improves performance in some cases, such as v2 GET API:
- /cps/api/v2/dataspaces/{dataspace}/anchors/{anchor}/node
Testing shows 3x faster response time.

Issue-ID: CPS-2417
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I80d97d8cc24372eed70626ed840cad985cbe0a4b

6 months agoRevert "Correctly report trust level if DMI is down" 99/138999/4
danielhanrahan [Mon, 23 Sep 2024 11:36:17 +0000 (12:36 +0100)]
Revert "Correctly report trust level if DMI is down"

This reverts commit https://gerrit.onap.org/r/c/cps/+/138853
Reason for revert: Commit caused performance issues in CM handle search

Manual rebase was required to integrate changes for alternate-id
support made since original commit was merged.

Issue-ID: CPS-2375
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I5ef743ea053589fa17c10587317a3534c2714c4b

6 months agoMerge "Make nginx load balance using least-connections"
Toine Siebelink [Tue, 24 Sep 2024 07:54:18 +0000 (07:54 +0000)]
Merge "Make nginx load balance using least-connections"

6 months agoRetry mechanism (with back off algorithm) is removed with more frequent watchdog... 20/138920/8
sourabh_sourabh [Wed, 11 Sep 2024 15:51:01 +0000 (16:51 +0100)]
Retry mechanism (with back off algorithm) is removed with more frequent watchdog poll

- Increased watchdog frequency for locked cm handle.
- Removed retry backoff algorithm for locked cm handle.

Issue-ID: CPS-2395
Change-Id: I54d0ec8f9de53a7d181639c14aaaa93736f03e19
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
6 months agoMake nginx load balance using least-connections 83/138983/1
danielhanrahan [Wed, 18 Sep 2024 13:35:39 +0000 (14:35 +0100)]
Make nginx load balance using least-connections

By default, nginx uses round robin for load balancing. Round robin
works well when all requests take same amount of time, but is not
optimal for dynamic loads.

With least-connected load balancing, nginx will try not to overload
a busy server with excessive requests, sending requests to a less
busy server instead.

Issue-ID: CPS-2415
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I0e7d6b36a79aa6541f2f5f667746f2792d55e54b

6 months agoMerge "[Cps Path Parser] Fixes for parent path & normalization"
Lee Anjella Macabuhay [Thu, 19 Sep 2024 10:52:52 +0000 (10:52 +0000)]
Merge "[Cps Path Parser] Fixes for parent path & normalization"

6 months agoMerge "Change execution order of the legacy async batch read operation in k6 suite"
Daniel Hanrahan [Thu, 19 Sep 2024 10:44:51 +0000 (10:44 +0000)]
Merge "Change execution order of the legacy async batch read operation in k6 suite"

6 months agoChange execution order of the legacy async batch read operation in k6 suite 55/138955/4
halil.cakal [Wed, 18 Sep 2024 11:48:27 +0000 (12:48 +0100)]
Change execution order of the legacy async batch read operation in k6 suite

- run the test after the parallel cases
- add 30 seconds safe-period of time waiting for completion of
  active threads (remained from previous test cases)

Issue-ID: CPS-2414

Change-Id: I1260b37cd4f1974eebcccf867b36172b407a9646
Signed-off-by: halil.cakal <halil.cakal@est.tech>
6 months ago[1/2] Move Swagger API to docs folder 56/138956/2
egernug [Wed, 18 Sep 2024 12:32:19 +0000 (13:32 +0100)]
[1/2] Move Swagger API to docs folder

Code generation moved to docs folder for cps-core

Issue-ID: CPS-2332

Change-Id: I9a8c5007a3039e0f46114c589de0756d6de91127
Signed-off-by: egernug <gerard.nugent@est.tech>
6 months ago[Cps Path Parser] Fixes for parent path & normalization 03/138903/4
danielhanrahan [Wed, 4 Sep 2024 17:15:17 +0000 (18:15 +0100)]
[Cps Path Parser] Fixes for parent path & normalization

This commit fixes issues with Cps Path Parser related to
path normalization and parent path generation when using
descendant paths and ancestor axis.

Issue-ID: CPS-2365
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I728fc379b134bd62c39a7085650930450c8a8597

6 months agoMerge "Refactored cps-ri package structure"
Sourabh Sourabh [Wed, 18 Sep 2024 13:23:15 +0000 (13:23 +0000)]
Merge "Refactored cps-ri package structure"

6 months agoRefactored cps-ri package structure 52/138952/1
leventecsanyi [Tue, 17 Sep 2024 10:07:32 +0000 (12:07 +0200)]
Refactored cps-ri package structure

  - fixed import order and moved package structure

Issue-ID: CPS-2293
Change-Id: Ie2f9f057f261577054530feee7480850ba4b41e1
Signed-off-by: leventecsanyi <levente.csanyi@est.tech>
6 months agoMerge "Policy Executor API: Add Prefix in Yaml"
Sourabh Sourabh [Tue, 17 Sep 2024 09:16:08 +0000 (09:16 +0000)]
Merge "Policy Executor API: Add Prefix in Yaml"

6 months agoPolicy Executor API: Add Prefix in Yaml 21/138921/6
ToineSiebelink [Mon, 16 Sep 2024 14:42:30 +0000 (15:42 +0100)]
Policy Executor API: Add Prefix in Yaml

- Remove servers from yaml.
- Add prefix to path in yaml.
- Remove basepath prefix from Controller Impl.
- Added policy server port to 8093.
- Exposed docker env. vars for policy services.

Issue-ID: CPS-2291
Change-Id: Idcc9e23f9d63bad2480537dc8d39523fe82b4b83
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
6 months agoMerge "Fix violation reported by sonarqube"
Lee Anjella Macabuhay [Mon, 16 Sep 2024 13:40:00 +0000 (13:40 +0000)]
Merge "Fix violation reported by sonarqube"

6 months agoFix violation reported by sonarqube 22/138922/1
halil.cakal [Fri, 13 Sep 2024 12:52:57 +0000 (13:52 +0100)]
Fix violation reported by sonarqube

Issue-ID: CPS-89

Change-Id: Ia6e3bb26ffb433f3610f53f408fad12489f2e3b3
Signed-off-by: halil.cakal <halil.cakal@est.tech>
6 months agoAdd CPS limit to the plots 92/138892/4
halil.cakal [Tue, 3 Sep 2024 14:38:01 +0000 (15:38 +0100)]
Add CPS limit to the plots

- as of now the cps limits given manually according to
  the latest result
- there will be another commit will change the plot
  job in cicd repo to reflect the values on the plots

Issue-ID: CPS-2350

Change-Id: Ic371f46894f1825dca1d26ec65ae0f62aa06c327
Signed-off-by: halil.cakal <halil.cakal@est.tech>
6 months agoCalculate throughput for legacy batch read data operations 60/138860/8
halil.cakal [Wed, 28 Aug 2024 09:19:26 +0000 (10:19 +0100)]
Calculate throughput for legacy batch read data operations

- stress the cps-and-ncmp module for batch read data operation
- for this aim use shared-ireations during async http request
- as parallel to this, consume expected number of messages
  as fast as possible

Issue-ID: CPS-2268

Change-Id: I1b6724479dac7391cbb6407fda52d15176aa8985
Signed-off-by: halil.cakal <halil.cakal@est.tech>
6 months agoMerge "Renaming Delta Operations as per RFC 9144"
Priyank Maheshwari [Wed, 11 Sep 2024 09:14:14 +0000 (09:14 +0000)]
Merge "Renaming Delta Operations as per RFC 9144"

6 months ago[k6] Fix incorrect results when errors occur in k6 tests (case 2) 12/138912/3
egernug [Fri, 6 Sep 2024 11:09:31 +0000 (12:09 +0100)]
[k6] Fix incorrect results when errors occur in k6 tests (case 2)

De-registration is reported as 200,000 handles per second if all operations fail
- Added a running total for successful de-registrations of CM Handles

Issue-ID: CPS-2346

Change-Id: Ib4302d3b605ff1eaf4ecadac11b4e851a69210df
Signed-off-by: egernug <gerard.nugent@est.tech>
6 months agoMerge "Fix high-cardinality metrics in k6, causing high memory use"
Toine Siebelink [Tue, 10 Sep 2024 13:34:35 +0000 (13:34 +0000)]
Merge "Fix high-cardinality metrics in k6, causing high memory use"

6 months agoMerge "Uplift liquibase-core to 4.29.0"
Toine Siebelink [Tue, 10 Sep 2024 12:08:18 +0000 (12:08 +0000)]
Merge "Uplift liquibase-core to 4.29.0"

6 months agoMerge "NCMP should not use CPS SPI classes"
Toine Siebelink [Tue, 10 Sep 2024 10:34:28 +0000 (10:34 +0000)]
Merge "NCMP should not use CPS SPI classes"

6 months agoRenaming Delta Operations as per RFC 9144 87/138687/7
Arpit Singh [Wed, 4 Sep 2024 05:32:06 +0000 (11:02 +0530)]
Renaming Delta Operations as per RFC 9144

RFC 9144 recommends naming convention for operations that can be performed on any datastore.
The recommended names are:
- create
- remove
- replace
Whereas in CPS Delta feature these operaitions were named:
- add
- remove
- update
So, "add" and "update" operations have been changed to "create" and "replace" respectively.

Issue-ID: CPS-2393
Change-Id: Ie39f5f8ccc91e96cf484ca7fbc833a6be8758054
Signed-off-by: Arpit Singh <as00745003@techmahindra.com>
6 months agoFix high-cardinality metrics in k6, causing high memory use 79/138879/7
sourabh_sourabh [Fri, 30 Aug 2024 15:25:04 +0000 (16:25 +0100)]
Fix high-cardinality metrics in k6, causing high memory use

- Added unique IDs as metric tags for all the endpoints.
- Re-arranged order of public and prive js methods.

Issue-ID: CPS-2331
Change-Id: Ib876a647fb35110c50670c7222986e8a8a6f5ca0
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
6 months agoUplift liquibase-core to 4.29.0 17/138917/1
mpriyank [Mon, 9 Sep 2024 14:12:56 +0000 (15:12 +0100)]
Uplift liquibase-core to 4.29.0

- uplift liquibase-core to 4.29.0 to resolve the internal licensing
  issue

Issue-ID: CPS-2392
Change-Id: Ia06bbb095a480c910d0df3d23ac719a746508856
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
6 months agoRefactor FragmentQueryBuilder 86/138886/5
danielhanrahan [Mon, 2 Sep 2024 11:04:51 +0000 (12:04 +0100)]
Refactor FragmentQueryBuilder

This is a refactoring of FragmentQueryBuilder, there are
no functional changes. This refactoring is needed before
main patch of CPS-2365.

Issue-ID: CPS-2365
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I648616159719e084ba0e7cb4b09649cd17560cab

6 months agoNCMP should not use CPS SPI classes 09/138909/1
danielhanrahan [Thu, 5 Sep 2024 15:38:22 +0000 (16:38 +0100)]
NCMP should not use CPS SPI classes

CpsDataPersistenceService is an internal interface of CPS, part of
the SPI, intended only to be used by CPS. Instead, NCMP should use
the public API of CPS, i.e. CpsDataService and CpsQueryService.

This is an architectural issue, but it will affect upcoming changes,
where changes to the SPI implementation should not affect public API.

Issue-ID: CPS-2398
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I89339e1a3566a25e33168c24c8b8401ca52224e9

6 months ago[k6] Minor correction to plot titles 05/138905/1
danielhanrahan [Thu, 5 Sep 2024 11:06:30 +0000 (12:06 +0100)]
[k6] Minor correction to plot titles

Plot title say Module filter instead of Module and Property.

Issue-ID: CPS-2349
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ie23080a6085960b89132ad38d07d0c2b6f12471b

6 months agoMerge "[Cps Path Parser] Move NCMP-specific logic to NCMP"
Priyank Maheshwari [Thu, 5 Sep 2024 09:49:38 +0000 (09:49 +0000)]
Merge "[Cps Path Parser] Move NCMP-specific logic to NCMP"

6 months agoMerge "[k6] Align tests with updated requirements"
Priyank Maheshwari [Thu, 5 Sep 2024 09:45:25 +0000 (09:45 +0000)]
Merge "[k6] Align tests with updated requirements"

6 months ago[Cps Path Parser] Move NCMP-specific logic to NCMP 84/138884/3
danielhanrahan [Thu, 29 Aug 2024 19:00:40 +0000 (20:00 +0100)]
[Cps Path Parser] Move NCMP-specific logic to NCMP

Some special case code to disable ancestor-axis was added for
CM-handle search (see CPS-2308). It is now relocated to NCMP.
This makes other needed improvements of Cps Path Parser easier.

- Move special case code into NCMP
- Add integration test to ensure CM-handle search works

Issue-ID: CPS-2365
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I168d6156be559166f115aa42e21cd987d98b7d41

6 months agoMerge "Support alternate id for CPS-E05 GetCmHandleDetailsById"
Daniel Hanrahan [Wed, 4 Sep 2024 16:19:02 +0000 (16:19 +0000)]
Merge "Support alternate id for CPS-E05 GetCmHandleDetailsById"

6 months agoMerge "[Cps Path Parser] Refactoring leaf conditions"
Priyank Maheshwari [Wed, 4 Sep 2024 16:18:43 +0000 (16:18 +0000)]
Merge "[Cps Path Parser] Refactoring leaf conditions"

6 months agoMerge "K6 Performance Graph fails frequently"
Daniel Hanrahan [Wed, 4 Sep 2024 15:56:15 +0000 (15:56 +0000)]
Merge "K6 Performance Graph fails frequently"

6 months agoK6 Performance Graph fails frequently 97/138897/3
sourabh_sourabh [Wed, 4 Sep 2024 13:02:17 +0000 (14:02 +0100)]
K6 Performance Graph fails frequently

- Overrides initial processing delay of every new module set tag for DMI plugin stub from 2 min (default) to 0 as a temp. solution.

Issue-ID: CPS-2395
Change-Id: Ic875262314406ec55457e31a69c00d55293c1b74
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
6 months ago[Cps Path Parser] Remove unneeded validation in grammar 83/138883/2
danielhanrahan [Sat, 31 Aug 2024 21:06:12 +0000 (22:06 +0100)]
[Cps Path Parser] Remove unneeded validation in grammar

The invalid prefix/postfix is not needed; existing tests show
that PathParsingException is thrown in for invalid paths.

Issue-ID: CPS-2365
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I202f0ca0bc5eb768fc3af711180405882dafc22d

6 months ago[k6] Align tests with updated requirements 58/138858/7
danielhanrahan [Sun, 25 Aug 2024 20:35:19 +0000 (21:35 +0100)]
[k6] Align tests with updated requirements

- Test load increased to 5 CM handle searches and 5 Id searches to
  reflect current requirements.
- CM handles searches now use a combined search filter including
  1 module and 1 public property, so we are testing with a heavier
  load than required, for early warning of trouble.
- Test load reduced for passthrough operations to 4 VUs per operation
  to reflect addition of batch operation.
- Added test of passthrough write operation with alternate ID.
- Additional refactoring.

Issue-ID: CPS-2349
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I88d2fe431f74821e8e0e976441efcf82aa320849

6 months agoMerge "Support Alternate-id for CPS-E05 module search"
Daniel Hanrahan [Wed, 4 Sep 2024 10:05:25 +0000 (10:05 +0000)]
Merge "Support Alternate-id for CPS-E05 module search"

7 months agoMerge "Remove event notifications when removing trust level entries"
Priyank Maheshwari [Wed, 4 Sep 2024 09:33:48 +0000 (09:33 +0000)]
Merge "Remove event notifications when removing trust level entries"

7 months agoSupport Alternate-id for CPS-E05 module search 44/138844/7
seanbeirne [Mon, 26 Aug 2024 09:48:20 +0000 (10:48 +0100)]
Support Alternate-id for CPS-E05 module search

Issue-ID: CPS-2370
Change-Id: I0ca3a1df232ad9fd9f76695cd6ea21d36919130d
Signed-off-by: seanbeirne <sean.beirne@est.tech>