cps.git
5 months agoAdded additional logging 00/139200/5
sourabh_sourabh [Tue, 15 Oct 2024 11:11:16 +0000 (12:11 +0100)]
Added additional logging

- Added logging for non existing xpath while updating data notes and its
  decendants.

Issue-ID: CPS-2403
Change-Id: I1ecaf2aed77aec8d266f6a758fe80ee96717d9c7
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
5 months agoMerge "Copy Swagger file to docs folder"
Toine Siebelink [Mon, 14 Oct 2024 12:13:35 +0000 (12:13 +0000)]
Merge "Copy Swagger file to docs folder"

5 months agoMerge "use one hazelcast instance per jvm"
Lee Anjella Macabuhay [Thu, 10 Oct 2024 12:51:40 +0000 (12:51 +0000)]
Merge "use one hazelcast instance per jvm"

5 months agouse one hazelcast instance per jvm 30/139130/5
mpriyank [Tue, 10 Sep 2024 15:12:28 +0000 (16:12 +0100)]
use one hazelcast instance per jvm

- updated all the cache(distributed datastructures) to use the same instance config now.
- there will be just one instance per JVM now
- Better cleanup of hz instance in the testware
- Refactored the testware to verify the configs
- Expected Impact on lowering the memory usage, less number of TCP
  communications between members
- NOTE: we need to do a full regression as the changes impacts all the
  use cases which involves cache

Issue-ID: CPS-2408
Change-Id: I7564992a9990f44ef3defb4f50cb7d094cad7b92
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
5 months agoCopy Swagger file to docs folder 82/139182/1
egernug [Thu, 10 Oct 2024 09:11:27 +0000 (10:11 +0100)]
Copy Swagger file to docs folder

To avoid manual copy/paste errors updates to docs folder for RTD are generated and copied when compiling CPS

Issue-ID: CPS-2332

Change-Id: I1a64f7cc74a82341403f8125adc5216ef046429c
Signed-off-by: egernug <gerard.nugent@est.tech>
5 months agoMerge "Autodiscover cps-and-ncmp containers when load balancing"
Daniel Hanrahan [Thu, 10 Oct 2024 09:53:47 +0000 (09:53 +0000)]
Merge "Autodiscover cps-and-ncmp containers when load balancing"

5 months agoMerge "Added architecture tests to validate ONLY NCMP REST/Service dependencies"
Sourabh Sourabh [Wed, 9 Oct 2024 15:09:11 +0000 (15:09 +0000)]
Merge "Added architecture tests to validate ONLY NCMP REST/Service dependencies"

5 months agoAdded architecture tests to validate ONLY NCMP REST/Service dependencies 03/139003/17
leventecsanyi [Mon, 23 Sep 2024 14:01:03 +0000 (16:01 +0200)]
Added architecture tests to validate ONLY NCMP REST/Service dependencies

  - removed old tests and added new test cases for the NCMP controller and service layers
  - other dependencies will be added in later commits

Issue-ID: CPS-2422
Change-Id: Iabc4b53e9414477cf3d0b2e1e5a03ba8b4995169
Signed-off-by: leventecsanyi <levente.csanyi@est.tech>
5 months agoMerge "Added K6 script test to check a throughput of CPS kafka consumer"
Daniel Hanrahan [Wed, 9 Oct 2024 11:22:46 +0000 (11:22 +0000)]
Merge "Added K6 script test to check a throughput of CPS kafka consumer"

5 months agoMerge "Implementation of Data validation feature in Create a Node API"
Toine Siebelink [Wed, 9 Oct 2024 09:37:42 +0000 (09:37 +0000)]
Merge "Implementation of Data validation feature in Create a Node API"

5 months agoAutodiscover cps-and-ncmp containers when load balancing 72/139172/4
halil.cakal [Mon, 7 Oct 2024 15:48:05 +0000 (16:48 +0100)]
Autodiscover cps-and-ncmp containers when load balancing

- remove hard-coded container names from load balancer(nginx) and
  prometheus

Issue-ID: CPS-2437

Change-Id: Ia64b949c2eb3e4d67e123b046e66d5bfdda2c801
Signed-off-by: halil.cakal <halil.cakal@est.tech>
5 months agoFine tune JVM parameters for cps-and-ncmp 97/138997/6
halil.cakal [Mon, 23 Sep 2024 10:05:16 +0000 (11:05 +0100)]
Fine tune JVM parameters for cps-and-ncmp

- limit the total memory with 2G
- set heap memory 75% of the total RAM

Issue-ID: CPS-2180

Change-Id: Ia8a6c2bd40866052caba4f332ac563e1ed829212
Signed-off-by: halil.cakal <halil.cakal@est.tech>
5 months agoImplementation of Data validation feature in Create a Node API 76/138676/28
Arpit Singh [Fri, 9 Aug 2024 06:53:49 +0000 (12:23 +0530)]
Implementation of Data validation feature in Create a Node API

Added support to validate JSON/XML data without the need of persisting
it in the databse.
 - added "dryRunInQuery" flag as a new query parameter
 - added new method as part of CpsDataService layer to perform data
   validation
 - added new method in yang parser "validateData" to validate
   data without persisting it

Issue-ID: CPS-2361
Change-Id: I43dd33cc6120576d0fac606d5c4b0168d107311d
Signed-off-by: Arpit Singh <as00745003@techmahindra.com>
5 months agoMerge "Policy Executor: handle errors, part 2 (fighting between IntelliJ and Checksty...
Sourabh Sourabh [Tue, 8 Oct 2024 15:51:01 +0000 (15:51 +0000)]
Merge "Policy Executor: handle errors, part 2 (fighting between IntelliJ and Checkstyle best practices)"

5 months agoMerge "Re-adjust cps expectations"
Lee Anjella Macabuhay [Mon, 7 Oct 2024 14:08:34 +0000 (14:08 +0000)]
Merge "Re-adjust cps expectations"

5 months agoMerge "RTD update for delta API 2"
Priyank Maheshwari [Mon, 7 Oct 2024 08:42:08 +0000 (08:42 +0000)]
Merge "RTD update for delta API 2"

6 months agoBump CPS to 3.5.4-SNAPSHOT 22/139122/1
mpriyank [Fri, 4 Oct 2024 15:59:40 +0000 (16:59 +0100)]
Bump CPS to 3.5.4-SNAPSHOT

- bump cps to 3.5.4-SNAPSHOT
- updated release notes

Issue-ID: CPS-2434
Change-Id: I3d5e6f37711bb29b61b598cc5af57ecfdc35cbba
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
6 months ago[CPS] Step-3: OSLO release changes 21/139121/1
sourabh_sourabh [Fri, 4 Oct 2024 15:23:06 +0000 (16:23 +0100)]
[CPS] Step-3: OSLO release changes

- cps maven stage docker yaml is created for release 3.5.3

Issue-ID:CPS-2434

Change-Id: I4048b9f54b9c0b0eb433fef9d9c403d54c39bc35
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
6 months ago[CPS] Step-2: OSLO release changes 20/139120/2
sourabh_sourabh [Fri, 4 Oct 2024 15:09:08 +0000 (16:09 +0100)]
[CPS] Step-2: OSLO release changes

- cps maven stage master yaml is created for release 3.5.3

Issue-ID:CPS-2434
Change-Id: I5353eb64252b3e071b5203d931f0f646ef290c13
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
6 months ago[CPS] OSLO release changes 18/139118/3
sourabh_sourabh [Fri, 4 Oct 2024 13:35:32 +0000 (14:35 +0100)]
[CPS] OSLO release changes

- Updated open api for cps, ncmp and policy apis.
- Updated release notes with bus and features.

Issue-ID: CPS-2434
Change-Id: Ib4d80f3939b6e267f228177c817e43832a9384db
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
6 months agoMerge "[BUG] Fix memory leak related to using arrays in Hibernate"
Lee Anjella Macabuhay [Fri, 4 Oct 2024 12:11:17 +0000 (12:11 +0000)]
Merge "[BUG] Fix memory leak related to using arrays in Hibernate"

6 months agoRevert "[1/2] Move Swagger API to docs folder" 35/139135/2
Lee Anjella Macabuhay [Thu, 3 Oct 2024 13:47:14 +0000 (13:47 +0000)]
Revert "[1/2] Move Swagger API to docs folder"

This reverts commit b76392e2d1629d4eb67b10c450cdd954ef678966.

Reason for revert: Release blocker

Change-Id: Ic477dbaaad34992c8f440981d92f12b977cdf96e
Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech>
6 months agoRTD update for delta API 2 65/137565/25
Arpit Singh [Mon, 6 May 2024 14:42:52 +0000 (20:12 +0530)]
RTD update for delta API 2

 - Documentation update for Delta Feature and Delta API 1
 - Added documentation for API 2: Delta between anchor
   and JSON payload

Issue-ID: CPS-2212
Signed-off-by: Arpit Singh <as00745003@techmahindra.com>
Change-Id: Id74cd930ce48e5cb414aa62c5381b79675346a37

6 months ago[BUG] Fix memory leak related to using arrays in Hibernate 13/139113/5
danielhanrahan [Wed, 2 Oct 2024 20:31:04 +0000 (21:31 +0100)]
[BUG] Fix memory leak related to using arrays in Hibernate

The use of arrays like String[] instead of Collection<String> in
JpaRepository methods is causing a memory leak, most likely due to a
bug in the hypersistence-utils dependency which provides the feature.
Note code using arrays in JDBC (via jdbcTemplate) is not affected.

This patch removes most uses of arrays in Java, except one needed for
FragmentPrefetchRepository using JDBC setArray(), which is safe.

Issue-ID: CPS-2430
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I94f8c3d4c8c32ebe0978c08d3226a196a95bf3b9

6 months agoRevert "Fix inputSpec for openapi code gen" 34/139134/3
Lee Anjella Macabuhay [Thu, 3 Oct 2024 13:46:59 +0000 (13:46 +0000)]
Revert "Fix inputSpec for openapi code gen"

This reverts commit 4baa75b06dcfb9365a8ea5cde03d77d7f21fa01c.

Reason for revert: Release blocker

Change-Id: I6156461de64dcec41e3f84cbd0cf90f63938bea6
Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech>
6 months agoRevert "CI: Add test (silent) checkov scan as part of the verify process" 16/139116/1
Jessica Wagantall [Thu, 3 Oct 2024 16:08:35 +0000 (09:08 -0700)]
Revert "CI: Add test (silent) checkov scan as part of the verify process"

This reverts commit c8e2b3383830395bb4bc37371a9c6119a316cb53.

Issue-ID: CIMAN-33
Change-Id: I5120097ad05394e3667a868c4b7edd44ef1aa070
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
6 months agoPolicy Executor: handle errors, part 2 34/139034/10
ToineSiebelink [Tue, 1 Oct 2024 17:40:39 +0000 (18:40 +0100)]
Policy Executor: handle errors, part 2
(fighting between IntelliJ and Checkstyle best practices)

- non-2xx responses are processed using web client exceptions
- handle unknown host exception
- upgraded spotbugs (checkstyle and related mvn plugin)
- fixed some small spotbugs due to upgrade
- added commented instructions in docker compose to enable debugging
- added some environment variables for policy executor configuration
- extract out Sleeper in stub service to achieve 100% coverage
- added cause to Policy Executor exceptions where applicable
- ignored (new) spotbug rule about catch NPE because of issue in 3pp
- ignored (new) spotbug rule about \n in string due to multiline string block

Issue-ID: CPS-2412
Change-Id: I6835a73320c436cbeea12cc7a06f15899eec7bf1
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
6 months agoAdded K6 script test to check a throughput of CPS kafka consumer 40/139040/4
sourabh_sourabh [Mon, 30 Sep 2024 10:13:30 +0000 (11:13 +0100)]
Added K6 script test to check a throughput of CPS kafka consumer

- Added new K6 script to produce cm avc enents. (It is not a part
  regular automated tests)
- Added a json resource (avc event)

Issue-ID: CPS-2329
Change-Id: I6446bc120382257c5039df17fd34b84b7c6f6550
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
6 months agoRe-adjust cps expectations 38/139038/3
halil.cakal [Fri, 27 Sep 2024 13:43:00 +0000 (14:43 +0100)]
Re-adjust cps expectations

- aligning cps expectation for new plots

Issue-ID: CPS-2350

Change-Id: I5b300c4f5432bf2bdfe8a8b0d52b2ce60bd338d6
Signed-off-by: halil.cakal <halil.cakal@est.tech>
6 months agoUpdate JIRA and Wiki links host name from wiki.onap.org to lf-onap.atlassian.net 95/139095/10
sourabh_sourabh [Wed, 2 Oct 2024 15:37:47 +0000 (16:37 +0100)]
Update JIRA and Wiki links host name from wiki.onap.org to lf-onap.atlassian.net

Issue-ID: CPS-2432
Change-Id: I53f380a4fe0968d1ec9f1393651c7c43f03d0d95
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
6 months agoMerge "BugFix: 500 Server error when upgrade and service restart with data in DB"
Sourabh Sourabh [Mon, 30 Sep 2024 11:10:19 +0000 (11:10 +0000)]
Merge "BugFix: 500 Server error when upgrade and service restart with data in DB"

6 months agoFix inputSpec for openapi code gen 37/139037/1
egernug [Fri, 27 Sep 2024 12:33:24 +0000 (13:33 +0100)]
Fix inputSpec for openapi code gen

Existing inputSpec only accessible from cps root. Changing path to be accessible elsewhere

Issue-ID: CPS-2332

Change-Id: I94ada93ea3c88d2fdca84d49c21e1ffc2923f4b2
Signed-off-by: egernug <gerard.nugent@est.tech>
6 months agoMerge "[1/2] Move Swagger API to docs folder"
Toine Siebelink [Thu, 26 Sep 2024 08:59:12 +0000 (08:59 +0000)]
Merge "[1/2] Move Swagger API to docs folder"

6 months agoCI: Add test (silent) checkov scan as part of the verify process 33/139033/1
Jessica Wagantall [Wed, 25 Sep 2024 20:17:55 +0000 (13:17 -0700)]
CI: Add test (silent) checkov scan as part of the verify process

Issue-ID: CIMAN-33
Change-Id: I70b171824acb913f67fd28c119bfbcbc1cc1e470
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
6 months agoBugFix: 500 Server error when upgrade and service restart with data in DB 32/139032/1
halil.cakal [Wed, 25 Sep 2024 10:07:01 +0000 (11:07 +0100)]
BugFix: 500 Server error when upgrade and service restart with data in DB

- return NONE for get effective trust level api if the trustlevel caches
  empty (restart case)

Issue-ID: CPS-2409

Change-Id: I72a755ca6ba5d4a78f5458a235c0d1f43aaa8d53
Signed-off-by: halil.cakal <halil.cakal@est.tech>
6 months agoMerge "[k6] CM handles searches using different filters"
Priyank Maheshwari [Wed, 25 Sep 2024 09:26:32 +0000 (09:26 +0000)]
Merge "[k6] CM handles searches using different filters"

6 months agoPolicy Executor: handle errors 82/138982/6
ToineSiebelink [Thu, 19 Sep 2024 16:23:58 +0000 (17:23 +0100)]
Policy Executor: handle errors

- configurable default answer
- apply default answer upon non 2xx response
- delayed default webclient read timeout
- add custom timeout method with original read timeout in seconds
- apply default answer upon timeout
- add integration test with short timeout error scenario

Issue-ID: CPS-2412
Change-Id: I62527a27e426c2f01fda2182ebd2513242c29ac1
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
6 months agoRemove Hazelcast cache for prefix resolver (CPS-2417 #2) 85/138985/3
danielhanrahan [Tue, 17 Sep 2024 16:48:22 +0000 (17:48 +0100)]
Remove Hazelcast cache for prefix resolver (CPS-2417 #2)

This patch removes the AnchorDataCache from CPS, which is used for
prefix resolution in get/query operations.

As such, Hazelcast is no longer a dependency of CPS, only NCMP.

- Changed PrefixResolver to be more efficient.
- Removed AnchorDataCache and associated classes.
- Moved HazelcastCacheConfig to NCMP.
- Removed Hazelcast dependency from cps-service/pom.xml

This shows good performance improvements in some APIs such as v2 GET
which is nearly 2x faster (also 5x faster including base patch).

Issue-ID: CPS-2417
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I24768469f24e90b70f7a6187faa4f5b3d75777d2

6 months agoFix intermittent error in integration test 09/139009/1
ToineSiebelink [Tue, 24 Sep 2024 17:16:31 +0000 (18:16 +0100)]
Fix intermittent error in integration test

- make collection verification order independent

Issue-ID: CPS-475
Change-Id: Ib070fd5360463c93ba6193a84a1dfcd11cd802fa
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
6 months agoMerge "Reduce anchor lookups related to PrefixResolver (CPS-2417 #1)"
Toine Siebelink [Tue, 24 Sep 2024 16:23:40 +0000 (16:23 +0000)]
Merge "Reduce anchor lookups related to PrefixResolver (CPS-2417 #1)"

6 months agoMerge "Revert "Correctly report trust level if DMI is down""
Daniel Hanrahan [Tue, 24 Sep 2024 16:01:48 +0000 (16:01 +0000)]
Merge "Revert "Correctly report trust level if DMI is down""

6 months ago[k6] CM handles searches using different filters 96/138996/2
danielhanrahan [Sat, 21 Sep 2024 15:47:43 +0000 (16:47 +0100)]
[k6] CM handles searches using different filters

Each 5 VUs for searches uses a different filter:
no filter, module, property, cps path, and trust level.
This is needed since each search may have different
performance. This also helps reduce load versus doing
a combined module and property search all the time.

Issue-ID: CPS-2349
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I33a75260f97697d2061d10b80ba7eeb2f6c346f5

6 months agoReduce anchor lookups related to PrefixResolver (CPS-2417 #1) 84/138984/2
danielhanrahan [Tue, 17 Sep 2024 19:50:29 +0000 (20:50 +0100)]
Reduce anchor lookups related to PrefixResolver (CPS-2417 #1)

Instead of looking up same Anchor many times inside a for-loop,
do it once outside the loop.

This greatly improves performance in some cases, such as v2 GET API:
- /cps/api/v2/dataspaces/{dataspace}/anchors/{anchor}/node
Testing shows 3x faster response time.

Issue-ID: CPS-2417
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I80d97d8cc24372eed70626ed840cad985cbe0a4b

6 months agoRevert "Correctly report trust level if DMI is down" 99/138999/4
danielhanrahan [Mon, 23 Sep 2024 11:36:17 +0000 (12:36 +0100)]
Revert "Correctly report trust level if DMI is down"

This reverts commit https://gerrit.onap.org/r/c/cps/+/138853
Reason for revert: Commit caused performance issues in CM handle search

Manual rebase was required to integrate changes for alternate-id
support made since original commit was merged.

Issue-ID: CPS-2375
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I5ef743ea053589fa17c10587317a3534c2714c4b

6 months agoMerge "Make nginx load balance using least-connections"
Toine Siebelink [Tue, 24 Sep 2024 07:54:18 +0000 (07:54 +0000)]
Merge "Make nginx load balance using least-connections"

6 months agoRetry mechanism (with back off algorithm) is removed with more frequent watchdog... 20/138920/8
sourabh_sourabh [Wed, 11 Sep 2024 15:51:01 +0000 (16:51 +0100)]
Retry mechanism (with back off algorithm) is removed with more frequent watchdog poll

- Increased watchdog frequency for locked cm handle.
- Removed retry backoff algorithm for locked cm handle.

Issue-ID: CPS-2395
Change-Id: I54d0ec8f9de53a7d181639c14aaaa93736f03e19
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
6 months agoMake nginx load balance using least-connections 83/138983/1
danielhanrahan [Wed, 18 Sep 2024 13:35:39 +0000 (14:35 +0100)]
Make nginx load balance using least-connections

By default, nginx uses round robin for load balancing. Round robin
works well when all requests take same amount of time, but is not
optimal for dynamic loads.

With least-connected load balancing, nginx will try not to overload
a busy server with excessive requests, sending requests to a less
busy server instead.

Issue-ID: CPS-2415
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I0e7d6b36a79aa6541f2f5f667746f2792d55e54b

6 months agoMerge "[Cps Path Parser] Fixes for parent path & normalization"
Lee Anjella Macabuhay [Thu, 19 Sep 2024 10:52:52 +0000 (10:52 +0000)]
Merge "[Cps Path Parser] Fixes for parent path & normalization"

6 months agoMerge "Change execution order of the legacy async batch read operation in k6 suite"
Daniel Hanrahan [Thu, 19 Sep 2024 10:44:51 +0000 (10:44 +0000)]
Merge "Change execution order of the legacy async batch read operation in k6 suite"

6 months agoChange execution order of the legacy async batch read operation in k6 suite 55/138955/4
halil.cakal [Wed, 18 Sep 2024 11:48:27 +0000 (12:48 +0100)]
Change execution order of the legacy async batch read operation in k6 suite

- run the test after the parallel cases
- add 30 seconds safe-period of time waiting for completion of
  active threads (remained from previous test cases)

Issue-ID: CPS-2414

Change-Id: I1260b37cd4f1974eebcccf867b36172b407a9646
Signed-off-by: halil.cakal <halil.cakal@est.tech>
6 months ago[1/2] Move Swagger API to docs folder 56/138956/2
egernug [Wed, 18 Sep 2024 12:32:19 +0000 (13:32 +0100)]
[1/2] Move Swagger API to docs folder

Code generation moved to docs folder for cps-core

Issue-ID: CPS-2332

Change-Id: I9a8c5007a3039e0f46114c589de0756d6de91127
Signed-off-by: egernug <gerard.nugent@est.tech>
6 months ago[Cps Path Parser] Fixes for parent path & normalization 03/138903/4
danielhanrahan [Wed, 4 Sep 2024 17:15:17 +0000 (18:15 +0100)]
[Cps Path Parser] Fixes for parent path & normalization

This commit fixes issues with Cps Path Parser related to
path normalization and parent path generation when using
descendant paths and ancestor axis.

Issue-ID: CPS-2365
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I728fc379b134bd62c39a7085650930450c8a8597

6 months agoMerge "Refactored cps-ri package structure"
Sourabh Sourabh [Wed, 18 Sep 2024 13:23:15 +0000 (13:23 +0000)]
Merge "Refactored cps-ri package structure"

6 months agoRefactored cps-ri package structure 52/138952/1
leventecsanyi [Tue, 17 Sep 2024 10:07:32 +0000 (12:07 +0200)]
Refactored cps-ri package structure

  - fixed import order and moved package structure

Issue-ID: CPS-2293
Change-Id: Ie2f9f057f261577054530feee7480850ba4b41e1
Signed-off-by: leventecsanyi <levente.csanyi@est.tech>
6 months agoMerge "Policy Executor API: Add Prefix in Yaml"
Sourabh Sourabh [Tue, 17 Sep 2024 09:16:08 +0000 (09:16 +0000)]
Merge "Policy Executor API: Add Prefix in Yaml"

6 months agoPolicy Executor API: Add Prefix in Yaml 21/138921/6
ToineSiebelink [Mon, 16 Sep 2024 14:42:30 +0000 (15:42 +0100)]
Policy Executor API: Add Prefix in Yaml

- Remove servers from yaml.
- Add prefix to path in yaml.
- Remove basepath prefix from Controller Impl.
- Added policy server port to 8093.
- Exposed docker env. vars for policy services.

Issue-ID: CPS-2291
Change-Id: Idcc9e23f9d63bad2480537dc8d39523fe82b4b83
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
6 months agoMerge "Fix violation reported by sonarqube"
Lee Anjella Macabuhay [Mon, 16 Sep 2024 13:40:00 +0000 (13:40 +0000)]
Merge "Fix violation reported by sonarqube"

6 months agoFix violation reported by sonarqube 22/138922/1
halil.cakal [Fri, 13 Sep 2024 12:52:57 +0000 (13:52 +0100)]
Fix violation reported by sonarqube

Issue-ID: CPS-89

Change-Id: Ia6e3bb26ffb433f3610f53f408fad12489f2e3b3
Signed-off-by: halil.cakal <halil.cakal@est.tech>
6 months agoAdd CPS limit to the plots 92/138892/4
halil.cakal [Tue, 3 Sep 2024 14:38:01 +0000 (15:38 +0100)]
Add CPS limit to the plots

- as of now the cps limits given manually according to
  the latest result
- there will be another commit will change the plot
  job in cicd repo to reflect the values on the plots

Issue-ID: CPS-2350

Change-Id: Ic371f46894f1825dca1d26ec65ae0f62aa06c327
Signed-off-by: halil.cakal <halil.cakal@est.tech>
6 months agoCalculate throughput for legacy batch read data operations 60/138860/8
halil.cakal [Wed, 28 Aug 2024 09:19:26 +0000 (10:19 +0100)]
Calculate throughput for legacy batch read data operations

- stress the cps-and-ncmp module for batch read data operation
- for this aim use shared-ireations during async http request
- as parallel to this, consume expected number of messages
  as fast as possible

Issue-ID: CPS-2268

Change-Id: I1b6724479dac7391cbb6407fda52d15176aa8985
Signed-off-by: halil.cakal <halil.cakal@est.tech>
6 months agoMerge "Renaming Delta Operations as per RFC 9144"
Priyank Maheshwari [Wed, 11 Sep 2024 09:14:14 +0000 (09:14 +0000)]
Merge "Renaming Delta Operations as per RFC 9144"

6 months ago[k6] Fix incorrect results when errors occur in k6 tests (case 2) 12/138912/3
egernug [Fri, 6 Sep 2024 11:09:31 +0000 (12:09 +0100)]
[k6] Fix incorrect results when errors occur in k6 tests (case 2)

De-registration is reported as 200,000 handles per second if all operations fail
- Added a running total for successful de-registrations of CM Handles

Issue-ID: CPS-2346

Change-Id: Ib4302d3b605ff1eaf4ecadac11b4e851a69210df
Signed-off-by: egernug <gerard.nugent@est.tech>
6 months agoMerge "Fix high-cardinality metrics in k6, causing high memory use"
Toine Siebelink [Tue, 10 Sep 2024 13:34:35 +0000 (13:34 +0000)]
Merge "Fix high-cardinality metrics in k6, causing high memory use"

6 months agoMerge "Uplift liquibase-core to 4.29.0"
Toine Siebelink [Tue, 10 Sep 2024 12:08:18 +0000 (12:08 +0000)]
Merge "Uplift liquibase-core to 4.29.0"

6 months agoMerge "NCMP should not use CPS SPI classes"
Toine Siebelink [Tue, 10 Sep 2024 10:34:28 +0000 (10:34 +0000)]
Merge "NCMP should not use CPS SPI classes"

6 months agoRenaming Delta Operations as per RFC 9144 87/138687/7
Arpit Singh [Wed, 4 Sep 2024 05:32:06 +0000 (11:02 +0530)]
Renaming Delta Operations as per RFC 9144

RFC 9144 recommends naming convention for operations that can be performed on any datastore.
The recommended names are:
- create
- remove
- replace
Whereas in CPS Delta feature these operaitions were named:
- add
- remove
- update
So, "add" and "update" operations have been changed to "create" and "replace" respectively.

Issue-ID: CPS-2393
Change-Id: Ie39f5f8ccc91e96cf484ca7fbc833a6be8758054
Signed-off-by: Arpit Singh <as00745003@techmahindra.com>
6 months agoFix high-cardinality metrics in k6, causing high memory use 79/138879/7
sourabh_sourabh [Fri, 30 Aug 2024 15:25:04 +0000 (16:25 +0100)]
Fix high-cardinality metrics in k6, causing high memory use

- Added unique IDs as metric tags for all the endpoints.
- Re-arranged order of public and prive js methods.

Issue-ID: CPS-2331
Change-Id: Ib876a647fb35110c50670c7222986e8a8a6f5ca0
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
6 months agoUplift liquibase-core to 4.29.0 17/138917/1
mpriyank [Mon, 9 Sep 2024 14:12:56 +0000 (15:12 +0100)]
Uplift liquibase-core to 4.29.0

- uplift liquibase-core to 4.29.0 to resolve the internal licensing
  issue

Issue-ID: CPS-2392
Change-Id: Ia06bbb095a480c910d0df3d23ac719a746508856
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
6 months agoRefactor FragmentQueryBuilder 86/138886/5
danielhanrahan [Mon, 2 Sep 2024 11:04:51 +0000 (12:04 +0100)]
Refactor FragmentQueryBuilder

This is a refactoring of FragmentQueryBuilder, there are
no functional changes. This refactoring is needed before
main patch of CPS-2365.

Issue-ID: CPS-2365
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I648616159719e084ba0e7cb4b09649cd17560cab

6 months agoNCMP should not use CPS SPI classes 09/138909/1
danielhanrahan [Thu, 5 Sep 2024 15:38:22 +0000 (16:38 +0100)]
NCMP should not use CPS SPI classes

CpsDataPersistenceService is an internal interface of CPS, part of
the SPI, intended only to be used by CPS. Instead, NCMP should use
the public API of CPS, i.e. CpsDataService and CpsQueryService.

This is an architectural issue, but it will affect upcoming changes,
where changes to the SPI implementation should not affect public API.

Issue-ID: CPS-2398
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I89339e1a3566a25e33168c24c8b8401ca52224e9

6 months ago[k6] Minor correction to plot titles 05/138905/1
danielhanrahan [Thu, 5 Sep 2024 11:06:30 +0000 (12:06 +0100)]
[k6] Minor correction to plot titles

Plot title say Module filter instead of Module and Property.

Issue-ID: CPS-2349
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ie23080a6085960b89132ad38d07d0c2b6f12471b

6 months agoMerge "[Cps Path Parser] Move NCMP-specific logic to NCMP"
Priyank Maheshwari [Thu, 5 Sep 2024 09:49:38 +0000 (09:49 +0000)]
Merge "[Cps Path Parser] Move NCMP-specific logic to NCMP"

6 months agoMerge "[k6] Align tests with updated requirements"
Priyank Maheshwari [Thu, 5 Sep 2024 09:45:25 +0000 (09:45 +0000)]
Merge "[k6] Align tests with updated requirements"

6 months ago[Cps Path Parser] Move NCMP-specific logic to NCMP 84/138884/3
danielhanrahan [Thu, 29 Aug 2024 19:00:40 +0000 (20:00 +0100)]
[Cps Path Parser] Move NCMP-specific logic to NCMP

Some special case code to disable ancestor-axis was added for
CM-handle search (see CPS-2308). It is now relocated to NCMP.
This makes other needed improvements of Cps Path Parser easier.

- Move special case code into NCMP
- Add integration test to ensure CM-handle search works

Issue-ID: CPS-2365
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I168d6156be559166f115aa42e21cd987d98b7d41

6 months agoMerge "Support alternate id for CPS-E05 GetCmHandleDetailsById"
Daniel Hanrahan [Wed, 4 Sep 2024 16:19:02 +0000 (16:19 +0000)]
Merge "Support alternate id for CPS-E05 GetCmHandleDetailsById"

6 months agoMerge "[Cps Path Parser] Refactoring leaf conditions"
Priyank Maheshwari [Wed, 4 Sep 2024 16:18:43 +0000 (16:18 +0000)]
Merge "[Cps Path Parser] Refactoring leaf conditions"

6 months agoMerge "K6 Performance Graph fails frequently"
Daniel Hanrahan [Wed, 4 Sep 2024 15:56:15 +0000 (15:56 +0000)]
Merge "K6 Performance Graph fails frequently"

6 months agoK6 Performance Graph fails frequently 97/138897/3
sourabh_sourabh [Wed, 4 Sep 2024 13:02:17 +0000 (14:02 +0100)]
K6 Performance Graph fails frequently

- Overrides initial processing delay of every new module set tag for DMI plugin stub from 2 min (default) to 0 as a temp. solution.

Issue-ID: CPS-2395
Change-Id: Ic875262314406ec55457e31a69c00d55293c1b74
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
6 months ago[Cps Path Parser] Remove unneeded validation in grammar 83/138883/2
danielhanrahan [Sat, 31 Aug 2024 21:06:12 +0000 (22:06 +0100)]
[Cps Path Parser] Remove unneeded validation in grammar

The invalid prefix/postfix is not needed; existing tests show
that PathParsingException is thrown in for invalid paths.

Issue-ID: CPS-2365
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I202f0ca0bc5eb768fc3af711180405882dafc22d

6 months ago[k6] Align tests with updated requirements 58/138858/7
danielhanrahan [Sun, 25 Aug 2024 20:35:19 +0000 (21:35 +0100)]
[k6] Align tests with updated requirements

- Test load increased to 5 CM handle searches and 5 Id searches to
  reflect current requirements.
- CM handles searches now use a combined search filter including
  1 module and 1 public property, so we are testing with a heavier
  load than required, for early warning of trouble.
- Test load reduced for passthrough operations to 4 VUs per operation
  to reflect addition of batch operation.
- Added test of passthrough write operation with alternate ID.
- Additional refactoring.

Issue-ID: CPS-2349
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I88d2fe431f74821e8e0e976441efcf82aa320849

6 months agoMerge "Support Alternate-id for CPS-E05 module search"
Daniel Hanrahan [Wed, 4 Sep 2024 10:05:25 +0000 (10:05 +0000)]
Merge "Support Alternate-id for CPS-E05 module search"

7 months agoMerge "Remove event notifications when removing trust level entries"
Priyank Maheshwari [Wed, 4 Sep 2024 09:33:48 +0000 (09:33 +0000)]
Merge "Remove event notifications when removing trust level entries"

7 months agoSupport Alternate-id for CPS-E05 module search 44/138844/7
seanbeirne [Mon, 26 Aug 2024 09:48:20 +0000 (10:48 +0100)]
Support Alternate-id for CPS-E05 module search

Issue-ID: CPS-2370
Change-Id: I0ca3a1df232ad9fd9f76695cd6ea21d36919130d
Signed-off-by: seanbeirne <sean.beirne@est.tech>
7 months agoMerge "Support Alternate-id for CPS-E05 module definition"
Priyank Maheshwari [Wed, 4 Sep 2024 07:57:04 +0000 (07:57 +0000)]
Merge "Support Alternate-id for CPS-E05 module definition"

7 months agoSupport alternate id for CPS-E05 GetCmHandleDetailsById 93/138893/1
mpriyank [Tue, 3 Sep 2024 16:00:17 +0000 (17:00 +0100)]
Support alternate id for CPS-E05 GetCmHandleDetailsById

- added support for alternate id when retreiving cmhandle details by id
- Note : The performance for alternate id fetching will be taken care as
  part of a separate story.

Issue-ID: CPS-2385
Change-Id: I019b85d128e4b0a1f1d61623c92e1a2381c406c0
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
7 months agoMerge "Remove unused and duplicate dependencies in poms"
Sourabh Sourabh [Tue, 3 Sep 2024 09:43:14 +0000 (09:43 +0000)]
Merge "Remove unused and duplicate dependencies in poms"

7 months agoMerge "Fix huge logs produced when errors occur in k6"
Daniel Hanrahan [Tue, 3 Sep 2024 08:45:50 +0000 (08:45 +0000)]
Merge "Fix huge logs produced when errors occur in k6"

7 months agoMerge "[k6] Report failure rate for all tests in a single plot"
Priyank Maheshwari [Mon, 2 Sep 2024 16:10:16 +0000 (16:10 +0000)]
Merge "[k6] Report failure rate for all tests in a single plot"

7 months agoRemove event notifications when removing trust level entries 88/138888/3
halil.cakal [Mon, 2 Sep 2024 13:30:02 +0000 (14:30 +0100)]
Remove event notifications when removing trust level entries

Issue-ID: CPS-2315

Change-Id: I4f0cb7c2066e0ab9c62ac9cdb0fc39ce7de7791b
Signed-off-by: halil.cakal <halil.cakal@est.tech>
7 months agoSupport Alternate-id for CPS-E05 module definition 46/138846/7
seanbeirne [Mon, 26 Aug 2024 13:13:29 +0000 (14:13 +0100)]
Support Alternate-id for CPS-E05 module definition

Issue-ID: CPS-2379
Change-Id: Idd180c5792575522ceaaa094b94c8f5b36790186
Signed-off-by: seanbeirne <sean.beirne@est.tech>
7 months agoRemove unused and duplicate dependencies in poms 87/138887/1
danielhanrahan [Sun, 18 Aug 2024 00:28:39 +0000 (01:28 +0100)]
Remove unused and duplicate dependencies in poms

- remove duplicate dependencies from cps-service pom
- remove unused spring security dependency from cps-application pom

Issue-ID: CPS-2390
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Id49705e8790df7d7e98d80fa0ccb639eb4f1995f

7 months agoPolicy Executor: pass Change request as Object 67/138867/5
sourabh_sourabh [Fri, 30 Aug 2024 12:17:57 +0000 (13:17 +0100)]
Policy Executor: pass Change request as Object

- Added unit test for new exception handling

Issue-ID: CPS-2335
Change-Id: I5517c91edf505932d0a94165c19420922b3f5f51
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
7 months agoMerge "Support alternate id interface for CPS-E-05 - publicProperties"
Daniel Hanrahan [Mon, 2 Sep 2024 09:22:13 +0000 (09:22 +0000)]
Merge "Support alternate id interface for CPS-E-05 - publicProperties"

7 months ago[Cps Path Parser] Refactoring leaf conditions 82/138882/1
danielhanrahan [Sat, 31 Aug 2024 21:30:16 +0000 (22:30 +0100)]
[Cps Path Parser] Refactoring leaf conditions

Instead of DataLeaf and ComparativeOperators classes,
we more simply have LeafConditions class with all info.

Issue-ID: CPS-2365
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I8fd18bb56b8ed9d26a3f9d36f487d00a9274c8bc

7 months ago[k6] Report failure rate for all tests in a single plot 56/138856/4
danielhanrahan [Tue, 27 Aug 2024 16:57:26 +0000 (17:57 +0100)]
[k6] Report failure rate for all tests in a single plot

Instead of separate plots of test failures for each test case,
we have one single plot showing failures.

Issue-ID: CPS-2347
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ie79b75c34903a7df2037474b3536d4867c1a61e0

7 months agoMerge "Support 2 CPS instances in Prometheus/Grafana"
Priyank Maheshwari [Fri, 30 Aug 2024 15:39:58 +0000 (15:39 +0000)]
Merge "Support 2 CPS instances in Prometheus/Grafana"

7 months ago[k6] Fix k6 tests after DMI stub changes 77/138877/1
danielhanrahan [Fri, 30 Aug 2024 14:03:06 +0000 (15:03 +0100)]
[k6] Fix k6 tests after DMI stub changes

A change to DMI stub has broken k6 module search due
to module rename and increased registration time.

- increase setupTimeout by 2 minutes to factor in DMI startup delay
- module search uses module common to all module set tags
- only record times for searches returning number of CM-handles

Issue-ID: CPS-2353
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I9067dc73015cfb475cdce1e7a3cc2d5401449caf

7 months agoFix huge logs produced when errors occur in k6 54/138854/2
sourabh_sourabh [Tue, 27 Aug 2024 14:38:41 +0000 (15:38 +0100)]
Fix huge logs produced when errors occur in k6

 - Changed k6 config to ignore all the error logs.

Issue-ID: CPS-2344
Change-Id: Iad7be43ae5bb909518f9f7d8e386a97c96c349e3
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
7 months agoSupport alternate id interface for CPS-E-05 - publicProperties 55/138855/2
mpriyank [Tue, 27 Aug 2024 15:08:31 +0000 (16:08 +0100)]
Support alternate id interface for CPS-E-05 - publicProperties

- added alternate id support for the existing endpoint
- Incorporated previous comments on the indendation
- Added new component cmHandleReferenceInPath in openapi docs to depict
  the correct example

Issue-ID: CPS-2378
Change-Id: I63e752fbb6cb0bde49d1ced53f063743d904d74e
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
7 months agoMerge "[k6] Exclude timings when error occur"
Priyank Maheshwari [Thu, 29 Aug 2024 13:25:41 +0000 (13:25 +0000)]
Merge "[k6] Exclude timings when error occur"