cps.git
2 days agoAdded JettyConfig with support for ambiguous path separators 26/140926/4
sourabh_sourabh [Wed, 21 May 2025 11:54:15 +0000 (12:54 +0100)]
Added JettyConfig with support for ambiguous path separators

- Added JettyConfig to customize Jetty's HttpConfiguration
- For ambiguous path separators via UriCompliance settings.

Issue-ID: CPS-2819
Change-Id: I30cbb14cf31ac808a9a9761cf8dbf9ba0df870ae
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2 days agoMerge "Make Groovy Performance Test Stable"
Sourabh Sourabh [Wed, 21 May 2025 13:56:57 +0000 (13:56 +0000)]
Merge "Make Groovy Performance Test Stable"

2 days agoMake Groovy Performance Test Stable 23/140923/4
ToineSiebelink [Tue, 20 May 2025 15:43:49 +0000 (16:43 +0100)]
Make Groovy Performance Test Stable

- Removed average calculation where this lead to very small duration and therefore seemingly more variation
- Tweak a few expected duration to get closer to actual average
- Remove memory assertion from resource meter test since they are unreliable and cause the odd error
- Corrected wording on description related to limits

Issue-ID:CPS-2743
Change-Id: Id30bdbe7074297c32a236322fc66ee698fdd8e2b
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
2 days agoMerge openapi.yml and dmi-components.yml into one file 21/140921/2
egernug [Tue, 20 May 2025 12:33:51 +0000 (13:33 +0100)]
Merge openapi.yml and dmi-components.yml into one file

To aid downloading of Openapi spec for downloading in DMI-Plugin the spec and components file are bing combined.
This allows for the use of the existing openapi-generator-maven-pluginto be used to grab the file from CPS repo.

Issue-ID: CPS-2820

Change-Id: I08b4ae15fffb195e8bdd7a243de557f4b4fe9d60
Signed-off-by: egernug <gerard.nugent@est.tech>
3 days agoMake Groovy Performance Test Stable 16/140916/2
ToineSiebelink [Tue, 20 May 2025 08:50:41 +0000 (09:50 +0100)]
Make Groovy Performance Test Stable

- Tweak/correct more graphs
- Fix functional issue in test for alt-id lookup

Issue-ID: CPS-2743
Change-Id: I6f7a8d7cdac5b32a0a988fdac88c33b36f621499
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
3 days agoMerge "Fix broken docker-compose link"
Toine Siebelink [Tue, 20 May 2025 10:58:43 +0000 (10:58 +0000)]
Merge "Fix broken docker-compose link"

3 days agoFix broken docker-compose link 18/140918/1
halil.cakal [Tue, 20 May 2025 10:40:32 +0000 (11:40 +0100)]
Fix broken docker-compose link

- There is still a link that point the old docker-compose file that
  should be updated in deployment.rst

Issue-ID: CPS-2730

Change-Id: I3c328435531f6e14b84f2f2cf2b8cd9934ae46b4
Signed-off-by: halil.cakal <halil.cakal@est.tech>
3 days agoMerge "Remove -quiet option on k6 run"
Toine Siebelink [Tue, 20 May 2025 10:13:45 +0000 (10:13 +0000)]
Merge "Remove -quiet option on k6 run"

3 days agoRemove -quiet option on k6 run 17/140917/1
emaclee [Mon, 19 May 2025 10:30:23 +0000 (11:30 +0100)]
Remove -quiet option on k6 run

     - Removing -quiet option to unsuppressed the progress logs by k6

Issue-ID: CPS-2794
Change-Id: I5bd81b68cef41522433faba714f1427d5df05cbd
Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech>
4 days agoMerge "Make Groovy Performance Test Stable"
Lee Anjella Macabuhay [Mon, 19 May 2025 13:49:22 +0000 (13:49 +0000)]
Merge "Make Groovy Performance Test Stable"

4 days agoMake Groovy Performance Test Stable 89/140889/2
ToineSiebelink [Thu, 15 May 2025 12:08:15 +0000 (13:08 +0100)]
Make Groovy Performance Test Stable

- Disable memory assertion
- Allow default 50% margin on expected time
- Allow 300% margin on very fast test < 1ms
- Extend resource meter method with optional exceptional margin for an unstable test
- Update all time limits on plotted test to be average (rather then limit)

Issue-ID: CPS-2743
Change-Id: Icdcbe01b048c2d04166684cda4bf341a204c9a1d
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
7 days agoSplit big docker-compose into smaller ones 70/140870/3
halil.cakal [Fri, 9 May 2025 10:13:07 +0000 (11:13 +0100)]
Split big docker-compose into smaller ones

- split the big docker-compose and remove
- two new docker-compose: cps-base and dmi-services
- remove monitoring profile from: kafka-ui, prometheus, and grafana
  since they are not an overhead to performance
- kept 'tracing' profile for 'jaeger-service' because it reduced the
  performance by around 4 times (see the jira for more details)
- documentation updated

Issue-ID: CPS-2730

Change-Id: I4d9cdd82c96c2864c8eb7415b6857a4d2df898fc
Signed-off-by: halil.cakal <halil.cakal@est.tech>
7 days agoMerge "Fix component field in Datajobs API"
Sourabh Sourabh [Fri, 16 May 2025 08:55:45 +0000 (08:55 +0000)]
Merge "Fix component field in Datajobs API"

7 days agoMerge "Use an environment variable for the on-demand KPI pipeline"
Sourabh Sourabh [Fri, 16 May 2025 08:53:10 +0000 (08:53 +0000)]
Merge "Use an environment variable for the on-demand KPI pipeline"

7 days agoFix component field in Datajobs API 94/140894/2
egernug [Fri, 16 May 2025 08:23:25 +0000 (09:23 +0100)]
Fix component field in Datajobs API

When using an external component file it can be named anything but
an internal file must use components as the top level key

Issue-ID: CPS-2811

Change-Id: Id2e5ae24c9a78afe81b92fc972fcc2ffe38fab74
Signed-off-by: egernug <gerard.nugent@est.tech>
8 days agoUse an environment variable for the on-demand KPI pipeline 90/140890/2
halil.cakal [Thu, 15 May 2025 15:28:42 +0000 (16:28 +0100)]
Use an environment variable for the on-demand KPI pipeline

- introduce an environment variable that can be set (export) from
  the onap-cps-kpi-on-demand job (jenkins)

Issue-ID: CPS-2694

Change-Id: I33a2e583e06c4b5cbafcdcf0d05e8cb89a573c8e
Signed-off-by: halil.cakal <halil.cakal@est.tech>
8 days agoMerge changes Ie80fdd4c,I1663d8aa
Sourabh Sourabh [Thu, 15 May 2025 15:14:08 +0000 (15:14 +0000)]
Merge changes Ie80fdd4c,I1663d8aa

* changes:
  DCM Write optimization using batching
  Use Alternate Id to CM-handle ID map for DCM write datajobs

8 days agoMerge "Update KPI Test load for 2 tests"
Lee Anjella Macabuhay [Thu, 15 May 2025 10:27:53 +0000 (10:27 +0000)]
Merge "Update KPI Test load for 2 tests"

8 days agoDCM Write optimization using batching 85/140885/4
ToineSiebelink [Thu, 8 May 2025 16:24:39 +0000 (17:24 +0100)]
DCM Write optimization using batching

- Add method to AlternateIdMatcher to match a batch of paths and return the relevant yang model cm handles
- Extended inventory service to load cm handles without properties (i.e. descendants)
- Switch from NcmpServiceCmHandles to YangModel Cm Handles
- Preload required cm handles (without properties) using batching
- Removed (now) unused methods
- Improved test coverage on some legacy but related functionality

Issue-ID: CPS-2743
Change-Id: Ie80fdd4c12b72fc72ab1a87aa463ec0e6b664e3a
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
8 days agoMerge "Enhance the Multipart Delta API to support JSON file"
Priyank Maheshwari [Thu, 15 May 2025 09:19:48 +0000 (09:19 +0000)]
Merge "Enhance the Multipart Delta API to support JSON file"

8 days agoMerge "Move DMI-Plugin and DataJob API into CPS repo"
Toine Siebelink [Thu, 15 May 2025 09:15:55 +0000 (09:15 +0000)]
Merge "Move DMI-Plugin and DataJob API into CPS repo"

8 days agoUpdate KPI Test load for 2 tests 80/140880/2
ToineSiebelink [Tue, 13 May 2025 14:27:34 +0000 (15:27 +0100)]
Update KPI Test load for 2 tests

- TODO Remove KPI test tweaks (for local testing)
- Removed storm load from CM AVC background load
- Changed Legacy Batch test to fixed rate of 200 request events per second
- Changed Legacy Batch test to fixed time of 15 min (=180,000 events)

Issue-ID: CPS-2743
Change-Id: Id1c795a56797637efc7cbd74b5c2bbb688648ac9
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
9 days agoUse Alternate Id to CM-handle ID map for DCM write datajobs 27/140827/2
emaclee [Mon, 28 Apr 2025 10:13:15 +0000 (11:13 +0100)]
Use Alternate Id to CM-handle ID map for DCM write datajobs

Issue-Id: CPS-2743
Change-Id: I1663d8aab4a3d4cee898a3171dad926643ccbe79
Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech>
9 days agoEnhance the Multipart Delta API to support JSON file 68/140268/20
Rudrangi Anupriya [Wed, 9 Apr 2025 09:55:17 +0000 (15:25 +0530)]
Enhance the Multipart Delta API to support JSON file

Here we upload Json file as payloads instead of  JSON data as text input

Issue-ID: CPS-2657
Change-Id: Ibc9bbb99aabb07302366321c6d5c2eade0e7e5fb
Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>
9 days agoMerge "Increase startup delay from 1 sec to 2 sec"
Lee Anjella Macabuhay [Wed, 14 May 2025 08:55:08 +0000 (08:55 +0000)]
Merge "Increase startup delay from 1 sec to 2 sec"

10 days agoIncrease startup delay from 1 sec to 2 sec 72/140872/2
halil.cakal [Tue, 13 May 2025 10:19:54 +0000 (11:19 +0100)]
Increase startup delay from 1 sec to 2 sec

- if cps-ncmp-1 got a head start before cps-ncmp-0, and spin-off faster
  than cps-ncmp-0 the current delay '1 sec' is not enough to prevent race
condition between liquibase instances

Issue-ID: CPS-2807

Change-Id: I9ed5780daea4b95e6a2026e46427c7f4e464818e
Signed-off-by: halil.cakal <halil.cakal@est.tech>
10 days agoTest Cm Event Storm 74/140874/2
ToineSiebelink [Tue, 13 May 2025 11:41:50 +0000 (12:41 +0100)]
Test Cm Event Storm

- 6 minutes CM Event Background load to 26*500 = 13000 AVC/sec
- 8 minutes CM Event Background load to 6*500 = 3000 AVC/sec

Issue-ID: CPS-2743
Change-Id: Ide4201609e5d9766b3d3ab7c855dfd11159dafd0
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
10 days agoMove DMI-Plugin and DataJob API into CPS repo 73/140873/1
egernug [Tue, 13 May 2025 11:10:54 +0000 (12:10 +0100)]
Move DMI-Plugin and DataJob API into CPS repo

Issue-ID: CPS-2811

Change-Id: Idc5931ab81c7d6b836e589a024cb7df153bd48c8
Signed-off-by: egernug <gerard.nugent@est.tech>
10 days agoMerge "Ensure teardown script removes all onap images"
Sourabh Sourabh [Tue, 13 May 2025 08:58:02 +0000 (08:58 +0000)]
Merge "Ensure teardown script removes all onap images"

2 weeks agoIncrease Kafka rate in KPI 63/140863/1
egernug [Fri, 9 May 2025 10:47:36 +0000 (11:47 +0100)]
Increase Kafka rate in KPI

Issue-ID: CPS-2805

Change-Id: I7c0feb72d844bfd251c5dea9d3d22c3c35645884
Signed-off-by: egernug <gerard.nugent@est.tech>
2 weeks agoMerge "Adjusted NCMP architecture rule"
Toine Siebelink [Thu, 8 May 2025 13:06:41 +0000 (13:06 +0000)]
Merge "Adjusted NCMP architecture rule"

2 weeks agoAdjusted NCMP architecture rule 47/140847/3
leventecsanyi [Wed, 7 May 2025 10:20:04 +0000 (12:20 +0200)]
Adjusted NCMP architecture rule

  - removed spi package from ncmp service test

Issue-ID: CPS-2767
Change-Id: I482685ef4f8760a3cff3f3a6dfca12298b467209
Signed-off-by: leventecsanyi <levente.csanyi@est.tech>
2 weeks agoEnsure teardown script removes all onap images 50/140850/1
emaclee [Thu, 8 May 2025 07:29:30 +0000 (08:29 +0100)]
Ensure teardown script removes all onap images

- ensure that script removes test profile containers, volumes
  and images
- add method to remove all onap images when
  testProfile=endurance
- add method to verify that no onap images is left dangling when
  endurance test profile is finished

Issue-ID: CPS-2802
Change-Id: Ib319f8595555d315c4ec97795827d99d22fb29af
Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech>
2 weeks agoMerge "Revert "Add CM Write Data Job Scenario to K6 Test Suite""
Toine Siebelink [Wed, 7 May 2025 17:49:40 +0000 (17:49 +0000)]
Merge "Revert "Add CM Write Data Job Scenario to K6 Test Suite""

2 weeks agoRevert "Add CM Write Data Job Scenario to K6 Test Suite" 97/140797/1
Sourabh Sourabh [Wed, 7 May 2025 16:57:30 +0000 (16:57 +0000)]
Revert "Add CM Write Data Job Scenario to K6 Test Suite"

This reverts commit affb89aa97cc436d873065a5daec2fc3eb103f8f.

Reason for revert: Performance

Change-Id: Ic3a441f8d83ba2c347a23026348db09347956f85
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2 weeks agoMerge "Extend CpsPathQuerySpec for dangerous SQLi characters"
Toine Siebelink [Wed, 7 May 2025 12:42:28 +0000 (12:42 +0000)]
Merge "Extend CpsPathQuerySpec for dangerous SQLi characters"

2 weeks agoMerge "Fix the tests that failing irregularly in groovy pipeline"
Toine Siebelink [Wed, 7 May 2025 12:41:19 +0000 (12:41 +0000)]
Merge "Fix the tests that failing irregularly in groovy pipeline"

2 weeks agoAdd CM Write Data Job Scenario to K6 Test Suite 67/140667/10
sourabh_sourabh [Tue, 8 Apr 2025 17:08:09 +0000 (18:08 +0100)]
Add CM Write Data Job Scenario to K6 Test Suite

- Added 'write_data_job_scenario' in the K6 test configuration for small
  and large data size.
- It helps to simulate lower / higher load and monitor system behavior under infrequent
  write operations.

Issue-ID: CPS-2716
Change-Id: Ic5a5f3642ff57b294c822541dcc4b1458301ccac
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2 weeks agoFix the tests that failing irregularly in groovy pipeline 43/140843/2
halil.cakal [Tue, 6 May 2025 12:40:46 +0000 (13:40 +0100)]
Fix the tests that failing irregularly in groovy pipeline

- Don't check if the key is present in the map anymore instead keep
  looking for if the key deleted after TTL period
- Optimize the test by reducing the max wait time from 10 seconds to 1
  second

Issue-ID: CPS-2798

Change-Id: Ibbd40e682c28fa0c52da3f07d3687927fa42aa02
Signed-off-by: halil.cakal <halil.cakal@est.tech>
2 weeks agoInconsistency With JSON Response(List Items) Using GetANode API 45/140045/31
Rudrangi Anupriya [Mon, 5 May 2025 14:34:39 +0000 (20:04 +0530)]
Inconsistency With JSON Response(List Items) Using GetANode API

Jira - https://lf-onap.atlassian.net/browse/CPS-2566
Documentation -https://lf-onap.atlassian.net/wiki/x/OQA1AQ

- Introduced versioned V3 approach to retrieve list elements
- added controller method getNodeByDataspaceAndAnchorV3
- Updated listelementAsMap,containerElementsAsMap to group list elements

Issue-ID: CPS-2566
Change-Id: Ice9ffb99aabb03702355321c6d5c2eade0e7ef5b
Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>
3 weeks agoAdd non blocking comments from previous commit 40/140840/1
emaclee [Fri, 2 May 2025 12:23:37 +0000 (13:23 +0100)]
Add non blocking comments from previous commit

- Adding non blocking comments changes from 140838: Increase CM Notification Event KPI Background Load | https://gerrit.onap.org/r/c/cps/+/140838

Issue-ID: CPS-2752
Change-Id: I14c28466f69d9ff83692075f4283b7631e947fd3
Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech>
3 weeks agoIncrease CM Notification Event KPI Background Load 38/140838/1
ToineSiebelink [Tue, 29 Apr 2025 11:13:26 +0000 (12:13 +0100)]
Increase CM Notification Event KPI Background Load

- Update KPI script to send batches of messages
- Increased KPI load from 500 to 2,500 events/sec using 10 VUs and batchsize of 250
- Added instrumentation on cm avc forwarding method
- verified using Grafana (instrumentation) and kafka ui to confirm correct load of messages consumed and forwarded

Issue-ID: CPS-2752
Change-Id: I90951bf7844e087b908082c2609d0f3a564d50f9
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
3 weeks agoExtend CpsPathQuerySpec for dangerous SQLi characters 35/140835/2
egernug [Wed, 30 Apr 2025 09:16:40 +0000 (10:16 +0100)]
Extend CpsPathQuerySpec for dangerous SQLi characters

Issue-ID: CPS-2781

Change-Id: Ic658028d0ed48c3345db218ab913732d7665e2c6
Signed-off-by: egernug <gerard.nugent@est.tech>
3 weeks agoControl start-up delay: set host-names explicitly in docker compose 26/140826/7
ToineSiebelink [Mon, 28 Apr 2025 08:15:06 +0000 (09:15 +0100)]
Control start-up delay: set host-names explicitly in docker compose

- Use sequenced hostnames for cps-ncmp  in docker compose based on a common template
- Use explicit port definitions for each instance instead of range (instrumentation/Grafana)
- Calculated startup delay as sequence number times 1 second when correct pattern used
- Fall back on hash code modulus 10 seconds otherwise
- clear logging which algorithm is used, why and outcome
- update kpi and endurance environment files to use correct instance names

Issue-ID:CPS-2752

Change-Id: Ie788aff60d6e9c470136b2a2051d0c5ccc173e9a
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
3 weeks agoMerge "Increase the setup timeout in k6"
Sourabh Sourabh [Thu, 1 May 2025 09:55:36 +0000 (09:55 +0000)]
Merge "Increase the setup timeout in k6"

3 weeks agoMerge "Adjust KPI threshold and rate as per latest KPI agreement"
Priyank Maheshwari [Thu, 1 May 2025 09:47:46 +0000 (09:47 +0000)]
Merge "Adjust KPI threshold and rate as per latest KPI agreement"

3 weeks agoIncrease the setup timeout in k6 33/140833/2
halil.cakal [Thu, 1 May 2025 08:30:46 +0000 (09:30 +0100)]
Increase the setup timeout in k6

- Since the new server for on demand KPI jobs is slower than our regular
  server and fails because of a limit on the set up time this patch is
to increase that

Issue-ID: CPS-2694

Change-Id: I406faadec474f3147eaedee67ba1137a82207ccb
Signed-off-by: halil.cakal <halil.cakal@est.tech>
3 weeks agoAdjust KPI threshold and rate as per latest KPI agreement 34/140834/1
ToineSiebelink [Wed, 30 Apr 2025 13:25:29 +0000 (14:25 +0100)]
Adjust KPI threshold and rate as per latest KPI agreement

- CM Handle ID searches now can take 2.6 sec
- CM Handle searches now can take 24 sec
- Updated 'blue' lines to reflect KPI
- Updated 'green' lines to reflect current average (i.e. we expect measurements to be 'around' this)
- Experimental: increased cm notification forwarding by using more VUs
- Add timer metric to be able to check cm notification forwarding rate
- Temporary: removed redirecting of warning and errors to null device

Issue-ID: CPS-2793
Change-Id: I560985d4f9ce7a4c8aa27b3ae089a43a8a76b28a
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
3 weeks agoMerge "Fix legacy batch response with not found CM-handle"
Priyank Maheshwari [Wed, 30 Apr 2025 14:40:52 +0000 (14:40 +0000)]
Merge "Fix legacy batch response with not found CM-handle"

4 weeks agoFix legacy batch response with not found CM-handle 54/140654/3
danielhanrahan [Thu, 17 Apr 2025 17:21:14 +0000 (18:21 +0100)]
Fix legacy batch response with not found CM-handle

An unhandled exception leads to the entire batch operation failing,
even if only a single CM handle in the batch was not found. This
fixes the issue by swallowing the exception. Note there is existing
logic in DmiDataOperationsHelper for reporting not-found CM handles.

Issue-ID: CPS-2769
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ifff2c4bd291107480ca7462d721a02f8ea138e58

4 weeks agoRemove Basic Authentication References 08/140708/16
ToineSiebelink [Wed, 16 Apr 2025 12:47:09 +0000 (13:47 +0100)]
Remove Basic Authentication References

- Remove whole â€˜security’ section from application.yml file(s) also in testware
- Remove whole â€˜security’ section from  open api yml file(s) except Policy Executor
- Remove whole â€˜component.securitySchemes’ section from  open api yml files
- Remove references to security.auth.username from xml config files
- Removed cps authentication details from Docker compose and readme files (tested)
- Removed authentication groovy test rom cps-application
- Side note: cps-application added test, coverage increased to 100%
- Updated Docker-Compose (CSIT & K6) to latest version
- Minor csit script updates to help troubleshooting
- Removed auth header from ALL csit tests files

Issue-ID: CPS-2600
Change-Id: Ie1cf02461943f4e43029a3dbfaef052e347a4d4d
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
4 weeks agoMerge "Automate the CPS-NCMP metrics scraping"
Toine Siebelink [Thu, 24 Apr 2025 16:42:12 +0000 (16:42 +0000)]
Merge "Automate the CPS-NCMP metrics scraping"

4 weeks agoAutomate the CPS-NCMP metrics scraping 87/140787/6
halil.cakal [Thu, 24 Apr 2025 08:47:57 +0000 (09:47 +0100)]
Automate the CPS-NCMP metrics scraping

- import ScrapeMetrics, test_ScrapeMetrics and run from conf .py to
  automate the scraping
- the scripts will be running with tox when building the docs
- update unit tests to use test specific names
- highlight the header row of metrics' table

Issue-ID: CPS-2778

Change-Id: I843221ec21454220ea1182c488f88813c0691c4a
Signed-off-by: halil.cakal <halil.cakal@est.tech>
4 weeks agoMerge "Add integration tests of legacy batch data operation"
Toine Siebelink [Thu, 24 Apr 2025 11:48:51 +0000 (11:48 +0000)]
Merge "Add integration tests of legacy batch data operation"

4 weeks agoMerge "Scrape all metrics and add them to cps documentation"
Toine Siebelink [Wed, 23 Apr 2025 15:54:47 +0000 (15:54 +0000)]
Merge "Scrape all metrics and add them to cps documentation"

4 weeks agoScrape all metrics and add them to cps documentation 00/140700/8
halil.cakal [Thu, 10 Apr 2025 13:02:54 +0000 (14:02 +0100)]
Scrape all metrics and add them to cps documentation

- add a new dummy annotation (TimedCustom) for Gauge metrics (cm handle state)
- add a description field for the counter annotation (CountCmHandleSearchExecution)
- add a python script (with tests) to scrape the metrics mentioned above
  using a single regex for all
- remove redundancies (distribution management) from pom.xml of
  checkstyle module since there is no artifact released from checkstyle
- import and display the output (metrics.cvs) in admin-guide docs

Issue-ID: CPS-2709

Change-Id: Iaee23f0a20c05e5aea033baacad1f23cb61e8b34
Signed-off-by: halil.cakal <halil.cakal@est.tech>
4 weeks agoAdd integration tests of legacy batch data operation 10/140710/3
danielhanrahan [Wed, 16 Apr 2025 11:20:58 +0000 (12:20 +0100)]
Add integration tests of legacy batch data operation

Add integration tests for:
1. happy path: batch request forwarded to DMI
2. negative case: not ready CM-handles report error
3. negative case: not existing CM-handles report error

Case 3 is Ignored as there is currently a bug which causes the
request to fail, instead of reporting error on kafka topic.

Issue-ID: CPS-2769
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ic31bf856c09dd63ba80019fc251a437458184bba

5 weeks agoSimplify hostname-based startup delay logic with consistent hash-based approach 11/140711/1
sourabh_sourabh [Thu, 17 Apr 2025 16:38:54 +0000 (17:38 +0100)]
Simplify hostname-based startup delay logic with consistent hash-based approach

- Removed conditional logic based on last character of the hostname

- Applied a uniform hash-based delay calculation up to 5000 ms

- Improves consistency and simplifies the code logic

- Updated JavaDoc to reflect the new approach

Issue-ID: CPS-2752
Change-Id: Ic9e5cffb7396695e90b94edba79984f7f0f859aa
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
5 weeks agoBump CPS to 3.6.3-SNAPSHOT 07/140707/1
mpriyank [Wed, 16 Apr 2025 12:16:25 +0000 (13:16 +0100)]
Bump CPS to 3.6.3-SNAPSHOT

- bumped CPS-NCMP to 3.6.3-SNAPSHOT version

Issue-ID: CPS-2770
Change-Id: I7221363b048abacdf2ef9011efb5b98caeaa043c
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
5 weeks agoMerge "Container yaml for release"
Lee Anjella Macabuhay [Wed, 16 Apr 2025 12:09:36 +0000 (12:09 +0000)]
Merge "Container yaml for release"

5 weeks agoContainer yaml for release 06/140706/1
mpriyank [Wed, 16 Apr 2025 11:35:23 +0000 (12:35 +0100)]
Container yaml for release

Issue-ID: CPS-2770
Change-Id: If0f852fcdf32b658343de1d682019d6edb27f2b7
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
5 weeks agoMaven stage yaml 05/140705/1
mpriyank [Wed, 16 Apr 2025 11:23:56 +0000 (12:23 +0100)]
Maven stage yaml

- maven stage yaml for 3.6.2 release

Issue-ID: CPS-2770
Change-Id: Ia143acf696ddc7e8c31d2b1ea8e8772f1452a3e7
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
5 weeks agoRelease note to uplift spring boot 04/140704/2
mpriyank [Wed, 16 Apr 2025 10:32:02 +0000 (11:32 +0100)]
Release note to uplift spring boot

Issue-ID: CPS-2770
Change-Id: Icc66cde5de51e1f5d74d52556cd3068e07600f2e
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
5 weeks agoUpdate info and OAS for the release 03/140703/1
mpriyank [Wed, 16 Apr 2025 09:58:13 +0000 (10:58 +0100)]
Update info and OAS for the release

- info version set to 3.6.2 as it is the version being released

Issue-ID: CPS-2770
Change-Id: Idd78aded458710b39fcaaab24343fd93aee3f658
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
5 weeks agoReview RTD Configuration Properties 02/140702/2
ToineSiebelink [Tue, 15 Apr 2025 15:17:14 +0000 (16:17 +0100)]
Review RTD Configuration Properties

- split into 3 tables
- introduced CSV-file-defined tables for easier maintenance
- removed unnecessary 3PP properties definitions
- checked all custom properties still used and default values
- corrected some default values (sometime just wrong casing)
- updated application yaml spring.datasource.url so it has a default host value
- removed code-formatting for property values (except for URLS to prevent failing link-checks)
- added notes about using environment variables where applicable

Issue-ID: CPS-2596
Change-Id: I6f413dd726088ca8a002da145bf4f6c52a1c7b2a
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
5 weeks agoMerge "Use a Hazelcast map of alternate ids to Cm handle ids"
Daniel Hanrahan [Tue, 15 Apr 2025 10:44:56 +0000 (10:44 +0000)]
Merge "Use a Hazelcast map of alternate ids to Cm handle ids"

5 weeks agoUse a Hazelcast map of alternate ids to Cm handle ids 64/140664/9
emaclee [Tue, 8 Apr 2025 11:48:48 +0000 (12:48 +0100)]
Use a Hazelcast map of alternate ids to Cm handle ids

Issue-ID: CPS-2753
Change-Id: I6f3657bf14d78e412f650ee536e79beefb601c09
Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech>
5 weeks agoAdd HTTP 409 Conflict error response to NCMP OpenAPI Doc 91/140691/2
ToineSiebelink [Mon, 14 Apr 2025 12:50:50 +0000 (13:50 +0100)]
Add HTTP 409 Conflict error response to NCMP OpenAPI Doc

- This error can occur when Policy Execution denies write operation but was never documented

Issue-ID: CPS-2764
Change-Id: Idc47371b63b1348f86384e1ff3a8ae0d50056cdf
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
6 weeks agoMerge "Refactoring to combine LcmEventsCreator and LcmEventsCreatorHelper"
Priyank Maheshwari [Fri, 11 Apr 2025 13:08:58 +0000 (13:08 +0000)]
Merge "Refactoring to combine LcmEventsCreator and LcmEventsCreatorHelper"

6 weeks agoIntroduced a startup delay mechanism based on the container hostname 59/140659/8
sourabh_sourabh [Tue, 8 Apr 2025 10:13:08 +0000 (11:13 +0100)]
Introduced a startup delay mechanism based on the container hostname

- Extracted hostname using InetAddress and Calculated delay or fallback hash-based delay.
- This helps prevent the 'relation "databasechangelog" already exists' error seen in concurrent startups.

Issue-ID:CPS-2752
Change-Id: I051a8edd5ddab5a9fb012183b0526c113d90304e
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
6 weeks agoRefactoring to combine LcmEventsCreator and LcmEventsCreatorHelper 58/140658/2
emaclee [Mon, 7 Apr 2025 12:44:28 +0000 (13:44 +0100)]
Refactoring to combine LcmEventsCreator and LcmEventsCreatorHelper

- 1. Move all in LcmEventsCreatorHelper to LcmEventsCreator
- 2. Rename LcmEventsCreator to LcmEventsProducerHelper
- 3. Rename and update relevant tests
- 4. Refactor words 'publisher' to 'producer' and 'publish' to 'send' where applicable

Issue-ID: CPS-2726
Change-Id: Id0d53f851d25823e0456e95917c2be8856a20c40
Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech>
6 weeks agoMerge "Fix release notes: only mention end user features"
Toine Siebelink [Wed, 9 Apr 2025 16:00:49 +0000 (16:00 +0000)]
Merge "Fix release notes: only mention end user features"

6 weeks agoFix release notes: only mention end user features 53/140653/3
ToineSiebelink [Thu, 3 Apr 2025 17:23:10 +0000 (18:23 +0100)]
Fix release notes: only mention end user features

Issue-ID: CPS-2169

Change-Id: I5c1572d687fa7b4b0de0f57a318b63988b508121
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
6 weeks agoUpdate the script that is responsible for copyright checks 65/140665/1
halil.cakal [Mon, 7 Apr 2025 16:26:59 +0000 (17:26 +0100)]
Update the script that is responsible for copyright checks

- Since no more Nordix Foundation license header must be present in the codes, the script
  should look for OpenInfra Foundation Europe
- As an example LcmEventsCmHandleStateHandler.java has been modified and
  there is no issue reported

Issue-ID: CPS-2754

Change-Id: I4f251328649916fc482756ec94302b426e224e3b
Signed-off-by: halil.cakal <halil.cakal@est.tech>
6 weeks agoMerge "Refactor Cps Delta code from CpsDataService to CpsDeltaService"
Priyank Maheshwari [Mon, 7 Apr 2025 10:23:28 +0000 (10:23 +0000)]
Merge "Refactor Cps Delta code from CpsDataService to CpsDeltaService"

6 weeks agoMerge "Refactor Consumers/Producers based on agreed format #2"
Priyank Maheshwari [Mon, 7 Apr 2025 09:45:24 +0000 (09:45 +0000)]
Merge "Refactor Consumers/Producers based on agreed format #2"

6 weeks agoRefactor Consumers/Producers based on agreed format #2 07/140607/3
emaclee [Thu, 27 Mar 2025 12:40:26 +0000 (12:40 +0000)]
Refactor Consumers/Producers based on agreed format #2

- 'EventsPublisher' to 'EventsProducer'
- terms 'publish' is replaced with 'send' to follow kafka
  methods
- LcmEventsProducer is not fully changed as it may affect
  metrics; will handle on seperate patch

Issue-ID: CPS-2597
Change-Id: I310fc60fd0ff85eb83f2f3c6f9b54c569b3ff902
Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech>
7 weeks agoUpdate Springboot to eliminate transitive vulnerabilities 33/140633/2
egernug [Tue, 1 Apr 2025 09:12:57 +0000 (10:12 +0100)]
Update Springboot to eliminate transitive vulnerabilities

Issue-ID: CPS-2725

Change-Id: Iafa07af4d95bc2f396385c607dde63f2d3eab8c8
Signed-off-by: egernug <gerard.nugent@est.tech>
7 weeks agoMerge "Set initial delay in ModuleSyncWatchdog and DataSyncWatchdog"
Sourabh Sourabh [Thu, 3 Apr 2025 17:01:51 +0000 (17:01 +0000)]
Merge "Set initial delay in ModuleSyncWatchdog and DataSyncWatchdog"

7 weeks agoMerge "Use alternate IDs for Legacy Batch Read operation"
Daniel Hanrahan [Thu, 3 Apr 2025 12:09:05 +0000 (12:09 +0000)]
Merge "Use alternate IDs for Legacy Batch Read operation"

7 weeks agoSet initial delay in ModuleSyncWatchdog and DataSyncWatchdog 10/140610/11
leventecsanyi [Thu, 27 Mar 2025 15:26:31 +0000 (16:26 +0100)]
Set initial delay in ModuleSyncWatchdog and DataSyncWatchdog

  - added initial delay values to yml files
  - modified the RTD

Issue-ID: CPS-2731
Change-Id: I3dc8fd9ee44428c91c7f73e9e6a351ea61c0653c
Signed-off-by: leventecsanyi <levente.csanyi@est.tech>
7 weeks agoUse alternate IDs for Legacy Batch Read operation 44/140644/2
halil.cakal [Wed, 2 Apr 2025 09:49:07 +0000 (10:49 +0100)]
Use alternate IDs for Legacy Batch Read operation

- generate a random set of alternate ids out of 50K
- each alternate id is unique in the batches
- this is for both kpi and endurance pipelines

Issue-ID: CPS-2417

Change-Id: Id848593d472fa1df677ded301948e526e193aa5a
Signed-off-by: halil.cakal <halil.cakal@est.tech>
7 weeks agoRefactor Cps Delta code from CpsDataService to CpsDeltaService 43/140343/22
Arpit Singh [Wed, 12 Mar 2025 07:03:35 +0000 (12:33 +0530)]
Refactor Cps Delta code from CpsDataService to CpsDeltaService

- added new open api yaml file defining CPS delta endpoints
- Added new REST controller for CPS Delta
- Moved CPS Delta endpoints to new java interface, out from CPS Data
- Added integration tests for CPS Delta
- CPS Delta rest controller and java interface have the two delta
  endpoints: Delta between anchors and Delta between anchor and payload.

Issue-ID: CPS-2320
Change-Id: I556e70623a8c18f8cde3cd28bac890296019c0e2
Signed-off-by: Arpit Singh <AS00745003@techmahindra.com>
7 weeks agoAdd 'unpublished' REST interface for DataJobs Write to allow K6 testing 04/140604/8
sourabh_sourabh [Thu, 27 Mar 2025 10:15:46 +0000 (10:15 +0000)]
Add 'unpublished' REST interface for DataJobs Write to allow K6 testing

- Created a new hidden REST endpoint to use write job for internal
  testing like KPI and endurence.

Issue-ID: CPS-2718
Change-Id: I1c4c7bc145dd522d344bd98ea04f05068b3e43eb
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
7 weeks agoMerge "Move SDNC and PNFSim into docker-compose for CSITs"
Priyank Maheshwari [Wed, 2 Apr 2025 12:23:00 +0000 (12:23 +0000)]
Merge "Move SDNC and PNFSim into docker-compose for CSITs"

7 weeks agoPerformance fix for Module search outputting Alternate IDs 40/140640/3
danielhanrahan [Tue, 1 Apr 2025 15:04:39 +0000 (16:04 +0100)]
Performance fix for Module search outputting Alternate IDs

Use OMIT_DESCENDANTS instead of INCLUDE_ALL_DESCENDANTS.

Issue-ID: CPS-2738
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: If0ee84baa297137f7275e611905e4183a16c7ee1

7 weeks agoMerge "Added logging around write job operation to verify request and response detail...
Toine Siebelink [Wed, 2 Apr 2025 08:33:43 +0000 (08:33 +0000)]
Merge "Added logging around write job operation to verify request and response details, with a focus on the number of jobs involved"

7 weeks agoMerge "Fix for Trust Level search with alternate ID output"
Toine Siebelink [Wed, 2 Apr 2025 07:42:35 +0000 (07:42 +0000)]
Merge "Fix for Trust Level search with alternate ID output"

7 weeks agoFix for Trust Level search with alternate ID output 41/140641/1
danielhanrahan [Tue, 1 Apr 2025 14:18:48 +0000 (15:18 +0100)]
Fix for Trust Level search with alternate ID output

Currently the requests are timing out, over 60 seconds.
This commit restores the older implementation not using
attribute-axis, so that when the DataNodes are fetched,
we can get both the Cm Handle ID and Alternate ID.

Issue-ID: CPS-2738
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I2072eb5cae2fd04d8217be48c261e219c948a339

7 weeks agoAdded logging around write job operation to verify request and response details,... 24/140624/8
sourabh_sourabh [Sat, 29 Mar 2025 15:26:06 +0000 (15:26 +0000)]
Added logging around write job operation to verify request and response details, with a focus on the number of jobs involved

Issue-ID: CPS-2735
Change-Id: I8dd6260556b4949bda5695d22f3f862c0786e385
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
7 weeks agoMerge "Update release notes for Flux improvement"
Priyank Maheshwari [Tue, 1 Apr 2025 14:15:07 +0000 (14:15 +0000)]
Merge "Update release notes for Flux improvement"

7 weeks agoUpdate batch id searches to return alternate ids 12/140612/3
halil.cakal [Thu, 27 Mar 2025 11:13:03 +0000 (11:13 +0000)]
Update batch id searches to return alternate ids

- keep the query parameter (?outputAlternateId=true) common for id-searches and searches
  since it does not have any negative impact for the searches when provided this way the code
  is more simple and readable
- function names kept the same to comply with the response messages
- the plot names kept the same to maintain the history in local storage of kpi plot job

Issue-ID: CPS-2715

Change-Id: I8146e7ecb04ddd4b907fae94e91fd78bd535b58c
Signed-off-by: halil.cakal <halil.cakal@est.tech>
7 weeks agoUpdate release notes for Flux improvement 23/140623/1
danielhanrahan [Fri, 28 Mar 2025 18:52:38 +0000 (18:52 +0000)]
Update release notes for Flux improvement

Issue-ID: CPS-2712
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I0b78b6a1c37b0b5b06b0e3894d717ab2f64eabe4

8 weeks agoMove SDNC and PNFSim into docker-compose for CSITs 02/140402/3
danielhanrahan [Tue, 11 Feb 2025 18:34:57 +0000 (18:34 +0000)]
Move SDNC and PNFSim into docker-compose for CSITs

To improve reliability of healthchecks in CSITs, they are implemented
in docker-compose. This commit moves SDNC and dependencies into main
docker-compose and adds healthchecks to them. This improves the CSIT
in that tests will quickly fail if SDNC or PNFsim containers fail to
start (currently tests run anyway even if containers are not ready).

- Move SDNC, MariaDB and PNFSim containers to main docker-compose
- Add healthchecks and timeouts for those containers
- Move node mounting script as a sidecar container so it is protected
  by healthchecks

Issue-ID: CPS-2632
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ib53522c2f756d3ce2c6d6b7472cb4c65359fe355

8 weeks agoMerge "Efficient implementation of Attribute Axis in SQL"
Lee Anjella Macabuhay [Thu, 27 Mar 2025 10:56:13 +0000 (10:56 +0000)]
Merge "Efficient implementation of Attribute Axis in SQL"

8 weeks agoMerge "Fix NullPointerException for cm handle state metrics"
Toine Siebelink [Thu, 27 Mar 2025 10:14:21 +0000 (10:14 +0000)]
Merge "Fix NullPointerException for cm handle state metrics"

8 weeks agoMerge "Refactor Consumers/Producers based on agreed format"
Priyank Maheshwari [Thu, 27 Mar 2025 10:14:12 +0000 (10:14 +0000)]
Merge "Refactor Consumers/Producers based on agreed format"

8 weeks agoEfficient implementation of Attribute Axis in SQL 99/140599/3
danielhanrahan [Mon, 2 Sep 2024 17:59:41 +0000 (18:59 +0100)]
Efficient implementation of Attribute Axis in SQL

Attribute Axis is the feature which allows fetching only a single
attribute, e.g. //books[@title='Matilda']/@price -> [15]
This implements the attribute axis feature directly in SQL, giving
much higher performance e.g. for CM-handle ID searches in NCMP.
The native SQL implementation directly returns data leaves from DB,
not requiring conversions to FragmentEntity, DataNode, etc.

Issue-ID: CPS-2623
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I54f517e47ca6bcddfae356f98857b05fd2e1229e

8 weeks agoMerge "Use Flux stream processing for CM-handle searches"
Toine Siebelink [Thu, 27 Mar 2025 08:36:32 +0000 (08:36 +0000)]
Merge "Use Flux stream processing for CM-handle searches"

8 weeks agoAdd CloudEventBuilder to SpringBootTest 00/140600/1
egernug [Wed, 26 Mar 2025 16:00:47 +0000 (16:00 +0000)]
Add CloudEventBuilder to SpringBootTest

There are test failures on:
DmiInEventProducerSpec
NcmpOutEventProducerSpec
stemming from CloudEventBuilder being missing from the SpringBootTest. Adding this to clear issues.

Issue-ID: CPS-2727

Change-Id: I5e838741d7fc54bc91c5d91497085da68ab391d4
Signed-off-by: egernug <gerard.nugent@est.tech>