cps.git
10 months agoMerge "Normalize parent xpath when building datanodes in CpsDataService"
Toine Siebelink [Thu, 29 Jun 2023 13:49:21 +0000 (13:49 +0000)]
Merge "Normalize parent xpath when building datanodes in CpsDataService"

10 months agoMerge "Update write performance test timings"
Toine Siebelink [Thu, 29 Jun 2023 13:48:20 +0000 (13:48 +0000)]
Merge "Update write performance test timings"

10 months agoUpdate write performance test timings 46/135246/2
danielhanrahan [Thu, 29 Jun 2023 10:28:47 +0000 (11:28 +0100)]
Update write performance test timings

Issue-ID: CPS-1673
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ia2e5bf557bf2ade98fba64b7923856f727049d84

10 months agoImproved code coverage (branches) around multipart file utils 44/135244/1
ToineSiebelink [Thu, 29 Jun 2023 09:42:23 +0000 (10:42 +0100)]
Improved code coverage (branches) around multipart file utils

- added test
- moved wrongly placed test (from MultipartFileUtilSpec to DataMapUtilsSpec)
- applied curent conventions to DataMapUtilsSpec

Issue-ID: CPS-475

Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: Ie8dc7f6802bb6ff0a256dc05df73da6af75409f3

10 months agoMerge "Client to NCMP Subscription schema change"
Priyank Maheshwari [Wed, 28 Jun 2023 15:53:55 +0000 (15:53 +0000)]
Merge "Client to NCMP Subscription schema change"

10 months agoClient to NCMP Subscription schema change 86/135186/8
emaclee [Tue, 27 Jun 2023 09:45:52 +0000 (10:45 +0100)]
Client to NCMP Subscription schema change

- Schema changes only
- Following code change changes on seperate commit

Issue-ID: CPS-1736
Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech>
Change-Id: I37f67a325d800a479c065ff72822b3e51d7f525f

10 months agoImproved code covage (branches) around cm handle state changes 06/135206/1
ToineSiebelink [Wed, 28 Jun 2023 11:09:29 +0000 (12:09 +0100)]
Improved code covage (branches) around cm handle state changes

- added test
- removed redundant (unreachable) check in production code

Issue-ID: CPS-475
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: Ia39ac7a1473a248710c2a52f65715fb6b6a85fca

10 months agoMerge "Subscription dmi_to_ncmp package clean-up"
Luke Gleeson [Wed, 28 Jun 2023 09:44:46 +0000 (09:44 +0000)]
Merge "Subscription dmi_to_ncmp package clean-up"

10 months agoMerge "Streamline outcome response for subscription creation"
Priyank Maheshwari [Wed, 28 Jun 2023 09:18:36 +0000 (09:18 +0000)]
Merge "Streamline outcome response for subscription creation"

10 months agoNormalize parent xpath when building datanodes in CpsDataService 81/135181/3
danielhanrahan [Mon, 26 Jun 2023 12:21:21 +0000 (13:21 +0100)]
Normalize parent xpath when building datanodes in CpsDataService

Data nodes are being saved with non-normalized xpaths, resuling in
data nodes that cannot be operated on. This affects all operations
including get, query, update, and delete.

Issue-ID: CPS-1765
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I5352182d79daec67805753ca5943b1a86c18159f

10 months agoSubscription dmi_to_ncmp package clean-up 97/135197/2
seanbeirne [Tue, 27 Jun 2023 15:56:31 +0000 (16:56 +0100)]
Subscription dmi_to_ncmp package clean-up

Issue-ID: CPS-1738
Signed-off-by: seanbeirne <sean.beirne@est.tech>
Change-Id: I7e01c89cb187233b53b8786e42d2fe2c0f24dbd9

10 months agoNCMP : Handle non-existing and non-ready cm handles 82/135182/6
sourabh_sourabh [Mon, 26 Jun 2023 10:15:57 +0000 (11:15 +0100)]
NCMP : Handle non-existing and non-ready cm handles

- Modified data operation schema to contains cm handle as steing and
  response content field name as result.
- Added a new common cloud event builder for NCMP to create an event.
- Added data operation event creater that uses cloud event builder to
  create a cloud event.
- Introduced a new method onto json object mapper to convert json object
  to bytes.
- Modified EventDateTimeFormatter and added a new method to convert date timestamp into offsetdateTime.
- Added a new code into ResourceRequestUtil to identify non-ready cm
  handle and non-existing cm handle and later publish it as cloud event
  to tha client given topic.
- Introduced CpsApplicationContext to get spring mannaged bean into non
  spring managed java object.

Issue-ID: CPS-1583, CPS-1614

Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: I24a39d2cb2c54dea25cd2f17e7748e21cd83a088
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
10 months agoGenerate docs using OpenAPI 3.0 85/135185/1
egernug [Tue, 27 Jun 2023 08:31:23 +0000 (09:31 +0100)]
Generate docs using OpenAPI 3.0

Doc generation in OpenAPI 3.0 requires executions for each file being generated.
This change adds those executions and updates the dependency to copy these files.

Issue-ID:CPS-1745

Signed-off-by: egernug <gerard.nugent@est.tech>
Change-Id: I144b49b0066f798f15408d955dcd914dd2c0bc91

10 months agoStreamline outcome response for subscription creation 59/134859/15
halil.cakal [Wed, 14 Jun 2023 07:10:15 +0000 (08:10 +0100)]
Streamline outcome response for subscription creation

- Pivot to outcome response generated from db
  for the corresponding subscription
- Fine tuning to wait till the cutoff time
  for all cm handles that subject to subscription
- Update cm handles as rejected if dmi plugin or
  cm handles does not exist in db
- Add more test for missed branches in mappers
  and response outcome

Issue-ID: CPS-1741
Change-Id: I82ab6958e673059977abc7708974d5a2cb13e339
Signed-off-by: halil.cakal <halil.cakal@est.tech>
10 months agoMerge "KafkaListener config for CloudEvents"
Sourabh Sourabh [Mon, 26 Jun 2023 12:17:36 +0000 (12:17 +0000)]
Merge "KafkaListener config for CloudEvents"

10 months agoKafkaListener config for CloudEvents 54/135154/3
mpriyank [Fri, 23 Jun 2023 14:01:38 +0000 (15:01 +0100)]
KafkaListener config for CloudEvents

- Introduced kafkalistener factory configs to correctly use the configs
  and convert the incoming events to CloudEvents
- Also legacy events to use a separate listener factory
- Increased the wait time from 100ms to 300ms to listen to the message
  in the test

Issue-ID: CPS-1764
Change-Id: I8b9fe0ba82ef87d52b7731941ccd3af8ae980109
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
10 months agoChange exclude path for mappers into more general 55/135155/1
halil.cakal [Mon, 26 Jun 2023 11:00:51 +0000 (12:00 +0100)]
Change exclude path for mappers into more general

Issue-ID: CPS-1741
Change-Id: I5ad92f19d65bbd7577014b6ea2ffbfffd41e0b00
Signed-off-by: halil.cakal <halil.cakal@est.tech>
10 months agoMerge "Adding logic to read response files from"
Toine Siebelink [Mon, 26 Jun 2023 09:46:10 +0000 (09:46 +0000)]
Merge "Adding logic to read response files from"

10 months agoMerge "#2 NCMP : Replacing the word 'Batch' as 'DataOperation'."
Toine Siebelink [Mon, 26 Jun 2023 09:29:58 +0000 (09:29 +0000)]
Merge "#2 NCMP : Replacing the word 'Batch' as 'DataOperation'."

10 months agoMerge "Chsnge to OpenApi 3.0"
Toine Siebelink [Mon, 26 Jun 2023 09:28:34 +0000 (09:28 +0000)]
Merge "Chsnge to OpenApi 3.0"

10 months agoAdding logic to read response files from 50/135050/5
waqas.ikram [Wed, 21 Jun 2023 13:16:22 +0000 (14:16 +0100)]
Adding logic to read response files from

filesystem for client integration tests

Issue-ID: CPS-1751
Change-Id: I62e24d09474184716c56149a06deae4500b7b286
Signed-off-by: waqas.ikram <waqas.ikram@est.tech>
10 months agoChsnge to OpenApi 3.0 43/135043/6
egernug [Wed, 21 Jun 2023 10:16:02 +0000 (11:16 +0100)]
Chsnge to OpenApi 3.0

In preparation for the Java 17 and Springboot 3 migrations Swagger Codegen has to upgrade to OpenApi 3.0
This effects cps-rest and cps-ncmp-rest

Issue-Id: CPS-1745

Signed-off-by: egernug <gerard.nugent@est.tech>
Change-Id: I18b8e9ca26d0123009702c2474efbfbeec6d4c44

10 months ago#2 NCMP : Replacing the word 'Batch' as 'DataOperation'. 35/135135/1
raviteja.karumuri [Fri, 23 Jun 2023 12:52:18 +0000 (13:52 +0100)]
#2 NCMP : Replacing the word 'Batch' as 'DataOperation'.

Issue-ID: CPS-1754
Signed-off-by: raviteja.karumuri <raviteja.karumuri@est.tech>
Change-Id: Ib04e9d8a909e8868bf094e2625412996409b3a67

10 months agoMerge "Improve code coverage"
Luke Gleeson [Fri, 23 Jun 2023 10:48:27 +0000 (10:48 +0000)]
Merge "Improve code coverage"

10 months agoMerge "NCMP : Replacing the word 'Batch' as 'DataOperation'."
Sourabh Sourabh [Fri, 23 Jun 2023 10:15:25 +0000 (10:15 +0000)]
Merge "NCMP : Replacing the word 'Batch' as 'DataOperation'."

10 months agoImprove code coverage 98/135098/1
ToineSiebelink [Thu, 22 Jun 2023 15:53:33 +0000 (16:53 +0100)]
Improve code coverage

cleaned up the modified test a bit as well

Issue-ID: CPS-475
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I0a6ac50bb5720935c5c1c7036ada8423b3174fda

10 months agoSubscription Response Event Schema 41/135041/6
seanbeirne [Wed, 21 Jun 2023 09:42:14 +0000 (10:42 +0100)]
Subscription Response Event Schema

Issue-ID: CPS-1738
Signed-off-by: seanbeirne <sean.beirne@est.tech>
Change-Id: I409baf66ca702166308821eeeff4887e9385148e

10 months agoNCMP : Replacing the word 'Batch' as 'DataOperation'. 36/135036/5
raviteja.karumuri [Tue, 20 Jun 2023 18:10:21 +0000 (19:10 +0100)]
NCMP : Replacing the word 'Batch' as 'DataOperation'.

Issue-ID: CPS-1754
Signed-off-by: raviteja.karumuri <raviteja.karumuri@est.tech>
Change-Id: Ie9f56effb160582ba7177331fb6260b2bde2342b

10 months agoMerge "Improve performance of updateDataLeaves"
Toine Siebelink [Thu, 22 Jun 2023 10:17:18 +0000 (10:17 +0000)]
Merge "Improve performance of updateDataLeaves"

10 months agoAdd RTD link to CPS in Release notes 78/135078/1
ToineSiebelink [Thu, 22 Jun 2023 08:24:04 +0000 (09:24 +0100)]
Add RTD link to CPS in Release notes

Issue-ID: REQ-1580
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: Id7f561a4686f62697bb98f7049199b44ebd387d4

10 months agoMerge "Remove irrelevant groovy class that tests nothing"
Toine Siebelink [Wed, 21 Jun 2023 17:04:52 +0000 (17:04 +0000)]
Merge "Remove irrelevant groovy class that tests nothing"

10 months agoImprove handling of legacy events 57/135057/1
ToineSiebelink [Wed, 21 Jun 2023 15:45:47 +0000 (16:45 +0100)]
Improve handling of legacy events

Legacy (non-cloud) events would cause a NPE in the event filter
Although the event woudl still be ignored it would lead to uncessary error logging
This fix addressed this issue using a trace level message instead
(also some improvements in the associated test)

Issue-ID: CPS-1724
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: Ibf71b13e4a47aaf705e32df5fe50cf41de5f558f

10 months agoRemove irrelevant groovy class that tests nothing 56/135056/1
halil.cakal [Wed, 21 Jun 2023 15:54:09 +0000 (16:54 +0100)]
Remove irrelevant groovy class that tests nothing

Issue-ID: CPS-1741
Change-Id: If27a74e2496b952eb4803bbabb8363400c57915a
Signed-off-by: halil.cakal <halil.cakal@est.tech>
10 months agoMerge "Patch # 3: Data operation response event (NCMP → Client App) to comply with...
Toine Siebelink [Wed, 21 Jun 2023 14:44:41 +0000 (14:44 +0000)]
Merge "Patch # 3: Data operation response event (NCMP → Client App) to comply with CloudEvents"

10 months agoImprove performance of updateDataLeaves 52/135052/1
danielhanrahan [Wed, 21 Jun 2023 12:53:32 +0000 (13:53 +0100)]
Improve performance of updateDataLeaves

Avoid fetching descendants during batchUpdateDataLeaves,
as descendants are not needed:
- Remove prefetch descendants step from getFragmentEntities;
- Explicitly prefetch descendants in operations requiring it;
- Update performance tests (5x faster for batch update).

Issue-ID: CPS-1675
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I7442a6f799cc0803b3a78f09d1ee53377f24b0b7

10 months agoAdd performance tests for updateDataLeaves 46/135046/3
danielhanrahan [Wed, 21 Jun 2023 11:22:46 +0000 (12:22 +0100)]
Add performance tests for updateDataLeaves

Issue-ID: CPS-1675
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I4dcd2ff0d0f6c3acf7373357da0ef7217a691573

10 months agoPatch # 3: Data operation response event (NCMP → Client App) to comply with CloudEvents 04/135004/5
sourabh_sourabh [Mon, 19 Jun 2023 16:35:15 +0000 (17:35 +0100)]
Patch # 3: Data operation response event (NCMP → Client App) to comply with CloudEvents

- Modified data operation record strategy to consume cloud event.
- Modified NCMP data operation event consumer to read cloud event
  header. (prefixed with ce_)
- Modified event publisher to support legacy and cloud event based on
  event type (if legacy event use legacy kafka template else cloud kafka
  template).
- Introduced a new method onto json object mapper to convert json object
  to bytes.
- Modified data operation consumer spec to produce a cloud event and validate it.
- Added Kafka Integration Test (for filtering)

Issue-ID: CPS-1724

Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: Ide701b1ff952f57413cb4e4aa0d55c08753f0298
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
10 months agopublishLcmEventBatchAsynchronously is not async 38/135038/4
danielhanrahan [Tue, 20 Jun 2023 17:19:59 +0000 (18:19 +0100)]
publishLcmEventBatchAsynchronously is not async

Spring @Async has a limitation where calling the async method from
within the same class won't work, as the method needs to be proxied,
and calling from the same class bypasses the proxy and calls the
underlying method directly, thus running synchronously.

Issue-ID: CPS-1749
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I7f198487dc18b359654dc38b5cf8fd600d33e189

10 months agoApply latest ReedTheDoc config rules 40/135040/1
ToineSiebelink [Wed, 21 Jun 2023 09:13:46 +0000 (10:13 +0100)]
Apply latest ReedTheDoc config rules

Issue-ID: CPS-1757
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: Icdf5fe5b5b1d6628bed786a1d99816e7990302cc

10 months agoRefactored code to enable its utilization in Junit 42/134942/3
waqas.ikram [Fri, 16 Jun 2023 12:05:04 +0000 (13:05 +0100)]
Refactored code to enable its utilization in Junit

tests for the client

Change-Id: I847ae0f48444af907039bfddb9879581003c0f35
Issue-ID: CPS-1751
Signed-off-by: waqas.ikram <waqas.ikram@est.tech>
10 months agoMerge "Fix test-deregistration script"
Toine Siebelink [Tue, 20 Jun 2023 09:00:13 +0000 (09:00 +0000)]
Merge "Fix test-deregistration script"

10 months agoMerge "DMI Data AVC to cloud events"
Luke Gleeson [Tue, 20 Jun 2023 08:38:01 +0000 (08:38 +0000)]
Merge "DMI Data AVC to cloud events"

10 months agoMerge "Patch # 1: Data operation response event (NCMP → Client App) to comply with...
Toine Siebelink [Tue, 20 Jun 2023 08:36:39 +0000 (08:36 +0000)]
Merge "Patch # 1: Data operation response event (NCMP → Client App) to comply with CloudEvents"

10 months agoMerge "Replace deprecated WebSecurityConfigurerAdapter"
Toine Siebelink [Tue, 20 Jun 2023 07:30:56 +0000 (07:30 +0000)]
Merge "Replace deprecated WebSecurityConfigurerAdapter"

10 months agoFix test-deregistration script 94/134994/1
danielhanrahan [Fri, 2 Jun 2023 10:44:09 +0000 (11:44 +0100)]
Fix test-deregistration script

- Fix issue where script breaks with newer version of docker-compose
  caused by profile names being omitted during `docker-compose down`
- Use main .gitignore file to ignore metrics-reports directory
- Fix Intelli-j/shellcheck warnings (mostly variables needing quotes)
- Script prints summary report at exit

Issue-ID: CPS-1552
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ia51e51c1c199de8cc2f4878bfb04cb09fbb0b0e7

10 months agoUpdate performance test timings 73/134973/2
danielhanrahan [Mon, 19 Jun 2023 13:44:21 +0000 (14:44 +0100)]
Update performance test timings

The fixes to lower memory usage in FragmentRepository have
positively affected read performance, so tests must be updated.

Issue-ID: CPS-1716
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I2c8b882f56fb8c1eaefe5b802da1d308d16b1f02

10 months agoPatch # 1: Data operation response event (NCMP → Client App) to comply with CloudEvents 92/134792/14
sourabh_sourabh [Thu, 8 Jun 2023 14:25:33 +0000 (15:25 +0100)]
Patch # 1: Data operation response event (NCMP → Client App) to comply with CloudEvents

 - Removed header definitions (since we now use CloudEvents)
 - Used 'dataOperation' instead of batch where appropriate.
 - Modified test json

Issue-ID: CPS-1724
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: Ic0f65297b944adf9cf5f3c2cbec679a031a675ec
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
10 months agoMerge "Code to persist or update the yang model subscription into db"
Priyank Maheshwari [Mon, 19 Jun 2023 13:49:11 +0000 (13:49 +0000)]
Merge "Code to persist or update the yang model subscription into db"

10 months agoCode to persist or update the yang model subscription into db 01/134801/7
halil.cakal [Fri, 9 Jun 2023 13:58:00 +0000 (14:58 +0100)]
Code to persist or update the yang model subscription into db

- Change subscription persistence for subscription registry
  to handle adding or updating cm handles into db

Issue-ID: CPS-1735
Change-Id: Icae609e810b2a097a4a174de1691892526f6b5c0
Signed-off-by: halil.cakal <halil.cakal@est.tech>
10 months agoDMI Data AVC to cloud events 38/134938/2
mpriyank [Fri, 16 Jun 2023 14:55:52 +0000 (15:55 +0100)]
DMI Data AVC to cloud events

- DMI Data AVC to be consumed as CloudEvents now
- Removed the schema header as it is taken care by cloudevent headers
- Implemented naming and packaging comments on the schema
- Test cases refactoring

Issue-ID: CPS-1719
Change-Id: I9864f90446720fe903fb3c1502d86035d886751d
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
10 months agoMove integration test (clean-up, last phase) 15/134815/8
ToineSiebelink [Mon, 12 Jun 2023 15:24:02 +0000 (16:24 +0100)]
Move integration test (clean-up, last phase)

- Moved session manager test
- Improved schemaset concurrency test (retry testing)
- Cleaned up all ri (container based) integration test (setup)
- Applied some groovy best practice where needed
- internal ri module cover now down to 29%, covred by integration instead with
  - Line coverage up by 41 lines to 99% (was 97%)
  - Branch coverage up by 3 branches to 96% (was 93%)

Issue-ID: CPS-1687
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: Ifb77a053e5a5db62a3f6a32ae60a3a8b10918efd

10 months agoMerge "Patch #2 : Introduce kafka template for cloud events"
Toine Siebelink [Thu, 15 Jun 2023 16:44:02 +0000 (16:44 +0000)]
Merge "Patch #2 : Introduce kafka template for cloud events"

10 months agoPatch #2 : Introduce kafka template for cloud events 96/134896/3
sourabh_sourabh [Thu, 15 Jun 2023 02:12:29 +0000 (03:12 +0100)]
Patch #2 : Introduce kafka template for cloud events

 - Introduced a new cloud kafka template for cloud events that reads
   it's configuration from application.yml
 - Kept legacy kafka template for backward compatibility utill all cps
   events moved to cloud event comply.
 - Modified application.yml producer and consumer value deserializer properties to support cloud events.
 - Added new cloudevents-bom used into cps-ncmp-service pom.
 - For the time being we will have 2 kafkatemplates (legacyEventKafkaTemplate, cloudEventKafkaTemplate) into EventsPublisher until we fully move to
  cloudevents for all events. Once all cps events will be cloud event compy, we have TODO task where Deprecated: legacyKafkaEventTemplate will be removed with its
  java configuration file KafkaTemplateConfig.

Issue-ID: CPS-1724

Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: I78c15bd480db063b89c6630c46c2d3a328b4fae4
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
10 months agoBump CPS version to 3.3.3-SNAPSHOT 07/134907/1
danielhanrahan [Thu, 15 Jun 2023 12:52:34 +0000 (13:52 +0100)]
Bump CPS version to 3.3.3-SNAPSHOT

- Update version properties as well as we prepare for the next release.

Issue-ID: CPS-1750
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I0626eee7834499d2fa96812fb416ce7daebf0529

10 months agoAdd container yaml for 3.3.2 05/134905/1
danielhanrahan [Thu, 15 Jun 2023 12:31:24 +0000 (13:31 +0100)]
Add container yaml for 3.3.2

- Step 6 of the release process

Issue-ID: CPS-1703
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I58e54ecd17878d3b9b6bc69ac9d254e454bd68b2

10 months agoAdd maven stage yaml for 3.3.2 03/134903/2
danielhanrahan [Thu, 15 Jun 2023 11:37:24 +0000 (12:37 +0100)]
Add maven stage yaml for 3.3.2

- Step 5 of the release process

Issue-ID: CPS-1703
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I105dedd6b1a634ca8ac4409ed682f6a10d1b018a

10 months agoReplace deprecated WebSecurityConfigurerAdapter 02/134902/1
egernug [Thu, 15 Jun 2023 10:44:50 +0000 (11:44 +0100)]
Replace deprecated WebSecurityConfigurerAdapter

WebSecurityConfigurerAdapter has been deprecated in Spring 2.7. Change needed to use SecurityFilterChain and InMemoryUserDetailsManager.
Note: WebMvcTest does not autoimport configurations that use beans.

Issue-Id: CPS-1742

Signed-off-by: egernug <gerard.nugent@est.tech>
Change-Id: I100884f10beaac6ce87e603302280aeb78559017

10 months agoUpdate release docs 99/134899/1
danielhanrahan [Thu, 15 Jun 2023 10:08:54 +0000 (11:08 +0100)]
Update release docs

- Also synched up the API changes in openapi yml file
- Step 1 and 2 of the release process

Issue-ID: CPS-1703
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Id47822737df3b89dffddb90413a53cac2b7ced6a

10 months agoLower memory usage in FragmentRepository 91/134791/6
danielhanrahan [Thu, 8 Jun 2023 13:37:17 +0000 (14:37 +0100)]
Lower memory usage in FragmentRepository

Avoid using Spring Data "interface projection" in FragmentRepository.
The use of FragmentExtract in FragmentRepository is causing an
overhead of around 5 kilobytes per fragment, which is leading to
abnormally high memory usage when queries return a large number of
nodes. For example, around 250MB of additional memory is needlessly
used when fetching 50,000 datanodes.

- Remove FragmentExtract interface and FragmentEntityArranger class.
- Add FragmentPrefetchRepository, using JdbcTemplate and RowMapper
  to fetch FragmentEntity descendants in a single SQL query.
- Many CpsDataService operations have memory reductions:
  - queryDataNodes
  - getDataNodesForMultipleXpaths
  - updateDataNodesAndDescendants
  - updateNodeLeaves
  - and any NCMP methods using the above.

Issue-ID: CPS-1716
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ic47a2c9eb34150ed76bd5ce452fe1c9aaf9b4c5c

10 months agoMerge "Move integration test (DataService)"
Toine Siebelink [Tue, 13 Jun 2023 10:56:29 +0000 (10:56 +0000)]
Merge "Move integration test (DataService)"

10 months agoFix for recursive SQL returning extra level of descendants 14/134814/1
danielhanrahan [Mon, 12 Jun 2023 15:24:55 +0000 (16:24 +0100)]
Fix for recursive SQL returning extra level of descendants

Recursive SQL queries are incorrectly returning maxDepth + 1 levels
of fragments due to incorrect comparison in the WHERE clause.
This code is used by getDataNodes and queryDataNodes to
prefetch the descendants before converting to datanodes.
Prefetching extra descendants causes high memory usage.

Issue-ID: CPS-1716
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ie22215c0c7018cd293f73cf75adca2cd55f218a6

10 months agoMove integration test (DataService) 97/134797/3
ToineSiebelink [Tue, 23 May 2023 08:30:37 +0000 (09:30 +0100)]
Move integration test (DataService)

- Moved all Data Intergation Test
- Added/improved some module Moudle Servcie test too
- Removed some unused (impossible) code-paths in prod code
- Replaced null's with meaningful strings (in edge case scenarios)
- Replaced @Shared with static (shorter) in some test

Issue-ID: CPS-1687
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: Ib4b1ec1706ace7e3f5ac03d85177a6f7d4f6b8d0

11 months agoAgreed outstanding comments including package refactoring 78/134778/2
halil.cakal [Tue, 6 Jun 2023 15:20:55 +0000 (16:20 +0100)]
Agreed outstanding comments including package refactoring

- Change package of AVC Subscription relevent codes into their place
- Change package of Subscription Event Cache config inot its place
- Add more branches for subscription outcome mapper
- Add more branches for subscription event response consumer
- Change unit test method params in order not to use deprecated methods

Issue-ID: CPS-1730
Change-Id: Ieda587d5be318db8360d52d49dc38d7ce3dd85cd
Signed-off-by: halil.cakal <halil.cakal@est.tech>
11 months agoFix some code smells 83/134783/1
halil.cakal [Wed, 7 Jun 2023 10:02:30 +0000 (11:02 +0100)]
Fix some code smells

Issue-ID: CPS-1507
Change-Id: Ic11ccbb37538242cb65ab1dbdde4fc77d9f94bdf
Signed-off-by: halil.cakal <halil.cakal@est.tech>
11 months agoMerge "NCMP : forward bulk response messages to client topic"
Sourabh Sourabh [Wed, 7 Jun 2023 08:49:32 +0000 (08:49 +0000)]
Merge "NCMP : forward bulk response messages to client topic"

11 months agoNCMP : forward bulk response messages to client topic 52/134752/3
raviteja.karumuri [Fri, 2 Jun 2023 09:45:13 +0000 (10:45 +0100)]
NCMP : forward bulk response messages to client topic

# Fixing the avc subscription event is not consuming even there is a record published on to the topic.

Issue-ID: CPS-1557
Signed-off-by: raviteja.karumuri <raviteja.karumuri@est.tech>
Change-Id: If09fd1849f467785141cc56639839ddda9f2c0de

11 months agoMerge "<,> operators document"
Luke Gleeson [Tue, 6 Jun 2023 08:45:35 +0000 (08:45 +0000)]
Merge "<,> operators document"

11 months agoSubscription Create Event Outcome Kafka Part 14/134514/20
halil.cakal [Mon, 8 May 2023 13:18:26 +0000 (14:18 +0100)]
Subscription Create Event Outcome Kafka Part

- Add subscription event outcome schema with java type for pojos
- Add subscription event outcome json for testing
- Add mapper to convert subscription response to event outcome
- Add a bean to handle subscription response outcome tasks
- Change response consumer to publish outcome for client app
- Change response timeout task to publish outcome for client app
- Change subscription persistance to read datanodes
- Add helper to extract cm handle to status mapping from data nodes
  event
- Fix code smells

Issue-ID: CPS-1507
Change-Id: I70195073490f456f014e53c1f59d1b6761d18cd4
Signed-off-by: halil.cakal <halil.cakal@est.tech>
11 months agoNCMP : forward bulk response messages to client topic 37/134737/1
raviteja.karumuri [Thu, 1 Jun 2023 09:05:44 +0000 (10:05 +0100)]
NCMP : forward bulk response messages to client topic

# Fixing the NullPointer Exception if the 'eventType' header not availabe.
# eventType header is a mandatory header but still we are doing null check for supporting old events (AsyncResponseEvent is not moved to separate kafka headers)

Issue-ID: CPS-1557
Signed-off-by: raviteja.karumuri <raviteja.karumuri@est.tech>
Change-Id: Ie7923d0e2674402fa36cb28cde966575b899cedb

11 months ago<,> operators document 24/134724/3
Rudrangi Anupriya [Wed, 31 May 2023 16:03:32 +0000 (21:33 +0530)]
<,> operators document

Issue-ID: CPS-1273
Change-Id: Ibbbbe90a28c3711ef211136cd8f8c0f265a51537
Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>
11 months agoMerge "Add <,> operators support to cps-path"
Luke Gleeson [Wed, 31 May 2023 11:07:57 +0000 (11:07 +0000)]
Merge "Add <,> operators support to cps-path"

11 months agoAdd <,> operators support to cps-path 88/133588/30
Rudrangi Anupriya [Tue, 30 May 2023 11:50:20 +0000 (17:20 +0530)]
Add <,> operators support to cps-path

Issue-ID: CPS-1273
Change-Id: I5d562463b9a49abfe0436047a637857d10596fff
Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>
11 months agoMerge "Improve error scenarios SubscriptionEventForwarder"
Priyank Maheshwari [Tue, 30 May 2023 10:24:25 +0000 (10:24 +0000)]
Merge "Improve error scenarios SubscriptionEventForwarder"

11 months agoMerge "NCMP: Update existing Batch endpoint (Moving url param into rest body)"
Toine Siebelink [Mon, 29 May 2023 08:52:22 +0000 (08:52 +0000)]
Merge "NCMP: Update existing Batch endpoint (Moving url param into rest body)"

11 months agoImprove error scenarios SubscriptionEventForwarder 88/134588/3
lukegleeson [Thu, 18 May 2023 15:40:04 +0000 (16:40 +0100)]
Improve error scenarios SubscriptionEventForwarder

- Added placeholder for emtpy outcome response when no cmhandles match targets

Issue-ID: CPS-1564
Signed-off-by: lukegleeson <luke.gleeson@est.tech>
Change-Id: Ia04bb656c639946d259da779971ab29b6e91acea

11 months agoNCMP: Update existing Batch endpoint (Moving url param into rest body) 30/134530/12
sourabh_sourabh [Tue, 9 May 2023 13:34:59 +0000 (14:34 +0100)]
NCMP: Update existing Batch endpoint (Moving url param into rest body)

- NCMP batch endpoint is updated to accept details into request payload.
- Removed unused code of previous impl.

Issue-ID: CPS-1635

Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: Ic290b750557da06b861c5a4a9bb12debc495ec2e
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
11 months agoAvc Event compliant with RFC8641 86/134586/6
mpriyank [Thu, 18 May 2023 13:56:07 +0000 (14:56 +0100)]
Avc Event compliant with RFC8641

- The event section of the payload will be now in the format of RFC8641.
- Once this event is updated , the corresponding code in the dmi plugin
  will be changed
- Schema is present in cps-ncmp codebase so we will create a release and
  then it will be used in the dmi plugin

Issue-ID: CPS-1668
Change-Id: Ibd811f318658d8b510b42575fad5d21357af9047
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
11 months agoNCMP : forward bulk response messages to client topic 00/134200/8
raviteja.karumuri [Thu, 13 Apr 2023 22:44:41 +0000 (23:44 +0100)]
NCMP : forward bulk response messages to client topic

Issue-ID: CPS-1557
Signed-off-by: raviteja.karumuri <raviteja.karumuri@est.tech>
Change-Id: I0ea040e98987f992f46105afea0a171c4031d64f

11 months agoMerge "LcmEvent to have header now"
Luke Gleeson [Mon, 22 May 2023 08:48:28 +0000 (08:48 +0000)]
Merge "LcmEvent to have header now"

11 months agoEntity ID types do not match types in database definition 68/134568/2
ToineSiebelink [Wed, 17 May 2023 15:40:44 +0000 (16:40 +0100)]
Entity ID types do not match types in database definition

As Liquibase is used, JPA does not manage database definition.
Within Liquibase changelog, only Anchor and Fragment have BIGINT (Long)
IDs. Other entities have INTEGER IDs.

- Change AnchorEntity/AnchorRepository ID type to Long
- Change YangResourceEntity/YangResourceRepository and
  ModuleReferenceRepository ID to Integer
- Modify datatypes of foreign key references in liquibase changelog to
  use correct types for dataspace_id, schema_set_id, yang_resource_id
- Update test liquibase changelog

Issue-ID: CPS-1676
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I26de6712b2e74d166fc5a48b1f8149de6a1e5199
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
11 months agoMerge "Change dmi plugin version to 1.4.0-SNAPHOST"
Luke Gleeson [Wed, 17 May 2023 13:17:25 +0000 (13:17 +0000)]
Merge "Change dmi plugin version to 1.4.0-SNAPHOST"

11 months agoChange dmi plugin version to 1.4.0-SNAPHOST 65/134565/2
halil.cakal [Tue, 16 May 2023 10:06:57 +0000 (11:06 +0100)]
Change dmi plugin version to 1.4.0-SNAPHOST

Issue-ID: CPS-1507
Change-Id: Ic812bd1be455d9df663e018383dd1d5c15359362
Signed-off-by: halil.cakal <halil.cakal@est.tech>
11 months agoMove integration test for adminService 59/134559/2
ToineSiebelink [Mon, 15 May 2023 16:32:28 +0000 (17:32 +0100)]
Move integration test for adminService

- finalized moving of admin service interation test
(just some scenarios were missing)
- deleted old test class (code coverage in RI module dropped)
- line/branch coverage now 100/100 (was 97/91)
- small refactoring; renaming and removed unnecessary check

Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: Ic683548ea5e1e4e252f257c0f1034c5cf76e498d

11 months agoLcmEvent to have header now 44/134544/7
mpriyank [Thu, 11 May 2023 12:55:55 +0000 (13:55 +0100)]
LcmEvent to have header now

- Introduce LcmEventHeader as per our agreement with stakeholders
- The version remains v1 as we just add the header and dont change
  anything in the existing event payload.
- Later we will remove the header fields from the event payload.
- tests modification and little code refractor for dmi data avc as well

Issue-ID: CPS-1695
Change-Id: Ibef1138a6d0cc7ffec50b4c201a4d3417b99e27e
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
11 months agoFixing minor compilation 64/134564/2
waqas.ikram [Mon, 15 May 2023 11:03:33 +0000 (12:03 +0100)]
Fixing minor compilation

- Complilation issues while importing project in eclipse
- Fixing CSIT

Change-Id: Idb85b08066d27a7f2a9b297d1580915df2b67541
Issue-ID: CPS-1672
Signed-off-by: waqas.ikram <waqas.ikram@est.tech>
11 months agoRemove dataspace_id column from Fragment table 91/134491/7
danielhanrahan [Thu, 4 May 2023 21:34:41 +0000 (22:34 +0100)]
Remove dataspace_id column from Fragment table

- Remove dataspace_id column from fragment table and drop associated
  index, foreign key constraint and uniqueness constaint.
- Add uniqueness constraint using only fragment xpath and anchor_id.
- Add not-null constraint to anchor_id.
- Update code and test data to remove references to dataspace_id.
- Repopulate the dataspace_id in Liquibase rollback
  (verified with manual testing of update/rollback).

Issue-ID: CPS-1677
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I467cccba25ac5d884ec790064ca58150281b7e1d

11 months agoMove integration test for moduleService 48/134548/2
ToineSiebelink [Mon, 8 May 2023 16:54:46 +0000 (17:54 +0100)]
Move integration test for moduleService

- all modules service interation test moved
- added some aditional scenarios to increase coverage
- removed unused method(s)
- remove javdoc on private methods
- some minor refactoring like replace 'var' with correct type

Issue-ID: CPS-1687
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I03ff1f3562cfe38318e8b9af81be47a1fe667da2

11 months agoSubscription Create Event Outcome Database Part 44/134444/6
halil.cakal [Wed, 3 May 2023 12:22:33 +0000 (13:22 +0100)]
Subscription Create Event Outcome Database Part

- Add mapper to convert subscription response to Yang model
- Add update operation to update Yang model into database
- Change Subscription persistence to support both save and update operations

Issue-ID: CPS-1669
Change-Id: I40cab1052ada5846945c67cac8640c393358e988
Signed-off-by: halil.cakal <halil.cakal@est.tech>
11 months agoMerge "DMI Data AVC to use kafka headers"
Toine Siebelink [Wed, 10 May 2023 14:20:09 +0000 (14:20 +0000)]
Merge "DMI Data AVC to use kafka headers"

11 months agoDMI Data AVC to use kafka headers 80/134480/14
mpriyank [Thu, 4 May 2023 10:24:29 +0000 (11:24 +0100)]
DMI Data AVC to use kafka headers

- POC done keeping AvcEvent schema in mind.
- Approach to have header schema per event schema.
- Moved the header information from AvcEvent to separate AvcEventHeader
  schema.
- Added Jsr303 annotation support for required field check

Issue-ID: CPS-1671
Change-Id: I2e4f969e8ca4f6282d1b9aa5fd52d16174a26084
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
11 months agoMove integration test (DataService) 05/134505/2
ToineSiebelink [Mon, 8 May 2023 16:07:54 +0000 (17:07 +0100)]
Move integration test (DataService)

- dropped ri module coverage to alow for move
- removed performance tests (already better tests in intergration module)
- added 'save' iue cases in new integration-test
- removed no longer used version for save single data node
(todo: remove old tests for similar functionality, will wait until new test cover same)

Issue-ID: CPS-1687
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I4c2e788e0a313938a923cd1e7e52584d739ec6f4

11 months agoMerge "Change default use of Model Loader to TRUE"
Priyank Maheshwari [Wed, 10 May 2023 11:25:24 +0000 (11:25 +0000)]
Merge "Change default use of  Model Loader to TRUE"

12 months agoMigrate cps-ri performance tests to integration-test module 18/134518/2
danielhanrahan [Tue, 9 May 2023 12:33:37 +0000 (13:33 +0100)]
Migrate cps-ri performance tests to integration-test module

- Migrate update and delete data service tests, using openroadm model
- Move module service tests to integration-test module
- Update performance test timings
- Remove all performance-related files from cps-ri
- Remove performance profile from cps-ri pom.xml

Issue-ID: CPS-1687
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Id9d864f8cab0377cb37c7967943d738748e1b6c5

12 months agoMerge "Support for Patch across multiple data nodes"
Toine Siebelink [Tue, 9 May 2023 12:44:19 +0000 (12:44 +0000)]
Merge "Support for Patch across multiple data nodes"

12 months agoSupport for Patch across multiple data nodes 19/134019/17
--global [Wed, 5 Apr 2023 11:38:08 +0000 (17:08 +0530)]
Support for Patch across multiple data nodes

- Added new method updateMultipleDataLeaves to perform Update on
  multiple data nodes
- Deprecated singular method of update data node(updateDataLeaves)
- Refactored code where singular version was used
- Updated release notes

Issue-ID: CPS-1006
Signed-off-by: <as00745003@techmahindra.com>
Change-Id: If67280e2dd3ad566de9a8217489f168415e624bc

12 months agoNCMP: Delete DatastoreType enum from cps-ncmp-rest 13/134513/1
sourabh_sourabh [Tue, 9 May 2023 09:34:02 +0000 (10:34 +0100)]
NCMP: Delete DatastoreType enum from cps-ncmp-rest

- Delete duplicate enum of datastore
- Use datastore from cps-ncmp-service

Issue-ID: CPS-1635

Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: I36484d6a327c81d81741b18a932fc6a1febfc520

12 months agoChange default use of Model Loader to TRUE 08/134508/1
emaclee [Tue, 9 May 2023 08:18:05 +0000 (09:18 +0100)]
Change default use of  Model Loader to TRUE

Issue-Id: CPS-1688
Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech>
Change-Id: I18bb5e3ad45ccdf5f3a1e88dd5a34dc830e06f4a

12 months agoMerge "Use recursive SQL to fetch descendants in CpsPath queries (CPS-1664 #4)"
Sourabh Sourabh [Mon, 8 May 2023 09:57:37 +0000 (09:57 +0000)]
Merge "Use recursive SQL to fetch descendants in CpsPath queries (CPS-1664 #4)"

12 months agoMerge "Fix and refactor query across anchors (CPS-1664 #3)"
Sourabh Sourabh [Mon, 8 May 2023 09:51:19 +0000 (09:51 +0000)]
Merge "Fix and refactor query across anchors (CPS-1664 #3)"

12 months agoMerge "[CPS] RI: Code Refactoring # Replace '[\s\S]' to '.' as it is same as '.'...
Toine Siebelink [Mon, 8 May 2023 09:26:46 +0000 (09:26 +0000)]
Merge "[CPS] RI: Code Refactoring # Replace '[\s\S]' to '.' as it is same as '.' while grouping in regex # Updated the groovy files to add 'assert' and to replace deprecated method # Removed deprecated 'storeDataNode' method."