cps.git
5 weeks agoScheduled task for Subscription Response 16/137516/13
mpriyank [Thu, 14 Mar 2024 09:11:27 +0000 (09:11 +0000)]
Scheduled task for Subscription Response

- Need to send the response back to the client in max 30 secs
- Also have the capability to send the response right away
- Testware added for the same
- Also added code to cancel the scheduled task
- Added state for cancelling the task

Issue-ID: CPS-2140
Change-Id: I3ab321d8221cd8f697c26be46d2e63d89b360923
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
6 weeks agoSet Postgres shared_buffers in docker-compose 31/137531/1
danielhanrahan [Thu, 14 Mar 2024 18:02:09 +0000 (18:02 +0000)]
Set Postgres shared_buffers in docker-compose

By default, Postgres uses value of 128MB for shared_buffers,
regardless of available memory. It is recommended to use
between 25% and 40% of memory for shared_buffers in Postgres.
Current value can be checked in SQL using 'SHOW shared_buffers'.

Issue-ID: CPS-2156
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ib20502f159f005ba941f39758555551f447d2b67

6 weeks agoConfigure Hazelcast to have 1 backup to reduce memory 17/137517/3
danielhanrahan [Fri, 15 Mar 2024 12:42:15 +0000 (12:42 +0000)]
Configure Hazelcast to have 1 backup to reduce memory

Currently Hazelcast is configured to have 3 synchronous backups
and 3 asynchronous backups. These are separate, meaning there will
be 7 copies in the cluster (1 original + 3 sync + 3 async backups).
Even if only 1 instance of CPS/NCMP is running, it will have 7 copies
in memory.

Given that CPS/NCMP is typically only deployed using 1 or 2 instances,
the settings are changed to 1 synchronous backup (same as Hazelcast's
default configuration). This change has been tested and shown to reduce
heap usage by around 100MB during 20K CM-handle registration.

Issue-ID: CPS-2146
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I4a5f6d83bc35e2c13cfb32002e38dc365da34c8e

6 weeks agoMerge "[Bug] Removing inner TaskExecutor to call DMI"
Sourabh Sourabh [Fri, 15 Mar 2024 10:13:55 +0000 (10:13 +0000)]
Merge "[Bug] Removing inner TaskExecutor to call DMI"

6 weeks agoMerge "Performance tests of alternate-id/module-set-tag lookup"
Toine Siebelink [Thu, 14 Mar 2024 15:17:57 +0000 (15:17 +0000)]
Merge "Performance tests of alternate-id/module-set-tag lookup"

6 weeks ago[Bug] Removing inner TaskExecutor to call DMI 91/137491/8
mpriyank [Fri, 8 Mar 2024 12:50:53 +0000 (12:50 +0000)]
[Bug] Removing inner TaskExecutor to call DMI

- Removing TaskExecutor from cps-ncmp-service package as anyways the
  call coming over from the cps-ncmp-rest layer is managed by a
  different thread executor(CpsNcmpTaskExecutor)
- Provided 60secs timeout value.
- Spawning new thread from a different executor might not be needed
- Removing @Async from a non spring managed method as its of no use

Issue-ID: CPS-2150
Change-Id: Ic99632983aff2c40df81421d782cf98ec600fc41
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
6 weeks agoModify lcm events to include dataProducerIdentifier and moduleSetTag (CPS-1964 3) 74/137474/8
JvD_Ericsson [Mon, 4 Mar 2024 16:23:25 +0000 (16:23 +0000)]
Modify lcm events to include dataProducerIdentifier and moduleSetTag (CPS-1964 3)

Issue-ID: CPS-2114
Change-Id: I52ae18fe849801706491742818b316df9f8cb554
Signed-off-by: JvD_Ericsson <jeff.van.dam@est.tech>
7 weeks agoModify the get cmHandle api to return dataProducerIdentifier, moduleSetTag, and alter... 65/137465/9
JvD_Ericsson [Mon, 26 Feb 2024 13:45:47 +0000 (13:45 +0000)]
Modify the get cmHandle api to return dataProducerIdentifier, moduleSetTag, and alternateId (CPS-1964 2)

Issue-ID: CPS-2113
Signed-off-by: JvD_Ericsson <jeff.van.dam@est.tech>
Change-Id: I672e5c8408e77a5fa15d6fc2f41438aca705484c

7 weeks agoProvide support for data producer identifier during registration (CPS-1964 1) 41/137341/13
JvD_Ericsson [Fri, 16 Feb 2024 09:30:45 +0000 (09:30 +0000)]
Provide support for data producer identifier during registration (CPS-1964 1)

- Added support for the dataProducerIdentifier to be added,
and updated from blank to something once
- The dataProducerIdentifier is persisted in the DB

Issue-ID: CPS-2105
Signed-off-by: JvD_Ericsson <jeff.van.dam@est.tech>
Change-Id: I1c230be8f9402154a761e8c2ba4d8a6331965b3c

7 weeks agoPerformance tests of alternate-id/module-set-tag lookup 15/137215/5
danielhanrahan [Sun, 11 Feb 2024 16:30:44 +0000 (16:30 +0000)]
Performance tests of alternate-id/module-set-tag lookup

- Add some tests of CPS get/query operations comparing
  look up of cm-handle id vs alternate-id
- Add test of querying all CM handles by module-set-tag
- Test accuracy is improved by performing warmup operations
  (reported results are faster with more warmup iterations)

Issue-ID: CPS-2087
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I4214e157ccf93f751c69b8a17d55f4185749ca90

7 weeks agoMapper to form CmNotificationSubscriptionNcmpOutEvent 83/137483/2
mpriyank [Wed, 6 Mar 2024 11:54:17 +0000 (11:54 +0000)]
Mapper to form CmNotificationSubscriptionNcmpOutEvent

- mapper to be used in the subsequent patches
- test case to support the mapping of
  CmNotificationSubscriptionNcmpOutEvent

Issue-ID: CPS-2147
Change-Id: I7f0be81941b774a27ddc343ed304a68c23796a60
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
8 weeks agoMerge "Advance DMI Stub to change module set (tags) used by cm handle"
Priyank Maheshwari [Wed, 6 Mar 2024 15:23:44 +0000 (15:23 +0000)]
Merge "Advance DMI Stub to change module set (tags) used by cm handle"

8 weeks agoMerge "Test of retry of failed module sync"
Toine Siebelink [Wed, 6 Mar 2024 14:36:40 +0000 (14:36 +0000)]
Merge "Test of retry of failed module sync"

8 weeks agoAdvance DMI Stub to change module set (tags) used by cm handle 51/137251/3
sourabh_sourabh [Mon, 19 Feb 2024 03:42:54 +0000 (03:42 +0000)]
Advance DMI Stub to change module set (tags) used by cm handle

- Renamed json response file name based on module set tag into Dmi plugin stub.
- Added in memory map to hold the information of cm handle per
  module set tag int dmi plugin stub.
- Exposed a REST api to perform diff. operations on memory map to replicate dmi plugin stub.

Issue-ID: CPS-2079
Change-Id: I7818fa4c47be9a02068976fd116a1dbc36b134ef
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
8 weeks agoMerge "Log all incoming HTTP requests to NCMP with Authorization Header"
Toine Siebelink [Wed, 6 Mar 2024 10:53:26 +0000 (10:53 +0000)]
Merge "Log all incoming HTTP requests to NCMP with Authorization Header"

8 weeks agoMerge "Update postman collection to utilize newest yang files"
Toine Siebelink [Wed, 6 Mar 2024 09:39:36 +0000 (09:39 +0000)]
Merge "Update postman collection to utilize newest yang files"

8 weeks agoMerge "Error reporting when registering cm handle with alternate id 2 - update scenario"
Toine Siebelink [Wed, 6 Mar 2024 09:33:19 +0000 (09:33 +0000)]
Merge "Error reporting when registering cm handle with alternate id 2 - update scenario"

8 weeks agoError reporting when registering cm handle with alternate id 2 - update scenario 71/137371/10
halil.cakal [Tue, 5 Mar 2024 17:15:42 +0000 (17:15 +0000)]
Error reporting when registering cm handle with alternate id 2 - update scenario

    - added error collection to cmhandle update

Issue-ID: CPS-2109
Change-Id: I751cad7c35ec53e914888760dd66eb52054da36e
Signed-off-by: halil.cakal <halil.cakal@est.tech>
8 weeks agoIntegration test of Bearer Token pass-through (CPS-2126 #5) 59/137459/4
danielhanrahan [Fri, 1 Mar 2024 10:56:19 +0000 (10:56 +0000)]
Integration test of Bearer Token pass-through (CPS-2126 #5)

This covers REST endpoints of GET, POST, PUT, PATCH, DELETE
of /ncmp/v1/ch/{cmHandleId}/data/ds/{datastoreName}
and the async REST endpoint of POST /ncmp/v1/data

It verifies that:
- bearer token is passed from NCMP to DMI
- basic auth header is not passed from NCMP to DMI

Issue-ID: CPS-2137
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ie4761a848904175a9d8cd5b917817e85f5b69813

2 months agoBump CPS to 3.4.7-SNAPSHOT 51/137451/2
mpriyank [Thu, 29 Feb 2024 17:47:47 +0000 (17:47 +0000)]
Bump CPS to 3.4.7-SNAPSHOT

- bump cps to 3.4.7-snapshot
- updated the relese notes

Issue-ID: CPS-2135
Change-Id: Ib166ebae2e3de9a727b3d0d4d21832c8858f4108
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 months agoMerge "Add container yaml for release"
Sourabh Sourabh [Thu, 29 Feb 2024 17:39:44 +0000 (17:39 +0000)]
Merge "Add container yaml for release"

2 months agoAdd container yaml for release 50/137450/1
mpriyank [Thu, 29 Feb 2024 16:45:40 +0000 (16:45 +0000)]
Add container yaml for release

- add container yaml to relase CPS 3.4.6 version

Issue-ID: CPS-2135
Change-Id: Iabf6b3de1e29c894e95bb5659c0a357dd9876991
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 months agoAdd maven stage yaml 49/137449/1
mpriyank [Thu, 29 Feb 2024 16:33:25 +0000 (16:33 +0000)]
Add maven stage yaml

- maven stage yaml to release 3.4.6 version of CPS

Issue-ID: CPS-2135
Change-Id: Icce73ceece62c889596bd757abe74844cd2f9531
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 months agoMerge "Release notes added for 3.4.6"
Sourabh Sourabh [Thu, 29 Feb 2024 16:29:58 +0000 (16:29 +0000)]
Merge "Release notes added for 3.4.6"

2 months agoAdd bearer token to NCMP async batch data passthrough (CPS-2126 #4) 48/137448/1
danielhanrahan [Thu, 29 Feb 2024 15:52:46 +0000 (15:52 +0000)]
Add bearer token to NCMP async batch data passthrough (CPS-2126 #4)

Async processing was not passing bearer token.

Issue-ID: CPS-2128
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I75da35cce555c49cca638b68f106a9daba79c24a

2 months agoRelease notes added for 3.4.6 47/137447/2
mpriyank [Thu, 29 Feb 2024 14:39:00 +0000 (14:39 +0000)]
Release notes added for 3.4.6

- added the release notes and openapi documentation

Issue-ID: CPS-2135
Change-Id: Ib224353ca5a529b6788ecb67c6432b806acdf557
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 months agoUpdate postman collection to utilize newest yang files 55/137455/1
JvD_Ericsson [Wed, 28 Feb 2024 15:02:53 +0000 (15:02 +0000)]
Update postman collection to utilize newest yang files

Issue-ID: CPS-2054
Change-Id: Id7aec68922157b1e662418de880742114b6ccc9e
Signed-off-by: JvD_Ericsson <jeff.van.dam@est.tech>
2 months agoAdd bearer token to NCMP passthrough operations (CPS-2126 #2) 41/137441/6
danielhanrahan [Wed, 28 Feb 2024 16:05:07 +0000 (16:05 +0000)]
Add bearer token to NCMP passthrough operations (CPS-2126 #2)

For NCMP resource data passthrough operations, accept an
authorization header and propagate it to outgoing DMI request
if it has a bearer token, otherwise use same behaviour as before

Issue-ID: CPS-2128
Change-Id: Ib3bf401abce4221a8b706989fb6f07618aa33fe2
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
2 months agoMerge "Remove Http Status 401 for cps core and NCMP(CPS-2126 #3)"
Sourabh Sourabh [Thu, 29 Feb 2024 11:54:45 +0000 (11:54 +0000)]
Merge "Remove Http Status 401 for cps core and NCMP(CPS-2126 #3)"

2 months agoRemove Http Status 401 for cps core and NCMP(CPS-2126 #3) 44/137444/3
mpriyank [Thu, 29 Feb 2024 09:22:36 +0000 (09:22 +0000)]
Remove Http Status 401 for cps core and NCMP(CPS-2126 #3)

- Removing Unauthorized Http Status code 401 from the repo for CPS-Core
  and NCMP

Issue-ID: CPS-2134
Change-Id: I535156fa30dd4292ad390177c217ddb3e8942d34
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 months agoDisable Spring Security and HTTP Basic Auth (CPS-2126 #1) 45/137445/2
danielhanrahan [Wed, 28 Feb 2024 07:13:50 +0000 (07:13 +0000)]
Disable Spring Security and HTTP Basic Auth (CPS-2126 #1)

This allows any authorization header to be passed in.

Issue-ID: CPS-2127
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ib1c5bd7024eed39afd1ae6e19325ed4733c853d4

2 months agoRevert Spring Boot Uplift 38/137438/2
mpriyank [Wed, 28 Feb 2024 14:10:54 +0000 (14:10 +0000)]
Revert Spring Boot Uplift

- reverting back to spring 3.1.2
- manual changes to revert as other apps not fully ready for the
  delivery

Issue-ID: CPS-2133
Change-Id: I670b67916fa4a5d4e5bced6548b7fcd5ee12e855
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 months agoLog all incoming HTTP requests to NCMP with Authorization Header 34/137434/7
sourabh_sourabh [Wed, 28 Feb 2024 09:06:24 +0000 (09:06 +0000)]
Log all incoming HTTP requests to NCMP with Authorization Header

- Enabled configuration to log NCMP request header and payload.
- Enabled the DEBUG level log into application.yml.

Issue-ID: CPS-2131

Change-Id: I458b1148894b8f460b013f0fa93032231ac47938
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2 months agoBump CPS to 3.4.6-SNAPSHOT 29/137429/1
mpriyank [Tue, 27 Feb 2024 14:04:38 +0000 (14:04 +0000)]
Bump CPS to 3.4.6-SNAPSHOT

- bumping CPS to 3.4.6-SNAPSHOT
- updated the release notes

Issue-ID: CPS-2124
Change-Id: Id28af9340da9c1ce118b2122e97b8717860c7932
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 months agoAdd maven container yaml 28/137428/1
mpriyank [Tue, 27 Feb 2024 13:39:14 +0000 (13:39 +0000)]
Add maven container yaml

- maven container yaml for the cps 3.4.5 release

Issue-ID: CPS-2124
Change-Id: I185562247b39b7eed0f489be2f1cabd432a53a9b
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 months agoAdd maven stage yaml 27/137427/1
mpriyank [Tue, 27 Feb 2024 13:18:09 +0000 (13:18 +0000)]
Add maven stage yaml

- maven stage yaml for 3.4.5 version of CPS

Issue-ID: CPS-2124
Change-Id: I61aadcf6400ba47ee511b4da1e324909473d6860
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 months agoTest of retry of failed module sync 68/137368/2
danielhanrahan [Tue, 20 Feb 2024 14:50:14 +0000 (14:50 +0000)]
Test of retry of failed module sync

A new integration test verifies that LOCKED handles that failed
module sync will be retried after a few minutes.

Issue-ID: CPS-2033
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: If194daeb315090f2710a017270875b6301f7140a

2 months agoRelease notes update 26/137426/1
mpriyank [Tue, 27 Feb 2024 12:21:53 +0000 (12:21 +0000)]
Release notes update

- Added the spring boot uplift ticket

Issue-ID: CPS-2124
Change-Id: I55b1e246d5bce92ed0a34deb19afe4903db4a7bc
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 months agoupdate link to upper-constraints.txt 98/137398/1
thmsdt [Tue, 27 Feb 2024 09:47:03 +0000 (10:47 +0100)]
update link to upper-constraints.txt

Issue-ID: DOC-823

Change-Id: I797b8e1058e8afe511bb9b140c75c887728ba311
Signed-off-by: thmsdt <thomas.kulik@telekom.de>
2 months agoMerge "Populate cmSubscription cache"
Priyank Maheshwari [Mon, 26 Feb 2024 17:06:21 +0000 (17:06 +0000)]
Merge "Populate cmSubscription cache"

2 months agoMerge "Create shared postman collections"
Sourabh Sourabh [Mon, 26 Feb 2024 16:58:22 +0000 (16:58 +0000)]
Merge "Create shared postman collections"

2 months agoMerge "Add passthrough read operation (synchronous) to DMI stub"
Lee Anjella Macabuhay [Mon, 26 Feb 2024 16:57:42 +0000 (16:57 +0000)]
Merge "Add passthrough read operation (synchronous) to DMI stub"

2 months agoPopulate cmSubscription cache 76/137076/15
emaclee [Wed, 24 Jan 2024 15:22:58 +0000 (15:22 +0000)]
Populate cmSubscription cache

       - Create subscription cache handler
       - Add unit tests

Issue-ID: CPS-2025
Change-Id: I54d505369a8429d6f6a87a2af0b169f9530622c3
Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech>
2 months agoMerge "Get Subscription Delta"
Priyank Maheshwari [Mon, 26 Feb 2024 16:10:23 +0000 (16:10 +0000)]
Merge "Get Subscription Delta"

2 months agoMerge "Limit max connections in DMI stub"
Priyank Maheshwari [Mon, 26 Feb 2024 11:56:23 +0000 (11:56 +0000)]
Merge "Limit max connections in DMI stub"

2 months agoMerge "Use default values for timers in docker-compose"
Priyank Maheshwari [Mon, 26 Feb 2024 11:54:05 +0000 (11:54 +0000)]
Merge "Use default values for timers in docker-compose"

2 months agoAdd passthrough read operation (synchronous) to DMI stub 67/137367/2
danielhanrahan [Wed, 29 Nov 2023 18:51:16 +0000 (18:51 +0000)]
Add passthrough read operation (synchronous) to DMI stub

- Implement DMI stub endpoint needed for for NCMP passthrough data
  operation, returning a fixed JSON response and adding delay.
    POST /dmi/v1/ch/{cmHandleId}/data/ds/{datastoreName}
- Add script to measure average overhead for many requests

Test results indicate the NCMP adds up to 150ms per request.

Issue-ID: CPS-2099
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I2aba8285f8a52f3570fc1699a8687caffa9d9c77

2 months agoMerge "Remove subscriptionModelLoader flag"
Lee Anjella Macabuhay [Mon, 26 Feb 2024 10:37:17 +0000 (10:37 +0000)]
Merge "Remove subscriptionModelLoader flag"

2 months agoMerge "Uplift Spring Boot to 3.2.2"
Priyank Maheshwari [Mon, 26 Feb 2024 10:32:19 +0000 (10:32 +0000)]
Merge "Uplift Spring Boot to 3.2.2"

2 months agoMerge "Add health check endpoint to DMI stub"
Priyank Maheshwari [Mon, 26 Feb 2024 10:25:13 +0000 (10:25 +0000)]
Merge "Add health check endpoint to DMI stub"

2 months agoRemove subscriptionModelLoader flag 54/137354/2
mpriyank [Fri, 23 Feb 2024 10:35:37 +0000 (10:35 +0000)]
Remove subscriptionModelLoader flag

- removed the flag as the functionality around model loader is stable
  now
- removed the config parameter from the application yaml file as well

Issue-ID: CPS-2112
Change-Id: Ie4e3317c0689615b900dcd9bbf1896bc3a5f37b3
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 months agoBump CPS to 3.4.5-SNAPSHOT 59/137359/1
danielhanrahan [Fri, 23 Feb 2024 15:27:17 +0000 (15:27 +0000)]
Bump CPS to 3.4.5-SNAPSHOT

- bump CPS to 3.4.5-SNAPSHOT and update the docs

Issue-ID: CPS-2120
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ia2ea81b40ec19b87d3b25359fbb25ffdcc3285dd

2 months agoAdd container yaml for CPS 3.4.4 58/137358/1
danielhanrahan [Fri, 23 Feb 2024 14:52:06 +0000 (14:52 +0000)]
Add container yaml for CPS 3.4.4

- added container yaml to release the docker images for CPS 3.4.4

Issue-ID: CPS-2120
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I9786dd5a3fc4d1597e24304236c427c0658c44b1

2 months agoAdd maven stage yaml for CPS 3.4.4 57/137357/1
danielhanrahan [Fri, 23 Feb 2024 14:06:20 +0000 (14:06 +0000)]
Add maven stage yaml for CPS 3.4.4

- maven stage yaml to release 3.4.4

Issue-ID: CPS-2120
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Iad423641f9fb214d195a6376d5c0496dff52bc79

2 months agoUpdate release notes for CPS 3.4.4 56/137356/1
danielhanrahan [Fri, 23 Feb 2024 13:27:37 +0000 (13:27 +0000)]
Update release notes for CPS 3.4.4

- updated release notes for 3.4.4
- no openapi changes

Issue-ID: CPS-2120
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I1634ed15b058e50c205dcf15ad59c21cce0e4972

2 months agoCreate shared postman collections 21/137221/5
JvD_Ericsson [Thu, 15 Feb 2024 11:34:36 +0000 (11:34 +0000)]
Create shared postman collections

Issue-ID: CPS-2054
Signed-off-by: JvD_Ericsson <jeff.van.dam@est.tech>
Change-Id: I0a7e941bc4accfe00edaf7157df5a8ebabadb187

2 months agoLimit max connections in DMI stub 55/137355/2
danielhanrahan [Fri, 23 Feb 2024 10:19:23 +0000 (10:19 +0000)]
Limit max connections in DMI stub

DMI plugin in a real deployment could only handle a limited number
of parallel connections. From CPS-1968 of testing in a realistic
deployment, 25 max connections is chosen as a reasonable default.
This allows us to simulate rate limiting with DMI stub.

Functionality has been verified by changing the max connections and
registering 5000 CM-handles. It takes:
- 120 seconds with 8 max DMI connections
- 80 seconds with 16 max DMI connections

Issue-ID: CPS-2117
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I6aaf43f721d6f2e6dcff45929fc9699e26980cdb

2 months agoGet Subscription Delta 72/137172/10
seanbeirne [Tue, 6 Feb 2024 10:37:02 +0000 (10:37 +0000)]
Get Subscription Delta

-Added @AllArgsConstructor to improve readability

Issue-ID: CPS-1953
Change-Id: Id457561edb8ce8c00c9c93a1453ec37da5174b38
Signed-off-by: seanbeirne <sean.beirne@est.tech>
2 months agoMerge "CmNotificationSubscription event from Dmi Plugin"
Lee Anjella Macabuhay [Fri, 23 Feb 2024 11:09:00 +0000 (11:09 +0000)]
Merge "CmNotificationSubscription event from Dmi Plugin"

2 months agoUse default values for timers in docker-compose 52/137352/1
danielhanrahan [Fri, 23 Feb 2024 10:27:00 +0000 (10:27 +0000)]
Use default values for timers in docker-compose

docker-compose.yml is overriding NCMP timers, e.g. module sync timer
is overridden from default of 5 seconds to 30 seconds, which
negatively affects performance testing.

Issue-ID: CPS-2119
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ic664169ddeed9cce59416dde50ec919cfcea522a

2 months agoAdd health check endpoint to DMI stub 51/137351/1
danielhanrahan [Fri, 23 Feb 2024 10:16:43 +0000 (10:16 +0000)]
Add health check endpoint to DMI stub

DMI stub health check endpoint /actuator/health is needed
for NCMP to set trust level. Can be verified using:
  curl "http://localhost:8784/actuator/health"
which reports:
  {"status":"UP"}

Issue-ID: CPS-2118
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I4477a389a585280d36028b72d280028705e0f8f2

2 months agoError reporting when registering cm handle with alternate id 39/137239/9
halil.cakal [Mon, 19 Feb 2024 21:37:25 +0000 (21:37 +0000)]
Error reporting when registering cm handle with alternate id

- refactored registration code all use cases; 1 method for each action
- introduce new cps error code; 111

- TODO: error reporting registration, UPDATE

Issue-ID: CPS-2100

Change-Id: I5049777ee4e08fdc94aa1db09e668e952ed8e1c3
Signed-off-by: halil.cakal <halil.cakal@est.tech>
2 months agoMerge "NCMP to DMI CmSubscription Event"
Lee Anjella Macabuhay [Fri, 23 Feb 2024 10:24:09 +0000 (10:24 +0000)]
Merge "NCMP to DMI CmSubscription Event"

2 months agoUplift Spring Boot to 3.2.2 32/137332/4
waqas.ikram [Wed, 21 Feb 2024 11:31:00 +0000 (11:31 +0000)]
Uplift Spring Boot to 3.2.2

- Upgraded hypersistence-utils-hibernate-60 to hypersistence-utils-hibernate-63
- Introduced @Qualifier annotations where needed

Issue-ID: CPS-2101
Change-Id: I8303592ed91584deb6ca5cd4e49800210b6ea989
Signed-off-by: waqas.ikram <waqas.ikram@est.tech>
2 months agoMerge "NCMP to Client Cm Subscription Event"
Lee Anjella Macabuhay [Thu, 22 Feb 2024 15:47:20 +0000 (15:47 +0000)]
Merge "NCMP to Client Cm Subscription Event"

2 months agoIntroduce class for Module Delta during module sync 40/137340/4
danielhanrahan [Thu, 22 Feb 2024 10:20:17 +0000 (10:20 +0000)]
Introduce class for Module Delta during module sync

Removed ImmutableTriple<String, Map<String, String>, Collection<ModuleReference>>
and replaced it with a ModuleDelta class to hold results.

Split syncAndCreateOrUpgradeSchemaSetAndAnchor into methods
syncAndCreateSchemaSetAndAnchor and syncAndUpgradeSchemaSet

Issue-ID: CPS-2027
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I95462ab55dcae7d98b9e2671fc278c4ded45f3fc

2 months agoImprove performance of moduleSetTag lookup 47/137347/2
danielhanrahan [Wed, 21 Feb 2024 16:37:19 +0000 (16:37 +0000)]
Improve performance of moduleSetTag lookup

- Using absolute cps-path /dmi-registry/cm-handles is faster
  than descendant cps-path //cm-handles due to CPS-2087.
- Converting DataNodes to YangModelCmHandle allows checking if
  the handles are READY without needing to send additional DB
  queries via cmHandleQueries::cmHandleHasState.

Issue-ID: CPS-2027
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ied7e884b0f779f394854603a7f8b186d09eb5be8

2 months agoMerge "Refactor YangDataConverter.convertCmHandleToYangModel"
Sourabh Sourabh [Thu, 22 Feb 2024 10:51:18 +0000 (10:51 +0000)]
Merge "Refactor YangDataConverter.convertCmHandleToYangModel"

2 months agoRefactor check for no new modules in Module Sync 46/137346/1
danielhanrahan [Wed, 21 Feb 2024 19:50:32 +0000 (19:50 +0000)]
Refactor check for no new modules in Module Sync

This change moves the check for no new module references into
DmiModelOperations::getNewYangResourcesFromDmi
(The method is only used by ModuleSyncService)

Issue-ID: CPS-2027
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I3cc65e7a327148ebe01b98b61dc64b70872e2cf5

2 months agoRefactor YangDataConverter.convertCmHandleToYangModel 45/137345/1
danielhanrahan [Wed, 21 Feb 2024 17:31:44 +0000 (17:31 +0000)]
Refactor YangDataConverter.convertCmHandleToYangModel

This method redundantly takes cmHandleId as a parameter, even though
the DataNode representing a CM-handle is guaranteed to have the ID
in a leaf called 'id'.

Issue-ID: CPS-245
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ib10097b6e13f7188fd933d6f6913110ec664273e

2 months agoImprove unit test for regsitration, upgrade 39/137339/2
ToineSiebelink [Wed, 21 Feb 2024 15:42:45 +0000 (15:42 +0000)]
Improve unit test for regsitration, upgrade

- verify schema set is created
- add test case for upgarde to same tag

Issue-ID: CPS-2027
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: Id853f477d2a90a86d13828a27b9269c62e1d9462

2 months agoMerge "adding isUniqueSubscriptionId method"
Priyank Maheshwari [Wed, 21 Feb 2024 15:10:02 +0000 (15:10 +0000)]
Merge "adding isUniqueSubscriptionId method"

2 months agoUpgrade yang modules using module set tag 52/137252/2
danielhanrahan [Wed, 21 Feb 2024 12:38:05 +0000 (12:38 +0000)]
Upgrade yang modules using module set tag

- New algorithm is implemented as per the pseudo code -> https://wiki.onap.org/display/DW/CPS-1733%3A+Upgrade+YANG+Schema-Set+for+CM+Handle+Using+a+Module+Set+Tag

Issue-ID: CPS-2027
Change-Id: I78fc1c8f927de1a1b814c3c1c62351dda58f2139
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
2 months agoNCMP to Client Cm Subscription Event 33/137333/1
mpriyank [Wed, 21 Feb 2024 12:21:18 +0000 (12:21 +0000)]
NCMP to Client Cm Subscription Event

- CmNotificationSubscriptionNcmpOut event to inform the client about the
  subscription he requested
- Testware for the same
- Code to provide the data to this producer will be part of other
  patches

Issue-ID: CPS-2108
Change-Id: Ic302e798a20da54dd3c56d2cb792b800446cc2d3
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 months agoFix handling of blank moduleSetTag 28/137328/1
danielhanrahan [Wed, 21 Feb 2024 10:02:19 +0000 (10:02 +0000)]
Fix handling of blank moduleSetTag

Handling of blank moduleSetTag is very inconsistent.
Sometimes it will be returned as null, '' or 'not-specified'.
Specifically in the case of MODULE_UPGRADE_FAILED, on the retry,
a blank moduleSetTag will be returned as 'not-specified', leading
to incorrect upgrade.

Issue-ID: CPS-2027
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I3f4290dc5659c386c7b2b924325da2560d00d73b

2 months agoAdd moduleSetTag when getting a CM handle 86/137286/3
danielhanrahan [Mon, 19 Feb 2024 18:39:37 +0000 (18:39 +0000)]
Add moduleSetTag when getting a CM handle

Test changes:
- Check moduleSetTag (and alternateId) in unit test of get CM handle.
- Verify moduleSetTag is updated in tests of CM handle upgrade.

Code changes:
- Set moduleSetTag when converting a YangModelCmHandle to
  NcmpServiceCmHandle.
- Set moduleSetTag in YangModelCmHandle copy constructor.
- Minor refactor using StringUtils::isBlank.

Issue-ID: CPS-2027
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I6a9a92aa58d15c6ecf5a6bb21aa5c9d6ec8dc817

2 months agoRemove alternate ID caches 94/137294/3
leventecsanyi [Tue, 20 Feb 2024 12:15:11 +0000 (13:15 +0100)]
Remove alternate ID caches

    - removed alternate id cache related classes and updated deployment.rst

Issue-ID: CPS-2107
Change-Id: I567af7b2f44107fceffaef370205b00a5cf79522
Signed-off-by: leventecsanyi <levente.csanyi@est.tech>
2 months agoCmNotificationSubscription event from Dmi Plugin 23/137323/1
mpriyank [Tue, 20 Feb 2024 13:32:23 +0000 (13:32 +0000)]
CmNotificationSubscription event from Dmi Plugin

- Added infrastructure related code to read event from the dmi plugin
- Just consuming and reading the message.
- Core logic to introduced to be part of subsequent patches

Issue-ID: CPS-2106
Change-Id: Ic58f4f80b3dded32df95daecac42315fa9212f58
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 months agoMerge "Use DB for checking Alternate IDs"
Toine Siebelink [Mon, 19 Feb 2024 17:29:57 +0000 (17:29 +0000)]
Merge "Use DB for checking Alternate IDs"

2 months agoadding isUniqueSubscriptionId method 82/137282/1
emaclee [Mon, 19 Feb 2024 12:56:46 +0000 (12:56 +0000)]
adding isUniqueSubscriptionId method

Issue-ID: CPS-2044
Change-Id: I26b209ead80add7e60b4de9257a9f5c326ddea6f
Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech>
2 months agoNCMP to DMI CmSubscription Event 79/137279/2
mpriyank [Mon, 19 Feb 2024 11:30:01 +0000 (11:30 +0000)]
NCMP to DMI CmSubscription Event

- Ncmp to have a producer which converts
  CmNotificationSubscriptionDmiInEvent to Cloud event and then publishes
  it
- For now , using the old topic prefix as topic name. Once topic names
  are finalized then they can be configured
- Testware for the publishing code

Issue-ID: CPS-2103
Change-Id: I83b6ce5b6647e92313839b25492cb076e528ce3e
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 months agoTests of CM-handle module upgrade & moduleSetTag 76/137276/2
danielhanrahan [Mon, 19 Feb 2024 09:55:37 +0000 (09:55 +0000)]
Tests of CM-handle module upgrade & moduleSetTag

Integration tests covering scenarios of the creation and upgrade of
CM-handles with:
- no moduleSetTag
- new moduleSetTag
- existing moduleSetTag
Failing tests are marked @Ignore.

Issue-ID: CPS-2102
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Id5567e08a13d8c26dc7b6458f0af8449bef6f082

2 months agoUse DB for checking Alternate IDs 11/137211/8
halil.cakal [Mon, 12 Feb 2024 11:41:00 +0000 (11:41 +0000)]
Use DB for checking Alternate IDs

- do NOT use hazelcast cache
- introduced AlternateIcChecker
- use checks during Inital registration

- TODO: remvoe Hazalcast Cache (once perf test are OK)
- TODO: use checks during UPDATE registration
- TODO: error reporting during registration

Issue-Id: CPS-2049

Change-Id: I0adcac52a7a49e26301758eafc4684152ddfcaf5
Signed-off-by: halil.cakal <halil.cakal@est.tech>
2 months agoAdd NCMP to integration-test module 14/137214/4
danielhanrahan [Tue, 13 Feb 2024 11:12:29 +0000 (11:12 +0000)]
Add NCMP to integration-test module

- Add NCMP to integration test base class
- Use MockRestServiceServer to mock DMI responses in groovy tests
- Added tests for CM-handle registration to READY and LOCKED states
- Add test of CM-handle deregistration

Issue-ID: CPS-2033
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I89f9f8aab2bba9d4eb5b3136a2feb238e7cf1b06

2 months agoMerge "Mapper to form event for DMI Plugin"
Lee Anjella Macabuhay [Wed, 14 Feb 2024 14:26:52 +0000 (14:26 +0000)]
Merge "Mapper to form event for DMI Plugin"

2 months agoMerge "Clean up of integration-test base classes"
Lee Anjella Macabuhay [Wed, 14 Feb 2024 11:48:45 +0000 (11:48 +0000)]
Merge "Clean up of integration-test base classes"

2 months agoMerge "Faster absolute xpath query using parent path lookup"
Lee Anjella Macabuhay [Wed, 14 Feb 2024 11:30:37 +0000 (11:30 +0000)]
Merge "Faster absolute xpath query using parent path lookup"

2 months agoClean up of integration-test base classes 12/137212/4
danielhanrahan [Tue, 13 Feb 2024 11:12:29 +0000 (11:12 +0000)]
Clean up of integration-test base classes

- Remove TestConfig and instead use auto-configuration
- Add needed application.yml config
- Refactor helper methods for creating JSON

Issue-ID: CPS-2090
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I4f618a1d4a83e9c75e468928c257b817d449c22f

2 months agoMerge "Remove liquibase test changelog"
Lee Anjella Macabuhay [Wed, 14 Feb 2024 11:13:01 +0000 (11:13 +0000)]
Merge "Remove liquibase test changelog"

2 months agoFaster absolute xpath query using parent path lookup 13/137213/3
danielhanrahan [Mon, 20 Nov 2023 16:37:46 +0000 (16:37 +0000)]
Faster absolute xpath query using parent path lookup

Issue-ID: CPS-2087
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I62b452f34f08a32a6d80671c7af560fbceec1285

2 months agoMerge "Update release notes for Extending Module API retrieval"
Toine Siebelink [Wed, 14 Feb 2024 08:43:58 +0000 (08:43 +0000)]
Merge "Update release notes for Extending Module API retrieval"

2 months agoRemove liquibase test changelog 09/137209/2
danielhanrahan [Tue, 13 Feb 2024 09:53:07 +0000 (09:53 +0000)]
Remove liquibase test changelog

Complete liquibase changelog takes under 1 second to run,
so having separate changelog for integration tests offers
no benefit.

Issue-ID: CPS-2090
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ib0e928f35d401b91c23dfcb5dc82c77778a044dc

2 months agoMapper to form event for DMI Plugin 40/137140/5
mpriyank [Tue, 6 Feb 2024 13:39:29 +0000 (13:39 +0000)]
Mapper to form event for DMI Plugin

- cmSubscriptionPredicate to CmNotificationSubscriptionDmiInEvent mapper code and
  testware.
- Based on the delta we find , we will be sending the events to the dmi
  plugins. Finding delta code will be separate patchset

Issue-ID: CPS-2060
Change-Id: If1e15679b72c39352e3cf166651226df7e207ce0
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 months agoUpdate release notes for Extending Module API retrieval 87/137187/2
halil.cakal [Thu, 8 Feb 2024 12:15:48 +0000 (12:15 +0000)]
Update release notes for Extending Module API retrieval

- Add the user story: CPS-1135 for CPS version 3.4.3

Issue-ID: CPS-1135

Change-Id: I9e9cdc5f648d38eb4dbd76cc98a884fed03a28b2
Signed-off-by: halil.cakal <halil.cakal@est.tech>
2 months agoPrefix with CmNotificationSubscription refactoring 10/137210/1
mpriyank [Tue, 13 Feb 2024 11:50:23 +0000 (11:50 +0000)]
Prefix with CmNotificationSubscription refactoring

- event schemas are now suffixed with CmNotificationSubscription
- testware updated appropriately

Issue-ID: CPS-2091
Change-Id: I9d0c4bdfaf7ea94812d35b071824d0fb083aa845
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 months agoMerge "Update cm-data-subscriptions model"
Lee Anjella Macabuhay [Tue, 13 Feb 2024 10:54:02 +0000 (10:54 +0000)]
Merge "Update cm-data-subscriptions model"

2 months agoMerge "Add a configurable delay to the DMI stub"
Sourabh Sourabh [Tue, 13 Feb 2024 10:08:11 +0000 (10:08 +0000)]
Merge "Add a configurable delay to the DMI stub"

2 months agoMerge "Fix test failure by ordering leaf-lists"
Priyank Maheshwari [Tue, 13 Feb 2024 09:47:35 +0000 (09:47 +0000)]
Merge "Fix test failure by ordering leaf-lists"

2 months agoMerge "Create schemas for notification service in cps-core"
Priyank Maheshwari [Tue, 13 Feb 2024 09:24:00 +0000 (09:24 +0000)]
Merge "Create schemas for notification service in cps-core"

2 months agoUpdate cm-data-subscriptions model 05/137205/3
mpriyank [Mon, 12 Feb 2024 16:16:36 +0000 (16:16 +0000)]
Update cm-data-subscriptions model

- rename leaf-list subscribers to subscriptionIds
- code using the same also updated
- not creating a new version of model as nobody is using it at the
  moment

Issue-ID: CPS-2088
Change-Id: I43d679047bdfc665b56f262757acc4bbaba2733f
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>