cps.git
2 years agoMerge "Handle partial failure"
Toine Siebelink [Tue, 13 Sep 2022 09:07:54 +0000 (09:07 +0000)]
Merge "Handle partial failure"

2 years agoHandle partial failure 52/130852/3
mpriyank [Tue, 6 Sep 2022 17:29:34 +0000 (18:29 +0100)]
Handle partial failure

- Removing the transaction boundaries as it was getting rollbacked on
  partial failures
- Handled adding the elements in batch and if it fails try them
  individually
- Refactored code a bit and when there is partial failure we try one
  more time in sequence and even if there are failures we collect the
  failures

Issue-ID: CPS-1232
Issue-ID: CPS-1126
Change-Id: I7824c9f37f80cbaeedd5dc06d598ca0e3a69c59b
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 years agoUpdate doc and release notes after endpoint merge 01/130801/1
bmiklos [Wed, 7 Sep 2022 14:11:57 +0000 (16:11 +0200)]
Update doc and release notes after endpoint merge

- Update generated swagger files according to changes
- Add release notes for the parent ticket

Issue-ID: CPS-1001
Issue-ID: CPS-1256
Change-Id: I09aaec26971ae89f4588bfc80d2b80cc4778dfc8
Signed-off-by: bmiklos <miklos.baranyak@est.tech>
2 years agoHandle invalid operations on merged datastore endpoint 51/130751/4
bmiklos [Mon, 5 Sep 2022 16:25:46 +0000 (18:25 +0200)]
Handle invalid operations on merged datastore endpoint

- Move the other endpoints under the same thats handling the get
- Add exceptions to handle invalid cases
- Add new tests for testing the invalid cases

Issue-ID: CPS-1001
Issue-ID: CPS-1219
Change-Id: I26fd57b5921f976cde7089b4cf1e8c0e5d8dc43b
Signed-off-by: bmiklos <miklos.baranyak@est.tech>
2 years agoMerge "Code to Interface : InventoryPersistence"
Toine Siebelink [Tue, 6 Sep 2022 15:34:41 +0000 (15:34 +0000)]
Merge "Code to Interface : InventoryPersistence"

2 years agoMinor code change from POC || 50/130750/1
sourabh_sourabh [Tue, 6 Sep 2022 09:22:49 +0000 (10:22 +0100)]
Minor code change from POC ||

-Explicitly use YangModelCmHandle#id to computing equals and hashcode

Issue-ID: CPS-1126
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: Ib5332843c71522a85640f225298b680d4cd29557

2 years agoXpath to NodeId invalid 53/130653/8
Gandhapu, Yashwanth [Fri, 2 Sep 2022 05:31:31 +0000 (11:01 +0530)]
Xpath to NodeId invalid

Issue-ID: CPS-1244
Signed-off-by: Gandhapu, Yashwanth <gandhapu.yashwanth@wipro.com>
Change-Id: Ib091de1247cb0bb48d3157bf2c5becfc9189b773

2 years agoCode to Interface : InventoryPersistence 37/130737/1
mpriyank [Mon, 5 Sep 2022 13:33:02 +0000 (14:33 +0100)]
Code to Interface : InventoryPersistence

- Extracted interface for InventoryPersistence using automatic
  extraction of IDE.
- We call the InventoryPersistence to InventoryPersistenceImpl
- Used the correct type in the test

Issue-ID: CPS-1241
Change-Id: Ieceaec69c1063d7762abc7d8e389d36011a93860
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 years agoPerformance Improvement: Fix Insert Yang Resource IDs (Schemset) 25/130725/2
ToineSiebelink [Mon, 5 Sep 2022 11:08:09 +0000 (12:08 +0100)]
Performance Improvement: Fix Insert Yang Resource IDs (Schemset)

- Add robustness to handle insertion/get of empty collection
- Fix business logic to pass down ALL module references during module Sync
- Update Java doc etc to clarify ALL module references are needed (not just new)

Issue-ID: CPS-1246
Issue-ID: CPS-1126
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: Ic8ff4bdfef646e98ef61a6732c6d5ecb4b762e29

2 years agoMerge "Update documentation for NCMP inventory openAPI"
Toine Siebelink [Mon, 5 Sep 2022 08:13:57 +0000 (08:13 +0000)]
Merge "Update documentation for NCMP inventory openAPI"

2 years agoUpdate documentation for NCMP inventory openAPI 86/130686/7
emaclee [Fri, 2 Sep 2022 13:40:17 +0000 (14:40 +0100)]
Update documentation for NCMP inventory openAPI

- added JIRA (CPS-1136) to release notes
- updated RTD copies of open api documentation for both cps-rest and cps-ncmp-rest
   - files were updated by copying the generated sources as outlined on https://wiki.onap.org/display/DW/CPS+Release+Process

Issue-ID: CPS-1136
Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech>
Change-Id: I929e0e774d8a42ad943eaf8a6ccf4e8891da4b77

2 years agoLogging: get resource ids 75/130675/3
mpriyank [Fri, 2 Sep 2022 12:48:54 +0000 (13:48 +0100)]
Logging: get resource ids

- Added logging while fetching resource ids using Module references.
- Minor refactoring related to stream and forEach
- Fixed minor sonar violations

Issue-ID: CPS-1246
Change-Id: I631d7acd3f29e6bdc2995726e17f40bd7de118b1
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 years agoPerformance Improvement: Use save batches of cmhandles 77/130677/2
sourabh_sourabh [Fri, 2 Sep 2022 08:40:35 +0000 (09:40 +0100)]
Performance Improvement: Use save batches of cmhandles

-Used cm handle batch to persist from state handler.

Issue-ID: CPS-1230
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: I68b7fde7dc85818b818f1af588344c26b549d87b

2 years agoMerge "Implement merging all ncmp datastore endpoints into one"
Toine Siebelink [Fri, 2 Sep 2022 12:00:13 +0000 (12:00 +0000)]
Merge "Implement merging all ncmp datastore endpoints into one"

2 years agoMerge "Code to Interface: CmHandleQueries"
Joseph Keenan [Fri, 2 Sep 2022 11:27:39 +0000 (11:27 +0000)]
Merge "Code to Interface: CmHandleQueries"

2 years agoCode to Interface: CmHandleQueries 43/130643/2
mpriyank [Thu, 1 Sep 2022 15:34:02 +0000 (16:34 +0100)]
Code to Interface: CmHandleQueries

- Extracting the interface CmHandleQueries using IDE intelligence
- Used intreface when mocking in the tests

Issue-ID: CPS-1240
Change-Id: I5f537eaac992c68b65468e344830a04f1dda7e45
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 years agoMerge "Adding config for RTD"
Toine Siebelink [Thu, 1 Sep 2022 16:38:12 +0000 (16:38 +0000)]
Merge "Adding config for RTD"

2 years agoImplement merging all ncmp datastore endpoints into one 46/130246/35
bmiklos [Thu, 25 Aug 2022 16:28:16 +0000 (18:28 +0200)]
Implement merging all ncmp datastore endpoints into one

- Merging all endpoints
under /v1/ch/{cm-handle}/data/ds/ncmp-datastore:*
to /v1/ch/{cm-handle}/data/ds/{ncmp-datastore-name}

- Implementing missing tests from parent
- Introducing abstract class to keep the common code and just pass in
  the supplier to be executed in sync or async manner
- Removed the existing get endpoints for passthrough-running,
  passthrough-operational and operational and merged them into a common
  get endpoint

Issue-ID: CPS-1178
Issue-ID: CPS-1001
Change-Id: I6956c81d5acfa8fb11217bcc16cb795b62070fa3
Signed-off-by: bmiklos <miklos.baranyak@est.tech>
2 years agoMerge "Get all cm handles by DMI plugin Identifier"
Sourabh Sourabh [Thu, 1 Sep 2022 14:40:03 +0000 (14:40 +0000)]
Merge "Get all cm handles by DMI plugin Identifier"

2 years agoAdding config for RTD 22/130622/5
JosephKeenan [Thu, 1 Sep 2022 10:39:47 +0000 (11:39 +0100)]
Adding config for RTD

-added config to files related to https://gerrit.onap.org/r/c/cps/+/130246/29

Issue-ID: CPS-1237
Signed-off-by: JosephKeenan <joseph.keenan@est.tech>
Change-Id: Ia2fe7d2ecf3bb33608139c5fbf1328dc211f5629

2 years agoMerge "Performance Improvement: Enhance state handler"
Sourabh Sourabh [Thu, 1 Sep 2022 13:46:53 +0000 (13:46 +0000)]
Merge "Performance Improvement: Enhance state handler"

2 years agoPerformance Improvement: Enhance state handler 37/130537/7
mpriyank [Tue, 30 Aug 2022 22:23:53 +0000 (23:23 +0100)]
Performance Improvement: Enhance state handler

- Introduced batch handling capability to state handler
- Refactored methods in CompositeStateUtils
- Renamed saveCmHandleStates to saveCmHandleStateBatch
- Decoupled processing of events in bulk
- Test scenarios for the new functionality

Issue-ID: CPS-1231
Issue-ID: CPS-1126
Change-Id: Ifacdeb7bbed14712ecf4f5e2a4d9b324bae278d8
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 years agoGet all cm handles by DMI plugin Identifier 84/130384/16
emaclee [Fri, 19 Aug 2022 08:26:39 +0000 (09:26 +0100)]
Get all cm handles by DMI plugin Identifier

- api added to get cm handles by dmi plugin identifier
- response object refactored from RestOutputCmHandle to collection of
  Strings (cm handle ids)
- added public and private methods in CmHandleQueries to get cm handle ID by dmi plugin
- added unit tests including test to show that there are no duplicates
  on response

Issue-ID: CPS-1136
Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech>
Change-Id: Ia3bdc16172a90ad3a3f9ae11cddcad1352188726

2 years agoMerge "Fixing more RTD warnings"
Sourabh Sourabh [Thu, 1 Sep 2022 09:00:07 +0000 (09:00 +0000)]
Merge "Fixing more RTD warnings"

2 years agoPerformance Improvement: Watchdog Parallel execution with configuration 34/130534/3
sourabh_sourabh [Wed, 31 Aug 2022 10:58:09 +0000 (11:58 +0100)]
Performance Improvement: Watchdog Parallel execution with configuration

- Introduced AsyncSyncExecutor to get task and execute it with
  configured number of parallel threads.
- Number of parallel thread can be configured from application.yml.
- AsyncTaskExecutorSpec is added
- Fixed existing grovvy test now async task would be submitted.

Issue-ID: CPS-1200
Change-Id: I58c0368b945c90e619c2acfc7458ba58de047484
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2 years agoFixing more RTD warnings 91/130591/1
ToineSiebelink [Thu, 1 Sep 2022 07:14:46 +0000 (08:14 +0100)]
Fixing more RTD warnings

Issue-ID: CPS-1237
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: Ic2ccad169b9ca96ab1788d2ae33d7ead8180b968

2 years agoFixing RTD warnings 43/130543/2
emaclee [Wed, 31 Aug 2022 14:53:10 +0000 (15:53 +0100)]
Fixing RTD warnings

Issue-ID: CPS-1237
Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech>
Change-Id: I0f09d6355d23451207d2d937fbda155b1407fb5a

2 years agoForce py3.8 in tox.ini 55/130555/1
Cédric Ollivier [Wed, 31 Aug 2022 19:17:16 +0000 (21:17 +0200)]
Force py3.8 in tox.ini

It allows leveraging on OpenStack Yoga's upper-contraints
without any exception.

Issue-ID: DOC-782

Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
Change-Id: I02b6a9c1595248e5d7e79a4a6ceb2cd38b5cc6ee

2 years agoMerge "Update CM Handle Query RTD with Casing Convention"
Toine Siebelink [Mon, 29 Aug 2022 17:44:26 +0000 (17:44 +0000)]
Merge "Update CM Handle Query RTD with Casing Convention"

2 years agoPerformance Improvement:save cmhandles capability 54/130454/5
mpriyank [Fri, 26 Aug 2022 12:26:01 +0000 (13:26 +0100)]
Performance Improvement:save cmhandles capability

- add saveCmHandleBatch in InventoryPersistence
- add saveListElementsBatch in CpsDataService
- have addListElementsBatch in CpsDataPersistenceService
- Test scenarios for the same

Issue-ID: CPS-1229
Issue-ID: CPS-1126
Change-Id: I0a1401818da5a4e523d7d0751cac6a526d1611b2
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 years agoUpdate CM Handle Query RTD with Casing Convention 53/130453/2
lukegleeson [Fri, 26 Aug 2022 10:46:22 +0000 (11:46 +0100)]
Update CM Handle Query RTD with Casing Convention

Details the casing differences between the response body and CPS Path querying

Issue-ID: CPS-1211
Signed-off-by: lukegleeson <luke.gleeson@est.tech>
Change-Id: Iec46df1401df4647cd324eea0e0e5f305b5cdbcc

2 years agoMerge "Performance Improvement: Use hazelcast blocking queue"
Sourabh Sourabh [Mon, 29 Aug 2022 12:27:55 +0000 (12:27 +0000)]
Merge "Performance Improvement: Use hazelcast blocking queue"

2 years agoPerformance Improvement: Use hazelcast blocking queue 50/130450/8
ToineSiebelink [Mon, 29 Aug 2022 11:35:33 +0000 (12:35 +0100)]
Performance Improvement: Use hazelcast blocking queue

- Introducing hazelcast for queue and progress map
- process batch of 100 at the time
- decreased module sync watchdog sleeptime to 5 seconds
- separate module sync tasks in new class and some other async preparations and easier testing
- tests for batching in module sync watchdog
- remove qualifiers annotation (support) where no longer needed

Issue-ID: CPS-1210
Issue-ID: CPS-1126
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I0a7d3755bf774e27c5688741bddb01f427d4a8a7

2 years agoMerge "Added eventSchemaVersion for async schema"
Toine Siebelink [Fri, 26 Aug 2022 09:47:50 +0000 (09:47 +0000)]
Merge "Added eventSchemaVersion for async schema"

2 years agoMerge "Performance Improvement: Batch Update DataNodes"
Sourabh Sourabh [Thu, 25 Aug 2022 14:35:18 +0000 (14:35 +0000)]
Merge "Performance Improvement: Batch Update DataNodes"

2 years agoPerformance Improvement: Batch Update DataNodes 92/130392/4
lukegleeson [Thu, 18 Aug 2022 16:47:24 +0000 (17:47 +0100)]
Performance Improvement: Batch Update DataNodes

Implemented methods to perform a batch operation on updating datanodes
Refactored replace data node(s) tree methods to update data node(s) and descendants

Issue-ID: CPS-1203
Signed-off-by: lukegleeson <luke.gleeson@est.tech>
Change-Id: I365d657422b19c9ce384110c9a23d041eaed06f4

2 years agoAdded eventSchemaVersion for async schema 35/130435/1
mpriyank [Tue, 23 Aug 2022 16:22:46 +0000 (17:22 +0100)]
Added eventSchemaVersion for async schema

- Added event schema version in the request and response schema.
- Enhanced the mapper to adhere to new field.

Issue-ID: CPS-1145
Change-Id: I408f73adefc2104b2e421bc3a3fe8f78dcba80f0
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 years agoPerformance Improvement: LCM Events 22/130422/1
mpriyank [Tue, 23 Aug 2022 11:31:23 +0000 (12:31 +0100)]
Performance Improvement: LCM Events

- Changed scope of method responsible for creating and publishing the
  LCM events.
- Use a dedicated thread pool for publishing the LCM events.

Issue-ID: CPS-1223
Change-Id: Ib46e3c1ff8d80f724ad9db30fca6e8125ca0775a
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 years agoMove Advised CM Handles Log to debug level 19/130419/1
lukegleeson [Tue, 23 Aug 2022 08:38:08 +0000 (09:38 +0100)]
Move Advised CM Handles Log to debug level

Issue-ID: CPS-1218
Signed-off-by: lukegleeson <luke.gleeson@est.tech>
Change-Id: I77a799cfb86fe98e526519c7555bbbd85d289f04

2 years agoMerge "Distributed datastore solution for Data Sync Watchdog"
Toine Siebelink [Mon, 22 Aug 2022 16:56:45 +0000 (16:56 +0000)]
Merge "Distributed datastore solution for Data Sync Watchdog"

2 years agoMerge "Performance Improvement: Insert Yang Resources"
Toine Siebelink [Mon, 22 Aug 2022 14:38:37 +0000 (14:38 +0000)]
Merge "Performance Improvement: Insert Yang Resources"

2 years agoDistributed datastore solution for Data Sync Watchdog 99/129999/16
kissand [Thu, 21 Jul 2022 12:53:37 +0000 (14:53 +0200)]
Distributed datastore solution for Data Sync Watchdog

- update lombok config to handle Qualifier annotation
- update Semaphore config to use ConcurrentMap
- update SyncUtils to return a list of cm handles
- update DataSyncWatchdog and ModuleSyncWatchdog with Qualifier
- update DataSyncWatchdog to handle a list of cm handles
- Use get with xpath to check cm handle state

Issue-ID: CPS-1015
Change-Id: Icb39bd29f89e0020d49a1f8960476ffe81b12362
Signed-off-by: kissand <andras.zoltan.kiss@est.tech>
2 years agoMerge "Performance Improvement: Retreive Yang Resources"
Sourabh Sourabh [Mon, 22 Aug 2022 12:35:12 +0000 (12:35 +0000)]
Merge "Performance Improvement: Retreive Yang Resources"

2 years agoPerformance Improvement: Insert Yang Resources 53/130353/6
ToineSiebelink [Thu, 18 Aug 2022 12:15:22 +0000 (13:15 +0100)]
Performance Improvement: Insert Yang Resources

Insert batches of yang resources for a schema set into one operation
add batch-sized tests to ensure batch-size logic is covered for all branches
fixed legacy issues in changed test classes

Issue-ID: CPS-1208
Issue-ID: CPS-1126

Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I5b2a54ed0895999079975d777ba89d589ed4688b

2 years agoPerformance Improvement: Retreive Yang Resources 56/130356/7
ToineSiebelink [Thu, 18 Aug 2022 14:03:16 +0000 (15:03 +0100)]
Performance Improvement: Retreive Yang Resources

Native query to more efficiently get all yangResourceIds
Combined Hibernate Fragment Repository with Native impl.

Issue-ID: CPS-1206
Issue-ID: CPS-1126
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I93fbd5347dd8f9fc48d44e69e47e8aed2d7ac030

2 years agoMerge "Performance Improvement: Detach FragmentEntities"
Sourabh Sourabh [Mon, 22 Aug 2022 11:14:48 +0000 (11:14 +0000)]
Merge "Performance Improvement: Detach FragmentEntities"

2 years agoPerformance Improvement: Detach FragmentEntities 89/130389/2
mpriyank [Fri, 19 Aug 2022 14:25:58 +0000 (15:25 +0100)]
Performance Improvement: Detach FragmentEntities

- Querying fragment entities and moving them to detached state as they
  are not needed to be managed by the JPA provider.
- Addressed legacy change to provide actual type instead of var generic
  type.

Issue-ID: CPS-1126
Issue-ID: CPS-1207
Change-Id: Ibbdbb197a6d480a8b2f7b9cae6052b82ea3686c6
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 years agoMerge "Fix: CM Handles for only hasAllModules Query"
Sourabh Sourabh [Mon, 22 Aug 2022 10:43:24 +0000 (10:43 +0000)]
Merge "Fix: CM Handles for only hasAllModules Query"

2 years agoFix: CM Handles for only hasAllModules Query 37/130337/3
lukegleeson [Wed, 17 Aug 2022 17:06:59 +0000 (18:06 +0100)]
Fix: CM Handles for only hasAllModules Query

hasAllModules was returning no CMHandles when queried by itself, only when paired with another query condition e.g. hasAllProperties
This fix means endpoint works as expected.
Reverted null check

Issue-ID: CPS-977
Signed-off-by: lukegleeson <luke.gleeson@est.tech>
Change-Id: Idafad4c9877941d121f5cbb98617c56a35a1cc94

2 years agoMerge "Performance Improvement: Temporal event"
Joseph Keenan [Fri, 19 Aug 2022 10:48:59 +0000 (10:48 +0000)]
Merge "Performance Improvement: Temporal event"

2 years agoPerformance Improvement: Temporal event 63/130363/5
mpriyank [Thu, 18 Aug 2022 14:20:09 +0000 (15:20 +0100)]
Performance Improvement: Temporal event

- Call the admin service in the async pool thread to fetch the anchor
  details in case of raising event on save, update and delete a node or
  delete list elements
- Admin service is still called in the main thread in case of
  DeleteNodes method as we require to have anchor as we are deleting all
  the anchors.
- Fixed the test scenarios related to processing async event
- Rearranged method formal params

Issue-ID: CPS-1126
Issue-ID: CPS-1209
Change-Id: Ibd8f001c4d7c7de63914f6de24042aaf01a063b2
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 years agoMerge "Update Cps Events LCM RTD"
Joseph Keenan [Fri, 19 Aug 2022 10:28:32 +0000 (10:28 +0000)]
Merge "Update Cps Events LCM RTD"

2 years agoUpdate Cps Events LCM RTD 19/130319/3
emaclee [Wed, 17 Aug 2022 08:52:32 +0000 (09:52 +0100)]
Update Cps Events LCM RTD

Issue-ID: CPS-1196
Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech>
Change-Id: I6f3588e072faba5fd35bbca92990f68dd70322c8

2 years agoMerge "Ensure Additional Properties for CM Handles are included"
Joseph Keenan [Fri, 19 Aug 2022 08:49:14 +0000 (08:49 +0000)]
Merge "Ensure Additional Properties for CM Handles are included"

2 years agoMerge "Performance Improvement: Async Notification Pool Configuration"
Joseph Keenan [Fri, 19 Aug 2022 08:47:08 +0000 (08:47 +0000)]
Merge "Performance Improvement: Async Notification Pool Configuration"

2 years agoMerge "Update info.yml for new Committer"
Joseph Keenan [Thu, 18 Aug 2022 12:44:57 +0000 (12:44 +0000)]
Merge "Update info.yml for new Committer"

2 years agoMerge "Performance Improvement: Tune Resttemplate"
Joseph Keenan [Thu, 18 Aug 2022 11:23:19 +0000 (11:23 +0000)]
Merge "Performance Improvement: Tune Resttemplate"

2 years agoMerge "Set Sleep Times as Environment Variable in CSIT"
Joseph Keenan [Thu, 18 Aug 2022 10:52:15 +0000 (10:52 +0000)]
Merge "Set Sleep Times as Environment Variable in CSIT"

2 years agoPerformance Improvement: Async Notification Pool Configuration 49/130349/2
ToineSiebelink [Thu, 18 Aug 2022 07:50:06 +0000 (08:50 +0100)]
Performance Improvement: Async Notification Pool Configuration

Reduce threads available for notifications
 to allow more resources for module sync

Issue-ID: CPS-1199
Issue-ID: CPS-1126

Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: Iff56dd614ea0074a1b6aa08b81e324e2d0ce33c1

2 years agoPerformance Improvement: Tune Resttemplate 51/130351/1
mpriyank [Thu, 18 Aug 2022 09:35:12 +0000 (10:35 +0100)]
Performance Improvement: Tune Resttemplate

- Increase connection and read timeout for Resttemplate to a higer value

Issue-ID: CPS-1201
Change-Id: Ia159b022698ff60dbd02a5877899f520b91bf883
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 years agoEnsure Additional Properties for CM Handles are included 50/130350/1
ToineSiebelink [Thu, 18 Aug 2022 09:15:43 +0000 (10:15 +0100)]
Ensure Additional Properties for CM Handles are included

- seems like bug is not present on master
- modified test to ensure the behavior is tested now
- (intelliJ) cleaned up so unused stuff

Issue-ID: CPS-1212

Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I6882df4277e50ef869c6a1e9b093ee83dbcdf78e

2 years agoPerformance Improvement: Postgres Pool Configuration 48/130348/1
ToineSiebelink [Thu, 18 Aug 2022 07:43:07 +0000 (08:43 +0100)]
Performance Improvement: Postgres Pool Configuration

Issue-ID: CPS-1198
Issue-ID: CPS-1126

Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I6e1e6b28cd8e20800df61deae1826ecc2db2945e

2 years agoUpdate info.yml for new Committer 28/130328/1
sourabh_sourabh [Wed, 17 Aug 2022 15:48:19 +0000 (16:48 +0100)]
Update info.yml for new Committer

- Updating INFO.yaml to include new committer for CPS (Sourabh)

Issue-ID: CPS-1213
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: If0260910ab903e0cd064ed0bbc5de79309cd1e40

2 years agoSet Sleep Times as Environment Variable in CSIT 01/130301/1
lukegleeson [Tue, 16 Aug 2022 15:52:53 +0000 (16:52 +0100)]
Set Sleep Times as Environment Variable in CSIT

Sleep times for Model and Data sync were being set to 2 seconds in the docker-compose file so that it may be used in CSIT tests
This change defaults them to 30 seconds and adds environment variables which are overridden by the test.properties for CSIT tests

Issue-ID: CPS-1195
Signed-off-by: lukegleeson <luke.gleeson@est.tech>
Change-Id: Id254e3ec82fb4685fd6678d1a345b16f108b2707

2 years agoMerge "CmHandleState transition using state handler"
Toine Siebelink [Fri, 12 Aug 2022 13:41:17 +0000 (13:41 +0000)]
Merge "CmHandleState transition using state handler"

2 years agoMerge "Get Node API fix"
Toine Siebelink [Fri, 12 Aug 2022 13:38:37 +0000 (13:38 +0000)]
Merge "Get Node API fix"

2 years agoGet Node API fix 58/130258/2
puthuparambil.aditya [Thu, 11 Aug 2022 09:55:43 +0000 (10:55 +0100)]
Get Node API fix

Issue-ID: CPS-1179
Signed-off-by: puthuparambil.aditya <aditya.puthuparambil@bell.ca>
Change-Id: I1450ed5d05afd6dd30484eaf19b1e40fd586df8e

2 years agoMerge "Chore: Fix typo in INFO.yaml file"
Toine Siebelink [Thu, 11 Aug 2022 10:19:26 +0000 (10:19 +0000)]
Merge "Chore: Fix typo in INFO.yaml file"

2 years agoCmHandleState transition using state handler 02/130202/8
mpriyank [Fri, 5 Aug 2022 15:34:33 +0000 (16:34 +0100)]
CmHandleState transition using state handler

- Raise LCM Event when state transition happens.
- Raised events for ADVISED to READY, ADVISED to LOCKED, LOCKED to
  ADVISED.
- Refactor existing code to raise the events correctly.
- Refactored existing test scenarios to comply with the code change.

Issue-ID: CPS-1034
Change-Id: Ie548e644f6133304d7fa36c892ca2bec7393c074
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 years agoMerge "CSITs for DataStore SyncState"
Toine Siebelink [Wed, 10 Aug 2022 13:04:03 +0000 (13:04 +0000)]
Merge "CSITs for DataStore SyncState"

2 years agoCSITs for DataStore SyncState 08/130208/4
emaclee [Tue, 9 Aug 2022 08:03:57 +0000 (09:03 +0100)]
CSITs for DataStore SyncState

- add new test for dataSyncEnabled API
- verify data sync follows after setting dataSyncEnabled flag

Issue-ID: CPS-1168
Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech>
Change-Id: Ie22a6b8d6be15a05bfa3ee90218af27fe7092dd0

2 years agoChore: Fix typo in INFO.yaml file 16/130216/1
Bengt Thuree [Wed, 10 Aug 2022 04:35:51 +0000 (14:35 +1000)]
Chore: Fix typo in INFO.yaml file

Remove duplicate on PTL, and fix his UserID.

Signed-off-by: Bengt Thuree <bthuree@linuxfoundation.org>
Change-Id: I3ed42eac79e16690f930a65687fd1e88030644c8

2 years agoModule Sync, Data Sync, Retry Mechanism RTD 03/130203/5
DylanB95EST [Mon, 8 Aug 2022 14:22:54 +0000 (15:22 +0100)]
Module Sync, Data Sync, Retry Mechanism RTD

- Add Documentation for Module Sync, Data Sync,
  and Retry Mechanism Proccesses for RTD
- Update CPS-Core header on the Index to just CPS
- Add Cps-NCMP Customizations for retry mechanism and
  data sync

Issue-ID: CPS-1181
Change-Id: Idb76de380e49e42ca337cabcf2708b940e15f46f
Signed-off-by: DylanB95EST <dylan.byrne@est.tech>
2 years agoMerge "Rename name column in yang resource table"
Joseph Keenan [Mon, 8 Aug 2022 09:36:00 +0000 (09:36 +0000)]
Merge "Rename name column in yang resource table"

2 years agoMerge "Upgrade Postgres Cps-Core"
Joseph Keenan [Mon, 8 Aug 2022 09:35:06 +0000 (09:35 +0000)]
Merge "Upgrade Postgres Cps-Core"

2 years agoRename name column in yang resource table 80/130180/2
DylanB95EST [Thu, 4 Aug 2022 17:07:24 +0000 (18:07 +0100)]
Rename name column in yang resource table

- Rename name column in yang_resource table to file_name
- Refactor instances where it occures in code and tests
- Add new changelog file to rename occurance

Issue-ID: CPS-529
Change-Id: I2aff97410bdb041b90d8ceaeff75e562ef8d9238
Signed-off-by: DylanB95EST <dylan.byrne@est.tech>
2 years agoUpgrade Postgres Cps-Core 81/130181/2
DylanB95EST [Thu, 4 Aug 2022 12:25:31 +0000 (13:25 +0100)]
Upgrade Postgres Cps-Core

Upgrade postgres version to 14 in accordance with Seccom

Issue-ID: CPS-897
Change-Id: I332c88f3c3e1ec3dd55afe31af5714b690b6bd07
Signed-off-by: DylanB95EST <dylan.byrne@est.tech>
2 years agoState Model RTD 77/130177/7
DylanB95EST [Thu, 4 Aug 2022 13:16:33 +0000 (14:16 +0100)]
State Model RTD

Add Read The Doc documentation for CM-Handle state model
Briefly explain all 5 potnetial cm handle states

Issue-ID: CPS-1162
Change-Id: Iee52f9c280929337abf4aac2f5c9bbfb1edde30f
Signed-off-by: DylanB95EST <dylan.byrne@est.tech>
2 years agoDocument DataStore Sync State(RTD) 34/130034/3
bmiklos [Thu, 4 Aug 2022 13:36:11 +0000 (15:36 +0200)]
Document DataStore Sync State(RTD)

Added a CM-Handle data sync state description section under
Cps-Modelling -> NCMP-Modelling -> Basic Concepts -> Cm Handle

Issue-ID: CPS-1164
Change-Id: Ia53a1ccbacda388ec98a3de936af0913edad1c58
Signed-off-by: bmiklos <miklos.baranyak@est.tech>
2 years agoMerge "Query CM Handles RTD"
Joseph Keenan [Thu, 4 Aug 2022 12:20:29 +0000 (12:20 +0000)]
Merge "Query CM Handles RTD"

2 years agoQuery CM Handles RTD 23/130023/5
lukegleeson [Mon, 25 Jul 2022 10:00:11 +0000 (11:00 +0100)]
Query CM Handles RTD

Updates to RTD for CM Handles Querying
Removed documented limitation in CPS path which has since been implemeneted
Updated dmi-registry file in RTD modeling

Issue-ID: CPS-1161
Signed-off-by: lukegleeson <luke.gleeson@est.tech>
Change-Id: I19ff09dadb43a46e14844e198278949f059c6e68

2 years agoLcmEvent state handler refactoring 06/130006/13
mpriyank [Thu, 21 Jul 2022 15:38:55 +0000 (16:38 +0100)]
LcmEvent state handler refactoring

- State handler will now handle new structure of LcmEvent.
- We have 3 types of events i.e create, update and delete.
- Introduced the LcmEventCreatorHelper to delegate some of the
  responsibility of event creation based on event type and the state.
- New tests and existing refactoring
- Code rebased
- Refactored name as per group code review
- Code rebase with other commits
- Used copy constructor for deep copy operation of NcmpServiceCmhandle
  and CompositeState
- UPCOMING : Related user stories to trigger the event publishing using
  state handler.

Issue-ID: CPS-1128
Change-Id: I94b5a87d37d6a174c017ee0aa37cd0f0f74ba084
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 years agoMerge "Enable/Disable Data Sync for Cm Handle"
Joseph Keenan [Tue, 2 Aug 2022 12:59:07 +0000 (12:59 +0000)]
Merge "Enable/Disable Data Sync for Cm Handle"

2 years agoMerge "Incorrect response to {} for cmhandle id-searches"
Joseph Keenan [Tue, 2 Aug 2022 12:47:52 +0000 (12:47 +0000)]
Merge "Incorrect response to {} for cmhandle id-searches"

2 years agoMerge "Query CmHandles using CPS path"
Joseph Keenan [Tue, 2 Aug 2022 12:26:46 +0000 (12:26 +0000)]
Merge "Query CmHandles using CPS path"

2 years agoIncorrect response to {} for cmhandle id-searches 59/130059/2
lukegleeson [Thu, 28 Jul 2022 16:00:46 +0000 (17:00 +0100)]
Incorrect response to {} for cmhandle id-searches

An empty body for the CmHandles id-searches api returns an anchor instead of all cm-handles
Now returns all childnode ids for the dmi-registry

Issue-ID: CPS-1175
Signed-off-by: lukegleeson <luke.gleeson@est.tech>
Change-Id: I358970170a4eb7905e9646605f0f373fbcd78945

2 years agoMerge "Upgrade Liquibase to Version 4.14.0"
Joseph Keenan [Fri, 29 Jul 2022 14:35:27 +0000 (14:35 +0000)]
Merge "Upgrade Liquibase to Version 4.14.0"

2 years agoMerge "LCM events RTD"
Joseph Keenan [Fri, 29 Jul 2022 14:22:56 +0000 (14:22 +0000)]
Merge "LCM events RTD"

2 years agoLCM events RTD 49/130049/5
emaclee [Thu, 28 Jul 2022 05:52:39 +0000 (06:52 +0100)]
LCM events RTD

Issue-ID: CPS-1165
Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech>
Change-Id: Ib6743bc73b260b4345a1bedf0de2ef892a6b3c87

2 years agoQuery CmHandles using CPS path 53/129953/9
lukegleeson [Mon, 11 Jul 2022 09:55:53 +0000 (10:55 +0100)]
Query CmHandles using CPS path

Added withCpsPath condition parameter
Validated to prevent misuse and blocking of querying using private properties
Updated OpenAPI with examples and links to documentation
Moved methods related to cmHandle querying using cps path from InventoryPersistence to CmHandleQueries
Renamed private method deleteSchemaSetAndListElementByCmHandleId to deleteCmHandleByCmHandleId

Issue-ID: CPS-977
Change-Id: I83827215b7e58de74f8f62cd0140516d217d93f1
Signed-off-by: lukegleeson <luke.gleeson@est.tech>
2 years agoEnable/Disable Data Sync for Cm Handle 86/129986/11
DylanB95EST [Tue, 19 Jul 2022 10:36:10 +0000 (11:36 +0100)]
Enable/Disable Data Sync for Cm Handle

-Create API Which will enable/disable data sync enabled flag
-Default functionality of module sync watchdog is to set to false
-Remove global config param
-Will set initial sync state based on data sync enabled flag
-Throws an Exception if the same data sync enabled flag tries
to be set
-Throws Exception if state is not in READY
-Data Sync enabled must be true to complete data sync process
- Delete all resource data within fragment table related
to synced cm handle when data sync is set to false

Issue-ID: CPS-1133
Change-Id: Ib47bbd8293f083c1d705d91bd0def74e6a105c72
Signed-off-by: DylanB95EST <dylan.byrne@est.tech>
2 years agoUpgrade Liquibase to Version 4.14.0 43/130043/1
lukegleeson [Wed, 27 Jul 2022 14:13:45 +0000 (15:13 +0100)]
Upgrade Liquibase to Version 4.14.0

liquibase-core 4.14.0 has been separated from commercial liquibase
As such, license should be resolved

Issue-ID: CPS-963
Signed-off-by: lukegleeson <luke.gleeson@est.tech>
Change-Id: I319c4e33741164b509ed770b1ca095a763ef9fcb

2 years agoDistributed datastore solution for Module Sync Watchdog 22/129922/16
kissand [Thu, 14 Jul 2022 10:37:14 +0000 (12:37 +0200)]
Distributed datastore solution for Module Sync Watchdog

- use semaphore map in ModuleSyncWatchdog
- increase test timeout, because it needs more time for hazelcast
initialization

Issue-ID: CPS-1015
Change-Id: I71feed8fbbd047af9fabba29a5f762a1f17a1c78
Signed-off-by: kissand <andras.zoltan.kiss@est.tech>
2 years agoCmHandle creation performance degradation 29/129829/11
sourabh_sourabh [Thu, 7 Jul 2022 15:53:45 +0000 (16:53 +0100)]
CmHandle creation performance degradation

- Created a dedicated threadpool for scheduler.
- Tuned async threadpool of notification executor from setting RejectedExecutionHandler and application.yml.

Issue-ID: CPS-1126
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: I2afe3c76c1aec78751777df0d2f08ddb8dcee102

2 years agoDe-registration: send event(s) using central state 32/129932/16
sourabh_sourabh [Mon, 25 Jul 2022 08:49:40 +0000 (09:49 +0100)]
De-registration: send event(s) using central state

- refactored method "parseAndRemoveCmHandlesInDmiRegistration"
- added method to send events for DELETING and DELETED
- introduced private method in state handler
- updated unit tests for registration spec
- added unit test to cover new private method in state handler

Issue-ID: CPS-1003
Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech>
Change-Id: Ifc1d1e94b7103d85b6352b196edf7075257c211a
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2 years agoRefractor code to use the new LcmEvent schema 33/129933/12
bmiklos [Thu, 14 Jul 2022 15:15:43 +0000 (17:15 +0200)]
Refractor code to use the new LcmEvent schema

-Old schema deleted and comments from tests are removed
-EventDateTimeFormatter utility added
-From LcmEventsCreator.populateLcmEvent the NcmpServiceCmHandle param
removed and tests are fixed accordingly
-Changes made in test of the LcmEventsCreator for payload testing
to make it more clear

Issue-ID: CPS-1147
Change-Id: Id87c81f2755a6617a0fc68ba92a3d0e3170fd20f
Signed-off-by: bmiklos <miklos.baranyak@est.tech>
2 years agoUpdate dependencies (release task) 66/129966/1
ToineSiebelink [Mon, 18 Jul 2022 10:38:37 +0000 (11:38 +0100)]
Update dependencies (release task)

Removed unnecessary direct dependency
- spring-context

Issue-ID: CPS-1106

Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I4c3b2e8ffb16504f442f2eda5553c185dc290f4b

2 years agoUpdate dependencies (release task) 27/129927/2
ToineSiebelink [Thu, 14 Jul 2022 12:38:26 +0000 (13:38 +0100)]
Update dependencies (release task)

Update to latest dependencies (minor) releases
-spring-boot-dependencies 2.6.9
-spring-cloud-dependencies 2021.0.3

Issue-ID: CPS-1106

Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I4780ca9a6f87d257d430b6c6fefd2c0621aeeec4

2 years agoRemove ALL Docker File Versions 19/129919/2
sourabh_sourabh [Thu, 14 Jul 2022 09:23:35 +0000 (10:23 +0100)]
Remove ALL Docker File Versions

Issue-ID: CPS-1095
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: I11790e271ec784dfa9dbdaea497d50e95f884600

2 years agoMerge "Code Refactoring Ncmp* to Lcm* as per new scope"
Toine Siebelink [Thu, 14 Jul 2022 09:18:17 +0000 (09:18 +0000)]
Merge "Code Refactoring Ncmp* to Lcm* as per new scope"