cps.git
2 years agoAvc Event prop rename 98/133698/1
mpriyank [Mon, 20 Mar 2023 10:55:32 +0000 (10:55 +0000)]
Avc Event prop rename

- rename the eventTarget property as eventSource as per the schema in
  the spike
- fixed the integration test and sample file

Issue-ID: CPS-1554
Change-Id: Ieb8b749c6f64d69d87cacadae17a4589e9908cf2
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 years agoMerge "Enable all Sphinx build warnings as errors"
Toine Siebelink [Mon, 20 Mar 2023 09:26:59 +0000 (09:26 +0000)]
Merge "Enable all Sphinx build warnings as errors"

2 years agoEnable all Sphinx build warnings as errors 75/133675/2
Cédric Ollivier [Wed, 15 Mar 2023 20:40:48 +0000 (21:40 +0100)]
Enable all Sphinx build warnings as errors

It also fixes several issues to pass the gate.
Please note that .. code-block :: json content must be json.

Issue-ID: DOC-798
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
Change-Id: I38a751b67ab70a5ed5d61cdd49fe914c7b9270cd

2 years agoMerge "Enhance DMI Data AVC Event schema"
Luke Gleeson [Wed, 15 Mar 2023 10:51:38 +0000 (10:51 +0000)]
Merge "Enhance DMI Data AVC Event schema"

2 years agoMerge "Forward Subscription Information to DMI Plugin(s)"
Priyank Maheshwari [Wed, 15 Mar 2023 09:37:04 +0000 (09:37 +0000)]
Merge "Forward Subscription Information to DMI Plugin(s)"

2 years agoReduce dataspace/anchor lookups in CpsDataService 27/133627/6
danielhanrahan [Tue, 7 Mar 2023 20:53:04 +0000 (20:53 +0000)]
Reduce dataspace/anchor lookups in CpsDataService

- Use Anchor object instead of names in processDataUpdatedEventAsync
- Use Anchor object instead of names in buildDataNodes
- Avoid unnecessary ArrayList copy in updateDataNodesAndDescendants

Issue-ID: CPS-1536
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I46936a655a3d151357e55b68e1c4161e07100e84

2 years agoMerge "Fetch fragment entities using recursive SQL query"
Toine Siebelink [Tue, 14 Mar 2023 16:27:11 +0000 (16:27 +0000)]
Merge "Fetch fragment entities using recursive SQL query"

2 years agoMerge "subscription-registry node in subscription loader"
Priyank Maheshwari [Tue, 14 Mar 2023 15:47:14 +0000 (15:47 +0000)]
Merge "subscription-registry node in subscription loader"

2 years agoEnhance DMI Data AVC Event schema 52/133652/1
mpriyank [Tue, 14 Mar 2023 13:51:15 +0000 (13:51 +0000)]
Enhance DMI Data AVC Event schema

- Enhancing the DMI Data Avc Event to have the event section as Object
  at the moment.

Issue-ID: CPS-1554
Change-Id: I7f9e1d66ab146f4133886188cd3e4049ac17c76b
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 years agosubscription-registry node in subscription loader 33/133633/2
lukegleeson [Mon, 13 Mar 2023 15:00:19 +0000 (15:00 +0000)]
subscription-registry node in subscription loader

Adding top node subscription-registry for the adding of subscriptions

Issue-ID: CPS-1548
Signed-off-by: lukegleeson <luke.gleeson@est.tech>
Change-Id: I4107293ce023e4c53dbb8b5d3feb0922cc3d4817

2 years agoForward Subscription Information to DMI Plugin(s) 31/133431/9
lukegleeson [Thu, 23 Feb 2023 11:43:13 +0000 (11:43 +0000)]
Forward Subscription Information to DMI Plugin(s)

Note: Implemented common EventsPublisher class

Issue-ID: CPS-1431
Signed-off-by: lukegleeson <luke.gleeson@est.tech>
Change-Id: I292a95f2c990a140f5fd63622ca4eba3f8284b9e

2 years agoFetch fragment entities using recursive SQL query 11/133511/12
danielhanrahan [Wed, 8 Mar 2023 13:50:34 +0000 (13:50 +0000)]
Fetch fragment entities using recursive SQL query

- Add SQL query that can fetch needed fragments to any given depth
- Update getFragmentEntities to use new query
- Remove now unused FragmentRepositoryMultiPathQuery
- Remove unused TempTableCreator method
- Result: getDataNodesForMultipleXpaths is up to 10 times faster

Issue-ID: CPS-1525
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I07cbc9da5ab994ce7e0c2b02d7ca05089f05dab0

2 years agoMerge "Reduce dataspace/anchor lookups in CpsDataPersistenceService"
Toine Siebelink [Mon, 13 Mar 2023 14:18:02 +0000 (14:18 +0000)]
Merge "Reduce dataspace/anchor lookups in CpsDataPersistenceService"

2 years agoMerge "Add new metrics to the get data node method"
Toine Siebelink [Mon, 13 Mar 2023 14:09:27 +0000 (14:09 +0000)]
Merge "Add new metrics to the get data node method"

2 years agoReduce dataspace/anchor lookups in CpsDataPersistenceService 99/133599/5
danielhanrahan [Tue, 7 Mar 2023 20:35:14 +0000 (20:35 +0000)]
Reduce dataspace/anchor lookups in CpsDataPersistenceService

- Remove unneeded calls to DataspaceRepository::getByName
- Remove unneeded calls to AnchorRepository::getByDataspaceAndName
- Refactor FragmentRepository

Issue-ID: CPS-1536
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I2121f6247070ee4a7c000e06ec66a6278b758540

2 years agoMake single deleteDataNode use plural deleteDataNodes 82/133582/7
danielhanrahan [Fri, 10 Mar 2023 13:27:01 +0000 (13:27 +0000)]
Make single deleteDataNode use plural deleteDataNodes

- Make deleteDataNode and deleteListDataNode call deleteDataNodes
- Add onlySupportListDeletion option to deleteDataNodes to support
  original deleteListDataNode behaviour
- Allow delete root xpath in deleteDataNodes
- Fix incorrect use of PathParsingException in deleteDataNode
- Update performance tests timings

Issue-ID: CPS-1523
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I92c3c3ce606a5ab2cb8e6779d1ee0f9853529982

2 years agoAdd schema_set_yang_resources index to tests 11/133611/4
danielhanrahan [Thu, 9 Mar 2023 17:26:34 +0000 (17:26 +0000)]
Add schema_set_yang_resources index to tests

- Add missing index to liquibase test-changelog.yaml

Issue-ID: CPS-1535
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I2f8ee9f66b3f433be2f61e17759f0f6c9f2e4707

2 years agoBump CPS version 19/133619/1
mpriyank [Fri, 10 Mar 2023 13:48:20 +0000 (13:48 +0000)]
Bump CPS version

- bump cps version to 3.2.6-snapshot

Issue-ID: CPS-1547
Change-Id: I397b8e5f22269c270c2fa7823713869b478622f9
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 years agoAdd container yaml 18/133618/1
mpriyank [Fri, 10 Mar 2023 12:50:47 +0000 (12:50 +0000)]
Add container yaml

- add container yaml for 3.2.5

Issue-ID: CPS-1546
Change-Id: I1d505c67cfdfbc83f461fb561557783b43615df3
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 years agomaven stage yaml 17/133617/1
mpriyank [Fri, 10 Mar 2023 12:33:27 +0000 (12:33 +0000)]
maven stage yaml

- add maven stage yaml file for 3.2.5 cps-core

Issue-ID: CPS-1546
Change-Id: Ic167b58b2770de6e025e704311cf32435ad27e5c
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 years agoRelease notes update 14/133614/1
mpriyank [Fri, 10 Mar 2023 11:48:23 +0000 (11:48 +0000)]
Release notes update

- add release notes for introduction of the control switch

Issue-ID: CPS-1545
Change-Id: I738b2f8ceea089af5eebd362c03ca2eb13f351d8
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 years agoIntroduce control switch 13/133613/1
mpriyank [Fri, 10 Mar 2023 10:07:19 +0000 (10:07 +0000)]
Introduce control switch

- introduced a control switch and keeping it as false due to init issues

Issue-ID: CPS-1537
Change-Id: I8455a5ff6fea09527c6f810ffbc620b4e38c488c
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 years agoBump CPS version 10/133610/1
mpriyank [Thu, 9 Mar 2023 17:58:31 +0000 (17:58 +0000)]
Bump CPS version

- bump the CPS version from 3.2.4-SNAPSHOT to 3.2.5-SNAPSHOT

Issue-ID: CPS-1543
Change-Id: I28e46fdeff86c71bd1ad89ae9aaef8ddb4994bfc
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 years agoAdd container yaml 09/133609/1
mpriyank [Thu, 9 Mar 2023 17:23:42 +0000 (17:23 +0000)]
Add container yaml

- Step 6 of the release process to add container YAML file

Issue-ID: CPS-1542
Change-Id: I244dfaec50828e0135350589d848ca79b300971f
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 years agoAdd Maven yaml 08/133608/1
mpriyank [Thu, 9 Mar 2023 17:05:38 +0000 (17:05 +0000)]
Add Maven yaml

- Step 5 of the release process to add maven yaml

Issue-ID: CPS-1542
Change-Id: I793c942af949a4be1c1d9c0efd573e30158fea8e
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 years agoRelease notes for the bugs 05/133605/1
mpriyank [Thu, 9 Mar 2023 16:33:02 +0000 (16:33 +0000)]
Release notes for the bugs

- Release notes

Issue-ID: CPS-1541
Change-Id: I0339dff64c695cb0c9b605a38572696895a09683
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 years agoBug:SubscriptionModelLoader.getFileContentAsString 04/133604/2
emaclee [Thu, 9 Mar 2023 15:50:32 +0000 (15:50 +0000)]
Bug:SubscriptionModelLoader.getFileContentAsString

Issue-ID: CPS-1537
Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech>
Change-Id: Icc88a8bf40830853a208e79658f3c590185ad051

2 years agoMerge "Create Base and Sample Performance Integration Tests"
Sourabh Sourabh [Thu, 9 Mar 2023 13:01:12 +0000 (13:01 +0000)]
Merge "Create Base and Sample Performance Integration Tests"

2 years agoCreate Base and Sample Performance Integration Tests 78/133478/4
ToineSiebelink [Tue, 28 Feb 2023 18:12:51 +0000 (18:12 +0000)]
Create Base and Sample Performance Integration Tests

- added data (folders) for different models and json data
- added base and data for cps-ncmp (registry) specific test cases
- integrated Ahila's (large) openroadm model and data
  (corrected version provided by Lee Anjella)
- created profiles to exclude/include *PerfTest
- fixed dependency test check
- included integration test for overall coverage
- increased margings in existing perf test to reduce nordix intermitten failures

Issue-ID: CPS-1516
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: Ia82826f610636c14aa7e8939b385c278e5039817

2 years agoAdd new metrics to the get data node method 93/133593/2
kissand [Thu, 2 Mar 2023 12:30:17 +0000 (13:30 +0100)]
Add new metrics to the get data node method

Issue-ID: CPS-1509
Change-Id: I2eded9a23b56f5263591c710c7a707cf281f93d7
Signed-off-by: kissand <andras.zoltan.kiss@est.tech>
2 years agoMerge "[CPS] Add configurable params for topic"
Toine Siebelink [Wed, 8 Mar 2023 18:07:13 +0000 (18:07 +0000)]
Merge "[CPS] Add configurable params for topic"

2 years ago[CPS] Add configurable params for topic 85/133585/3
mpriyank [Wed, 8 Mar 2023 15:29:57 +0000 (15:29 +0000)]
[CPS] Add configurable params for topic

- Configurable parameter for topic dmi-cm-events

Issue-ID: CPS-1538
Change-Id: I5396ee8f50f33ea3c117291a314f10c1870966bc
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 years agoMerge "Add index to schema_set_yang_resources table"
Sourabh Sourabh [Wed, 8 Mar 2023 09:52:59 +0000 (09:52 +0000)]
Merge "Add index to schema_set_yang_resources table"

2 years agoMerge "Update CPSarchitecture release notes"
Luke Gleeson [Wed, 8 Mar 2023 09:45:45 +0000 (09:45 +0000)]
Merge "Update CPSarchitecture release notes"

2 years agoAdd index to schema_set_yang_resources table 72/133572/2
danielhanrahan [Tue, 7 Mar 2023 16:56:08 +0000 (16:56 +0000)]
Add index to schema_set_yang_resources table

Currently, the database is sequentially scanning when deleting from
schema_set_yang_resources table as there are no indexes on the table.
Adding an index doubles performance of NCMP CM-handle deregistration.

Issue-ID: CPS-1535
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ia6ac7ef95d1662f49a77fc72112f1e29a67079ff

2 years agoDrop temp table when transaction finishes 70/133570/1
danielhanrahan [Thu, 2 Mar 2023 23:27:09 +0000 (23:27 +0000)]
Drop temp table when transaction finishes

By default, temp tables are persisted until the end of the session.
This causes "Out of shared memory" errors in the database, which then
auto-vacuums the orphaned temp tables to free memory. To fix this, we
drop temp tables when the transaction finishes, using ON COMMIT DROP.

Issue-ID: CPS-1533
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Iac47d85e9621531a4a95b99c4becc8cc95242721

2 years agoRelease process stage 7 66/133566/1
seanbeirne [Tue, 7 Mar 2023 13:42:04 +0000 (13:42 +0000)]
Release process stage 7

Issue-ID: CPS-1529
Signed-off-by: seanbeirne <sean.beirne@est.tech>
Change-Id: Ie6c6276f4612b919762479d2461b4e72cb886245

2 years agoRelease process stage 6 63/133563/2
seanbeirne [Tue, 7 Mar 2023 11:20:56 +0000 (11:20 +0000)]
Release process stage 6

Issue-ID: CPS-1529
Signed-off-by: seanbeirne <sean.beirne@est.tech>
Change-Id: I32fd42c2fc11279710f9d88b8c41df5e8b3a6d1b

2 years agoRelease process step 5 61/133561/2
seanbeirne [Tue, 7 Mar 2023 10:36:18 +0000 (10:36 +0000)]
Release process step 5

Issue-ID: CPS-1529
Signed-off-by: seanbeirne <sean.beirne@est.tech>
Change-Id: I338f7bbaea92a4e6bdbbb6f632976ac81a6f5c63

2 years agoUpdate CPSarchitecture release notes 57/133557/1
emaclee [Tue, 7 Mar 2023 09:29:11 +0000 (09:29 +0000)]
Update CPSarchitecture release notes

Issue-ID: CPS-1532
Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech>
Change-Id: If82b02e7ee4d28299f0c362ad483988a6358c1aa

2 years agoMerge "JAVA onboarding of YANG model"
Priyank Maheshwari [Mon, 6 Mar 2023 09:56:23 +0000 (09:56 +0000)]
Merge "JAVA onboarding of YANG model"

2 years agoJAVA onboarding of YANG model 17/133217/16
emaclee [Thu, 9 Feb 2023 15:17:13 +0000 (15:17 +0000)]
JAVA onboarding of YANG model

- removed docker compose init container
- removed init container script files
- created new model loader class to onboard YANG model which
  runs after the application has set up
- new model loader class is created in new package in
  cps-service (org.onap.cps.ncmp.init)

Issue-ID: CPS-1467
Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech>
Change-Id: I35fe82817aff7783520acf8db66ad504271026bd

2 years agoRefactor cmHandle(ID) queries 22/133522/4
seanbeirne [Thu, 23 Feb 2023 15:16:38 +0000 (15:16 +0000)]
Refactor cmHandle(ID) queries

- first execute cm-handle id search only
- get cm handles for ids only when needed using multiple-get-method
- use Collection interface where posisble (instead of Set)
- use java Function to combine multiple queries in a more genric way

Issue-ID: CPS-1494
Signed-off-by: seanbeirne <sean.beirne@est.tech>
Change-Id: Icfcb8ec94f75e261303aaee5c9034b920c01f3c4

2 years agoAdd DataNodeNotFoundException to deleteDataNodes 82/133482/2
danielhanrahan [Tue, 28 Feb 2023 16:25:08 +0000 (16:25 +0000)]
Add DataNodeNotFoundException to deleteDataNodes

Current implementation of NCMP handle de-registration relies on
DataNodeNotFoundException being thrown to report errors.

- Make deleteDataNodes throw DataNodeFoundExceptionBatch
- Update performance test timings

Issue-ID: CPS-1481
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ib833bdb4a23d24e1784bdaf4e5e5e8a9acb41c54

2 years agoSkip deleting list xpaths that are list elements 20/133420/4
danielhanrahan [Sun, 26 Feb 2023 14:27:06 +0000 (14:27 +0000)]
Skip deleting list xpaths that are list elements

List elements cannot be lists. Filtering list elements before trying
to delete lists by xpaths doubles performance when deleting list
elements (which doubles performance of CM handle de-registration).

Issue-ID: CPS-1511
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ieb6002212d006396d468f27f853708b5aa1e31f2

2 years agoMerge "Expand CPS Service Integration Test (framework)"
Toine Siebelink [Mon, 27 Feb 2023 17:21:26 +0000 (17:21 +0000)]
Merge "Expand CPS Service Integration Test (framework)"

2 years agoExpand CPS Service Integration Test (framework) 01/133401/4
ToineSiebelink [Thu, 23 Feb 2023 15:49:40 +0000 (15:49 +0000)]
Expand CPS Service Integration Test (framework)

- Created package structure
- Created several test bases
- Created complete test set for Admin service
- Created first test for Data service
- Added human-readable toString() to FetchDescendantsOption for test reporting and debuging purposes
- Renamed fetch descendants (enum) direct children option for consistency with others options
- TODO: Add sample performance test (and base)

Issue-ID: CPS-475
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I75317686161be41662b6bf81314a9cd425ddd6eb

2 years agoImprove performance for update data nodes 88/133388/4
danielhanrahan [Tue, 21 Feb 2023 22:55:05 +0000 (22:55 +0000)]
Improve performance for update data nodes

- Extract method getFragmentEntities from getDataNodes
- Remove unused code from getFragmentEntity
- Use bulk getFragmentEntities in updateDataNodesAndDescendants
- Update performance test timings

Issue-ID: CPS-1504
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Icd2c6a0e6009e152de43090cbc23a21349703da2

2 years agoRevert "Omit descendants when not need in ncmp inventory queries" 99/133399/1
lukegleeson [Thu, 23 Feb 2023 14:08:10 +0000 (14:08 +0000)]
Revert "Omit descendants when not need in ncmp inventory queries"

This reverts commit eba77598601a7e3430f9582dfe433495ded3244c.

Signed-off-by: lukegleeson <luke.gleeson@est.tech>
Change-Id: I76a0d1252177c4115de9190ee7f3bb6d15c4e19a

2 years agoAdd performance tests for update data nodes 75/133375/3
danielhanrahan [Tue, 21 Feb 2023 22:09:59 +0000 (22:09 +0000)]
Add performance tests for update data nodes

Issue-ID: CPS-1504
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I1356e4a67a40d03cd71a98dad1571583229f414d

2 years agoMerge "Improve performance of deleteDataNodes SQL"
Luke Gleeson [Wed, 22 Feb 2023 15:24:43 +0000 (15:24 +0000)]
Merge "Improve performance of deleteDataNodes SQL"

2 years agoOmit descendants when not need in ncmp inventory queries 63/133363/3
seanbeirne [Mon, 20 Feb 2023 15:00:11 +0000 (15:00 +0000)]
Omit descendants when not need in ncmp inventory queries

-Seperated tests by Id and details

Issue-ID: CPS-1494
Signed-off-by: seanbeirne <sean.beirne@est.tech>
Change-Id: Ia1c2b637aa5ab2ffe3a92930d74af1dd7d0dbe92

2 years agoImprove performance of deleteDataNodes SQL 47/133347/5
danielhanrahan [Mon, 20 Feb 2023 14:01:45 +0000 (14:01 +0000)]
Improve performance of deleteDataNodes SQL

- Use SQL IN operator instead of temp table when deleting nodes
- Use Postgresql LIKE ANY array operator when deleting lists
- Update delete perf test timings
- Refactor adding cascade delete constraint

Issue-ID: CPS-1502
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ic90b867e7c71ec1981f05a9122322ece84dd8bde

2 years agoMerge "Fix sonar code smell"
Priyank Maheshwari [Tue, 21 Feb 2023 09:37:05 +0000 (09:37 +0000)]
Merge "Fix sonar code smell"

2 years agoMerge "Update performance test timings"
Toine Siebelink [Tue, 21 Feb 2023 08:09:33 +0000 (08:09 +0000)]
Merge "Update performance test timings"

2 years agoMerge "FragmentEntity stuck in memory on "Already exists" error"
Toine Siebelink [Mon, 20 Feb 2023 16:58:49 +0000 (16:58 +0000)]
Merge "FragmentEntity stuck in memory on "Already exists" error"

2 years agoUpdate performance test timings 45/133345/2
danielhanrahan [Mon, 20 Feb 2023 15:16:06 +0000 (15:16 +0000)]
Update performance test timings

- Tighten delete perf test timings
- Fix timings for getDataNodes

Issue-ID: CPS-1502
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ib89a70ef90b1b30e2720e48ebfb66650a2fafa12

2 years agoFix sonar code smell 46/133346/1
halil.cakal [Mon, 20 Feb 2023 16:17:11 +0000 (16:17 +0000)]
Fix sonar code smell

Issue-ID: CPS-475
Change-Id: I6c213090862b68ccf44fb95395c57c717f52a9b2
Signed-off-by: halil.cakal <halil.cakal@est.tech>
2 years agoRTD support to get all data nodes under an anchor 39/133339/1
arpitsingh [Mon, 20 Feb 2023 11:11:03 +0000 (16:41 +0530)]
RTD support to get all data nodes under an anchor

Signed-off-by: arpitsingh <as00745003@techmahindra.com>
Change-Id: I118b6475d2e8c5953b520112deb79869802bac7a

2 years agoMerge "CPS-1401 Implement V2 of GET Data Node API"
Luke Gleeson [Mon, 20 Feb 2023 09:57:05 +0000 (09:57 +0000)]
Merge "CPS-1401 Implement V2 of GET Data Node API"

2 years agoCPS-1401 Implement V2 of GET Data Node API 86/132886/21
arpitsingh [Mon, 9 Jan 2023 14:23:10 +0000 (19:53 +0530)]
CPS-1401 Implement V2 of GET Data Node API

- Modified the GET Data Node API so it returns all the data nodes when
  xpath set to root "/"
- Fragment Repository now returns a collection of Fragment Entities
- Instead of returning only the first Fragment Entity now all fragment
  entities are returned when xpath is set to root
- The Fragemnt Entities are further processed to a Collection of Data
  Nodes. As opposed to singular Data Node in current implementation.
- Finally the DataRestController also returns a Collection of Data
  Nodes when xpath is set to root and valid data is present
- Response body changed from JSON object to JSON Array.
- Exception handling for invalid xpath and non-existing xpath is now
  done separately at persistence layer.
- Refactored code against CPS-1422
- Deprecated getDataNode method from Service and Persistence layer
- Modified V1 of Get Data Node API to use the getDataNodes method and
  get the first data node from the collection returned.
- Modified NCMP to use getDataNodes method
- NCMP still does not support multiple data nodes. It retrieves the
  first data node from the collection returned by getDataNodes

Signed-off-by: arpitsingh <as00745003@techmahindra.com>
Change-Id: I494a5740a53f65376d135fcb9f1e2e8900a2803e

2 years agoFragmentEntity stuck in memory on "Already exists" error 80/133280/1
kissand [Thu, 16 Feb 2023 14:16:10 +0000 (15:16 +0100)]
FragmentEntity stuck in memory on "Already exists" error

- exclude childFragments from toString method

Issue-ID: CPS-1493
Change-Id: I3d0c7ef601f06ed36265d88cadb6fb0ab0cb6ea2
Signed-off-by: kissand <andras.zoltan.kiss@est.tech>
2 years agoImprove batch delete schemasets performance 74/133274/1
danielhanrahan [Wed, 15 Feb 2023 19:00:37 +0000 (19:00 +0000)]
Improve batch delete schemasets performance

- Bulk delete anchors and datanodes associated with schemasets.
  Improves de-registration performance by approx 10%

Issue-ID: CPS-1423
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ie38e8b4c64356bf5935d8c7a5d3f5bfa73fb1714

2 years agoBulk delete schemasets in CM handle deregistration 56/133256/3
danielhanrahan [Tue, 14 Feb 2023 13:24:40 +0000 (13:24 +0000)]
Bulk delete schemasets in CM handle deregistration

- Batch delete schema sets in single query
- Call deleteUnusedYangResourceModules once per batch, not per CM handle
- Results for deregistering 10k: 14 mins before; 6 mins after

Issue-ID: CPS-1423
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ia3a86a0dc88677323e2f386253a99022a7f02603

2 years agoMerge "Use getDataNodes (plural version) into NCMP to get CM handles Performance...
Toine Siebelink [Wed, 15 Feb 2023 10:52:19 +0000 (10:52 +0000)]
Merge "Use getDataNodes (plural version) into NCMP to get CM handles Performance improvement"

2 years agoIntroducing Grafana 91/133191/3
ToineSiebelink [Tue, 7 Feb 2023 10:30:03 +0000 (10:30 +0000)]
Introducing Grafana

- Added profiles to docker compose: dmi-service/dmi-stub, monitoring
- NOTE: now need to add  '--profile dmi-service' to enable dm-plugin (e.g. for csit)
- using container names to resolve hosts for prometheus

Issue-ID: CPS-1465

Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I5d7365a3f789cc698e2702d72700a002cf03eaef

2 years agoUse getDataNodes (plural version) into NCMP to get CM handles Performance improvement 32/133232/3
seanbeirne [Fri, 10 Feb 2023 16:42:38 +0000 (16:42 +0000)]
Use getDataNodes (plural version) into NCMP to get CM handles Performance improvement

- Code clean-up on existing classes
- Removed duplicate code

Issue-ID: CPS-1426
Signed-off-by: seanbeirne <sean.beirne@est.tech>
Change-Id: I3e2ce5b804d7a6e87838490d970cde3cd563f06b

2 years agoMerge "Test for yang file archive limitation"
Luke Gleeson [Mon, 13 Feb 2023 16:08:47 +0000 (16:08 +0000)]
Merge "Test for yang file archive limitation"

2 years agoMerge "Generate report of Instrumentation Metrics"
Toine Siebelink [Mon, 13 Feb 2023 15:18:52 +0000 (15:18 +0000)]
Merge "Generate report of Instrumentation Metrics"

2 years agoTest for yang file archive limitation 18/133218/7
halil.cakal [Thu, 9 Feb 2023 15:51:57 +0000 (15:51 +0000)]
Test for yang file archive limitation

- add new test
- fixed misleading method names

Issue-ID: CPS-1480
Change-Id: I9dd8f41d0acfc8b1c063aab65567d619670f4905
Signed-off-by: halil.cakal <halil.cakal@est.tech>
2 years agoGenerate report of Instrumentation Metrics 29/133229/4
danielhanrahan [Sat, 11 Feb 2023 13:25:22 +0000 (13:25 +0000)]
Generate report of Instrumentation Metrics

Generate a tab-seperated-value report of Prometheus metrics.
By default, assume CPS/NCMP is running on localhost:8887,
and output file named metrics-YYYY-MM-DDThh:mm:ss+00:00.tsv

Issue-ID: CPS-1488
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I8bb356a1b6caacaec4a025a7d6d29cd34420e6c9

2 years agoUse getDataNodes (plural) in CmHandle deregistration 19/133219/2
danielhanrahan [Thu, 9 Feb 2023 15:52:33 +0000 (15:52 +0000)]
Use getDataNodes (plural) in CmHandle deregistration

Issue-ID: CPS-1426
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I0ad79bf36840cdecc13898e185dc618cedafd529

2 years agoMerge "Produce subscription create event for subscription-topic: cm-avc-subscription"
Toine Siebelink [Thu, 9 Feb 2023 14:38:50 +0000 (14:38 +0000)]
Merge "Produce subscription create event for subscription-topic: cm-avc-subscription"

2 years agoMerge "Update CmHandleState on deregistration response"
Priyank Maheshwari [Thu, 9 Feb 2023 11:41:55 +0000 (11:41 +0000)]
Merge "Update CmHandleState on deregistration response"

2 years agoProduce subscription create event for subscription-topic: cm-avc-subscription 74/133174/4
sourabh_sourabh [Fri, 3 Feb 2023 16:20:58 +0000 (16:20 +0000)]
Produce subscription create event for subscription-topic: cm-avc-subscription

- Groovy Kafka (component level) : Produce subscription create event.

Issue-ID: CPS-1394

Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: Ib020c0a4458406925792fc074afd998aa09af158
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2 years agoMerge "Correct calculation of zip archive size for schemaset upload"
Toine Siebelink [Wed, 8 Feb 2023 17:35:12 +0000 (17:35 +0000)]
Merge "Correct calculation of zip archive size for schemaset upload"

2 years agoUse getDataNodes (plural version) into NCMP to get CM handles 19/133119/10
sourabh_sourabh [Thu, 2 Feb 2023 11:22:20 +0000 (11:22 +0000)]
Use getDataNodes (plural version) into NCMP to get CM handles

- Use plural getYangModelCmHandles get YangModelCmHandles by CmHandles in batches.

Issue-ID: CPS-1426

Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: Ie175b471bd98b02b572ca4a3ff699d721b5b782a
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2 years agoRelease Process Step 7 92/133192/3
mpriyank [Wed, 8 Feb 2023 11:21:08 +0000 (11:21 +0000)]
Release Process Step 7

- updated pom to 3.2.3-SNAPSHOT
- updated version.properties and updated the release notes to properly
  depict the correct version

Issue-ID: CPS-1476
Change-Id: Ied60194c6dd4153b58a43715cb729ad75b8e12bf
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 years agoRelease Step 6 89/133189/1
mpriyank [Wed, 8 Feb 2023 10:40:18 +0000 (10:40 +0000)]
Release Step 6

- adding container yaml file for the release

Issue-ID: CPS-1476
Change-Id: Ib61a14591b9fe62e96d24ae3c557f720a23217c7
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 years agoRelease process Step 4 88/133188/1
mpriyank [Wed, 8 Feb 2023 09:58:24 +0000 (09:58 +0000)]
Release process Step 4

- adding maven stage yaml file

Issue-ID: CPS-1476
Change-Id: Iafbc5ed63cf4edc275e5b4087c154439c5c5ac55
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 years agoRelease Steps 1 and 2 81/133181/2
mpriyank [Tue, 7 Feb 2023 17:55:10 +0000 (17:55 +0000)]
Release Steps 1 and 2

- added the jira ticket as part of step1
- copied the changes of openapi.yaml as part of step2 of release process

Issue-ID: CPS-1476
Change-Id: I877757e0396e6349b1ab5e966ae4440c4fead829
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2 years agoCorrect calculation of zip archive size for schemaset upload 87/133187/3
Arne Chrestin [Tue, 7 Feb 2023 16:39:24 +0000 (17:39 +0100)]
Correct calculation of zip archive size for schemaset upload

Correct calculation of zip archive size for schemaset upload

Issue-ID: CPS-1477
Change-Id: I73f97f986a817d423f93a8d922dcd9647b0203ab
Signed-off-by: Arne Chrestin <arne.chrestin@telekom.de>
2 years agoUpdate CmHandleState on deregistration response 47/133147/3
danielhanrahan [Fri, 3 Feb 2023 18:35:16 +0000 (18:35 +0000)]
Update CmHandleState on deregistration response

Current implementation incorrectly sets CmHandleState to DELETED for
CM handles that were not deleted.
- Update CmHandleState to DELETED only for deleted CM handles
- Minor test refactoring

Issue-ID: CPS-1471
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ibf8b6d2b87d46a7633f0497b065e3d4099851fd6

2 years agoMerge "Descendants description changed in openApi"
Luke Gleeson [Fri, 3 Feb 2023 10:08:57 +0000 (10:08 +0000)]
Merge "Descendants description changed in openApi"

2 years agoDescendants description changed in openApi 38/133138/1
rajesh.kumar [Fri, 3 Feb 2023 06:36:40 +0000 (06:36 +0000)]
Descendants description changed in openApi

Issue-ID: CPS-1381
Change-ID: I73f97f986a817d423f93a8d922dcd9647b0203aa
Signed-off-by: rajesh.kumar <rk00747546@techmahindra.com>
2 years agoCmHandle batch deletion 14/133114/6
danielhanrahan [Wed, 1 Feb 2023 14:48:52 +0000 (14:48 +0000)]
CmHandle batch deletion

- Use plural deleteDataNodes to remove CmHandles in batches,
  falling back to individual delete on error
- Use single deleteDataNode instead of deleteListOrListElement for
  individual delete

Issue-ID: CPS-1464
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: If09f22478df8703290c8fc24aa6fe2a11c90788a

2 years agoModify CSIT script to have absolute root path env 18/133118/4
emaclee [Thu, 2 Feb 2023 10:58:35 +0000 (10:58 +0000)]
Modify CSIT script to have absolute root path env

        - change set up to call docker compose for cps-ncmp
        - add root as a property in 'test.properties'
        - root is specified to be root directory folder for the current
          project
        - modified docker compose init container to use ROOT env var

Issue-ID: CPS-1472
Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech>
Change-Id: Ie5ac6bbff7cb1e761501bd0db8447f86af06b95f

2 years agoCreate plural version of deleteDataNode 40/133040/25
danielhanrahan [Tue, 24 Jan 2023 11:23:02 +0000 (11:23 +0000)]
Create plural version of deleteDataNode

- Add method to CpsDataService to batch delete data nodes and lists
- Use native queries to batch delete fragment entities by xpaths,
  for data nodes and lists
- Add performance tests for batch delete
- Refactor FragmentNativeRepository
- Add single-column version of createTemporaryTable
- Renamed metric cps.data.service.datanode.batch.delete
  to cps.data.service.datanode.all.delete

Issue-ID: CPS-1438
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I1851f9c7ef0b1be4bd421b3352d9697a2dd23f79

2 years agoMerge "Introduce Instrumentation"
Luke Gleeson [Tue, 31 Jan 2023 11:41:54 +0000 (11:41 +0000)]
Merge "Introduce Instrumentation"

2 years agoIntroduce Instrumentation 60/132960/7
ToineSiebelink [Thu, 19 Jan 2023 16:45:58 +0000 (16:45 +0000)]
Introduce Instrumentation

- Add instrumentation related dependency
- Added Timed Instrumentation
- CPS-Service Crud methods
      - CPS Yang parsing
      - NCMP Registration methods
      - NCMP Events handling
- Remove manual Gauge for YanResources Cache
 as (better!) instrumentation is already built into the 3PP
- Sorted dependecies alphabetically (as we used to enforce, to prevent duplicates)
- Added ##  P E R F O R M A N C E   T E S T   R E S U L T S  ### mini report
- (unrelated) test improvement (because of bug that turned out to be invalid)
Reviewers: Sourabh,Priyank, Luke

Issue-ID: CPS-1457

Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I34b20bece2f59488b022b8effa9470704c57be4d

2 years agoEdit example given for ncmp datastore 68/133068/1
seanbeirne [Mon, 30 Jan 2023 16:53:37 +0000 (16:53 +0000)]
Edit example given for ncmp datastore

Issue-ID: CPS-1468
Signed-off-by: seanbeirne <sean.beirne@est.tech>
Change-Id: Id746604fca83333f4ee186ce96ea80cb779c4ac2

2 years agoFix for integration test module tests 65/133065/1
lukegleeson [Mon, 30 Jan 2023 15:18:54 +0000 (15:18 +0000)]
Fix for integration test module tests

Added Fragment Native Repository to test config
Removed test profile

Issue-ID: CPS-1466
Signed-off-by: lukegleeson <luke.gleeson@est.tech>
Change-Id: Id845b713e760801be480aa9aea6cb304decd05b9

2 years agoUpdate version files for release process 55/133055/1
seanbeirne [Fri, 27 Jan 2023 18:09:33 +0000 (18:09 +0000)]
Update version files for release process

- Updated version.properties
- Updated release-notes
- Updated pom files

Issue-ID: CPS-1835
Signed-off-by: seanbeirne <sean.beirne@est.tech>
Change-Id: I0f837ac41d24b41d7c2b9e3dfe2dacc550c0aafa

2 years agoCreate release container yaml file for release process 54/133054/1
seanbeirne [Fri, 27 Jan 2023 16:56:00 +0000 (16:56 +0000)]
Create release container yaml file for release process

Issue-ID: CPS-1385
Signed-off-by: seanbeirne <sean.beirne@est.tech>
Change-Id: I95f3f76c49ea9eb9556fb4b32dbe10dc3e8283ad

2 years agoMerge "Create release yaml file for release process"
Priyank Maheshwari [Fri, 27 Jan 2023 16:54:31 +0000 (16:54 +0000)]
Merge "Create release yaml file for release process"

2 years agoMerge "Release Note Update"
Sourabh Sourabh [Fri, 27 Jan 2023 16:41:37 +0000 (16:41 +0000)]
Merge "Release Note Update"

2 years agoRelease Note Update 22/133022/4
lukegleeson [Wed, 25 Jan 2023 15:06:52 +0000 (15:06 +0000)]
Release Note Update

Added Query large outputs using limit/depth/pagination
Reordered existing items in ascending Jira number order
Added missing Jiras

Issue-ID: CPS-1381
Signed-off-by: lukegleeson <luke.gleeson@est.tech>
Change-Id: I1a124db8d8a4adffc7807aecddb378876b732cd7

2 years agoCreate release yaml file for release process 37/133037/3
seanbeirne [Thu, 26 Jan 2023 14:19:31 +0000 (14:19 +0000)]
Create release yaml file for release process

Issue-ID: CPS-1385
Signed-off-by: seanbeirne <sean.beirne@est.tech>
Change-Id: I98c7cafb3b850ef6a419c87c242e1f469ccf600e

2 years agoMerge "Define and onboard model for subscription events"
Luke Gleeson [Fri, 27 Jan 2023 13:37:17 +0000 (13:37 +0000)]
Merge "Define and onboard model for subscription events"

2 years agoDefine and onboard model for subscription events 18/133018/8
emaclee [Wed, 25 Jan 2023 12:18:15 +0000 (12:18 +0000)]
Define and onboard model for subscription events

- created new yang model for subscription events; added in
  ncmp-service
- created new container/service to run an init container
- added script file to the container
- added Dockerfile to be the base image of the init container
- health check for CPS added as part of script
- removed previous patch modification in setup
- changed path of models and dockerfile to main project
- latest patch: changed connection to external ports of CPS

Issue-ID: CPS-1393
Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech>
Change-Id: I2367020e57b34a669dda93050eeaeede5846304e