cps.git
14 months agoSkip deleting list xpaths that are list elements 20/133420/4
danielhanrahan [Sun, 26 Feb 2023 14:27:06 +0000 (14:27 +0000)]
Skip deleting list xpaths that are list elements

List elements cannot be lists. Filtering list elements before trying
to delete lists by xpaths doubles performance when deleting list
elements (which doubles performance of CM handle de-registration).

Issue-ID: CPS-1511
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ieb6002212d006396d468f27f853708b5aa1e31f2

14 months agoMerge "Expand CPS Service Integration Test (framework)"
Toine Siebelink [Mon, 27 Feb 2023 17:21:26 +0000 (17:21 +0000)]
Merge "Expand CPS Service Integration Test (framework)"

14 months agoExpand CPS Service Integration Test (framework) 01/133401/4
ToineSiebelink [Thu, 23 Feb 2023 15:49:40 +0000 (15:49 +0000)]
Expand CPS Service Integration Test (framework)

- Created package structure
- Created several test bases
- Created complete test set for Admin service
- Created first test for Data service
- Added human-readable toString() to FetchDescendantsOption for test reporting and debuging purposes
- Renamed fetch descendants (enum) direct children option for consistency with others options
- TODO: Add sample performance test (and base)

Issue-ID: CPS-475
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I75317686161be41662b6bf81314a9cd425ddd6eb

14 months agoImprove performance for update data nodes 88/133388/4
danielhanrahan [Tue, 21 Feb 2023 22:55:05 +0000 (22:55 +0000)]
Improve performance for update data nodes

- Extract method getFragmentEntities from getDataNodes
- Remove unused code from getFragmentEntity
- Use bulk getFragmentEntities in updateDataNodesAndDescendants
- Update performance test timings

Issue-ID: CPS-1504
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Icd2c6a0e6009e152de43090cbc23a21349703da2

14 months agoRevert "Omit descendants when not need in ncmp inventory queries" 99/133399/1
lukegleeson [Thu, 23 Feb 2023 14:08:10 +0000 (14:08 +0000)]
Revert "Omit descendants when not need in ncmp inventory queries"

This reverts commit eba77598601a7e3430f9582dfe433495ded3244c.

Signed-off-by: lukegleeson <luke.gleeson@est.tech>
Change-Id: I76a0d1252177c4115de9190ee7f3bb6d15c4e19a

14 months agoAdd performance tests for update data nodes 75/133375/3
danielhanrahan [Tue, 21 Feb 2023 22:09:59 +0000 (22:09 +0000)]
Add performance tests for update data nodes

Issue-ID: CPS-1504
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I1356e4a67a40d03cd71a98dad1571583229f414d

14 months agoMerge "Improve performance of deleteDataNodes SQL"
Luke Gleeson [Wed, 22 Feb 2023 15:24:43 +0000 (15:24 +0000)]
Merge "Improve performance of deleteDataNodes SQL"

14 months agoOmit descendants when not need in ncmp inventory queries 63/133363/3
seanbeirne [Mon, 20 Feb 2023 15:00:11 +0000 (15:00 +0000)]
Omit descendants when not need in ncmp inventory queries

-Seperated tests by Id and details

Issue-ID: CPS-1494
Signed-off-by: seanbeirne <sean.beirne@est.tech>
Change-Id: Ia1c2b637aa5ab2ffe3a92930d74af1dd7d0dbe92

14 months agoImprove performance of deleteDataNodes SQL 47/133347/5
danielhanrahan [Mon, 20 Feb 2023 14:01:45 +0000 (14:01 +0000)]
Improve performance of deleteDataNodes SQL

- Use SQL IN operator instead of temp table when deleting nodes
- Use Postgresql LIKE ANY array operator when deleting lists
- Update delete perf test timings
- Refactor adding cascade delete constraint

Issue-ID: CPS-1502
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ic90b867e7c71ec1981f05a9122322ece84dd8bde

14 months agoMerge "Fix sonar code smell"
Priyank Maheshwari [Tue, 21 Feb 2023 09:37:05 +0000 (09:37 +0000)]
Merge "Fix sonar code smell"

14 months agoMerge "Update performance test timings"
Toine Siebelink [Tue, 21 Feb 2023 08:09:33 +0000 (08:09 +0000)]
Merge "Update performance test timings"

14 months agoMerge "FragmentEntity stuck in memory on "Already exists" error"
Toine Siebelink [Mon, 20 Feb 2023 16:58:49 +0000 (16:58 +0000)]
Merge "FragmentEntity stuck in memory on "Already exists" error"

14 months agoUpdate performance test timings 45/133345/2
danielhanrahan [Mon, 20 Feb 2023 15:16:06 +0000 (15:16 +0000)]
Update performance test timings

- Tighten delete perf test timings
- Fix timings for getDataNodes

Issue-ID: CPS-1502
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ib89a70ef90b1b30e2720e48ebfb66650a2fafa12

14 months agoFix sonar code smell 46/133346/1
halil.cakal [Mon, 20 Feb 2023 16:17:11 +0000 (16:17 +0000)]
Fix sonar code smell

Issue-ID: CPS-475
Change-Id: I6c213090862b68ccf44fb95395c57c717f52a9b2
Signed-off-by: halil.cakal <halil.cakal@est.tech>
14 months agoRTD support to get all data nodes under an anchor 39/133339/1
arpitsingh [Mon, 20 Feb 2023 11:11:03 +0000 (16:41 +0530)]
RTD support to get all data nodes under an anchor

Signed-off-by: arpitsingh <as00745003@techmahindra.com>
Change-Id: I118b6475d2e8c5953b520112deb79869802bac7a

14 months agoMerge "CPS-1401 Implement V2 of GET Data Node API"
Luke Gleeson [Mon, 20 Feb 2023 09:57:05 +0000 (09:57 +0000)]
Merge "CPS-1401 Implement V2 of GET Data Node API"

15 months agoCPS-1401 Implement V2 of GET Data Node API 86/132886/21
arpitsingh [Mon, 9 Jan 2023 14:23:10 +0000 (19:53 +0530)]
CPS-1401 Implement V2 of GET Data Node API

- Modified the GET Data Node API so it returns all the data nodes when
  xpath set to root "/"
- Fragment Repository now returns a collection of Fragment Entities
- Instead of returning only the first Fragment Entity now all fragment
  entities are returned when xpath is set to root
- The Fragemnt Entities are further processed to a Collection of Data
  Nodes. As opposed to singular Data Node in current implementation.
- Finally the DataRestController also returns a Collection of Data
  Nodes when xpath is set to root and valid data is present
- Response body changed from JSON object to JSON Array.
- Exception handling for invalid xpath and non-existing xpath is now
  done separately at persistence layer.
- Refactored code against CPS-1422
- Deprecated getDataNode method from Service and Persistence layer
- Modified V1 of Get Data Node API to use the getDataNodes method and
  get the first data node from the collection returned.
- Modified NCMP to use getDataNodes method
- NCMP still does not support multiple data nodes. It retrieves the
  first data node from the collection returned by getDataNodes

Signed-off-by: arpitsingh <as00745003@techmahindra.com>
Change-Id: I494a5740a53f65376d135fcb9f1e2e8900a2803e

15 months agoFragmentEntity stuck in memory on "Already exists" error 80/133280/1
kissand [Thu, 16 Feb 2023 14:16:10 +0000 (15:16 +0100)]
FragmentEntity stuck in memory on "Already exists" error

- exclude childFragments from toString method

Issue-ID: CPS-1493
Change-Id: I3d0c7ef601f06ed36265d88cadb6fb0ab0cb6ea2
Signed-off-by: kissand <andras.zoltan.kiss@est.tech>
15 months agoImprove batch delete schemasets performance 74/133274/1
danielhanrahan [Wed, 15 Feb 2023 19:00:37 +0000 (19:00 +0000)]
Improve batch delete schemasets performance

- Bulk delete anchors and datanodes associated with schemasets.
  Improves de-registration performance by approx 10%

Issue-ID: CPS-1423
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ie38e8b4c64356bf5935d8c7a5d3f5bfa73fb1714

15 months agoBulk delete schemasets in CM handle deregistration 56/133256/3
danielhanrahan [Tue, 14 Feb 2023 13:24:40 +0000 (13:24 +0000)]
Bulk delete schemasets in CM handle deregistration

- Batch delete schema sets in single query
- Call deleteUnusedYangResourceModules once per batch, not per CM handle
- Results for deregistering 10k: 14 mins before; 6 mins after

Issue-ID: CPS-1423
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ia3a86a0dc88677323e2f386253a99022a7f02603

15 months agoMerge "Use getDataNodes (plural version) into NCMP to get CM handles Performance...
Toine Siebelink [Wed, 15 Feb 2023 10:52:19 +0000 (10:52 +0000)]
Merge "Use getDataNodes (plural version) into NCMP to get CM handles Performance improvement"

15 months agoIntroducing Grafana 91/133191/3
ToineSiebelink [Tue, 7 Feb 2023 10:30:03 +0000 (10:30 +0000)]
Introducing Grafana

- Added profiles to docker compose: dmi-service/dmi-stub, monitoring
- NOTE: now need to add  '--profile dmi-service' to enable dm-plugin (e.g. for csit)
- using container names to resolve hosts for prometheus

Issue-ID: CPS-1465

Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I5d7365a3f789cc698e2702d72700a002cf03eaef

15 months agoUse getDataNodes (plural version) into NCMP to get CM handles Performance improvement 32/133232/3
seanbeirne [Fri, 10 Feb 2023 16:42:38 +0000 (16:42 +0000)]
Use getDataNodes (plural version) into NCMP to get CM handles Performance improvement

- Code clean-up on existing classes
- Removed duplicate code

Issue-ID: CPS-1426
Signed-off-by: seanbeirne <sean.beirne@est.tech>
Change-Id: I3e2ce5b804d7a6e87838490d970cde3cd563f06b

15 months agoMerge "Test for yang file archive limitation"
Luke Gleeson [Mon, 13 Feb 2023 16:08:47 +0000 (16:08 +0000)]
Merge "Test for yang file archive limitation"

15 months agoMerge "Generate report of Instrumentation Metrics"
Toine Siebelink [Mon, 13 Feb 2023 15:18:52 +0000 (15:18 +0000)]
Merge "Generate report of Instrumentation Metrics"

15 months agoTest for yang file archive limitation 18/133218/7
halil.cakal [Thu, 9 Feb 2023 15:51:57 +0000 (15:51 +0000)]
Test for yang file archive limitation

- add new test
- fixed misleading method names

Issue-ID: CPS-1480
Change-Id: I9dd8f41d0acfc8b1c063aab65567d619670f4905
Signed-off-by: halil.cakal <halil.cakal@est.tech>
15 months agoGenerate report of Instrumentation Metrics 29/133229/4
danielhanrahan [Sat, 11 Feb 2023 13:25:22 +0000 (13:25 +0000)]
Generate report of Instrumentation Metrics

Generate a tab-seperated-value report of Prometheus metrics.
By default, assume CPS/NCMP is running on localhost:8887,
and output file named metrics-YYYY-MM-DDThh:mm:ss+00:00.tsv

Issue-ID: CPS-1488
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I8bb356a1b6caacaec4a025a7d6d29cd34420e6c9

15 months agoUse getDataNodes (plural) in CmHandle deregistration 19/133219/2
danielhanrahan [Thu, 9 Feb 2023 15:52:33 +0000 (15:52 +0000)]
Use getDataNodes (plural) in CmHandle deregistration

Issue-ID: CPS-1426
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I0ad79bf36840cdecc13898e185dc618cedafd529

15 months agoMerge "Produce subscription create event for subscription-topic: cm-avc-subscription"
Toine Siebelink [Thu, 9 Feb 2023 14:38:50 +0000 (14:38 +0000)]
Merge "Produce subscription create event for subscription-topic: cm-avc-subscription"

15 months agoMerge "Update CmHandleState on deregistration response"
Priyank Maheshwari [Thu, 9 Feb 2023 11:41:55 +0000 (11:41 +0000)]
Merge "Update CmHandleState on deregistration response"

15 months agoProduce subscription create event for subscription-topic: cm-avc-subscription 74/133174/4
sourabh_sourabh [Fri, 3 Feb 2023 16:20:58 +0000 (16:20 +0000)]
Produce subscription create event for subscription-topic: cm-avc-subscription

- Groovy Kafka (component level) : Produce subscription create event.

Issue-ID: CPS-1394

Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: Ib020c0a4458406925792fc074afd998aa09af158
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
15 months agoMerge "Correct calculation of zip archive size for schemaset upload"
Toine Siebelink [Wed, 8 Feb 2023 17:35:12 +0000 (17:35 +0000)]
Merge "Correct calculation of zip archive size for schemaset upload"

15 months agoUse getDataNodes (plural version) into NCMP to get CM handles 19/133119/10
sourabh_sourabh [Thu, 2 Feb 2023 11:22:20 +0000 (11:22 +0000)]
Use getDataNodes (plural version) into NCMP to get CM handles

- Use plural getYangModelCmHandles get YangModelCmHandles by CmHandles in batches.

Issue-ID: CPS-1426

Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: Ie175b471bd98b02b572ca4a3ff699d721b5b782a
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
15 months agoRelease Process Step 7 92/133192/3
mpriyank [Wed, 8 Feb 2023 11:21:08 +0000 (11:21 +0000)]
Release Process Step 7

- updated pom to 3.2.3-SNAPSHOT
- updated version.properties and updated the release notes to properly
  depict the correct version

Issue-ID: CPS-1476
Change-Id: Ied60194c6dd4153b58a43715cb729ad75b8e12bf
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
15 months agoRelease Step 6 89/133189/1
mpriyank [Wed, 8 Feb 2023 10:40:18 +0000 (10:40 +0000)]
Release Step 6

- adding container yaml file for the release

Issue-ID: CPS-1476
Change-Id: Ib61a14591b9fe62e96d24ae3c557f720a23217c7
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
15 months agoRelease process Step 4 88/133188/1
mpriyank [Wed, 8 Feb 2023 09:58:24 +0000 (09:58 +0000)]
Release process Step 4

- adding maven stage yaml file

Issue-ID: CPS-1476
Change-Id: Iafbc5ed63cf4edc275e5b4087c154439c5c5ac55
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
15 months agoRelease Steps 1 and 2 81/133181/2
mpriyank [Tue, 7 Feb 2023 17:55:10 +0000 (17:55 +0000)]
Release Steps 1 and 2

- added the jira ticket as part of step1
- copied the changes of openapi.yaml as part of step2 of release process

Issue-ID: CPS-1476
Change-Id: I877757e0396e6349b1ab5e966ae4440c4fead829
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
15 months agoCorrect calculation of zip archive size for schemaset upload 87/133187/3
Arne Chrestin [Tue, 7 Feb 2023 16:39:24 +0000 (17:39 +0100)]
Correct calculation of zip archive size for schemaset upload

Correct calculation of zip archive size for schemaset upload

Issue-ID: CPS-1477
Change-Id: I73f97f986a817d423f93a8d922dcd9647b0203ab
Signed-off-by: Arne Chrestin <arne.chrestin@telekom.de>
15 months agoUpdate CmHandleState on deregistration response 47/133147/3
danielhanrahan [Fri, 3 Feb 2023 18:35:16 +0000 (18:35 +0000)]
Update CmHandleState on deregistration response

Current implementation incorrectly sets CmHandleState to DELETED for
CM handles that were not deleted.
- Update CmHandleState to DELETED only for deleted CM handles
- Minor test refactoring

Issue-ID: CPS-1471
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ibf8b6d2b87d46a7633f0497b065e3d4099851fd6

15 months agoMerge "Descendants description changed in openApi"
Luke Gleeson [Fri, 3 Feb 2023 10:08:57 +0000 (10:08 +0000)]
Merge "Descendants description changed in openApi"

15 months agoDescendants description changed in openApi 38/133138/1
rajesh.kumar [Fri, 3 Feb 2023 06:36:40 +0000 (06:36 +0000)]
Descendants description changed in openApi

Issue-ID: CPS-1381
Change-ID: I73f97f986a817d423f93a8d922dcd9647b0203aa
Signed-off-by: rajesh.kumar <rk00747546@techmahindra.com>
15 months agoCmHandle batch deletion 14/133114/6
danielhanrahan [Wed, 1 Feb 2023 14:48:52 +0000 (14:48 +0000)]
CmHandle batch deletion

- Use plural deleteDataNodes to remove CmHandles in batches,
  falling back to individual delete on error
- Use single deleteDataNode instead of deleteListOrListElement for
  individual delete

Issue-ID: CPS-1464
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: If09f22478df8703290c8fc24aa6fe2a11c90788a

15 months agoModify CSIT script to have absolute root path env 18/133118/4
emaclee [Thu, 2 Feb 2023 10:58:35 +0000 (10:58 +0000)]
Modify CSIT script to have absolute root path env

        - change set up to call docker compose for cps-ncmp
        - add root as a property in 'test.properties'
        - root is specified to be root directory folder for the current
          project
        - modified docker compose init container to use ROOT env var

Issue-ID: CPS-1472
Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech>
Change-Id: Ie5ac6bbff7cb1e761501bd0db8447f86af06b95f

15 months agoCreate plural version of deleteDataNode 40/133040/25
danielhanrahan [Tue, 24 Jan 2023 11:23:02 +0000 (11:23 +0000)]
Create plural version of deleteDataNode

- Add method to CpsDataService to batch delete data nodes and lists
- Use native queries to batch delete fragment entities by xpaths,
  for data nodes and lists
- Add performance tests for batch delete
- Refactor FragmentNativeRepository
- Add single-column version of createTemporaryTable
- Renamed metric cps.data.service.datanode.batch.delete
  to cps.data.service.datanode.all.delete

Issue-ID: CPS-1438
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I1851f9c7ef0b1be4bd421b3352d9697a2dd23f79

15 months agoMerge "Introduce Instrumentation"
Luke Gleeson [Tue, 31 Jan 2023 11:41:54 +0000 (11:41 +0000)]
Merge "Introduce Instrumentation"

15 months agoIntroduce Instrumentation 60/132960/7
ToineSiebelink [Thu, 19 Jan 2023 16:45:58 +0000 (16:45 +0000)]
Introduce Instrumentation

- Add instrumentation related dependency
- Added Timed Instrumentation
- CPS-Service Crud methods
      - CPS Yang parsing
      - NCMP Registration methods
      - NCMP Events handling
- Remove manual Gauge for YanResources Cache
 as (better!) instrumentation is already built into the 3PP
- Sorted dependecies alphabetically (as we used to enforce, to prevent duplicates)
- Added ##  P E R F O R M A N C E   T E S T   R E S U L T S  ### mini report
- (unrelated) test improvement (because of bug that turned out to be invalid)
Reviewers: Sourabh,Priyank, Luke

Issue-ID: CPS-1457

Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I34b20bece2f59488b022b8effa9470704c57be4d

15 months agoEdit example given for ncmp datastore 68/133068/1
seanbeirne [Mon, 30 Jan 2023 16:53:37 +0000 (16:53 +0000)]
Edit example given for ncmp datastore

Issue-ID: CPS-1468
Signed-off-by: seanbeirne <sean.beirne@est.tech>
Change-Id: Id746604fca83333f4ee186ce96ea80cb779c4ac2

15 months agoFix for integration test module tests 65/133065/1
lukegleeson [Mon, 30 Jan 2023 15:18:54 +0000 (15:18 +0000)]
Fix for integration test module tests

Added Fragment Native Repository to test config
Removed test profile

Issue-ID: CPS-1466
Signed-off-by: lukegleeson <luke.gleeson@est.tech>
Change-Id: Id845b713e760801be480aa9aea6cb304decd05b9

15 months agoUpdate version files for release process 55/133055/1
seanbeirne [Fri, 27 Jan 2023 18:09:33 +0000 (18:09 +0000)]
Update version files for release process

- Updated version.properties
- Updated release-notes
- Updated pom files

Issue-ID: CPS-1835
Signed-off-by: seanbeirne <sean.beirne@est.tech>
Change-Id: I0f837ac41d24b41d7c2b9e3dfe2dacc550c0aafa

15 months agoCreate release container yaml file for release process 54/133054/1
seanbeirne [Fri, 27 Jan 2023 16:56:00 +0000 (16:56 +0000)]
Create release container yaml file for release process

Issue-ID: CPS-1385
Signed-off-by: seanbeirne <sean.beirne@est.tech>
Change-Id: I95f3f76c49ea9eb9556fb4b32dbe10dc3e8283ad

15 months agoMerge "Create release yaml file for release process"
Priyank Maheshwari [Fri, 27 Jan 2023 16:54:31 +0000 (16:54 +0000)]
Merge "Create release yaml file for release process"

15 months agoMerge "Release Note Update"
Sourabh Sourabh [Fri, 27 Jan 2023 16:41:37 +0000 (16:41 +0000)]
Merge "Release Note Update"

15 months agoRelease Note Update 22/133022/4
lukegleeson [Wed, 25 Jan 2023 15:06:52 +0000 (15:06 +0000)]
Release Note Update

Added Query large outputs using limit/depth/pagination
Reordered existing items in ascending Jira number order
Added missing Jiras

Issue-ID: CPS-1381
Signed-off-by: lukegleeson <luke.gleeson@est.tech>
Change-Id: I1a124db8d8a4adffc7807aecddb378876b732cd7

15 months agoCreate release yaml file for release process 37/133037/3
seanbeirne [Thu, 26 Jan 2023 14:19:31 +0000 (14:19 +0000)]
Create release yaml file for release process

Issue-ID: CPS-1385
Signed-off-by: seanbeirne <sean.beirne@est.tech>
Change-Id: I98c7cafb3b850ef6a419c87c242e1f469ccf600e

15 months agoMerge "Define and onboard model for subscription events"
Luke Gleeson [Fri, 27 Jan 2023 13:37:17 +0000 (13:37 +0000)]
Merge "Define and onboard model for subscription events"

15 months agoDefine and onboard model for subscription events 18/133018/8
emaclee [Wed, 25 Jan 2023 12:18:15 +0000 (12:18 +0000)]
Define and onboard model for subscription events

- created new yang model for subscription events; added in
  ncmp-service
- created new container/service to run an init container
- added script file to the container
- added Dockerfile to be the base image of the init container
- health check for CPS added as part of script
- removed previous patch modification in setup
- changed path of models and dockerfile to main project
- latest patch: changed connection to external ports of CPS

Issue-ID: CPS-1393
Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech>
Change-Id: I2367020e57b34a669dda93050eeaeede5846304e

15 months agoMerge "NCMP De-Register response payload and status is invalid"
Sourabh Sourabh [Thu, 26 Jan 2023 17:39:23 +0000 (17:39 +0000)]
Merge "NCMP De-Register response payload and status is invalid"

15 months agoMerge "Disable persistence integration tests"
Toine Siebelink [Thu, 26 Jan 2023 17:36:17 +0000 (17:36 +0000)]
Merge "Disable persistence integration tests"

15 months agoDisable persistence integration tests 41/133041/1
lukegleeson [Thu, 26 Jan 2023 17:17:56 +0000 (17:17 +0000)]
Disable persistence integration tests

Disable tests for deploy
Enable tests in Nordix jobs

Issue-ID: CPS-1466
Signed-off-by: lukegleeson <luke.gleeson@est.tech>
Change-Id: I2aa4952ea800d3499ea7380faefdcac1d8955cb4

15 months agoNCMP De-Register response payload and status is invalid 27/133027/5
sourabh_sourabh [Wed, 25 Jan 2023 17:13:38 +0000 (17:13 +0000)]
NCMP De-Register response payload and status is invalid

-Refactored "DELETING" and "DELETED" update state to avoid wrong
response code.

Issue-ID: CPS-1463

Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: I639ba94986c4ffe48d2d42f14f6a6b4e2bd2713e
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
15 months agoMerge "Springboot Integration tests improvements"
Sourabh Sourabh [Thu, 26 Jan 2023 12:03:10 +0000 (12:03 +0000)]
Merge "Springboot Integration tests improvements"

15 months agoUpdate openAPI documentation for release process 32/133032/1
seanbeirne [Thu, 26 Jan 2023 09:21:44 +0000 (09:21 +0000)]
Update openAPI documentation for release process

Issue-ID: CPS-1385
Signed-off-by: seanbeirne <sean.beirne@est.tech>
Change-Id: Ia0b011d4497e7d95f9b625514e8b3216575df044

15 months agoSpringboot Integration tests improvements 07/133007/3
lukegleeson [Wed, 11 Jan 2023 09:45:53 +0000 (09:45 +0000)]
Springboot Integration tests improvements

Creation of CpsIntegrationSpecBase
Demonstration of test class implementing CpsIntegrationSpecBase in CpsPersistenceSpec
Tests use reduced liquibase steps, basic bookstore yang model and bookstore json payload

Issue-ID: CPS-1379
Signed-off-by: lukegleeson <luke.gleeson@est.tech>
Change-Id: I38202d0888808d08d85fce1aab45fc43e8b0cec3

15 months agoMerge "Update INFO.yaml Committers"
Toine Siebelink [Tue, 24 Jan 2023 17:36:46 +0000 (17:36 +0000)]
Merge "Update INFO.yaml Committers"

15 months agoUpdate INFO.yaml Committers 97/132997/1
lukegleeson [Tue, 24 Jan 2023 12:20:28 +0000 (12:20 +0000)]
Update INFO.yaml Committers

Issue-ID: CPS-1462
Signed-off-by: lukegleeson <luke.gleeson@est.tech>
Change-Id: I0b5bd7baffc03ae8c91a73926c4fa7596dba1a96

15 months agoHandle root xpaths in getDataNodes 83/132983/5
danielhanrahan [Thu, 19 Jan 2023 18:20:11 +0000 (18:20 +0000)]
Handle root xpaths in getDataNodes

Issue-ID: CPS-1458
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I64abf97317afe4335c8d04169689ee1396e75860

15 months agoMerge "Use native query to delete data nodes"
Toine Siebelink [Tue, 24 Jan 2023 10:12:16 +0000 (10:12 +0000)]
Merge "Use native query to delete data nodes"

15 months agoUse native query to delete data nodes 84/132984/5
sourabh_sourabh [Mon, 23 Jan 2023 12:51:15 +0000 (12:51 +0000)]
Use native query to delete data nodes

- Used hashcode to remove child fragment based on it's xpath
- Used native query to delete fragment by altering fragment_parent_id_fkey CONSTRAINT.

Issue-ID: CPS-1439

Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: If19c449818e18f8fd666503b7346704eeb4a95d0
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
15 months agoMerge "Further TTL increase"
Toine Siebelink [Mon, 23 Jan 2023 16:32:55 +0000 (16:32 +0000)]
Merge "Further TTL increase"

15 months agoIncrease duration for CpsPathUtilPerfTest 85/132985/2
danielhanrahan [Mon, 23 Jan 2023 14:35:44 +0000 (14:35 +0000)]
Increase duration for CpsPathUtilPerfTest

Issue-ID: CPS-1460
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I47f226c44351513b77b5cbb9b7d91e13f8bf4db6

15 months agoMerge "Added depth parameter in query nodes API."
Toine Siebelink [Mon, 23 Jan 2023 12:07:27 +0000 (12:07 +0000)]
Merge "Added depth parameter in query nodes API."

15 months agoMerge "[NCMP] Consume & Forward to client topic"
Toine Siebelink [Mon, 23 Jan 2023 11:08:07 +0000 (11:08 +0000)]
Merge "[NCMP] Consume & Forward to client topic"

15 months ago[NCMP] Consume & Forward to client topic 80/132680/11
seanbeirne [Fri, 13 Jan 2023 17:13:25 +0000 (17:13 +0000)]
[NCMP] Consume & Forward to client topic

-Consumes event from dmi-cm-events
-Immediately forwards to static topic (topic selection for events comes
later from subscription information)
-Added Kafka test
-SHOULD BE MERGED BEFORE DMI PART

Issue-ID: CPS-138
Signed-off-by: JosephKeenan <joseph.keenan@est.tech>
Change-Id: I0a426381e2c3f9173b8d3916960c05722ad4f77d
Signed-off-by: seanbeirne <sean.beirne@est.tech>
15 months agoAdded depth parameter in query nodes API. 89/132589/17
rajesh.kumar [Tue, 6 Dec 2022 11:47:24 +0000 (11:47 +0000)]
Added depth parameter in query nodes API.

Issue-ID: CPS-1381
Change-ID: I73f97f986a817d423f93a8d922dcd9647b1206bb
Signed-off-by: rajesh.kumar <rk00747546@techmahindra.com>
15 months agoFurther TTL increase 72/132972/1
mpriyank [Fri, 20 Jan 2023 12:54:32 +0000 (12:54 +0000)]
Further TTL increase

- increased TTL to 10mins or 600secs now.
- Added few more log informations.

Issue-ID: CPS-1446
Change-Id: Ie0508b426fb8d1232f0cc3c1d428a27178294323
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
16 months agoNormalize xpaths for getDataNodes 38/132938/8
danielhanrahan [Wed, 18 Jan 2023 13:14:57 +0000 (13:14 +0000)]
Normalize xpaths for getDataNodes

Issue-ID: CPS-1457
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I93d19666c168aa69da73eadbfef0fc54181aec52

16 months agoOptimal TTL value for Module Sync WD 53/132953/2
mpriyank [Thu, 19 Jan 2023 14:31:54 +0000 (14:31 +0000)]
Optimal TTL value for Module Sync WD

- setting optimal value for TTL to 120secs from 200secs.
- Since the retry parameter is configurable , not changing the same in
  patch , but can be set as per the environment we are working on

Issue-ID: CPS-1446
Change-Id: Ifaca07ad1a049a88b2609fd7a1a8e1e2fa478dd9
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
16 months agoMerge "Bug Analysis for LOCKED cmHandles"
Toine Siebelink [Wed, 18 Jan 2023 15:24:15 +0000 (15:24 +0000)]
Merge "Bug Analysis for LOCKED cmHandles"

16 months agoBug Analysis for LOCKED cmHandles 20/132920/6
mpriyank [Mon, 16 Jan 2023 14:14:41 +0000 (14:14 +0000)]
Bug Analysis for LOCKED cmHandles

- Increased TTL time from 60secs to 200secs
- Added few info level logs to analyze more

Issue-ID: CPS-1446
Change-Id: If4456040e51bd33a5bf321f48440e44af7698154
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
16 months agoAdd delete performance tests 05/132905/7
danielhanrahan [Thu, 12 Jan 2023 09:55:56 +0000 (09:55 +0000)]
Add delete performance tests

- move existing delete tests to new class
- add tests for deleting lists and list elements
- add test for deleting root node

Issue-ID: CPS-1437
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I81228aa9473ed28d550db64b28c38abb1c1016f5

16 months agoMerge "Fetch CM handles by collection of xpaths"
Sourabh Sourabh [Mon, 16 Jan 2023 08:40:23 +0000 (08:40 +0000)]
Merge "Fetch CM handles by collection of xpaths"

16 months agoMerge "Updating CmHandleStates using batch operation"
Toine Siebelink [Thu, 12 Jan 2023 16:31:07 +0000 (16:31 +0000)]
Merge "Updating CmHandleStates using batch operation"

16 months agoUpdating CmHandleStates using batch operation 98/132898/3
JosephKeenan [Wed, 11 Jan 2023 09:18:07 +0000 (09:18 +0000)]
Updating CmHandleStates using batch operation

Issue-ID: CPS-1424
Signed-off-by: JosephKeenan <joseph.keenan@est.tech>
Change-Id: Ia67db468ece4a7ab694d95cb63a954f24dd8cb55

16 months agoFetch CM handles by collection of xpaths 62/132762/4
seanbeirne [Thu, 15 Dec 2022 16:06:20 +0000 (16:06 +0000)]
Fetch CM handles by collection of xpaths

- Added FragmentRepositoryMultiPathQuery
- Removed Hibernate method for same
- Added perf. test
- Handle escaping of single qoutes in sql-data
- Increased timing for path paser performance test

Issue-ID: CPS-1422
Signed-off-by: seanbeirne <sean.beirne@est.tech>
Change-Id: Ibea12a44bffd29ed43cc1560b507d1fa7e968b8b

16 months agoCPS PATCH operation does not merge existing data 90/132890/3
lukegleeson [Mon, 9 Jan 2023 16:14:36 +0000 (16:14 +0000)]
CPS PATCH operation does not merge existing data

Altered code to add attributes and not overwrite attributes + tests

Issue-ID: CPS-1442
Signed-off-by: lukegleeson <luke.gleeson@est.tech>
Change-Id: I23c5f6a65b98ea1b05af62a38a874c228cc82067

16 months agoDocBuilderFactory one instance 84/132884/1
mpriyank [Mon, 9 Jan 2023 09:58:28 +0000 (09:58 +0000)]
DocBuilderFactory one instance

- setting the property for DocumentBuilderFactory only once.

Issue-ID: CPS-1435
Change-Id: Ib1509e86c2f5572d184945dc55f1818a7147c259
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
16 months agoLocal DocBuilderFactory fix XEE 76/132876/1
mpriyank [Fri, 6 Jan 2023 10:12:59 +0000 (10:12 +0000)]
Local DocBuilderFactory fix XEE

- local DocumentBuilderFactory fix for prevention of XML External Entity

Issue-ID: CPS-1435
Change-Id: Ib88268edc5975bf0fe4e3e56bc704f266280af4b
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
16 months agoTest XEE in SonarQube 75/132875/1
mpriyank [Thu, 5 Jan 2023 17:16:39 +0000 (17:16 +0000)]
Test XEE in SonarQube

- test by removing the attributes for transformerfactory to check if
  sonarqube gives the vulnerability or not

Issue-ID: CPS-1435
Change-Id: I087796b1bbc465655fd741f678a9b2b417d174dd
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
16 months agoLocal DocumentBuilderFactory 74/132874/1
mpriyank [Thu, 5 Jan 2023 15:35:59 +0000 (15:35 +0000)]
Local DocumentBuilderFactory

- trying to use local DocumentBuilderFactory without any condition

Issue-ID: CPS-1435
Change-Id: I209a31633427cbf6fdac3e465154a1b89b9ca89f
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
16 months agoXEE prevention with all props 71/132871/1
mpriyank [Thu, 5 Jan 2023 12:57:49 +0000 (12:57 +0000)]
XEE prevention with all props

- include all properties to prevent XEE for DocumentBuilderFactory

Issue-ID: CPS-1435
Change-Id: I5a740f34072af348fe2df282fba7babeff4299d8
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
16 months agoAdd delete performance tests 70/132870/2
danielhanrahan [Thu, 5 Jan 2023 11:03:17 +0000 (11:03 +0000)]
Add delete performance tests

Issue-ID: CPS-1437
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: If940d7633ab6f85b6e1e311af7077ce2006cfa6a

16 months agoDocument and Transformer Factory refactor 69/132869/1
mpriyank [Thu, 5 Jan 2023 11:04:34 +0000 (11:04 +0000)]
Document and Transformer Factory refactor

- Extracted out document and transformer factory code

Issue-ID: CPS-1435
Change-Id: I7c296a797871b0724f6de51833396a5d822c7d76
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
16 months agoXEE remove few property 67/132867/1
mpriyank [Wed, 4 Jan 2023 18:22:11 +0000 (18:22 +0000)]
XEE remove few property

- remove and test if it fixes the issue

Issue-ID: CPS-1435
Change-Id: Id70509494c105c3e73e8217d0bfb6334aa0cc609
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
16 months agoXEE add more property 65/132865/2
mpriyank [Wed, 4 Jan 2023 15:33:42 +0000 (15:33 +0000)]
XEE add more property

- adding few more properties to fix the problem

Issue-ID: CPS-1435
Change-Id: I2e952b38bae5dab396911ea1ce4a1125d0402369
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
16 months agoXXE prevention 46/132846/5
mpriyank [Tue, 3 Jan 2023 14:17:36 +0000 (14:17 +0000)]
XXE prevention

- xml external entity prevention in the XmlFileUtils
- setting the features only once for the document builder factory

Issue-ID: CPS-1435
Change-Id: I06f9ac4bcdb0a90262f237489c6c50d8fde33c0d
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
16 months agoMinor documentation fix 44/132844/1
mpriyank [Tue, 3 Jan 2023 11:41:07 +0000 (11:41 +0000)]
Minor documentation fix

- minor tag fix for param in documentation

Issue-ID: CPS-1435
Change-Id: I23ff1ebce54ee30cd84a58be0340ab41ad32b69a
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
16 months agoMerge "Fix SonarQube warnings"
Toine Siebelink [Fri, 23 Dec 2022 14:40:44 +0000 (14:40 +0000)]
Merge "Fix SonarQube warnings"

16 months agoFix SonarQube warnings 31/132831/2
ToineSiebelink [Fri, 23 Dec 2022 12:09:19 +0000 (12:09 +0000)]
Fix SonarQube warnings

- fixed most new warnings (required some rerfactoring and renaming)
- had to change method order (becasue private/public)
  which loosk liek a big change because gerrit cannot match thee old/new correctly :-(
- some reformatting to improve readability
- used autoclosable fro streams
- use of deprecated ODL methods will be addressed by separate commit (Lee-Anjella)

Issue-ID: CPS-475
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I4b2f74492ac9e2c644659edab1e7ff05dddcbd20

16 months agoMerge "Update Release Notes"
Toine Siebelink [Fri, 23 Dec 2022 11:41:20 +0000 (11:41 +0000)]
Merge "Update Release Notes"

16 months agoUpdate Release Notes 30/132830/1
ToineSiebelink [Fri, 23 Dec 2022 11:07:42 +0000 (11:07 +0000)]
Update Release Notes

- Added CPS-1171 and others
- Corrected mistakes

Issue-ID: CPS-1171
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I30f51c1b295e2e28fa07a2174fcc2c5db761ce5e