danielhanrahan [Mon, 20 Feb 2023 15:16:06 +0000 (15:16 +0000)]
Update performance test timings
- Tighten delete perf test timings
- Fix timings for getDataNodes
Issue-ID: CPS-1502
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ib89a70ef90b1b30e2720e48ebfb66650a2fafa12
arpitsingh [Mon, 20 Feb 2023 11:11:03 +0000 (16:41 +0530)]
RTD support to get all data nodes under an anchor
Signed-off-by: arpitsingh <as00745003@techmahindra.com>
Change-Id: I118b6475d2e8c5953b520112deb79869802bac7a
Luke Gleeson [Mon, 20 Feb 2023 09:57:05 +0000 (09:57 +0000)]
Merge "CPS-1401 Implement V2 of GET Data Node API"
arpitsingh [Mon, 9 Jan 2023 14:23:10 +0000 (19:53 +0530)]
CPS-1401 Implement V2 of GET Data Node API
- Modified the GET Data Node API so it returns all the data nodes when
xpath set to root "/"
- Fragment Repository now returns a collection of Fragment Entities
- Instead of returning only the first Fragment Entity now all fragment
entities are returned when xpath is set to root
- The Fragemnt Entities are further processed to a Collection of Data
Nodes. As opposed to singular Data Node in current implementation.
- Finally the DataRestController also returns a Collection of Data
Nodes when xpath is set to root and valid data is present
- Response body changed from JSON object to JSON Array.
- Exception handling for invalid xpath and non-existing xpath is now
done separately at persistence layer.
- Refactored code against CPS-1422
- Deprecated getDataNode method from Service and Persistence layer
- Modified V1 of Get Data Node API to use the getDataNodes method and
get the first data node from the collection returned.
- Modified NCMP to use getDataNodes method
- NCMP still does not support multiple data nodes. It retrieves the
first data node from the collection returned by getDataNodes
Signed-off-by: arpitsingh <as00745003@techmahindra.com>
Change-Id: I494a5740a53f65376d135fcb9f1e2e8900a2803e
danielhanrahan [Wed, 15 Feb 2023 19:00:37 +0000 (19:00 +0000)]
Improve batch delete schemasets performance
- Bulk delete anchors and datanodes associated with schemasets.
Improves de-registration performance by approx 10%
Issue-ID: CPS-1423
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ie38e8b4c64356bf5935d8c7a5d3f5bfa73fb1714
danielhanrahan [Tue, 14 Feb 2023 13:24:40 +0000 (13:24 +0000)]
Bulk delete schemasets in CM handle deregistration
- Batch delete schema sets in single query
- Call deleteUnusedYangResourceModules once per batch, not per CM handle
- Results for deregistering 10k: 14 mins before; 6 mins after
Issue-ID: CPS-1423
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ia3a86a0dc88677323e2f386253a99022a7f02603
Toine Siebelink [Wed, 15 Feb 2023 10:52:19 +0000 (10:52 +0000)]
Merge "Use getDataNodes (plural version) into NCMP to get CM handles Performance improvement"
ToineSiebelink [Tue, 7 Feb 2023 10:30:03 +0000 (10:30 +0000)]
Introducing Grafana
- Added profiles to docker compose: dmi-service/dmi-stub, monitoring
- NOTE: now need to add '--profile dmi-service' to enable dm-plugin (e.g. for csit)
- using container names to resolve hosts for prometheus
Issue-ID: CPS-1465
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I5d7365a3f789cc698e2702d72700a002cf03eaef
seanbeirne [Fri, 10 Feb 2023 16:42:38 +0000 (16:42 +0000)]
Use getDataNodes (plural version) into NCMP to get CM handles Performance improvement
- Code clean-up on existing classes
- Removed duplicate code
Issue-ID: CPS-1426
Signed-off-by: seanbeirne <sean.beirne@est.tech>
Change-Id: I3e2ce5b804d7a6e87838490d970cde3cd563f06b
Luke Gleeson [Mon, 13 Feb 2023 16:08:47 +0000 (16:08 +0000)]
Merge "Test for yang file archive limitation"
Toine Siebelink [Mon, 13 Feb 2023 15:18:52 +0000 (15:18 +0000)]
Merge "Generate report of Instrumentation Metrics"
halil.cakal [Thu, 9 Feb 2023 15:51:57 +0000 (15:51 +0000)]
Test for yang file archive limitation
- add new test
- fixed misleading method names
Issue-ID: CPS-1480
Change-Id: I9dd8f41d0acfc8b1c063aab65567d619670f4905
Signed-off-by: halil.cakal <halil.cakal@est.tech>
danielhanrahan [Sat, 11 Feb 2023 13:25:22 +0000 (13:25 +0000)]
Generate report of Instrumentation Metrics
Generate a tab-seperated-value report of Prometheus metrics.
By default, assume CPS/NCMP is running on localhost:8887,
and output file named metrics-YYYY-MM-DDThh:mm:ss+00:00.tsv
Issue-ID: CPS-1488
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I8bb356a1b6caacaec4a025a7d6d29cd34420e6c9
danielhanrahan [Thu, 9 Feb 2023 15:52:33 +0000 (15:52 +0000)]
Use getDataNodes (plural) in CmHandle deregistration
Issue-ID: CPS-1426
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I0ad79bf36840cdecc13898e185dc618cedafd529
Toine Siebelink [Thu, 9 Feb 2023 14:38:50 +0000 (14:38 +0000)]
Merge "Produce subscription create event for subscription-topic: cm-avc-subscription"
Priyank Maheshwari [Thu, 9 Feb 2023 11:41:55 +0000 (11:41 +0000)]
Merge "Update CmHandleState on deregistration response"
sourabh_sourabh [Fri, 3 Feb 2023 16:20:58 +0000 (16:20 +0000)]
Produce subscription create event for subscription-topic: cm-avc-subscription
- Groovy Kafka (component level) : Produce subscription create event.
Issue-ID: CPS-1394
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: Ib020c0a4458406925792fc074afd998aa09af158
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Toine Siebelink [Wed, 8 Feb 2023 17:35:12 +0000 (17:35 +0000)]
Merge "Correct calculation of zip archive size for schemaset upload"
sourabh_sourabh [Thu, 2 Feb 2023 11:22:20 +0000 (11:22 +0000)]
Use getDataNodes (plural version) into NCMP to get CM handles
- Use plural getYangModelCmHandles get YangModelCmHandles by CmHandles in batches.
Issue-ID: CPS-1426
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: Ie175b471bd98b02b572ca4a3ff699d721b5b782a
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
mpriyank [Wed, 8 Feb 2023 11:21:08 +0000 (11:21 +0000)]
Release Process Step 7
- updated pom to 3.2.3-SNAPSHOT
- updated version.properties and updated the release notes to properly
depict the correct version
Issue-ID: CPS-1476
Change-Id: Ied60194c6dd4153b58a43715cb729ad75b8e12bf
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
mpriyank [Wed, 8 Feb 2023 10:40:18 +0000 (10:40 +0000)]
Release Step 6
- adding container yaml file for the release
Issue-ID: CPS-1476
Change-Id: Ib61a14591b9fe62e96d24ae3c557f720a23217c7
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
mpriyank [Wed, 8 Feb 2023 09:58:24 +0000 (09:58 +0000)]
Release process Step 4
- adding maven stage yaml file
Issue-ID: CPS-1476
Change-Id: Iafbc5ed63cf4edc275e5b4087c154439c5c5ac55
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
mpriyank [Tue, 7 Feb 2023 17:55:10 +0000 (17:55 +0000)]
Release Steps 1 and 2
- added the jira ticket as part of step1
- copied the changes of openapi.yaml as part of step2 of release process
Issue-ID: CPS-1476
Change-Id: I877757e0396e6349b1ab5e966ae4440c4fead829
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
Arne Chrestin [Tue, 7 Feb 2023 16:39:24 +0000 (17:39 +0100)]
Correct calculation of zip archive size for schemaset upload
Correct calculation of zip archive size for schemaset upload
Issue-ID: CPS-1477
Change-Id: I73f97f986a817d423f93a8d922dcd9647b0203ab
Signed-off-by: Arne Chrestin <arne.chrestin@telekom.de>
danielhanrahan [Fri, 3 Feb 2023 18:35:16 +0000 (18:35 +0000)]
Update CmHandleState on deregistration response
Current implementation incorrectly sets CmHandleState to DELETED for
CM handles that were not deleted.
- Update CmHandleState to DELETED only for deleted CM handles
- Minor test refactoring
Issue-ID: CPS-1471
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ibf8b6d2b87d46a7633f0497b065e3d4099851fd6
Luke Gleeson [Fri, 3 Feb 2023 10:08:57 +0000 (10:08 +0000)]
Merge "Descendants description changed in openApi"
rajesh.kumar [Fri, 3 Feb 2023 06:36:40 +0000 (06:36 +0000)]
Descendants description changed in openApi
Issue-ID: CPS-1381
Change-ID: I73f97f986a817d423f93a8d922dcd9647b0203aa
Signed-off-by: rajesh.kumar <rk00747546@techmahindra.com>
danielhanrahan [Wed, 1 Feb 2023 14:48:52 +0000 (14:48 +0000)]
CmHandle batch deletion
- Use plural deleteDataNodes to remove CmHandles in batches,
falling back to individual delete on error
- Use single deleteDataNode instead of deleteListOrListElement for
individual delete
Issue-ID: CPS-1464
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: If09f22478df8703290c8fc24aa6fe2a11c90788a
emaclee [Thu, 2 Feb 2023 10:58:35 +0000 (10:58 +0000)]
Modify CSIT script to have absolute root path env
- change set up to call docker compose for cps-ncmp
- add root as a property in 'test.properties'
- root is specified to be root directory folder for the current
project
- modified docker compose init container to use ROOT env var
Issue-ID: CPS-1472
Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech>
Change-Id: Ie5ac6bbff7cb1e761501bd0db8447f86af06b95f
danielhanrahan [Tue, 24 Jan 2023 11:23:02 +0000 (11:23 +0000)]
Create plural version of deleteDataNode
- Add method to CpsDataService to batch delete data nodes and lists
- Use native queries to batch delete fragment entities by xpaths,
for data nodes and lists
- Add performance tests for batch delete
- Refactor FragmentNativeRepository
- Add single-column version of createTemporaryTable
- Renamed metric cps.data.service.datanode.batch.delete
to cps.data.service.datanode.all.delete
Issue-ID: CPS-1438
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I1851f9c7ef0b1be4bd421b3352d9697a2dd23f79
Luke Gleeson [Tue, 31 Jan 2023 11:41:54 +0000 (11:41 +0000)]
Merge "Introduce Instrumentation"
ToineSiebelink [Thu, 19 Jan 2023 16:45:58 +0000 (16:45 +0000)]
Introduce Instrumentation
- Add instrumentation related dependency
- Added Timed Instrumentation
- CPS-Service Crud methods
- CPS Yang parsing
- NCMP Registration methods
- NCMP Events handling
- Remove manual Gauge for YanResources Cache
as (better!) instrumentation is already built into the 3PP
- Sorted dependecies alphabetically (as we used to enforce, to prevent duplicates)
- Added ## P E R F O R M A N C E T E S T R E S U L T S ### mini report
- (unrelated) test improvement (because of bug that turned out to be invalid)
Reviewers: Sourabh,Priyank, Luke
Issue-ID: CPS-1457
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I34b20bece2f59488b022b8effa9470704c57be4d
seanbeirne [Mon, 30 Jan 2023 16:53:37 +0000 (16:53 +0000)]
Edit example given for ncmp datastore
Issue-ID: CPS-1468
Signed-off-by: seanbeirne <sean.beirne@est.tech>
Change-Id: Id746604fca83333f4ee186ce96ea80cb779c4ac2
lukegleeson [Mon, 30 Jan 2023 15:18:54 +0000 (15:18 +0000)]
Fix for integration test module tests
Added Fragment Native Repository to test config
Removed test profile
Issue-ID: CPS-1466
Signed-off-by: lukegleeson <luke.gleeson@est.tech>
Change-Id: Id845b713e760801be480aa9aea6cb304decd05b9
seanbeirne [Fri, 27 Jan 2023 18:09:33 +0000 (18:09 +0000)]
Update version files for release process
- Updated version.properties
- Updated release-notes
- Updated pom files
Issue-ID: CPS-1835
Signed-off-by: seanbeirne <sean.beirne@est.tech>
Change-Id: I0f837ac41d24b41d7c2b9e3dfe2dacc550c0aafa
seanbeirne [Fri, 27 Jan 2023 16:56:00 +0000 (16:56 +0000)]
Create release container yaml file for release process
Issue-ID: CPS-1385
Signed-off-by: seanbeirne <sean.beirne@est.tech>
Change-Id: I95f3f76c49ea9eb9556fb4b32dbe10dc3e8283ad
Priyank Maheshwari [Fri, 27 Jan 2023 16:54:31 +0000 (16:54 +0000)]
Merge "Create release yaml file for release process"
Sourabh Sourabh [Fri, 27 Jan 2023 16:41:37 +0000 (16:41 +0000)]
Merge "Release Note Update"
lukegleeson [Wed, 25 Jan 2023 15:06:52 +0000 (15:06 +0000)]
Release Note Update
Added Query large outputs using limit/depth/pagination
Reordered existing items in ascending Jira number order
Added missing Jiras
Issue-ID: CPS-1381
Signed-off-by: lukegleeson <luke.gleeson@est.tech>
Change-Id: I1a124db8d8a4adffc7807aecddb378876b732cd7
seanbeirne [Thu, 26 Jan 2023 14:19:31 +0000 (14:19 +0000)]
Create release yaml file for release process
Issue-ID: CPS-1385
Signed-off-by: seanbeirne <sean.beirne@est.tech>
Change-Id: I98c7cafb3b850ef6a419c87c242e1f469ccf600e
Luke Gleeson [Fri, 27 Jan 2023 13:37:17 +0000 (13:37 +0000)]
Merge "Define and onboard model for subscription events"
emaclee [Wed, 25 Jan 2023 12:18:15 +0000 (12:18 +0000)]
Define and onboard model for subscription events
- created new yang model for subscription events; added in
ncmp-service
- created new container/service to run an init container
- added script file to the container
- added Dockerfile to be the base image of the init container
- health check for CPS added as part of script
- removed previous patch modification in setup
- changed path of models and dockerfile to main project
- latest patch: changed connection to external ports of CPS
Issue-ID: CPS-1393
Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech>
Change-Id: I2367020e57b34a669dda93050eeaeede5846304e
Sourabh Sourabh [Thu, 26 Jan 2023 17:39:23 +0000 (17:39 +0000)]
Merge "NCMP De-Register response payload and status is invalid"
Toine Siebelink [Thu, 26 Jan 2023 17:36:17 +0000 (17:36 +0000)]
Merge "Disable persistence integration tests"
lukegleeson [Thu, 26 Jan 2023 17:17:56 +0000 (17:17 +0000)]
Disable persistence integration tests
Disable tests for deploy
Enable tests in Nordix jobs
Issue-ID: CPS-1466
Signed-off-by: lukegleeson <luke.gleeson@est.tech>
Change-Id: I2aa4952ea800d3499ea7380faefdcac1d8955cb4
sourabh_sourabh [Wed, 25 Jan 2023 17:13:38 +0000 (17:13 +0000)]
NCMP De-Register response payload and status is invalid
-Refactored "DELETING" and "DELETED" update state to avoid wrong
response code.
Issue-ID: CPS-1463
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: I639ba94986c4ffe48d2d42f14f6a6b4e2bd2713e
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Sourabh Sourabh [Thu, 26 Jan 2023 12:03:10 +0000 (12:03 +0000)]
Merge "Springboot Integration tests improvements"
seanbeirne [Thu, 26 Jan 2023 09:21:44 +0000 (09:21 +0000)]
Update openAPI documentation for release process
Issue-ID: CPS-1385
Signed-off-by: seanbeirne <sean.beirne@est.tech>
Change-Id: Ia0b011d4497e7d95f9b625514e8b3216575df044
lukegleeson [Wed, 11 Jan 2023 09:45:53 +0000 (09:45 +0000)]
Springboot Integration tests improvements
Creation of CpsIntegrationSpecBase
Demonstration of test class implementing CpsIntegrationSpecBase in CpsPersistenceSpec
Tests use reduced liquibase steps, basic bookstore yang model and bookstore json payload
Issue-ID: CPS-1379
Signed-off-by: lukegleeson <luke.gleeson@est.tech>
Change-Id: I38202d0888808d08d85fce1aab45fc43e8b0cec3
Toine Siebelink [Tue, 24 Jan 2023 17:36:46 +0000 (17:36 +0000)]
Merge "Update INFO.yaml Committers"
lukegleeson [Tue, 24 Jan 2023 12:20:28 +0000 (12:20 +0000)]
Update INFO.yaml Committers
Issue-ID: CPS-1462
Signed-off-by: lukegleeson <luke.gleeson@est.tech>
Change-Id: I0b5bd7baffc03ae8c91a73926c4fa7596dba1a96
danielhanrahan [Thu, 19 Jan 2023 18:20:11 +0000 (18:20 +0000)]
Handle root xpaths in getDataNodes
Issue-ID: CPS-1458
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I64abf97317afe4335c8d04169689ee1396e75860
Toine Siebelink [Tue, 24 Jan 2023 10:12:16 +0000 (10:12 +0000)]
Merge "Use native query to delete data nodes"
sourabh_sourabh [Mon, 23 Jan 2023 12:51:15 +0000 (12:51 +0000)]
Use native query to delete data nodes
- Used hashcode to remove child fragment based on it's xpath
- Used native query to delete fragment by altering fragment_parent_id_fkey CONSTRAINT.
Issue-ID: CPS-1439
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: If19c449818e18f8fd666503b7346704eeb4a95d0
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Toine Siebelink [Mon, 23 Jan 2023 16:32:55 +0000 (16:32 +0000)]
Merge "Further TTL increase"
danielhanrahan [Mon, 23 Jan 2023 14:35:44 +0000 (14:35 +0000)]
Increase duration for CpsPathUtilPerfTest
Issue-ID: CPS-1460
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I47f226c44351513b77b5cbb9b7d91e13f8bf4db6
Toine Siebelink [Mon, 23 Jan 2023 12:07:27 +0000 (12:07 +0000)]
Merge "Added depth parameter in query nodes API."
Toine Siebelink [Mon, 23 Jan 2023 11:08:07 +0000 (11:08 +0000)]
Merge "[NCMP] Consume & Forward to client topic"
seanbeirne [Fri, 13 Jan 2023 17:13:25 +0000 (17:13 +0000)]
[NCMP] Consume & Forward to client topic
-Consumes event from dmi-cm-events
-Immediately forwards to static topic (topic selection for events comes
later from subscription information)
-Added Kafka test
-SHOULD BE MERGED BEFORE DMI PART
Issue-ID: CPS-138
Signed-off-by: JosephKeenan <joseph.keenan@est.tech>
Change-Id: I0a426381e2c3f9173b8d3916960c05722ad4f77d
Signed-off-by: seanbeirne <sean.beirne@est.tech>
rajesh.kumar [Tue, 6 Dec 2022 11:47:24 +0000 (11:47 +0000)]
Added depth parameter in query nodes API.
Issue-ID: CPS-1381
Change-ID: I73f97f986a817d423f93a8d922dcd9647b1206bb
Signed-off-by: rajesh.kumar <rk00747546@techmahindra.com>
mpriyank [Fri, 20 Jan 2023 12:54:32 +0000 (12:54 +0000)]
Further TTL increase
- increased TTL to 10mins or 600secs now.
- Added few more log informations.
Issue-ID: CPS-1446
Change-Id: Ie0508b426fb8d1232f0cc3c1d428a27178294323
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
danielhanrahan [Wed, 18 Jan 2023 13:14:57 +0000 (13:14 +0000)]
Normalize xpaths for getDataNodes
Issue-ID: CPS-1457
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I93d19666c168aa69da73eadbfef0fc54181aec52
mpriyank [Thu, 19 Jan 2023 14:31:54 +0000 (14:31 +0000)]
Optimal TTL value for Module Sync WD
- setting optimal value for TTL to 120secs from 200secs.
- Since the retry parameter is configurable , not changing the same in
patch , but can be set as per the environment we are working on
Issue-ID: CPS-1446
Change-Id: Ifaca07ad1a049a88b2609fd7a1a8e1e2fa478dd9
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
Toine Siebelink [Wed, 18 Jan 2023 15:24:15 +0000 (15:24 +0000)]
Merge "Bug Analysis for LOCKED cmHandles"
mpriyank [Mon, 16 Jan 2023 14:14:41 +0000 (14:14 +0000)]
Bug Analysis for LOCKED cmHandles
- Increased TTL time from 60secs to 200secs
- Added few info level logs to analyze more
Issue-ID: CPS-1446
Change-Id: If4456040e51bd33a5bf321f48440e44af7698154
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
danielhanrahan [Thu, 12 Jan 2023 09:55:56 +0000 (09:55 +0000)]
Add delete performance tests
- move existing delete tests to new class
- add tests for deleting lists and list elements
- add test for deleting root node
Issue-ID: CPS-1437
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I81228aa9473ed28d550db64b28c38abb1c1016f5
Sourabh Sourabh [Mon, 16 Jan 2023 08:40:23 +0000 (08:40 +0000)]
Merge "Fetch CM handles by collection of xpaths"
Toine Siebelink [Thu, 12 Jan 2023 16:31:07 +0000 (16:31 +0000)]
Merge "Updating CmHandleStates using batch operation"
JosephKeenan [Wed, 11 Jan 2023 09:18:07 +0000 (09:18 +0000)]
Updating CmHandleStates using batch operation
Issue-ID: CPS-1424
Signed-off-by: JosephKeenan <joseph.keenan@est.tech>
Change-Id: Ia67db468ece4a7ab694d95cb63a954f24dd8cb55
seanbeirne [Thu, 15 Dec 2022 16:06:20 +0000 (16:06 +0000)]
Fetch CM handles by collection of xpaths
- Added FragmentRepositoryMultiPathQuery
- Removed Hibernate method for same
- Added perf. test
- Handle escaping of single qoutes in sql-data
- Increased timing for path paser performance test
Issue-ID: CPS-1422
Signed-off-by: seanbeirne <sean.beirne@est.tech>
Change-Id: Ibea12a44bffd29ed43cc1560b507d1fa7e968b8b
lukegleeson [Mon, 9 Jan 2023 16:14:36 +0000 (16:14 +0000)]
CPS PATCH operation does not merge existing data
Altered code to add attributes and not overwrite attributes + tests
Issue-ID: CPS-1442
Signed-off-by: lukegleeson <luke.gleeson@est.tech>
Change-Id: I23c5f6a65b98ea1b05af62a38a874c228cc82067
mpriyank [Mon, 9 Jan 2023 09:58:28 +0000 (09:58 +0000)]
DocBuilderFactory one instance
- setting the property for DocumentBuilderFactory only once.
Issue-ID: CPS-1435
Change-Id: Ib1509e86c2f5572d184945dc55f1818a7147c259
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
mpriyank [Fri, 6 Jan 2023 10:12:59 +0000 (10:12 +0000)]
Local DocBuilderFactory fix XEE
- local DocumentBuilderFactory fix for prevention of XML External Entity
Issue-ID: CPS-1435
Change-Id: Ib88268edc5975bf0fe4e3e56bc704f266280af4b
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
mpriyank [Thu, 5 Jan 2023 17:16:39 +0000 (17:16 +0000)]
Test XEE in SonarQube
- test by removing the attributes for transformerfactory to check if
sonarqube gives the vulnerability or not
Issue-ID: CPS-1435
Change-Id: I087796b1bbc465655fd741f678a9b2b417d174dd
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
mpriyank [Thu, 5 Jan 2023 15:35:59 +0000 (15:35 +0000)]
Local DocumentBuilderFactory
- trying to use local DocumentBuilderFactory without any condition
Issue-ID: CPS-1435
Change-Id: I209a31633427cbf6fdac3e465154a1b89b9ca89f
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
mpriyank [Thu, 5 Jan 2023 12:57:49 +0000 (12:57 +0000)]
XEE prevention with all props
- include all properties to prevent XEE for DocumentBuilderFactory
Issue-ID: CPS-1435
Change-Id: I5a740f34072af348fe2df282fba7babeff4299d8
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
danielhanrahan [Thu, 5 Jan 2023 11:03:17 +0000 (11:03 +0000)]
Add delete performance tests
Issue-ID: CPS-1437
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: If940d7633ab6f85b6e1e311af7077ce2006cfa6a
mpriyank [Thu, 5 Jan 2023 11:04:34 +0000 (11:04 +0000)]
Document and Transformer Factory refactor
- Extracted out document and transformer factory code
Issue-ID: CPS-1435
Change-Id: I7c296a797871b0724f6de51833396a5d822c7d76
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
mpriyank [Wed, 4 Jan 2023 18:22:11 +0000 (18:22 +0000)]
XEE remove few property
- remove and test if it fixes the issue
Issue-ID: CPS-1435
Change-Id: Id70509494c105c3e73e8217d0bfb6334aa0cc609
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
mpriyank [Wed, 4 Jan 2023 15:33:42 +0000 (15:33 +0000)]
XEE add more property
- adding few more properties to fix the problem
Issue-ID: CPS-1435
Change-Id: I2e952b38bae5dab396911ea1ce4a1125d0402369
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
mpriyank [Tue, 3 Jan 2023 14:17:36 +0000 (14:17 +0000)]
XXE prevention
- xml external entity prevention in the XmlFileUtils
- setting the features only once for the document builder factory
Issue-ID: CPS-1435
Change-Id: I06f9ac4bcdb0a90262f237489c6c50d8fde33c0d
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
mpriyank [Tue, 3 Jan 2023 11:41:07 +0000 (11:41 +0000)]
Minor documentation fix
- minor tag fix for param in documentation
Issue-ID: CPS-1435
Change-Id: I23ff1ebce54ee30cd84a58be0340ab41ad32b69a
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
Toine Siebelink [Fri, 23 Dec 2022 14:40:44 +0000 (14:40 +0000)]
Merge "Fix SonarQube warnings"
ToineSiebelink [Fri, 23 Dec 2022 12:09:19 +0000 (12:09 +0000)]
Fix SonarQube warnings
- fixed most new warnings (required some rerfactoring and renaming)
- had to change method order (becasue private/public)
which loosk liek a big change because gerrit cannot match thee old/new correctly :-(
- some reformatting to improve readability
- used autoclosable fro streams
- use of deprecated ODL methods will be addressed by separate commit (Lee-Anjella)
Issue-ID: CPS-475
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I4b2f74492ac9e2c644659edab1e7ff05dddcbd20
Toine Siebelink [Fri, 23 Dec 2022 11:41:20 +0000 (11:41 +0000)]
Merge "Update Release Notes"
ToineSiebelink [Fri, 23 Dec 2022 11:07:42 +0000 (11:07 +0000)]
Update Release Notes
- Added CPS-1171 and others
- Corrected mistakes
Issue-ID: CPS-1171
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I30f51c1b295e2e28fa07a2174fcc2c5db761ce5e
ToineSiebelink [Fri, 23 Dec 2022 10:11:38 +0000 (10:11 +0000)]
Organize performance test
- Separated CpsPathUtil performance tests in dedicated file
- Used smaller sample to speed up CpsPathUtil test
- Increased margin on CpsModuleReferenceRepository performance test to rpevent accidental failures
- Added profiles to cps path parser module
- Use Regex to include any (future) perforamcne test using same naming
Issue-ID: CPS-1421
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I9da1614b6f067e946a4f088c6417479fa6c71988
Toine Siebelink [Thu, 22 Dec 2022 16:01:28 +0000 (16:01 +0000)]
Merge "Temp Table Creation improvements"
Toine Siebelink [Thu, 22 Dec 2022 15:46:16 +0000 (15:46 +0000)]
Merge "XML content on create anchors node support"
ToineSiebelink [Thu, 22 Dec 2022 14:47:07 +0000 (14:47 +0000)]
Temp Table Creation improvements
- extracted methods for more generic temp table creation
- optimized
- added perf test on use case that depends on temp table creation
- had some doubts about stopwatch use in recent tests, but was all OK, just renamed some
Issue-ID: CPS-1422
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I22cabb9b0ba1b0aa8576a2d756d77af46eebc1b0
Toine Siebelink [Thu, 22 Dec 2022 15:12:50 +0000 (15:12 +0000)]
Merge "Add fix for posting nodes with xPath with '/'"
Michal Jagiello [Tue, 13 Dec 2022 07:40:19 +0000 (07:40 +0000)]
XML content on create anchors node support
Add XML content type support on anchor node creation.
Issue-ID: CPS-1257
Change-Id: I7e7a9a1961b6e81de93a4e32e842b47f8a163a09
Signed-off-by: Michal Jagiello <michal.jagiello@t-mobile.pl>
Signed-off-by: Lee Anjella Macabuhay <lee.anjella.macabuhay@est.tech>
Toine Siebelink [Thu, 22 Dec 2022 12:22:19 +0000 (12:22 +0000)]
Merge "Consume Subscription Creation Event"
seanbeirne [Tue, 6 Dec 2022 11:12:18 +0000 (11:12 +0000)]
Consume Subscription Creation Event
Issue-ID: CPS-1392
Signed-off-by: seanbeirne <sean.beirne@est.tech>
Change-Id: I0a5a8c256319a1a2944ee6606db1c14b50e8f8e4
mpriyank [Thu, 22 Dec 2022 10:16:28 +0000 (10:16 +0000)]
CSIT fix to use correct lib
- Explicitly setting pylibs.txt file to use 3.0.0 version of
robotframework-selenium2library
- Also updating the script to be read from WORKSPACE.
Change-Id: Ic66c5357f53e73678aac439d095d318dbd03c333
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
emaclee [Wed, 21 Dec 2022 09:29:24 +0000 (09:29 +0000)]
Add fix for posting nodes with xPath with '/'
- YangUtils method changed from using REGEX to cps path parser
- unit test added for cps path util
Issue-ID: CPS-1433
Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech>
Change-Id: Ibb9efdd09423f9bade4a4a557d7d9ed49aa44ef4
ToineSiebelink [Wed, 21 Dec 2022 09:29:54 +0000 (09:29 +0000)]
CpsPath Query Optimization
- Optimized CpsPathqueries with descendants that only care about the xpath (no attribuets checks)
- Use native query with regular expression for target xpath and descendants
- Refactored so existing sql-geneartion code can be re-used in different repository implementations
- Adjusted related performance test expectations
Issue-ID: CPS-1421
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I3a807a14478c4b3272a5335d31c9aa3615eb2bee
Joseph Keenan [Wed, 21 Dec 2022 11:42:39 +0000 (11:42 +0000)]
Merge "API versioning supported and added different versions for POST APIs"
ToineSiebelink [Tue, 20 Dec 2022 11:32:26 +0000 (11:32 +0000)]
Check preformance impacts
-Added a basic perf. test to process 200K CPS paths using parser
Issue-ID: CPS-1409
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: Ibf8aed53506d28ba08f7166a44bd6fe9aefa5952
ToineSiebelink [Mon, 19 Dec 2022 14:19:29 +0000 (14:19 +0000)]
Fix Delete uses case with '/' in path
-Extend and use cpsPath parser (util) classes instead of regex
Issue-ID: CPS-1409
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I230c9eb71cc34264983830b39149511b95c4b4a6
Code Review / cps.git / log