cps.git
8 months agoMerge "CI: Add test rtdv3 GHA workflow"
Toine Siebelink [Mon, 21 Aug 2023 16:43:52 +0000 (16:43 +0000)]
Merge "CI: Add test rtdv3 GHA workflow"

8 months agoMerge "Cm Subscription: PENDING logic handling in NCMP"
Luke Gleeson [Mon, 21 Aug 2023 10:20:09 +0000 (10:20 +0000)]
Merge "Cm Subscription: PENDING logic handling in NCMP"

8 months agoCI: Add test rtdv3 GHA workflow 74/135774/1
Jessica Wagantall [Fri, 18 Aug 2023 17:48:11 +0000 (10:48 -0700)]
CI: Add test rtdv3 GHA workflow

This is not a voting workflow, but rather it replicates what
Jenkins does in GHA.
The idea is to have this workflow tested first before we deprecate
the global-jjb rtdv3 job and switch to GHA.

Issue-ID: CIMAN-33
Change-Id: I83742b2fb9d16785a6932c93a2c701f13527307b
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
8 months agoMerge "Device heartbeat listener"
Luke Gleeson [Fri, 18 Aug 2023 13:17:32 +0000 (13:17 +0000)]
Merge "Device heartbeat listener"

8 months agoMerge "CPS-Core : Expose a java interface to update schema set"
Priyank Maheshwari [Fri, 18 Aug 2023 12:55:49 +0000 (12:55 +0000)]
Merge "CPS-Core : Expose a java interface to update schema set"

8 months agoDevice heartbeat listener 09/135609/10
mpriyank [Wed, 26 Jul 2023 16:33:35 +0000 (17:33 +0100)]
Device heartbeat listener

- Infrastructure code to have the kafka listener and distributed set in
  place
- performance tested locally
- testware added

Issue-ID: CPS-1642
Change-Id: I775dbe6e6b520b8777faa08610db439877757572
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
8 months agoCPS-Core : Expose a java interface to update schema set 28/135728/4
sourabh_sourabh [Tue, 15 Aug 2023 11:15:33 +0000 (12:15 +0100)]
CPS-Core : Expose a java interface to update schema set

- Exposed an interface to update anchor by schema set name.
- New interface is implemented into RI model.
- New native query is exposed to update id with given schema set name.
- A new integration test is written to test new interface into cps core.

Issue-ID: CPS-1800

Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: Ibf44712e11b53cb6673b04b9e3fd864321c90839
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
8 months agoModify data operation schema to accommodate additional information 70/135770/2
sourabh_sourabh [Fri, 18 Aug 2023 09:56:42 +0000 (10:56 +0100)]
Modify data operation schema to accommodate additional information

- Code is modify to add resourceIdentifier and option to data operation kafka response for status code 100, 101, 102 and 103.

Issue-ID: CPS-1837
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: I1f7870d9d373f69de03240700b9d74be46fbfb6c
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
8 months agoMerge "Bring back coverage to 1.00 for ncmp-rest"
Luke Gleeson [Fri, 18 Aug 2023 10:52:17 +0000 (10:52 +0000)]
Merge "Bring back coverage to 1.00 for ncmp-rest"

8 months agoMerge "Update of top-level data node fails with SQL error"
Sourabh Sourabh [Fri, 18 Aug 2023 10:52:15 +0000 (10:52 +0000)]
Merge "Update of top-level data node fails with SQL error"

8 months agoBring back coverage to 1.00 for ncmp-rest 71/135771/1
mpriyank [Fri, 18 Aug 2023 10:34:08 +0000 (11:34 +0100)]
Bring back coverage to 1.00 for ncmp-rest

Issue-ID: CPS-1832
Change-Id: I1704060d0cd56d7c2d2fb7b932a8142d3940120b
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
8 months agoMerge "Add code coverage for missing branches in pagination API(ep4)"
Luke Gleeson [Fri, 18 Aug 2023 10:03:31 +0000 (10:03 +0000)]
Merge "Add code coverage for missing branches in pagination API(ep4)"

8 months agoUpdate of top-level data node fails with SQL error 64/135764/4
danielhanrahan [Thu, 17 Aug 2023 14:43:34 +0000 (15:43 +0100)]
Update of top-level data node fails with SQL error

The error is caused by the fetch-descendants code using
ResultSet.getLong which returns a 'long' primitive instead
of a 'Long' object. Thus a parent ID of 'NULL' becomes '0',
which causes an error during update. To preserve the NULL
value, ResultSet.getObject must be used.

Issue-ID: CPS-1841
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ida6837a04954cd3c23f0f2faabd7d0712d8ee19d

8 months agoAdd code coverage for missing branches in pagination API(ep4) 00/135700/2
rajesh.kumar [Fri, 11 Aug 2023 08:47:49 +0000 (14:17 +0530)]
Add code coverage for missing branches in pagination API(ep4)

Add new test cases for missing branch covereage in Pagiantion API

- Added test cases in cps-rest to cover the scenario of missing
  page index and missing page size

Issue-ID:CPS-1835
Change-ID: I73f97f986a817d423f93a8d922dcd9647b1108bc
Signed-off-by: rajesh.kumar <rk00747546@techmahindra.com>
8 months agoCm Subscription: PENDING logic handling in NCMP 22/135722/5
mpriyank [Thu, 10 Aug 2023 22:09:38 +0000 (23:09 +0100)]
Cm Subscription: PENDING logic handling in NCMP

- Remove PENDING state from dmi schema
- Modify ncmp out event mapper to categorize
  response per details
- Rename class and method names as well as
  in unit tests
- resolved the merge conflicts

Issue-ID: CPS-1830
Change-Id: I5b7f523f546ec9940c246bd286586fdeba2f892e
Signed-off-by: halil.cakal <halil.cakal@est.tech>
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
8 months agoMerge "Fixing sonar issues"
Sourabh Sourabh [Thu, 17 Aug 2023 08:37:47 +0000 (08:37 +0000)]
Merge "Fixing sonar issues"

8 months agoModify data operation schema to accommodate additional information 57/135757/2
sourabh_sourabh [Wed, 16 Aug 2023 12:16:25 +0000 (13:16 +0100)]
Modify data operation schema to accommodate additional information

- Data operation schema is updated to add resourceIdentifier and
  options.
- dmi plugin stub is modified to add both the attributes into kafka
  response message.

Issue-ID:CPS-1837

Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: I624c32d07ccf396f3090766049622f43f5c3f44f

8 months agoFixing sonar issues 56/135756/1
danielhanrahan [Wed, 16 Aug 2023 09:39:48 +0000 (10:39 +0100)]
Fixing sonar issues

Issue-ID: CPS-475
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I7623a8301fd4567bf96c605eb26bf6c02b1d3c76

8 months agoUpdate performance test timing for warmup 30/135730/1
danielhanrahan [Tue, 15 Aug 2023 13:12:30 +0000 (14:12 +0100)]
Update performance test timing for warmup

Incorrect limit on the warmup results in performance pipeline
failing, as the first test that runs must include the time taken
to write initial data.

Issue-ID: CPS-1811
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I82353fcd5d0904f458808b32f4f01597ba25a688

8 months agoGeneric Cloud Event Mapper 20/135720/3
mpriyank [Thu, 10 Aug 2023 13:27:51 +0000 (14:27 +0100)]
Generic Cloud Event Mapper

- Introduced a generic event cloud mapper to be used all over the
  codebase
- Added as a utility so we dont have to inject it as spring bean
- Refactored code and added tests
- Incorporated review comments

Issue-ID: CPS-1827
Change-Id: I8dab914791e01bbf6492ce96e7ba6eaa8f75a6cb
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
9 months agoRefactoring Subscription Create LCM use case 13/135713/6
mpriyank [Tue, 8 Aug 2023 14:27:18 +0000 (15:27 +0100)]
Refactoring Subscription Create LCM use case

- Client to NCMP: CmSubscriptionNcmpInEvent
- NCMP to DMI: CmSubscriptionDmiInEvent
- DMI to NCMP: CmSubscriptionDmiOutEvent
- NCMP to Client: CmSubscriptionNcmpOutEvent
- code package changed from avcsubscription to cmsubscription
- Other classes name as per the events naming above
- Test classes refactored
- NO LOGIC changes incorporated in this patch

Issue-ID: CPS-1831
Change-Id: Id5ad5f799007deaaf6d6fc0f402c130339263d09
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
9 months agoModify DmiDataOperationRequest to make it identical as DataOperationRequest 14/135714/3
sourabh_sourabh [Tue, 8 Aug 2023 18:30:41 +0000 (19:30 +0100)]
Modify DmiDataOperationRequest to make it identical as DataOperationRequest

- Modified DmiDataOperationRequest attribute from operationType to
  operation.
- Wrapped operation details to parent attribute operationd array.

Issue-ID: CPS-1833

Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: I79aa943e10e1c32c3252a6eb4375c14496476f8b
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
9 months agoMerge "Subscription Creation: Fixes for code review after demo"
Priyank Maheshwari [Tue, 8 Aug 2023 08:48:40 +0000 (08:48 +0000)]
Merge "Subscription Creation: Fixes for code review after demo"

9 months agoUpdate performance test timings for larger dataset 75/135675/3
danielhanrahan [Fri, 4 Aug 2023 13:29:55 +0000 (14:29 +0100)]
Update performance test timings for larger dataset

- Populate 3 anchors with 1000 openroadm devices nodes for
  read and query tests (over 250,000 fragments)
- Increase update perf tests to update 100 out of 1000 devices
- Increase delete perf tests to delete 100 out of 300 devices
- Remove bookstore data from performance tests
- Update test timings

Issue-ID: CPS-1811
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Iaa36694be907278e772ae729a85510ea9d004fd1

9 months agoSubscription Creation: Fixes for code review after demo 39/135639/9
halil.cakal [Tue, 1 Aug 2023 15:35:58 +0000 (16:35 +0100)]
Subscription Creation: Fixes for code review after demo

- Change missleading subscription details in testware
- Change datastore check to comply with CPS enum
- Remove redundant CloudConstructionException
- Change exception handling in cloud event mappers
  to avoid loss of information
- Remove exception handling from scheduler service
  as discussed in the meeting

Issue-ID: CPS-1732
Change-Id: I9fee2eafd4db97a0eed80e39219463c904f5a980
Signed-off-by: halil.cakal <halil.cakal@est.tech>
9 months agoUpdate coverage ratio to 0.99 73/135673/1
mpriyank [Fri, 4 Aug 2023 11:42:21 +0000 (12:42 +0100)]
Update coverage ratio to 0.99

- in order to stabilize the builds , updating the coverage ratio for
  cps-ncmp-rest package to 0.99 from 1.00
- Will bring back to 1.00 once the necessary fixes are in place.

Issue-ID: CPS-1832
Change-Id: If0f106d7c1a44294ed67c83a049f667d9ce13b31
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
9 months agoAbility to disable sending auth header 34/135634/6
mpriyank [Tue, 1 Aug 2023 14:36:27 +0000 (15:36 +0100)]
Ability to disable sending auth header

- introduced configurable parameter to disable sending auth header in
  the request to dmi-plugin
- default is to include the auth header
- DMI_AUTH_ENABLED flag can be set to true/false to control the
  behaviour
- Also added the env variables to the CSITs
- updated release-notes

Issue-ID: CPS-1819
Change-Id: If97c06e33f06e391d2190b7300d73210ed2f4e27
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
9 months agoFix: integration test that is for add and delete top level list element 55/135655/1
halil.cakal [Thu, 3 Aug 2023 15:12:33 +0000 (16:12 +0100)]
Fix: integration test that is for add and delete top level list element

Issue-ID: CPS-1605
Change-Id: Ia6ba73d5e4930bcc547cf24d06c50c681c3a7452
Signed-off-by: halil.cakal <halil.cakal@est.tech>
9 months agoMerge "Support pagination in query across all anchors(ep4)"
Luke Gleeson [Thu, 3 Aug 2023 13:13:47 +0000 (13:13 +0000)]
Merge "Support pagination in query across all anchors(ep4)"

9 months agoSupport pagination in query across all anchors(ep4) 12/134312/38
rajesh.kumar [Tue, 25 Apr 2023 06:28:35 +0000 (11:58 +0530)]
Support pagination in query across all anchors(ep4)

Add pagination query parameters in query across all anchors API
pagination parameters (pageIndex and pageSize) are optional
default is to query all fragments
each pageSize represents number of records(number of anchors)
TotalRecords is returned in response header to find number of pages.

- If pagination option is provided in request then query number of
  anchors equal to pageSize. pageIndex is used for setting offset.

- return number of records(one anchor per record) as per pagesize
  and pageSize

Issue-ID: CPS-1605
Change-ID: I73f97f986a817d423f93a8d922dcd9647b2504bc
Signed-off-by: rajesh.kumar <rk00747546@techmahindra.com>
9 months agoMerge "Fix: Make bookstore data consistent"
Toine Siebelink [Tue, 1 Aug 2023 12:20:51 +0000 (12:20 +0000)]
Merge "Fix: Make bookstore data consistent"

9 months agoAdd 'direct' keyword to descendants option to query direct children (ep1) 06/135606/2
Rudrangi Anupriya [Mon, 31 Jul 2023 11:20:32 +0000 (16:50 +0530)]
Add 'direct' keyword to descendants option to query direct children (ep1)

-added 'direct' keyword also to Fetch Descendants Option along with 'all' and 'none' to Query direct child.
-added unit tests to test direct keyword

Issue-ID: CPS-1784
Change-Id: Icb7f59fbeebb03703626132d6d5d2cfde0e5ab4d
Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>
9 months agoMerge "Use constants for magic numbers in perf tests"
Toine Siebelink [Mon, 31 Jul 2023 10:31:59 +0000 (10:31 +0000)]
Merge "Use constants for magic numbers in perf tests"

9 months agoIncrease code coverage in cps-service module 76/135576/3
ToineSiebelink [Wed, 26 Jul 2023 16:49:02 +0000 (17:49 +0100)]
Increase code coverage in cps-service module

- After last rebase I had to remove 3 unused recent cloud eventd specific exceptions/constructors
- Moved the only used new exception from SPI to the relevant util package
(please NOTE not all exceptions belong in SPI and always question need for new exception
when there is no specific handling, try to use standard or existign CPS exception instead!)
- Increased cps-service module (line) coverage from 95 to 100%
- Added tests for missing exceptions (handling i.e. thrown up)
- Removed incorrect SPI defined OperationNotYetSupportedException
(replaced with standard java exception instead)
- Fixed some legacy issues with existign test classes I modified
(unnecessary setup, conventions etc)
- Increased coverage for DataNodeBuilder
- Added or modified test to include more spi models
- Added tests for Hazelcast Configs
- Added more tests for json object mapper
- Added test and fixed error handling in YangUtils/XmlFileUtils
(it was incorrectly converting a config exception to a data validation exception)

Issue-ID: CPS-475
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I5852ba01bc5b33ae361b8f29daae9868f05baa35

9 months agoUse constants for magic numbers in perf tests 78/135578/5
danielhanrahan [Thu, 27 Jul 2023 09:44:38 +0000 (10:44 +0100)]
Use constants for magic numbers in perf tests

Issue-ID: CPS-1811
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ic87bcb1847b97a0e8c59b2e3636e9ccd66f814ae

9 months agoFix pom.xml version 00/135600/1
leventecsanyi [Fri, 28 Jul 2023 15:36:55 +0000 (17:36 +0200)]
Fix pom.xml version

  - corrected version number

Issue-ID: CPS-1817
Signed-off-by: Levente Csanyi <levente.csanyi@est.tech>
Change-Id: Id97b3d55e7342619804b8e007238718b56ef3120

9 months agoAdd 'direct' keyword to descendants option to query direct children (ep1) 25/135525/5
Rudrangi Anupriya [Fri, 28 Jul 2023 08:58:54 +0000 (14:28 +0530)]
Add 'direct' keyword to descendants option to query direct children (ep1)

-added 'direct' keyword also to Fetch Descendants Option along with 'all' and 'none' to Query direct child.
-added unit tests to test direct keyword

Issue-ID:CPS-1784
Change-Id: Iab7f59fbeebb03703626132c6d5c2afde0e5ab4d
Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>
9 months agoMerge "Improve 32K limit tests"
Toine Siebelink [Thu, 27 Jul 2023 15:14:19 +0000 (15:14 +0000)]
Merge "Improve 32K limit tests"

9 months agoMerge "Add write performance tests"
Toine Siebelink [Thu, 27 Jul 2023 15:14:02 +0000 (15:14 +0000)]
Merge "Add write performance tests"

9 months agoUpdate RTD (CPS-1515: Spike: Support Multiple CM-Handles for NCMP Get Operation) 54/135554/5
sourabh_sourabh [Tue, 25 Jul 2023 09:58:52 +0000 (10:58 +0100)]
Update RTD (CPS-1515: Spike: Support Multiple CM-Handles for NCMP Get Operation)

- Added information about data operation NCMP endpoint and kafka event
  details.
- Re-structured "CPS-Events" and "Modeling" page.

Issue-ID:CPS-1785

Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: I2b077ccd7ccd6144a8f9f2183d927f922075f532
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
9 months agoImprove 32K limit tests 69/135569/3
danielhanrahan [Wed, 26 Jul 2023 12:19:50 +0000 (13:19 +0100)]
Improve 32K limit tests

Some operations were previously limited to 32,768 parameters,
such as getDataNodesForMultipleXpaths. After this limit was lifted,
the tests only verified that database exceptions were not thrown.
This commit improves the tests by verifying that the operations
return or delete the expected data.

- Create 33,000 data nodes
- Verify that getDataNodesForMultipleXpaths returns 33,000 nodes
- Verify than deleteDataNodes deletes 33,000 nodes

Issue-ID: CPS-1811
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I7029ea0cf1d4e95c5ca814ca62c222fc5b9ddc7e

9 months agoMerge "Allow getDataNodes to read whole lists(ep 1)"
Sourabh Sourabh [Thu, 27 Jul 2023 12:16:35 +0000 (12:16 +0000)]
Merge "Allow getDataNodes to read whole lists(ep 1)"

9 months agoAllow getDataNodes to read whole lists(ep 1) 59/134759/17
danielhanrahan [Thu, 18 May 2023 09:18:10 +0000 (10:18 +0100)]
Allow getDataNodes to read whole lists(ep 1)

 - getDataNodes can now retrieve list data nodes.

Issue-ID: CPS-1696
Signed-off-by: arpitsingh <as00745003@techmahindra.com>
Change-Id: I320a368d6cb73599d3f7c13fe9b8ab7d0cc69470

9 months agoMerge "Subscription Creation: NCMP to Client CloudEvent transformation"
Luke Gleeson [Thu, 27 Jul 2023 10:20:54 +0000 (10:20 +0000)]
Merge "Subscription Creation: NCMP to Client CloudEvent transformation"

9 months agoMerge "Performance test of reading non-existing datanodes"
Toine Siebelink [Thu, 27 Jul 2023 09:56:32 +0000 (09:56 +0000)]
Merge "Performance test of reading non-existing datanodes"

9 months agoFix: Make bookstore data consistent 47/135447/2
Arpit Singh [Thu, 27 Jul 2023 09:18:30 +0000 (14:48 +0530)]
Fix: Make bookstore data consistent

 - Removed list named invoice from all bookstore files
 - Added a new parent list as bookstore-address
 - Refactored tests

Issue-ID: CPS-1813
Signed-off-by: Arpit Singh <as00745003@techmahindra.com>
Change-Id: I5c7e83e1ee0ad9ac1d94d1906c2361c7a00d49af

9 months agoAdd write performance tests 73/135573/3
danielhanrahan [Wed, 26 Jul 2023 08:50:55 +0000 (09:50 +0100)]
Add write performance tests

- Add perf test for openroadm (linear time)
- Add perf test for bookstore (exponential time!)

Issue-ID: CPS-1811
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I9c1772a9e9684707f2dbb32b2c361ec248fb1c60

9 months agoPerformance test of reading non-existing datanodes 71/135571/1
danielhanrahan [Wed, 26 Jul 2023 15:42:39 +0000 (16:42 +0100)]
Performance test of reading non-existing datanodes

Issue-ID: CPS-1811
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I183129bb5b0a2d620b87e9b36f3113fe8e169798

9 months agoPerformance test of deleting non-existing datanodes 70/135570/1
danielhanrahan [Wed, 26 Jul 2023 15:56:10 +0000 (16:56 +0100)]
Performance test of deleting non-existing datanodes

Issue-ID: CPS-1811
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ifae635f45d8ec1990452ade958253972fde68415

9 months agoSubscription Creation: NCMP to Client CloudEvent transformation 13/135413/11
halil.cakal [Thu, 13 Jul 2023 10:28:18 +0000 (11:28 +0100)]
Subscription Creation: NCMP to Client CloudEvent transformation

- Delete legacy avc subscription event and event outcome schemas
- Change subscription response and outcome sample json file contents
- Change ncmp event response code to support avc subscriptions
- Add mapper that maps cloud event to subscription event response
- Add mapper that maps subscription event outcome to cloud event
- Change subscription event response consumer to consume CloudEvents
- Change time out task to support response event instead client id and
  name
- Change subscription event response mapper to support Cloud Event
- Change subscription outcome mapper to group subscription responses as
  per details and status
- Change subscription status to have fromString functionality
- Change all unit tests to support new functionalities
- Add cps exceptions for cloud event and outcome type
- Add details field in yang model
- Change data node helper to supoort details field
- Consolidate final subscription response codes
- Fix code smells reported by SonarLint

Issue-ID: CPS-1739
Change-Id: I5eadeb8ef40d3d7befb762b5a8d2139fe3c85d7e
Signed-off-by: halil.cakal <halil.cakal@est.tech>
9 months agoSome performance tests are skipped due to filename 68/135568/1
danielhanrahan [Wed, 26 Jul 2023 14:50:06 +0000 (15:50 +0100)]
Some performance tests are skipped due to filename

CPS POM is setup so that only files ending in 'Spec' or 'Test' are
run as tests, and files ending in 'PerfTest' will only run in the
'include-performance' profile. The 32K limit tests are currently
being skipped always, as they do not end in 'Spec', 'Test', or
'PerfTest'. This commit renames them to PerfTest.

Issue-ID: CPS-1811
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I75a4ae92edf324213f734503c23dedb2b31fd007

9 months agoUpdate dmi-stub details in test-deregistration script 43/135543/2
danielhanrahan [Mon, 24 Jul 2023 08:37:34 +0000 (09:37 +0100)]
Update dmi-stub details in test-deregistration script

dmi-plugin-stub was renamed to dmi-plugin-demo-and-csit-stub,
and port changed to 8092.

Issue-ID: CPS-1560
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I3005087ace35aa131df720ed04c642d2d72f1255

9 months agoFix code coverage reporting 46/135546/1
ToineSiebelink [Mon, 24 Jul 2023 11:23:05 +0000 (12:23 +0100)]
Fix code coverage reporting

- Fixed (partly duplicated) exclusion list: all exlusings now in PARENT pom only
(this means module reports and aggregate report use same exclusion
- Set common minimum to 100% (3 modules now achieve this :-) )
- Added./clean cm-parsre test to get too 10% in that module too
- Increased module specif minima to actual coverge today

Issue-ID: CPS-475
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: Ic155f963bfd472e11481fcab6ee8ca227903d9ae

9 months agoPrepare for next CPS-NCMP release 37/135537/3
egernug [Fri, 21 Jul 2023 12:27:24 +0000 (13:27 +0100)]
Prepare for next CPS-NCMP release

Issue-ID: CPS-1797

Signed-off-by: egernug <gerard.nugent@est.tech>
Change-Id: Ib2b840b6b37131f16afd879720658d2e45387d6a

9 months agoContainer for release 3.3.5 36/135536/2
egernug [Fri, 21 Jul 2023 12:19:01 +0000 (13:19 +0100)]
Container for release 3.3.5

Issue-ID: CPS-1797

Signed-off-by: egernug <gerard.nugent@est.tech>
Change-Id: Id8bcefe615947a7ec160275bf7a3326e80a0f2f3

9 months agoMaven stage yaml for 3.3.5 35/135535/1
egernug [Fri, 21 Jul 2023 12:11:04 +0000 (13:11 +0100)]
Maven stage yaml for 3.3.5

Issue-ID: CPS-1797

Signed-off-by: egernug <gerard.nugent@est.tech>
Change-Id: I99232dde07a4ad92fd124265214bc760a032de12

9 months agoIncrease timeout for PollingConditions 34/135534/3
egernug [Fri, 21 Jul 2023 10:14:23 +0000 (11:14 +0100)]
Increase timeout for PollingConditions

Test failing due to timing issues. Increasing timeout to rectify.

Issue-ID: CPS-475

Signed-off-by: egernug <gerard.nugent@est.tech>
Change-Id: I89b512809922470f10940fcdd739f057b8f9c80e

9 months agoMerge "Refactor ncmp request handlers (fix async issue)"
Sourabh Sourabh [Thu, 20 Jul 2023 15:29:14 +0000 (15:29 +0000)]
Merge "Refactor ncmp request handlers (fix async issue)"

9 months agoMerge "Apostrophe handling in CpsPathParser"
Toine Siebelink [Thu, 20 Jul 2023 13:27:18 +0000 (13:27 +0000)]
Merge "Apostrophe handling in CpsPathParser"

9 months agoRefactor ncmp request handlers (fix async issue) 03/135503/7
ToineSiebelink [Tue, 18 Jul 2023 12:23:29 +0000 (13:23 +0100)]
Refactor ncmp request handlers (fix async issue)

- simplified request handlers (sub)classes (no more need for default interface)
- fix issue with async execution of data operation requests (wasn't really async)
- removed redundant (unreachable) check in production code
- Improved code coverage (branches) ncmp request handlers added UNIT test
- removed MVC test scenarios now covered by appropriate unit level tests
- applied Spock Polling Condition for verifying async call

Issue-ID: CPS-475
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: Ibe601c709de65080fa8898f2419fdbd92c5ba27d

9 months agoMerge "Persisting a list element to a parent list (ep2)"
Luke Gleeson [Thu, 20 Jul 2023 09:47:48 +0000 (09:47 +0000)]
Merge "Persisting a list element to a parent list (ep2)"

9 months agoApostrophe handling in CpsPathParser 74/135474/7
danielhanrahan [Wed, 28 Jun 2023 11:55:20 +0000 (12:55 +0100)]
Apostrophe handling in CpsPathParser

Apostrophe is not currently handled correctly, and having apostrophe in
the xpath will lead to various errors.
For example, normalizing this xpath works:
  /path[@name="I'm quoted"] -> /path[@name='I\'m quoted']
However the resulting xpath will throw a PathParsingException if parsed!
(Thus path normalization is not idempotent.)

- Use '' for escaping apostrophe in single quoted leaf value,
  to comply with XPath standard (and use "" for escaping in ").
- Use Liquibase to make existing data comply with new rules.
- Leaf values in data leaves are now unescaped, e.g. "I'm quoted"
- Quoting is now consistent for leaf/text/contains conditions.

Issue-ID: CPS-1769
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Iafc287f738254d7f99706c6bc548091c0ecd5aa0

9 months agoMerge "Allow duplicate leaf names in Cps Path leaf condition"
Toine Siebelink [Thu, 20 Jul 2023 08:52:40 +0000 (08:52 +0000)]
Merge "Allow duplicate leaf names in Cps Path leaf condition"

9 months agoPrepare for next CPS-NCMP release 06/135506/7
egernug [Wed, 19 Jul 2023 10:05:00 +0000 (11:05 +0100)]
Prepare for next CPS-NCMP release

Issue-ID: CPS-1791

Signed-off-by: egernug <gerard.nugent@est.tech>
Change-Id: Ide0fcdfa41656e25395ac8478da1bb489fd8663e

9 months agoContainer yaml for release 94/135494/2
egernug [Tue, 18 Jul 2023 10:32:56 +0000 (11:32 +0100)]
Container yaml for release

Issue-ID: CPS-1793

Signed-off-by: egernug <gerard.nugent@est.tech>
Change-Id: If7ac05d814da42b12c66704afcb153f4af72f3e5

9 months agoMaven stage yaml 93/135493/1
egernug [Tue, 18 Jul 2023 10:26:18 +0000 (11:26 +0100)]
Maven stage yaml

Issue-ID: CPS-1793

Signed-off-by: egernug <gerard.nugent@est.tech>
Change-Id: Ie553550137038e4086f049beca76506df48e2a21

9 months agoUpdate release notes for release 3.3.4 92/135492/1
egernug [Tue, 18 Jul 2023 09:57:42 +0000 (10:57 +0100)]
Update release notes for release 3.3.4

Issue-ID: CPS-1792

Signed-off-by: egernug <gerard.nugent@est.tech>
Change-Id: I3f278d3e89dbc1a757f603ac312686d83612eec8

9 months agoPersisting a list element to a parent list (ep2) 23/134723/35
Rudrangi Anupriya [Mon, 17 Jul 2023 14:50:34 +0000 (20:20 +0530)]
Persisting a list element to a parent list (ep2)

Post List Element does not allow for create List Element, only appends onto existing node as children

-Add a check in saveListElements to see if the parent xpath is a root path ("/").If root node store list element as top node. Else add passed list element to parent xpath node.
-Add test for scenario for above
-Add test scenario Saving list element data fragment under Root node
-Add Integration Tests Add and Delete top-level list (element) data nodes with root node
-Update bookstore model with TopLevelList datanode

Issue-ID: CPS-1586
Change-Id: Iaa7f59fbeebb03703626132c6d5c2afde0e7ab4b
Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>
9 months agoAllow duplicate leaf names in Cps Path leaf condition 76/135476/3
danielhanrahan [Fri, 14 Jul 2023 12:09:41 +0000 (13:09 +0100)]
Allow duplicate leaf names in Cps Path leaf condition

Presently, a query using the same leaf name twice will fail:
 //books[@price > 10 and @price < 20]
It is caused by storing data leaves in a Map.
This is fixed by storing data leaves in a List<DataLeaf>.

Issue-ID: CPS-1779
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ie6990ea5e622cf37e986b720a0a07fb69ce4f03b

9 months agoImproved code coverage for CpsDataServiceImpl 80/135480/2
ToineSiebelink [Mon, 17 Jul 2023 08:22:05 +0000 (09:22 +0100)]
Improved code coverage for CpsDataServiceImpl

- Added (extended) test to cover missed scenarios
- Removed unnecesarry null check from Production code
- Improved error messages in production code
- Removed duplicate test
- Cleaned up existign test somewhat (labels and aligment)

Issue-ID: CPS-475
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I9761ad6d4777a6bcfee4cbe1b876dd39fa218fc8

9 months agoMerge "Improved code coverage for NetworkCmProxyDataServiceImpl"
Luke Gleeson [Fri, 14 Jul 2023 11:25:14 +0000 (11:25 +0000)]
Merge "Improved code coverage for NetworkCmProxyDataServiceImpl"

9 months agoMerge "Upgrade to Java 17"
Sourabh Sourabh [Fri, 14 Jul 2023 10:48:02 +0000 (10:48 +0000)]
Merge "Upgrade to Java 17"

9 months agoImproved code coverage for NetworkCmProxyDataServiceImpl 19/135419/1
ToineSiebelink [Thu, 13 Jul 2023 16:55:32 +0000 (17:55 +0100)]
Improved code coverage for NetworkCmProxyDataServiceImpl

- Production code had catch block that is no longer needed
- Cleaned up some test somewhat (close to the new tests)

Issue-ID: CPS-475
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: Iaa409e752a496bf088a58ccd422fe3d850442b59

9 months ago[CSIT] Testing DataOperations Request with positive Scenario 49/135349/13
raviteja.karumuri [Thu, 6 Jul 2023 12:48:31 +0000 (13:48 +0100)]
[CSIT] Testing DataOperations Request with positive Scenario

# Sending CMHandle with ready state to the NCMP then it forwarded to DMI to complete the read operation once done with success it produces a message to the client topic'

Issue-ID: CPS-1560
Signed-off-by: raviteja.karumuri <raviteja.karumuri@est.tech>
Change-Id: I0af68fdf97bf0a5063361ae08475dee38dbe7b8e

9 months agoIntroduced a dmi rest demo csit stub 72/135372/8
sourabh_sourabh [Tue, 4 Jul 2023 23:56:05 +0000 (00:56 +0100)]
Introduced a dmi rest demo csit stub

- Introduced a new response code 0 for data operational success response.
- Created a new dmi plugin rest stub for demo and performance testing.
- Removed dmi plugin stub of wiremocked.
- Updated parent pom to add newely created demo stub.
- updated docker compose to add neww demo stub with profile for CSIT,
  demo and performance testing.
- Removed target id list from unsupported operation/datastore from REST
  response.

Issue-ID: CPS-1560

Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: Ie8198366df820e7d4b8b6e8a85ff9cdb0fce7435
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
9 months agoFix handling of special characters in prefix names 79/135379/3
danielhanrahan [Wed, 12 Jul 2023 15:11:07 +0000 (16:11 +0100)]
Fix handling of special characters in prefix names

This fixes issues with special characters like square brackets
- Make PrefixResolver use CpsPathParser instead of regex
- Make DataMapUtils use CpsPathParser instead of String parsing

Issue-ID: CPS-1758
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I6dd66eee398a46a69c0229059195f5096ab6fdec

9 months agoMerge "Improved code coverage for record filter strategies"
Sourabh Sourabh [Thu, 13 Jul 2023 10:15:28 +0000 (10:15 +0000)]
Merge "Improved code coverage for record filter strategies"

9 months agoUpgrade to Java 17 53/135353/2
egernug [Mon, 3 Jul 2023 09:09:09 +0000 (10:09 +0100)]
Upgrade to Java 17

Upgrade CPS component to Java 17

Issue-ID:CPS-1767

Signed-off-by: egernug <gerard.nugent@est.tech>
Change-Id: Ide1e75205851e025371b4822ee161f71f04a6af1

9 months agoImproved code coverage for record filter strategies 81/135381/2
ToineSiebelink [Wed, 12 Jul 2023 16:05:41 +0000 (17:05 +0100)]
Improved code coverage for record filter strategies

(the semi-integration test dont cover all scenarios)

Issue-ID: CPS-475
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I717946cddec315a1e69ca8d104b07f91691c1b63

9 months agoImproved code coverage for Data Sync Watchdog 83/135383/1
ToineSiebelink [Wed, 12 Jul 2023 16:41:44 +0000 (17:41 +0100)]
Improved code coverage for Data Sync Watchdog

- also cleaned up existing test (c&p errors in descriptions)

Issue-ID: CPS-475

Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I20ed98faf0e05e2eb3b2aeaa92bed7dc73cbb358

9 months agoImproved code coverage (branches) for sync watchdog 77/135377/1
ToineSiebelink [Wed, 12 Jul 2023 14:30:48 +0000 (15:30 +0100)]
Improved code coverage (branches) for sync watchdog

- had to refactor prod. code slightly to make it testable
- added a batch size check although the task is a simple iteration there is overhead in creating a paralel task

Issue-ID: CPS-475

Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I0db7ba3a15f4edc469a5af1ddc6fd76922b16809

9 months agoCombine alreadyDefinedException classes 58/135258/7
danielhanrahan [Fri, 30 Jun 2023 11:14:07 +0000 (12:14 +0100)]
Combine alreadyDefinedException classes

Issue-ID: CPS-1774
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I52ff9074a9f8188e8635a811b0d2713a97cb4b10

10 months agoNCMP fails to start if /subscription-registry already exists 52/135352/2
danielhanrahan [Mon, 10 Jul 2023 09:47:36 +0000 (10:47 +0100)]
NCMP fails to start if /subscription-registry already exists

cpsDataService only throws AlreadyDefinedException, not
AlreadyDefinedExceptionBatch. This causes NcmpStartupException
to be thrown during startup, if /subscription-registry exists.

Issue-ID: CPS-1783
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I5667d891f855ef47ca2475891f9216be102932ed

10 months agoHandle special characters in CpsPath queries (CPS-1760 #2) 32/135132/12
danielhanrahan [Fri, 23 Jun 2023 13:37:12 +0000 (14:37 +0100)]
Handle special characters in CpsPath queries (CPS-1760 #2)

This fixes issues with special characters for CPS-500, CPS-1756,
CPS-1758, and CPS-1760. It also improves query performance.

- use SQL LIKE instead of regex in Cps Path queries

Issue-ID: CPS-1763
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I5c179882bfba71d3b009c60059e9073f46227e7d

10 months agoMerge "Subscription Creation: NCMP to DMI CloudEvent transformation"
Priyank Maheshwari [Mon, 10 Jul 2023 09:53:59 +0000 (09:53 +0000)]
Merge "Subscription Creation: NCMP to DMI CloudEvent transformation"

10 months agoMerge "Escape SQL LIKE wildcards in queries (CPS-1760 #1)"
Toine Siebelink [Mon, 10 Jul 2023 09:06:56 +0000 (09:06 +0000)]
Merge "Escape SQL LIKE wildcards in queries (CPS-1760 #1)"

10 months agoSubscription Creation: NCMP to DMI CloudEvent transformation 02/135302/8
halil.cakal [Mon, 3 Jul 2023 14:24:02 +0000 (15:24 +0100)]
Subscription Creation: NCMP to DMI CloudEvent transformation

- Add mapper to convert client event into ncmp event
- Add sample json object of ncmp version
- Change subscription event consumer to consume CloudEvents
- Change subscription event forwarder to publish CloudEvents
- Change test producer config to support CloudEvents
- Change sample subscription event json to comply with new schema
- Add more test for missing branches
- Change packages of the mappers into relevant directory

Issue-ID: CPS-1737
Change-Id: I8c9e6e7bf713a8fb530a0586dfb2bce796a462f5
Signed-off-by: halil.cakal <halil.cakal@est.tech>
10 months agoMerge "NCMP to client schema for Subscription"
Luke Gleeson [Fri, 7 Jul 2023 14:47:22 +0000 (14:47 +0000)]
Merge "NCMP to client schema for Subscription"

10 months agoFix doc issue 45/135345/2
lukegleeson [Fri, 7 Jul 2023 14:12:17 +0000 (15:12 +0100)]
Fix doc issue

Issue-ID: CPS-475
Signed-off-by: lukegleeson <luke.gleeson@est.tech>
Change-Id: Id1f15d148dca7801ecfabd447e9f87d56414a2b4

10 months agoEscape SQL LIKE wildcards in queries (CPS-1760 #1) 33/135133/6
danielhanrahan [Fri, 23 Jun 2023 11:49:27 +0000 (12:49 +0100)]
Escape SQL LIKE wildcards in queries (CPS-1760 #1)

If '%' and '_' are used in the contains-condition of a CpsPath query,
incorrect results will be returned. For example:
  /bookstore/categories[contains(@code, "%")]
Special characters in the contains-condition value must be escaped.

Issue-ID: CPS-1762
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I2fdd5a26433d510cd7d6af5b734a6779b537d63d

10 months agoNCMP to client schema for Subscription 31/135331/3
mpriyank [Wed, 5 Jul 2023 15:14:34 +0000 (16:14 +0100)]
NCMP to client schema for Subscription

- added schema to comply with the agreed upon changes and it include
  capability to have statusCode and statusMessage as well.
- Renamed existing schemas just for consistency

Issue-ID: CPS-1739
Change-Id: I6bb1d13bf5c84ca812c5cf37062a168845d6be1d
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
10 months agoMerge "Modify dmi plugin stub mapping of data operation"
Toine Siebelink [Mon, 3 Jul 2023 13:03:51 +0000 (13:03 +0000)]
Merge "Modify dmi plugin stub mapping of data operation"

10 months agoKafka (De)serializtion test case for legacy consumer 57/135257/3
raviteja.karumuri [Fri, 30 Jun 2023 12:23:33 +0000 (13:23 +0100)]
Kafka (De)serializtion test case for legacy consumer

# Added test case to verify the legacy consumer able to forward the event to client topic

Signed-off-by: raviteja.karumuri <raviteja.karumuri@est.tech>
Change-Id: I9c88023028eaa2453e5578b5afb4951a41bf4999

10 months agoModify dmi plugin stub mapping of data operation 65/135265/2
sourabh_sourabh [Fri, 30 Jun 2023 11:21:09 +0000 (12:21 +0100)]
Modify dmi plugin stub mapping of data operation

- Modifiying exising dmi stub behaviour to mock onap-dmi plugin to send response HTTP 501.

Issue-ID: CPS-1723

Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: I5dad5004ce69b29752821afdbb87839befa97cf5

10 months agoMerge "Kafka (De-)Serialization Test"
Sourabh Sourabh [Fri, 30 Jun 2023 11:54:40 +0000 (11:54 +0000)]
Merge "Kafka (De-)Serialization Test"

10 months agoPrepare for next CPS-NCMP release 56/135256/1
mpriyank [Fri, 30 Jun 2023 10:14:10 +0000 (11:14 +0100)]
Prepare for next CPS-NCMP release

- updated the docs.
- Bumped the version to 3.3.4-SNAPSHOT for the next release.

Issue-ID: CPS-1773
Change-Id: Ibe9a694baadb33d284913868452b143c14d105cc
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
10 months agoContainer yaml for release 55/135255/1
mpriyank [Fri, 30 Jun 2023 09:24:35 +0000 (10:24 +0100)]
Container yaml for release

- Release process

Issue-ID: CPS-1772
Change-Id: Ia3ce440f9a58d59ec9e44717c0c749d546c725b6
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
10 months agoMaven stage yaml 53/135253/1
mpriyank [Thu, 29 Jun 2023 17:25:20 +0000 (18:25 +0100)]
Maven stage yaml

Issue-ID: CPS-1772
Change-Id: I06bf1a018e891339d16bd8c2c8f935ef3b44ea84
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
10 months agoKafka (De-)Serialization Test 52/135252/1
ToineSiebelink [Thu, 29 Jun 2023 17:00:48 +0000 (18:00 +0100)]
Kafka (De-)Serialization Test

- added test that proof (de-)serialization of DataOperation CloudEvent
- extracted new baseclass for kafka consumer tests

Issue-ID: CPS-1746
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I3b04dc0ed8cd1f1f48206cbcf0fd739532ba423c

10 months agoOpenapi file and RTD update 51/135251/4
mpriyank [Thu, 29 Jun 2023 15:56:11 +0000 (16:56 +0100)]
Openapi file and RTD update

- Step-1 and 2 of the release process

Issue-ID: CPS-1771
Change-Id: I2c5f2f04fdf6344c788aa47a82090b2b37d1505f
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>