cps.git
9 months agoMerge "Use constants for magic numbers in perf tests"
Toine Siebelink [Mon, 31 Jul 2023 10:31:59 +0000 (10:31 +0000)]
Merge "Use constants for magic numbers in perf tests"

9 months agoIncrease code coverage in cps-service module 76/135576/3
ToineSiebelink [Wed, 26 Jul 2023 16:49:02 +0000 (17:49 +0100)]
Increase code coverage in cps-service module

- After last rebase I had to remove 3 unused recent cloud eventd specific exceptions/constructors
- Moved the only used new exception from SPI to the relevant util package
(please NOTE not all exceptions belong in SPI and always question need for new exception
when there is no specific handling, try to use standard or existign CPS exception instead!)
- Increased cps-service module (line) coverage from 95 to 100%
- Added tests for missing exceptions (handling i.e. thrown up)
- Removed incorrect SPI defined OperationNotYetSupportedException
(replaced with standard java exception instead)
- Fixed some legacy issues with existign test classes I modified
(unnecessary setup, conventions etc)
- Increased coverage for DataNodeBuilder
- Added or modified test to include more spi models
- Added tests for Hazelcast Configs
- Added more tests for json object mapper
- Added test and fixed error handling in YangUtils/XmlFileUtils
(it was incorrectly converting a config exception to a data validation exception)

Issue-ID: CPS-475
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I5852ba01bc5b33ae361b8f29daae9868f05baa35

9 months agoUse constants for magic numbers in perf tests 78/135578/5
danielhanrahan [Thu, 27 Jul 2023 09:44:38 +0000 (10:44 +0100)]
Use constants for magic numbers in perf tests

Issue-ID: CPS-1811
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ic87bcb1847b97a0e8c59b2e3636e9ccd66f814ae

9 months agoFix pom.xml version 00/135600/1
leventecsanyi [Fri, 28 Jul 2023 15:36:55 +0000 (17:36 +0200)]
Fix pom.xml version

  - corrected version number

Issue-ID: CPS-1817
Signed-off-by: Levente Csanyi <levente.csanyi@est.tech>
Change-Id: Id97b3d55e7342619804b8e007238718b56ef3120

9 months agoAdd 'direct' keyword to descendants option to query direct children (ep1) 25/135525/5
Rudrangi Anupriya [Fri, 28 Jul 2023 08:58:54 +0000 (14:28 +0530)]
Add 'direct' keyword to descendants option to query direct children (ep1)

-added 'direct' keyword also to Fetch Descendants Option along with 'all' and 'none' to Query direct child.
-added unit tests to test direct keyword

Issue-ID:CPS-1784
Change-Id: Iab7f59fbeebb03703626132c6d5c2afde0e5ab4d
Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>
9 months agoMerge "Improve 32K limit tests"
Toine Siebelink [Thu, 27 Jul 2023 15:14:19 +0000 (15:14 +0000)]
Merge "Improve 32K limit tests"

9 months agoMerge "Add write performance tests"
Toine Siebelink [Thu, 27 Jul 2023 15:14:02 +0000 (15:14 +0000)]
Merge "Add write performance tests"

9 months agoUpdate RTD (CPS-1515: Spike: Support Multiple CM-Handles for NCMP Get Operation) 54/135554/5
sourabh_sourabh [Tue, 25 Jul 2023 09:58:52 +0000 (10:58 +0100)]
Update RTD (CPS-1515: Spike: Support Multiple CM-Handles for NCMP Get Operation)

- Added information about data operation NCMP endpoint and kafka event
  details.
- Re-structured "CPS-Events" and "Modeling" page.

Issue-ID:CPS-1785

Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: I2b077ccd7ccd6144a8f9f2183d927f922075f532
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
9 months agoImprove 32K limit tests 69/135569/3
danielhanrahan [Wed, 26 Jul 2023 12:19:50 +0000 (13:19 +0100)]
Improve 32K limit tests

Some operations were previously limited to 32,768 parameters,
such as getDataNodesForMultipleXpaths. After this limit was lifted,
the tests only verified that database exceptions were not thrown.
This commit improves the tests by verifying that the operations
return or delete the expected data.

- Create 33,000 data nodes
- Verify that getDataNodesForMultipleXpaths returns 33,000 nodes
- Verify than deleteDataNodes deletes 33,000 nodes

Issue-ID: CPS-1811
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I7029ea0cf1d4e95c5ca814ca62c222fc5b9ddc7e

9 months agoMerge "Allow getDataNodes to read whole lists(ep 1)"
Sourabh Sourabh [Thu, 27 Jul 2023 12:16:35 +0000 (12:16 +0000)]
Merge "Allow getDataNodes to read whole lists(ep 1)"

9 months agoAllow getDataNodes to read whole lists(ep 1) 59/134759/17
danielhanrahan [Thu, 18 May 2023 09:18:10 +0000 (10:18 +0100)]
Allow getDataNodes to read whole lists(ep 1)

 - getDataNodes can now retrieve list data nodes.

Issue-ID: CPS-1696
Signed-off-by: arpitsingh <as00745003@techmahindra.com>
Change-Id: I320a368d6cb73599d3f7c13fe9b8ab7d0cc69470

9 months agoMerge "Subscription Creation: NCMP to Client CloudEvent transformation"
Luke Gleeson [Thu, 27 Jul 2023 10:20:54 +0000 (10:20 +0000)]
Merge "Subscription Creation: NCMP to Client CloudEvent transformation"

9 months agoMerge "Performance test of reading non-existing datanodes"
Toine Siebelink [Thu, 27 Jul 2023 09:56:32 +0000 (09:56 +0000)]
Merge "Performance test of reading non-existing datanodes"

9 months agoAdd write performance tests 73/135573/3
danielhanrahan [Wed, 26 Jul 2023 08:50:55 +0000 (09:50 +0100)]
Add write performance tests

- Add perf test for openroadm (linear time)
- Add perf test for bookstore (exponential time!)

Issue-ID: CPS-1811
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I9c1772a9e9684707f2dbb32b2c361ec248fb1c60

9 months agoPerformance test of reading non-existing datanodes 71/135571/1
danielhanrahan [Wed, 26 Jul 2023 15:42:39 +0000 (16:42 +0100)]
Performance test of reading non-existing datanodes

Issue-ID: CPS-1811
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I183129bb5b0a2d620b87e9b36f3113fe8e169798

9 months agoPerformance test of deleting non-existing datanodes 70/135570/1
danielhanrahan [Wed, 26 Jul 2023 15:56:10 +0000 (16:56 +0100)]
Performance test of deleting non-existing datanodes

Issue-ID: CPS-1811
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ifae635f45d8ec1990452ade958253972fde68415

9 months agoSubscription Creation: NCMP to Client CloudEvent transformation 13/135413/11
halil.cakal [Thu, 13 Jul 2023 10:28:18 +0000 (11:28 +0100)]
Subscription Creation: NCMP to Client CloudEvent transformation

- Delete legacy avc subscription event and event outcome schemas
- Change subscription response and outcome sample json file contents
- Change ncmp event response code to support avc subscriptions
- Add mapper that maps cloud event to subscription event response
- Add mapper that maps subscription event outcome to cloud event
- Change subscription event response consumer to consume CloudEvents
- Change time out task to support response event instead client id and
  name
- Change subscription event response mapper to support Cloud Event
- Change subscription outcome mapper to group subscription responses as
  per details and status
- Change subscription status to have fromString functionality
- Change all unit tests to support new functionalities
- Add cps exceptions for cloud event and outcome type
- Add details field in yang model
- Change data node helper to supoort details field
- Consolidate final subscription response codes
- Fix code smells reported by SonarLint

Issue-ID: CPS-1739
Change-Id: I5eadeb8ef40d3d7befb762b5a8d2139fe3c85d7e
Signed-off-by: halil.cakal <halil.cakal@est.tech>
9 months agoSome performance tests are skipped due to filename 68/135568/1
danielhanrahan [Wed, 26 Jul 2023 14:50:06 +0000 (15:50 +0100)]
Some performance tests are skipped due to filename

CPS POM is setup so that only files ending in 'Spec' or 'Test' are
run as tests, and files ending in 'PerfTest' will only run in the
'include-performance' profile. The 32K limit tests are currently
being skipped always, as they do not end in 'Spec', 'Test', or
'PerfTest'. This commit renames them to PerfTest.

Issue-ID: CPS-1811
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I75a4ae92edf324213f734503c23dedb2b31fd007

9 months agoUpdate dmi-stub details in test-deregistration script 43/135543/2
danielhanrahan [Mon, 24 Jul 2023 08:37:34 +0000 (09:37 +0100)]
Update dmi-stub details in test-deregistration script

dmi-plugin-stub was renamed to dmi-plugin-demo-and-csit-stub,
and port changed to 8092.

Issue-ID: CPS-1560
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I3005087ace35aa131df720ed04c642d2d72f1255

9 months agoFix code coverage reporting 46/135546/1
ToineSiebelink [Mon, 24 Jul 2023 11:23:05 +0000 (12:23 +0100)]
Fix code coverage reporting

- Fixed (partly duplicated) exclusion list: all exlusings now in PARENT pom only
(this means module reports and aggregate report use same exclusion
- Set common minimum to 100% (3 modules now achieve this :-) )
- Added./clean cm-parsre test to get too 10% in that module too
- Increased module specif minima to actual coverge today

Issue-ID: CPS-475
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: Ic155f963bfd472e11481fcab6ee8ca227903d9ae

9 months agoPrepare for next CPS-NCMP release 37/135537/3
egernug [Fri, 21 Jul 2023 12:27:24 +0000 (13:27 +0100)]
Prepare for next CPS-NCMP release

Issue-ID: CPS-1797

Signed-off-by: egernug <gerard.nugent@est.tech>
Change-Id: Ib2b840b6b37131f16afd879720658d2e45387d6a

9 months agoContainer for release 3.3.5 36/135536/2
egernug [Fri, 21 Jul 2023 12:19:01 +0000 (13:19 +0100)]
Container for release 3.3.5

Issue-ID: CPS-1797

Signed-off-by: egernug <gerard.nugent@est.tech>
Change-Id: Id8bcefe615947a7ec160275bf7a3326e80a0f2f3

9 months agoMaven stage yaml for 3.3.5 35/135535/1
egernug [Fri, 21 Jul 2023 12:11:04 +0000 (13:11 +0100)]
Maven stage yaml for 3.3.5

Issue-ID: CPS-1797

Signed-off-by: egernug <gerard.nugent@est.tech>
Change-Id: I99232dde07a4ad92fd124265214bc760a032de12

9 months agoIncrease timeout for PollingConditions 34/135534/3
egernug [Fri, 21 Jul 2023 10:14:23 +0000 (11:14 +0100)]
Increase timeout for PollingConditions

Test failing due to timing issues. Increasing timeout to rectify.

Issue-ID: CPS-475

Signed-off-by: egernug <gerard.nugent@est.tech>
Change-Id: I89b512809922470f10940fcdd739f057b8f9c80e

9 months agoMerge "Refactor ncmp request handlers (fix async issue)"
Sourabh Sourabh [Thu, 20 Jul 2023 15:29:14 +0000 (15:29 +0000)]
Merge "Refactor ncmp request handlers (fix async issue)"

9 months agoMerge "Apostrophe handling in CpsPathParser"
Toine Siebelink [Thu, 20 Jul 2023 13:27:18 +0000 (13:27 +0000)]
Merge "Apostrophe handling in CpsPathParser"

9 months agoRefactor ncmp request handlers (fix async issue) 03/135503/7
ToineSiebelink [Tue, 18 Jul 2023 12:23:29 +0000 (13:23 +0100)]
Refactor ncmp request handlers (fix async issue)

- simplified request handlers (sub)classes (no more need for default interface)
- fix issue with async execution of data operation requests (wasn't really async)
- removed redundant (unreachable) check in production code
- Improved code coverage (branches) ncmp request handlers added UNIT test
- removed MVC test scenarios now covered by appropriate unit level tests
- applied Spock Polling Condition for verifying async call

Issue-ID: CPS-475
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: Ibe601c709de65080fa8898f2419fdbd92c5ba27d

9 months agoMerge "Persisting a list element to a parent list (ep2)"
Luke Gleeson [Thu, 20 Jul 2023 09:47:48 +0000 (09:47 +0000)]
Merge "Persisting a list element to a parent list (ep2)"

9 months agoApostrophe handling in CpsPathParser 74/135474/7
danielhanrahan [Wed, 28 Jun 2023 11:55:20 +0000 (12:55 +0100)]
Apostrophe handling in CpsPathParser

Apostrophe is not currently handled correctly, and having apostrophe in
the xpath will lead to various errors.
For example, normalizing this xpath works:
  /path[@name="I'm quoted"] -> /path[@name='I\'m quoted']
However the resulting xpath will throw a PathParsingException if parsed!
(Thus path normalization is not idempotent.)

- Use '' for escaping apostrophe in single quoted leaf value,
  to comply with XPath standard (and use "" for escaping in ").
- Use Liquibase to make existing data comply with new rules.
- Leaf values in data leaves are now unescaped, e.g. "I'm quoted"
- Quoting is now consistent for leaf/text/contains conditions.

Issue-ID: CPS-1769
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Iafc287f738254d7f99706c6bc548091c0ecd5aa0

9 months agoMerge "Allow duplicate leaf names in Cps Path leaf condition"
Toine Siebelink [Thu, 20 Jul 2023 08:52:40 +0000 (08:52 +0000)]
Merge "Allow duplicate leaf names in Cps Path leaf condition"

9 months agoPrepare for next CPS-NCMP release 06/135506/7
egernug [Wed, 19 Jul 2023 10:05:00 +0000 (11:05 +0100)]
Prepare for next CPS-NCMP release

Issue-ID: CPS-1791

Signed-off-by: egernug <gerard.nugent@est.tech>
Change-Id: Ide0fcdfa41656e25395ac8478da1bb489fd8663e

9 months agoContainer yaml for release 94/135494/2
egernug [Tue, 18 Jul 2023 10:32:56 +0000 (11:32 +0100)]
Container yaml for release

Issue-ID: CPS-1793

Signed-off-by: egernug <gerard.nugent@est.tech>
Change-Id: If7ac05d814da42b12c66704afcb153f4af72f3e5

9 months agoMaven stage yaml 93/135493/1
egernug [Tue, 18 Jul 2023 10:26:18 +0000 (11:26 +0100)]
Maven stage yaml

Issue-ID: CPS-1793

Signed-off-by: egernug <gerard.nugent@est.tech>
Change-Id: Ie553550137038e4086f049beca76506df48e2a21

9 months agoUpdate release notes for release 3.3.4 92/135492/1
egernug [Tue, 18 Jul 2023 09:57:42 +0000 (10:57 +0100)]
Update release notes for release 3.3.4

Issue-ID: CPS-1792

Signed-off-by: egernug <gerard.nugent@est.tech>
Change-Id: I3f278d3e89dbc1a757f603ac312686d83612eec8

9 months agoPersisting a list element to a parent list (ep2) 23/134723/35
Rudrangi Anupriya [Mon, 17 Jul 2023 14:50:34 +0000 (20:20 +0530)]
Persisting a list element to a parent list (ep2)

Post List Element does not allow for create List Element, only appends onto existing node as children

-Add a check in saveListElements to see if the parent xpath is a root path ("/").If root node store list element as top node. Else add passed list element to parent xpath node.
-Add test for scenario for above
-Add test scenario Saving list element data fragment under Root node
-Add Integration Tests Add and Delete top-level list (element) data nodes with root node
-Update bookstore model with TopLevelList datanode

Issue-ID: CPS-1586
Change-Id: Iaa7f59fbeebb03703626132c6d5c2afde0e7ab4b
Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>
9 months agoAllow duplicate leaf names in Cps Path leaf condition 76/135476/3
danielhanrahan [Fri, 14 Jul 2023 12:09:41 +0000 (13:09 +0100)]
Allow duplicate leaf names in Cps Path leaf condition

Presently, a query using the same leaf name twice will fail:
 //books[@price > 10 and @price < 20]
It is caused by storing data leaves in a Map.
This is fixed by storing data leaves in a List<DataLeaf>.

Issue-ID: CPS-1779
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ie6990ea5e622cf37e986b720a0a07fb69ce4f03b

9 months agoImproved code coverage for CpsDataServiceImpl 80/135480/2
ToineSiebelink [Mon, 17 Jul 2023 08:22:05 +0000 (09:22 +0100)]
Improved code coverage for CpsDataServiceImpl

- Added (extended) test to cover missed scenarios
- Removed unnecesarry null check from Production code
- Improved error messages in production code
- Removed duplicate test
- Cleaned up existign test somewhat (labels and aligment)

Issue-ID: CPS-475
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I9761ad6d4777a6bcfee4cbe1b876dd39fa218fc8

9 months agoMerge "Improved code coverage for NetworkCmProxyDataServiceImpl"
Luke Gleeson [Fri, 14 Jul 2023 11:25:14 +0000 (11:25 +0000)]
Merge "Improved code coverage for NetworkCmProxyDataServiceImpl"

9 months agoMerge "Upgrade to Java 17"
Sourabh Sourabh [Fri, 14 Jul 2023 10:48:02 +0000 (10:48 +0000)]
Merge "Upgrade to Java 17"

9 months agoImproved code coverage for NetworkCmProxyDataServiceImpl 19/135419/1
ToineSiebelink [Thu, 13 Jul 2023 16:55:32 +0000 (17:55 +0100)]
Improved code coverage for NetworkCmProxyDataServiceImpl

- Production code had catch block that is no longer needed
- Cleaned up some test somewhat (close to the new tests)

Issue-ID: CPS-475
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: Iaa409e752a496bf088a58ccd422fe3d850442b59

9 months ago[CSIT] Testing DataOperations Request with positive Scenario 49/135349/13
raviteja.karumuri [Thu, 6 Jul 2023 12:48:31 +0000 (13:48 +0100)]
[CSIT] Testing DataOperations Request with positive Scenario

# Sending CMHandle with ready state to the NCMP then it forwarded to DMI to complete the read operation once done with success it produces a message to the client topic'

Issue-ID: CPS-1560
Signed-off-by: raviteja.karumuri <raviteja.karumuri@est.tech>
Change-Id: I0af68fdf97bf0a5063361ae08475dee38dbe7b8e

9 months agoIntroduced a dmi rest demo csit stub 72/135372/8
sourabh_sourabh [Tue, 4 Jul 2023 23:56:05 +0000 (00:56 +0100)]
Introduced a dmi rest demo csit stub

- Introduced a new response code 0 for data operational success response.
- Created a new dmi plugin rest stub for demo and performance testing.
- Removed dmi plugin stub of wiremocked.
- Updated parent pom to add newely created demo stub.
- updated docker compose to add neww demo stub with profile for CSIT,
  demo and performance testing.
- Removed target id list from unsupported operation/datastore from REST
  response.

Issue-ID: CPS-1560

Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: Ie8198366df820e7d4b8b6e8a85ff9cdb0fce7435
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
9 months agoFix handling of special characters in prefix names 79/135379/3
danielhanrahan [Wed, 12 Jul 2023 15:11:07 +0000 (16:11 +0100)]
Fix handling of special characters in prefix names

This fixes issues with special characters like square brackets
- Make PrefixResolver use CpsPathParser instead of regex
- Make DataMapUtils use CpsPathParser instead of String parsing

Issue-ID: CPS-1758
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I6dd66eee398a46a69c0229059195f5096ab6fdec

9 months agoMerge "Improved code coverage for record filter strategies"
Sourabh Sourabh [Thu, 13 Jul 2023 10:15:28 +0000 (10:15 +0000)]
Merge "Improved code coverage for record filter strategies"

9 months agoUpgrade to Java 17 53/135353/2
egernug [Mon, 3 Jul 2023 09:09:09 +0000 (10:09 +0100)]
Upgrade to Java 17

Upgrade CPS component to Java 17

Issue-ID:CPS-1767

Signed-off-by: egernug <gerard.nugent@est.tech>
Change-Id: Ide1e75205851e025371b4822ee161f71f04a6af1

9 months agoImproved code coverage for record filter strategies 81/135381/2
ToineSiebelink [Wed, 12 Jul 2023 16:05:41 +0000 (17:05 +0100)]
Improved code coverage for record filter strategies

(the semi-integration test dont cover all scenarios)

Issue-ID: CPS-475
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I717946cddec315a1e69ca8d104b07f91691c1b63

9 months agoImproved code coverage for Data Sync Watchdog 83/135383/1
ToineSiebelink [Wed, 12 Jul 2023 16:41:44 +0000 (17:41 +0100)]
Improved code coverage for Data Sync Watchdog

- also cleaned up existing test (c&p errors in descriptions)

Issue-ID: CPS-475

Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I20ed98faf0e05e2eb3b2aeaa92bed7dc73cbb358

9 months agoImproved code coverage (branches) for sync watchdog 77/135377/1
ToineSiebelink [Wed, 12 Jul 2023 14:30:48 +0000 (15:30 +0100)]
Improved code coverage (branches) for sync watchdog

- had to refactor prod. code slightly to make it testable
- added a batch size check although the task is a simple iteration there is overhead in creating a paralel task

Issue-ID: CPS-475

Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I0db7ba3a15f4edc469a5af1ddc6fd76922b16809

9 months agoCombine alreadyDefinedException classes 58/135258/7
danielhanrahan [Fri, 30 Jun 2023 11:14:07 +0000 (12:14 +0100)]
Combine alreadyDefinedException classes

Issue-ID: CPS-1774
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I52ff9074a9f8188e8635a811b0d2713a97cb4b10

10 months agoNCMP fails to start if /subscription-registry already exists 52/135352/2
danielhanrahan [Mon, 10 Jul 2023 09:47:36 +0000 (10:47 +0100)]
NCMP fails to start if /subscription-registry already exists

cpsDataService only throws AlreadyDefinedException, not
AlreadyDefinedExceptionBatch. This causes NcmpStartupException
to be thrown during startup, if /subscription-registry exists.

Issue-ID: CPS-1783
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I5667d891f855ef47ca2475891f9216be102932ed

10 months agoHandle special characters in CpsPath queries (CPS-1760 #2) 32/135132/12
danielhanrahan [Fri, 23 Jun 2023 13:37:12 +0000 (14:37 +0100)]
Handle special characters in CpsPath queries (CPS-1760 #2)

This fixes issues with special characters for CPS-500, CPS-1756,
CPS-1758, and CPS-1760. It also improves query performance.

- use SQL LIKE instead of regex in Cps Path queries

Issue-ID: CPS-1763
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I5c179882bfba71d3b009c60059e9073f46227e7d

10 months agoMerge "Subscription Creation: NCMP to DMI CloudEvent transformation"
Priyank Maheshwari [Mon, 10 Jul 2023 09:53:59 +0000 (09:53 +0000)]
Merge "Subscription Creation: NCMP to DMI CloudEvent transformation"

10 months agoMerge "Escape SQL LIKE wildcards in queries (CPS-1760 #1)"
Toine Siebelink [Mon, 10 Jul 2023 09:06:56 +0000 (09:06 +0000)]
Merge "Escape SQL LIKE wildcards in queries (CPS-1760 #1)"

10 months agoSubscription Creation: NCMP to DMI CloudEvent transformation 02/135302/8
halil.cakal [Mon, 3 Jul 2023 14:24:02 +0000 (15:24 +0100)]
Subscription Creation: NCMP to DMI CloudEvent transformation

- Add mapper to convert client event into ncmp event
- Add sample json object of ncmp version
- Change subscription event consumer to consume CloudEvents
- Change subscription event forwarder to publish CloudEvents
- Change test producer config to support CloudEvents
- Change sample subscription event json to comply with new schema
- Add more test for missing branches
- Change packages of the mappers into relevant directory

Issue-ID: CPS-1737
Change-Id: I8c9e6e7bf713a8fb530a0586dfb2bce796a462f5
Signed-off-by: halil.cakal <halil.cakal@est.tech>
10 months agoMerge "NCMP to client schema for Subscription"
Luke Gleeson [Fri, 7 Jul 2023 14:47:22 +0000 (14:47 +0000)]
Merge "NCMP to client schema for Subscription"

10 months agoFix doc issue 45/135345/2
lukegleeson [Fri, 7 Jul 2023 14:12:17 +0000 (15:12 +0100)]
Fix doc issue

Issue-ID: CPS-475
Signed-off-by: lukegleeson <luke.gleeson@est.tech>
Change-Id: Id1f15d148dca7801ecfabd447e9f87d56414a2b4

10 months agoEscape SQL LIKE wildcards in queries (CPS-1760 #1) 33/135133/6
danielhanrahan [Fri, 23 Jun 2023 11:49:27 +0000 (12:49 +0100)]
Escape SQL LIKE wildcards in queries (CPS-1760 #1)

If '%' and '_' are used in the contains-condition of a CpsPath query,
incorrect results will be returned. For example:
  /bookstore/categories[contains(@code, "%")]
Special characters in the contains-condition value must be escaped.

Issue-ID: CPS-1762
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I2fdd5a26433d510cd7d6af5b734a6779b537d63d

10 months agoNCMP to client schema for Subscription 31/135331/3
mpriyank [Wed, 5 Jul 2023 15:14:34 +0000 (16:14 +0100)]
NCMP to client schema for Subscription

- added schema to comply with the agreed upon changes and it include
  capability to have statusCode and statusMessage as well.
- Renamed existing schemas just for consistency

Issue-ID: CPS-1739
Change-Id: I6bb1d13bf5c84ca812c5cf37062a168845d6be1d
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
10 months agoMerge "Modify dmi plugin stub mapping of data operation"
Toine Siebelink [Mon, 3 Jul 2023 13:03:51 +0000 (13:03 +0000)]
Merge "Modify dmi plugin stub mapping of data operation"

10 months agoKafka (De)serializtion test case for legacy consumer 57/135257/3
raviteja.karumuri [Fri, 30 Jun 2023 12:23:33 +0000 (13:23 +0100)]
Kafka (De)serializtion test case for legacy consumer

# Added test case to verify the legacy consumer able to forward the event to client topic

Signed-off-by: raviteja.karumuri <raviteja.karumuri@est.tech>
Change-Id: I9c88023028eaa2453e5578b5afb4951a41bf4999

10 months agoModify dmi plugin stub mapping of data operation 65/135265/2
sourabh_sourabh [Fri, 30 Jun 2023 11:21:09 +0000 (12:21 +0100)]
Modify dmi plugin stub mapping of data operation

- Modifiying exising dmi stub behaviour to mock onap-dmi plugin to send response HTTP 501.

Issue-ID: CPS-1723

Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: I5dad5004ce69b29752821afdbb87839befa97cf5

10 months agoMerge "Kafka (De-)Serialization Test"
Sourabh Sourabh [Fri, 30 Jun 2023 11:54:40 +0000 (11:54 +0000)]
Merge "Kafka (De-)Serialization Test"

10 months agoPrepare for next CPS-NCMP release 56/135256/1
mpriyank [Fri, 30 Jun 2023 10:14:10 +0000 (11:14 +0100)]
Prepare for next CPS-NCMP release

- updated the docs.
- Bumped the version to 3.3.4-SNAPSHOT for the next release.

Issue-ID: CPS-1773
Change-Id: Ibe9a694baadb33d284913868452b143c14d105cc
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
10 months agoContainer yaml for release 55/135255/1
mpriyank [Fri, 30 Jun 2023 09:24:35 +0000 (10:24 +0100)]
Container yaml for release

- Release process

Issue-ID: CPS-1772
Change-Id: Ia3ce440f9a58d59ec9e44717c0c749d546c725b6
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
10 months agoMaven stage yaml 53/135253/1
mpriyank [Thu, 29 Jun 2023 17:25:20 +0000 (18:25 +0100)]
Maven stage yaml

Issue-ID: CPS-1772
Change-Id: I06bf1a018e891339d16bd8c2c8f935ef3b44ea84
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
10 months agoKafka (De-)Serialization Test 52/135252/1
ToineSiebelink [Thu, 29 Jun 2023 17:00:48 +0000 (18:00 +0100)]
Kafka (De-)Serialization Test

- added test that proof (de-)serialization of DataOperation CloudEvent
- extracted new baseclass for kafka consumer tests

Issue-ID: CPS-1746
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I3b04dc0ed8cd1f1f48206cbcf0fd739532ba423c

10 months agoOpenapi file and RTD update 51/135251/4
mpriyank [Thu, 29 Jun 2023 15:56:11 +0000 (16:56 +0100)]
Openapi file and RTD update

- Step-1 and 2 of the release process

Issue-ID: CPS-1771
Change-Id: I2c5f2f04fdf6344c788aa47a82090b2b37d1505f
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
10 months agoMerge "Clearing code smells"
Toine Siebelink [Thu, 29 Jun 2023 14:56:12 +0000 (14:56 +0000)]
Merge "Clearing code smells"

10 months agoMerge "NCMP to DMI subscription event schema compliant with Cloud Events"
Priyank Maheshwari [Thu, 29 Jun 2023 14:42:13 +0000 (14:42 +0000)]
Merge "NCMP to DMI subscription event schema compliant with Cloud Events"

10 months agoClearing code smells 49/135249/2
ToineSiebelink [Thu, 29 Jun 2023 14:08:14 +0000 (15:08 +0100)]
Clearing code smells

- added/updated @deprecated javadoc in EventPublisher

Issue-ID: CPS-475
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: Ic0fa3efbc7dc580368bed6782db683b3c7a4d918

10 months agoMerge "Updating the Kafka listener compliance to could events and legacy"
Toine Siebelink [Thu, 29 Jun 2023 14:26:55 +0000 (14:26 +0000)]
Merge "Updating the Kafka listener compliance to could events and legacy"

10 months agoUpdating the Kafka listener compliance to could events and legacy 04/135204/6
raviteja.karumuri [Thu, 29 Jun 2023 12:57:40 +0000 (13:57 +0100)]
Updating the Kafka listener compliance to could events and legacy

# Added filter property to the legay(DMIAsync) Consumer
# Added containerFactory property to the CloudEvent(DataOperation) consumer to use 'cloudEventDeserializtion'

Issue-ID: CPS-1746
Signed-off-by: raviteja.karumuri <raviteja.karumuri@est.tech>
Change-Id: I73315b816eb6446b7e163fd8779bc35fd6cbef63

10 months agoNCMP : NCMP : Handle non responding DMI-Plugin 45/135245/6
sourabh_sourabh [Wed, 28 Jun 2023 11:28:27 +0000 (12:28 +0100)]
NCMP : NCMP : Handle non responding DMI-Plugin

- Added new response code for non-responding dmi and non-success.
- Captured any exception after sending request to dmi service and then
  create a cloud event with error code and publish it to client topic.
- Minor modificarion into resource data operation util class.

Issue-ID: CPS-1558

Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: I39d409fb42856b816bf9833c2a23f7fa250a1e62
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
10 months agoMerge "Normalize parent xpath when building datanodes in CpsDataService"
Toine Siebelink [Thu, 29 Jun 2023 13:49:21 +0000 (13:49 +0000)]
Merge "Normalize parent xpath when building datanodes in CpsDataService"

10 months agoMerge "Update write performance test timings"
Toine Siebelink [Thu, 29 Jun 2023 13:48:20 +0000 (13:48 +0000)]
Merge "Update write performance test timings"

10 months agoUpdate write performance test timings 46/135246/2
danielhanrahan [Thu, 29 Jun 2023 10:28:47 +0000 (11:28 +0100)]
Update write performance test timings

Issue-ID: CPS-1673
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ia2e5bf557bf2ade98fba64b7923856f727049d84

10 months agoNCMP to DMI subscription event schema compliant with Cloud Events 34/135134/13
halil.cakal [Fri, 23 Jun 2023 13:30:52 +0000 (14:30 +0100)]
NCMP to DMI subscription event schema compliant with Cloud Events

- Change avc subscription schema into cloud events format
- Remove json objects that is provided in header of cloud event
- Add CmHandle object for private properties into json schema
- Remove required field and additional properties
  from subscriptionevent and subscriptioneventresponse

Issue-ID: CPS-1737
Change-Id: I9648e5f0264ac56153ff477b1242d3c7b53b2844
Signed-off-by: halil.cakal <halil.cakal@est.tech>
10 months agoImproved code coverage (branches) around multipart file utils 44/135244/1
ToineSiebelink [Thu, 29 Jun 2023 09:42:23 +0000 (10:42 +0100)]
Improved code coverage (branches) around multipart file utils

- added test
- moved wrongly placed test (from MultipartFileUtilSpec to DataMapUtilsSpec)
- applied curent conventions to DataMapUtilsSpec

Issue-ID: CPS-475

Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: Ie8dc7f6802bb6ff0a256dc05df73da6af75409f3

10 months agoMerge "Client to NCMP Subscription schema change"
Priyank Maheshwari [Wed, 28 Jun 2023 15:53:55 +0000 (15:53 +0000)]
Merge "Client to NCMP Subscription schema change"

10 months agoClient to NCMP Subscription schema change 86/135186/8
emaclee [Tue, 27 Jun 2023 09:45:52 +0000 (10:45 +0100)]
Client to NCMP Subscription schema change

- Schema changes only
- Following code change changes on seperate commit

Issue-ID: CPS-1736
Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech>
Change-Id: I37f67a325d800a479c065ff72822b3e51d7f525f

10 months agoImproved code covage (branches) around cm handle state changes 06/135206/1
ToineSiebelink [Wed, 28 Jun 2023 11:09:29 +0000 (12:09 +0100)]
Improved code covage (branches) around cm handle state changes

- added test
- removed redundant (unreachable) check in production code

Issue-ID: CPS-475
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: Ia39ac7a1473a248710c2a52f65715fb6b6a85fca

10 months agoMerge "Subscription dmi_to_ncmp package clean-up"
Luke Gleeson [Wed, 28 Jun 2023 09:44:46 +0000 (09:44 +0000)]
Merge "Subscription dmi_to_ncmp package clean-up"

10 months agoMerge "Streamline outcome response for subscription creation"
Priyank Maheshwari [Wed, 28 Jun 2023 09:18:36 +0000 (09:18 +0000)]
Merge "Streamline outcome response for subscription creation"

10 months agoNormalize parent xpath when building datanodes in CpsDataService 81/135181/3
danielhanrahan [Mon, 26 Jun 2023 12:21:21 +0000 (13:21 +0100)]
Normalize parent xpath when building datanodes in CpsDataService

Data nodes are being saved with non-normalized xpaths, resuling in
data nodes that cannot be operated on. This affects all operations
including get, query, update, and delete.

Issue-ID: CPS-1765
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I5352182d79daec67805753ca5943b1a86c18159f

10 months agoSubscription dmi_to_ncmp package clean-up 97/135197/2
seanbeirne [Tue, 27 Jun 2023 15:56:31 +0000 (16:56 +0100)]
Subscription dmi_to_ncmp package clean-up

Issue-ID: CPS-1738
Signed-off-by: seanbeirne <sean.beirne@est.tech>
Change-Id: I7e01c89cb187233b53b8786e42d2fe2c0f24dbd9

10 months agoNCMP : Handle non-existing and non-ready cm handles 82/135182/6
sourabh_sourabh [Mon, 26 Jun 2023 10:15:57 +0000 (11:15 +0100)]
NCMP : Handle non-existing and non-ready cm handles

- Modified data operation schema to contains cm handle as steing and
  response content field name as result.
- Added a new common cloud event builder for NCMP to create an event.
- Added data operation event creater that uses cloud event builder to
  create a cloud event.
- Introduced a new method onto json object mapper to convert json object
  to bytes.
- Modified EventDateTimeFormatter and added a new method to convert date timestamp into offsetdateTime.
- Added a new code into ResourceRequestUtil to identify non-ready cm
  handle and non-existing cm handle and later publish it as cloud event
  to tha client given topic.
- Introduced CpsApplicationContext to get spring mannaged bean into non
  spring managed java object.

Issue-ID: CPS-1583, CPS-1614

Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: I24a39d2cb2c54dea25cd2f17e7748e21cd83a088
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
10 months agoGenerate docs using OpenAPI 3.0 85/135185/1
egernug [Tue, 27 Jun 2023 08:31:23 +0000 (09:31 +0100)]
Generate docs using OpenAPI 3.0

Doc generation in OpenAPI 3.0 requires executions for each file being generated.
This change adds those executions and updates the dependency to copy these files.

Issue-ID:CPS-1745

Signed-off-by: egernug <gerard.nugent@est.tech>
Change-Id: I144b49b0066f798f15408d955dcd914dd2c0bc91

10 months agoStreamline outcome response for subscription creation 59/134859/15
halil.cakal [Wed, 14 Jun 2023 07:10:15 +0000 (08:10 +0100)]
Streamline outcome response for subscription creation

- Pivot to outcome response generated from db
  for the corresponding subscription
- Fine tuning to wait till the cutoff time
  for all cm handles that subject to subscription
- Update cm handles as rejected if dmi plugin or
  cm handles does not exist in db
- Add more test for missed branches in mappers
  and response outcome

Issue-ID: CPS-1741
Change-Id: I82ab6958e673059977abc7708974d5a2cb13e339
Signed-off-by: halil.cakal <halil.cakal@est.tech>
10 months agoMerge "KafkaListener config for CloudEvents"
Sourabh Sourabh [Mon, 26 Jun 2023 12:17:36 +0000 (12:17 +0000)]
Merge "KafkaListener config for CloudEvents"

10 months agoKafkaListener config for CloudEvents 54/135154/3
mpriyank [Fri, 23 Jun 2023 14:01:38 +0000 (15:01 +0100)]
KafkaListener config for CloudEvents

- Introduced kafkalistener factory configs to correctly use the configs
  and convert the incoming events to CloudEvents
- Also legacy events to use a separate listener factory
- Increased the wait time from 100ms to 300ms to listen to the message
  in the test

Issue-ID: CPS-1764
Change-Id: I8b9fe0ba82ef87d52b7731941ccd3af8ae980109
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
10 months agoChange exclude path for mappers into more general 55/135155/1
halil.cakal [Mon, 26 Jun 2023 11:00:51 +0000 (12:00 +0100)]
Change exclude path for mappers into more general

Issue-ID: CPS-1741
Change-Id: I5ad92f19d65bbd7577014b6ea2ffbfffd41e0b00
Signed-off-by: halil.cakal <halil.cakal@est.tech>
10 months agoMerge "Adding logic to read response files from"
Toine Siebelink [Mon, 26 Jun 2023 09:46:10 +0000 (09:46 +0000)]
Merge "Adding logic to read response files from"

10 months agoMerge "#2 NCMP : Replacing the word 'Batch' as 'DataOperation'."
Toine Siebelink [Mon, 26 Jun 2023 09:29:58 +0000 (09:29 +0000)]
Merge "#2 NCMP : Replacing the word 'Batch' as 'DataOperation'."

10 months agoMerge "Chsnge to OpenApi 3.0"
Toine Siebelink [Mon, 26 Jun 2023 09:28:34 +0000 (09:28 +0000)]
Merge "Chsnge to OpenApi 3.0"

10 months agoAdding logic to read response files from 50/135050/5
waqas.ikram [Wed, 21 Jun 2023 13:16:22 +0000 (14:16 +0100)]
Adding logic to read response files from

filesystem for client integration tests

Issue-ID: CPS-1751
Change-Id: I62e24d09474184716c56149a06deae4500b7b286
Signed-off-by: waqas.ikram <waqas.ikram@est.tech>
10 months agoChsnge to OpenApi 3.0 43/135043/6
egernug [Wed, 21 Jun 2023 10:16:02 +0000 (11:16 +0100)]
Chsnge to OpenApi 3.0

In preparation for the Java 17 and Springboot 3 migrations Swagger Codegen has to upgrade to OpenApi 3.0
This effects cps-rest and cps-ncmp-rest

Issue-Id: CPS-1745

Signed-off-by: egernug <gerard.nugent@est.tech>
Change-Id: I18b8e9ca26d0123009702c2474efbfbeec6d4c44

10 months ago#2 NCMP : Replacing the word 'Batch' as 'DataOperation'. 35/135135/1
raviteja.karumuri [Fri, 23 Jun 2023 12:52:18 +0000 (13:52 +0100)]
#2 NCMP : Replacing the word 'Batch' as 'DataOperation'.

Issue-ID: CPS-1754
Signed-off-by: raviteja.karumuri <raviteja.karumuri@est.tech>
Change-Id: Ib04e9d8a909e8868bf094e2625412996409b3a67

10 months agoMerge "Improve code coverage"
Luke Gleeson [Fri, 23 Jun 2023 10:48:27 +0000 (10:48 +0000)]
Merge "Improve code coverage"

10 months agoMerge "NCMP : Replacing the word 'Batch' as 'DataOperation'."
Sourabh Sourabh [Fri, 23 Jun 2023 10:15:25 +0000 (10:15 +0000)]
Merge "NCMP : Replacing the word 'Batch' as 'DataOperation'."

10 months agoImprove code coverage 98/135098/1
ToineSiebelink [Thu, 22 Jun 2023 15:53:33 +0000 (16:53 +0100)]
Improve code coverage

cleaned up the modified test a bit as well

Issue-ID: CPS-475
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I0a6ac50bb5720935c5c1c7036ada8423b3174fda