seanbeirne [Wed, 21 Jun 2023 09:42:14 +0000 (10:42 +0100)]
Subscription Response Event Schema
Issue-ID: CPS-1738
Signed-off-by: seanbeirne <sean.beirne@est.tech>
Change-Id: I409baf66ca702166308821eeeff4887e9385148e
Toine Siebelink [Thu, 22 Jun 2023 10:17:18 +0000 (10:17 +0000)]
Merge "Improve performance of updateDataLeaves"
ToineSiebelink [Thu, 22 Jun 2023 08:24:04 +0000 (09:24 +0100)]
Add RTD link to CPS in Release notes
Issue-ID: REQ-1580
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: Id7f561a4686f62697bb98f7049199b44ebd387d4
Toine Siebelink [Wed, 21 Jun 2023 17:04:52 +0000 (17:04 +0000)]
Merge "Remove irrelevant groovy class that tests nothing"
ToineSiebelink [Wed, 21 Jun 2023 15:45:47 +0000 (16:45 +0100)]
Improve handling of legacy events
Legacy (non-cloud) events would cause a NPE in the event filter
Although the event woudl still be ignored it would lead to uncessary error logging
This fix addressed this issue using a trace level message instead
(also some improvements in the associated test)
Issue-ID: CPS-1724
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: Ibf71b13e4a47aaf705e32df5fe50cf41de5f558f
halil.cakal [Wed, 21 Jun 2023 15:54:09 +0000 (16:54 +0100)]
Remove irrelevant groovy class that tests nothing
Issue-ID: CPS-1741
Change-Id: If27a74e2496b952eb4803bbabb8363400c57915a
Signed-off-by: halil.cakal <halil.cakal@est.tech>
Toine Siebelink [Wed, 21 Jun 2023 14:44:41 +0000 (14:44 +0000)]
Merge "Patch # 3: Data operation response event (NCMP → Client App) to comply with CloudEvents"
danielhanrahan [Wed, 21 Jun 2023 12:53:32 +0000 (13:53 +0100)]
Improve performance of updateDataLeaves
Avoid fetching descendants during batchUpdateDataLeaves,
as descendants are not needed:
- Remove prefetch descendants step from getFragmentEntities;
- Explicitly prefetch descendants in operations requiring it;
- Update performance tests (5x faster for batch update).
Issue-ID: CPS-1675
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I7442a6f799cc0803b3a78f09d1ee53377f24b0b7
danielhanrahan [Wed, 21 Jun 2023 11:22:46 +0000 (12:22 +0100)]
Add performance tests for updateDataLeaves
Issue-ID: CPS-1675
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I4dcd2ff0d0f6c3acf7373357da0ef7217a691573
sourabh_sourabh [Mon, 19 Jun 2023 16:35:15 +0000 (17:35 +0100)]
Patch # 3: Data operation response event (NCMP → Client App) to comply with CloudEvents
- Modified data operation record strategy to consume cloud event.
- Modified NCMP data operation event consumer to read cloud event
header. (prefixed with ce_)
- Modified event publisher to support legacy and cloud event based on
event type (if legacy event use legacy kafka template else cloud kafka
template).
- Introduced a new method onto json object mapper to convert json object
to bytes.
- Modified data operation consumer spec to produce a cloud event and validate it.
- Added Kafka Integration Test (for filtering)
Issue-ID: CPS-1724
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: Ide701b1ff952f57413cb4e4aa0d55c08753f0298
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
danielhanrahan [Tue, 20 Jun 2023 17:19:59 +0000 (18:19 +0100)]
publishLcmEventBatchAsynchronously is not async
Spring @Async has a limitation where calling the async method from
within the same class won't work, as the method needs to be proxied,
and calling from the same class bypasses the proxy and calls the
underlying method directly, thus running synchronously.
Issue-ID: CPS-1749
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I7f198487dc18b359654dc38b5cf8fd600d33e189
ToineSiebelink [Wed, 21 Jun 2023 09:13:46 +0000 (10:13 +0100)]
Apply latest ReedTheDoc config rules
Issue-ID: CPS-1757
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: Icdf5fe5b5b1d6628bed786a1d99816e7990302cc
waqas.ikram [Fri, 16 Jun 2023 12:05:04 +0000 (13:05 +0100)]
Refactored code to enable its utilization in Junit
tests for the client
Change-Id: I847ae0f48444af907039bfddb9879581003c0f35
Issue-ID: CPS-1751
Signed-off-by: waqas.ikram <waqas.ikram@est.tech>
Toine Siebelink [Tue, 20 Jun 2023 09:00:13 +0000 (09:00 +0000)]
Merge "Fix test-deregistration script"
Luke Gleeson [Tue, 20 Jun 2023 08:38:01 +0000 (08:38 +0000)]
Merge "DMI Data AVC to cloud events"
Toine Siebelink [Tue, 20 Jun 2023 08:36:39 +0000 (08:36 +0000)]
Merge "Patch # 1: Data operation response event (NCMP → Client App) to comply with CloudEvents"
Toine Siebelink [Tue, 20 Jun 2023 07:30:56 +0000 (07:30 +0000)]
Merge "Replace deprecated WebSecurityConfigurerAdapter"
danielhanrahan [Fri, 2 Jun 2023 10:44:09 +0000 (11:44 +0100)]
Fix test-deregistration script
- Fix issue where script breaks with newer version of docker-compose
caused by profile names being omitted during `docker-compose down`
- Use main .gitignore file to ignore metrics-reports directory
- Fix Intelli-j/shellcheck warnings (mostly variables needing quotes)
- Script prints summary report at exit
Issue-ID: CPS-1552
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ia51e51c1c199de8cc2f4878bfb04cb09fbb0b0e7
danielhanrahan [Mon, 19 Jun 2023 13:44:21 +0000 (14:44 +0100)]
Update performance test timings
The fixes to lower memory usage in FragmentRepository have
positively affected read performance, so tests must be updated.
Issue-ID: CPS-1716
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I2c8b882f56fb8c1eaefe5b802da1d308d16b1f02
sourabh_sourabh [Thu, 8 Jun 2023 14:25:33 +0000 (15:25 +0100)]
Patch # 1: Data operation response event (NCMP → Client App) to comply with CloudEvents
- Removed header definitions (since we now use CloudEvents)
- Used 'dataOperation' instead of batch where appropriate.
- Modified test json
Issue-ID: CPS-1724
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: Ic0f65297b944adf9cf5f3c2cbec679a031a675ec
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Priyank Maheshwari [Mon, 19 Jun 2023 13:49:11 +0000 (13:49 +0000)]
Merge "Code to persist or update the yang model subscription into db"
halil.cakal [Fri, 9 Jun 2023 13:58:00 +0000 (14:58 +0100)]
Code to persist or update the yang model subscription into db
- Change subscription persistence for subscription registry
to handle adding or updating cm handles into db
Issue-ID: CPS-1735
Change-Id: Icae609e810b2a097a4a174de1691892526f6b5c0
Signed-off-by: halil.cakal <halil.cakal@est.tech>
mpriyank [Fri, 16 Jun 2023 14:55:52 +0000 (15:55 +0100)]
DMI Data AVC to cloud events
- DMI Data AVC to be consumed as CloudEvents now
- Removed the schema header as it is taken care by cloudevent headers
- Implemented naming and packaging comments on the schema
- Test cases refactoring
Issue-ID: CPS-1719
Change-Id: I9864f90446720fe903fb3c1502d86035d886751d
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
ToineSiebelink [Mon, 12 Jun 2023 15:24:02 +0000 (16:24 +0100)]
Move integration test (clean-up, last phase)
- Moved session manager test
- Improved schemaset concurrency test (retry testing)
- Cleaned up all ri (container based) integration test (setup)
- Applied some groovy best practice where needed
- internal ri module cover now down to 29%, covred by integration instead with
- Line coverage up by 41 lines to 99% (was 97%)
- Branch coverage up by 3 branches to 96% (was 93%)
Issue-ID: CPS-1687
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: Ifb77a053e5a5db62a3f6a32ae60a3a8b10918efd
Toine Siebelink [Thu, 15 Jun 2023 16:44:02 +0000 (16:44 +0000)]
Merge "Patch #2 : Introduce kafka template for cloud events"
sourabh_sourabh [Thu, 15 Jun 2023 02:12:29 +0000 (03:12 +0100)]
Patch #2 : Introduce kafka template for cloud events
- Introduced a new cloud kafka template for cloud events that reads
it's configuration from application.yml
- Kept legacy kafka template for backward compatibility utill all cps
events moved to cloud event comply.
- Modified application.yml producer and consumer value deserializer properties to support cloud events.
- Added new cloudevents-bom used into cps-ncmp-service pom.
- For the time being we will have 2 kafkatemplates (legacyEventKafkaTemplate, cloudEventKafkaTemplate) into EventsPublisher until we fully move to
cloudevents for all events. Once all cps events will be cloud event compy, we have TODO task where Deprecated: legacyKafkaEventTemplate will be removed with its
java configuration file KafkaTemplateConfig.
Issue-ID: CPS-1724
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: I78c15bd480db063b89c6630c46c2d3a328b4fae4
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
danielhanrahan [Thu, 15 Jun 2023 12:52:34 +0000 (13:52 +0100)]
Bump CPS version to 3.3.3-SNAPSHOT
- Update version properties as well as we prepare for the next release.
Issue-ID: CPS-1750
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I0626eee7834499d2fa96812fb416ce7daebf0529
danielhanrahan [Thu, 15 Jun 2023 12:31:24 +0000 (13:31 +0100)]
Add container yaml for 3.3.2
- Step 6 of the release process
Issue-ID: CPS-1703
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I58e54ecd17878d3b9b6bc69ac9d254e454bd68b2
danielhanrahan [Thu, 15 Jun 2023 11:37:24 +0000 (12:37 +0100)]
Add maven stage yaml for 3.3.2
- Step 5 of the release process
Issue-ID: CPS-1703
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I105dedd6b1a634ca8ac4409ed682f6a10d1b018a
egernug [Thu, 15 Jun 2023 10:44:50 +0000 (11:44 +0100)]
Replace deprecated WebSecurityConfigurerAdapter
WebSecurityConfigurerAdapter has been deprecated in Spring 2.7. Change needed to use SecurityFilterChain and InMemoryUserDetailsManager.
Note: WebMvcTest does not autoimport configurations that use beans.
Issue-Id: CPS-1742
Signed-off-by: egernug <gerard.nugent@est.tech>
Change-Id: I100884f10beaac6ce87e603302280aeb78559017
danielhanrahan [Thu, 15 Jun 2023 10:08:54 +0000 (11:08 +0100)]
Update release docs
- Also synched up the API changes in openapi yml file
- Step 1 and 2 of the release process
Issue-ID: CPS-1703
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Id47822737df3b89dffddb90413a53cac2b7ced6a
danielhanrahan [Thu, 8 Jun 2023 13:37:17 +0000 (14:37 +0100)]
Lower memory usage in FragmentRepository
Avoid using Spring Data "interface projection" in FragmentRepository.
The use of FragmentExtract in FragmentRepository is causing an
overhead of around 5 kilobytes per fragment, which is leading to
abnormally high memory usage when queries return a large number of
nodes. For example, around 250MB of additional memory is needlessly
used when fetching 50,000 datanodes.
- Remove FragmentExtract interface and FragmentEntityArranger class.
- Add FragmentPrefetchRepository, using JdbcTemplate and RowMapper
to fetch FragmentEntity descendants in a single SQL query.
- Many CpsDataService operations have memory reductions:
- queryDataNodes
- getDataNodesForMultipleXpaths
- updateDataNodesAndDescendants
- updateNodeLeaves
- and any NCMP methods using the above.
Issue-ID: CPS-1716
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ic47a2c9eb34150ed76bd5ce452fe1c9aaf9b4c5c
Toine Siebelink [Tue, 13 Jun 2023 10:56:29 +0000 (10:56 +0000)]
Merge "Move integration test (DataService)"
danielhanrahan [Mon, 12 Jun 2023 15:24:55 +0000 (16:24 +0100)]
Fix for recursive SQL returning extra level of descendants
Recursive SQL queries are incorrectly returning maxDepth + 1 levels
of fragments due to incorrect comparison in the WHERE clause.
This code is used by getDataNodes and queryDataNodes to
prefetch the descendants before converting to datanodes.
Prefetching extra descendants causes high memory usage.
Issue-ID: CPS-1716
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Ie22215c0c7018cd293f73cf75adca2cd55f218a6
ToineSiebelink [Tue, 23 May 2023 08:30:37 +0000 (09:30 +0100)]
Move integration test (DataService)
- Moved all Data Intergation Test
- Added/improved some module Moudle Servcie test too
- Removed some unused (impossible) code-paths in prod code
- Replaced null's with meaningful strings (in edge case scenarios)
- Replaced @Shared with static (shorter) in some test
Issue-ID: CPS-1687
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: Ib4b1ec1706ace7e3f5ac03d85177a6f7d4f6b8d0
halil.cakal [Tue, 6 Jun 2023 15:20:55 +0000 (16:20 +0100)]
Agreed outstanding comments including package refactoring
- Change package of AVC Subscription relevent codes into their place
- Change package of Subscription Event Cache config inot its place
- Add more branches for subscription outcome mapper
- Add more branches for subscription event response consumer
- Change unit test method params in order not to use deprecated methods
Issue-ID: CPS-1730
Change-Id: Ieda587d5be318db8360d52d49dc38d7ce3dd85cd
Signed-off-by: halil.cakal <halil.cakal@est.tech>
halil.cakal [Wed, 7 Jun 2023 10:02:30 +0000 (11:02 +0100)]
Fix some code smells
Issue-ID: CPS-1507
Change-Id: Ic11ccbb37538242cb65ab1dbdde4fc77d9f94bdf
Signed-off-by: halil.cakal <halil.cakal@est.tech>
Sourabh Sourabh [Wed, 7 Jun 2023 08:49:32 +0000 (08:49 +0000)]
Merge "NCMP : forward bulk response messages to client topic"
raviteja.karumuri [Fri, 2 Jun 2023 09:45:13 +0000 (10:45 +0100)]
NCMP : forward bulk response messages to client topic
# Fixing the avc subscription event is not consuming even there is a record published on to the topic.
Issue-ID: CPS-1557
Signed-off-by: raviteja.karumuri <raviteja.karumuri@est.tech>
Change-Id: If09fd1849f467785141cc56639839ddda9f2c0de
Luke Gleeson [Tue, 6 Jun 2023 08:45:35 +0000 (08:45 +0000)]
Merge "<,> operators document"
halil.cakal [Mon, 8 May 2023 13:18:26 +0000 (14:18 +0100)]
Subscription Create Event Outcome Kafka Part
- Add subscription event outcome schema with java type for pojos
- Add subscription event outcome json for testing
- Add mapper to convert subscription response to event outcome
- Add a bean to handle subscription response outcome tasks
- Change response consumer to publish outcome for client app
- Change response timeout task to publish outcome for client app
- Change subscription persistance to read datanodes
- Add helper to extract cm handle to status mapping from data nodes
event
- Fix code smells
Issue-ID: CPS-1507
Change-Id: I70195073490f456f014e53c1f59d1b6761d18cd4
Signed-off-by: halil.cakal <halil.cakal@est.tech>
raviteja.karumuri [Thu, 1 Jun 2023 09:05:44 +0000 (10:05 +0100)]
NCMP : forward bulk response messages to client topic
# Fixing the NullPointer Exception if the 'eventType' header not availabe.
# eventType header is a mandatory header but still we are doing null check for supporting old events (AsyncResponseEvent is not moved to separate kafka headers)
Issue-ID: CPS-1557
Signed-off-by: raviteja.karumuri <raviteja.karumuri@est.tech>
Change-Id: Ie7923d0e2674402fa36cb28cde966575b899cedb
Rudrangi Anupriya [Wed, 31 May 2023 16:03:32 +0000 (21:33 +0530)]
<,> operators document
Issue-ID: CPS-1273
Change-Id: Ibbbbe90a28c3711ef211136cd8f8c0f265a51537
Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>
Luke Gleeson [Wed, 31 May 2023 11:07:57 +0000 (11:07 +0000)]
Merge "Add <,> operators support to cps-path"
Rudrangi Anupriya [Tue, 30 May 2023 11:50:20 +0000 (17:20 +0530)]
Add <,> operators support to cps-path
Issue-ID: CPS-1273
Change-Id: I5d562463b9a49abfe0436047a637857d10596fff
Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>
Priyank Maheshwari [Tue, 30 May 2023 10:24:25 +0000 (10:24 +0000)]
Merge "Improve error scenarios SubscriptionEventForwarder"
Toine Siebelink [Mon, 29 May 2023 08:52:22 +0000 (08:52 +0000)]
Merge "NCMP: Update existing Batch endpoint (Moving url param into rest body)"
lukegleeson [Thu, 18 May 2023 15:40:04 +0000 (16:40 +0100)]
Improve error scenarios SubscriptionEventForwarder
- Added placeholder for emtpy outcome response when no cmhandles match targets
Issue-ID: CPS-1564
Signed-off-by: lukegleeson <luke.gleeson@est.tech>
Change-Id: Ia04bb656c639946d259da779971ab29b6e91acea
sourabh_sourabh [Tue, 9 May 2023 13:34:59 +0000 (14:34 +0100)]
NCMP: Update existing Batch endpoint (Moving url param into rest body)
- NCMP batch endpoint is updated to accept details into request payload.
- Removed unused code of previous impl.
Issue-ID: CPS-1635
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: Ic290b750557da06b861c5a4a9bb12debc495ec2e
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
mpriyank [Thu, 18 May 2023 13:56:07 +0000 (14:56 +0100)]
Avc Event compliant with RFC8641
- The event section of the payload will be now in the format of RFC8641.
- Once this event is updated , the corresponding code in the dmi plugin
will be changed
- Schema is present in cps-ncmp codebase so we will create a release and
then it will be used in the dmi plugin
Issue-ID: CPS-1668
Change-Id: Ibd811f318658d8b510b42575fad5d21357af9047
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
raviteja.karumuri [Thu, 13 Apr 2023 22:44:41 +0000 (23:44 +0100)]
NCMP : forward bulk response messages to client topic
Issue-ID: CPS-1557
Signed-off-by: raviteja.karumuri <raviteja.karumuri@est.tech>
Change-Id: I0ea040e98987f992f46105afea0a171c4031d64f
Luke Gleeson [Mon, 22 May 2023 08:48:28 +0000 (08:48 +0000)]
Merge "LcmEvent to have header now"
ToineSiebelink [Wed, 17 May 2023 15:40:44 +0000 (16:40 +0100)]
Entity ID types do not match types in database definition
As Liquibase is used, JPA does not manage database definition.
Within Liquibase changelog, only Anchor and Fragment have BIGINT (Long)
IDs. Other entities have INTEGER IDs.
- Change AnchorEntity/AnchorRepository ID type to Long
- Change YangResourceEntity/YangResourceRepository and
ModuleReferenceRepository ID to Integer
- Modify datatypes of foreign key references in liquibase changelog to
use correct types for dataspace_id, schema_set_id, yang_resource_id
- Update test liquibase changelog
Issue-ID: CPS-1676
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I26de6712b2e74d166fc5a48b1f8149de6a1e5199
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Luke Gleeson [Wed, 17 May 2023 13:17:25 +0000 (13:17 +0000)]
Merge "Change dmi plugin version to 1.4.0-SNAPHOST"
halil.cakal [Tue, 16 May 2023 10:06:57 +0000 (11:06 +0100)]
Change dmi plugin version to 1.4.0-SNAPHOST
Issue-ID: CPS-1507
Change-Id: Ic812bd1be455d9df663e018383dd1d5c15359362
Signed-off-by: halil.cakal <halil.cakal@est.tech>
ToineSiebelink [Mon, 15 May 2023 16:32:28 +0000 (17:32 +0100)]
Move integration test for adminService
- finalized moving of admin service interation test
(just some scenarios were missing)
- deleted old test class (code coverage in RI module dropped)
- line/branch coverage now 100/100 (was 97/91)
- small refactoring; renaming and removed unnecessary check
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: Ic683548ea5e1e4e252f257c0f1034c5cf76e498d
mpriyank [Thu, 11 May 2023 12:55:55 +0000 (13:55 +0100)]
LcmEvent to have header now
- Introduce LcmEventHeader as per our agreement with stakeholders
- The version remains v1 as we just add the header and dont change
anything in the existing event payload.
- Later we will remove the header fields from the event payload.
- tests modification and little code refractor for dmi data avc as well
Issue-ID: CPS-1695
Change-Id: Ibef1138a6d0cc7ffec50b4c201a4d3417b99e27e
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
waqas.ikram [Mon, 15 May 2023 11:03:33 +0000 (12:03 +0100)]
Fixing minor compilation
- Complilation issues while importing project in eclipse
- Fixing CSIT
Change-Id: Idb85b08066d27a7f2a9b297d1580915df2b67541
Issue-ID: CPS-1672
Signed-off-by: waqas.ikram <waqas.ikram@est.tech>
danielhanrahan [Thu, 4 May 2023 21:34:41 +0000 (22:34 +0100)]
Remove dataspace_id column from Fragment table
- Remove dataspace_id column from fragment table and drop associated
index, foreign key constraint and uniqueness constaint.
- Add uniqueness constraint using only fragment xpath and anchor_id.
- Add not-null constraint to anchor_id.
- Update code and test data to remove references to dataspace_id.
- Repopulate the dataspace_id in Liquibase rollback
(verified with manual testing of update/rollback).
Issue-ID: CPS-1677
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I467cccba25ac5d884ec790064ca58150281b7e1d
ToineSiebelink [Mon, 8 May 2023 16:54:46 +0000 (17:54 +0100)]
Move integration test for moduleService
- all modules service interation test moved
- added some aditional scenarios to increase coverage
- removed unused method(s)
- remove javdoc on private methods
- some minor refactoring like replace 'var' with correct type
Issue-ID: CPS-1687
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I03ff1f3562cfe38318e8b9af81be47a1fe667da2
halil.cakal [Wed, 3 May 2023 12:22:33 +0000 (13:22 +0100)]
Subscription Create Event Outcome Database Part
- Add mapper to convert subscription response to Yang model
- Add update operation to update Yang model into database
- Change Subscription persistence to support both save and update operations
Issue-ID: CPS-1669
Change-Id: I40cab1052ada5846945c67cac8640c393358e988
Signed-off-by: halil.cakal <halil.cakal@est.tech>
Toine Siebelink [Wed, 10 May 2023 14:20:09 +0000 (14:20 +0000)]
Merge "DMI Data AVC to use kafka headers"
mpriyank [Thu, 4 May 2023 10:24:29 +0000 (11:24 +0100)]
DMI Data AVC to use kafka headers
- POC done keeping AvcEvent schema in mind.
- Approach to have header schema per event schema.
- Moved the header information from AvcEvent to separate AvcEventHeader
schema.
- Added Jsr303 annotation support for required field check
Issue-ID: CPS-1671
Change-Id: I2e4f969e8ca4f6282d1b9aa5fd52d16174a26084
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
ToineSiebelink [Mon, 8 May 2023 16:07:54 +0000 (17:07 +0100)]
Move integration test (DataService)
- dropped ri module coverage to alow for move
- removed performance tests (already better tests in intergration module)
- added 'save' iue cases in new integration-test
- removed no longer used version for save single data node
(todo: remove old tests for similar functionality, will wait until new test cover same)
Issue-ID: CPS-1687
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I4c2e788e0a313938a923cd1e7e52584d739ec6f4
Priyank Maheshwari [Wed, 10 May 2023 11:25:24 +0000 (11:25 +0000)]
Merge "Change default use of Model Loader to TRUE"
danielhanrahan [Tue, 9 May 2023 12:33:37 +0000 (13:33 +0100)]
Migrate cps-ri performance tests to integration-test module
- Migrate update and delete data service tests, using openroadm model
- Move module service tests to integration-test module
- Update performance test timings
- Remove all performance-related files from cps-ri
- Remove performance profile from cps-ri pom.xml
Issue-ID: CPS-1687
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: Id9d864f8cab0377cb37c7967943d738748e1b6c5
Toine Siebelink [Tue, 9 May 2023 12:44:19 +0000 (12:44 +0000)]
Merge "Support for Patch across multiple data nodes"
--global [Wed, 5 Apr 2023 11:38:08 +0000 (17:08 +0530)]
Support for Patch across multiple data nodes
- Added new method updateMultipleDataLeaves to perform Update on
multiple data nodes
- Deprecated singular method of update data node(updateDataLeaves)
- Refactored code where singular version was used
- Updated release notes
Issue-ID: CPS-1006
Signed-off-by: <as00745003@techmahindra.com>
Change-Id: If67280e2dd3ad566de9a8217489f168415e624bc
sourabh_sourabh [Tue, 9 May 2023 09:34:02 +0000 (10:34 +0100)]
NCMP: Delete DatastoreType enum from cps-ncmp-rest
- Delete duplicate enum of datastore
- Use datastore from cps-ncmp-service
Issue-ID: CPS-1635
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: I36484d6a327c81d81741b18a932fc6a1febfc520
emaclee [Tue, 9 May 2023 08:18:05 +0000 (09:18 +0100)]
Change default use of Model Loader to TRUE
Issue-Id: CPS-1688
Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech>
Change-Id: I18bb5e3ad45ccdf5f3a1e88dd5a34dc830e06f4a
Sourabh Sourabh [Mon, 8 May 2023 09:57:37 +0000 (09:57 +0000)]
Merge "Use recursive SQL to fetch descendants in CpsPath queries (CPS-1664 #4)"
Sourabh Sourabh [Mon, 8 May 2023 09:51:19 +0000 (09:51 +0000)]
Merge "Fix and refactor query across anchors (CPS-1664 #3)"
Toine Siebelink [Mon, 8 May 2023 09:26:46 +0000 (09:26 +0000)]
Merge "[CPS] RI: Code Refactoring # Replace '[\s\S]' to '.' as it is same as '.' while grouping in regex # Updated the groovy files to add 'assert' and to replace deprecated method # Removed deprecated 'storeDataNode' method."
Sourabh Sourabh [Mon, 8 May 2023 09:06:32 +0000 (09:06 +0000)]
Merge "Don't handle ancestor-axis using regex quickfind (CPS-1664 #2)"
raviteja.karumuri [Fri, 5 May 2023 13:52:07 +0000 (14:52 +0100)]
[CPS] RI: Code Refactoring
# Replace '[\s\S]' to '.' as it is same as '.' while grouping in regex
# Updated the groovy files to add 'assert' and to replace deprecated method
# Removed deprecated 'storeDataNode' method.
Issue-ID: CPS-1594
Signed-off-by: raviteja.karumuri <raviteja.karumuri@est.tech>
Change-Id: Iaa5f8b0cc8d66473c993060a8fc5a54e37e373b1
danielhanrahan [Thu, 4 May 2023 22:19:19 +0000 (23:19 +0100)]
Fragment Entity does not represent the correct relationship with Anchor
FragmentEntity incorrectly has OneToOne relationship to AnchorEntity,
instead of ManyToOne. The reason this has not caused errors so far is
that Liquibase is being used to generate database definition, not JPA.
Issue-ID: CPS-863
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I8d3c5dc19193a84fa59526de7f2cbab7ab20a8fc
danielhanrahan [Wed, 3 May 2023 18:51:27 +0000 (19:51 +0100)]
Use recursive SQL to fetch descendants in CpsPath queries (CPS-1664 #4)
- Add recursive SQL method to fetch descendants from queries.
This changes worst-case complexity from quadratic to linear,
resulting in extremely large performance increase for large
number of datanodes.
- Remove RegexQuickFind algorithm as it is no longer faster.
- Updated query performance test timings
Issue-ID: CPS-1664
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: If5f0b54a88af1cb681006bbeca7043345dcdc8da
danielhanrahan [Tue, 2 May 2023 19:01:34 +0000 (20:01 +0100)]
Fix and refactor query across anchors (CPS-1664 #3)
- Fix CPS-1580: Query Across All Anchors Does NOT Filter on Dataspace
- Fix CPS-1582: NullPointerException in queryDataNodesAcrossAnchors
- Improve performance of queryDataNodesAcrossAnchors
- Refactor queryDataNodes
- Refactor FragmentQueryBuilder
Issue-ID: CPS-1580
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I8b83d4c580280d8719ab2ac33f3914f53e798774
danielhanrahan [Tue, 2 May 2023 20:37:56 +0000 (21:37 +0100)]
Don't handle ancestor-axis using regex quickfind (CPS-1664 #2)
This is a preparatory patch to make refactoring easier in CPS-1664 #3
Issue-ID: CPS-1664
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I1f94fef137ae41f7dedc585d8314f13493885d1b
Toine Siebelink [Thu, 4 May 2023 14:14:08 +0000 (14:14 +0000)]
Merge "Sensible equals and hashCode for FragmentEntity (CPS-1664 #1)"
ToineSiebelink [Thu, 4 May 2023 10:35:41 +0000 (11:35 +0100)]
Fix for quickfind with descendants incl. list entries
- add specialized regex
- fixed legacy issues with absolute paths (added ^ in regex's)
- split into 2 methods for normal queries and quickfind
- fixed order of private methods
Issue-ID: CPS-1671
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: I785a230cd0e3e0359e3cbb0fa754a21b6e2be7ff
danielhanrahan [Wed, 3 May 2023 14:07:34 +0000 (15:07 +0100)]
Sensible equals and hashCode for FragmentEntity (CPS-1664 #1)
Include Anchor and Xpath in equals and hashCode methods for
FragmentEntity. (This also requires adding equals and hashCode for
AnchorEntity and DataspaceEntity.) The combination of dataspace,
anchor, and xpath uniquely identify a fragment/datanode.
This allows FragmentEntity objects returned from query across anchors
to be stored in Set collections.
Performance was observed to be unaffected by the change.
Issue-ID: CPS-1664
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: I2c7e3957e392af36f5230d08c9bbd550f44e7444
ToineSiebelink [Wed, 3 May 2023 17:21:46 +0000 (18:21 +0100)]
Refactor datatrore handling in API requests
- Fully parameterized datastore(name)
- Improved controler testing to verify all parameter handling
- Simplified service inject (alwasy using contructors instead of @AutoWire)
- Added delay to fix isuse with intermitting failign test
that tests invocation of a method that is executed on a separate thread
Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
Change-Id: Iba5d118d2484badee9c7c90ec7694882f6557d16
mpriyank [Wed, 3 May 2023 10:10:08 +0000 (11:10 +0100)]
Bump CPS version to 3.3.2-SNAPSHOT
- Update version properties as well as we prepare for the next release.
Issue-ID: CPS-1666
Change-Id: I44e44ac1755c44d7050ce30c9e40f83628fe9338
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
Sourabh Sourabh [Wed, 3 May 2023 09:54:31 +0000 (09:54 +0000)]
Merge "Add container yaml for 3.3.1"
Toine Siebelink [Wed, 3 May 2023 09:52:00 +0000 (09:52 +0000)]
Merge "canUseRegexQuickfind does not test for contains-condition"
mpriyank [Wed, 3 May 2023 09:36:30 +0000 (10:36 +0100)]
Add container yaml for 3.3.1
- Step 6 of the release process
Change-Id: I18185934b9c9e801d0269652658ae3965da5b119
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
danielhanrahan [Tue, 2 May 2023 17:00:53 +0000 (18:00 +0100)]
canUseRegexQuickfind does not test for contains-condition
This bug causes queries using contains-condition to return wrong
results, when INCLUDE_ALL_DESCENDANTS option is set.
Issue-ID: CPS-1663
Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech>
Change-Id: If323e0e3b54d8c9a213dffda46897b2efb01fe2a
mpriyank [Wed, 3 May 2023 09:17:09 +0000 (10:17 +0100)]
Add maven stage yaml for 3.3.1
- Step 5 of the release process
Change-Id: I5b796f909395e10c0b4f783358496719f0d4b492
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
mpriyank [Tue, 2 May 2023 14:58:55 +0000 (15:58 +0100)]
Update release docs
- Also synched up the API changes in openapi yml file
- Step 1 and 2 of the release process
Issue-ID: CPS-1662
Change-Id: I5b9485497a27a3dee8639faedfa9c1bd8ed3265e
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
mpriyank [Fri, 28 Apr 2023 14:02:00 +0000 (15:02 +0100)]
Align the hazelcast cluster names
- We need to align the hazelcast cluster names as members of the same
cluster can join each other. We will still have exclusivity as the
instance names and configs are different for each distributed object.
- Exposing env variable to override the cluster name depending on the
env it is run on.
- Modified test cases to validate the cluster names as well
Issue-ID: CPS-1637
Change-Id: Ib0f8c80dc9b2268f976b0c2d3ccd6d64792d4781
Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
Luke Gleeson [Fri, 28 Apr 2023 13:59:06 +0000 (13:59 +0000)]
Merge "Add document for support for contains condition to cps-path"
lukegleeson [Thu, 6 Apr 2023 14:28:56 +0000 (15:28 +0100)]
Timeout for Subscription Create Partial Response
- Implemented default 30s timeout for DMI Responses
- Placeholders have been TODO'd for Outcome Response generation and Persisted Subscription Updating
- Refactored common HazelcastCacheConfig methods
- Some tests required a blocking variable due to seperate thread usage
Issue-ID: CPS-1599
Signed-off-by: lukegleeson <luke.gleeson@est.tech>
Change-Id: I2b1a35e93939daa0524d379ac4736d714ef61a6f
Luke Gleeson [Thu, 27 Apr 2023 15:26:21 +0000 (15:26 +0000)]
Merge "Add retry mechanism on Subscription loader"
emaclee [Sun, 16 Apr 2023 16:48:15 +0000 (17:48 +0100)]
Add retry mechanism on Subscription loader
Issue-ID: CPS-1568
Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech>
Change-Id: I35697c260cc1a774f50d12552996b39f812fc2de
sourabh_sourabh [Mon, 27 Mar 2023 13:36:20 +0000 (14:36 +0100)]
CPS-1553 :REST endpoint to accept collection of cm handles for GET operation
-Introduced an interface having all un-implemented or default defination
of any method that is further extended or implemented by abstract or
stubbed controller.
-Exposed an endpoint to accept bulk request
-Modified an existing methods for more readability. (Parameterized datasource for ... methods)
-Code impl. for getResourceDataFromDmi method to send bulk request to
new proposed dmi-plugin endpoint that would accept bulk request.
-We need to investigate a groovy test that fails only into onap CICD pipeline.
For now it's commented out by added TODO tag. (org/onap/cps/ncmp/rest/controller/NetworkCmProxyControllerSpec.groovy:205)
Issue-ID: CPS-1553
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Change-Id: Ieac39690956e3a0eab41068db57c5d93a16d04f0
Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
Rudrangi Anupriya [Thu, 27 Apr 2023 06:34:52 +0000 (12:04 +0530)]
Add document for support for contains condition to cps-path
Issue-ID: CPS-1272
Change-Id: I32c03d5ed80e5d16d4b5c0f726539a6da487abd6
Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>
Luke Gleeson [Thu, 27 Apr 2023 09:02:23 +0000 (09:02 +0000)]
Merge "Ordering of leaf elements to support combination of AND/OR in cps-path"
raviteja.karumuri [Tue, 25 Apr 2023 18:41:29 +0000 (19:41 +0100)]
[CPS] RI, Service: Code Refactoring
# Remove unused method "updateDataNodeAndDescendants"
# Remove unused production code
Issue-ID: CPS-1594
Signed-off-by: raviteja.karumuri <raviteja.karumuri@est.tech>
Change-Id: Id909e9da3e9181c181dfdfe62f051451d8e488a2
Rudrangi Anupriya [Tue, 25 Apr 2023 07:43:16 +0000 (13:13 +0530)]
Ordering of leaf elements to support combination of AND/OR in cps-path
Issue-ID: CPS-1629
Change-Id: Ib9df87e8171f4d53c49d80d27e3c6cf75aa3b209
Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>