1 .. This work is licensed under a Creative Commons Attribution 4.0
2 International License. http://creativecommons.org/licenses/by/4.0
8 CCVPN (Cross Domain and Cross Layer VPN)
9 ----------------------------------------
10 Update for kohn Release
11 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
13 The kohn release enhances the CCVPN use-case by introducing the following three features (REQ-1268):
14 1. E-LINE (P2P connection) support for the Cloud Leased Line (CLL) service delivery
15 2. Enhancing the Closed-Loop Automation of CCVPN services by using DCAE SDK dmaap-client lib in slice analysis MS
16 3. Enhancing TN NSSMF NBI to align with the latest IETF specification (SO changes)
18 Kohn Scope and Impacted modules
19 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
21 The Kohn release is an enhancement release. E-LINE service model (P2P connection) is added to the Cloud Leased Line (CLL) service.
22 Also, slice analysis MS is enhanced to use DCAE SDK dmaap-client lib.
23 And lastly, TN NSSMF northbound is aligned with the latest IETF transport slice definition model (SO changes).
25 The impacted ONAP modules are: CCSDK, SDN-C, DCAE, and SO.
27 Installation Procedure
28 ~~~~~~~~~~~~~~~~~~~~~~
30 For Kohn new features, the integration test environment is similar to that of
31 the Jakarta release: an ONAP instance with Kohn release interfacing with 3rd party
32 transport domain controllers should be established.
34 Functional/Integration Test Cases
35 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
37 The testing procedure is described in the following few test cases:
38 - Create and delete single CLL instance which accesses single cloud, and monitor if the closed-loop call flow is getting triggered.
39 - Modify the bandwidth of a connection link of an existing CLL instance, and monitor if the closed-loop call flow is getting triggered.
42 Update for Jakarta Release
43 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
45 The Jakarta release enhances the CCVPN use-case by introducing the following three features (REG-1076):
46 1. Support for IBN service discovery by registering Cloud Leased Line (CLL) and Transport Slicing services to MSB
47 2. Support for 1+1 protection of Cloud Leased Line (CLL)
48 3. Support for closed-loop and user-triggered intent update
50 Jakarta Scope and Impacted modules
51 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
53 The "CCVPN closed-loop" feature and the "user-triggered intent update" feature require both a front-end and a back-end system.
54 The front-end would be different for IBN and CCVPN, but the two features can share a common back-end.
55 As a first step, current bandwidth usage of a CLL should be collected from the physical network. Then VES collector API
56 should be called to send this information to DCAE. DCAE would then publish a new DMaaP topic to be consumed by DCAE slice
57 analysis micro-service. This module will then send this notification to Policy.
59 In Jakarta, the goal of both user-triggered intent update and CCVPN closed-loop is to ensure the max-bandwidth of the CLL service
60 can satisfy user's intent throughout the intent life cycle. Thus, the modify-CLL operation triggered by DCAE and Policy is
61 common to IBN and CCVPN. So a common back-end mechanism is implemented to support both use-cases.
63 The impacted ONAP modules are: CCSDK, SDN-C, A&AI, DCAE, POLICY, and SO.
65 Installation Procedure
66 ~~~~~~~~~~~~~~~~~~~~~~
68 For Jakarta new features, the integration test environment is similar to that of
69 the Istanbul release: an ONAP instance with Istanbul release interfacing with 3rd party
70 transport domain controllers should be established.
72 Functional/Integration Test Cases
73 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
75 The testing procedure is described in the following few test cases:
76 - Create and delete single CLL instance which accesses single cloud, and monitor if the closed-loop call flow is getting triggered.
77 - Create and delete single CLL instance which access multiple clouds, and monitor if the closed-loop call flow is getting triggered.
78 - Create and delete multiple CLL instances which access single cloud, and monitor if the closed-loop call flow is getting triggered.
79 - Create and delete multiple CLL instances which access multiple clouds, and monitor if the closed-loop call flow is getting triggered.
80 - Create a CLL instance which have connection links with different bandwidth, and monitor if the closed-loop call flow is getting triggered.
81 - Modify the bandwidth of a connection link of an existing CLL instance, and monitor if the closed-loop call flow is getting triggered.
82 - Modify an existing CLL instance by add a new connection link, and monitor if the closed-loop call flow is getting triggered.
85 Update for Istanbul Release
86 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
88 The Istanbul release introduces a new functionality for the CCVPN use-case:
89 Cloud Lease Line (CLL) service support. The following three main operations were
90 added in Istanbul release (REQ-719):
92 1. The support for creating an E-Tree service, which has one ROOT (Cloud POP) and may have
93 one or more LEAFs (i.e. ONUs) as its branches.
94 2. The support for modifying the maximum bandwidth supported by a given E-Tree.
95 3. The support for deleting an E-Tree service.
97 Istanbul Scope and Impacted modules
98 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
100 For operation #1 mentioned above, the user should be able to "create" an E-Tree service.
101 The modification operation is able to support the following scenarios:
103 a. An E-Tree can have one or more branches (LEAFs) located in one or multiple (different)
105 b. When multiple LEAFs are physically located in a single OLT node, those LEAFs
106 should re-use or share the same OTN tunnels, therefore the path computation
107 mechanism should only be called once.
109 By operation #2 mentioned above, a user can change/modify the maximum bandwidth supported
112 And by operation #3 mentioned above, a user can delete a given E-Tree.
114 The impacted ONAP modules are: SO, SDN-C, and A&AI.
116 For A&AI, additional edge-rules were introduced between two connectivity nodes as well as
117 between a connectivity and a uni node.
119 In SDN-C, additional Directed Graphs (DGs) were implemented to support the above-mentioned
120 features. These new DGs are placed under the generic-resource-api folder in SDNC.
122 Installation Procedure
123 ~~~~~~~~~~~~~~~~~~~~~~
125 For Istanbul new features, the integration test environment is similar to that of
126 the Honolulu release: an ONAP instance with Istanbul release interfacing with 3rd party
127 transport domain controllers should be established.
129 For E-Tree support, the installation procedure is similar to that of the E2E
130 Network Slicing use case. In other words, we need to bring up the required modules
131 including SO, ADNS, A&AI, and UUI. We also need to configure these modules along
132 with the mandatory common modules such as DMaaP.
134 Functional/Integration Test Cases
135 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
137 The testing procedure is described in the following few test cases:
139 - create an E-Tree with one ROOT and one or multiple LEAF(s) in a multi-domain topology
140 - modify the maximum bw of a given E-Tree or add a new connection link to a given E-Tree
141 - delete a given E-Tree
143 To run such test cases, the user must first add (register) the domain controllers as the ESR
144 3rd party controllers. As a result of this registration, a round of topology discovery gets
145 triggered. After that, network-routes or UNI Endpoints have to be created in A&AI. This step
146 is similar to that of Guilin release, and is described in the following link:
147 https://wiki.onap.org/display/DW/Transport+Slicing+Configuration+and+Operation+Guidance
149 Then an E-Tree creation, modification and deletion can be triggered from SO APIs.
153 Update for Honolulu Release
154 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
156 The Honolulu release continued to support and extend the Transport Slicing functionality
157 developed in Guilin release. Two main features were aded in Honolulu release (REQ-456):
159 1. The support for reuse and modification of an existing TN NSSI has been developed.
160 2. In addition, the Honolulu release also continuted to support and extend the CCVPN
161 use-case and in particular, the support for inter-domain connections of three or
162 more network domains has been introduced in Honolulu release. (CCVPN in previous
163 releases were only be able to connect two domains).
165 Honolulu Scope and Impacted modules
166 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
168 For feature #1 mentioned above, the user should be able to "modify" a Transport Slice.
169 The modification operation is able to support the following three scenarios:
171 a. A user may "Add" one or more new service(s)/connections link(s) to a given slice
172 (TN NSSI) that is already created.
173 b. A user may need to change or modify the maximum bandwidth attribute (i.e. the SLA
174 agreement) using which a given slice is created.
175 c. Both of the above operations.
177 For feature #2 mentioned above, now in H release, we can have and support an artibrary
178 number of domains inter-connected to each other and we can support a cross-layer
179 cross-domain VPN connectivity and transport slicing for these kinds of scenarios as well.
181 Impacted ONAP modules include: SO, SDN-C, CCSDK, A&AI.
183 In CCSDk, a path computation engine (PCE) mechanism is introduced to support a
184 graph-based path computation in a multi-domain network topologies. This PCE system is
185 implemented as a SLI plugin to be called and used by Directed Graphs (DGs).
187 For A&AI, additional attributes were introduced to the connectivity node and vpn-binding node.
189 In SDN-C, additional Directed Graphs (DGs) were implemented to support the above-mentioned
192 Installation Procedure
193 ~~~~~~~~~~~~~~~~~~~~~~
195 For Honolulu new features, the integration test environment is similar to that of the Guilin
196 release: an ONAP instance with Honolulu release interfacing to 3rd party transport domain
197 controllers should be established.
199 For Transport Slicing, the installation procedure is similar to that of the E2E
200 Network Slicing use case. In other words, we need to bring up the required modules
201 including SDC, SO, A&AI, UUI and OOF. We also need to configure these modules along
202 with the mandatory common modules such as DMaaP.
204 Functional/Integration Test Cases
205 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
207 The testing procedure is described in the following few test cases:
209 - service/template design: Successful design of TN NSST and Slice Profile
210 - modify max-bandwidth of existing TN NSSI: Modify the maximum bandwidth of an existing TN NSSI
211 - modify connection links existing TN NSSI: Add new connection links to existing TN NSSI
212 - modify both max-bandwidth and connection links of TN NSSI: Modify both the maximum bandwidth and add new connection links to an existing TN NSSI
213 - three-domain network: Test create TN NSSI (or other NSI life cycle operations) on a three-domain network (i.e., need 3 ACTN PNC simulators)
217 Update for Guilin Release
218 ~~~~~~~~~~~~~~~~~~~~~~~~~
220 In Guilin Release, **MDONS** Extension feature is introduced.
222 In addition to the MDONS extension, CCVPN has also developed an
223 IETF/ACTN-based Transport Slicing solution (REQ-347). This development
224 enabled ONAP to offer the TN NSSMF functionality, which was used by
225 the E2E Network Slicing use case (REQ-342). The solution was built
226 upon the existing IETF/ACTN E-LINE over OTN NNI feature developed in Frankfurt release.
228 Guilin Scope and Impacted modules
229 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
230 MDONS Extension implementation for the Frankfurt release will incorporate the following:
232 - Support Asynchronous OpenRoadM OTN service activation notification handling
233 - Add OOF support for inter domain link/path selection
234 - Support Closed Loop sub-use case
236 Impacted ONAP modules include: OOF, SDN-C, SO and Holmes.
238 `Wiki link reference <https://wiki.onap.org/display/DW/MDONS+Extension+in+R7>`_
240 Transport Slicing in Guilin release has implemented the following TN NSSMF functionality:
247 The Tranport Slicing implementation has made code changes in the following modules:
249 - AAI (Schema changes only)
257 Functional/Integration Test Cases
258 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
260 For integration test case and description of MDONS extension, refer to this
261 `following wiki-page <https://wiki.onap.org/display/DW/Integration+Test+Cases+-+MDONS+Extension>`_.
263 For integration test case and description of Transport Slicing:
265 - `Guilin Test plan <https://wiki.onap.org/display/DW/CCVPN+-+Transport+Slicing+integration+test+plan+for+Guilin+release>`_
266 - `Guilin E2E Network Slicing <https://wiki.onap.org/display/DW/E2E+Network+Slicing+Use+Case+in+R7+Guilin>`_
268 Installation Procedure
269 ~~~~~~~~~~~~~~~~~~~~~~
271 For MDONS extension, the integration test environment is established to have ONAP instance with Guilin
272 release interfacing to 3rd party transport domain controllers. One controller
273 instance manages OpenROADM OTN topology and the other 2 instances manage TAPI
274 OTN topology. L0 infrastructure and WDM services are pre-provisioned to support
275 L1 topology discovery and OTN service orchestration from ONAP.
277 For Transport Slicing, the installation procedure is similar to that of the E2E
278 Network Slicing use case. In other words, we need to bring up the required modules
279 including SDC, SO, A&AI, UUI and OOF. We also need to configure these modules along
280 with the mandatory common modules such as DMaaP.
285 The testing procedure is described in:
287 - `Testing procedure for MDONS extension <https://wiki.onap.org/display/DW/Integration+Test+Cases+-+MDONS+Extension>`_
288 - `Testing procedure for Transport Slicing <https://wiki.onap.org/display/DW/CCVPN+-+Transport+Slicing+integration+test+plan+for+Guilin+release>`_
290 Update for Frankfurt release
291 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
293 In Frankfurt, we introduced two extensions in CCVPN use case. One is E-LINE service over OTN NNI handover, another is the
294 multi domain optical service which aims to provide end to end layer 1 service.
302 It is considered a typical scenario for operators to use OTN to interconnect its multiple transport network domains. Hence
303 the capabilities of orchestrating end-to-end E-LINE services across the domains over OTN is important for ONAP. When operating
304 with multiple domains with multi vendor solutions, it is also important to define and use standard and open
305 interfaces, such as the IETF ACTN-based transport `YANG models <https://tools.ietf.org/html/rfc8345>`_, as the southbound interface
306 of ONAP, in order to ensure interoperability. The SOTN NNI use-case aims to automate the design, service provision by independent
307 operational entities within a service provider network by delivering E-Line over OTN orchestration capabilities into ONAP. SOTN NNI
308 extends upon the CCVPN use-case by incorporating support for L1/L2 network management capabilities leveraging open standards & common
311 Frankfurt Scope and Impacted modules
312 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
314 The Frankfurt demonstration includes L1(OTN) and L2(ETH) Topology discovery from multiple domains controllers with in an operator
315 and provide VPN service provision in OTN and ETH network.
317 The ONAP components involved in this use case are: SDC, A&AI, UUI, SO, SDNC, OOF, MSB.
319 Functional Test Cases
320 ~~~~~~~~~~~~~~~~~~~~~
322 Usecase specific developments have been realized in SO, OOF, AAI, SDNC and UUI ONAP components..
327 SOTNVPNInfraService service design in SDC and distribute to AAI and SO.
330 All operation will be triggered by UUI, including service creation and termination,
331 link management and topology network display:
333 - `E-LINE over OTN Inter Domain Test Cases <https://wiki.onap.org/display/DW/E-LINE+over+OTN+Inter+Domain+Test+Cases>`_
334 - `Testing status <https://wiki.onap.org/display/DW/2%3A+Frankfurt+Release+Integration+Testing+Status>`_
336 MDONS (Multi-Domain Optical Network Services)
337 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
342 The MDONS use-case aims to automate the design, activation & operations resulting
343 from an optical transport (L0/L1) service request exchange between service providers and/or independent operational entities within a service provider network by delivering E2E optical orchestration capabilities into ONAP. MDONS extends upon the CCVPN use-case by incorporating support for L0/L1 network management capabilities leveraging open standards & common data models defined by OpenROADM, Transport-API & MEF.
345 Frankfurt Scope and Impacted modules
346 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
348 MDONS implementation for the Frankfurt release will incorporate the following:
349 - Design & modelling of optical services based on MEF L1 subscriber & operator properties
350 - E2E optical service workflow definitions for service instantiation & deletion
351 - UI portal with L1 service instantiation templates
352 - Optical Transport domain management (topology, resource onboarding) through standard models / APIs - OpenROADM, T-API
353 Impacted ONAP modules include: A&AI, SDC, SDN-C, SO, UUI
357 - `OpenROADM reference <https://github.com/OpenROADM/OpenROADM_MSA_Public>`_
358 - `ONF Transport-API (TAPI) <https://github.com/OpenNetworkingFoundation/TAPI>`_
359 - `MEF <https://wiki.mef.net/display/CESG/MEF+63+-+Subscriber+Layer+1+Service+Attributes>`_
361 Functional/Integration Test Cases
362 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
364 For integration test case and description, refer to this following
365 `wiki-page <https://wiki.onap.org/display/DW/MDONS+Integration+Test+Case>`_.
367 Installation Procedure
368 ~~~~~~~~~~~~~~~~~~~~~~
370 The integration test environment is established to have ONAP instance with
371 Frankfurt release interfacing to 3rd party transport domain controllers.
372 One controller instance manages OpenROADM OTN topology and the other 2 instances
373 manage TAPI OTN topology. L0 infrastructure and WDM services are pre-provisioned
374 to support L1 topology discovery and OTN service orchestration from ONAP.
379 Test environment is described in
380 `Installation and Test Procedure <https://wiki.onap.org/display/DW/MDONS+Integration+Test+Case>`_.
382 Update for Dublin release
383 ~~~~~~~~~~~~~~~~~~~~~~~~~
385 1. Service model optimization
387 In Dublin release,the design of CCVPN was optimized by having support of List type of Input in SDC.
388 During onboarding and design phase, one end to end service is created using SDC.
389 This service is composed of these two kinds of resources:
394 See the `Details of Targeted Service Template wiki page <https://wiki.onap.org/display/DW/Details+of+Targeted+Service+Template>`_
397 2. Closed Loop in bandwidth adjustment
398 Simulate alarm at the edge site branch and ONAP will execute close-loop automatically and trigger bandwidth to change higher.
401 Site can be add or delete according to the requirements
403 More information about:
405 - `CCVPN in Dublin release <https://wiki.onap.org/pages/viewpage.action?pageId=45296665>`_
406 - `Dublin test cases <https://wiki.onap.org/display/DW/CCVPN+Test+Cases+for+Dublin+Release>`_
407 - `CCVPN Test Status wiki page <https://wiki.onap.org/display/DW/CCVPN+Test+Status>`_
410 CCVPN integration testing coversed service design, service creation and
411 closed-loop bandwidth adjustments in Dublin release.
413 The service termination and service change will continue to be tested in E release.
414 During the integration testing, SDC, SO, SDC master branch are used which
415 includes the enhanced features for CCVPN use case.
417 Service used for CCVPN
418 ~~~~~~~~~~~~~~~~~~~~~~
420 - `SOTNVPNInfraService, SDWANVPNInfraService and SIteService <https://wiki.onap.org/display/DW/CCVPN+Service+Design>`_
421 - `WanConnectionService (Another way to describe CCVPN in a single service form which based on ONF CIM <https://wiki.onap.org/display/DW/CCVPN+Wan+Connection+Service+Design>`_
426 Cross-domain, cross-layer VPN (CCVPN) is one of the use cases of the ONAP
427 Casablanca release. This release demonstrates cross-operator ONAP orchestration
428 and interoperability with third party SDN controllers and enables cross-domain,
429 cross-layer and cross-operator service creation and assurance.
431 The demonstration includes two ONAP instances, one deployed by Vodafone and one
432 by China Mobile, both of which orchestrate the respective operator underlay OTN
433 networks and overlay SD-WAN networks and peer to each other for cross-operator
434 VPN service delivery.
436 `CCVPN Use Case Wiki Page <https://wiki.onap.org/display/DW/CCVPN%28Cross+Domain+and+Cross+Layer+VPN%29+USE+CASE>`_
438 The projects covered by this use case include: SDC, A&AI, UUI, SO, SDNC, OOF, Policy, DCAE(Holmes), External API, MSB
445 - `SOTNVPNInfraService, SDWANVPNInfraService and SIteService service Design steps <https://wiki.onap.org/display/DW/CCVPN+Service+Design>`_
446 - `WanConnectionService ( Another way to describe CCVPN in a single service form which based on ONF CIM ) <https://wiki.onap.org/display/DW/CCVPN+Wan+Connection+Service+Design>`_
450 - All operations will be triggered by UUI, including service creation and termination,
451 link management and topology network display.
454 See the `CCVPN Test Guide wiki page <https://wiki.onap.org/display/DW/CCVPN+Test+Guide>`_
457 Test Status and Plans
458 ~~~~~~~~~~~~~~~~~~~~~
460 - `All test case covered by this use case <https://wiki.onap.org/display/DW/CCVPN+Integration+Test+Case>`_
461 - `Test status <https://wiki.onap.org/display/DW/CCVPN++-Test+Status>`_
463 Known Issues and Resolutions
464 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
466 1) AAI-1923. Link Management, UUI can't delete the link to external onap otn domain.
468 For the manual steps provided by A&AI team, we should follow the steps as follow
469 the only way to delete is using the forceDeleteTool shell script in the graphadmin container.
470 First we will need to find the vertex id, you should be able to get the id by making the following GET request.
472 GET /aai/v14/network/ext-aai-networks/ext-aai-network/createAndDelete/esr-system-info/test-esr-system-info-id-val-0?format=raw
481 "node-type": "pserver",
482 "url": "/aai/v13/cloud-infrastructure/pservers/pserver/pserverid14503-as988q",
489 Same goes for the ext-aai-network:
491 GET /aai/v14/network/ext-aai-networks/ext-aai-network/createAndDelete?format=raw
493 Retrieve the id from the above output as that will be the vertex id that you want to remove.
495 Run the following command multiple times for both the esr-system-info and ext-aai-network:
499 kubectl exec -it $(kubectl get pods -lapp=aai-graphadmin -n onap --template 'range .items.metadata.name"\n"end' | head -1) -n onap gosu aaiadmin /opt/app/aai-graphadmin/scripts/forceDeleteTool.sh -action DELETE_NODE -userId YOUR_ID_ANY_VALUE -vertexId VERTEX_ID
501 From the above, remove the YOUR_ID_ANY_VALUE and VERTEX_ID with your info.
503 2) SDC-1955. Site service Distribution
505 To overcome the Service distribution, the SO catalog has to be populated with the model information of the services and resources.
506 a) Refering to the Csar that is generated in the SDC designed as per the details mentioned in the below link: https://wiki.onap.org/display/DW/CCVPN+Service+Design
507 b) Download the Csar from SDC thus generated.
508 c) copy the csar to SO sdc controller pod and bpmn pod
512 kubectl -n onap get pod|grep so
513 kubectl -n onap exec -it dev-so-so-sdc-controller-c949f5fbd-qhfbl /bin/sh
516 kubectl -n onap cp service-Sdwanvpninfraservice-csar.csar dev-so-so-bpmn-infra-58796498cf-6pzmz:null/ASDC/1/service-Sdwanvpninfraservice-csar.csar
517 kubectl -n onap cp service-Sdwanvpninfraservice-csar.csar dev-so-so-bpmn-infra-58796498cf-6pzmz:ASDC/1/service-Sdwanvpninfraservice-csar.csar
519 d) populate model information to SO db: the db script example can be seen in
520 https://wiki.onap.org/display/DW/Manual+steps+for+CCVPN+Integration+Testing
522 The same would also be applicable for the integration of the client to create the service and get the details.
523 Currently the testing has been performed using the postman calls to the corresponding APIs.
525 3) SDC-1955 & SDC-1958. Site service parsing Error
527 UUI: stored the csar which created based on beijing release under a fixed directory, If site servive can't parsed by SDC tosca parser, UUI will parse this default csar and get the input parameter
528 a) Make an available csar file for CCVPN use case.
529 b) Replace uuid of available files with what existing in SDC.
530 c) Put available csar files in UUI local path (/home/uui).
532 4) SO docker branch 1.3.5 has fixes for the issues 1SO-1248
534 After SDC distribution success, copy all csar files from so-sdc-controller:
536 - connect to so-sdc-controller ( eg: kubectl.exe exec -it -n onap dev-so-so-sdc-controller-77df99bbc9-stqdz /bin/sh )
537 - find out all csar files ( eg: find / -name "\*.csar" ), the csar files should
538 be in this path: /app/null/ASDC/ ( eg: /app/null/ASDC/1/service-Sotnvpninfraservice-csar.csar )
539 - exit from the so-sdc-controller ( eg: exit )
540 - copy all csar files to local derectory ( eg: kubectl.exe cp onap/dev-so-so-sdc-controller-6dfdbff76c-64nf9:/app/null/ASDC/tmp/service-DemoService-csar.csar service-DemoService-csar.csar -c so-sdc-controller )
542 Copy csar files, which got from so-sdc-controller, to so-bpmn-infra:
544 - connect to so-bpmn-infra ( eg: kubectl.exe -n onap exec -it dev-so-so-bpmn-infra-54db5cd955-h7f5s -c so-bpmn-infra /bin/sh )
545 - check the /app/ASDC directory, if doesn't exist, create it ( eg: mkdir /app/ASDC -p )
546 - exit from the so-bpmn-infra ( eg: exit )
547 - copy all csar files to so-bpmn-infra ( eg: kubectl.exe cp service-Siteservice-csar.csar onap/dev-so-so-bpmn-infra-54db5cd955-h7f5s:/app/ASDC/1/service-Siteservice-csar.csar )