1 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
2 .. http://creativecommons.org/licenses/by/4.0
5 .. _docs_vfw_edgex_multicloud_k8s:
7 vFW/Edgex with Multicloud Kubernetes Plugin: Setting Up and Configuration
8 -------------------------------------------------------------------------
12 This use case covers the deployment of vFW and Edgex HELM Charts in a Kubernetes based cloud region via the multicloud-k8s plugin.
13 <<<<<<< HEAD (2a9a2c Add missing dash in file ref)
14 The multicloud-k8s plugin provides APIs to upload self contained HELM Charts that can be customized via the profile api and later installed in a particular cloud region.
18 `Multicloud K8s API Page <https://wiki.onap.org/display/DW/MultiCloud+K8s-Plugin-service+API>`_
20 `Edgex Integration Test Status <https://wiki.onap.org/display/DW/Deploy+EdgeXFoundry+Helm+Chart+using+Multicloud+K8s+Plugin+in+Kubernetes+Cloud>`_
22 `vFW Integration Test Status <https://wiki.onap.org/pages/viewpage.action?pageId=64000842>`_
24 `Deploying vFW and EdgexFoundry on Kubernetes with ONAP <https://wiki.onap.org/display/DW/Deploying+vFw+and+EdgeXFoundry+Services+on+Kubernets+Cluster+with+ONAP>`_
28 Setting Up and Installation
29 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
30 Instructions for setting up and deploying the vFW and EdgeXFoundry HELM Charts can be found here: https://wiki.onap.org/display/DW/Deploying+vFw+and+EdgeXFoundry+Services+on+Kubernets+Cluster+with+ONAP
32 Install ONAP via OOM using the deploy script in the integration repo. Instructions for this can be found in this link https://onap.readthedocs.io/en/latest/submodules/oom.git/docs/oom_cloud_setup_guide.html.
34 When the installation is complete (all the pods are either in running or completed state)
37 The multicloud-k8s plugin provides APIs to upload self-contained HELM Charts that can be customized via the profile API and later installed in a particular cloud region.
39 When the installation is complete (all the pods are either in running or completed state)
44 https://github.com/onap/multicloud-k8s/tree/master/kud/demo/firewall
46 EdgeXFoundry Helm Chart link:
47 -----------------------------
49 https://github.com/onap/multicloud-k8s/tree/master/kud/tests/vnfs/edgex/helm/edgex
52 **Create CSAR with Helm chart as an artifact**
53 ----------------------------------------------
55 The CSAR is a heat template package with Helm chart in it. The basic package
56 consists of an **environment file**, **base_dummy.yaml file** (example),
57 **MANIFEST.json** and the **tar.gz** file (of Helm chart).
58 We need to zip all of these files before onboarding.
59 One thing to pay much attention to is the naming convention which must
60 be followed while making the tgz.
61 **NOTE: The Naming convention is for the helm chart tgz file.**
63 **Naming convention follows the format:**
65 <free format string>\_\ ***cloudtech***\ \_<technology>\_<subtype>.extension
67 1. *Cloudtech:* is a fixed pattern and should not be changed if not
69 2. *Technology:* k8s, azure, aws
70 3. *Subtype*: charts, day0, config template
71 4. *Extension*: zip, tgz, csar
73 NOTE: The .tgz file must be a tgz created from the top level helm chart
74 folder. I.e. a folder that contains a Chart.yaml file in it.
77 Listed below is an example of the contents inside a heat template
82 MANIFEST.json base_dummy.env base_dummy.yaml
83 vfw_cloudtech_k8s_charts.tgz vfw_k8s_demo.zip
91 Key thing is note the addition of cloud artifact
94 type: "CLOUD_TECHNOLOGY_SPECIFIC_ARTIFACTS"
101 "file": "base_dummy.yaml",
106 "file": "base_dummy.env",
112 "file": "vfw_cloudtech_k8s_charts.tgz",
113 "type": "CLOUD_TECHNOLOGY_SPECIFIC_ARTIFACTS"
119 ~~~~~~~~~~~~~~~~~~~~~
120 Designed to be minimal HEAT template
124 ##==================LICENSE_START========================================
126 ## Copyright (C) 2019 Intel Corporation
127 ## SPDX-License-Identifier: Apache-2.0
129 ##==================LICENSE_END===========================================
131 heat_template_version: 2016-10-14
132 description: Heat template to deploy dummy VNF
138 description: Dummy name
143 description: Provided by ONAP
148 description: Provided by ONAP
153 description: Provided by ONAP
157 label: Image name or ID
158 description: Dummy image name
163 description: Dummy flavor
167 type: OS::Nova::Server
169 name: { get_param: dummy_name_0 }
170 image: { get_param: dummy_image_name }
171 flavor: { get_param: dummy_flavor_name } metadata: { vnf_name: { get_param: vnf_name }, vnf_id: { get_param: vnf_id }, vf_module_id: { get_param: vf_module_id }}
183 vnf_id: PROVIDED_BY_ONAP
184 vnf_name: PROVIDED_BY_ONAP
185 vf_module_id: PROVIDED_BY_ONAP
186 dummy_name_0: dummy_1_0
187 dummy_image_name: dummy
188 dummy_flavor_name: dummy.default
193 For onboarding instructions please refer to steps 4-9 from the document
194 `here <https://wiki.onap.org/display/DW/vFWCL+instantiation%2C+testing%2C+and+debuging>`__.
196 **Steps for installing KUD Cloud**
197 ----------------------------------
199 Follow the link to install KUD Kubernetes Deployment. KUD contains all
200 the packages required for running vfw use case.
202 Kubernetes Baremetal deployment instructions here_
204 .. _here: https://wiki.onap.org/display/DW/Kubernetes+Baremetal+deployment+setup+instructions/
206 **REGISTER KUD CLOUD REGION with K8s-Plugin**
207 ---------------------------------------------
209 API to support Reachability for Kubernetes Cloud
211 **The command to POST connectivity info**
212 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
216 "cloud-region" : "<name>", // Must be unique across
217 "cloud-owner" : "<owner>",
218 "other-connectivity-list" : {
221 This is a multipart upload and here is how you do the POST for this.
223 #Using a json file (eg: post.json) containing content as above
226 curl -i -F "metadata=<post.json;type=application/json" -F file=@
227 /home/ad_kkkamine/.kube/config -X POST http://MSB_NODE_IP:30280/api/multicloud-k8s/v1/v1/connectivity-info
229 **Command to GET Connectivity Info**
230 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
234 curl -i -X GET http://MSB_NODE_IP:30280/api/multicloud-k8s/v1/v1/connectivity-info/{name}
237 **Command to DELETE Connectivity Info**
238 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
242 curl -i -X GET http://MSB_NODE_IP:30280/api/multicloud-k8s/v1/v1/connectivity-info/{name}
245 **Command to UPDATE/PUT Connectivity Info**
246 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
250 curl -i -X GET http://MSB_NODE_IP:30280/api/multicloud-k8s/v1/v1/connectivity-info/{name}
252 **Register KUD Cloud region with AAI**
253 --------------------------------------
255 With k8s cloud region, we need to add a tenant to the k8s cloud region.
256 The 'easy' way is to have the ESR information (in step 1 of cloud
257 registration) point to a real OpenStack tenant (e.g. the OOF tenant in
258 the lab where we tested).
260 This will cause multicloud to add the tenant to the k8s cloud region and
261 then, similar to #10 in the documentation
262 `here <https://onap.readthedocs.io/en/casablanca/submodules/integration.git/docs/docs_vfwHPA.html#docs-vfw-hpa>`__,
263 the service-subscription can be added to that object.
265 NOTE: use same name cloud-region and cloud-owner name
267 An example is shown below for K8s cloud but following the steps 1,2,3
269 `here <https://onap.readthedocs.io/en/latest/submodules/multicloud/framework.git/docs/multicloud-plugin-windriver/UserGuide-MultiCloud-WindRiver-TitaniumCloud.html#tutorial-onboard-instance-of-wind-river-titanium-cloud>`__.
270 The sample input below is for k8s cloud type.
272 **Step 1 - Cloud Registration/ Create a cloud region to represent the instance.**
275 Note: highlighted part of the body refers to an existing OpenStack
276 tenant (OOF in this case). Has nothing to do with the K8s cloud region
281 PUT https://{{AAI1_PUB_IP}}:{{AAI1_PUB_PORT}}/aai/v13/cloud-infrastructure/cloud-regions/cloud-region/k8scloudowner4/k8sregionfour
283 "cloud-owner": "k8scloudowner4",
284 "cloud-region-id": "k8sregionfour",
286 "owner-defined-type": "t1",
287 "cloud-region-version": "1.0",
288 "complex-name": "clli1",
289 "cloud-zone": "CloudZone",
290 "sriov-automation": false,
291 "cloud-extra-info":"{\"openstack-region-id\":\"k8sregionthree\"}",
292 "esr-system-info-list": {
295 "esr-system-info-id": "55f97d59-6cc3-49df-8e69-926565f00066",
296 "service-url": "http://10.12.25.2:5000/v3",
298 "password": "onapdemo",
299 "system-type": "VIM",
300 "ssl-insecure": true,
301 "cloud-domain": "Default",
302 "default-tenant": "OOF",
303 "tenant-id": "6bbd2981b210461dbc8fe846df1a7808",
304 "system-status": "active"
310 **Step 2 add a complex to the cloud**
312 Note: just adding one that exists already
316 PUT https://{{AAI1_PUB_IP}}:{{AAI1_PUB_PORT}}/aai/v13/cloud-infrastructure/cloud-regions/cloud-region/k8scloudowner4/k8sregionfour/relationship-list/relationship
318 "related-to": "complex",
319 "related-link": "/aai/v13/cloud-infrastructure/complexes/complex/clli1",
320 "relationship-data": [
322 "relationship-key": "complex.physical-location-id",
323 "relationship-value": "clli1"
328 **Step 3 - Trigger the Multicloud plugin registration process**
333 POST http://{{MSB_IP}}:{{MSB_PORT}}/api/multicloud-titaniumcloud/v1/k8scloudowner4/k8sregionfour/registry
336 This registers the K8S cloud with Multicloud it also reaches out and
337 adds tenant information to the cloud (see example below you'll see all
338 kinds of flavor, image information that is associated with the OOF
341 If we had not done it this way, then wed have to go in to AAI at this
342 point and manually add a tenant to the cloud region. The first time I
343 tried this (k8s region one), I just made up some random tenant id and
346 The tenant is there so you can add the service-subscription to it:
348 **Making a Service Type:**
352 PUT https://{{AAI1_PUB_IP}}:{{AAI1_PUB_PORT}}/aai/v13/service-design-and-creation/services/service/vfw-k8s
354 "service-description": "vfw-k8s",
355 "service-id": "vfw-k8s"
358 Add subscription to service type to the customer (Demonstration in this
359 case which was already created by running the robot demo scripts)
363 PUT https://{{AAI1_PUB_IP}}:{{AAI1_PUB_PORT}}/aai/v16/business/customers/customer/Demonstration/service-subscriptions/service-subscription/vfw-k8s
365 "service-type": "vfw-k8s"
368 Add Service-Subscription to the tenant (resource-version changes based
369 on actual value at the time):
373 PUT https://{{AAI1_PUB_IP}}:{{AAI1_PUB_PORT}}/aai/v16/cloud-infrastructure/cloud-regions/cloud-region/k8scloudowner4/k8sregionfour/tenants/tenant/6bbd2981b210461dbc8fe846df1a7808?resource-version=1559345527327
375 "tenant-id": "6bbd2981b210461dbc8fe846df1a7808",
376 "tenant-name": "OOF",
377 "resource-version": "1559345527327",
378 "relationship-list": {
381 "related-to": "service-subscription",
382 "relationship-label": "org.onap.relationships.inventory.Uses",
383 "related-link": "/aai/v13/business/customers/customer/Demonstration/service-subscriptions/service-subscription/vfw-k8s",
384 "relationship-data": [
386 "relationship-key": "customer.global-customer-id",
387 "relationship-value": "Demonstration"
390 "relationship-key": "service-subscription.service-type",
391 "relationship-value": "vfw-k8s"
399 **Distribute the CSAR**
400 -----------------------
401 Onboard a service it gets stored in SDC final action is distributed. SO
402 and other services are notified sdc listener in the multicloud sidecar.
403 When distribution happens it takes tar.gz file and uploads to k8s
406 **Create Profile Manually**
407 ---------------------------
409 K8s-plugin artifacts start in the form of Definitions. These are nothing
410 but Helm Charts wrapped with some metadata about the chart itself. Once
411 the Definitions are created, we are ready to create some profiles so
412 that we can customize that definition and instantiate it in Kubernetes.
414 NOTE: Refer this link_ for complete API lists and
417 .. _link : https://wiki.onap.org/display/DW/MultiCloud+K8s-Plugin-service+API
419 A profile consists of the following:
423 - Contains the details for the profile and everything contained within
425 A **HELM** values override yaml file.
427 - It can have any name as long as it matches the corresponding entry in the **manifest.yaml**
429 Any number of files organized in a folder structure
431 - All these files should have a corresponding entry in **manifest.yaml** file
433 **Creating a Profile Artifact**
434 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
438 > cd multicloud-k8s/kud/tests/vnfs/testrb/helm/profile
444 testfol/subdir/deployment.yaml
446 #Create profile tar.gz
448 > tar -cf profile.tar *
450 > mv profile.tar.gz ../
452 The manifest file contains the following
459 values: "values_override.yaml"
461 - filepath: testfol/subdir/deployment.yaml
462 chartpath: vault-consul-dev/templates/deployment.yaml
464 Note: values: "values\_override.yaml" can **be** empty **file** **if**
465 you are creating **a** dummy **profile**
467 Note: A dummy profile does not need any customization. The following is
468 optional in the manifest file.
473 - filepath: testfol/subdir/deployment.yaml
474 chartpath: vault-consul-dev/templates/deployment.yaml
477 With this information, we are ready to upload the profile with the
483 "rb-name": "test-rbdef",
485 "profile-name": "p1",
486 "release-name": "r1", //If release-name is not provided, profile-name will be used
487 "namespace": "testnamespace1",
488 "kubernetes-version": "1.12.3"
492 **Command to create (POST) Profile**
493 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
497 curl -i -d @create_rbprofile.json -X POST http://MSB_NODE_IP:30280/api/multicloud-k8s/v1/v1/rb/definition/test-rbdef/v1/profile
501 **Command to UPLOAD artifact for Profile**
505 curl -i --data-binary @profile.tar.gz -X POST http://MSB_NODE_IP:30280/api/multicloud-k8s/v1/v1/rb/definition/test-rbdef/v1/profile/p1/content
509 **Command to GET Profiles**
513 curl -i http://MSB_NODE_IP:30280/api/multicloud-k8s/v1/v1/rb/definition/test-rbdef/v1/profile
515 curl -i http://MSB_NODE_IP:30280/api/multicloud-k8s/v1/v1/rb/definition/test-rbdef/v1/profile/p1
519 **Command to DELETE Profile**
522 curl -i -X DELETE http://MSB_NODE_IP:30280/api/multicloud-k8s/v1/v1/rb/definition/test-rbdef/v1/profile/p1
528 Instantiation is done by SO. SO then talks to Multi Cloud-broker via MSB
529 and that in turn looks up the cloud region in AAI to find the endpoint.
530 If k8sregion one is registered with AAI and SO makes a call with that,
531 then the broker will know that it needs to talk to k8s-plugin based on
532 the type of the registration.
534 **Instantiate the created Profile via the following REST API**
538 Using the following JSON:
540 "cloud-region": "kud",
541 "profile-name": "p1",
542 "rb-name":"test-rbdef",
548 **NOTE**: Make sure that the namespace is already created before
551 Instantiate the profile with the ID provided above
553 **Command to Instantiate a Profile**
557 curl -d @create_rbinstance.json http://MSB_NODE_IP:30280/api/multicloud-k8s/v1/v1/instance
560 The command returns the following JSON
568 "profile-name": "profile1",
569 "cloud-region": "kud",
570 "namespace": "testns",
591 "Version": "v1beta1",
592 "Kind": "StatefulSet"
594 "Name": "profile1-mongo"
599 **Delete Instantiated Kubernetes resources**
601 The **id** field from the returned JSON can be used to **DELETE** the
602 resources created in the previous step. This executes a Delete operation
603 using the Kubernetes API.
607 curl -X DELETE http://MSB_NODE_IP:30280/api/multicloud-k8s/v1/v1/instance/ZKMTSaxv
610 **GET Instantiated Kubernetes resources**
613 The **id field** from the returned JSON can be used to **GET** the
614 resources created in the previous step. This executes a get operation
615 using the Kubernetes API.
619 curl -X GET http://MSB_NODE_IP:30280/api/multicloud-k8s/v1/v1/instance/ZKMTSaxv
622 `*\ https://github.com/onap/oom/blob/master/kubernetes/multicloud/resources/config/provider-plugin.json <https://github.com/onap/oom/blob/master/kubernetes/multicloud/resources/config/provider-plugin.json>`__
624 **Create User parameters**
626 We need to create parameters that ultimately get translated as:
633 "attribute_name": "definition-name",
634 "attribute_value": "edgex"
637 "attribute_name": "definition-version",
638 "attribute_value": "v1"
641 "attribute_name": "profile-name",
642 "attribute_value": "profile1"
646 >>>>>>> CHANGE (24cda1 Adding document for vfw/edgex use case)