1 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
2 .. http://creativecommons.org/licenses/by/4.0
4 Helm Chart Based DCAE Deployment
5 ================================
7 This document describes the details of the Helm chart based deployment process for R4 ONAP and how DCAE is deployed through this process.
10 ONAP Deployment Overview
11 ------------------------
13 ONAP R4 extends the Kubernetes deployment method introduced in R2 and continued in R3.
14 Kubernetes is a container orchestration technology that organizes containers into composites of various patterns for easy deployment, management, and scaling.
15 ONAP uses Kubernetes as the foundation for fulfilling its platform maturity promises.
17 ONAP manages Kubernetes specifications using Helm charts, under which all Kubernetes yaml-formatted resource specifications and additional files
18 are organized into a hierarchy of charts, sub-charts, and resources. These yaml files are further augmented with Helm's templating, which makes dependencies
19 and cross-references of parameters and parameter derivatives among resources manageable for a large and complex Kubernetes system such as ONAP.
21 At deployment time, with a single **helm deploy** command, Helm resolves all the templates and compiles the chart hierarchy into Kubernetes resource definitions,
22 and invokes Kubernetes deployment operations for all the resources.
24 All ONAP Helm charts are organized under the **kubernetes** directory of the **OOM** project, where roughly each ONAP component occupies a subdirectory.
25 DCAE charts are placed under the **dcaegen2** directory.
27 The PNDA data platform is an optional DCAE component that is placed under the **pnda**
28 directory. Details for how to configure values to enable PNDA installation during Helm install
29 are described in `Installing PNDA During Helm Chart Based DCAE Deployment
32 DCAE Chart Organization
33 -----------------------
35 Following Helm conventions, the DCAE Helm chart directory (``oom/kubernetes/dcaegen2``) consists of the following files and subdirectories:
37 * ``Chart.yaml``: metadata.
38 * ``requirements.yaml``: dependency charts.
39 * ``values.yaml``: values for Helm templating engine to expand templates.
40 * ``resources``: subdirectory for additional resource definitions such as configuration, scripts, etc.
41 * ``templates``: subdirectory for Kubernetes resource definition templates.
42 * ``charts``: subdirectory for sub-charts.
44 The dcaegen2 chart has the following sub-charts:
46 * ``dcae-bootstrap``: deploys the DCAE bootstrap service that performs some DCAE initialization and deploys additional DCAE components.
47 * ``dcae-cloudify-manager``: deploys the DCAE Cloudify Manager instance.
48 * ``dcae-config-binding-service``: deploys the DCAE config binding service.
49 * ``dcae-deployment-handler``: deploys the DCAE deployment handler service.
50 * ``dcae-healthcheck``: deploys the DCAE healthcheck service that provides an API to check the health of all DCAE components.
51 * ``dcae-policy-handler``: deploys the DCAE policy handler service.
52 * ``dcae-redis``: deploys the DCAE Redis cluster.
53 * ``dcae-servicechange-handler``: deploys the DCAE service change handler service. A subchart (``dcae-inventory-api``) deploys the DCAE inventory API service.
58 At deployment time, when the **helm deploy** command is executed,
59 all DCAE resources defined within the subcharts above are deployed.
62 * the DCAE bootstrap service
63 * the DCAE healthcheck service
64 * the DCAE platform components:
67 * Config binding service
70 * Service change handler
71 * Inventory API service (launched as a subchart of service change handler)
72 * Inventory postgres database service (launched as a dependency of the inventory API service)
73 * DCAE postgres database service (launched as a dependency of the bootstrap service)
76 Some of the DCAE subcharts include an initContainer that checks to see if
77 other services that they need in order to run have become ready. The installation
78 of these subcharts will pause until the needed services are available.
80 In addition, DCAE operations depends on a Consul server cluster.
81 For ONAP OOM deployment, the Consul cluster is provided as a shared
82 resource. Its charts are defined under the ``oom/kubernetes/consul``
83 directory, not as part of the DCAE chart hierarchy.
85 The dcae-bootstrap service has a number of prerequisites because the subsequently deployed DCAE components depends on a number of resources having entered their normal operation state. DCAE bootstrap job will not start before these resources are ready. They are:
87 * dcae-cloudify-manager
91 * dcae-config-binding-service
94 Once started, the DCAE bootstrap service will call Cloudify Manager to deploy
95 a series of blueprints which specify the additional DCAE R4 components.
96 These blueprints use the DCAE Kubernetes plugin (``k8splugin``) to deploy
97 Docker images into the ONAP Kubernetes cluster. For each component, the plugin
98 creates a Kubernetes deployment and other Kubernetes resources (services, volumes, logging sidecar, etc.)
101 The DCAE bootstrap service creates the following Kubernetes deployments:
103 * deploy/dep-dcae-dashboard
104 * deploy/dep-dcae-hv-ves-collector
105 * deploy/dep-dcae-prh
106 * deploy/dep-dcae-snmptrap-collector
107 * deploy/dep-dcae-tca-analytics
108 * deploy/dep-dcae-ves-collector
109 * deploy/dep-holmes-engine-mgmt
110 * deploy/dep-holmes-rule-mgmt
112 After deploying all of the blueprints, the DCAE bootstrap service
113 continues to run. The bootstrap container can be useful for
114 troubleshooting or for launching additional components. The bootstrap
115 container logs (accessed using the ``kubectl logs`` command) show the
116 details of all of the component deployments.
121 Deployment time configuration of DCAE components are defined in several places.
123 * Helm Chart templates:
124 * Helm/Kubernetes template files can contain static values for configuration parameters;
125 * Helm Chart resources:
126 * Helm/Kubernetes resources files can contain static values for configuration parameters;
127 * Helm values.yaml files:
128 * The values.yaml files supply the values that Helm templating engine uses to expand any templates defined in Helm templates;
129 * In a Helm chart hierarchy, values defined in values.yaml files in higher level supersedes values defined in values.yaml files in lower level;
130 * Helm command line supplied values supersedes values defined in any values.yaml files.
132 In addition, for DCAE components deployed through Cloudify Manager blueprints, their configuration parameters are defined in the following places:
134 * The blueprint files can contain static values for configuration parameters;
135 * The blueprint files are defined under the ``blueprints`` directory of the ``dcaegen2/platform/blueprints`` repo, named with "k8s" prefix.
136 * The blueprint files can specify input parameters and the values of these parameters will be used for configuring parameters in Blueprints. The values for these input parameters can be supplied in several ways as listed below in the order of precedence (low to high):
137 * The blueprint files can define default values for the input parameters;
138 * The blueprint input files can contain static values for input parameters of blueprints. These input files are provided as config resources under the dcae-bootstrap chart;
139 * The blueprint input files may contain Helm templates, which are resolved into actual deployment time values following the rules for Helm values.
142 Now we walk through an example, how to configure the Docker image for the DCAE dashboard, which is deployed by Cloudify Manager.
144 In the ``k8s-dashboard.yaml-template`` blueprint template, the Docker image to use is defined as an input parameter with a default value:
148 dashboard_docker_image:
149 description: 'Docker image for dashboard'
150 default: 'nexus3.onap.org:10001/onap/org.onap.ccsdk.dashboard.ccsdk-app-os:1.1.0-SNAPSHOT-latest'
152 Then in the input file, ``oom/kubernetes/dcaegen2/charts/dcae-bootstrap/resources/inputs/k8s-dashboard-inputs.yaml``,
153 it is defined again as:
157 dashboard_docker_image: {{ include "common.repository" . }}/{{ .Values.componentImages.dashboard }}
159 Thus, when ``common.repository`` and ``componentImages.policy_handler`` are defined in the ``values.yaml`` files,
160 their values will be plugged in here and the resulting ``policy_handler_image`` value
161 will be passed to the Policy Handler blueprint as the Docker image tag to use instead of the default value in the blueprint.
163 Indeed the ``componentImages.dashboard`` value is provided in the ``oom/kubernetes/dcaegen2/charts/dcae-bootstrap/values.yaml`` file:
168 dashboard: onap/org.onap.ccsdk.dashboard.ccsdk-app-os:1.1.0
170 The final result is that when DCAE bootstrap calls Cloudify Manager to deploy the DCAE dashboard, the 1.1.0 image will be deployed.
172 DCAE Service Endpoints
173 ----------------------
175 Below is a table of default hostnames and ports for DCAE component service endpoints in Kubernetes deployment:
176 ================== ================================= ======================================================
177 Component Cluster Internal (host:port) Cluster external (svc_name:port)
178 ================== ================================= ======================================================
179 VES dcae-ves-collector:8080 xdcae-ves-collector.onap:30235
180 HV-VES dcae-hv-ves-collector:6061 xdcae-hv-ves-collector.onap:30222
181 TCA dcae-tca-analytics:11011 xdcae-tca-analytics.onap:32010
183 SNMPTrap dcae-snmptrap-collector:6162/udp xdcae-snmptrap-collector:30470/UDP
184 Policy Handler policy-handler:25577 NA
185 Deployment Handler deployment-handler:8443 NA
186 Inventory inventory:8080 NA
187 Config binding config-binding-service:10000 config-binding-servicee:30415
188 DCAE Healthcheck dcae-healthcheck:80 NA
189 Cloudify Manager dcae-cloudify-manager:80 NA
190 DCAE Dashboard dcae-dashboard:8080/8443 xdcae-dashboard:30418/30419
191 ================== ================================= ======================================================
193 In addition, a number of ONAP service endpoints that are used by DCAE components are listed as follows
194 for reference by DCAE developers and testers:
196 ==================== ============================ ================================
197 Component Cluster Internal (host:port) Cluster external (svc_name:port)
198 ==================== ============================ ================================
199 Consul Server consul-server:8500 consul-server:30270
200 Robot robot:88 robot:30209 TCP
201 Message router message-router:3904 message-router:30227
202 Message router message-router:3905 message-router:30226
203 Message router Kafka message-router-kafka:9092 NA
204 MSB Discovery msb-discovery:10081 msb-discovery:30281
205 Logging log-kibana:5601 log-kibana:30253
206 AAI aai:8080 aai:30232
207 AAI aai:8443 aai:30233
208 ==================== ============================ ================================
213 All of the DCAE components deployed using the OOM Helm charts will be
214 deleted by the ``helm undeploy`` command. This command can be used to
215 uninstall all of ONAP by undeploying the top-level Helm release that was
216 created by the ``helm deploy`` command. The command can also be used to
217 uninstall just DCAE, by having the command undeploy the `top_level_release_name`-``dcaegen2``
220 Helm will undeploy only the components that were originally deployed using
221 Helm charts. Components deployed by Cloudify Manager are not deleted by
224 When uninstalling all of ONAP, it is sufficient to delete the namespace
225 used for the deployment (typically ``onap``) after running the undeploy
226 operation. Deleting the namespace will get rid of any remaining resources
227 in the namespace, including the components deployed by Cloudify Manager.
229 When uninstalling DCAE alone, deleting the namespace would delete the
230 rest of ONAP as well. To delete DCAE alone, and to make sure all of the
231 DCAE components deployed by Cloudify Manager are uninstalled:
233 * Find the Cloudify Manager pod identifier, using a command like:
235 ``kubectl -n onap get pods | grep dcae-cloudify-manager``
236 * Execute the DCAE cleanup script on the Cloudify Manager pod, using a command like:
238 ``kubectl -n onap exec`` `cloudify-manager-pod-id` ``-- /scripts/dcae-cleanup.sh``
239 * Finally, run ``helm undeploy`` against the DCAE Helm subrelease
241 The DCAE cleanup script uses Cloudify Manager and the DCAE Kubernetes
242 plugin to instruct Kubernetes to delete the components deployed by Cloudify
243 Manager. This includes the components deployed when the DCAE bootstrap
244 service ran and any components deployed after bootstrap.