1 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
2 .. http://creativecommons.org/licenses/by/4.0
4 DCAE Service Template - Overview
5 ================================
10 Prior to the ONAP Honolulu release, DCAE microservices were deployed
11 using the Cloudify orchestration tool. Each microservice had a Cloudify
12 *blueprint* containing the information needed for Cloudify to deploy the
13 microservice. The DCAE team provided a Cloudify plugin that used the
14 Kubernetes API to create the Kubernetes resources (including a
15 Kubernetes Deployment and a Kubernetes Service) that make up a running
16 instance of the microservice.
18 In the Honolulu release, DCAE began migrating to a new approach for
19 deploying DCAE microservices. Instead of using Cloudify with a
20 Cloudify blueprint for each microservice, DCAE began using Helm to
21 deploy microservices. This migration is now complete, and all
22 DCAE microservices have a Helm chart instead of a Cloudify blueprint.
24 DCAE Service Templates - Introduction
25 -------------------------------------
27 It would be possible to write a Helm chart for each microservice, each
28 completely unrelated to the others. We are taking a different approach. We are
29 providing shared Helm templates that (approximately) create the same
30 Kubernetes resources that the Cloudify plugin created when it processed
31 a blueprint. Creating a Helm chart for a microservice involves setting
32 up a Helm chart directory, which can be done by copying the chart
33 directory for an existing microservice and changing the ``Chart.yaml``
34 file (to set the name, description, and version of the chart) and the
35 ``values.yaml`` file (to customize the templates for the target
38 Once a chart for a microservice has been created, the chart can be used
39 to deploy the microservice, on demand, into a running instance of ONAP
40 and DCAE. This is similar to how we deployed microservices on demand
41 using a Cloudify blueprint and the Cloudify Manager (or the DCAE
44 The bulk of this document describes the different parameters that can be
45 set in the ``values.yaml`` file. There are two sets of parameters. One
46 set comes from the ONAP OOM common templates used by all of the ONAP
47 components that deployed via Helm. The other set consists of parameters
48 that are specific to the DCAE shared templates.
50 DCAE Service Templates - Location and content
51 ---------------------------------------------
52 The DCAE shared Helm charts for microservices are maintained in the
53 OOM repository, in the ``oom/kubernetes/dcaegen2-services/common/dcaegen2-services-common``
54 directory. In this directory subtree are:
56 - ``Chart.yaml``: the usual Helm chart definition file.
57 - ``requirements.yaml``: the dependencies for the chart. Only the OOM "common" chart is needed.
58 - ``values.yaml``: the file is present for completion, but there are no locally-defined values.
59 - ``templates/_configmap.tpl``: a template that sets up a configMap containing the microservices initial configuration and,
60 if needed, a configMap for filebeat logging configuration.
61 - ``templates/_deployment.tpl``: a template that sets up a Kubernetes Deployment for the microservice.
62 - ``templates/_filebeat-config.tpl``: a template containing the standard filebeat configuration for microservices that use filebeat logging.
63 It's used in the ``templates/_configmap.tpl`` template.
64 - ``templates/_job.tpl``: a template that creates a Kubernetes Job that runs when a microservice is deleted. The job brings up a container
65 that removes the microservice configuration information from Consul.
67 Setting variables in ``values.yaml`` for individual microservices
68 -----------------------------------------------------------------
70 Variables used by ONAP OOM common templates
71 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
75 Name and tag of the Docker image for the microservice.
76 Required. The image repository is set using the OOM common
77 ``repositoryGenerator.repository`` template. Normally this points to the
78 ONAP image repository, but it can be overridden on a global basis or a
79 per-chart basis. See the OOM documentation for more details.
85 image: onap/org.onap.dcaegen2.services.prh.prh-app-server:1.5.6
87 **imageRepositoryOverride**:
89 Alternative repository for the Docker image for the microservice.
90 Optional. If this value is set, the Docker image for the microservice
91 will be pulled from the repository specified by this value, instead of
92 from the repository set by ``repositoryGenerator.repository``. Note that
93 this alternative repository is used only for the microservice image and
94 not for other images (such as images for initContainers). Note also that
95 the alternative repository must not use any form of authentication,
96 because there is no way to provide credentials for the repository.
102 imageRepositoryOverride: "myrepo.example.org:5000"
104 **global.pullPolicy** and **pullPolicy**:
106 These settings control when
107 the image is pulled from the repository. ``Always`` means the image is
108 always pulled from the repository when a container is created from the
109 image, while ``IfNotPresent`` means that the image is pulled from the
110 repository only if the image is not already present on the host machine
111 where the container is being created. Typical ONAP OOM practice is to
112 set ``pullPolicy`` to ``Always`` in the chart. During development and
113 testing, this can be overriden during the Helm install with
114 ``global.pullPolicy`` set to ``IfNotPresent``, to speed up deployment by
115 reducing the number of times images are pulled from the repository.
125 Many microservices depend on having other services
126 up and running in the DCAE and ONAP environment - services like AAF to get
127 certificates or DMaaP to communicate with other services.
128 ``readinessCheck.wait_for`` is a list of the *containers* that the
129 microservice needs to have available. If this parameter is present, an
130 initContainer will run and wait for all of the listed containers to
131 become ready. (Unfortunately, it's necessary to know the name of a
132 *container*; it isn't sufficient to list the name of a service.)
140 - dcae-config-binding-service
145 If this parameter is present, a Kubernetes readiness
146 probe will be configured for the microservice. The template supports
147 either an HTTP(S) readiness probe or a script-based readiness probe. The
148 parameter has the following fields that apply to both types:
150 1. ``initialDelaySeconds``: The number of seconds to wait after container startup before attempting the first readiness probe. *[Optional, default 5]*
151 2. ``periodSeconds``: The number of seconds between readiness probes. *[Optional, default 15]*
152 3. ``timeoutSeconds``: The number of seconds to wait for a connection to the container before timing out. *[Optional, default 1]*
153 4. ``probeType``: The type of readiness probe - ``httpGet`` for an HTTP probe or ``exec`` for a script-based probe. *[Optional, default ``httpGet``]*
155 For HTTP(S) readiness probes, the following fields are *required*:
157 1. ``scheme``: ``HTTP`` or ``HTTPS``
158 2. ``path``: the path to the readiness probe API endpoint on the container
159 3. ``port``: the *container port* on which the microservice is listening for readiness probe requests.
160 (This is the *internal* port, not a NodePort or Ingress port.)
162 For script-based readiness probe, the following field is *required*:
163 1. ``command``: an array consisting of the command to be executed to run
164 the readiness probe and any command arguments that are needed.
166 Example (HTTP probe):
171 initialDelaySeconds: 5
177 Example (script-based probe):
183 initialDelaySeconds: 5
187 - /opt/ves-hv-collector/healthcheck.sh
189 Variables used by the DCAE services common templates
190 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
192 **applicationConfig:**
194 *[Optional]*. Initial configuration for
195 microservice. Pushed into Consul for retrieval by config-binding-service
196 and mapped to a file mounted at ``/app-config``. This is a YAML object
197 with keys and values as needed for the specific microservice. It will be
198 converted to JSON before being pushed to Consul or mounted as a file. If
199 not present, defaults to an empty object ({}).
203 Microservice-specific environment variables to be
204 set for the microservice's container. Environment variables can be set
205 to literal string values or a value from a Kubernetes Secret that has
206 been set up using the ONAP OOM common secret template.
208 For a literal string value, use the environment variable name as the
209 key, and the desired string as the value:
214 EXAMPLE_ENV_VAR: "example variable content"
216 For a value taken from a secret, use the environment variable name as
217 the key and set the value to an object with the following fields:
219 1. ``secretUid``: *[Required]* The ``uid`` of the secret (set up with the
220 ONAP OOM common secret template) from which the value will be taken.
221 2. ``key``: *[Required]* The key within the secret that holds the desired value.
222 (A secret can contain several values, each with its own key. One frequently
223 used form of secrets contains login credentials, with keys for username
226 Example of an environment variable set from a secret:
232 secretUid: example-secret
235 The ``applicationEnv`` section of ``values.yaml`` can contain an
236 arbitrary number of environment variables and can contain both literal
237 values and values from secrets. ``applicationEnv`` is optional. If it is
238 not present in the ``values.yaml`` file, no microservice-specific
239 environment variables will be set for the microservice's container.
241 Note that ``applicationEnv`` is a YAML object (or "dictionary"), not an
246 Controls microservice-specific volumes and volume
247 mounts. Allows a microservice to access an externally-created data
248 store. Currently only configMaps are supported. ``externalVolumes`` is a
249 YAML array of objects. Each object has three required fields and two
252 1. ``name``: *[Required]* The Kubernetes name of the configMap to be mounted.
253 The value is a case sensitive string. Because the names of configMaps are
254 sometimes set at deployment time (for instance, to prefix the Helm release to
255 the name), the string can be a Helm template fragment that will be expanded
257 2. ``type``: *[Required]* For now, this is always ``configMap``. This is a
258 case-insensitive string.
259 3. ``mountPath``: *[Required]* The path to the mount point for the volume
260 in the container file system. The value is a case-sensitive string.
261 4. ``readOnly``: *[Optional]* Boolean flag. Set to ``true`` to mount the volume
262 as read-only. Defaults to ``false``.
263 5. ``optional``: *[Optional]* Boolean flag. Set to ``true`` to make the
264 configMap optional (i.e., to allow the microservice's pod to start even
265 if the configMap doesn't exist). If set to ``false``, the configMap must
266 be present in order for the microservice's pod to start. Defaults to
267 ``true``. *Note that this default is the opposite of the Kubernetes
268 default. We've done this to be consistent with the behavior of the DCAE
269 Cloudify plugin for Kubernetes (``k8splugin``), which always set
270 ``optional`` to ``true`` and did not allow for overriding this value.*
272 Example of an ``externalVolumes`` section:
277 - name: my-example-configmap
279 mountPath: /opt/app/config
280 - name: '{{ include "common.release" . }}-another-example'
282 mountPath: /opt/app/otherconfig
284 The dcaegen2-services-common deployment template will set up a volume
285 pointing to the specific configMap in the microservice's pod and a
286 volume mount (mounted at ``mountPath`` on the microservice's container.)
288 The ``externalVolumes`` section is optional. If it is not present, no
289 external volumes will be set up for the microservice.
293 Path to the directory in the microservice's
294 container file system where TLS-certificate information from CMPv2 should
295 be mounted. This is an optional field. When it is present, the
296 dcaegen2-services-common deployment template will set up an
297 initContainer that retrieves the certificate information into a shared
298 volume, which will then be mounted at the path specified by
305 certDirectory: /etc/ves-hv/ssl
309 Path to the directory where the microservice writes
310 its log files. ``logDirectory`` is optional. If ``logDirectory`` is
311 present, the dcaegen2-services-common deployment template will deploy a
312 sidecar container that forwards the log file content to a log server.
318 logDirectory: /var/log/ONAP/dcae-hv-ves-collector
320 Note that ONAP is moving away from the sidecar approach and encouraging
321 applications (including DCAE microservices) to write log information to
322 ``stdout`` and ``stderr``.
326 If present, the dcaegen2-services-common deployment
327 template will deploy a sidecar container that polls the ONAP policy
328 subsystem for policy-driven configuration information.
330 ``policies`` is a YAML object ("dictionary") that can contain the
333 1. ``policyID``: *[Optional]* A string representation of a JSON array of policy ID
334 values that the sidecar should monitor. Default '[]'.
335 2. ``filter``: *[Optional]* A string representation of a JSON array of regular
336 expressions that match policy IDs that the sidecar should monitory. Default '[]'.
337 3. ``duration``: *[Optional]* The interval (in seconds) between polling requests
338 made by the sidecar to the policy subsystem. Default: 2600.
346 '["onap.vfirewall.tca","abc"]'
348 '["DCAE.Config_vfirewall_.*"]'
351 **dcaePolicySyncImage:**
353 Name and tag of the policy sidecar image to be
354 used. Required if the policy sidecar is being used. The image repository
355 is set using the OOM common ``repositoryGenerator.repository`` template.
356 Normally this points to the ONAP image repository, but it can be
357 overridden on a global basis or a per-chart basis. See the OOM
358 documentation for more details.
364 dcaePolicySyncImage: onap/org.onap.dcaegen2.deployments.dcae-services-policy-sync:1.0.1
366 **certProcessorImage:**
368 Name and tag of the CMPv2 certificate
369 initialization image to be used. Required if the microservice is
370 configured to act as a TLS client and/or server using CMPv2
371 certificates. This image runs in an initContainer and sets up trust
372 stores and keystores for CMPv2 use. The image repository is set using
373 the OOM common ``repositoryGenerator.repository`` template. Normally
374 this points to the ONAP image repository, but it can be overridden on a
375 global basis or a per-chart basis. See the OOM documentation for more
382 onap/org.onap.oom.platform.cert-service.oom-certservice-post-processor:2.1.0
385 Deploying multiple instances of a microservice
386 ----------------------------------------------
387 The dcaegen2-services-common charts can be used to deploy multiple instances of the same microservice. To do this successfully,
388 it's necessary to make sure that any Kubernetes service that the microservice exposes has different service names for each instance and,
389 if the service is exposed outside the Kubernetes cluster, a different external port assignment. This can be done by overriding the default
390 settings in the ``values.yaml`` file.
392 As an example, consider the DCAE VES collector (``dcae-ves-collector``). One instance of the VES collector is deployed by default when DCAE is installed using the ONAP installation
393 process. It exposes a service with the name ``dcae-ves-collector`` which is also exposed outside the Kubernetes cluster on NodePort 30417.
395 To deploy a second instance of the VES collector, we can create a YAML override file to define the service exposed by the second instance. The following
396 override file (``ves2.yaml``) will name the service as ``dcae-ves-collector-2`` and expose it on port 30499:
401 name: dcae-ves-collector-2
410 In the directory containing the ``dcae-ves-collector`` chart and the file ``ves.yaml``, running the following command will deploy a second instance
411 of the VES collector:
413 ``helm install -n onap --set global.masterPassword=whatever --set pullPolicy=IfNotPresent -f ves2.yaml ves2 .``
415 This creates a new Helm release called ``ves2``. The instance can be removed with:
417 ``helm delete -n onap ves2``
419 Note that if a component is using TLS with an AAF certificate, the DCAE certificate would need to include the new service name.
420 If a component is using an external certificate (CMPv2), the override file would need to supply the proper parameters to get a certificate with
421 correct common name/SAN.
423 Also note that if the chart for ``dcae-ves-collector`` has been pushed into a Helm repository, the ``helm install`` command can refer to the
424 repository (for instance, ``local/dcae-ves-collector``) instead of using the chart on the local filesystem.
427 Dynamic Topic and Feed Provisioning
428 -----------------------------------
429 This section introduces details on creation of dynamic Dmaap Topics in Message Router and Feeds in Data Router via DCAE helm charts.
431 Provisioning support through DCAE common-service template
432 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
434 When using DCAE common-service template in microservice chart ``deployment.yaml`` file it is required to include ``dcaegen2-services-common.microserviceDeployment`` template.
435 The dcaegen2-services-common include necessary ``common.dmaap.provisioning.initContainer`` template which provisions topics and feeds on Dmaap Message Router and Data Router.
437 Example : Including ``dcaegen2-services-common.microserviceDeployment`` template in ``deployment.yaml``.
441 {{ include "dcaegen2-services-common.microserviceDeployment" . }}
443 The ``common.dmaap.provisioning.initContainer`` template included in DCAE ``dcaegen2-services-common.microserviceDeployment`` makes use of
444 dmaap-bc client image to create Topics on Message Router and Feeds on Data Router microservice, with the help of ``dbc-client.sh`` script,
445 it uses Bus Controller REST API to create resources.
447 If the resource creation is successful via script, the response is logged in file with appropriate naming convention.
450 The configuration provided via ``values.yaml`` file, is consumed by ``common.dmaap.provisioning.initContainer`` template which runs two
451 init-container, First named init-dmaap-provisioning for creating resources on Dmaap, Second named init-merge-config which updates application config
452 with response generated as an outcome of operation by init-dmaap-provisioning container.
454 The figure below shows Dmaap Topics, Feeds Provisioning architecture via dcae-service-common helm charts.
457 The following diagram has been created on https://app.diagrams.net/. There is an editable version of the diagram
458 in repository under path docs/sections/images/dmaap_provisioning_architecture_diagram.drawio, import file to update diagram.
460 .. image:: images/dmaap_provisioning.png
462 Configuration to be added in ``values.yaml`` file.
465 For more information on attributes that are set in ``values.yaml`` for Data Router Feed, Publisher and Subscriber, Message Router Topic creation,
466 you can refer DMaaP Bus Controller API documentation at: https://docs.onap.org/projects/onap-dmaap-buscontroller/en/latest/apis/api.html
468 Dmaap Data Router Feeds creation input can be provided in below format. It consumes list of Feeds.
471 For DR Feed creation except ``feedName``, ``feedDescription``, ``feedVersion`` avoid update on other attributes.
472 All other attributes are mandatory, contains required default values.
477 - feedName: bulk_pm_feed
480 asprClassification: unclassified
481 feedDescription: DFC Feed Creation
483 Once the Feeds creation is successful we can attach Publisher and Subscriber to Feeds.
485 Dmaap Data Router Publisher config:
488 For DR Publisher creation except ``feedName`` avoid update on other attribute.
489 All other attributes are mandatory, contains required default values.
494 - feedName: bulk_pm_feed
495 dcaeLocationName: loc00
497 Dmaap Data Router Subscriber config:
500 For DR Subscriber creation except ``feedName`` avoid update on other attributes.
501 Attribute username, userpwd will be updated via init-merge-config init-container of ``common.dmaap.provisioning.initContainer`` template.
502 In case dcae-pm-mapper microservice is not the Subscriber, attribute deliveryURL need to be updated and privilegedSubscriber can be updated to False.
503 All other attributes are mandatory, contains required default values.
508 - feedName: bulk_pm_feed
510 username: ${DR_USERNAME}
511 userpwd: ${DR_PASSWORD}
512 dcaeLocationName: loc00
513 privilegedSubscriber: True
514 deliveryURL: https://dcae-pm-mapper:8443/delivery
516 Dmaap Message Router Topics creation input can be provided in below format. It consumes list of Topics.
517 Also we can attach Message Router Publisher and Subscriber at same time while creation of Topic.
520 For Message Router Topic creation except ``topicName`` and ``topicDescription`` avoid update on other attributes.
521 All other attributes are mandatory, contains required default values.
526 - topicName: PERFORMANCE_MEASUREMENTS
527 topicDescription: Description about Topic
531 - dcaeLocationName: san-francisco
532 clientRole: org.onap.dcae.pmPublisher
537 Volume configuration for configMap to be provided in ``values.yaml`` file.
543 path: /opt/app/config/feeds
545 path: /opt/app/config/dr_pubs
547 path: /opt/app/config/dr_subs
548 - name: topics-config
549 path: /opt/app/config/topics
552 For example directory containing ``dcae-datafile-collector``, ``dcae-pm-mapper`` chart under
553 dcaegen2-services in OOM repository we can find examples for Feed and Topic creation.
555 Provisioning support through DCAE When using custom deployment.yaml
556 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
558 When using custom ``deployment.yaml`` it is required explicitly to include ``common.dmaap.provisioning.initContainer`` template in
559 initContainer specs of ``deployment.yaml`` file.
561 Example : Including ``common.dmaap.provisioning.initContainer`` template in ``deployment.yaml`` file.
565 {{- include "common.dmaap.provisioning.initContainer" . | nindent XX }}
567 Note also need to take care of the ``Volumes`` that are required to be mounted on Application Pod in ``deployment.yaml``.
571 {{- include "common.dmaap.provisioning._volumes" . | nindent XX -}}
573 Configuration to be added in ``values.yaml`` file is similar to described in ``Provisioning support through DCAE common-service template``.
575 Removal of Data Router Feed, Publisher and Subscriber Or Message Router Topic
576 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
578 DCAE does not support automatic removal of Feed, Publisher and Subscriber from Data Router or Topic from Message Router at present.
579 So it is the responsibility of operator to manually remove Feeds and associated Publisher or Subscriber from Data Router and Topics
580 from Message Router after uninstalling microservice charts which created resources on installation.
582 Reference to DMAAP Bus Controller API documentation to figure out steps for manual removal of resources.
583 https://docs.onap.org/projects/onap-dmaap-buscontroller/en/latest/apis/api.html