-.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. This work is licensed under a Creative Commons Attribution 4.0
+.. International License.
.. http://creativecommons.org/licenses/by/4.0
-.. Copyright 2018 Amdocs, Bell Canada
+.. Copyright 2018-2020 Amdocs, Bell Canada, Orange, Samsung
.. Links
.. _Helm: https://docs.helm.sh/
+.. _Helm Charts: https://github.com/kubernetes/charts
.. _Kubernetes: https://Kubernetes.io/
.. _Docker: https://www.docker.com/
-.. _Nexus: https://nexus.onap.org/#welcome
+.. _Nexus: https://nexus.onap.org/
.. _AWS Elastic Block Store: https://aws.amazon.com/ebs/
.. _Azure File: https://docs.microsoft.com/en-us/azure/storage/files/storage-files-introduction
.. _GCE Persistent Disk: https://cloud.google.com/compute/docs/disks/
.. _Kubernetes Storage Class: https://Kubernetes.io/docs/concepts/storage/storage-classes/
.. _Assigning Pods to Nodes: https://Kubernetes.io/docs/concepts/configuration/assign-pod-node/
+
.. _developer-guide-label:
OOM Developer Guide
###################
-.. figure:: oomLogoV2-medium.png
+.. figure:: images/oom_logo/oomLogoV2-medium.png
:align: right
ONAP consists of a large number of components, each of which are substantial
Helm Charts
===========
+A Helm chart is a collection of files that describe a related set of Kubernetes
+resources. A simple chart might be used to deploy something simple, like a
+memcached pod, while a complex chart might contain many micro-service arranged
+in a hierarchy as found in the `aai` ONAP component.
+
+Charts are created as files laid out in a particular directory tree, then they
+can be packaged into versioned archives to be deployed. There is a public
+archive of `Helm Charts`_ on GitHub that includes many technologies applicable
+to ONAP. Some of these charts have been used in ONAP and all of the ONAP charts
+have been created following the guidelines provided.
+
+The top level of the ONAP charts is shown below:
+
+.. code-block:: bash
+
+ common
+ ├── cassandra
+ │ ├── Chart.yaml
+ │ ├── resources
+ │ │ ├── config
+ │ │ │ └── docker-entrypoint.sh
+ │ │ ├── exec.py
+ │ │ └── restore.sh
+ │ ├── templates
+ │ │ ├── backup
+ │ │ │ ├── configmap.yaml
+ │ │ │ ├── cronjob.yaml
+ │ │ │ ├── pv.yaml
+ │ │ │ └── pvc.yaml
+ │ │ ├── configmap.yaml
+ │ │ ├── pv.yaml
+ │ │ ├── service.yaml
+ │ │ └── statefulset.yaml
+ │ └── values.yaml
+ ├── common
+ │ ├── Chart.yaml
+ │ ├── templates
+ │ │ ├── _createPassword.tpl
+ │ │ ├── _ingress.tpl
+ │ │ ├── _labels.tpl
+ │ │ ├── _mariadb.tpl
+ │ │ ├── _name.tpl
+ │ │ ├── _namespace.tpl
+ │ │ ├── _repository.tpl
+ │ │ ├── _resources.tpl
+ │ │ ├── _secret.yaml
+ │ │ ├── _service.tpl
+ │ │ ├── _storage.tpl
+ │ │ └── _tplValue.tpl
+ │ └── values.yaml
+ ├── ...
+ └── postgres-legacy
+ ├── Chart.yaml
+ ├── charts
+ └── configs
+
+The common section of charts consists of a set of templates that assist with
+parameter substitution (`_name.tpl`, `_namespace.tpl` and others) and a set of
+charts for components used throughout ONAP. When the common components are used
+by other charts they are instantiated each time or we can deploy a shared
+instances for several components.
+
+All of the ONAP components have charts that follow the pattern shown below:
+
+.. code-block:: bash
+
+ name-of-my-component
+ ├── Chart.yaml
+ ├── component
+ │ └── subcomponent-folder
+ ├── charts
+ │ └── subchart-folder
+ ├── resources
+ │ ├── folder1
+ │ │ ├── file1
+ │ │ └── file2
+ │ └── folder1
+ │ ├── file3
+ │ └── folder3
+ │ └── file4
+ ├── templates
+ │ ├── NOTES.txt
+ │ ├── configmap.yaml
+ │ ├── deployment.yaml
+ │ ├── ingress.yaml
+ │ ├── job.yaml
+ │ ├── secrets.yaml
+ │ └── service.yaml
+ └── values.yaml
+
+Note that the component charts / components may include a hierarchy of sub
+components and in themselves can be quite complex.
+
+You can use either `charts` or `components` folder for your subcomponents.
+`charts` folder means that the subcomponent will always been deployed.
+
+`components` folders means we can choose if we want to deploy the
+subcomponent.
+
+This choice is done in root `values.yaml`:
-Standard Chart Format
----------------------
+.. code-block:: yaml
-Helm charts are available in the open-source community for a wide variety of
-common software components which are used in ONAP and where possible.
+ ---
+ global:
+ key: value
+ component1:
+ enabled: true
+ component2:
+ enabled: true
-Chart Hierarchy
----------------
+Then in `Chart.yaml` dependencies section, you'll use these values:
+
+.. code-block:: yaml
+
+ ---
+ dependencies:
+ - name: common
+ version: ~x.y-0
+ repository: '@local'
+ - name: component1
+ version: ~x.y-0
+ repository: 'file://components/component1'
+ condition: component1.enabled
+ - name: component2
+ version: ~x.y-0
+ repository: 'file://components/component2'
+ condition: component2.enabled
+
+Configuration of the components varies somewhat from component to component but
+generally follows the pattern of one or more `configmap.yaml` files which can
+directly provide configuration to the containers in addition to processing
+configuration files stored in the `config` directory. It is the responsibility
+of each ONAP component team to update these configuration files when changes
+are made to the project containers that impact configuration.
+
+The following section describes how the hierarchical ONAP configuration system
+is key to management of such a large system.
+
+Configuration Management
+========================
+
+ONAP is a large system composed of many components - each of which are complex
+systems in themselves - that needs to be deployed in a number of different
+ways. For example, within a single operator's network there may be R&D
+deployments under active development, pre-production versions undergoing system
+testing and production systems that are operating live networks. Each of these
+deployments will differ in significant ways, such as the version of the
+software images deployed. In addition, there may be a number of application
+specific configuration differences, such as operating system environment
+variables. The following describes how the Helm configuration management
+system is used within the OOM project to manage both ONAP infrastructure
+configuration as well as ONAP components configuration.
+
+One of the artifacts that OOM/Kubernetes uses to deploy ONAP components is the
+deployment specification, yet another yaml file. Within these deployment specs
+are a number of parameters as shown in the following example:
+
+.. code-block:: yaml
+
+ apiVersion: apps/v1
+ kind: StatefulSet
+ metadata:
+ labels:
+ app.kubernetes.io/name: zookeeper
+ helm.sh/chart: zookeeper
+ app.kubernetes.io/component: server
+ app.kubernetes.io/managed-by: Tiller
+ app.kubernetes.io/instance: onap-oof
+ name: onap-oof-zookeeper
+ namespace: onap
+ spec:
+ <...>
+ replicas: 3
+ selector:
+ matchLabels:
+ app.kubernetes.io/name: zookeeper
+ app.kubernetes.io/component: server
+ app.kubernetes.io/instance: onap-oof
+ serviceName: onap-oof-zookeeper-headless
+ template:
+ metadata:
+ labels:
+ app.kubernetes.io/name: zookeeper
+ helm.sh/chart: zookeeper
+ app.kubernetes.io/component: server
+ app.kubernetes.io/managed-by: Tiller
+ app.kubernetes.io/instance: onap-oof
+ spec:
+ <...>
+ affinity:
+ containers:
+ - name: zookeeper
+ <...>
+ image: gcr.io/google_samples/k8szk:v3
+ imagePullPolicy: Always
+ <...>
+ ports:
+ - containerPort: 2181
+ name: client
+ protocol: TCP
+ - containerPort: 3888
+ name: election
+ protocol: TCP
+ - containerPort: 2888
+ name: server
+ protocol: TCP
+ <...>
+
+Note that within the statefulset specification, one of the container arguments
+is the key/value pair image: gcr.io/google_samples/k8szk:v3 which
+specifies the version of the zookeeper software to deploy. Although the
+statefulset specifications greatly simplify statefulset, maintenance of the
+statefulset specifications themselves become problematic as software versions
+change over time or as different versions are required for different
+statefulsets. For example, if the R&D team needs to deploy a newer version of
+mariadb than what is currently used in the production environment, they would
+need to clone the statefulset specification and change this value. Fortunately,
+this problem has been solved with the templating capabilities of Helm.
+
+The following example shows how the statefulset specifications are modified to
+incorporate Helm templates such that key/value pairs can be defined outside of
+the statefulset specifications and passed during instantiation of the component.
+
+.. code-block:: yaml
+
+ apiVersion: apps/v1
+ kind: StatefulSet
+ metadata:
+ name: {{ include "common.fullname" . }}
+ namespace: {{ include "common.namespace" . }}
+ labels: {{- include "common.labels" . | nindent 4 }}
+ spec:
+ replicas: {{ .Values.replicaCount }}
+ selector:
+ matchLabels: {{- include "common.matchLabels" . | nindent 6 }}
+ # serviceName is only needed for StatefulSet
+ # put the postfix part only if you have add a postfix on the service name
+ serviceName: {{ include "common.servicename" . }}-{{ .Values.service.postfix }}
+ <...>
+ template:
+ metadata:
+ labels: {{- include "common.labels" . | nindent 8 }}
+ annotations: {{- include "common.tplValue" (dict "value" .Values.podAnnotations "context" $) | nindent 8 }}
+ name: {{ include "common.name" . }}
+ spec:
+ <...>
+ containers:
+ - name: {{ include "common.name" . }}
+ image: {{ .Values.image }}
+ imagePullPolicy: {{ .Values.global.pullPolicy | default .Values.pullPolicy }}
+ ports:
+ {{- range $index, $port := .Values.service.ports }}
+ - containerPort: {{ $port.port }}
+ name: {{ $port.name }}
+ {{- end }}
+ {{- range $index, $port := .Values.service.headlessPorts }}
+ - containerPort: {{ $port.port }}
+ name: {{ $port.name }}
+ {{- end }}
+ <...>
+
+This version of the statefulset specification has gone through the process of
+templating values that are likely to change between statefulsets. Note that the
+image is now specified as: image: {{ .Values.image }} instead of a
+string used previously. During the statefulset phase, Helm (actually the Helm
+sub-component Tiller) substitutes the {{ .. }} entries with a variable defined
+in a values.yaml file. The content of this file is as follows:
+
+.. code-block:: yaml
+
+ <...>
+ image: gcr.io/google_samples/k8szk:v3
+ replicaCount: 3
+ <...>
+
+
+Within the values.yaml file there is an image key with the value
+`gcr.io/google_samples/k8szk:v3` which is the same value used in
+the non-templated version. Once all of the substitutions are complete, the
+resulting statefulset specification ready to be used by Kubernetes.
+
+When creating a template consider the use of default values if appropriate.
+Helm templating has built in support for DEFAULT values, here is
+an example:
+
+.. code-block:: yaml
+
+ imagePullSecrets:
+ - name: "{{ .Values.nsPrefix | default "onap" }}-docker-registry-key"
+
+The pipeline operator ("|") used here hints at that power of Helm templates in
+that much like an operating system command line the pipeline operator allow
+over 60 Helm functions to be embedded directly into the template (note that the
+Helm template language is a superset of the Go template language). These
+functions include simple string operations like upper and more complex flow
+control operations like if/else.
+
+OOM is mainly helm templating. In order to have consistent deployment of the
+different components of ONAP, some rules must be followed.
+
+Templates are provided in order to create Kubernetes resources (Secrets,
+Ingress, Services, ...) or part of Kubernetes resources (names, labels,
+resources requests and limits, ...).
+
+a full list and simple description is done in
+`kubernetes/common/common/documentation.rst`.
+
+Service template
+----------------
+
+In order to create a Service for a component, you have to create a file (with
+`service` in the name.
+For normal service, just put the following line:
+
+.. code-block:: yaml
+
+ {{ include "common.service" . }}
+
+For headless service, the line to put is the following:
+
+.. code-block:: yaml
+
+ {{ include "common.headlessService" . }}
+
+The configuration of the service is done in component `values.yaml`:
+
+.. code-block:: yaml
+
+ service:
+ name: NAME-OF-THE-SERVICE
+ postfix: MY-POSTFIX
+ type: NodePort
+ annotations:
+ someAnnotationsKey: value
+ ports:
+ - name: tcp-MyPort
+ port: 5432
+ nodePort: 88
+ - name: http-api
+ port: 8080
+ nodePort: 89
+ - name: https-api
+ port: 9443
+ nodePort: 90
+
+`annotations` and `postfix` keys are optional.
+if `service.type` is `NodePort`, then you have to give `nodePort` value for your
+service ports (which is the end of the computed nodePort, see example).
+
+It would render the following Service Resource (for a component named
+`name-of-my-component`, with version `x.y.z`, helm deployment name
+`my-deployment` and `global.nodePortPrefix` `302`):
+
+.. code-block:: yaml
+
+ apiVersion: v1
+ kind: Service
+ metadata:
+ annotations:
+ someAnnotationsKey: value
+ name: NAME-OF-THE-SERVICE-MY-POSTFIX
+ labels:
+ app.kubernetes.io/name: name-of-my-component
+ helm.sh/chart: name-of-my-component-x.y.z
+ app.kubernetes.io/instance: my-deployment-name-of-my-component
+ app.kubernetes.io/managed-by: Tiller
+ spec:
+ ports:
+ - port: 5432
+ targetPort: tcp-MyPort
+ nodePort: 30288
+ - port: 8080
+ targetPort: http-api
+ nodePort: 30289
+ - port: 9443
+ targetPort: https-api
+ nodePort: 30290
+ selector:
+ app.kubernetes.io/name: name-of-my-component
+ app.kubernetes.io/instance: my-deployment-name-of-my-component
+ type: NodePort
+
+In the deployment or statefulSet file, you needs to set the good labels in
+order for the service to match the pods.
+
+here's an example to be sure it matches (for a statefulSet):
+
+.. code-block:: yaml
+
+ apiVersion: apps/v1
+ kind: StatefulSet
+ metadata:
+ name: {{ include "common.fullname" . }}
+ namespace: {{ include "common.namespace" . }}
+ labels: {{- include "common.labels" . | nindent 4 }}
+ spec:
+ selector:
+ matchLabels: {{- include "common.matchLabels" . | nindent 6 }}
+ # serviceName is only needed for StatefulSet
+ # put the postfix part only if you have add a postfix on the service name
+ serviceName: {{ include "common.servicename" . }}-{{ .Values.service.postfix }}
+ <...>
+ template:
+ metadata:
+ labels: {{- include "common.labels" . | nindent 8 }}
+ annotations: {{- include "common.tplValue" (dict "value" .Values.podAnnotations "context" $) | nindent 8 }}
+ name: {{ include "common.name" . }}
+ spec:
+ <...>
+ containers:
+ - name: {{ include "common.name" . }}
+ ports:
+ {{- range $index, $port := .Values.service.ports }}
+ - containerPort: {{ $port.port }}
+ name: {{ $port.name }}
+ {{- end }}
+ {{- range $index, $port := .Values.service.headlessPorts }}
+ - containerPort: {{ $port.port }}
+ name: {{ $port.name }}
+ {{- end }}
+ <...>
+
+The configuration of the service is done in component `values.yaml`:
+
+.. code-block:: yaml
+
+ service:
+ name: NAME-OF-THE-SERVICE
+ headless:
+ postfix: NONE
+ annotations:
+ anotherAnnotationsKey : value
+ publishNotReadyAddresses: true
+ headlessPorts:
+ - name: tcp-MyPort
+ port: 5432
+ - name: http-api
+ port: 8080
+ - name: https-api
+ port: 9443
+
+`headless.annotations`, `headless.postfix` and
+`headless.publishNotReadyAddresses` keys are optional.
+
+If `headless.postfix` is not set, then we'll add `-headless` at the end of the
+service name.
+
+If it set to `NONE`, there will be not postfix.
+
+And if set to something, it will add `-something` at the end of the service
+name.
+
+It would render the following Service Resource (for a component named
+`name-of-my-component`, with version `x.y.z`, helm deployment name
+`my-deployment` and `global.nodePortPrefix` `302`):
+
+.. code-block:: yaml
+
+ apiVersion: v1
+ kind: Service
+ metadata:
+ annotations:
+ anotherAnnotationsKey: value
+ name: NAME-OF-THE-SERVICE
+ labels:
+ app.kubernetes.io/name: name-of-my-component
+ helm.sh/chart: name-of-my-component-x.y.z
+ app.kubernetes.io/instance: my-deployment-name-of-my-component
+ app.kubernetes.io/managed-by: Tiller
+ spec:
+ clusterIP: None
+ ports:
+ - port: 5432
+ targetPort: tcp-MyPort
+ nodePort: 30288
+ - port: 8080
+ targetPort: http-api
+ nodePort: 30289
+ - port: 9443
+ targetPort: https-api
+ nodePort: 30290
+ publishNotReadyAddresses: true
+ selector:
+ app.kubernetes.io/name: name-of-my-component
+ app.kubernetes.io/instance: my-deployment-name-of-my-component
+ type: ClusterIP
+
+Previous example of StatefulSet would also match (except for the `postfix` part
+obviously).
+
+Creating Deployment or StatefulSet
+----------------------------------
+
+Deployment and StatefulSet should use the `apps/v1` (which has appeared in
+v1.9).
+As seen on the service part, the following parts are mandatory:
+
+.. code-block:: yaml
+
+ apiVersion: apps/v1
+ kind: StatefulSet
+ metadata:
+ name: {{ include "common.fullname" . }}
+ namespace: {{ include "common.namespace" . }}
+ labels: {{- include "common.labels" . | nindent 4 }}
+ spec:
+ selector:
+ matchLabels: {{- include "common.matchLabels" . | nindent 6 }}
+ # serviceName is only needed for StatefulSet
+ # put the postfix part only if you have add a postfix on the service name
+ serviceName: {{ include "common.servicename" . }}-{{ .Values.service.postfix }}
+ <...>
+ template:
+ metadata:
+ labels: {{- include "common.labels" . | nindent 8 }}
+ annotations: {{- include "common.tplValue" (dict "value" .Values.podAnnotations "context" $) | nindent 8 }}
+ name: {{ include "common.name" . }}
+ spec:
+ <...>
+ containers:
+ - name: {{ include "common.name" . }}
+
+ONAP Application Configuration
+------------------------------
Dependency Management
---------------------
.. code-block:: yaml
- apiVersion: extensions/v1beta1
+ apiVersion: apps/v1
kind: Deployment
metadata:
- name: {{ include "common.name" . }}
+ name: {{ include "common.fullname" . }}
namespace: {{ include "common.namespace" . }}
- labels:
- app: {{ include "common.name" . }}
- chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
- release: {{ .Release.Name }}
- heritage: {{ .Release.Service }}
+ labels: {{- include "common.labels" . | nindent 4 }}
spec:
replicas: {{ .Values.replicaCount }}
+ selector:
+ matchLabels: {{- include "common.matchLabels" . | nindent 6 }}
template:
metadata:
labels:
spec:
initContainers:
- command:
- - /root/ready.py
+ - /app/ready.py
args:
- --container-name
- so-mariadb
Name Spaces
-----------
-Within the namespaces are Kubernetes services that provide external connectivity to pods that host Docker containers.
+Within the namespaces are Kubernetes services that provide external
+connectivity to pods that host Docker containers.
ONAP Components to Kubernetes Object Relationships
--------------------------------------------------
.. database PV
.. @enduml
-.. figure:: kubernetes_objects.png
+.. figure:: images/k8s/kubernetes_objects.png
OOM uses these Kubernetes objects as described in the following sections.
~~~~~
OOM works with both physical and virtual worker machines.
-* Virtual Machine Deployments - If ONAP is to be deployed onto a set of virtual machines, the creation of the VMs is outside of the scope of OOM and could be done in many ways, such as
+* Virtual Machine Deployments - If ONAP is to be deployed onto a set of virtual
+ machines, the creation of the VMs is outside of the scope of OOM and could be
+ done in many ways, such as
- * manually, for example by a user using the OpenStack Horizon dashboard or AWS EC2, or
- * automatically, for example with the use of a OpenStack Heat Orchestration Template which builds an ONAP stack, Azure ARM template, AWS CloudFormation Template, or
- * orchestrated, for example with Cloudify creating the VMs from a TOSCA template and controlling their life cycle for the life of the ONAP deployment.
+ * manually, for example by a user using the OpenStack Horizon dashboard or
+ AWS EC2, or
+ * automatically, for example with the use of a OpenStack Heat Orchestration
+ Template which builds an ONAP stack, Azure ARM template, AWS CloudFormation
+ Template, or
+ * orchestrated, for example with Cloudify creating the VMs from a TOSCA
+ template and controlling their life cycle for the life of the ONAP
+ deployment.
-* Physical Machine Deployments - If ONAP is to be deployed onto physical machines there are several options but the recommendation is to use Rancher along with Helm to associate hosts with a Kubernetes cluster.
+* Physical Machine Deployments - If ONAP is to be deployed onto physical
+ machines there are several options but the recommendation is to use Rancher
+ along with Helm to associate hosts with a Kubernetes cluster.
Pods
~~~~
-A group of containers with shared storage and networking can be grouped together into a Kubernetes pod. All of the containers within a pod are co-located and co-scheduled so they operate as a single unit. Within ONAP Amsterdam release, pods are mapped one-to-one to docker containers although this may change in the future. As explained in the Services section below the use of Pods within each ONAP component is abstracted from other ONAP components.
+A group of containers with shared storage and networking can be grouped
+together into a Kubernetes pod. All of the containers within a pod are
+co-located and co-scheduled so they operate as a single unit. Within ONAP
+Amsterdam release, pods are mapped one-to-one to docker containers although
+this may change in the future. As explained in the Services section below the
+use of Pods within each ONAP component is abstracted from other ONAP
+components.
Services
~~~~~~~~
------------------------------
- DNS
-- Ports - Flattening the containers also expose port conflicts between the containers which need to be resolved.
+- Ports - Flattening the containers also expose port conflicts between the
+ containers which need to be resolved.
Node Ports
~~~~~~~~~~
nodeSelector:
failure-domain.beta.Kubernetes.io/region: {{ .Values.location }}
-- "location: West" is specified in the values.yaml file used to deploy
- one DCAE cluster and "location: East" is specified in a second values.yaml
+- "location: West" is specified in the `values.yaml` file used to deploy
+ one DCAE cluster and "location: East" is specified in a second `values.yaml`
file (see OOM Configuration Management for more information about
- configuration files like the values.yamlfile).
+ configuration files like the `values.yaml` file).
Node affinity can also be used to achieve geographic redundancy if pods are
assigned to multiple failure domains. For more information refer to `Assigning
used for all persistent state information.
-Configuration Management
-========================
-ONAP is a large system composed of many components - each of which are complex
-systems in themselves - that needs to be deployed in a number of different
-ways. For example, within a single operator's network there may be R&D
-deployments under active development, pre-production versions undergoing system
-testing and production systems that are operating live networks. Each of these
-deployments will differ in significant ways, such as the version of the
-software images deployed. In addition, there may be a number of application
-specific configuration differences, such as operating system environment
-variables. The following describes how the Helm configuration management
-system is used within the OOM project to manage both ONAP infrastructure
-configuration as well as ONAP components configuration.
+Environment Files
+~~~~~~~~~~~~~~~~~
-One of the artifacts that OOM/Kubernetes uses to deploy ONAP components is the
-deployment specification, yet another yaml file. Within these deployment specs
-are a number of parameters as shown in the following mariadb example:
+MSB Integration
+===============
-.. code-block:: yaml
+The \ `Microservices Bus
+Project <https://wiki.onap.org/pages/viewpage.action?pageId=3246982>`__ provides
+facilities to integrate micro-services into ONAP and therefore needs to
+integrate into OOM - primarily through Consul which is the backend of
+MSB service discovery. The following is a brief description of how this
+integration will be done:
- apiVersion: extensions/v1beta1
- kind: Deployment
- metadata:
- name: mariadb
- spec:
- <...>
- template:
- <...>
- spec:
- hostname: mariadb
- containers:
- - args:
- image: nexus3.onap.org:10001/mariadb:10.1.11
- name: "mariadb"
- env:
- - name: MYSQL_ROOT_PASSWORD
- value: password
- - name: MARIADB_MAJOR
- value: "10.1"
- <...>
- imagePullSecrets:
- - name: onap-docker-registry-key
-
-Note that within the deployment specification, one of the container arguments
-is the key/value pair image: nexus3.onap.org:10001/mariadb:10.1.11 which
-specifies the version of the mariadb software to deploy. Although the
-deployment specifications greatly simplify deployment, maintenance of the
-deployment specifications themselves become problematic as software versions
-change over time or as different versions are required for different
-deployments. For example, if the R&D team needs to deploy a newer version of
-mariadb than what is currently used in the production environment, they would
-need to clone the deployment specification and change this value. Fortunately,
-this problem has been solved with the templating capabilities of Helm.
+A registrator to push the service endpoint info to MSB service
+discovery.
-The following example shows how the deployment specifications are modified to
-incorporate Helm templates such that key/value pairs can be defined outside of
-the deployment specifications and passed during instantiation of the component.
+- The needed service endpoint info is put into the kubernetes yaml file
+ as annotation, including service name, Protocol,version, visual
+ range,LB method, IP, Port,etc.
+
+- OOM deploy/start/restart/scale in/scale out/upgrade ONAP components
+
+- Registrator watch the kubernetes event
+
+- When an ONAP component instance has been started/destroyed by OOM,
+ Registrator get the notification from kubernetes
+
+- Registrator parse the service endpoint info from annotation and
+ register/update/unregister it to MSB service discovery
+
+- MSB API Gateway uses the service endpoint info for service routing
+ and load balancing.
+
+Details of the registration service API can be found at \ `Microservice
+Bus API
+Documentation <https://wiki.onap.org/display/DW/Microservice+Bus+API+Documentation>`__.
+
+ONAP Component Registration to MSB
+----------------------------------
+The charts of all ONAP components intending to register against MSB must have
+an annotation in their service(s) template. A `sdc` example follows:
.. code-block:: yaml
- apiVersion: extensions/v1beta1
- kind: Deployment
- metadata:
- name: mariadb
- namespace: "{{ .Values.nsPrefix }}-mso"
- spec:
- <...>
- template:
- <...>
- spec:
- hostname: mariadb
- containers:
- - args:
- image: {{ .Values.image.mariadb }}
- imagePullPolicy: {{ .Values.pullPolicy }}
- name: "mariadb"
- env:
- - name: MYSQL_ROOT_PASSWORD
- value: password
- - name: MARIADB_MAJOR
- value: "10.1"
- <...>
- imagePullSecrets:
- - name: "{{ .Values.nsPrefix }}-docker-registry-key"apiVersion: extensions/v1beta1
- kind: Deployment
+ apiVersion: v1
+ kind: Service
metadata:
- name: mariadb
- namespace: "{{ .Values.nsPrefix }}-mso"
- spec:
- <...>
- template:
- <...>
- spec:
- hostname: mariadb
- containers:
- - args:
- image: {{ .Values.image.mariadb }}
- imagePullPolicy: {{ .Values.pullPolicy }}
- name: "mariadb"
- env:
- - name: MYSQL_ROOT_PASSWORD
- value: password
- - name: MARIADB_MAJOR
- value: "10.1"
- <...>
- imagePullSecrets:
- - name: "{{ .Values.nsPrefix }}-docker-registry-key"
+ labels:
+ app: sdc-be
+ name: sdc-be
+ namespace: "{{ .Values.nsPrefix }}"
+ annotations:
+ msb.onap.org/service-info: '[
+ {
+ "serviceName": "sdc",
+ "version": "v1",
+ "url": "/sdc/v1",
+ "protocol": "REST",
+ "port": "8080",
+ "visualRange":"1"
+ },
+ {
+ "serviceName": "sdc-deprecated",
+ "version": "v1",
+ "url": "/sdc/v1",
+ "protocol": "REST",
+ "port": "8080",
+ "visualRange":"1",
+ "path":"/sdc/v1"
+ }
+ ]'
+ ...
-This version of the deployment specification has gone through the process of
-templating values that are likely to change between deployments. Note that the
-image is now specified as: image: {{ .Values.image.mariadb }} instead of a
-string used previously. During the deployment phase, Helm (actually the Helm
-sub-component Tiller) substitutes the {{ .. }} entries with a variable defined
-in a values.yaml file. The content of this file is as follows:
-.. code-block:: yaml
+MSB Integration with OOM
+------------------------
+A preliminary view of the OOM-MSB integration is as follows:
- nsPrefix: onap
- pullPolicy: IfNotPresent
- image:
- readiness: oomk8s/readiness-check:1.0.0
- mso: nexus3.onap.org:10001/openecomp/mso:1.0-STAGING-latest
- mariadb: nexus3.onap.org:10001/mariadb:10.1.11
+.. figure:: images/msb/MSB-OOM-Diagram.png
-Within the values.yaml file there is an image section with the key/value pair
-mariadb: nexus3.onap.org:10001/mariadb:10.1.11 which is the same value used in
-the non-templated version. Once all of the substitutions are complete, the
-resulting deployment specification ready to be used by Kubernetes.
+A message sequence chart of the registration process:
-Also note that in this example, the namespace key/value pair is specified in
-the values.yaml file. This key/value pair will be global across the entire
-ONAP deployment and is therefore a prime example of where configuration
-hierarchy can be very useful.
+.. uml::
-When creating a deployment template consider the use of default values if
-appropriate. Helm templating has built in support for DEFAULT values, here is
-an example:
+ participant "OOM" as oom
+ participant "ONAP Component" as onap
+ participant "Service Discovery" as sd
+ participant "External API Gateway" as eagw
+ participant "Router (Internal API Gateway)" as iagw
-.. code-block:: yaml
+ box "MSB" #LightBlue
+ participant sd
+ participant eagw
+ participant iagw
+ end box
- imagePullSecrets:
- - name: "{{ .Values.nsPrefix | default "onap" }}-docker-registry-key"
+ == Deploy Servcie ==
-The pipeline operator ("|") used here hints at that power of Helm templates in
-that much like an operating system command line the pipeline operator allow
-over 60 Helm functions to be embedded directly into the template (note that the
-Helm template language is a superset of the Go template language). These
-functions include simple string operations like upper and more complex flow
-control operations like if/else.
+ oom -> onap: Deploy
+ oom -> sd: Register service endpoints
+ sd -> eagw: Services exposed to external system
+ sd -> iagw: Services for internal use
+ == Component Life-cycle Management ==
-ONAP Application Configuration
-------------------------------
+ oom -> onap: Start/Stop/Scale/Migrate/Upgrade
+ oom -> sd: Update service info
+ sd -> eagw: Update service info
+ sd -> iagw: Update service info
-Environment Files
-~~~~~~~~~~~~~~~~~
+ == Service Health Check ==
+
+ sd -> onap: Check the health of service
+ sd -> eagw: Update service status
+ sd -> iagw: Update service status
+
+
+MSB Deployment Instructions
+---------------------------
+MSB is helm installable ONAP component which is often automatically deployed.
+To install it individually enter::
+
+ > helm install <repo-name>/msb
+
+.. note::
+ TBD: Vaidate if the following procedure is still required.
+
+Please note that Kubernetes authentication token must be set at
+*kubernetes/kube2msb/values.yaml* so the kube2msb registrator can get the
+access to watch the kubernetes events and get service annotation by
+Kubernetes APIs. The token can be found in the kubectl configuration file
+*~/.kube/config*
+
+More details can be found here `MSB installation <https://docs.onap.org/projects/onap-msb-apigateway/en/latest/platform/installation.html>`_.
-MSB Integration
-===============
.. MISC
.. ====
.. Note that although OOM uses Kubernetes facilities to minimize the effort
-.. required of the ONAP component owners to implement a successful rolling upgrade
-.. strategy there are other considerations that must be taken into consideration.
+.. required of the ONAP component owners to implement a successful rolling
+.. upgrade strategy there are other considerations that must be taken into
+.. consideration.
.. For example, external APIs - both internal and external to ONAP - should be
-.. designed to gracefully accept transactions from a peer at a different software
-.. version to avoid deadlock situations. Embedded version codes in messages may
-.. facilitate such capabilities.
+.. designed to gracefully accept transactions from a peer at a different
+.. software version to avoid deadlock situations. Embedded version codes in
+.. messages may facilitate such capabilities.
..
-.. Within each of the projects a new configuration repository contains all of the
-.. project specific configuration artifacts. As changes are made within the
-.. project, it's the responsibility of the project team to make appropriate
+.. Within each of the projects a new configuration repository contains all of
+.. the project specific configuration artifacts. As changes are made within
+.. the project, it's the responsibility of the project team to make appropriate
.. changes to the configuration data.
Code Review / oom.git / blobdiff