-.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. This work is licensed under a Creative Commons Attribution 4.0
+.. International License.
.. http://creativecommons.org/licenses/by/4.0
-.. Copyright 2018 Amdocs, Bell Canada
+.. Copyright 2018-2020 Amdocs, Bell Canada, Orange, Samsung
.. _oom_user_guide:
.. Links
.. _Helm Documentation: https://docs.helm.sh/helm/
.. _Helm: https://docs.helm.sh/
.. _Kubernetes: https://Kubernetes.io/
-.. _Kubernetes LoadBalancer: https://kubernetes.io/docs/concepts/services-networking/service/#type-loadbalancer
+.. _Kubernetes LoadBalancer: https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer
.. _user-guide-label:
-OOM User Guide
-##############
+OOM User Guide helm3 (experimental)
+###################################
The ONAP Operations Manager (OOM) provide the ability to manage the entire
life-cycle of an ONAP installation, from the initial deployment to final
- Monitor_ - real-time health monitoring feeding to a Consul UI and Kubernetes
- Heal_- failed ONAP containers are recreated automatically
- Scale_ - cluster ONAP services to enable seamless scaling
-- Upgrade_ - change-out containers or configuration with little or no service impact
+- Upgrade_ - change-out containers or configuration with little or no service
+ impact
- Delete_ - cleanup individual containers or entire deployments
.. figure:: oomLogoV2-Deploy.png
Pre-requisites
--------------
-Your environment must have both the Kubernetes `kubectl` and Helm setup as a
-one time activity.
+Your environment must have the Kubernetes `kubectl` with Cert-Manager
+and Helm setup as a one time activity.
Install Kubectl
~~~~~~~~~~~~~~~
on other O/Ss), the Kubernetes command line interface used to manage a
Kubernetes cluster::
- > curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.8.10/bin/linux/amd64/kubectl
+ > curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.15.11/bin/linux/amd64/kubectl
> chmod +x ./kubectl
> sudo mv ./kubectl /usr/local/bin/kubectl
> mkdir ~/.kube
> kubectl get pods --all-namespaces
-At this point you should see six Kubernetes pods running.
+At this point you should see Kubernetes pods running.
+
+Install Cert-Manager
+~~~~~~~~~~~~~~~~~~~~
+Details on how to install Cert-Manager can be found
+:doc:`here <oom_setup_paas>`.
Install Helm
~~~~~~~~~~~~
Helm is used by OOM for package and configuration management. To install Helm,
enter the following::
- > wget http://storage.googleapis.com/kubernetes-helm/helm-v2.9.1-linux-amd64.tar.gz
- > tar -zxvf helm-v2.9.1-linux-amd64.tar.gz
+ > wget https://get.helm.sh/helm-v3.5.2-linux-amd64.tar.gz
+ > tar -zxvf helm-v3.5.2-linux-amd64.tar.gz
> sudo mv linux-amd64/helm /usr/local/bin/helm
Verify the Helm version with::
> helm version
-Install the Helm Tiller application and initialize with::
-
- > helm init
-
Install the Helm Repo
---------------------
Once kubectl and Helm are setup, one needs to setup a local Helm server to
To prepare your system for an installation of ONAP, you'll need to::
- > git clone -b frankfurt --recurse-submodules -j2 http://gerrit.onap.org/r/oom
+ > git clone -b guilin --recurse-submodules -j2 http://gerrit.onap.org/r/oom
> cd oom/kubernetes
+To install a local Helm server::
+
+ > curl -LO https://s3.amazonaws.com/chartmuseum/release/latest/bin/linux/amd64/chartmuseum
+ > chmod +x ./chartmuseum
+ > mv ./chartmuseum /usr/local/bin
+
To setup a local Helm server to server up the ONAP charts::
- > helm init
- > helm serve &
+ > mkdir -p ~/helm3-storage
+ > chartmuseum --storage local --storage-local-rootdir ~/helm3-storage -port 8879 &
Note the port number that is listed and use it in the Helm repo add as
follows::
Then build your local Helm repository::
- > make SKIP_LINT=TRUE all
+ > make SKIP_LINT=TRUE [HELM_BIN=<HELM_PATH>] all
+
+`HELM_BIN`
+ Sets the helm binary to be used. The default value use helm from PATH
The Helm search command reads through all of the repositories configured on the
system, and looks for matches::
- > helm search -l
+ > helm search repo local
NAME VERSION DESCRIPTION
local/appc 2.0.0 Application Controller
local/clamp 2.0.0 ONAP Clamp
In any case, setup of the Helm repository is a one time activity.
-Next, install Helm Plugins required to deploy the ONAP Casablanca release::
+Next, install Helm Plugins required to deploy the ONAP release::
- > cp -R helm/plugins/ ~/.helm
+ > cp -R ~/oom/kubernetes/helm/plugins/ ~/.local/share/helm/plugins
Once the repo is setup, installation of ONAP can be done with a single
command::
- > helm deploy development local/onap --namespace onap
+ > helm deploy development local/onap --namespace onap --set global.masterPassword=password
This will install ONAP from a local repository in a 'development' Helm release.
As described below, to override the default configuration values provided by
OOM, an environment file can be provided on the command line as follows::
- > helm deploy development local/onap --namespace onap -f overrides.yaml
+
+
+ > helm deploy development local/onap --namespace onap -f overrides.yaml --set global.masterPassword=password
+
+.. note::
+ Refer the Configure_ section on how to update overrides.yaml and values.yaml
To get a summary of the status of all of the pods (containers) running in your
deployment::
- > kubectl get pods --all-namespaces -o=wide
+ > kubectl get pods --namespace onap -o=wide
.. note::
The Kubernetes namespace concept allows for multiple instances of a component
(such as all of ONAP) to co-exist with other components in the same
Kubernetes cluster by isolating them entirely. Namespaces share only the
hosts that form the cluster thus providing isolation between production and
- development systems as an example. The OOM deployment of ONAP in Beijing is
- now done within a single Kubernetes namespace where in Amsterdam a namespace
- was created for each of the ONAP components.
+ development systems as an example.
.. note::
The Helm `--name` option refers to a release name and not a Kubernetes namespace.
To install a specific version of a single ONAP component (`so` in this example)
with the given release name enter::
- > helm deploy so onap/so --version 3.0.1
+ > helm deploy so onap/so --version 8.0.0 --set global.masterPassword=password --set global.flavor=unlimited --namespace onap
+
+.. note::
+ The dependent components should be installed for component being installed
+
To display details of a specific resource or group of resources type::
To deploy ONAP with this environment file, enter::
- > helm deploy local/onap -n onap -f environments/onap-production.yaml
+ > helm deploy local/onap -n onap -f onap/resources/environments/onap-production.yaml --set global.masterPassword=password
.. include:: environments_onap_demo.yaml
:code: yaml
dependencies:
<...>
- name: so
- version: ~2.0.0
+ version: ~8.0.0
repository: '@local'
condition: so.enabled
<...>
-The ~ operator in the `so` version value indicates that the latest "2.X.X"
+The ~ operator in the `so` version value indicates that the latest "8.X.X"
version of `so` shall be used thus allowing the chart to allow for minor
-upgrades that don't impact the so API; hence, version 2.0.1 will be installed
+upgrades that don't impact the so API; hence, version 8.0.1 will be installed
in this case.
-The onap/resources/environment/onap-dev.yaml (see the excerpt below) enables
+The onap/resources/environment/dev.yaml (see the excerpt below) enables
for fine grained control on what components are included as part of this
deployment. By changing this `so` line to `enabled: false` the `so` component
will not be deployed. If this change is part of an upgrade the existing `so`
------------------------------------------------------------
The ONAP deployment created by OOM operates in a private IP network that isn't
-publicly accessible (i.e. Openstack VMs with private internal network) which
+publicly accessible (i.e. OpenStack VMs with private internal network) which
blocks access to the ONAP Portal. To enable direct access to this Portal from a
user's own environment (a laptop etc.) the portal application's port 8989 is
exposed through a `Kubernetes LoadBalancer`_ object.
Typically, to be able to access the Kubernetes nodes publicly a public address
-is assigned. In Openstack this is a floating IP address.
+is assigned. In OpenStack this is a floating IP address.
When the `portal-app` chart is deployed a Kubernetes service is created that
instantiates a load balancer. The LB chooses the private interface of one of
the nodes as in the example below (10.0.0.4 is private to the K8s cluster only).
Then to be able to access the portal on port 8989 from outside the K8s &
-Openstack environment, the user needs to assign/get the floating IP address that
+OpenStack environment, the user needs to assign/get the floating IP address that
corresponds to the private IP as follows::
> kubectl -n onap get services|grep "portal-app"
In this example, use the 10.0.0.4 private address as a key find the
corresponding public address which in this example is 10.12.6.155. If you're
-using OpenStack you'll do the lookup with the horizon GUI or the Openstack CLI
+using OpenStack you'll do the lookup with the horizon GUI or the OpenStack CLI
for your tenant (openstack server list). That IP is then used in your
`/etc/hosts` to map the fixed DNS aliases required by the ONAP Portal as shown
below::
Ensure you've disabled any proxy settings the browser you are using to access
the portal and then simply access now the new ssl-encrypted URL:
-https://portal.api.simpledemo.onap.org:30225/ONAPPORTAL/login.htm
+``https://portal.api.simpledemo.onap.org:30225/ONAPPORTAL/login.htm``
.. note::
Using the HTTPS based Portal URL the Browser needs to be configured to accept
| Alternatives Considered:
- - Kubernetes port forwarding was considered but discarded as it would require
- the end user to run a script that opens up port forwarding tunnels to each of
- the pods that provides a portal application widget.
+ - Kubernetes port forwarding was considered but discarded as it would
+ require the end user to run a script that opens up port forwarding tunnels
+ to each of the pods that provides a portal application widget.
- Reverting to a VNC server similar to what was deployed in the Amsterdam
- release was also considered but there were many issues with resolution, lack
- of volume mount, /etc/hosts dynamic update, file upload that were a tall order
- to solve in time for the Beijing release.
+ release was also considered but there were many issues with resolution,
+ lack of volume mount, /etc/hosts dynamic update, file upload that were
+ a tall order to solve in time for the Beijing release.
Observations:
- - If you are not using floating IPs in your Kubernetes deployment and directly attaching
- a public IP address (i.e. by using your public provider network) to your K8S Node
- VMs' network interface, then the output of 'kubectl -n onap get services | grep "portal-app"'
+ - If you are not using floating IPs in your Kubernetes deployment and
+ directly attaching a public IP address (i.e. by using your public provider
+ network) to your K8S Node VMs' network interface, then the output of
+ 'kubectl -n onap get services | grep "portal-app"'
will show your public IP instead of the private network's IP. Therefore,
- you can grab this public IP directly (as compared to trying to find the floating
- IP first) and map this IP in /etc/hosts.
+ you can grab this public IP directly (as compared to trying to find the
+ floating IP first) and map this IP in /etc/hosts.
.. figure:: oomLogoV2-Monitor.png
:align: right
All highly available systems include at least one facility to monitor the
health of components within the system. Such health monitors are often used as
-inputs to distributed coordination systems (such as etcd, zookeeper, or consul)
-and monitoring systems (such as nagios or zabbix). OOM provides two mechanisms
+inputs to distributed coordination systems (such as etcd, Zookeeper, or Consul)
+and monitoring systems (such as Nagios or Zabbix). OOM provides two mechanisms
to monitor the real-time health of an ONAP deployment:
- a Consul GUI for a human operator or downstream monitoring systems and
.. figure:: consulHealth.png
:align: center
-To see the real-time health of a deployment go to: http://<kubernetes IP>:30270/ui/
+To see the real-time health of a deployment go to: ``http://<kubernetes IP>:30270/ui/``
where a GUI much like the following will be found:
+.. note::
+ If Consul GUI is not accessible, you can refer this
+ `kubectl port-forward <https://kubernetes.io/docs/tasks/access-application-cluster/port-forward-access-application-cluster/>`_ method to access an application
.. figure:: oomLogoV2-Heal.png
:align: right
In order to change the size of a cluster, an operator could use a helm upgrade
(described in detail in the next section) as follows::
- > helm upgrade --set replicaCount=3 onap/so/mariadb
+ > helm upgrade [RELEASE] [CHART] [flags]
+
+The RELEASE argument can be obtained from the following command::
+
+ > helm list
+
+Below is the example for the same::
+
+ > helm list
+ NAME REVISION UPDATED STATUS CHART APP VERSION NAMESPACE
+ dev 1 Wed Oct 14 13:49:52 2020 DEPLOYED onap-8.0.0 Honolulu onap
+ dev-cassandra 5 Thu Oct 15 14:45:34 2020 DEPLOYED cassandra-8.0.0 onap
+ dev-contrib 1 Wed Oct 14 13:52:53 2020 DEPLOYED contrib-8.0.0 onap
+ dev-mariadb-galera 1 Wed Oct 14 13:55:56 2020 DEPLOYED mariadb-galera-8.0.0 onap
+
+Here the Name column shows the RELEASE NAME, In our case we want to try the
+scale operation on cassandra, thus the RELEASE NAME would be dev-cassandra.
+
+Now we need to obtain the chart name for casssandra. Use the below
+command to get the chart name::
+
+ > helm search cassandra
+
+Below is the example for the same::
+
+ > helm search cassandra
+ NAME CHART VERSION APP VERSION DESCRIPTION
+ local/cassandra 8.0.0 ONAP cassandra
+ local/portal-cassandra 8.0.0 Portal cassandra
+ local/aaf-cass 8.0.0 ONAP AAF cassandra
+ local/sdc-cs 8.0.0 ONAP Service Design and Creation Cassandra
+
+Here the Name column shows the chart name. As we want to try the scale
+operation for cassandra, thus the correponding chart name is local/cassandra
+
+
+Now we have both the command's arguments, thus we can perform the
+scale opeartion for cassandra as follows::
+
+ > helm upgrade dev-cassandra local/cassandra --set replicaCount=3
+
+Using this command we can scale up or scale down the cassadra db instances.
+
The ONAP components use Kubernetes provided facilities to build clustered,
highly available systems including: Services_ with load-balancers, ReplicaSet_,
> helm list
NAME REVISION UPDATED STATUS CHART NAMESPACE
- so 1 Mon Feb 5 10:05:22 2018 DEPLOYED so-2.0.1 default
+ so 1 Mon Feb 5 10:05:22 2020 DEPLOYED so-8.0.0 onap
When upgrading a cluster a parameter controls the minimum size of the cluster
during the upgrade while another parameter controls the maximum number of nodes
For example, to upgrade a container by changing configuration, specifically an
environment value::
- > helm deploy onap onap/so --version 2.0.1 --set enableDebug=true
+ > helm upgrade so onap/so --version 8.0.1 --set enableDebug=true
Issuing this command will result in the appropriate container being stopped by
Kubernetes and replaced with a new container with the new environment value.
To upgrade a component to a new version with a new configuration file enter::
- > helm deploy onbap onap/so --version 2.0.2 -f environments/demo.yaml
+ > helm upgrade so onap/so --version 8.0.1 -f environments/demo.yaml
To fetch release history enter::
> helm history so
REVISION UPDATED STATUS CHART DESCRIPTION
- 1 Mon Feb 5 10:05:22 2018 SUPERSEDED so-2.0.1 Install complete
- 2 Mon Feb 5 10:10:55 2018 DEPLOYED so-2.0.2 Upgrade complete
+ 1 Mon Feb 5 10:05:22 2020 SUPERSEDED so-8.0.0 Install complete
+ 2 Mon Feb 5 10:10:55 2020 DEPLOYED so-8.0.1 Upgrade complete
Unfortunately, not all upgrades are successful. In recognition of this the
lineup of pods within an ONAP deployment is tagged such that an administrator
> helm history so
REVISION UPDATED STATUS CHART DESCRIPTION
- 1 Mon Feb 5 10:05:22 2018 SUPERSEDED so-2.0.1 Install complete
- 2 Mon Feb 5 10:10:55 2018 SUPERSEDED so-2.0.2 Upgrade complete
- 3 Mon Feb 5 10:14:32 2018 DEPLOYED so-2.0.1 Rollback to 1
+ 1 Mon Feb 5 10:05:22 2020 SUPERSEDED so-8.0.0 Install complete
+ 2 Mon Feb 5 10:10:55 2020 SUPERSEDED so-8.0.1 Upgrade complete
+ 3 Mon Feb 5 10:14:32 2020 DEPLOYED so-8.0.0 Rollback to 1
.. note::
Existing deployments can be partially or fully removed once they are no longer
needed. To minimize errors it is recommended that before deleting components
from a running deployment the operator perform a 'dry-run' to display exactly
-what will happen with a given command prior to actually deleting anything. For
-example::
+what will happen with a given command prior to actually deleting anything.
+For example::
> helm undeploy onap --dry-run
deployment.
To completely delete a release and remove it from the internal store enter::
- > helm undeploy onap --purge
+ > helm undeploy onap
+
+Once complete undeploy is done then delete the namespace as well
+using following command::
+
+ > kubectl delete namespace <name of namespace>
+
+.. note::
+ You need to provide the namespace name which you used during deployment,
+ below is the example::
+
+ > kubectl delete namespace onap
One can also remove individual components from a deployment by changing the
ONAP configuration values. For example, to remove `so` from a running
deployment enter::
- > helm undeploy onap-so --purge
+ > helm undeploy onap-so
will remove `so` as the configuration indicates it's no longer part of the
deployment. This might be useful if a one wanted to replace just `so` by
Code Review / oom.git / blobdiff