1 .. This work is licensed under a Creative Commons Attribution 4.0
2 .. International License.
3 .. http://creativecommons.org/licenses/by/4.0
4 .. Copyright (C) 2022 Nordix Foundation
7 .. _HELM Best Practices Guide: https://docs.helm.sh/chart_best_practices/#requirements
8 .. _helm installation guide: https://helm.sh/docs/intro/install/
9 .. _kubectl installation guide: https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/
10 .. _Curated applications for Kubernetes: https://github.com/kubernetes/charts
11 .. _Cert-Manager Installation documentation: https://cert-manager.io/docs/installation/kubernetes/
12 .. _Cert-Manager kubectl plugin documentation: https://cert-manager.io/docs/usage/kubectl-plugin/
13 .. _Strimzi Apache Kafka Operator helm Installation documentation: https://strimzi.io/docs/operators/in-development/deploying.html#deploying-cluster-operator-helm-chart-str
14 .. _ONAP Next Generation Security & Logging Structure: https://wiki.onap.org/pages/viewpage.action?pageId=103417456
15 .. _Istio setup guide: https://istio.io/latest/docs/setup/install/helm/
16 .. _Gateway-API: https://gateway-api.sigs.k8s.io/
17 .. _Istio-Gateway: https://istio.io/latest/docs/reference/config/networking/gateway/
18 .. _DefaultStorageClass: https://kubernetes.io/docs/tasks/administer-cluster/change-default-storage-class/
20 .. _oom_base_setup_guide:
25 As part of the initial base setup of the host Kubernetes cluster,
26 the following mandatory installation and configuration steps must be completed.
34 For additional platform add-ons, see the :ref:`oom_base_optional_addons` section.
36 Install & configure kubectl
37 ---------------------------
39 The Kubernetes command line interface used to manage a Kubernetes cluster needs to be installed
40 and configured to run as non root.
42 For additional information regarding kubectl installation and configuration see the `kubectl installation guide`_
44 To install kubectl, execute the following, replacing the <recommended-kubectl-version> with the version defined
45 in the :ref:`versions_table` table::
47 > curl -LO https://dl.k8s.io/release/v<recommended-kubectl-version>/bin/linux/amd64/kubectl
51 > sudo mv ./kubectl /usr/local/bin/kubectl
55 > cp kube_config_cluster.yml ~/.kube/config.onap
57 > export KUBECONFIG=~/.kube/config.onap
59 > kubectl config use-context onap
61 Validate the installation::
67 NAME STATUS ROLES AGE VERSION
68 onap-control-1 Ready controlplane,etcd 3h53m v1.23.8
69 onap-control-2 Ready controlplane,etcd 3h53m v1.23.8
70 onap-k8s-1 Ready worker 3h53m v1.23.8
71 onap-k8s-2 Ready worker 3h53m v1.23.8
72 onap-k8s-3 Ready worker 3h53m v1.23.8
73 onap-k8s-4 Ready worker 3h53m v1.23.8
74 onap-k8s-5 Ready worker 3h53m v1.23.8
75 onap-k8s-6 Ready worker 3h53m v1.23.8
78 Install & configure helm
79 ------------------------
81 Helm is used for package and configuration management of the relevant helm charts.
82 For additional information, see the `helm installation guide`_
84 To install helm, execute the following, replacing the <recommended-helm-version> with the version defined
85 in the :ref:`versions_table` table::
87 > wget https://get.helm.sh/helm-v<recommended-helm-version>-linux-amd64.tar.gz
89 > tar -zxvf helm-v<recommended-helm-version>-linux-amd64.tar.gz
91 > sudo mv linux-amd64/helm /usr/local/bin/helm
93 Verify the helm version with::
97 Helm's default CNCF provided `Curated applications for Kubernetes`_ repository called
98 *stable* can be removed to avoid confusion::
100 > helm repo remove stable
102 Install the additional OOM plugins required to un/deploy the OOM helm charts::
104 > git clone http://gerrit.onap.org/r/oom
106 > helm plugin install ~/oom/kubernetes/helm/plugins/deploy
108 > helm plugin install ~/oom/kubernetes/helm/plugins/undeploy
110 Verify the plugins are installed::
116 NAME VERSION DESCRIPTION
117 deploy 1.0.0 install (upgrade if release exists) parent chart and all subcharts as separate but related releases
118 undeploy 1.0.0 delete parent chart and subcharts that were deployed as separate releases
120 Set the default StorageClass
121 ----------------------------
123 In some ONAP components it is important to have a default storageClass defined (e.g. cassandra),
124 if you don't want to explicitly set it during the deployment via helm overrides.
126 Therefor you should set the default storageClass (if not done during the K8S cluster setup) via the command:
128 > kubectl patch storageclass <storageclass> -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
130 see `DefaultStorageClass`_
132 Install the Strimzi Kafka Operator
133 ----------------------------------
135 Strimzi Apache Kafka provides a way to run an Apache Kafka cluster on Kubernetes
136 in various deployment configurations by using kubernetes operators.
137 Operators are a method of packaging, deploying, and managing Kubernetes applications.
139 Strimzi Operators extend the Kubernetes functionality, automating common
140 and complex tasks related to a Kafka deployment. By implementing
141 knowledge of Kafka operations in code, the Kafka administration
142 tasks are simplified and require less manual intervention.
144 The Strimzi cluster operator is deployed using helm to install the parent chart
145 containing all of the required custom resource definitions. This should be done
146 by a kubernetes administrator to allow for deployment of custom resources in to
147 any kubernetes namespace within the cluster.
149 Full installation instructions can be found in the
150 `Strimzi Apache Kafka Operator helm Installation documentation`_.
152 To add the required helm repository, execute the following::
154 > helm repo add strimzi https://strimzi.io/charts/
156 To install the strimzi kafka operator, execute the following, replacing the <recommended-strimzi-version> with the version defined
157 in the :ref:`versions_table` table::
159 > helm install strimzi-kafka-operator strimzi/strimzi-kafka-operator --namespace strimzi-system --version <recommended-strimzi-version> --set watchAnyNamespace=true --create-namespace
161 Verify the installation::
163 > kubectl get po -n strimzi-system
167 NAME READY STATUS RESTARTS AGE
168 strimzi-cluster-operator-7f7d6b46cf-mnpjr 1/1 Running 0 2m
171 .. _oom_base_setup_cert_manager:
176 Cert-Manager is a native Kubernetes certificate management controller.
177 It can help with issuing certificates from a variety of sources, such as
178 Let’s Encrypt, HashiCorp Vault, Venafi, a simple signing key pair, self
179 signed or external issuers. It ensures certificates are valid and up to
180 date, and attempt to renew certificates at a configured time before expiry.
182 Cert-Manager is deployed using regular YAML manifests which include all
183 the needed resources (the CustomResourceDefinitions, cert-manager,
184 namespace, and the webhook component).
186 Full installation instructions, including details on how to configure extra
187 functionality in Cert-Manager can be found in the
188 `Cert-Manager Installation documentation`_.
190 There is also a kubectl plugin (kubectl cert-manager) that can help you
191 to manage cert-manager resources inside your cluster. For installation
192 steps, please refer to `Cert-Manager kubectl plugin documentation`_.
195 To install cert-manager, execute the following, replacing the <recommended-cm-version> with the version defined
196 in the :ref:`versions_table` table::
198 > kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v<recommended-cm-version>/cert-manager.yaml
200 Verify the installation::
202 > kubectl get po -n cert-manager
206 NAME READY STATUS RESTARTS AGE
207 cert-manager-776c4cfcb6-vgnpw 1/1 Running 0 2m
208 cert-manager-cainjector-7d9668978d-hdxf7 1/1 Running 0 2m
209 cert-manager-webhook-66c8f6c75-dxmtz 1/1 Running 0 2m
215 In London ONAP deployment supports the
216 `ONAP Next Generation Security & Logging Structure`_
218 ONAP is currenty supporting Istio as default ServiceMesh platform.
219 Therefor the following instructions describe the setup of Istio and required tools.
220 Used `Istio setup guide`_
222 .. _oom_base_optional_addons_istio_installation:
224 Istio Platform Installation
225 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
227 Install Istio Basic Platform
228 """"""""""""""""""""""""""""
230 - Configure the Helm repository::
232 > helm repo add istio https://istio-release.storage.googleapis.com/charts
236 - Create a namespace for "mesh-level" configurations::
238 > kubectl create namespace istio-config
240 - Create a namespace istio-system for Istio components::
242 > kubectl create namespace istio-system
244 - Install the Istio Base chart which contains cluster-wide resources used by the
245 Istio control plane, replacing the <recommended-istio-version> with the version
246 defined in the :ref:`versions_table` table::
248 > helm upgrade -i istio-base istio/base -n istio-system --version <recommended-istio-version>
250 - Create an override for istiod (e.g. istiod.yaml) to add the oauth2-proxy as external
251 authentication provider and apply some specific config settings
253 .. collapse:: istiod.yaml
255 .. include:: ../../resources/yaml/istiod.yaml
258 - Install the Istio Base Istio Discovery chart which deploys the istiod service, replacing the
259 <recommended-istio-version> with the version defined in the :ref:`versions_table` table::
261 > helm upgrade -i istiod istio/istiod -n istio-system --version <recommended-istio-version>
262 --wait -f ./istiod.yaml
264 Add an EnvoyFilter for HTTP header case
265 """""""""""""""""""""""""""""""""""""""
267 When handling HTTP/1.1, Envoy will normalize the header keys to be all
268 lowercase. While this is compliant with the HTTP/1.1 spec, in practice this
269 can result in issues when migrating existing systems that might rely on
270 specific header casing. In our case a problem was detected in the SDC client
271 implementation, which relies on uppercase header values. To solve this problem
272 in general we add a EnvoyFilter to keep the uppercase header in the
273 istio-config namespace to apply for all namespaces, but set the context to
274 SIDECAR_INBOUND to avoid problems in the connection between Istio-Gateway and
277 - Create a EnvoyFilter file (e.g. envoyfilter-case.yaml)
279 .. collapse:: envoyfilter-case.yaml
281 .. include:: ../../resources/yaml/envoyfilter-case.yaml
284 - Apply the change to Istio::
286 > kubectl apply -f envoyfilter-case.yaml
289 Ingress Controller Installation
290 -------------------------------
292 In the production setup 2 different Ingress setups are supported.
294 - Istio Gateway `Istio-Gateway`_ (currently tested, but in the future deprecated)
295 - Gateway API `Gateway-API`_ (in Alpha status, but will be standard in the future)
297 Depending on the solution, the ONAP helm values.yaml has to be configured.
298 See the :ref:`OOM customized deployment<oom_customize_overrides>` section for more details.
303 - Create a namespace istio-ingress for the Istio Ingress gateway
304 and enable istio-injection::
306 > kubectl create namespace istio-ingress
308 > kubectl label namespace istio-ingress istio-injection=enabled
310 - To expose additional ports besides HTTP/S (e.g. for external Kafka access, SDNC-callhome)
311 create an override file (e.g. istio-ingress.yaml)
313 .. collapse:: istio-ingress.yaml
315 .. include:: ../../resources/yaml/istio-ingress.yaml
318 - Install the Istio Gateway chart using the override file, replacing the
319 <recommended-istio-version> with the version defined in
320 the :ref:`versions_table` table::
322 > helm upgrade -i istio-ingress istio/gateway -n istio-ingress
323 --version <recommended-istio-version> -f ingress-istio.yaml --wait
329 - Install the Gateway-API CRDs replacing the
330 <recommended-gwapi-version> with the version defined in
331 the :ref:`versions_table` table::
333 > kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/<recommended-gwapi-version>/experimental-install.yaml
335 - Create a common Gateway instance
338 Keycloak Installation
339 ---------------------
341 - Add helm repositories
343 > helm repo add bitnami https://charts.bitnami.com/bitnami
345 > helm repo add codecentric https://codecentric.github.io/helm-charts
349 - create keycloak namespace
351 > kubectl create namespace keycloak
352 > kubectl label namespace keycloak istio-injection=enabled
354 Install Keycloak-Database
355 ^^^^^^^^^^^^^^^^^^^^^^^^^
357 - To configure the Postgres DB
358 create an override file (e.g. keycloak-db-values.yaml)
360 .. collapse:: keycloak-db-values.yaml
362 .. include:: ../../resources/yaml/keycloak-db-values.yaml
365 - Install the Postgres DB
367 > helm -n keycloak upgrade -i keycloak-db bitnami/postgresql --values ./keycloak-db-values.yaml
372 - To configure the Keycloak instance
373 create an override file (e.g. keycloak-server-values.yaml)
375 .. collapse:: keycloak-server-values.yaml
377 .. include:: ../../resources/yaml/keycloak-server-values.yaml
382 > helm -n keycloak upgrade -i keycloak codecentric/keycloak --values ./keycloak-server-values.yaml
384 The required Ingress entry and REALM will be provided by the ONAP "Platform"