1 .. This work is licensed under a Creative Commons Attribution 4.0
2 .. International License.
3 .. http://creativecommons.org/licenses/by/4.0
4 .. Copyright (C) 2022 Nordix Foundation
7 .. _HELM Best Practices Guide: https://docs.helm.sh/chart_best_practices/#requirements
8 .. _helm installation guide: https://helm.sh/docs/intro/install/
9 .. _kubectl installation guide: https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/
10 .. _Curated applications for Kubernetes: https://github.com/kubernetes/charts
11 .. _Cert-Manager Installation documentation: https://cert-manager.io/docs/installation/kubernetes/
12 .. _Cert-Manager kubectl plugin documentation: https://cert-manager.io/docs/usage/kubectl-plugin/
13 .. _Strimzi Apache Kafka Operator helm Installation documentation: https://strimzi.io/docs/operators/in-development/deploying.html#deploying-cluster-operator-helm-chart-str
14 .. _ONAP Next Generation Security & Logging Structure: https://wiki.onap.org/pages/viewpage.action?pageId=103417456
15 .. _Istio setup guide: https://istio.io/latest/docs/setup/install/helm/
16 .. _Gateway-API: https://gateway-api.sigs.k8s.io/
17 .. _Istio-Gateway: https://istio.io/latest/docs/reference/config/networking/gateway/
18 .. _DefaultStorageClass: https://kubernetes.io/docs/tasks/administer-cluster/change-default-storage-class/
20 .. _oom_base_setup_guide:
25 As part of the initial base setup of the host Kubernetes cluster,
26 the following mandatory installation and configuration steps must be completed.
34 For additional platform add-ons, see the :ref:`oom_base_optional_addons` section.
36 Install & configure kubectl
37 ---------------------------
39 The Kubernetes command line interface used to manage a Kubernetes cluster needs to be installed
40 and configured to run as non root.
42 For additional information regarding kubectl installation and configuration see the `kubectl installation guide`_
44 To install kubectl, execute the following, replacing the <recommended-kubectl-version> with the version defined
45 in the :ref:`versions_table` table::
47 > curl -LO https://dl.k8s.io/release/v<recommended-kubectl-version>/bin/linux/amd64/kubectl
51 > sudo mv ./kubectl /usr/local/bin/kubectl
55 > cp kube_config_cluster.yml ~/.kube/config.onap
57 > export KUBECONFIG=~/.kube/config.onap
59 > kubectl config use-context onap
61 Validate the installation::
67 NAME STATUS ROLES AGE VERSION
68 onap-control-1 Ready controlplane,etcd 3h53m v1.27.5
69 onap-control-2 Ready controlplane,etcd 3h53m v1.27.5
70 onap-k8s-1 Ready worker 3h53m v1.27.5
71 onap-k8s-2 Ready worker 3h53m v1.27.5
72 onap-k8s-3 Ready worker 3h53m v1.27.5
73 onap-k8s-4 Ready worker 3h53m v1.27.5
74 onap-k8s-5 Ready worker 3h53m v1.27.5
75 onap-k8s-6 Ready worker 3h53m v1.27.5
78 Install & configure helm
79 ------------------------
81 Helm is used for package and configuration management of the relevant helm charts.
82 For additional information, see the `helm installation guide`_
84 To install helm, execute the following, replacing the <recommended-helm-version> with the version defined
85 in the :ref:`versions_table` table::
87 > wget https://get.helm.sh/helm-v<recommended-helm-version>-linux-amd64.tar.gz
89 > tar -zxvf helm-v<recommended-helm-version>-linux-amd64.tar.gz
91 > sudo mv linux-amd64/helm /usr/local/bin/helm
93 Verify the helm version with::
97 Helm's default CNCF provided `Curated applications for Kubernetes`_ repository called
98 *stable* can be removed to avoid confusion::
100 > helm repo remove stable
102 Install the additional OOM plugins required to un/deploy the OOM helm charts::
104 > git clone http://gerrit.onap.org/r/oom
106 > helm plugin install ~/oom/kubernetes/helm/plugins/deploy
108 > helm plugin install ~/oom/kubernetes/helm/plugins/undeploy
110 Verify the plugins are installed::
116 NAME VERSION DESCRIPTION
117 deploy 1.0.0 install (upgrade if release exists) parent chart and all subcharts as separate but related releases
118 undeploy 1.0.0 delete parent chart and subcharts that were deployed as separate releases
120 Set the default StorageClass
121 ----------------------------
123 In some ONAP components it is important to have a default storageClass defined (e.g. cassandra),
124 if you don't want to explicitly set it during the deployment via helm overrides.
126 Therefor you should set the default storageClass (if not done during the K8S cluster setup) via the command::
128 > kubectl patch storageclass <storageclass> -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
130 see `DefaultStorageClass`_
132 Install the Strimzi Kafka Operator
133 ----------------------------------
135 Strimzi Apache Kafka provides a way to run an Apache Kafka cluster on Kubernetes
136 in various deployment configurations by using kubernetes operators.
137 Operators are a method of packaging, deploying, and managing Kubernetes applications.
139 Strimzi Operators extend the Kubernetes functionality, automating common
140 and complex tasks related to a Kafka deployment. By implementing
141 knowledge of Kafka operations in code, the Kafka administration
142 tasks are simplified and require less manual intervention.
144 The Strimzi cluster operator is deployed using helm to install the parent chart
145 containing all of the required custom resource definitions. This should be done
146 by a kubernetes administrator to allow for deployment of custom resources in to
147 any kubernetes namespace within the cluster.
149 Full installation instructions can be found in the
150 `Strimzi Apache Kafka Operator helm Installation documentation`_.
152 To add the required helm repository, execute the following::
154 > helm repo add strimzi https://strimzi.io/charts/
156 To install the strimzi kafka operator, execute the following, replacing the <recommended-strimzi-version> with the version defined
157 in the :ref:`versions_table` table::
159 > helm install strimzi-kafka-operator strimzi/strimzi-kafka-operator --namespace strimzi-system --version <recommended-strimzi-version> --set watchAnyNamespace=true --create-namespace
161 Verify the installation::
163 > kubectl get po -n strimzi-system
167 NAME READY STATUS RESTARTS AGE
168 strimzi-cluster-operator-7f7d6b46cf-mnpjr 1/1 Running 0 2m
171 .. _oom_base_setup_cert_manager:
176 Cert-Manager is a native Kubernetes certificate management controller.
177 It can help with issuing certificates from a variety of sources, such as
178 Let’s Encrypt, HashiCorp Vault, Venafi, a simple signing key pair, self
179 signed or external issuers. It ensures certificates are valid and up to
180 date, and attempt to renew certificates at a configured time before expiry.
182 Cert-Manager is deployed using regular YAML manifests which include all
183 the needed resources (the CustomResourceDefinitions, cert-manager,
184 namespace, and the webhook component).
186 Full installation instructions, including details on how to configure extra
187 functionality in Cert-Manager can be found in the
188 `Cert-Manager Installation documentation`_.
190 There is also a kubectl plugin (kubectl cert-manager) that can help you
191 to manage cert-manager resources inside your cluster. For installation
192 steps, please refer to `Cert-Manager kubectl plugin documentation`_.
195 To install cert-manager, execute the following, replacing the <recommended-cm-version> with the version defined
196 in the :ref:`versions_table` table::
198 > kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v<recommended-cm-version>/cert-manager.yaml
200 Verify the installation::
202 > kubectl get po -n cert-manager
206 NAME READY STATUS RESTARTS AGE
207 cert-manager-776c4cfcb6-vgnpw 1/1 Running 0 2m
208 cert-manager-cainjector-7d9668978d-hdxf7 1/1 Running 0 2m
209 cert-manager-webhook-66c8f6c75-dxmtz 1/1 Running 0 2m
215 The ONAP deployment supports the
216 `ONAP Next Generation Security & Logging Structure`_
218 ONAP is currenty supporting Istio as default ServiceMesh platform.
219 Therefor the following instructions describe the setup of Istio and required tools.
220 Used `Istio setup guide`_
222 .. _oom_base_optional_addons_istio_installation:
224 Istio Platform Installation
225 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
227 Install Istio Basic Platform
228 """"""""""""""""""""""""""""
230 - Configure the Helm repository::
232 > helm repo add istio https://istio-release.storage.googleapis.com/charts
236 - Create a namespace for "mesh-level" configurations::
238 > kubectl create namespace istio-config
240 - Create a namespace istio-system for Istio components::
242 > kubectl create namespace istio-system
244 - Install the Istio Base chart which contains cluster-wide resources used by the
245 Istio control plane, replacing the <recommended-istio-version> with the version
246 defined in the :ref:`versions_table` table::
248 > helm upgrade -i istio-base istio/base -n istio-system --version <recommended-istio-version>
250 - Create an override for istiod (e.g. istiod.yaml) to add the oauth2-proxy as external
251 authentication provider and apply some specific config settings
253 .. collapse:: istiod.yaml
255 .. include:: ../../resources/yaml/istiod.yaml
258 - Install the Istio Base Istio Discovery chart which deploys the istiod service, replacing the
259 <recommended-istio-version> with the version defined in the :ref:`versions_table` table::
261 > helm upgrade -i istiod istio/istiod -n istio-system --version <recommended-istio-version>
262 --wait -f ./istiod.yaml
264 Add an EnvoyFilter for HTTP header case
265 """""""""""""""""""""""""""""""""""""""
267 When handling HTTP/1.1, Envoy will normalize the header keys to be all
268 lowercase. While this is compliant with the HTTP/1.1 spec, in practice this
269 can result in issues when migrating existing systems that might rely on
270 specific header casing. In our case a problem was detected in the SDC client
271 implementation, which relies on uppercase header values. To solve this problem
272 in general we add a EnvoyFilter to keep the uppercase header in the
273 istio-config namespace to apply for all namespaces, but set the context to
274 SIDECAR_INBOUND to avoid problems in the connection between Istio-Gateway and
277 - Create a EnvoyFilter file (e.g. envoyfilter-case.yaml)
279 .. collapse:: envoyfilter-case.yaml
281 .. include:: ../../resources/yaml/envoyfilter-case.yaml
284 - Apply the change to Istio::
286 > kubectl apply -f envoyfilter-case.yaml
289 Ingress Controller Installation
290 -------------------------------
292 In the production setup 2 different Ingress setups are supported.
294 - Gateway API `Gateway-API`_ (recommended)
295 - Istio Gateway `Istio-Gateway`_ (alternative, but in the future deprecated)
297 Depending on the solution, the ONAP helm values.yaml has to be configured.
298 See the :ref:`OOM customized deployment<oom_customize_overrides>` section for more details.
300 Gateway-API (recommended)
301 ^^^^^^^^^^^^^^^^^^^^^^^^^
303 - Install the Gateway-API CRDs replacing the
304 <recommended-gwapi-version> with the version defined in
305 the :ref:`versions_table` table::
307 > kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/<recommended-gwapi-version>/experimental-install.yaml
309 - Create a common Gateway instance named "common-gateway"
310 The following example uses provides listeners for HTTP(s), UDP and TCP
312 .. collapse:: common-gateway.yaml
314 .. include:: ../../resources/yaml/common-gateway.yaml
319 > kubectl apply -f common-gateway.yaml
321 Istio Gateway (alternative)
322 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
324 - Create a namespace istio-ingress for the Istio Ingress gateway
325 and enable istio-injection::
327 > kubectl create namespace istio-ingress
329 > kubectl label namespace istio-ingress istio-injection=enabled
331 - To expose additional ports besides HTTP/S (e.g. for external Kafka access, SDNC-callhome)
332 create an override file (e.g. istio-ingress.yaml)
334 .. collapse:: istio-ingress.yaml
336 .. include:: ../../resources/yaml/istio-ingress.yaml
339 - Install the Istio Gateway chart using the override file, replacing the
340 <recommended-istio-version> with the version defined in
341 the :ref:`versions_table` table::
343 > helm upgrade -i istio-ingress istio/gateway -n istio-ingress
344 --version <recommended-istio-version> -f ingress-istio.yaml --wait
347 Keycloak Installation
348 ---------------------
350 - Add helm repositories::
352 > helm repo add bitnami https://charts.bitnami.com/bitnami
354 > helm repo add codecentric https://codecentric.github.io/helm-charts
358 - create keycloak namespace::
360 > kubectl create namespace keycloak
361 > kubectl label namespace keycloak istio-injection=disabled
363 Install Keycloak-Database
364 ^^^^^^^^^^^^^^^^^^^^^^^^^
366 - To configure the Postgres DB
367 create an override file (e.g. keycloak-db-values.yaml)
369 .. collapse:: keycloak-db-values.yaml
371 .. include:: ../../resources/yaml/keycloak-db-values.yaml
374 - Install the Postgres DB::
376 > helm -n keycloak upgrade -i keycloak-db bitnami/postgresql --values ./keycloak-db-values.yaml
381 - To configure the Keycloak instance
382 create an override file (e.g. keycloak-server-values.yaml)
384 .. collapse:: keycloak-server-values.yaml
386 .. include:: ../../resources/yaml/keycloak-server-values.yaml
391 > helm -n keycloak upgrade -i keycloak codecentric/keycloakx --values ./keycloak-server-values.yaml
393 The required Ingress entry and REALM will be provided by the ONAP "Platform"
396 - Create Ingress gateway entry for the keycloak web interface
397 using the configured Ingress <base-url> (here "simpledemo.onap.org")
398 as described in :ref:`oom_customize_overrides`
400 .. collapse:: keycloak-ingress.yaml
402 .. include:: ../../resources/yaml/keycloak-ingress.yaml
405 - Add the Ingress entry for Keycloak::
407 > kubectl -n keycloak apply -f keycloak-ingress.yaml