1 .. This work is licensed under a Creative Commons Attribution 4.0
2 .. International License.
3 .. http://creativecommons.org/licenses/by/4.0
4 .. Copyright 2020, Samsung Electronics
7 .. _HELM Best Practices Guide: https://docs.helm.sh/chart_best_practices/#requirements
8 .. _kubectl Cheat Sheet: https://kubernetes.io/docs/reference/kubectl/cheatsheet/
9 .. _Kubernetes documentation for emptyDir: https://kubernetes.io/docs/concepts/storage/volumes/#emptydir
10 .. _metallb Metal Load Balancer installation: https://metallb.universe.tf/installation/
11 .. _http://cd.onap.info:30223/mso/logging/debug: http://cd.onap.info:30223/mso/logging/debug
12 .. _Onboarding and Distributing a Vendor Software Product: https://wiki.onap.org/pages/viewpage.action?pageId=1018474
13 .. _README.md: https://gerrit.onap.org/r/gitweb?p=oom.git;a=blob;f=kubernetes/README.md
15 .. figure:: images/oom_logo/oomLogoV2-medium.png
18 .. _oom_setup_ingress_controller:
21 Ingress controller setup on HA Kubernetes Cluster
22 #################################################
24 This guide provides instruction how to setup experimental ingress controller
25 feature. For this, we are hosting our cluster on OpenStack VMs and using the
26 Rancher Kubernetes Engine (RKE) to deploy and manage our Kubernetes Cluster and
34 The result at the end of this tutorial will be:
36 #. Customization of the cluster.yaml file for ingress controller support
38 #. Installation and configuration test DNS server for ingress host resolution
41 #. Installation and configuration MLB (Metal Load Balancer) required for
42 exposing ingress service
44 #. Installation and configuration NGINX ingress controller
46 #. Additional info how to deploy ONAP with services exposed via Ingress
49 Customize cluster.yml file
50 ==========================
51 Before setup cluster for ingress purposes DNS cluster IP and ingress provider
52 should be configured and following:
66 - <custer_dns_ip>:31555
68 Where the <cluster_dns_ip> should be set to the same IP as the CONTROLPANE
71 For external load balancer purposes, minimum one of the worker node should be
72 configured with external IP address accessible outside the cluster. It can be
73 done using the following example node configuration:
79 - address: <external_ip>
80 internal_address: <internal_ip>
84 hostname_override: "onap-worker-0"
86 ssh_key_path: "~/.ssh/id_rsa"
89 Where the <external_ip> is external worker node IP address, and <internal_ip>
90 is internal node IP address if it is required.
93 DNS server configuration and installation
94 =========================================
95 DNS server deployed on the Kubernetes cluster makes it easy to use services
96 exposed through ingress controller because it resolves all subdomain related to
97 the ONAP cluster to the load balancer IP. Testing ONAP cluster requires a lot
98 of entries on the target machines in the /etc/hosts. Adding many entries into
99 the configuration files on testing machines is quite problematic and error
100 prone. The better wait is to create central DNS server with entries for all
101 virtual host pointed to simpledemo.onap.org and add custom DNS server as a
102 target DNS server for testing machines and/or as external DNS for Kubernetes
105 DNS server has automatic installation and configuration script, so installation
108 > cd kubernetes/contrib/dns-server-for-vhost-ingress-testing
112 After DNS deploy you need to setup DNS entry on the target testing machine.
113 Because DNS listen on non standard port configuration require iptables rules
114 on the target machine. Please follow the configuration proposed by the deploy
116 Example output depends on the IP address and example output looks like bellow::
118 DNS server already deployed:
119 1. You can add the DNS server to the target machine using following commands:
120 sudo iptables -t nat -A OUTPUT -p tcp -d 192.168.211.211 --dport 53 -j DNAT --to-destination 10.10.13.14:31555
121 sudo iptables -t nat -A OUTPUT -p udp -d 192.168.211.211 --dport 53 -j DNAT --to-destination 10.10.13.14:31555
122 sudo sysctl -w net.ipv4.conf.all.route_localnet=1
123 sudo sysctl -w net.ipv4.ip_forward=1
124 2. Update /etc/resolv.conf file with nameserver 192.168.211.211 entry on your target machine
127 MetalLB Load Balancer installation and configuration
128 ====================================================
130 By default pure Kubernetes cluster requires external load balancer if we want
131 to expose external port using LoadBalancer settings. For this purpose MetalLB
132 can be used. Before installing the MetalLB you need to ensure that at least one
133 worker has assigned IP accessible outside the cluster.
135 MetalLB Load balancer can be easily installed using automatic install script::
137 > cd kubernetes/contrib/metallb-loadbalancer-inst
139 > ./install-metallb-on-cluster.sh
142 Configuration Nginx ingress controller
143 ======================================
145 After installation DNS server and ingress controller we can install and
146 configure ingress controller.
147 It can be done using the following commands::
149 > cd kubernetes/contrib/ingress-nginx-post-inst
151 > kubectl apply -f nginx_ingress_cluster_config.yaml
153 > kubectl apply -f nginx_ingress_enable_optional_load_balacer_service.yaml
155 After deploy NGINX ingress controller you can ensure that the ingress port is
156 exposed as load balancer service with external IP address::
158 > kubectl get svc -n ingress-nginx
159 NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
160 default-http-backend ClusterIP 10.10.10.10 <none> 80/TCP 25h
161 ingress-nginx LoadBalancer 10.10.10.11 10.12.13.14 80:31308/TCP,443:30314/TCP 24h
164 ONAP with ingress exposed services
165 ==================================
166 If you want to deploy onap with services exposed through ingress controller you
167 can use full onap deploy script::
169 > onap/resources/overrides/onap-all-ingress-nginx-vhost.yaml
171 Ingress also can be enabled on any onap setup override using following code: