5 Copyright 2019 AT&T Intellectual Property. All rights reserved.
7 This file is licensed under the CREATIVE COMMONS ATTRIBUTION 4.0 INTERNATIONAL LICENSE
9 Full license text at <https://creativecommons.org/licenses/by/4.0/legalcode>
13 ONAP on AKS will orchestrate an Azure Kubernetes Service (AKS) deployment, a DevStack deployment, an ONAP + NFS deployment, as well as configuration to link the Azure resources together. After ONAP is installed, a cloud region will also be added to ONAP with the new DevStack details that can be used to instantiate a VNF.
17 The following software is required to be installed:
20 - [helm](https://helm.sh/docs/using_helm/)
21 - [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/)
22 - [azure command line](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli-apt?view=azure-cli-latest)
23 - make, openjdk-8-jdk, openjdk-8-jre (`apt-get update && apt-get install make openjdk-8-jre openjdk-8-jdk`)
25 Check the [OOM Cloud Setup Guide](https://docs.onap.org/en/latest/submodules/oom.git/docs/oom_cloud_setup_guide.html#cloud-setup-guide-label) for the versions of kubectl and helm to use.
27 After installing the above software, run `az login` and follow the instructions to finalize the azure command line installation. **You'll need to be either an owner or co-owner of the azure subscription, or some of the deployment steps may not complete successfully**. If you have multiple azure subscriptions, use `az account set --subscription <subscription name>` prior to running `az login` so that resources are deployed to the correct subscription. See [the azure docs](https://docs.microsoft.com/en-us/cli/azure/get-started-with-azure-cli?view=azure-cli-latest) for more details on using the azure command line.
29 ### The following resources will be created in Azure
31 - Kubernetes cluster via AKS (Azure Kubernetes Service)
32 - VM running NFS server application
33 - VM running latest DevStack version
39 `cloud.sh` is the main driver script, and deploys a Kubernetes Cluster (AKS), DevStack, NFS, and bootstraps ONAP with configuration needed to instantiate a VNF. The script creates ONAP in "about" an hour.
48 -f, --no-prompt executes with no prompt for confirmation
49 -n, --no-install don't install ONAP
50 -o, --override create integration override for robot configuration
51 -d, --no-validate dont validate pre-reqs before executing deployment
52 -p, --post-install execute post-install scripts
53 -h, --help provide brief overview of script
55 This script deploys a cloud environment in Azure.
57 - Uses Azure Kubernetes Service (AKS) to bootstrap a kubernetes cluster.
58 - Creates a VM to be used as NFS storage.
59 - Creates a VM and installs DevStack, to be used with ONAP.
60 - Creates an openstack cli pod that can be used for cli access to devstack
61 - Creates an integration-override.yaml file to configure robot
62 - Launches ONAP onto the AKS Cluster via OOM.
63 - Configures Networking, SSH Access, and Security Group Rules
70 $ ./cloud.sh --override
75 This file contains the parameters that will be used when executing `cloud.sh`. The parameter `BUILD` will be generated at runtime.
77 For an example with all of the parameters filled out, check [here](./cloud.conf.example). You can copy this and modify to suit your deployment. The parameters that MUST be modified from `cloud.conf.example` are `USER_PUBLIC_IP_PREFIX` and `BUILD_DIR`.
79 All other parameters will work out of the box, however you can also customize them to suit your own deployment. See below for a description of the available parameters and how they're used.
83 # The variable $BUILD will be generated dynamically when this file is sourced
85 RANDOM_STRING=`cat /dev/urandom | env LC_CTYPE=C tr -cd 'a-zA-Z0-9' | head -c 4`
86 BUILD= This is just a helper variable to create a random string to assign to various resources. Look at cloud.conf.example to see how it can be used.
89 LOCATION= Location in Azure to deploy resources
90 USER_PUBLIC_IP_PREFIX= Space delimited list of ip addresses/CIDR blocks that will be added to azure secuirty groups for access. Add the CIDR blocks to grant access for ssh, ONAP portal, and DevStack horizon access.
91 BUILD_DIR= /path/to/directory where build files, artifacts, and other files will be created.
94 AKS_RESOURCE_GROUP_NAME= Name of resource group in azure that will be created for the AKS resource. Must not already exist.
95 AKS_NAME= Name of AKS resource.
96 AKS_K8_VERSION= Kubernetes version, use az aks get-versions --location <location> to see available versions.
97 AKS_NODE_COUNT= Number of nodes that will comprise the AKS cluster.
98 AKS_NODE_SIZE= Flavor to use for AKS nodes.
99 AKS_VNET_NAME= Name of VNET that AKS nodes will attach to.
100 AKS_DNS_PREFIX= DNS prefix that will be used by kubernetes dns service.
101 AKS_POD_CIDR= CIDR used for pod ip allocation.
102 AKS_NODE_CIDR= CIDR used for node ip allocation.
103 AKS_SERVICE_CIDR= CIDR used for kubernetes service allocation.
104 AKS_DNS_IP= IP address to assign to kubernetes dns service. Should be from AKS_SERVICE_CIDR range.
105 AKS_ADMIN_USER= User name that will be created on AKS nodes. Use this user to ssh into AKS nodes if needed.
108 NFS_NAME= Name of NFS VM created in Azure.
109 NFS_RG= Name of resource group that will be created in Azure for the NFS VM. Must not already exist.
110 NFS_VM_SIZE= Flavor to use for NFS VM.
111 NFS_LOCATION= Azure location to deploy NFS VM.
112 NFS_CIDR= CIDR for NFS VNET.
113 NFS_ADMIN_USER= User name that will be created on NFS VM. Use this to ssh to NFS VM if needed.
114 NFS_VNET_NAME= Name of VNET that NFS VM will attach to.
115 NFS_SUBNET_NAME= Name of SUBNET attached to NFS_VNET_NAME.
116 NFS_DISK_SIZE= Size of OS Disk for NFS VM.
119 DEVSTACK_NAME= Name of DevStack VM created in Azure.
120 DEVSTACK_RG= Name of resource group that will be created in Azure for the DevStack VM. Must not already exist.
121 DEVSTACK_VM_SIZE= Flavor to use for DevStack VM.
122 DEVSTACK_LOCATION= Azure location to deploy DevStack VM.
123 DEVSTACK_CIDR= CIDR for DevStack VNET.
124 DEVSTACK_PRIVATE_IP= IP to allocate to DevStack VM. This should be from DEVSTACK_CIDR range, and will be used to communicate with DevStack from ONAP.
125 DEVSTACK_ADMIN_USER= User name that will be created on DevStack VM. Use this to ssh to DevStack VM if needed.
126 DEVSTACK_VNET_NAME= Name of VNET that DevStack VM will attach to.
127 DEVSTACK_SUBNET_NAME= Name of SUBNET attached to DEVSTACK_VNET_NAME.
128 DEVSTACK_DISK_SIZE= Size of OS Disk for DevStack VM.
129 DEVSTACK_BRANCH= Branch to use when installing devstack.
130 OPENSTACK_USER= User name that will be added to OpenStack after devstack has finished installing. This is also the username that will be used to create a cloud site in ONAP SO.
131 OPENSTACK_PASS= Password to use for OPENSTACK_USER.
132 OPENSTACK_TENANT= Tenant name that will be added to OpenStack after devstack has finished installing. This is also the username that will be used to create a cloud site in ONAP SO.
133 OPENSTACK_REGION= Only allows RegionOne for now, future enhancements will be added to allow multi-region.
134 IMAGE_LIST= Space delimited list of image urls to add to DevStack. Not required.
137 CLLI= Name of CLLI to be created in AAI.
138 CLOUD_OWNER= Name of Cloud Owner to be created in AAI.
139 CLOUD_REGION= Name of Cloud Region to be created in AAI.
140 CUSTOMER= Name of Customer to be created in AAI.
141 SUBSCRIBER= Name of Subscriber to be created in AAI.
142 SERVICE_TYPE= Name of Service Type to be created in AAI.
143 AZ= Name of Availability Zone to be created in AAI.
144 OE= Name of Owning Entity to be created in VID.
145 LOB= Name of Line of Business to be created in VID.
146 PROJECT= Name of Project to be created in VID.
147 PLATFORM= Name of Platform to be created in VID.
148 OS_ID= Primary key to be used when adding cloud site to mariadb pod.
149 OS_TENANT_ROLE= Only supports admin for now.
150 OS_KEYSTONE= Use KEYSTONE_V3 for now.
151 OOM_BRANCH= Branch of OOM to clone and use to install ONAP.
152 CHART_VERSION= Version of charts to use for ONAP install. This is needed in case multiple versions of the onap helm charts are present on the machine being used for the install.
153 OOM_OVERRIDES= Command line overrides to use when running helm deploy. --set <override value>, etc...
154 DOCKER_REPOSITORY= Image repository url to pull ONAP images to use for installation.
158 ### Integration Override
160 When you execute `cloud.sh`, you have the option to create an `integration-override.yaml` file that will be used during `helm deploy ...` to install ONAP. This is done by passing the `--override` flag to cloud.sh.
162 The template used to create the override file is `./util/integration-override.template`, and is invoked by `./util/create_robot_config.sh`. It's very possible this isn't complete or sufficient for how you'd like to customize your deployment. You can update the template file and/or the script to provide additional customization for your ONAP install.
166 In `cloud.conf`, there's a parameter `OOM_OVERRIDES` available that's used to provide command line overrides to `helm deploy`. This uses the standard helm syntax, so if you're using it the value should look like `OOM_OVERRIDES="--set vid.enabled=false,so.image=abc"`. If you don't want to override anything, just set this value to an empty string.
170 When you run `cloud.sh` it will execute `pre_install.sh` first, which checks a few things:
172 - It checks you have the correct pre-reqs installed. So, it'll make sure you have kubectl, azure cli, helm, etc...
173 - It checks that the version of kubernetes in `cloud.conf` is available in Azure.
174 - It checks the version of azure cli is >= to the baselined version (you can check this version by looking at the top of `pre_install.sh`). The Azure cli is introduced changes in minor versions that aren't backwards compatible.
175 - It checks that the version of kubectl installed is at **MOST** 1 minor version different than the version of kubernetes in `cloud.conf`.
177 If you would like to skip `pre_install.sh` and run the deployment anyways, pass the flag `--no-validate` to `cloud.sh`, like this:
180 $ ./cloud.sh --no-validate
186 After ONAP is deployed, you have the option of executing an arbitrary set of post-install scripts. This is enabled by passing the `--post-install` flag to `cloud.sh`, like this:
189 $ ./cloud.sh --post-install
193 These post-install scripts need to be executable from the command line, and will be provided two parameters that they can use to perform their function:
195 - /path/to/onap.conf : This is created during the deployment, and has various ONAP and OpenStack parameters.
196 - /path/to/cloud.conf : this is the same `cloud.conf` that's used during the original deployment.
198 Your post-install scripts can disregard these parameters, or source them and use the parameters as-needed.
200 Included with this repo is one post-install script (`000_bootstrap_onap.sh`)that bootstraps AAI, VID, and SO with cloud and customer details so that ONAP is ready to model and instantiate a VNF.
202 In order to include other custom post-install scripts, simply put them in the `post-install` directory, and make sure to set its mode to executable. They are executed in alphabetical order.
206 After ONAP and DevStack are deployed, there will be a `deployment.notes` file with instructions on how to access the various components. The `BUILD_DIR` specified in `cloud.conf` will contain a new ssh key, kubeconfig, and other deployment artifacts as well.
208 All of the access information below will be in `deployment.notes`.
210 ### Kubernetes Access
212 To access the Kubernetes dashboard:
214 `az aks browse --resource-group $AKS_RESOURCE_GROUP_NAME --name $AKS_NAME`
220 export KUBECONFIG=$BUILD_DIR/kubeconfig
229 Find the public IP address via the Azure portal, and go to
230 `http://$DEVSTACK_PUBLIC_IP`
232 SSH access to DevStack node:
234 `ssh -i $BUILD_DIR/id_rsa ${DEVSTACK_ADMIN_USER}@${DEVSTACK_PUBLIC_IP}`
236 OpenStack cli access:
238 There's an openstack cli pod that's created in the default kubernetes default namespace. To use it, run:
240 `kubectl exec $OPENSTACK_CLI_POD -- sh -lc "<openstack command>"`
244 `ssh -i $BUILD_DIR/id_rsa ${NFS_ADMIN_USER}@${NFS_PUBLIC_IP}`
246 ## Deleting the deployment
248 After deployment, there will be a script named `$BUILD_DIR/clean.sh` that can be used to delete the resource groups that were created during deployment. This script is not required; you can always just navigate to the Azure portal to delete the resource groups manually.
250 ## Running the scripts separately
252 Below are instructions for how to create DevStack, NFS, or AKS cluster separately if you don't want to create everything all at once.
254 **NOTE: The configuration to link components together (network peering, route table modification, NFS setup, etc...) and the onap-bootstrap will not occur if you run the scripts separately**
256 ### DevStack creation
260 $ ./create_devstack.sh --help
261 ./create_devstack.sh [options]
265 --public-key public key to add for admin user [required]
266 --user-public-ip public ip that will be granted access to VM [required]
267 -l, --location location to deploy VM [required]
268 -u, --admin-user admin user to create on VM [required]
271 -f, --no-prompt executes with no prompt for confirmation
272 -h, --help provide brief overview of script
273 -n, --name VM name [optional]
274 -g, --resource-group provide brief overview of script [optional]
275 -s, --size Azure flavor size for VM [optional]
276 -c, --cidr cidr for VNET to create for VM [optional]. If provided, must also provide --devstack-private-ip from same range.
277 -d, --directory directory to store cloud config data [optional]
278 --vnet-name name of Vnet to create for VM [optional]
279 --image-list space delimited list of image urls that will be added to devstack [optional]
280 --devstack-private-ip private ip assigned to VM [optional]. If provided, this value must come from the CIDR range of VNET.
281 --devstack-subnet-name subnet name created on VNET [optional]
282 --devstack-disk-size size of OS disk to be allocated [optional]
283 --openstack-username default user name for openstack [optional]
284 --openstack-password default password for openstack [optional]
285 --openstack-tenant default tenant name for openstack [optional]
293 $ ./create_nfs.sh --help
294 ./create_nfs.sh [options]
298 --public-key public key to add for admin user [required]
299 --user-public-ip public ip that will be granted access to VM [required]
300 -l, --location location to deploy VM [required]
301 -u, --admin-user admin user to create on VM [required]
302 --aks-node-cidr CIDR for Kubernetes nodes [required]. This is used during the NFS deploy to grant access to the NFS server from Kubernetes.
305 -f, --no-prompt executes with no prompt for confirmation
306 -h, --help provide brief overview of script
307 -n, --name VM name [optional]
308 -g, --resource-group resource group that will be created [optional]
309 -s, --size Azure flavor size for VM [optional]
310 -c, --cidr cidr for VNET to create for VM [optional].
311 -d, --directory directory to store cloud config data [optional]
312 --vnet-name name of Vnet to create for VM [optional]
313 --nfs-subnet-name subnet name created on VNET [optional]
314 --nfs-disk-size size of external disk to be mounted on NFS VM [optional]
322 $ ./create_aks.sh --help
323 ./create_aks.sh [options]
327 --user-public-ip public ip that will be granted access to AKS [required]
328 --admin-user admin user created on AKS nodes [required]
329 --public-key public key added for admin user [required]
330 -l, --location location to deploy AKS [required]
333 -f, --no-prompt executes with no prompt for confirmation
334 -h, --help provide brief overview of script
335 -n, --name AKS name [optional]
336 -g, --resource-group name of resource group that will be created [optional]
337 -s, --size azure flavor size for Kube nodes [optional]
338 -v, --kube-version version of Kubernetes for cluster [optional]
339 -c, --node-count number of nodes for cluster [optional]
340 --service-cidr cidr for Kuberenetes services [optional].
341 --dns-ip IP for Kuberenetes dns service [optional]. This should be from --service-cidr.
342 --pod-cidr cidr for Kuberenetes pods [optional].
343 --node-cidr cidr for Kuberenetes nodes [optional].
344 --vnet-name name of Vnet to create for Kubernetes Cluster [optional]