--- /dev/null
+# Infrastructure as code for OpenStack deployment of ONAP
+
+## Build your infrastructure with Terragrunt(Terraform) for ONAP
+
+Preparing ONAP for deployment requires Openstack VMs with Kubernetes and helm installed.
+Building underlying infrastructure for ONAP with Openstack GUI or command-line interfaces is not only time-consuming but also prone to mistakes.
+By providing Infrastructure as Code for deploying ONAP, building and managing the underlying infrastructure become simpler and easier.
+This [link](https://docs.onap.org/en/casablanca/submodules/oom.git/docs/oom_setup_kubernetes_rancher.html#onap-on-kubernetes-with-rancher) shows how to set up the underlying infrastructure with Openstack GUI and Command line tool.
+
+This Terragrunt(Terraform) code provides the same infrastructure as you would create through the process outlined in the link above.
+
+
+## Directory structure
+```
+openlab # Terragrunt scripts to feed configuration into the Terraform modules
+ └ RegionOne # For multi regions. e.g, us-east-1
+ └ stage # Environment specific configuration. e.g, QA/Stage/Prod
+ └ resource
+```
+
+Infrastrucuture is organized hierarchically in folders.
+The root level folder represents an account for clouds such as Openstack or AWS.
+The second and third levels represent the region in a cloud and environment under the region respectively.
+
+### Preparation
+1. You need a cloud storage bucket to store an intermediate state for your infrastructure. The remote state enables your team to work together as well. We tested this code using a Google Storage bucket. You can choose AWS S3 as well as others.
+
+
+2. Openstack is the primary target for this code. We tested this code onto Openstack v3.8 (Ocata)
+We deployed VMs and K8s with the scripts and after that we deployed ONAP Frankfurt version with OOM.
+
+### Usage
+#### Set up environment variables for your target cloud.
+
+1.a You need to export cloud storage credential.
+For instance, if you use Google Storage bucket, you can download the credentials from Google UI or the command-line tool.
+Go to Google Cloud project's `IAM & Admin` menu and choose the service account associated with the storage.
+You can export the credential as a JSON formatted file. Then
+`export GOOGLE_APPLICATION_CREDENTIALS=/path/to/credential-file`.
+Please, refer to the following [link](https://cloud.google.com/iam/docs/creating-managing-service-account-keys).
+
+Second, you need to export Openstack credentials. You can use the openstack.rc file downloaded from your Openstack project.
+Please, refer to the following [link](https://docs.openstack.org/ocata/user-guide/common/cli-set-environment-variables-using-openstack-rc.html) for details.
+
+#### Fill in files
+ - `account.hcl`: Top-level configuration for a cloud account.
+ - `region.hcl`: The second level configuration for a region within the cloud
+ - `env.hcl`: The third level configuration for an environment within the region
+ - `terragrunt.hcl`: files under the compute directory. Since Kubernetes deployment needs 2 types of nodes (control and worker) plus 1 NFS cluster, these files under the compute directory contain the configuration for Kubernentes and NFS nodes
+
+#### Building all modules for an environment
+Move to an environmental level folder, e.g stage.
+Then run `terragrunt apply-all` followed by `terraform init`
+If you run the command above for the first time and your version of Terraform is <= 0.13-beta, You may end up the error below
+`Provider "rke" not available for installation.`
+You have to manually download the `rke` provider and copy it under a Terraform's cached plug-in directory.
+After copying the rke provider, move to the `rancher` directory and run `terragrunt init`.
+Please, refer to the following [link](https://github.com/rancher/terraform-provider-rke#installing-the-provider) for details.
+
+https://github.com/gruntwork-io/terragrunt-infrastructure-live-example#deploying-all-modules-in-a-region
+
+#### Updating infrastructure version
+Infrastructure may evolve. You can use existing infrastructure as it is or updating the infrastructure to meet a new requirement.
+To deploy a different version of infrastructure, you can change a tag of `source` module version.
+Please, refer to the below document.
+If you like to test a new module (Terraform code) with Terragrunt, you just need to change the source attribute within Terrafrom block in each terragurnt.hcl file.
+[link](https://www.terraform.io/docs/modules/sources.html#generic-git-repository)
+
+#### Using Kubernetes and helm
+
+Please, refer to [link](https://github.com/gruntwork-io/terragrunt-infrastructure-live-example#example-infrastructure-live-for-terragrunt)
+
+#### Obtaining your KUBECONFIG
+Finally, You need to export Kubernenetes credentials.
+This credential is used when Helm service account is created.
+For example, `export KUBECONFIG=/path/to/kube_config_cluster.yaml`
+In default, `kube_config_cluster.yaml` will be created under `path/to/openlab/RegionOne/stage` directory once you run `terragrunt apply-all`
+
+## Google Cloud Backend for Terraform
+To use the Google Cloud Storage backend for Terraform -- it stores state and manages locking -- you'll need to install the Google Cloud SDK. Follow the instructions here https://cloud.google.com/sdk.
+You can do this task with Google Cloud's Web.
+
+1. Create a service account
+gcloud iam service-accounts create `service-account-name`
+
+2. Binding the service account with a role
+gcloud projects add-iam-policy-binding `project id` --member "serviceAccount:service-account-name-above@project-id.iam.gserviceaccount.com" --role "roles/proper-role-such-as-storage-user"
+
+3. To create a key for the service account created above
+gcloud iam service-accounts keys create account.json --iam-account `service-account-name-above@project-id.iam.gserviceaccount.com
+
+4. Create a storage bucket
+gsutil mb -p project-id gs://storage-bucket-name
+
+## Secrets
+How to hide your secret and provide it via a key management tool. Please, refer to the link below.
+Refer to https://nderraugh.github.io/
--- /dev/null
+# openlab
+
+Terraform scripts for Openlab(Windriver Lab)
--- /dev/null
+provider "helm" {
+ version = "~> 0.10.0"
+ init_helm_home = true
+ install_tiller = true
+ service_account = var.service_account
+ namespace = var.namespace
+ tiller_image = "gcr.io/kubernetes-helm/tiller:v2.16.6"
+
+ kubernetes {
+ config_path = "${var.kubernetes_home}/kube_config_cluster.yaml"
+ }
+}
+
+provider "kubernetes" {
+ version = ">= 1.10"
+ load_config_file = true
+}
+
+resource "kubernetes_service_account" "tiller" {
+ automount_service_account_token = true
+
+ metadata {
+ name = var.service_account
+ namespace = var.namespace
+ }
+}
+
+resource "kubernetes_cluster_role_binding" "tiller" {
+ metadata {
+ name = "tiller"
+ }
+
+ role_ref {
+ api_group = "rbac.authorization.k8s.io"
+ kind = "ClusterRole"
+ name = "cluster-admin"
+ }
+
+ subject {
+ kind = "ServiceAccount"
+ name = kubernetes_service_account.tiller.metadata.0.name
+ namespace = kubernetes_service_account.tiller.metadata.0.namespace
+ }
+
+ provisioner "local-exec" {
+ command = "helm init --service-account ${var.service_account};kubectl -n ${var.namespace} rollout status deployment/tiller-deploy"
+ }
+}
--- /dev/null
+variable "kubernetes_home" {}
+
+variable "service_account" {}
+
+variable "namespace" {}
\ No newline at end of file
--- /dev/null
+data "terraform_remote_state" "control_plane_floating_ips" {
+ backend = var.backend
+ config = {
+ bucket = var.backend_state
+ prefix = "${var.region}/${var.environment}/compute/control/terraform.tfstate"
+ }
+}
+
+data "terraform_remote_state" "worker_node_floating_ips" {
+ backend = var.backend
+ config = {
+ bucket = var.backend_state
+ prefix = "${var.region}/${var.environment}/compute/worker/terraform.tfstate"
+ }
+}
--- /dev/null
+# If your terraform version is < 0.13-beta, manual installation is needed.
+# https://github.com/rancher/terraform-provider-rke
+provider "rke" {}
+
+locals {
+ control_node = [for ip in (flatten(data.terraform_remote_state.control_plane_floating_ips.outputs.floating_ips)): {
+ address = ip
+ role = "control"
+ }]
+ worker_node = [for ip in (flatten(data.terraform_remote_state.worker_node_floating_ips.outputs.floating_ips)): {
+ address = ip
+ role = "worker"
+ }]
+
+ all_node = concat(local.control_node, local.worker_node)
+}
+
+resource "rke_cluster" "cluster" {
+ kubernetes_version = var.kubernetes_version
+ cluster_name = var.kubernetes_cluster_name
+
+ dynamic nodes {
+
+ for_each = local.all_node
+
+ content {
+ address = nodes.value.address
+ user = var.kubernetes_user
+ role = (nodes.value.role == "control") ? [ "controlplane", "etcd" ] : ["worker"]
+ }
+ }
+
+ # You have to have private key on your machine excuting terraform
+ # An Openstack keypair is generated and stored within the remote state at
+ # "${var.region}/${var.environment}/keypair/terraform.tfstate"
+ ssh_key_path = var.ssh_private_key_path
+
+ disable_port_check = false
+
+ depends_on = [null_resource.wait-for-docker]
+}
+
+resource "local_file" "kube_cluster_yaml" {
+ filename = "${var.kubernetes_home}/kube_config_cluster.yaml"
+ sensitive_content = rke_cluster.cluster.kube_config_yaml
+}
+
+resource "null_resource" "wait-for-docker" {
+ provisioner "local-exec" {
+ # wait untill VM's bootstrapping's done
+ # If your VMs for Computing node have finished bootstrapping already,
+ # you may not need this waiting time
+ command = "sleep 120"
+ }
+}
--- /dev/null
+output "ips" {
+ value = local.all_node
+}
+
+output "kube_config_file" {
+ value = local_file.kube_cluster_yaml.filename
+}
\ No newline at end of file
--- /dev/null
+variable "backend" {}
+
+variable "backend_state" {}
+
+variable "region" {}
+
+variable "environment" {}
+
+variable "ssh_private_key_path" {}
+
+variable "kubernetes_user" {}
+
+variable "kubernetes_version" {}
+
+variable "kubernetes_cluster_name" {}
+
+variable "kubernetes_home" {}
--- /dev/null
+data "terraform_remote_state" "keypair" {
+ backend = var.backend
+ config = {
+ bucket = var.backend_state
+ prefix = "${var.region}/${var.environment}/keypair/terraform.tfstate"
+ }
+}
+
+data "terraform_remote_state" "network" {
+ backend = var.backend
+ config = {
+ bucket = var.backend_state
+ prefix = "${var.region}/${var.environment}/network/terraform.tfstate"
+ }
+}
+
+data "terraform_remote_state" "securitygroup" {
+ backend = var.backend
+ config = {
+ bucket = var.backend_state
+ prefix = "${var.region}/${var.environment}/securitygroup/terraform.tfstate"
+ }
+}
+
+data "openstack_images_image_v2" "vm_image" {
+ name = var.image
+}
+
+data "openstack_compute_flavor_v2" "flavor" {
+ name = var.flavor
+}
+
+data "openstack_networking_network_v2" "egress_network" {
+ name = var.network
+}
+
--- /dev/null
+resource "openstack_compute_instance_v2" "nodes" {
+ name = "${var.cluster_name}-${var.node_name}-${count.index}"
+ image_id = data.openstack_images_image_v2.vm_image.id
+ flavor_id = data.openstack_compute_flavor_v2.flavor.id
+ key_pair = data.terraform_remote_state.keypair.outputs.name
+ network {
+ name = data.terraform_remote_state.network.outputs.name
+ }
+ security_groups = [ data.terraform_remote_state.securitygroup.outputs.name ]
+
+ user_data = var.user_data
+
+ count = var.node_count
+}
+
+resource "openstack_compute_floatingip_v2" "floatingip" {
+ pool = var.floating_ip_pool
+
+ count = var.node_count
+}
+
+resource "openstack_compute_floatingip_associate_v2" "floatipassociation" {
+ floating_ip = openstack_compute_floatingip_v2.floatingip[count.index].address
+ instance_id = openstack_compute_instance_v2.nodes[count.index].id
+
+ count = var.node_count
+}
--- /dev/null
+output "floating_ips" {
+ value = [openstack_compute_floatingip_associate_v2.floatipassociation.*.floating_ip]
+}
+
+output "fixed_ips" {
+ value = [openstack_compute_instance_v2.nodes.*.access_ip_v4]
+}
+
+output "hostnames" {
+ value = [openstack_compute_instance_v2.nodes.*.name]
+}
\ No newline at end of file
--- /dev/null
+variable "cluster_name" {
+ description = "A name for the cluster"
+ type = string
+}
+
+variable "backend" {}
+variable "backend_state" {}
+variable "region" {}
+variable "environment" {}
+
+variable "node_name" {}
+variable "image" {}
+variable "flavor" {}
+variable "network" {}
+variable "floating_ip_pool" {}
+variable "user_data" {}
+
+variable "node_count" {}
--- /dev/null
+resource "openstack_compute_keypair_v2" "key" {
+ # You cna find a public/private key pair from your remote state storage.
+ name = "${var.cluster_name}-key"
+ # In order to generate a new keypair via existing public key
+ public_key = var.ssh_public_key
+}
--- /dev/null
+output "name" {
+ value = openstack_compute_keypair_v2.key.name
+}
\ No newline at end of file
--- /dev/null
+variable "cluster_name" {
+ description = "A name for the cluster"
+ type = string
+}
+
+variable "ssh_public_key" { }
--- /dev/null
+data "openstack_networking_network_v2" "egress_network" {
+ name = var.network
+}
\ No newline at end of file
--- /dev/null
+resource "openstack_networking_network_v2" "network" {
+ name = "${var.cluster_name}-network"
+ admin_state_up = "true"
+}
+
+resource "openstack_networking_subnet_v2" "subnet" {
+ name = "${var.cluster_name}-subnet"
+ network_id = openstack_networking_network_v2.network.id
+ cidr = "192.168.64.0/24"
+ ip_version = 4
+ gateway_ip = "192.168.64.1"
+ enable_dhcp = "true"
+ dns_nameservers = [ "8.8.8.8", "8.8.4.4" ]
+}
+
+resource "openstack_networking_router_v2" "router" {
+ name = "${var.cluster_name}-router"
+ admin_state_up = true
+ external_network_id = data.openstack_networking_network_v2.egress_network.id
+}
+
+resource "openstack_networking_router_interface_v2" "router_interface" {
+ router_id = openstack_networking_router_v2.router.id
+ subnet_id = openstack_networking_subnet_v2.subnet.id
+}
\ No newline at end of file
--- /dev/null
+output "name" {
+ value = openstack_networking_network_v2.network.name
+}
\ No newline at end of file
--- /dev/null
+variable "cluster_name" {
+ description = "A name for the cluster"
+ type = string
+}
+
+variable "network" {
+ description = "A network to the Internet"
+ type = string
+}
--- /dev/null
+resource "openstack_networking_secgroup_v2" "securitygroup" {
+ name = "${var.cluster_name}-securitygroup"
+ description = "RKE security group"
+}
+
+resource "openstack_networking_secgroup_rule_v2" "securitygroup_rule" {
+ direction = "ingress"
+ ethertype = "IPv4"
+ security_group_id = openstack_networking_secgroup_v2.securitygroup.id
+}
--- /dev/null
+output "name" {
+ value = openstack_networking_secgroup_v2.securitygroup.name
+}
\ No newline at end of file
--- /dev/null
+variable "cluster_name" {
+ description = "A name for the cluster"
+ type = string
+}
Code Review / integration / terraform.git / commitdiff