multicloud/k8s.git
3 years agoUpdate multus-daemonset.yml to that of multus-cni v3.6 release 65/114465/2
Todd Malsbary [Fri, 30 Oct 2020 20:31:04 +0000 (13:31 -0700)]
Update multus-daemonset.yml to that of multus-cni v3.6 release

The only change to the upstream yml is the removal of the
kube-multus-ds-ppc64le DaemonSet and the replacement of
"default-cni-network" with "cni0".

Note also that the v3.6 yml actually uses the v3.4.1 image tag.  The
yml now points to a v3.4.1 image with the addition of code to merge
the results from all delegates to support Virtlet.

Issue-ID: MULTICLOUD-1230
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
Change-Id: I0e18644a567facfac1fd7dc1c053002b2d906288

3 years agoUpgrade kubespray from 2.12.6 to 2.14.1 64/114464/1
Todd Malsbary [Tue, 27 Oct 2020 21:46:29 +0000 (14:46 -0700)]
Upgrade kubespray from 2.12.6 to 2.14.1

- Replace move of ansible.cfg from kubespray distribution to
  /etc/ansible with ANSIBLE_CONFIG environment variable.  Ansible
  modifies ansible.cfg during installation, and the paths in it are
  relative.

- kubespray 2.14.1 requires a kubernetes version > 1.16.  Use the
  default versions of kubernetes and helm provided by kubespray
  2.14.1.

- kubespray 2.14.1 replaces helm 2 with helm 3.  This removes support
  for helm init and helm serve.  It is no longer necessary to call
  helm init, and the helm serve repository is replaced with file
  relative URLs.  This also triggered a subsequent update of the
  kubernetes-helm ansible module to include the newer helm versions.

- Add "storageType: hostPath" to etcd/values.yaml.  Helm deploy of
  etcd will fail without this due to nil
  PersistentVolume.metadata.labels.type.

- The mitogen module used by kubespray/ansible requires python2 on the
  hosts.  Use the linear strategy to bypass mitogen and install
  python2 on the cluster hosts.

Issue-ID: MULTICLOUD-1230
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
Change-Id: I9f50bb4e123fdcacab6b6a97e79cd09fb5c96634

3 years agoMerge "Update EMCO OpenAPI spec with DCM APIs"
Eric Multanen [Fri, 16 Oct 2020 23:27:28 +0000 (23:27 +0000)]
Merge "Update EMCO OpenAPI spec with DCM APIs"

3 years agoMerge "Restore commented unit test TestDeleteLogicalCloud"
Eric Multanen [Fri, 16 Oct 2020 23:25:49 +0000 (23:25 +0000)]
Merge "Restore commented unit test TestDeleteLogicalCloud"

3 years agoMerge "delete should return 404 error for invalid name"
Eric Multanen [Fri, 16 Oct 2020 23:22:22 +0000 (23:22 +0000)]
Merge "delete should return 404 error for invalid name"

3 years agoRestore commented unit test TestDeleteLogicalCloud 49/113749/1
Igor D.C [Thu, 8 Oct 2020 18:44:14 +0000 (18:44 +0000)]
Restore commented unit test TestDeleteLogicalCloud

Restore the previously commented-out unit test TestDeleteLogicalCloud.
That test was disabled due to a failure introduced by interacting with
AppContext for the first time in module/logicalcloud.go and it not
being ready to do so.

This commit restores it and modifies code so dependent mocks can plug
in correctly. This was done in order to keep testing the code that was
previously being tested, not so much to add additional coverage.

Although it would be a significant undertaking, the different types and
interfaces in pkg/module should be redesigned to achieve better
decoupling and thus make unit testing more straightforward.

Issue-ID: MULTICLOUD-1143
Change-Id: I1e6b7bb9111fc6883f0c9cee887329a9e0b27fbd
Signed-off-by: Igor D.C <igor.duarte.cardoso@intel.com>
3 years agoUpdate EMCO OpenAPI spec with DCM APIs 61/111061/11
Igor D.C [Fri, 7 Aug 2020 23:29:02 +0000 (23:29 +0000)]
Update EMCO OpenAPI spec with DCM APIs

Update emco_apis.yaml with all new API paths provided by DCM and
its /logical-clouds URL prefix. Update schemas and parameters too.

Issue-ID: MULTICLOUD-1143
Change-Id: I9d94cb2954c2f0131ffb8c3061a87a6b6a235cba
Signed-off-by: Igor D.C <igor.duarte.cardoso@intel.com>
3 years agoFix mispelled key name in kubeconfig 07/113707/1
Igor D.C [Thu, 8 Oct 2020 23:50:55 +0000 (23:50 +0000)]
Fix mispelled key name in kubeconfig

Fixes the mispelled "currentcontext" with the right key name
"current-context" which was introduced as a regression with or
before the latest version of the kubeconfig code (probably a bad
search/replace).

This 1-character long bug was preventing the generated kubeconfigs
from being correctly interpreted by kubectl. Thus, kubectl couldn't
set any context of the kubeconfig as the current context.

Observable output before this fix:
$ kubectl get pods
The connection to the server localhost:8080 was refused - did you
specify the right host or port?

Issue-ID: MULTICLOUD-1143
Change-Id: I617d1e20b7be2567729b84d3746b22e4ceaf8b9d
Signed-off-by: Igor D.C <igor.duarte.cardoso@intel.com>
3 years agoUpdating the API definition 36/113136/5
Ritu Sood [Thu, 24 Sep 2020 20:35:02 +0000 (13:35 -0700)]
Updating the API definition

API definition updated to add
generic placemnent intent under
deployment group

Issue-ID: MULTICLOUD-1096
Signed-off-by: Ritu Sood <ritu.sood@intel.com>
Change-Id: Ida830de0f17038925651bc604440b11547b7320d

3 years agoMerge "Add k8splugin releases file for 0.7.0"
Ritu Sood [Thu, 8 Oct 2020 21:29:54 +0000 (21:29 +0000)]
Merge "Add k8splugin releases file for 0.7.0"

3 years agoAdd k8splugin releases file for 0.7.0 06/113706/1
Eric Multanen [Fri, 9 Oct 2020 04:37:14 +0000 (21:37 -0700)]
Add k8splugin releases file for 0.7.0

Add k8splugin file for 0.7.0

Issue-ID: MULTICLOUD-1213
Signed-off-by: Eric Multanen <eric.w.multanen@intel.com>
Change-Id: I54d7610f39be4d05e176ff7e2f67bd7594e50677

3 years agoMerge "Reduce duplication around DeleteCompositeApp calls"
Ritu Sood [Thu, 8 Oct 2020 21:12:12 +0000 (21:12 +0000)]
Merge "Reduce duplication around DeleteCompositeApp calls"

3 years agoUpdate k8splugin for G release 88/113688/1 0.7.0
Eric Multanen [Thu, 8 Oct 2020 20:50:55 +0000 (13:50 -0700)]
Update k8splugin for G release

Update image version of k8splugin to 0.7.0-SNAPSHOT
This will be then be used to create the release image

Issue-ID: MULTICLOUD-1213
Signed-off-by: Eric Multanen <eric.w.multanen@intel.com>
Change-Id: Ice57a61bc98397a2bbb5c313452459b02ec86139

3 years agoReduce duplication around DeleteCompositeApp calls 61/113661/1
Igor D.C [Thu, 8 Oct 2020 02:54:54 +0000 (02:54 +0000)]
Reduce duplication around DeleteCompositeApp calls

DCM's apply function had a lot of error handling code that cleans up
the AppContext if something goes wrong while adding to AppContext.

Most of that handling was using multiline duplicate code with the only
change being in the log/error strings. This commit attempts to reuse
all that error handling code and make it easier to follow main code.

Issue-ID: MULTICLOUD-1143
Change-Id: I3a35387b1ed46279c1b973dbd852352276ff5cc8
Signed-off-by: Igor D.C <igor.duarte.cardoso@intel.com>
3 years agoMerge "Apply corrections to EMCO v2 helm README"
Ritu Sood [Tue, 6 Oct 2020 21:06:21 +0000 (21:06 +0000)]
Merge "Apply corrections to EMCO v2 helm README"

3 years agoApply corrections to EMCO v2 helm README 90/113590/1
Igor D.C [Tue, 6 Oct 2020 20:45:19 +0000 (20:45 +0000)]
Apply corrections to EMCO v2 helm README

Mostly just make the identifiers consistent with each other
so copy-paste works without surprises. No more rel- prefix, just emco-.

Additionally suggest a 2nd workaround to the persistentvolumes issue.

Issue-ID: MULTICLOUD-1143
Change-Id: I7ad1d3a4b20f7563226a9d487c388275e56429b5
Signed-off-by: Igor D.C <igor.duarte.cardoso@intel.com>
3 years agoAdd DCM to Helm deployment folder 95/113395/4
Igor D.C [Wed, 30 Sep 2020 17:34:28 +0000 (17:34 +0000)]
Add DCM to Helm deployment folder

Also add DCM to cleanup-emco.sh.

Issue-ID: MULTICLOUD-1143
Change-Id: I2496c1fef2e2588c1c8e750105568afc210d54c3
Signed-off-by: Igor D.C <igor.duarte.cardoso@intel.com>
3 years agoMerge "Remove unused mongo db methods from emco code"
Ritu Sood [Mon, 5 Oct 2020 22:47:13 +0000 (22:47 +0000)]
Merge "Remove unused mongo db methods from emco code"

3 years agoMerge "Ovnaction and vfw updates for deploy api change"
Ritu Sood [Mon, 5 Oct 2020 22:45:38 +0000 (22:45 +0000)]
Merge "Ovnaction and vfw updates for deploy api change"

3 years agoMerge "Modify GenericPlacement APIs to include DepIntGrp"
Ritu Sood [Mon, 5 Oct 2020 22:45:20 +0000 (22:45 +0000)]
Merge "Modify GenericPlacement APIs to include DepIntGrp"

3 years agoOvnaction and vfw updates for deploy api change 40/113440/4
Eric Multanen [Thu, 1 Oct 2020 05:38:59 +0000 (22:38 -0700)]
Ovnaction and vfw updates for deploy api change

Update the ovnaction controller APIs to support
the api change of including the deployment intent
group in the URL.  Also fixup:
- vfw and other test cases to support the change
- updates to emcoctl tool and examples

Issue-ID: MULTICLOUD-1218
Signed-off-by: Eric Multanen <eric.w.multanen@intel.com>
Change-Id: Icadacb5ec6d7c238bb3bf8a44a39c30692ecebee

3 years agoModify GenericPlacement APIs to include DepIntGrp 45/113145/6
Rajamohan Raj [Fri, 25 Sep 2020 03:16:40 +0000 (03:16 +0000)]
Modify GenericPlacement APIs to include DepIntGrp

Modify the genericPlacementIntent API such that deploymentIntentGroup
becomes a mandatory parameter.

Issue-ID: MULTICLOUD-1218
Signed-off-by: Rajamohan Raj <rajamohan.raj@intel.com>
Change-Id: I33d2eeac5b60228e9c08921c9347b1b6aa3f8d28

3 years agoDCM E2E testing fixes 50/113450/6
Igor D.C [Fri, 2 Oct 2020 01:40:09 +0000 (01:40 +0000)]
DCM E2E testing fixes

A set of small fixes after discovery during DCM's end-to-end testing:
- properly detect that a cert hasn't been issued yet (don't crash)
- in Monitor, don't pass namespace when querying for CSRs
- fixed incorrect /kubeconfig file encoding from yaml to json

Issue-ID: MULTICLOUD-1143
Change-Id: Ie813e377070b0751f0bfdabac8da50e3288090de
Signed-off-by: Igor D.C <igor.duarte.cardoso@intel.com>
3 years agoMerge "Set name of resources correctly in appcontext"
Ritu Sood [Fri, 2 Oct 2020 23:20:57 +0000 (23:20 +0000)]
Merge "Set name of resources correctly in appcontext"

3 years agoMerge "Reimplement Terminate to be compatible with Status"
Ritu Sood [Fri, 2 Oct 2020 23:20:30 +0000 (23:20 +0000)]
Merge "Reimplement Terminate to be compatible with Status"

3 years agoRemove unused mongo db methods from emco code 47/113447/2
Eric Multanen [Thu, 1 Oct 2020 23:29:08 +0000 (16:29 -0700)]
Remove unused mongo db methods from emco code

Remove the set of unused (legacy from v1) mongo
db methods so that they are not used in the emco
code base.

Issue-ID: MULTICLOUD-1227
Signed-off-by: Eric Multanen <eric.w.multanen@intel.com>
Change-Id: Ibe94892e5b1d0f33192be7ea3db5c55b3f8cbf60

3 years agoSet name of resources correctly in appcontext 49/113449/4
Igor D.C [Fri, 2 Oct 2020 00:20:09 +0000 (00:20 +0000)]
Set name of resources correctly in appcontext

Previously the DCM K8s resources in appcontext were in the format:
<logical cloud name>+<resource type>

Now they are in the correct format:
<resource name>+<ResourceType>

Issue-ID: MULTICLOUD-1143
Change-Id: I271c70587244921cc2ffb7ad571ad8eb162cb10f
Signed-off-by: Igor D.C <igor.duarte.cardoso@intel.com>
3 years agoReimplement Terminate to be compatible with Status 45/113445/4
Igor D.C [Thu, 1 Oct 2020 17:50:20 +0000 (17:50 +0000)]
Reimplement Terminate to be compatible with Status

This also includes modifying Apply and Delete, since there are
strict conditions that need to be met in each, to prevent
Logical Clouds (LCs) from entering a bad state.

Summary of what's being done here:
- When applying:
  - set tag 'lccontext' in the LC to the context ID (was already done)
  - and let rsync know about the appcontext (grpc) (was already done)
  - if tag was already set, check current context /status
  - if context /status is actually Terminated,  'lccontext' is set to
    new context ID and previous AppContext deleted
- When terminating:
  - lets rsync know about the termination request (grpc)
- When deleting:
  - checks whether the current context /status is Terminated
  - if it is, then it will remove the latest LC context

This particular commit disables the TestDeleteLogicalCloud test
until a known issue behind the test is resolved.

This commit does not leverage the full capacity of the Status
framework, but is sufficient to support all operations. A future
patch will entirely migrate DCM to the Status framework. Until
then, a known issue exists where DCM will forget about context IDs
previously associated to a particular Logical Cloud (only keeps last).

Issue-ID: MULTICLOUD-1143
Change-Id: I7a6034eba543c2a27daa41b7fe6298cb2a85f9ce
Signed-off-by: Igor D.C <igor.duarte.cardoso@intel.com>
3 years agoAdding CSR Approval functionality 44/113444/4
Ritu Sood [Thu, 1 Oct 2020 22:05:42 +0000 (15:05 -0700)]
Adding CSR Approval functionality

Update rsync to be able to approve CSR

Issue-ID: MULTICLOUD-1143
Signed-off-by: Ritu Sood <ritu.sood@intel.com>
Change-Id: I0b2bec3475a3453a2d8fc9c2e87cfc4531b0e2f3

3 years agoImprove DCM test script: apply, term, kubeconfig 48/113448/3
Igor D.C [Thu, 1 Oct 2020 23:44:46 +0000 (23:44 +0000)]
Improve DCM test script: apply, term, kubeconfig

Adds apply and terminate operations, while decoupling them in two
different script branches.

Add kubeconfig test into a third script branch.

Also, $cluster_2_name wasn't defined, so this fixes that.

Issue-ID: MULTICLOUD-1143
Change-Id: I574934078644ec83224bcac74b5b17783330aaaa
Signed-off-by: Igor D.C <igor.duarte.cardoso@intel.com>
3 years agoMerge "Fix rsync retry logic for unreachable cluster"
Ritu Sood [Fri, 2 Oct 2020 20:10:22 +0000 (20:10 +0000)]
Merge "Fix rsync retry logic for unreachable cluster"

3 years agoFix rsync retry logic for unreachable cluster 39/113439/1
Eric Multanen [Tue, 22 Sep 2020 18:10:32 +0000 (11:10 -0700)]
Fix rsync retry logic for unreachable cluster

Fix rsync to only retry reaching clusters until the
timeout occurs.

Issue-ID: MULTICLOUD-1226
Signed-off-by: Eric Multanen <eric.w.multanen@intel.com>
Change-Id: I5e8202358cfd8cdd15c9feb4fcbefe2683b7139a

3 years agoSubtle refactoring in a few functions 32/113432/2
Igor D.C [Wed, 30 Sep 2020 21:30:06 +0000 (21:30 +0000)]
Subtle refactoring in a few functions

In DCM.

Essentially refactored two different areas:
- rename apply/terminate functions in module to clarify what they do
- split gets from getAlls in API code of the 5 DCM resource types

And cleaned up here and there.

Issue-ID: MULTICLOUD-1143
Change-Id: I9b72c77ba34a1febd5df4a339e87968ddc4a7891
Signed-off-by: Igor D.C <igor.duarte.cardoso@intel.com>
3 years agoFix arg mismatch in GetLogicalCloudContext 08/113408/1
Igor D.C [Thu, 1 Oct 2020 00:18:45 +0000 (00:18 +0000)]
Fix arg mismatch in GetLogicalCloudContext

The previous commits were merged in the wrong order, which caused a bad
3-way merge resulting in the arg mismatch as witnessed below:

k8s/src/dcm/pkg/modulepkg/module/cluster.go:263:57: not enough arguments
in call to lcClient.GetLogicalCloudContext
        have (string)
        want (string, string)
Makefile:13: recipe for target 'all' failed
make: *** [all] Error 2

The CI merge job doesn't run tests before merging, so this slipped in.

Issue-ID: MULTICLOUD-1143
Change-Id: I0168345af6bac7886cba32d958f241687ab24a81
Signed-off-by: Igor D.C <igor.duarte.cardoso@intel.com>
3 years agoMerge "Implement Kubeconfig endpoint in DCM"
Ritu Sood [Wed, 30 Sep 2020 23:48:26 +0000 (23:48 +0000)]
Merge "Implement Kubeconfig endpoint in DCM"

3 years agoMerge "Update ResourceBundleState CRD and monitor-deploy"
Eric Multanen [Wed, 30 Sep 2020 23:28:04 +0000 (23:28 +0000)]
Merge "Update ResourceBundleState CRD and monitor-deploy"

3 years agoMerge "Fix Status API to actually provide instance status"
Eric Multanen [Wed, 30 Sep 2020 23:22:46 +0000 (23:22 +0000)]
Merge "Fix Status API to actually provide instance status"

3 years agoUpdate ResourceBundleState CRD and monitor-deploy 99/113399/2
Igor D.C [Wed, 30 Sep 2020 19:37:51 +0000 (19:37 +0000)]
Update ResourceBundleState CRD and monitor-deploy

Update ResourceBundleState CRD and monitor-deploy.yaml with
csrStatuses.

This was missing from the previous patch implementing CSR support
in Monitor. Depending on how the environment was brought up, attempting
to read the CSR would result in forbidden access.

Issue-ID: MULTICLOUD-1143
Change-Id: I86ffb8df8faa045e57fd553e5784d308e1900ee4
Signed-off-by: Igor D.C <igor.duarte.cardoso@intel.com>
3 years agoMerge "Add DCM to K8s deployment file"
Ritu Sood [Wed, 30 Sep 2020 17:45:29 +0000 (17:45 +0000)]
Merge "Add DCM to K8s deployment file"

3 years agodelete should return 404 error for invalid name 39/113139/3
Aditya Sharoff [Thu, 24 Sep 2020 22:45:15 +0000 (15:45 -0700)]
delete should return 404 error for invalid name

Issue-ID: MULTICLOUD-1150

Change-Id: I04c9584e35672b6b40047a67be4d2026002204e1
Signed-off-by: Aditya Sharoff <aditya.sharoff@intel.com>
3 years agoAdd DCM to K8s deployment file 93/113393/2
Igor D.C [Wed, 30 Sep 2020 17:09:28 +0000 (17:09 +0000)]
Add DCM to K8s deployment file

Issue-ID: MULTICLOUD-1143
Change-Id: Iaca9bd6cb3614e0ae825d9231476524ce09db550
Signed-off-by: Igor D.C <igor.duarte.cardoso@intel.com>
3 years agoFix Status API to actually provide instance status 71/113171/4
Konrad Bańka [Fri, 25 Sep 2020 14:35:02 +0000 (16:35 +0200)]
Fix Status API to actually provide instance status

Provide information about instance resources and Pods inside status
response.

Issue-ID: MULTICLOUD-1177
Signed-off-by: Konrad Bańka <k.banka@samsung.com>
Change-Id: Iee6fd56120d091dddfa6b6d0e4aa7eb36d40e888

3 years agoUse project name provided by API in DCM 43/113343/2
Igor D.C [Tue, 29 Sep 2020 19:00:37 +0000 (19:00 +0000)]
Use project name provided by API in DCM

DCM apply/terminate code was still using a hardcoded "test-project"
string for the project name that logical clouds should associate to.

This simple patch makes DCM use the project name provided by the API.

It also checks that the project exists before creating a Logical Cloud.

Issue-ID: MULTICLOUD-1143
Change-Id: I05c8a2309ed07a7c96da30bf2461c43abaccc9d5
Signed-off-by: Igor D.C <igor.duarte.cardoso@intel.com>
3 years agoImplement Kubeconfig endpoint in DCM 04/113204/6
Igor D.C [Fri, 25 Sep 2020 05:28:09 +0000 (05:28 +0000)]
Implement Kubeconfig endpoint in DCM

The /kubeconfig API path allows a client to retrieve a kubeconfig
file for a specified cluster reference of a logical cloud.
- includes CA cert, address, user private key and signed cert.

This commit includes the "lazy-loading" implementation of certificate
retrieval per cluster from Rsync (which happens when clients call).

The certificate is read from the cluster status in appcontext.
Thus, Monitor and Rsync need to be configured and running.

Issue-ID: MULTICLOUD-1143
Change-Id: Ie94cd128e14c8a944861eced2bdc886d95fab6ed
Signed-off-by: Igor D.C <igor.duarte.cardoso@intel.com>
3 years agoMerge "Add plugin_fw.sh test for v2 and run as part of installer."
Ritu Sood [Tue, 29 Sep 2020 19:58:19 +0000 (19:58 +0000)]
Merge "Add plugin_fw.sh test for v2 and run as part of installer."

3 years agoMerge "Add a couple of sanity checks in DCM"
Ritu Sood [Mon, 28 Sep 2020 23:51:43 +0000 (23:51 +0000)]
Merge "Add a couple of sanity checks in DCM"

3 years agoRequest CSR approvals via /subresource level 07/113207/2
Igor D.C [Sat, 26 Sep 2020 02:54:02 +0000 (02:54 +0000)]
Request CSR approvals via /subresource level

Make use of the new /subresource level (under a resource) to
request rsync to process approvals (a K8s subresource) for CSRs.

Generic usage of instructions (order, dependency) for consistency.

This also introduces a 'subresources' package in 'appcontext' with
the first member being the approval subresource. Since subresources
aren't necessarily created with yaml, this package will allow
developers to specify what should be the "interface" in appcontext
for each type of subresource when the different services need to
exchange this data (e.g. DCM and rsync).

Issue-ID: MULTICLOUD-1143
Change-Id: I8c85b4ee8c5654036a1c7ebb39d111ba4518b468
Signed-off-by: Igor D.C <igor.duarte.cardoso@intel.com>
3 years agoMerge "Implement rest of Apply operation in DCM"
Ritu Sood [Mon, 28 Sep 2020 23:27:04 +0000 (23:27 +0000)]
Merge "Implement rest of Apply operation in DCM"

3 years agoMerge "Implement Terminate operation in DCM"
Ritu Sood [Mon, 28 Sep 2020 23:26:25 +0000 (23:26 +0000)]
Merge "Implement Terminate operation in DCM"

3 years agoFix special characters issue in quotas in DCM 42/113142/8
Igor D.C [Fri, 25 Sep 2020 00:15:58 +0000 (00:15 +0000)]
Fix special characters issue in quotas in DCM

Before this patch, any resource quotas, say, with a dot (like
limits.cpu), would not be stored properly in the database and thus
unusable and ignored by DCM logical cloud apply operation.

Issue-ID: MULTICLOUD-1143
Change-Id: I178e66756bc9bb2798427233d15196d0e2559a99
Signed-off-by: Igor D.C <igor.duarte.cardoso@intel.com>
3 years agoImplement rest of Apply operation in DCM 06/113206/1
Igor D.C [Sat, 26 Sep 2020 02:19:38 +0000 (02:19 +0000)]
Implement rest of Apply operation in DCM

"Rest of" meaning the Apply operation now effectively
instantiates logical cloud resources on K8s cluster.

More specifically, this commit now enables:
- DCM calling rsync over gRPC on apply
- The following K8s resources are now being created per cluster:
  - namespace, csr, resourcequotas, roles, rolebindings
- Tested-working for multiple clusters simultaneously
- Generating and storing logical cloud user private key in MongoDB

This commit doesn't include any logic to request
or retrieve CSR approvals and signed user certificates.

Issue-ID: MULTICLOUD-1143
Change-Id: I92c4270678d05a5cb531791cef199a7dd79403d3
Signed-off-by: Igor D.C <igor.duarte.cardoso@intel.com>
3 years agoAdd a couple of sanity checks in DCM 05/113205/1
Igor D.C [Sat, 26 Sep 2020 01:55:04 +0000 (01:55 +0000)]
Add a couple of sanity checks in DCM

When applying, check whether logical cloud is already applied.
When deleting logical cloud, check if it exists.

Issue-ID: MULTICLOUD-1143
Change-Id: I6aa11d8921db76e3d9e9632f915b55fe7ab8efab
Signed-off-by: Igor D.C <igor.duarte.cardoso@intel.com>
3 years agoImplement Terminate operation in DCM 03/113203/2
Igor D.C [Fri, 25 Sep 2020 22:31:11 +0000 (22:31 +0000)]
Implement Terminate operation in DCM

Also makes minor changes to non-terminate code as a side-effect of
supporting the new Terminate operation (such as including tagContext in
the LogicalCloudClient implementation of LogicalCloudManager interface).
These changes are/will also be leveraged by other operations.

Issue-ID: MULTICLOUD-1143
Change-Id: Idbd2ec9f6cf0e5584a0f51cf4c16144db56d9fa0
Signed-off-by: Igor D.C <igor.duarte.cardoso@intel.com>
3 years agoIntroduce Monitor support for CSR resource 26/111926/11
Igor D.C [Mon, 31 Aug 2020 22:46:10 +0000 (22:46 +0000)]
Introduce Monitor support for CSR resource

These changes allow the Monitor to also track CSR
(CertificateSigningResource) resources which will make
it possible to know when a certificate has been issued by
the K8s cluster signer. In turn, DCM will be able to read,
store and use that certificate to generate kubeconfigs.

Out-of-tree actions required:
- publish monitor's docker image built from this source
  onto emcov2/monitor:latest

Issue-ID: MULTICLOUD-1143
Change-Id: I7facd27bbfe08891151bb3b6a9a19948435e24e4
Signed-off-by: Igor D.C <igor.duarte.cardoso@intel.com>
3 years agoMerge "Enhance error handling and HTTP codes in DCM"
Ritu Sood [Fri, 25 Sep 2020 16:36:24 +0000 (16:36 +0000)]
Merge "Enhance error handling and HTTP codes in DCM"

3 years agoIf getAppContextStatus fails, cancel waitForDone 38/113138/2
Igor D.C [Thu, 24 Sep 2020 22:01:35 +0000 (22:01 +0000)]
If getAppContextStatus fails, cancel waitForDone

And log error appropriately.

Essentially, add basic error handling and return from waitForDone.

It is possible to trigger this error easily by having an rsync client
wipe out etcd right after issuing an InvokeUninstallApp, as demonstrated
by DCM (before it was adapted to use the new asynchronous rsync).

Issue-ID: MULTICLOUD-1143
Change-Id: Id5c0d8da928738cb3cdc862eaefdb91db2eadf5f
Signed-off-by: Igor D.C <igor.duarte.cardoso@intel.com>
3 years agoEnhance error handling and HTTP codes in DCM 41/113141/7
Igor D.C [Fri, 25 Sep 2020 00:04:58 +0000 (00:04 +0000)]
Enhance error handling and HTTP codes in DCM

This improves error handling between DCM and the database resources and
adds/corrects a lot more HTTP return codes in the API, respectively.

Issue-ID: MULTICLOUD-1143
Change-Id: I3abc8025660e042f4c946f8bbfd280e1eb4c9583
Signed-off-by: Igor D.C <igor.duarte.cardoso@intel.com>
3 years agoMerge "Let instructions exist under /subresource level"
Ritu Sood [Fri, 25 Sep 2020 02:15:45 +0000 (02:15 +0000)]
Merge "Let instructions exist under /subresource level"

3 years agoAdd default/template config.json for DCM 40/113140/1
Igor D.C [Thu, 24 Sep 2020 23:37:33 +0000 (23:37 +0000)]
Add default/template config.json for DCM

DCM listening on port 9077 as default.

The IP addresses for MongoDB and etcd are what we get by default
on docker installations of those using emco deployments.

Issue-ID: MULTICLOUD-1143
Change-Id: I9b30b278e3cf4fdb428665986e05c6916000b4e7
Signed-off-by: Igor D.C <igor.duarte.cardoso@intel.com>
3 years agoLet instructions exist under /subresource level 27/111927/4
Igor D.C [Mon, 31 Aug 2020 23:00:46 +0000 (23:00 +0000)]
Let instructions exist under /subresource level

Instructions (order/dependency) can now belong to an
app, a resource or a subresource.

Issue-ID: MULTICLOUD-1143
Signed-off-by: Igor D.C <igor.duarte.cardoso@intel.com>
Change-Id: Iee55c2bbb569de242e66f5a5456ac52025a00f0e

3 years agoCleanup tmp resources after instantiation 38/112638/2
Rajamohan Raj [Mon, 14 Sep 2020 19:47:54 +0000 (19:47 +0000)]
Cleanup tmp resources after instantiation

In this patch,we clean up the tmp files
and directories generated during the
process of instantiation.
They include the tmp dirs with
prefix : /tmp/helm-tmpl and /tmp/k8s-ext

Issue-ID: MULTICLOUD-1206
Signed-off-by: Rajamohan Raj <rajamohan.raj@intel.com>
Change-Id: I02d11bb2f8d920e35aae7343f041a53b1cd3f057

3 years agoinitial commit of emcoui web app 08/113208/1
vikaskumar [Thu, 24 Sep 2020 07:37:57 +0000 (13:07 +0530)]
initial commit of emcoui web app

Signed-off-by: vikaskumar <vkumar@aarnanetworks.com>
Change-Id: I78b2f9f4ec3aa0b7a660a21e2ee2d61a3cddadc6
Issue-ID: MULTICLOUD-1225

3 years agoAdd plugin_fw.sh test for v2 and run as part of installer. 81/113081/1
Todd Malsbary [Tue, 8 Sep 2020 18:31:52 +0000 (11:31 -0700)]
Add plugin_fw.sh test for v2 and run as part of installer.

To deploy to multiple clusters, set the KUD_PLUGIN_FW_CLUSTERS
environment variable to the following format (an array of cluster data
objects):

[
  {
    "metadata": {
      "name": "NAME",
      "description": "DESCRIPTION",
      "userData1": "USER_DATA_1",
      "userData2": "USER_DATA_2"
    },
    "file": "KUBECONFIG_PATH"
  },
  {
    ...
  }
]

Issue-ID: MULTICLOUD-1217
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
Change-Id: I4c80fbcef1162b441c4dfba4ce2bfd3ac419bc25

3 years agoMerge "Add playbooks for v2 emco chart."
Ritu Sood [Wed, 23 Sep 2020 20:25:04 +0000 (20:25 +0000)]
Merge "Add playbooks for v2 emco chart."

3 years agoMerge "Create helm chart for ICN/SDEWAN controllers"
Ritu Sood [Wed, 23 Sep 2020 20:01:28 +0000 (20:01 +0000)]
Merge "Create helm chart for ICN/SDEWAN controllers"

3 years agoAdd playbooks for v2 emco chart. 10/111810/7
Todd Malsbary [Fri, 7 Aug 2020 17:20:12 +0000 (10:20 -0700)]
Add playbooks for v2 emco chart.

Rename v2/onap4k8s to v2/emco, and rename sanity-check-for-v2.sh to
emco.sh.  This allows --plugins emco to be passed to installer.sh in
place of --plugins onap4k8s.

Issue-ID: MULTICLOUD-1181
Signed-off-by: Todd <todd.malsbary@intel.com>
Change-Id: Idb427a8aa4c8aaff181965a540078c8cf6dd88aa

3 years agoEnable installer.sh to use --plugins emco in place of onap4k8s. 09/111809/7
Todd [Wed, 26 Aug 2020 21:48:28 +0000 (21:48 +0000)]
Enable installer.sh to use --plugins emco in place of onap4k8s.

Issue-ID: MULTICLOUD-1181
Signed-off-by: Todd <todd.malsbary@intel.com>
Change-Id: Ibfdf401d40398bf6b94543dedf4c860951d50de7

3 years agoCreate helm chart for ICN/SDEWAN controllers 37/109637/12
Yao Le [Mon, 29 Jun 2020 14:05:35 +0000 (22:05 +0800)]
Create helm chart for ICN/SDEWAN controllers

Create helm chart for sdewan-controller in KUD

Issue-ID: MULTICLOUD-1104
Signed-off-by: Yao Le <le.yao@intel.com>
Change-Id: Ic5d8daecdecba52c05d3ed38fa91ebd555ce5533

3 years agoFix missing operator-sdk dependency goautoneg 45/112945/10
Igor D.C [Mon, 21 Sep 2020 17:46:52 +0000 (17:46 +0000)]
Fix missing operator-sdk dependency goautoneg

goautoneg used to be available from bitbucket.org/ww/goautoneg
but not anymore. operator-sdk has been importing it from
github.com/munnerz/goautoneg for some time now but since we were
using operator-sdk v0.9.0, it was still pointing at the old (now
deleted) repository.

This patch bumps operator-sdk to v0.19.0 (which doesn't use
goautoneg from bitbucket), controller-runtime to v0.5.11, helm to
v2.16.12 and all k8s.io dependencies to v0.16.9 (consistent with
the k8s v1.16.9 we use).

Code changes have been made to make all modules compatible with
the new dependencies.

All src go.sums have been cleaned (deleted and re-built).

Issue-ID: MULTICLOUD-1143
Change-Id: I89fc39e9595b4a6a38dd90028b161bcecd3ef349
Signed-off-by: Igor D.C <igor.duarte.cardoso@intel.com>
3 years agoDelete issues error code 404 when project name is invalid 34/112434/6
Aditya Sharoff [Wed, 9 Sep 2020 23:22:32 +0000 (16:22 -0700)]
Delete issues error code 404 when project name is invalid

Issue-ID: MULTICLOUD-1147

Signed-off-by: Aditya Sharoff <aditya.sharoff@intel.com>
Change-Id: I659dfcb00b3f15bcf787ac3947f8755737b1fc39

3 years agoMerge "Provide tests for nested labels provisioning"
Eric Multanen [Tue, 15 Sep 2020 03:04:14 +0000 (03:04 +0000)]
Merge "Provide tests for nested labels provisioning"

3 years agoProvide tests for nested labels provisioning 93/112493/3
Konrad Bańka [Thu, 10 Sep 2020 14:19:02 +0000 (16:19 +0200)]
Provide tests for nested labels provisioning

Issue-ID: MULTICLOUD-1179
Signed-off-by: Konrad Bańka <k.banka@samsung.com>
Change-Id: I2f8facefc5299408a47dd9f8bdba9410aa8171de

3 years agoMerge "Use external service address and ports in sanity-check-for-v2.sh."
Eric Multanen [Fri, 11 Sep 2020 20:39:57 +0000 (20:39 +0000)]
Merge "Use external service address and ports in sanity-check-for-v2.sh."

3 years agoUse external service address and ports in sanity-check-for-v2.sh. 07/111807/6
Todd Malsbary [Tue, 18 Aug 2020 22:44:34 +0000 (15:44 -0700)]
Use external service address and ports in sanity-check-for-v2.sh.

This allows it to be used as an automated test.

Issue-ID: MULTICLOUD-1181
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
Change-Id: Ib8636159291243bbb60f974984f296b67f6e397e

3 years agoMerge "Adding validations for all JSON data across modules"
Ritu Sood [Fri, 11 Sep 2020 00:04:32 +0000 (00:04 +0000)]
Merge "Adding validations for all JSON data across modules"

3 years agoMerge "Add log level support for orchestrator"
Eric Multanen [Thu, 10 Sep 2020 23:47:04 +0000 (23:47 +0000)]
Merge "Add log level support for orchestrator"

3 years agoAdding validations for all JSON data across modules 22/111722/4
Rajamohan Raj [Wed, 26 Aug 2020 00:28:12 +0000 (00:28 +0000)]
Adding validations for all JSON data across modules

Issue-ID: MULTICLOUD-1198
Signed-off-by: Rajamohan Raj <rajamohan.raj@intel.com>
Change-Id: Idc2fbfd6e875bd7b662c7d60925aeb776b3f7bea

3 years agoMerge "Provide capability to specify release-name during instantiation"
Eric Multanen [Thu, 10 Sep 2020 13:12:14 +0000 (13:12 +0000)]
Merge "Provide capability to specify release-name during instantiation"

3 years agoMerge "Create monitor deployment yaml file"
Ritu Sood [Thu, 10 Sep 2020 00:49:56 +0000 (00:49 +0000)]
Merge "Create monitor deployment yaml file"

3 years agoCreate monitor deployment yaml file 66/112066/4
Eric Multanen [Wed, 2 Sep 2020 21:27:28 +0000 (14:27 -0700)]
Create monitor deployment yaml file

Combine all resources into a yaml file for deployment
of the monitor program in a cluster.

Issue-ID: MULTICLOUD-1065
Signed-off-by: Eric Multanen <eric.w.multanen@intel.com>
Change-Id: I4b7029bb62d01dc166ad4e25a89f99d139057e0c

3 years agoChanges to add state and retry logic to rsync 52/111052/25
Manjunath Ranganathaiah [Fri, 7 Aug 2020 19:06:22 +0000 (19:06 +0000)]
Changes to add state and retry logic to rsync

- Adds retry watcher and related functionality.
- Adds code to update, get the status from appcontext.
- Adds logic to handle state transition during terminate.

Issue-ID: MULTICLOUD-1005
Signed-off-by: Manjunath Ranganathaiah <manjunath.ranganathaiah@intel.com>
Change-Id: I2ed76efd9d8b6f40fec547bbe8b7d8a86f69ce07

3 years agoMerge "Changes to add state and retry logic to rsync"
Eric Multanen [Thu, 10 Sep 2020 00:05:47 +0000 (00:05 +0000)]
Merge "Changes to add state and retry logic to rsync"

3 years agoAdd log level support for orchestrator 57/111857/3
Rajamohan Raj [Sat, 29 Aug 2020 02:47:00 +0000 (02:47 +0000)]
Add log level support for orchestrator

In this patch, a new config item for log-level
is added, default log-level is set as "warn",
for detailed logs, set log-level as "info"

Issue-ID: MULTICLOUD-1200
Signed-off-by: Rajamohan Raj <rajamohan.raj@intel.com>
Change-Id: I3205ce110a492ecc6a7c680e3d35e173a5624bb0

3 years agoMerge "GET project Returns 404 instead of 500"
Eric Multanen [Wed, 9 Sep 2020 17:54:10 +0000 (17:54 +0000)]
Merge "GET project Returns 404 instead of 500"

3 years agoProvide capability to specify release-name during instantiation 67/112167/3
Konrad Bańka [Fri, 4 Sep 2020 12:44:45 +0000 (14:44 +0200)]
Provide capability to specify release-name during instantiation

Allow release-name property to be provided during instantiation that, if
provided, overrides release-name specified in profile.
Additionally updated Makefile to allow easy compilation with different
go version easily.

Issue-ID: MULTICLOUD-1175
Signed-off-by: Konrad Bańka <k.banka@samsung.com>
Change-Id: Id8db484369045cfb0bc99543a80317644fc838f9

3 years agoUpdate project maturity status 06/112206/1
Bin Yang [Mon, 7 Sep 2020 07:11:38 +0000 (15:11 +0800)]
Update project maturity status

Issue-ID: MULTICLOUD-1193
Signed-off-by: Bin Yang <bin.yang@windriver.com>
Change-Id: I6d5b19f6e2da393b609732397d6903131f2fa50f

3 years agoGET project Returns 404 instead of 500 69/111769/10
Aditya Sharoff [Wed, 26 Aug 2020 17:21:34 +0000 (10:21 -0700)]
GET project Returns 404 instead of 500

Issue-ID: MULTICLOUD-1148

Signed-off-by: Aditya Sharoff <aditya.sharoff@intel.com>
Change-Id: I482cb8ff06ebf945020c5fc185479d9911aa946a

3 years agoMerge "Update CLI with complete get implementation"
Eric Multanen [Wed, 2 Sep 2020 21:28:23 +0000 (21:28 +0000)]
Merge "Update CLI with complete get implementation"

3 years agoUpdate CLI with complete get implementation 56/111856/3
Ritu Sood [Sat, 29 Aug 2020 08:27:53 +0000 (01:27 -0700)]
Update CLI with complete get implementation

CLI updated with get functionality. Also adding
vFw test case

Issue-ID: MULTICLOUD-1065
Signed-off-by: Ritu Sood <ritu.sood@intel.com>
Change-Id: I7bd22aca9fac9cb7b1f4c93d0ffad5b07b62cced

3 years agoUpdate packages images path 65/112065/2
Ritu Sood [Wed, 2 Sep 2020 20:10:57 +0000 (13:10 -0700)]
Update packages images path

Issue-ID: MULTICLOUD-1065
Signed-off-by: Ritu Sood <ritu.sood@intel.com>
Change-Id: I1ae50bafe7b4e40811d5d2061460e1c0240e3dde

3 years agoMerge "Replace invalid literal true with valid string "true" in yaml."
Ritu Sood [Wed, 2 Sep 2020 01:19:16 +0000 (01:19 +0000)]
Merge "Replace invalid literal true with valid string "true" in yaml."

3 years agofix the vfw demo version 38/110638/2
Kuralamudhan Ramakrishnan [Tue, 28 Jul 2020 01:11:10 +0000 (01:11 +0000)]
fix the vfw demo version

Issue-ID: MULTICLOUD-1146
Signed-off-by: Kuralamudhan Ramakrishnan <kuralamudhan.ramakrishnan@intel.com>
Change-Id: I221cfc74809c3179c6bc389b513f20128138d24f

3 years agoMerge "Enhance the status query API"
Ritu Sood [Tue, 1 Sep 2020 21:18:45 +0000 (21:18 +0000)]
Merge "Enhance the status query API"

3 years agoMerge "Fix typos in README.md commands."
Ritu Sood [Tue, 1 Sep 2020 17:00:09 +0000 (17:00 +0000)]
Merge "Fix typos in README.md commands."

3 years agoReplace invalid literal true with valid string "true" in yaml. 56/111256/2
Todd Malsbary [Thu, 13 Aug 2020 22:22:36 +0000 (15:22 -0700)]
Replace invalid literal true with valid string "true" in yaml.

Prior to this change qat_plugin_privileges.yaml fails to kubectl apply
due to a validation error.

Issue-ID: MULTICLOUD-1182
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
Change-Id: Ibe73c1b39d1164fe05ea5cdede74dc93f846c943

3 years agoMerge "Add rsync registration changes to ncm too"
Ritu Sood [Mon, 31 Aug 2020 22:56:13 +0000 (22:56 +0000)]
Merge "Add rsync registration changes to ncm too"

3 years agoEnhance the status query API 01/111501/4
Eric Multanen [Wed, 12 Aug 2020 22:33:12 +0000 (15:33 -0700)]
Enhance the status query API

This patch enhances the status query API.
- The ResourceBundleState CRD is modified to just use the
  k8s Pod structure instead of a customized struct.
- Status queries can either present results showing
  the rsync status of the composite app and resources
  or from information received from the cluster via
  the ResourceBundleState CR
- Query parameters are provided to the API call to
  customize the query and response
- Support for querying status of cluster network
  intents is added

Issue-ID: MULTICLOUD-1042
Signed-off-by: Eric Multanen <eric.w.multanen@intel.com>
Change-Id: Icca4cdd901e2f2b446414fade256fc24d87594cd

3 years agoAdd rsync registration changes to ncm too 15/111915/1
Eric Multanen [Mon, 31 Aug 2020 18:29:22 +0000 (11:29 -0700)]
Add rsync registration changes to ncm too

Changes made to remove dependency of rsyn
registraion from orchestrator need to be added
to ncm as well.

Issue-ID: MULTICLOUD-1196
Signed-off-by: Eric Multanen <eric.w.multanen@intel.com>
Change-Id: Ie572a127fd87cf50a1bb03163d98d3ebe901437e

3 years agoUpdate go.mod for rsync 35/111435/3
Rajamohan Raj [Wed, 19 Aug 2020 19:33:57 +0000 (19:33 +0000)]
Update go.mod for rsync

Issue-ID: MULTICLOUD-1196
Signed-off-by: Rajamohan Raj <rajamohan.raj@intel.com>
Change-Id: Ib5b59d025151f53d64d089a649a5b25f572d2c60

3 years agoRemove the need for rysnc registration in orchestrator 97/111397/3
Rajamohan Raj [Wed, 19 Aug 2020 00:32:30 +0000 (00:32 +0000)]
Remove the need for rysnc registration in orchestrator

Removed dependency of rsync registration
from orchestrator.RSYNC shall have a function NewRsyncInfo to initiate a
new rsync independent of the orchestrator and make
gRPC calls.

Issue-ID: MULTICLOUD-1196
Signed-off-by: Rajamohan Raj <rajamohan.raj@intel.com>
Change-Id: I16bbac6a6865cf3c4ee7b763dac72abe2ed1ad0a