From: zhang ab Date: Sun, 25 Mar 2018 07:05:28 +0000 (+0000) Subject: Merge "Spec for elastic API exposure" X-Git-Tag: v1.1.2~49 X-Git-Url: https://gerrit.onap.org/r/gitweb?p=multicloud%2Fframework.git;a=commitdiff_plain;h=84973dec870bbc33625dd641b5486afa2d8b1d4e;hp=589deee5d3c321a20131e03a580cc367d49ebf2e Merge "Spec for elastic API exposure" --- diff --git a/.gitignore b/.gitignore index 495cfc8..9787425 100644 --- a/.gitignore +++ b/.gitignore @@ -5,6 +5,7 @@ target/ logs/*.log *.pyc +*.swp # Test related files multivimbroker/.coverage diff --git a/docs/specs/multicloud_event_federation.rst b/docs/specs/multicloud_event_federation.rst new file mode 100644 index 0000000..02de57e --- /dev/null +++ b/docs/specs/multicloud_event_federation.rst @@ -0,0 +1,145 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. Copyright (c) 2017-2018 VMware, Inc. + +================= +Event/Alert/Metrics Federation +================= + +As a cloud mediation layer, Multicloud could be invoked by many projects, through this feature, Multicloud will provide +VM status/events check and also can customize the type of event which user would like to receive. There are some +kinds of VM status can be chosen: DELETE, PAUSE, POWER_OFF, REBUILD，SHUT_DOWN, SOFT_DELETE, etc.. In VMware VIO Plugin, +once any change of VM status is detected of a given type, Multicloud will catch the event and throw it to DMaaP. +Other projects can try this way of getting VM status messages in the future. Also, for other Multicloud plugin providers, +due to some issues, there will be rest apis for them to grab the VM status messages. + +The APP-C won't be impacted since APP-C can still call the existing API which implemented in Amsterdam Release + and the API is an existing API + +Use Cases +=================== + +In VIO, one typical use case is to allow VIO users to fetch messages from DMaaP, this will provide a easier way for fetching status of +VMs, it may drastically reduce the time of close loop, for other Multicloud plugin providers, Multicloud will provide a set of +rest apis to get them + + +Proposed change +=================== + +In VIO plugin: + +The proposed change will include two parts: * listener: to listen the events of the status change of VM, for others it +will have rest apis to get the messages * publisher: to throw the event to DMaaP.The message we try to send is something like this: +{ + "state_description": "powering-off", + "availability_zone": "nova", + "terminated_at": "", + "ephemeral_gb": 0, + "instance_type_id": 5, + "deleted_at": "", + "reservation_id": "r-pvx3l6s2", + "memory_mb": 2048, + "display_name": "VM1", + "hostname": "vm1", + "state": "active", + "progress": "", + "launched_at": "2018-03-07T05:59:46.000000", + "metadata": {}, + "node": "domain-c202.22bfc05c-da55-4ba6-ba93-08d9a067138e", + "ramdisk_id": "", + "access_ip_v6": null, + "disk_gb": 20, + "access_ip_v4": null, + "kernel_id": "", + "host": "compute01", + "user_id": "aa90efa5c84c4084b39094da952e0bd1", + "image_ref_url": "http://10.154.9.172:9292/images/207b9b7c-9450-4a95-852b-0d6d41f35d24", + "cell_name": "", + "root_gb": 20, + "tenant_id": "943ecb804cdf4103976b8a02cea12fdb", + "created_at": "2018-03-07 05:58:01+00:00", + "instance_id": "4f398943-7d39-4119-8058-74768d6dfa52", + "instance_type": "m1.small", + "vcpus": 1, + "image_meta": { + "is_copying": "1", + "container_format": "bare", + "min_ram": "0", + "vmware_disktype": "streamOptimized", + "disk_format": "vmdk", + "source_type": "url", + "image_url": "https://cloud-images.ubuntu.com/releases/14.04/release/ubuntu-14.04-server-cloudimg-amd64-disk1.img", + "vmware_adaptertype": "lsiLogic", + "min_disk": "20", + "base_image_ref": "207b9b7c-9450-4a95-852b-0d6d41f35d24" + }, + "architecture": null, + "os_type": null, + "instance_flavor_id": "2" +} + +The eventual work flow looks like as follows: + + +------------------+ + | | + | Multicloud | + | Broker | + | | + +---------+--------+ + | + | + V + +-----------------------+ +------------------+ + | Multicloud VIO Plugin |----------->| Dmaap | + | | Event | | + +--------|-----^--------+ +------------------+ + Oslo | | + Listener | | + V | + +----------------------+ + | VIO | + +----------------------+ + + +In Other Plugins: + +Since the security rules of VIMs and network connectivity issues, other multicloud plugins won't be suitable for the +oslo notification listener, so we will provide rest apis for them, the specific implementation will be dicided by them + +Input of /check_vim_status will be + +:: + { + "states": array // the set of VIM status which user wants to get + } + +Output of check_vim_status will be + +:: + { + "state_description": string // VIM's state + "launched_at": string // time of state change + } + +The work flow looks like as follows: + + +------------------+ + | | + | Multicloud | + | Broker | + | | + +---------+--------+ + | + | + V + +-----------------------+ + | Multicloud Plugins | + | | + +--------|-----^--------+ + polling | | + or other way | | + V | + +----------------------+ + | Openstack | + +----------------------+ diff --git a/docs/specs/multicloud_image_service.rst b/docs/specs/multicloud_image_service.rst new file mode 100644 index 0000000..728d389 --- /dev/null +++ b/docs/specs/multicloud_image_service.rst @@ -0,0 +1,117 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. Copyright (c) 2017-2018 VMware, Inc. + + +================= +Image Service +================= + +Because Multicloud provides a cloud mediation layer supporting multiple clouds. It's necessary to +introduces some function enhancements in it. Image Service could let user upload/download images +in a convinient way just by using Multicloud. + + +Problem Description +=================== + +The original functions which Multicloud possesses are to use urls to upload images, while in this +spec we intend to upload images as raw file which means it has to store a copy in Multicloud then +upload the images to the backend openstack. So this spec is to extend multicloud to support +download/upload images as raw file rather than a through a url + + +Use Cases +=================== + +One typical use case is to allow users to upload/download images by Multicloud + + +Proposed change +=================== + +The proposed change mainly means introducing glance python APIs to enable multicloud support openstack +image service. This feature needs two changes: Upload API to import an image to backend OpenStack +and the image that just imported can be queried from MultiCloud. Download API to download an image +from backend Openstack and the image can be downloaded from MultiCloud. + +The eventual work flow looks like as follows: + + user request to upload image + | + V + +------------------+ + | | + | image file(iso, | + | vmdk... ) | + | | + +---------+--------+ + | + | + | + +-----------|----------+ + | multicloud| | + | V | + | +------------------+ | + | | image service API| | + | +---------+--------+ | + +-----------|----------+ + | glance + | + V + +----------------------+ + | openstack | + +----------------------+ + +The APIs look like this: + +upload: + +Input of /{vimid}/{tenantid}/images/file will be + +:: + required: image file + { + "imageType": string, // image type: ami, ari, aki, vhd, vhdx, vmdk, raw, qcow2, vdi, iso + "containerFormat": string, // image container format: ami, ari, aki, bare, ovf, ova, docker + "visibility": string, // public, private, shared, or community + "properties": arrary // list of properties + } + +Output of upload_image will be + +:: + "responses": { + "201": { + "description": "upload successfully", + }, + "404": { + "description": "the vim id or tenant UUID is wrong" + }, + "500": { + "description": "the vim image is not accessable" + } + +download: + +Input of /{vimid}/{tenantid}/images/file/{imageid} will be + +:: + { + "imagepath": string, // the path of the downloaded image + "properties": arrary // list of properties + } + +Output of download_image will be + +:: + "responses": { + "200": { + "description": "download successfully", + }, + "404": { + "description": "the vim id or tenant UUID is wrong" + }, + "500": { + "description": "the vim image is not accessable" + } diff --git a/docs/specs/parallelism_improvement.rst b/docs/specs/parallelism_improvement.rst new file mode 100644 index 0000000..86f39d8 --- /dev/null +++ b/docs/specs/parallelism_improvement.rst @@ -0,0 +1,199 @@ +.. + This work is licensed under a Creative Commons Attribution 4.0 + International License. + +=============================================== +Parallelism improvement of Multi Cloud Services +=============================================== + + +Problem Description +=================== + +Multi-Cloud runs Django by using Django's built-in webserver currently. +According to Django Document[Django_Document]_, this mode should not be used +in production. This mode has not gone through security audits or performance +tests, and should only be used in development. From test on local computer, +this mode can only handle ONE API request at one time. This can not meet the +performance requirement. + +.. [Django_Document] https://docs.djangoproject.com/en/dev/ref/django-admin/#runserver + +Although security and scalability might be improved as the side effect of +resolving the performance issue, this spec will only focus on how to improve +the parallelism(performance) of current MultiCloud API framework. + +Possible Solutions +================== + +Solution 1 +---------- + +Django is a mature framework. And it has its own way to improve parallelism. +Instead of running Django's build-in webserver, Django APP can be deployed in +some dedicated web server. Django’s primary deployment platform is WSGI[django_deploy]_, +the Python standard for web servers and applications. + +.. [django_deploy] https://docs.djangoproject.com/en/2.0/howto/deployment/wsgi/ + + +But on the other side, Danjgo is very huge. And Django is a black box if one +doesn't have good knowledge of it. Adding feature based on Django may be +time-consuming. For example, the unit test[unit_test]_ of Multi-Cloud can't use +regular python test library because of Django. The unit test has to base on +Django's test framework. When we want to improve the parallelism of Multi-Cloud +services, we need to find out how Django can implement it, instead of using some +common method. + +.. [unit_test] https://gerrit.onap.org/r/#/c/8909/ + +Besides, Django's code pattern is too much like web code. And, most famous use +cases of Django are web UI. Current code of Multi-Cloud puts many logic in +files named `views.py`, but actually there is no view to expose. It is confusing. + +The benefit of this solution is that most current code needs no change. + +Solution 2 +---------- + +Given the fact that Django has shortcomings to move on, this solution propose +to use a alternative framework. Eventlet[Eventlet]_ with Pecan[Pecan]_ will be the +idea web framework in this case, because it is lightweight, lean and widely +used. + +.. [Eventlet] http://eventlet.net/doc/modules/wsgi.html + +.. [Pecan] https://pecan.readthedocs.io/en/latest/ + +For example, most OpenStack projects use such framework. This framework is so +thin that it can provide flexibility for future architecture design. + +However, it needs to change existing code of API exposing. + + +Performance Test Comparison +=========================== + +Test Environment +---------------- + +Apache Benchmark is used as test tool. It is shipped with Ubuntu, if you +don’t find it, just run “sudo apt install -y apache2-utils” + +2 Virtual Machine with Ubuntu1604. Virtual Machines are hosted in a multi-core +hardware server. One VM is for Apache Benchmark. This VM is 1 CPU core, 8G mem. +The other VM is for Multicloud. The VM is 4 CPU core, 6G mem. + +Test Command +~~~~~~~~~~~~ + +`ab -n -c http:///api/multicloud/v0/vim_types` + +Test result +----------- + +It should be noted that data may vary in different test run, but overall result is +similar as below. + +100 requests, concurrency level 1 +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Command: `ab -n 100 -c 1 http:///api/multicloud/v0/vim_types` +Result: + Django runserver: total takes 0.512 seconds, all requests success + Django+uwsgi: totally takes 0.671 seconds, all requests success. + Pecan+eventlet: totally takes 0.149 seconds, all requests success. + +10000 requests, concurrency level 100 +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Command: `ab -n 10000 -c 100 http:///api/multicloud/v0/vim_types` +Result: + Django runserver: total takes 85.326 seconds, all requests success + Django+uwsgi: totally takes 3.808 seconds, all requests success. + Pecan+eventlet: totally takes 3.181 seconds, all requests success. + +100000 requests, concurrency level 1000 +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Command: `ab -n 100000 -c 1000 http:///api/multicloud/v0/vim_types` +Result: + Django runserver: Apache Benchmark quit because it reports timeout after + running a random portion of all requests. + Django+uwsgi: totally takes 37.316 seconds, about 32% requests fail. I see + some error says that tcp socket open too many. + Pecan+eventlet: totally takes 35.315 seconds, all requests success. + +Proposed Change +=============== + +Given the test result above, this spec proposes to use solution 2. Based on +the consideration of Elastic API exposure[jira_workitem]_, Multi-Cloud will +provide a new way to expose its API. That is to say, existing code of API +exposing needs rewrite in [jira_workitem]_. So the disadvantage of solution +2 doesn't exist. + +.. [jira_workitem] https://jira.onap.org/browse/MULTICLOUD-152 + +To define a clear scope of this spec, VoLTE is the use case that will be used +to perform test to this spec. All functionality that VoLTE needed should be +implemented in this spec and [jira_workitem]_. + +Backward compatibility +---------------------- + +This spec will NOT change current API. This spec will NOT replace the current +API framework in R2, nor will switch to new API framework in R2. Instead, +this spec will provide a configuration option, named `web_framework`, to make +sure use case and functionalities not be broken. Default value of the +configuration will BE `django`, which will still run current Django API +framework. An alternative value is `pecan`, which will run the API framework +proposed in this spec. So users don't care about the change won't be +affected. + +WSGI Server +----------- + +No matter what API framework will be used, a WSGI Server needs to be provided. +This spec will use Eventlet WSGI server. API framework will be run as an +application in WSGI server. + +Multi processes framework +------------------------- + +This spec proposes to run Multi-Cloud API server in multiple processes mode. +Multi-process can provide parallel API handlers. So, when multiple API +requests come to Multi-Cloud, they can be handled simultaneously. On the other +hand, different processes can effectively isolate different API request. So +that, one API request will not affect another. + +Managing multiple processes could be overwhelming difficult and sometimes +dangerous. Some mature library could be used to reduce related work here, for +example oslo.service[oslo_service]_. Since oslo is used by all OpenStack +projects for many releases, and oslo project is actively updated, it can be +seen as a stable library. + +.. [oslo_service] https://github.com/openstack/oslo.service + +Number of processes +~~~~~~~~~~~~~~~~~~~ + +To best utilize multi-core CPU, the number of processes will be set to the +number of CPU cores by default. + +Shared socket file +~~~~~~~~~~~~~~~~~~ + +To make multiple processes work together and provide a unified port number, +multiple processes need to share a socket file. To achieve this, a bootstrap +process will be started and will initialize the socket file. Other processes +can be forked from this bootstrap process. + +Work Items +========== + +#. Add WSGI server. +#. Run Pecan application in WSGI server. +#. Add multiple processes support. +#. Update deploy script to support new API framework. + diff --git a/multivimbroker/multivimbroker/api_v2/__init__.py b/multivimbroker/multivimbroker/api_v2/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/multivimbroker/multivimbroker/api_v2/api_router/__init__.py b/multivimbroker/multivimbroker/api_v2/api_router/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/multivimbroker/multivimbroker/api_v2/api_router/root.py b/multivimbroker/multivimbroker/api_v2/api_router/root.py new file mode 100644 index 0000000..0f98e93 --- /dev/null +++ b/multivimbroker/multivimbroker/api_v2/api_router/root.py @@ -0,0 +1,27 @@ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from pecan import rest + +from multivimbroker.api_v2.api_router import v0_controller + + +class MultiCloudController(rest.RestController): + v0 = v0_controller.V0_Controller() + + +class APIController(rest.RestController): + multicloud = MultiCloudController() + + +class RootController(object): + api = APIController() diff --git a/multivimbroker/multivimbroker/api_v2/api_router/v0_controller.py b/multivimbroker/multivimbroker/api_v2/api_router/v0_controller.py new file mode 100644 index 0000000..99c1b08 --- /dev/null +++ b/multivimbroker/multivimbroker/api_v2/api_router/v0_controller.py @@ -0,0 +1,93 @@ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import logging +import pecan + +from multivimbroker.pub import exceptions +from multivimbroker.pub.utils import restcall +from multivimbroker.pub.utils import syscomm +from multivimbroker.swagger import utils + + +logger = logging.getLogger(__name__) + +# TODO: Move to a constant file. +REGISTRY_URI = "registry" +UNREGISTRY_URI = "" +IDENTITY_URI = "identity/v3" +IDENTITY_AUTH_URI = "identity/v3/auth/tokens" + + +class V0_Controller(object): + + @pecan.expose('json') + def vim_types(self): + return syscomm.getVIMTypes() + + @pecan.expose('json', route="swagger.json") + def swagger_json(self): + return utils.get_swagger_json_data() + + def _filter_illegal_uri(self, uri, method): + """ + Filter unsupported actions, so they can be stopped at begginning. + """ + + if uri == REGISTRY_URI and method != "POST": + pecan.abort(405) + + if uri == UNREGISTRY_URI and method != "DELETE": + pecan.abort(405) + + if (uri in (IDENTITY_URI, IDENTITY_AUTH_URI) and + method not in ("POST", "GET")): + pecan.abort(405) + + @pecan.expose() + def _route(self, remainder, request): + uri = "/".join(remainder[1:]) + method = request.method + self._filter_illegal_uri(uri, method) + + return self.forwarder, remainder + + @pecan.expose('json') + def forwarder(self, *remainder, **kwargs): + """ Forward any requests that don't have a specific match """ + + # TODO(xiaohhui): Add validator for vim_id. + vim_id = remainder[0] + request = pecan.request + try: + vim_url = syscomm.getMultivimDriver(vim_id, + full_path=request.path) + + # NOTE: Not sure headers should be set here. According to original + # code, headers are discarded. + retcode, content, status_code, resp = restcall.req_by_msb( + vim_url, request.method, content=request.body) + except exceptions.NotFound as e: + pecan.abort(404, detail=str(e)) + except Exception as e: + pecan.abort(500, detail=str(e)) + + if retcode: + # Execptions are handled within req_by_msb + logger.error("Status code is %s, detail is %s.", + status_code, content) + response = pecan.Response(body=content, status=status_code) + + for k in syscomm.getHeadersKeys(resp): + response.headers[k] = resp[k] + + return response diff --git a/multivimbroker/multivimbroker/api_v2/app.py b/multivimbroker/multivimbroker/api_v2/app.py new file mode 100644 index 0000000..86777ee --- /dev/null +++ b/multivimbroker/multivimbroker/api_v2/app.py @@ -0,0 +1,33 @@ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import pecan + + +def setup_app(config=None): + app_conf = { + 'root': "multivimbroker.api_v2.api_router.root.RootController", + 'modules': ["multivimbroker.api_v2"], + 'debug': True, + # NOTE: By default, guess_content_type_from_ext is True, and Pecan will + # strip the file extension from url. For example, ../../swagger.json + # will look like ../../swagger to Pecan API router. This makes other + # url like ../../swagger.txt get the same API route. Set this to False + # to do strict url mapping. + 'guess_content_type_from_ext': False + } + app = pecan.make_app( + app_conf.pop('root'), + **app_conf + ) + + return app diff --git a/multivimbroker/multivimbroker/api_v2/service.py b/multivimbroker/multivimbroker/api_v2/service.py new file mode 100644 index 0000000..228cc63 --- /dev/null +++ b/multivimbroker/multivimbroker/api_v2/service.py @@ -0,0 +1,52 @@ +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +from oslo_concurrency import processutils +from oslo_config import cfg +from oslo_service import service +from oslo_service import wsgi + +from multivimbroker.api_v2 import app + + +CONF = cfg.CONF + + +class WSGIService(service.ServiceBase): + """Provides ability to launch API from wsgi app.""" + + def __init__(self): + self.app = app.setup_app() + + self.workers = processutils.get_worker_count() + + self.server = wsgi.Server( + CONF, + "multivimbroker", + self.app, + # TODO(xiaohhui): these should be configurable. + host="0.0.0.0", + port="9002", + use_ssl=False + ) + + def start(self): + self.server.start() + + def stop(self): + self.server.stop() + + def wait(self): + self.server.wait() + + def reset(self): + self.server.reset() diff --git a/multivimbroker/multivimbroker/forwarder/urls.py b/multivimbroker/multivimbroker/forwarder/urls.py index 6a60df4..771f052 100644 --- a/multivimbroker/multivimbroker/forwarder/urls.py +++ b/multivimbroker/multivimbroker/forwarder/urls.py @@ -16,6 +16,7 @@ from django.conf.urls import url from rest_framework.urlpatterns import format_suffix_patterns +from multivimbroker.forwarder.views import CheckCapacity from multivimbroker.forwarder.views import Extension from multivimbroker.forwarder.views import Forward from multivimbroker.forwarder.views import Identity @@ -27,6 +28,8 @@ from multivimbroker.forwarder.views import VIMTypes urlpatterns = [ url(r'^api/multicloud/v0/vim_types$', VIMTypes.as_view()), + url(r'^api/multicloud/v0/check_vim_capacity$', + CheckCapacity.as_view()), url(r'^api/multicloud/v0/(?P[0-9a-zA-Z_-]+)/identity/v3$', Identity.as_view()), url(r'^api/multicloud/v0/(?P[0-9a-zA-Z_-]+)/identity/v3' diff --git a/multivimbroker/multivimbroker/forwarder/views.py b/multivimbroker/multivimbroker/forwarder/views.py index 7935642..d1763c2 100644 --- a/multivimbroker/multivimbroker/forwarder/views.py +++ b/multivimbroker/multivimbroker/forwarder/views.py @@ -13,15 +13,14 @@ # See the License for the specific language governing permissions and # limitations under the License. -import os import json from rest_framework.views import APIView from rest_framework.views import Response from rest_framework.views import status from multivimbroker.forwarder.base import BaseHandler - -# +from multivimbroker.pub.utils.syscomm import originHeaders +from multivimbroker.pub.utils import syscomm class BaseServer(BaseHandler, APIView): @@ -50,18 +49,21 @@ class Identity(BaseServer): def get(self, request, vimid): - return self.send(vimid, request.get_full_path(), request.body, "GET") + return self.send(vimid, request.get_full_path(), request.body, "GET", + headers=originHeaders(request)) def post(self, request, vimid): - return self.send(vimid, request.get_full_path(), request.body, "POST") + return self.send(vimid, request.get_full_path(), request.body, "POST", + headers=originHeaders(request)) class Registry(BaseServer): def post(self, request, vimid): - return self.send(vimid, request.get_full_path(), request.body, "POST") + return self.send(vimid, request.get_full_path(), request.body, "POST", + headers=originHeaders(request)) class UnRegistry(BaseServer): @@ -69,30 +71,52 @@ class UnRegistry(BaseServer): def delete(self, request, vimid): return self.send(vimid, request.get_full_path(), request.body, - "DELETE") + "DELETE", headers=originHeaders(request)) class Extension(BaseServer): def get(self, request, vimid): - return self.send(vimid, request.get_full_path(), request.body, "GET") + return self.send(vimid, request.get_full_path(), request.body, "GET", + headers=originHeaders(request)) class VIMTypes(BaseServer): def get(self, request): - # Fix here unless we have plugin registry - json_file = os.path.join(os.path.dirname(__file__), - '../pub/config/provider-plugin.json') - with open(json_file, "r") as f: - plugins = json.load(f) - ret = [] - for k, v in plugins.items(): - item = {} - item["vim_type"] = v.get("vim_type") - item["versions"] = [k for k in v.get('versions', {})] - ret.append(item) + return Response(data=syscomm.getVIMTypes(), status=status.HTTP_200_OK) + + +class CheckCapacity(BaseServer): + + def post(self, request): + try: + body = json.loads(request.body) + except ValueError as e: + return Response( + data={'error': 'Invalidate request body %s.' % e}, + status=status.HTTP_400_BAD_REQUEST) + + ret = {"VIMs": []} + newbody = { + "vCPU": body.get("vCPU", 0), + "Memory": body.get("Memory", 0), + "Storage": body.get("Storage", 0) + } + for vim in body.get("VIMs", []): + url = request.get_full_path().replace( + "check_vim_capacity", "%s/capacity_check" % vim) + resp = self.send(vim, url, newbody, "POST") + if resp.status_code != status.HTTP_200_OK: + continue + try: + resp_body = json.loads(resp.body) + except ValueError: + continue + if not resp_body.get("result", False): + continue + ret['VIMs'].append(vim) return Response(data=ret, status=status.HTTP_200_OK) diff --git a/multivimbroker/multivimbroker/middleware.py b/multivimbroker/multivimbroker/middleware.py new file mode 100644 index 0000000..5b320a3 --- /dev/null +++ b/multivimbroker/multivimbroker/middleware.py @@ -0,0 +1,64 @@ +# Copyright (c) 2017-2018 VMware, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + + +import uuid +from onaplogging.mdcContext import MDC +from multivimbroker.pub.config.config import SERVICE_NAME +from multivimbroker.pub.config.config import FORWARDED_FOR_FIELDS + + +class LogContextMiddleware(object): + + # the last IP behind multiple proxies, if no exist proxies + # get local host ip. + def _getLastIp(self, request): + + ip = "" + try: + for field in FORWARDED_FOR_FIELDS: + if field in request.META: + if ',' in request.META[field]: + parts = request.META[field].split(',') + ip = parts[-1].strip().split(":")[0] + else: + ip = request.META[field].split(":")[0] + + if ip == "": + ip = request.META.get("HTTP_HOST").split(":")[0] + + except Exception: + pass + + return ip + + def process_request(self, request): + + # Fetch TRANSACTIONID Id and pass to plugin server + ReqeustID = request.META.get("HTTP_X_TRANSACTIONID", None) + if ReqeustID is None: + ReqeustID = uuid.uuid3(uuid.NAMESPACE_URL, SERVICE_NAME) + request.META["HTTP_X_TRANSACTIONID"] = ReqeustID + MDC.put("requestID", ReqeustID) + # generate the unique id + InovocationID = uuid.uuid3(uuid.NAMESPACE_DNS, SERVICE_NAME) + MDC.put("invocationID", InovocationID) + MDC.put("serviceName", SERVICE_NAME) + # access ip + MDC.put("serviceIP", self._getLastIp(request)) + + return None + + def process_response(self, request, response): + + MDC.clear() + return response diff --git a/multivimbroker/multivimbroker/pub/config/config.py b/multivimbroker/multivimbroker/pub/config/config.py index 8fba115..192c743 100644 --- a/multivimbroker/multivimbroker/pub/config/config.py +++ b/multivimbroker/multivimbroker/pub/config/config.py @@ -24,26 +24,15 @@ MSB_SERVICE_PORT = '10080' AAI_ADDR = "aai.api.simpledemo.openecomp.org" AAI_PORT = "8443" AAI_SERVICE_URL = 'https://%s:%s/aai' % (AAI_ADDR, AAI_PORT) -AAI_SCHEMA_VERSION = "v11" +AAI_SCHEMA_VERSION = "v13" AAI_USERNAME = 'AAI' AAI_PASSWORD = 'AAI' +# [MDC] +SERVICE_NAME = "multicloud-broker" +FORWARDED_FOR_FIELDS = ["HTTP_X_FORWARDED_FOR", "HTTP_X_FORWARDED_HOST", + "HTTP_X_FORWARDED_SERVER"] + # [IMAGE LOCAL PATH] ROOT_PATH = os.path.dirname( os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) - -# [register] -REG_TO_MSB_WHEN_START = False -REG_TO_MSB_REG_URL = "/api/microservices/v1/services" -REG_TO_MSB_REG_PARAM = { - "serviceName": "multicloud", - "version": "v0", - "url": "/api/multicloud/v0", - "protocol": "REST", - "visualRange": "1", - "nodes": [{ - "ip": "127.0.0.1", - "port": "9001", - "ttl": 0 - }] -} diff --git a/multivimbroker/multivimbroker/pub/config/log.yml b/multivimbroker/multivimbroker/pub/config/log.yml index 12da69f..09be40d 100644 --- a/multivimbroker/multivimbroker/pub/config/log.yml +++ b/multivimbroker/multivimbroker/pub/config/log.yml @@ -12,14 +12,14 @@ handlers: class: "logging.handlers.RotatingFileHandler" filename: "/var/log/onap/multicloud/multivimbroker/multivimbroker.log" formatter: "mdcFormat" - maxBytes: 1024*1024*50 + maxBytes: 52428800 backupCount: 10 formatters: standard: - format: "%(asctime)s:[%(name)s]:[%(filename)s]-[%(lineno)d] [%(levelname)s]:%(message)s" + format: "%(asctime)s|||||%(name)s||%(thread)||%(funcName)s||%(levelname)s||%(message)s" mdcFormat: - format: "%(asctime)s:[%(name)s]:[%(filename)s]-[%(lineno)d] [%(levelname)s]:[%(mdc)s]: %(message)s" - mdcfmt: "{requestID}" + format: "%(asctime)s|||||%(name)s||%(thread)s||%(funcName)s||%(levelname)s||%(message)s||||%(mdc)s \t" + mdcfmt: "{requestID} {invocationID} {serviceName} {serviceIP}" datefmt: "%Y-%m-%d %H:%M:%S" (): onaplogging.mdcformatter.MDCFormatter diff --git a/multivimbroker/multivimbroker/pub/utils/syscomm.py b/multivimbroker/multivimbroker/pub/utils/syscomm.py index 0b2b557..337a1bd 100644 --- a/multivimbroker/multivimbroker/pub/utils/syscomm.py +++ b/multivimbroker/multivimbroker/pub/utils/syscomm.py @@ -36,6 +36,14 @@ def getHeadersKeys(response): return [header for header in response.keys() if header not in hopbyhop] +# trim out 'HTTP_' prefix part and replace "_" wiht "-". +def originHeaders(request): + regex = re.compile('^HTTP_') + return dict((regex.sub('', header).replace("_", "-"), value) + for (header, value) in request.META.items() + if header.startswith('HTTP_')) + + def findMultivimDriver(vim=None): json_file = os.path.join(os.path.dirname(__file__), '../config/provider-plugin.json') @@ -54,3 +62,19 @@ def getMultivimDriver(vimid, full_path=""): vim = get_vim_by_id(vimid) multclouddriver = findMultivimDriver(vim=vim) return re.sub(multcloud, multclouddriver, full_path) + + +def getVIMTypes(): + # Fix here unless we have plugin registry + json_file = os.path.join(os.path.dirname(__file__), + '../config/provider-plugin.json') + with open(json_file, "r") as f: + plugins = json.load(f) + ret = [] + for k, v in plugins.items(): + item = {} + item["vim_type"] = v.get("vim_type") + item["versions"] = [k for k in v.get('versions', {})] + ret.append(item) + + return ret diff --git a/multivimbroker/multivimbroker/scripts/__init__.py b/multivimbroker/multivimbroker/scripts/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/multivimbroker/multivimbroker/scripts/api.py b/multivimbroker/multivimbroker/scripts/api.py new file mode 100644 index 0000000..01b69a2 --- /dev/null +++ b/multivimbroker/multivimbroker/scripts/api.py @@ -0,0 +1,35 @@ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +import eventlet +eventlet.monkey_patch() + +from oslo_config import cfg # noqa +from oslo_service import service # noqa +import sys # noqa + +from multivimbroker.api_v2 import service as api_service # noqa + + +def main(): + try: + api_server = api_service.WSGIService() + launcher = service.launch(cfg.CONF, + api_server, + workers=api_server.workers) + launcher.wait() + except RuntimeError as excp: + sys.stderr.write("ERROR: %s\n" % excp) + sys.exit(1) + + +if __name__ == '__main__': + main() diff --git a/multivimbroker/multivimbroker/settings.py b/multivimbroker/multivimbroker/settings.py index dca2dd6..c1d31d0 100644 --- a/multivimbroker/multivimbroker/settings.py +++ b/multivimbroker/multivimbroker/settings.py @@ -26,9 +26,9 @@ BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) SECRET_KEY = '3o-wney!99y)^h3v)0$j16l9=fdjxcb+a8g+q3tfbahcnu2b0o' # SECURITY WARNING: don't run with debug turned on in production! -DEBUG = True +# DEBUG = True -ALLOWED_HOSTS = [] +ALLOWED_HOSTS = ['*'] # Application definition @@ -51,6 +51,7 @@ MIDDLEWARE_CLASSES = [ 'django.contrib.auth.middleware.SessionAuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', + 'multivimbroker.middleware.LogContextMiddleware', ] ROOT_URLCONF = 'multivimbroker.urls' @@ -96,7 +97,6 @@ config.yamlConfig(filepath=LOGGING_FILE, watchDog=True) if 'test' in sys.argv: from multivimbroker.pub.config import config - config.REG_TO_MSB_WHEN_START = False DATABASES = {} DATABASES['default'] = { 'ENGINE': 'django.db.backends.sqlite3', diff --git a/multivimbroker/multivimbroker/swagger/utils.py b/multivimbroker/multivimbroker/swagger/utils.py new file mode 100644 index 0000000..05c92da --- /dev/null +++ b/multivimbroker/multivimbroker/swagger/utils.py @@ -0,0 +1,94 @@ +# Copyright (c) 2017 Wind River Systems, Inc. +# Copyright (c) 2017-2018 VMware, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + +import json +import os + + +def get_swagger_json_data(): + json_file = os.path.join(os.path.dirname(__file__), + 'multivim.flavor.swagger.json') + f = open(json_file) + json_data = json.JSONDecoder().decode(f.read()) + f.close() + json_file = os.path.join(os.path.dirname(__file__), + 'multivim.image.swagger.json') + f = open(json_file) + json_data_temp = json.JSONDecoder().decode(f.read()) + f.close() + json_data["paths"].update(json_data_temp["paths"]) + json_data["definitions"].update(json_data_temp["definitions"]) + json_file = os.path.join(os.path.dirname(__file__), + 'multivim.network.swagger.json') + f = open(json_file) + json_data_temp = json.JSONDecoder().decode(f.read()) + f.close() + json_data["paths"].update(json_data_temp["paths"]) + json_data["definitions"].update(json_data_temp["definitions"]) + json_file = os.path.join(os.path.dirname(__file__), + 'multivim.subnet.swagger.json') + f = open(json_file) + json_data_temp = json.JSONDecoder().decode(f.read()) + f.close() + json_data["paths"].update(json_data_temp["paths"]) + json_data["definitions"].update(json_data_temp["definitions"]) + json_file = os.path.join(os.path.dirname(__file__), + 'multivim.server.swagger.json') + f = open(json_file) + json_data_temp = json.JSONDecoder().decode(f.read()) + f.close() + json_data["paths"].update(json_data_temp["paths"]) + json_data["definitions"].update(json_data_temp["definitions"]) + json_file = os.path.join(os.path.dirname(__file__), + 'multivim.volume.swagger.json') + f = open(json_file) + json_data_temp = json.JSONDecoder().decode(f.read()) + f.close() + json_data["paths"].update(json_data_temp["paths"]) + json_data["definitions"].update(json_data_temp["definitions"]) + json_file = os.path.join(os.path.dirname(__file__), + 'multivim.vport.swagger.json') + f = open(json_file) + json_data_temp = json.JSONDecoder().decode(f.read()) + f.close() + json_data["paths"].update(json_data_temp["paths"]) + json_data["definitions"].update(json_data_temp["definitions"]) + json_file = os.path.join(os.path.dirname(__file__), + 'multivim.tenant.swagger.json') + f = open(json_file) + json_data_temp = json.JSONDecoder().decode(f.read()) + f.close() + json_data["paths"].update(json_data_temp["paths"]) + json_data["definitions"].update(json_data_temp["definitions"]) + json_file = os.path.join(os.path.dirname(__file__), + 'multivim.host.swagger.json') + f = open(json_file) + json_data_temp = json.JSONDecoder().decode(f.read()) + f.close() + json_data["paths"].update(json_data_temp["paths"]) + json_data["definitions"].update(json_data_temp["definitions"]) + json_file = os.path.join(os.path.dirname(__file__), + 'multivim.limit.swagger.json') + f = open(json_file) + json_data_temp = json.JSONDecoder().decode(f.read()) + f.close() + json_data["paths"].update(json_data_temp["paths"]) + json_data["definitions"].update(json_data_temp["definitions"]) + json_file = os.path.join(os.path.dirname(__file__), + 'multivim.identity.swagger.json') + f = open(json_file) + json_data_temp = json.JSONDecoder().decode(f.read()) + f.close() + json_data["paths"].update(json_data_temp["paths"]) + json_data["definitions"].update(json_data_temp["definitions"]) + + return json_data diff --git a/multivimbroker/multivimbroker/swagger/views.py b/multivimbroker/multivimbroker/swagger/views.py index 00cf297..6897270 100644 --- a/multivimbroker/multivimbroker/swagger/views.py +++ b/multivimbroker/multivimbroker/swagger/views.py @@ -10,96 +10,14 @@ # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -import json -import logging -import os -# import traceback # from rest_framework import status from rest_framework.response import Response from rest_framework.views import APIView -# from multivimbroker.pub.exceptions import VimBrokerException - -logger = logging.getLogger(__name__) +from multivimbroker.swagger import utils class SwaggerJsonView(APIView): def get(self, request): - json_file = os.path.join(os.path.dirname(__file__), - 'multivim.flavor.swagger.json') - f = open(json_file) - json_data = json.JSONDecoder().decode(f.read()) - f.close() - json_file = os.path.join(os.path.dirname(__file__), - 'multivim.image.swagger.json') - f = open(json_file) - json_data_temp = json.JSONDecoder().decode(f.read()) - f.close() - json_data["paths"].update(json_data_temp["paths"]) - json_data["definitions"].update(json_data_temp["definitions"]) - json_file = os.path.join(os.path.dirname(__file__), - 'multivim.network.swagger.json') - f = open(json_file) - json_data_temp = json.JSONDecoder().decode(f.read()) - f.close() - json_data["paths"].update(json_data_temp["paths"]) - json_data["definitions"].update(json_data_temp["definitions"]) - json_file = os.path.join(os.path.dirname(__file__), - 'multivim.subnet.swagger.json') - f = open(json_file) - json_data_temp = json.JSONDecoder().decode(f.read()) - f.close() - json_data["paths"].update(json_data_temp["paths"]) - json_data["definitions"].update(json_data_temp["definitions"]) - json_file = os.path.join(os.path.dirname(__file__), - 'multivim.server.swagger.json') - f = open(json_file) - json_data_temp = json.JSONDecoder().decode(f.read()) - f.close() - json_data["paths"].update(json_data_temp["paths"]) - json_data["definitions"].update(json_data_temp["definitions"]) - json_file = os.path.join(os.path.dirname(__file__), - 'multivim.volume.swagger.json') - f = open(json_file) - json_data_temp = json.JSONDecoder().decode(f.read()) - f.close() - json_data["paths"].update(json_data_temp["paths"]) - json_data["definitions"].update(json_data_temp["definitions"]) - json_file = os.path.join(os.path.dirname(__file__), - 'multivim.vport.swagger.json') - f = open(json_file) - json_data_temp = json.JSONDecoder().decode(f.read()) - f.close() - json_data["paths"].update(json_data_temp["paths"]) - json_data["definitions"].update(json_data_temp["definitions"]) - json_file = os.path.join(os.path.dirname(__file__), - 'multivim.tenant.swagger.json') - f = open(json_file) - json_data_temp = json.JSONDecoder().decode(f.read()) - f.close() - json_data["paths"].update(json_data_temp["paths"]) - json_data["definitions"].update(json_data_temp["definitions"]) - json_file = os.path.join(os.path.dirname(__file__), - 'multivim.host.swagger.json') - f = open(json_file) - json_data_temp = json.JSONDecoder().decode(f.read()) - f.close() - json_data["paths"].update(json_data_temp["paths"]) - json_data["definitions"].update(json_data_temp["definitions"]) - json_file = os.path.join(os.path.dirname(__file__), - 'multivim.limit.swagger.json') - f = open(json_file) - json_data_temp = json.JSONDecoder().decode(f.read()) - f.close() - json_data["paths"].update(json_data_temp["paths"]) - json_data["definitions"].update(json_data_temp["definitions"]) - # - json_file = os.path.join(os.path.dirname(__file__), - 'multivim.identity.swagger.json') - f = open(json_file) - json_data_temp = json.JSONDecoder().decode(f.read()) - f.close() - json_data["paths"].update(json_data_temp["paths"]) - json_data["definitions"].update(json_data_temp["definitions"]) - return Response(json_data) + return Response(utils.get_swagger_json_data()) diff --git a/multivimbroker/multivimbroker/tests/test_check_capacity.py b/multivimbroker/multivimbroker/tests/test_check_capacity.py new file mode 100644 index 0000000..60035e0 --- /dev/null +++ b/multivimbroker/multivimbroker/tests/test_check_capacity.py @@ -0,0 +1,93 @@ +# Copyright (c) 2017-2018 VMware, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + +import mock +import unittest + +from rest_framework import status + +from multivimbroker.forwarder.views import CheckCapacity + + +class CheckCapacityTest(unittest.TestCase): + + def setUp(self): + self.view = CheckCapacity() + super(CheckCapacityTest, self).setUp() + + def tearDown(self): + pass + + def test_check_capacity_success(self): + req = mock.Mock() + req.body = """ + { + "vCPU": 1, + "Memory": 1, + "Storage": 500, + "VIMs": ["openstack_RegionOne"] + }""" + req.get_full_path.return_value = ("http://msb.onap.org/api/multicloud" + "/v0/check_vim_capacity") + with mock.patch.object(self.view, "send") as send: + plugin_resp = mock.Mock() + plugin_resp.body = """{ + "result": true + }""" + plugin_resp.status_code = status.HTTP_200_OK + send.return_value = plugin_resp + + resp = self.view.post(req) + expect_body = { + "VIMs": ["openstack_RegionOne"] + } + self.assertEqual(status.HTTP_200_OK, resp.status_code) + self.assertDictEqual(expect_body, resp.data) + + def test_check_capacity_no_suitable_vim(self): + req = mock.Mock() + req.body = """ + { + "vCPU": 1, + "Memory": 1, + "Storage": 500, + "VIMs": ["openstack_RegionOne"] + }""" + req.get_full_path.return_value = ("http://msb.onap.org/api/multicloud" + "/v0/check_vim_capacity") + with mock.patch.object(self.view, "send") as send: + plugin_resp = mock.Mock() + plugin_resp.body = """{ + "result": false + }""" + plugin_resp.status_code = status.HTTP_200_OK + send.return_value = plugin_resp + + resp = self.view.post(req) + expect_body = { + "VIMs": [] + } + self.assertEqual(status.HTTP_200_OK, resp.status_code) + self.assertDictEqual(expect_body, resp.data) + + def test_check_capacity_invalid_input(self): + req = mock.Mock() + req.body = "hello world" + req.get_full_path.return_value = ("http://msb.onap.org/api/multicloud" + "/v0/check_vim_capacity") + expect_body = { + "error": ("Invalidate request body " + "No JSON object could be decoded.") + } + resp = self.view.post(req) + self.assertEqual(status.HTTP_400_BAD_REQUEST, resp.status_code) + self.assertDictEqual(expect_body, resp.data) diff --git a/multivimbroker/multivimbroker/tests/test_restcall.py b/multivimbroker/multivimbroker/tests/test_restcall.py new file mode 100644 index 0000000..b76b0a6 --- /dev/null +++ b/multivimbroker/multivimbroker/tests/test_restcall.py @@ -0,0 +1,24 @@ +# Copyright (c) 2017-2018 VMware, Inc. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + +import unittest + +from multivimbroker.pub.utils import restcall + + +class TestRestCall(unittest.TestCase): + + def test_combine_url(self): + url = ["http://a.com/test/", "http://a.com/test/", + "http://a.com/test"] + res = ["/resource", "resource", "/resource"] + expected = "http://a.com/test/resource" + for i in range(len(url)): + self.assertEqual(expected, restcall.combine_url(url[i], res[i])) diff --git a/multivimbroker/multivimbroker/tests/test_urls.py b/multivimbroker/multivimbroker/tests/test_vim_types.py similarity index 55% rename from multivimbroker/multivimbroker/tests/test_urls.py rename to multivimbroker/multivimbroker/tests/test_vim_types.py index 7990033..b8a5080 100644 --- a/multivimbroker/multivimbroker/tests/test_urls.py +++ b/multivimbroker/multivimbroker/tests/test_vim_types.py @@ -8,20 +8,19 @@ # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -import json import mock import unittest -from multivimbroker.pub.config import config -from multivimbroker import urls +from rest_framework.views import status + +from multivimbroker.forwarder.views import VIMTypes class TestUrls(unittest.TestCase): + def setUp(self): + self.view = VIMTypes() - def test_request_msb(self): - with mock.patch("multivimbroker.pub.utils.restcall." - "req_by_msb") as req_by_msb: - urls.req_msb(True) - req_by_msb.assert_called_once_with( - config.REG_TO_MSB_REG_URL, "POST", - json.JSONEncoder().encode(config.REG_TO_MSB_REG_PARAM)) + def test_vim_types_success(self): + resp = self.view.get(mock.Mock()) + self.assertEqual(status.HTTP_200_OK, resp.status_code) + self.assertEqual(2, len(resp.data)) diff --git a/multivimbroker/multivimbroker/urls.py b/multivimbroker/multivimbroker/urls.py index 0843bb3..c393210 100644 --- a/multivimbroker/multivimbroker/urls.py +++ b/multivimbroker/multivimbroker/urls.py @@ -11,23 +11,8 @@ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. from django.conf.urls import include, url -import json - -from multivimbroker.pub.config import config - urlpatterns = [ url(r'^', include('multivimbroker.swagger.urls')), url(r'^', include('multivimbroker.forwarder.urls')), ] - - -def req_msb(request_when_start): - # regist to MSB when startup - if request_when_start: - from multivimbroker.pub.utils.restcall import req_by_msb - req_by_msb(config.REG_TO_MSB_REG_URL, "POST", - json.JSONEncoder().encode(config.REG_TO_MSB_REG_PARAM)) - - -req_msb(config.REG_TO_MSB_WHEN_START) diff --git a/multivimbroker/requirements.txt b/multivimbroker/requirements.txt index 1378612..5aadc57 100644 --- a/multivimbroker/requirements.txt +++ b/multivimbroker/requirements.txt @@ -24,4 +24,15 @@ mock==2.0.0 unittest_xml_reporting==1.12.0 # for onap logging -onappylog>=1.0.5 \ No newline at end of file +onappylog>=1.0.6 + +# for pecan framework +pecan>=1.2.1 +oslo.concurrency>=3.21.0 +oslo.config>=4.11.0 +oslo.service>=1.25.0 +eventlet>=0.20.0 + +# uwsgi for parallel processing +uwsgi + diff --git a/multivimbroker/run.sh b/multivimbroker/run.sh index 35f2b9e..8270deb 100755 --- a/multivimbroker/run.sh +++ b/multivimbroker/run.sh @@ -26,10 +26,15 @@ if [ ! -x $logDir ]; then mkdir -p $logDir fi -nohup python manage.py runserver 0.0.0.0:9001 2>&1 & +if [ "$WEB_FRAMEWORK" == "pecan" ] +then + python multivimbroker/scripts/api.py +else + # nohup python manage.py runserver 0.0.0.0:9001 2>&1 & + nohup uwsgi --http :9001 --module multivimbroker.wsgi --master --processes 4 & -while [ ! -f $logDir/multivimbroker.log ]; do - sleep 1 -done + while [ ! -f $logDir/multivimbroker.log ]; do + sleep 1 + done -tail -F $logDir/multivimbroker.log + tail -F $logDir/multivimbroker.log diff --git a/multivimbroker/stop.sh b/multivimbroker/stop.sh index 4a2e5c3..ba0a2c8 100755 --- a/multivimbroker/stop.sh +++ b/multivimbroker/stop.sh @@ -11,4 +11,5 @@ # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -ps auxww | grep 'manage.py runserver 0.0.0.0:9001' | awk '{print $2}' | xargs kill -9 +# ps auxww | grep 'manage.py runserver 0.0.0.0:9001' | awk '{print $2}' | xargs kill -9 +ps auxww |grep 'uwsgi --http :9001 --module multivimbroker.wsgi --master' |awk '{print $2}' |xargs kill -9 diff --git a/pom.xml b/pom.xml index 4c6b8fb..bdd5736 100644 --- a/pom.xml +++ b/pom.xml @@ -18,7 +18,7 @@ org.onap.oparent oparent - 0.1.1 + 1.1.0 ../oparent 4.0.0