1 VF Module Scale Out Use Case
2 ----------------------------
7 - Base VNF template file: https://git.onap.org/demo/plain/heat/vLBMS/base_vlb.yaml
8 - Base VNF environment file: https://git.onap.org/demo/plain/heat/vLBMS/base_vlb.env
10 - VF module scaling template file: https://git.onap.org/demo/plain/heat/vLBMS/dnsscaling.yaml
11 - VF module scaling environment file: https://git.onap.org/demo/plain/heat/vLBMS/dnsscaling.env
15 The Scale Out use case shows how users/network operators can add capacity to an existing VNF. ONAP Casablanca release supports scale out with manual trigger from VID and closed-loop enabled automation from Policy. This is demonstrated against the vLB/vDNS VNFs. For Casablanca, both APPC and SDNC controllers are supported. APPC is the official controller used for this use case and it can be used to scale multiple VNF types. SDNC is experimental for now and it can scale only the vDNS VNF developed for ONAP.
17 The Casablanca Scaling Use Case Wiki Page can be found here: https://wiki.onap.org/display/DW/Scaling+Use+Case+Extension
21 Scaling VF modules manually requires the user/network operator to trigger the scale out operation from the VID portal. VID translates the operation into a call to SO. Scaling VF modules in an automated manner instead requires the user/network operator to design and deploy a closed loop for scale out that includes policies (e.g. threshold-crossing conditions), guard policies that determine when it's safe to scale out, and microservices that analyze events coming from the network in order to discover situations.
23 Both manual and automated scale out activate the scale out workflow in the Service Orchestrator (SO). The workflow runs as follows:
25 - SO sends a request to APPC to run health check against the VNF;
26 - If the VNF is healthy, SO instantiates a new VF module and sends a request to APPC to reconfigure the VNF;
27 - APPC reconfigures the VNF, without interrupting the service;
28 - SO sends a request to APPC to run health check against the VNF again, to validate that the scale out operation didn't impact the running VNF.
30 The vLB has a Northbound API that allows an upstream system (e.g. ONAP) to change the internal configuration by updating the list of active vDNS instances (i.e. VNF reconfiguration). The Northbound API framework has been built using FD.io-based Honeycomb 1707, and supports both RESTconf and NETCONF protocols. Below is an example of vDNS instances contained in the vLB, in JSON format:
34 "vlb-business-vnf-onap-plugin": {
38 "ip-addr": "192.168.10.211",
39 "oam-ip-addr": "10.0.150.2",
47 The parameters required for VNF reconfiguration (i.e. "ip-addr", "oam-ip-addr", and "enabled" in case of vLB/vDNS) can be specified in the VID GUI when triggering the workflow manually or in CLAMP when designing a closed loop for the automated case. In both cases, the format used for specifying the parameters and their values is a JSON path. SO will use the provided paths to access parameters' name and value in the VF module preload received from SDNC before instantiating a new VF module.
49 VID accepts a JSON array in the "Configuration Parameter" box (see later), for example:
52 [{"ip-addr":"$.vf-module-topology.vf-module-parameters.param[10].value","oam-ip-addr":"$.vf-module-topology.vf-module-parameters.param[15].value","enabled":"$.vf-module-topology.vf-module-parameters.param[22].value"}]
54 CLAMP, instead, accepts a YAML file in the "Payload" box in the Policy Creation form, for example:
57 requestParameters: '{"usePreload":true,"userParams":[]}'
58 configurationParameters: '[{"ip-addr":"$.vf-module-topology.vf-module-parameters.param[10].value","oam-ip-addr":"$.vf-module-topology.vf-module-parameters.param[15].value","enabled":"$.vf-module-topology.vf-module-parameters.param[22].value"}]'
60 Note that Policy requires an additional object, called "requestParameters" in which "usePreload" should be set to true and the "userParams" array should be left empty.
62 The JSON path to the parameters used for VNF reconfiguration, including array locations, should be set as described above. Finally, although the VNF supports to update multiple vDNS records in the same call, for Casablanca release APPC updates a single vDNS instance at a time.
64 When using APPC, before running scale out, the user needs to create a VNF template using the Controller Design Tool (CDT), a design-time tool that allows users to create and on-board VNF templates into the APPC. The template describes which control operation can be executed against the VNF (e.g. scale out, health check, modify configuration, etc.), the protocols that the VNF supports, port numbers, VNF APIs, and credentials for authentication. Being VNF agnostic, APPC uses these templates to "learn" about specific VNFs and the supported operations.
65 CDT requires two input:
67 1) the list of parameters that APPC will receive (ip-addr, oam-ip-addr, enabled in the example above);
69 2) the VNF API that APPC will use to reconfigure the VNF.
71 Below is an example of the parameters file (yaml format), which we call parameters.yaml:
104 Here is an example of API for the vLB VNF used for this use case. We name the file after the vnf-type contained in SDNC (i.e. Vloadbalancerms..dnsscaling..module-1):
107 <vlb-business-vnf-onap-plugin xmlns="urn:opendaylight:params:xml:ns:yang:vlb-business-vnf-onap-plugin">
110 <ip-addr>${ip-addr}</ip-addr>
111 <oam-ip-addr>${oam-ip-addr}</oam-ip-addr>
112 <enabled>${enabled}</enabled>
115 </vlb-business-vnf-onap-plugin>
117 To create the VNF template in CDT, the following steps are required:
119 - Connect to the CDT GUI: http://APPC-IP:8080 (in Heat-based ONAP deployments) or http://ANY-K8S-IP:30289 (in OOM/K8S-based ONAP deployments)
120 - Click "My VNF" Tab. Create your user ID, if necessary
121 - Click "Create new VNF" entering the VNF type as reported in VID or AAI, e.g. vLoadBalancerMS/vLoadBalancerMS 0
122 - Select "ConfigScaleOut" action
123 - Create a new template identifier using the vnf-type name in SDNC as template name, e.g. Vloadbalancerms..dnsscaling..module-1
124 - Select protocol (Netconf-XML), VNF username (admin), and VNF port number (2831 for NETCONF)
125 - Click "Parameter Definition" Tab and upload the parameters (.yaml) file
126 - Click "Template Tab" and upload API template (.yaml) file
127 - Click "Reference Data" Tab
128 - Click "Save All to APPC"
130 For health check operation, we just need to specify the protocol, the port number and username of the VNF (REST, 8183, and "admin" respectively, in the case of vLB/vDNS) and the API. For the vLB/vDNS, the API is:
133 restconf/operational/health-vnf-onap-plugin:health-vnf-onap-plugin-state/health-check
135 Note that we don't need to create a VNF template for health check, so the "Template" flag can be set to "N". Again, the user has to click "Save All to APPC" to update the APPC database.
136 At this time, CDT doesn't allow users to provide VNF password from the GUI. To update the VNF password we need to log into the APPC Maria DB container and change the password manually:
139 mysql -u sdnctl -p (type "gamma" when password is prompted)
141 UPDATE DEVICE_AUTHENTICATION SET PASSWORD='admin' WHERE
142 VNF_TYPE='vLoadBalancerMS/vLoadBalancerMS 0'; (use your VNF type)
144 To trigger the scale out workflow manually, the user/network operator can log into VID from the ONAP Portal (demo/demo123456! as username/password), select "VNF Changes" and then the "New (+)" button. The user/network operator needs to fill in the "VNF Change Form" by selecting Subscriber, Service Type, NF Role, Model Version, VNF, Scale Out from the Workflow dropdown window, and insert the JSON path array described above in the "Configuration Parameter" box. After clicking "Next", in the following window the user/network operator has to select the VF Module to scale by clicking on the VNF and then on the appropriate VF Module checkbox. Finally, by clicking on the "Schedule" button, the scale out use case will run as described above.
146 Automated scale out requires the user to onboard a DCAE blueprint in SDC when creating the service. To design a closed loop for scale out, the user needs to access the CLAMP GUI (https://clamp.api.simpledemo.onap.org:30258/designer/index.html) and execute the following operations:
148 - Click the "Closed loop" dropdown window and select "Open CL"
149 - Select the closed loop model and click "OK"
150 - In the next screen, click the "Policy" box to create a policy for closed loop, including guard policies
151 - After creating the policies, click "TCA" and review the blueprint uploaded during service creation and distributed by SDC to CLAMP
152 - Click the "Manage" dropdown and then "Submit" to push the policies to the Policy Engine
153 - From the same "Manage" dropdown, click "Deploy" to deploy the TCA blueprint to DCAE
155 The vLB/vDNS VNF generates traffic and reports metrics to the VES collector in the DCAE platform. The number of incoming packets to the vLB is used to evaluate the policy defined for closed loop. If the provided threshold is crossed, DCAE generates an event that reaches the Policy Engine, which in turn activates the scale out closed loop described above.
157 For more information about scale out, videos, and material used for running the use case, please look at the wiki page: https://wiki.onap.org/display/DW/Running+Scale+Out+Use+Case+for+Casablanca
159 Test Status and Plans
160 ~~~~~~~~~~~~~~~~~~~~~
161 Casablanca Scale Out completed all tests as found here: https://wiki.onap.org/pages/viewpage.action?pageId=36964241#UseCaseTracking(CasablancaScaling)-Testing
163 Known Issues and Resolutions
164 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
165 1) When running closed loop-enabled scale out, the closed loop designed in CLAMP conflicts with the default closed loop defined for the old vLB/vDNS use case
167 Resolution: Change TCA configuration for the old vLB/vDNS use case
169 - Connect to Consul: http://<ANY K8S VM IP ADDRESS>:30270 and click on "Key/Value" → "dcae-tca-analytics"
170 - Change "eventName" in the vLB default policy to something different, for example "vLB" instead of the default value "vLoadBalancer"
171 - Change "subscriberConsumerGroup" in the TCA configuration to something different, for example "OpenDCAE-c13" instead of the default value "OpenDCAE-c12"
172 - Click "UPDATE" to upload the new TCA configuration
174 2) When running closed loop-enabled scale out, the permitAll guard policy conflicts with the scale out guard policy
176 Resolution: Undeploy the permitAll guard policy
178 - Connect to the Policy GUI, either through the ONAP Portal (https://portal.api.simpledemo.onap.org:30225/ONAPPORTAL/login.htm) or directly (https://policy.api.simpledemo.onap.org:30219/onap/login.htm)
179 - If using the ONAP Portal, use demo/demo123456! as credentials, otherwise, if accessing Policy GUI directly, use demo/demo
180 - Click "Policy" → "Push" on the left panel
181 - Click the pencil symbol next to "default" in the PDP Groups table
182 - Select "Decision_AllPermitGuard"
185 3) When using SDNC, the JSON template for DMaaP messages has a wrong "input" field
187 Resolution: Replace the "input" field in the JSON template with "output"
189 - Connect to the SDNC container from the Rancher VM in the Kubernetes cluster, for example
193 kubectl exec -it -n onap dev-sdnc-sdnc-0 bash
195 - Install your favorite text editor, for example
199 apt-get update; apt-get install vim -y
201 - Open /opt/onap/sdnc/restapi/templates/lcm-dmaap-publish-template.json and replace "input" with "output" in the JSON object body
203 - If SDNC is deployed in cluster mode (3 SDNC replicas, dev-sdnc-sdnc-0, dev-sdnc-sdnc-1, dev-sdnc-sdnc-2), apply the same change to all the replicas in the cluster.