2 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
3 .. http://creativecommons.org/licenses/by/4.0
5 *****************************************
6 Scalability, Resiliency and Manageability
7 *****************************************
12 The new Beijing release scalability, resiliency, and manageablity are described here. These capabilities apply to the OOM/Kubernetes installation.
16 Follow the OOM installation instructions at http://onap.readthedocs.io/en/latest/submodules/oom.git/docs/index.html
18 Overview of the running system
19 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
21 Upon initialization, you should see pools of 4 PDP-Ds and 2 PDP-Xs:
26 kubectl get pods --all-namespaces -o=wid
28 onap dev-brmsgw-5dbc4c8dc4-llk5s 1/1 Running 0 18m 10.42.120.43 k8sx
29 onap dev-drools-0 1/1 Running 0 18m 10.42.60.27 k8sx
30 onap dev-drools-1 1/1 Running 0 16m 10.42.105.190 k8sx
31 onap dev-drools-2 1/1 Running 0 15m 10.42.139.82 k8sx
32 onap dev-drools-3 1/1 Running 0 15m 10.42.128.4 k8sx
33 onap dev-nexus-7d96568f5f-qp5td 1/1 Running 0 18m 10.42.172.8 k8sx
34 onap dev-pap-8587696769-vwj6k 2/2 Running 0 18m 10.42.19.137 k8sx
35 onap dev-pdp-0 2/2 Running 0 18m 10.42.144.218 k8sx
36 onap dev-pdp-1 2/2 Running 0 15m 10.42.233.111 k8sx
37 onap dev-policydb-587d55bdff-4f5dz 1/1 Running 0 18m 10.42.12.242 k8sx
40 and a service for every component:
43 :caption: verify services
45 kubectl get services --all-namespaces
47 onap brmsgw NodePort 10.43.209.173 <none> 9989:30216/TCP 24m
48 onap drools NodePort 10.43.27.92 <none> 6969:30217/TCP,9696:30221/TCP 24m
49 onap nexus NodePort 10.43.19.171 <none> 8081:30236/TCP 24m
50 onap pap NodePort 10.43.9.166 <none> 8443:30219/TCP,9091:30218/TCP 24m
51 onap pdp ClusterIP None <none> 8081/TCP 24m
52 onap policydb ClusterIP None <none> 3306/TCP 24m
54 Config and Decision policy requests will be distributed across PDP-Xs through the *pdp* service. PDP-X clients (such as DCAE) should configure their URLs to go through the *pdp* service. Their requests will be distributed across the available PDP-X replicas. The PDP-Xs can be also accessed individually (dev-pdp-0 and dev-pdp-1 above), but is preferable to that external clients use the service.
56 PDP-Ds are also accessible on a group fashion by using the service IP. Nevertheless, as DMaaP is the main means of communication with other ONAP components, the service interface is not used heavily.
62 Verify that the policy healtcheck passes by the robot framework:
65 :caption: ~/oom/kubernetes/robot/ete-k8s.sh onap health 2> /dev/null | grep PASS
67 Basic Policy Health Check | PASS |
69 A policy healthcheck (with more detailed output) can be done directly to the drools service in the policy VM.
72 :caption: Healtcheck on the PDP-D service
74 curl --silent --user '<username>:<password> -X GET http://localhost:30217/healthcheck | python -m json.tool
90 "url": "http://pap:9091/pap/test"
97 "url": "http://pdp:8081/pdp/test"
104 PDP-X active/active pool
105 ^^^^^^^^^^^^^^^^^^^^^^^^
107 The policy engine UI (console container in the pap pod) can be used to check that the the 2 individual PDP-Xs are synchronized.
108 The console URL is accessible at ``http://<oom-vm>:30219/onap/login.htm``. Select the PDP tab.
110 .. image:: srmPdpxPdpMgmt.png
112 After initialization, there will be no policies loaded into the policy subsystem. You can verify it, by accessing the Editor tab in the UI.
115 PDP-D Active/Active Pool
116 ^^^^^^^^^^^^^^^^^^^^^^^^
118 The PDP-Ds replicas will come up with the amsterdam controller installed in brainless mode (no maven coordinates) since the controller has not been associated with a set of drools rules to run (control loop rules).
120 The following command can be issued on each of the PDP-D replicas IPs:
123 :caption: Querying the rules association for a PDP-D replica
125 curl --silent --user '<username>:<password>' -X GET http://<drools-replica-ip>:9696/policy/pdp/engine/controllers/amsterdam/drools | python -m json.tool
129 "artifactId": "NO-ARTIFACT-ID",
131 "canonicalSessionNames": [],
133 "groupId": "NO-GROUP-ID",
135 "recentSinkEvents": [],
136 "recentSourceEvents": [],
138 "version": "NO-VERSION"
144 The OOM default installation will come with no policies pre-configured. There is a sample script used by integration teams to load policies to support all 4 use cases at: /tmp/policy-install/config/push-policies.sh in the pap container within the pap pod. This script can be modified for your own particular installation, for example if only interested in vCPE use cases, remove those vCPE related API REST calls. For the vFW use case, you may want to edit the encoded operational policy to point to the proper resourceID in your installation.
146 The above mentioned push-policies.sh script can be executed as follows:
149 :caption: Installing the default policies
151 kubectl exec -it dev-pap-8587696769-vwj6k -c pap -n onap -- bash -c "export PRELOAD_POLICIES=true; /tmp/policy-install/config/push-policies.sh"
155 Create BRMSParam Operational Policies
157 Create BRMSParamvFirewall Policy
159 Transaction ID: ef08cc65-9950-4478-a4ab-0f3bc2519f60 --Policy with the name com.Config_BRMS_Param_BRMSParamvFirewall.1.xml was successfully created.Create BRMSParamvDNS Policy
161 Transaction ID: 52e33efe-ba66-47de-b404-8d441107d8a9 --Policy with the name com.Config_BRMS_Param_BRMSParamvDNS.1.xml was successfully created.Create BRMSParamVOLTE Policy
163 Transaction ID: f13072b7-6258-4c16-99da-f908d29363ec --Policy with the name com.Config_BRMS_Param_BRMSParamVOLTE.1.xml was successfully created.Create BRMSParamvCPE Policy
165 Transaction ID: 616f970a-b45e-40f7-88cd-d63000d22cca --Policy with the name com.Config_BRMS_Param_BRMSParamvCPE.1.xml was successfully created.Create MicroService Config Policies
166 Create MicroServicevFirewall Policy
168 Transaction ID: 4c143a15-20af-408a-9285-bc7940261829 --Policy with the name com.Config_MS_MicroServicevFirewall.1.xml was successfully created.Create MicroServicevDNS Policy
170 Transaction ID: 1e54ae73-509b-490e-bf62-1fea7989fd5f --Policy with the name com.Config_MS_MicroServicevDNS.1.xml was successfully created.Create MicroServicevCPE Policy
172 Transaction ID: 32239868-bab2-4e12-9fd9-81a0ed4a6b1c --Policy with the name com.Config_MS_MicroServicevCPE.1.xml was successfully created.Creating Decision Guard policy
174 Transaction ID: b43cb9d5-42c7-4654-aacf-d4898c4d13bb --Policy with the name com.Decision_AllPermitGuard.1.xml was successfully created.Push Decision policy
176 Transaction ID: 3c1e4ae6-6991-415b-9f2d-c665a8c5a026 --Policy 'com.Decision_AllPermitGuard.1.xml' was successfully pushed to the PDP group 'default'.Pushing BRMSParam Operational policies
178 Transaction ID: 58d26d03-b5b8-4fd3-b2df-1411a1c36420 --Policy 'com.Config_BRMS_Param_BRMSParamvFirewall.1.xml' was successfully pushed to the PDP group 'default'.pushPolicy : PUT : com.BRMSParamvDNS
180 Transaction ID: 0854e54a-504b-4f06-bc2f-30f491cb9f5a --Policy 'com.Config_BRMS_Param_BRMSParamvDNS.1.xml' was successfully pushed to the PDP group 'default'.pushPolicy : PUT : com.BRMSParamVOLTE
182 Transaction ID: d33c7dde-5c99-4dab-b4ff-9988473cd88d --Policy 'com.Config_BRMS_Param_BRMSParamVOLTE.1.xml' was successfully pushed to the PDP group 'default'.pushPolicy : PUT : com.BRMSParamvCPE
184 Transaction ID: e8c8a73e-127c-4318-9e59-3cae9dcbe011 --Policy 'com.Config_BRMS_Param_BRMSParamvCPE.1.xml' was successfully pushed to the PDP group 'default'.Pushing MicroService Config policies
186 Transaction ID: ec0429d7-e35f-4978-8a6c-40d2b5b3be61 --Policy 'com.Config_MS_MicroServicevFirewall.1.xml' was successfully pushed to the PDP group 'default'.pushPolicy : PUT : com.MicroServicevDNS
188 Transaction ID: f7072f05-7b74-45b5-9bd3-99b7f8023e3e --Policy 'com.Config_MS_MicroServicevDNS.1.xml' was successfully pushed to the PDP group 'default'.pushPolicy : PUT : com.MicroServicevCPE
190 Transaction ID: 6d47db63-7956-4f5f-ab34-aeb5a124a90d --Policy 'com.Config_MS_MicroServicevCPE.1.xml' was successfully pushed to the PDP group 'default'.
193 The policies pushed can be viewed through the Policy UI:
195 .. image:: srmEditor.png
197 As a consequence of pushing the policies, the brmsgw component will compose drools rules artifacts and publish them to the nexus respository at ``http://<oom-vm>:30236/nexus/``
199 .. image:: srmNexus.png
201 At the same time each replica of the PDP-Ds will receive notifications for each new version of the policies to run for the amsterdam controller. You can run the following command to see how the amsterdam controller is associated with the latest rules version. The following command can be used for verification for each replica:
205 :caption: Querying the rules association of a PDP-D replica
207 curl --silent --user '<username><password> -X GET http://<replica-ip>:9696/policy/pdp/engine/controllers/amsterdam/drools | python -m json.tool
210 "artifactId": "policy-amsterdam-rules",
212 "groupId": "org.onap.policy-engine.drools.amsterdam",
214 "modelClassLoaderHash": 1223551265,
215 "recentSinkEvents": [],
216 "recentSourceEvents": [],
217 "sessionCoordinates": [
218 "org.onap.policy-engine.drools.amsterdam:policy-amsterdam-rules:0.4.0:closedloop-amsterdam"
221 "closedloop-amsterdam"
226 Likewise, for verification purposes, each PDP-X replica can be queried directly to retrieve policy information. The following commands can be used to query a policy through the pdp service:
230 :caption: Querying the "pdp" service for the vFirewal policy
232 ubuntu@k8sx:~$ kubectl exec -it dev-pap-8587696769-vwj6k -c pap -n onap bash
233 policy@dev-pap-8587696769-vwj6k:/tmp/policy-install$ curl --silent -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'ClientAuth: cHl0aG9uOnRlc3Q=' --header 'Authorization: Basic dGVzdHBkcDphbHBoYTEyMw==' --header 'Environment: TEST' -d '{"policyName": ".*vFirewall.*"}' http://pdp:8081/pdp/api/getConfig | python -m json.tool
236 "config": "{\"service\":\"tca_policy\",\"location\":\"SampleServiceLocation\",\"uuid\":\"test\",\"policyName\":\"MicroServicevFirewall\",\"description\":\"MicroService vFirewall Policy\",\"configName\":\"SampleConfigName\",\"templateVersion\":\"OpenSource.version.1\",\"version\":\"1.1.0\",\"priority\":\"1\",\"policyScope\":\"resource=SampleResource,service=SampleService,type=SampleType,closedLoopControlName=ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a\",\"riskType\":\"SampleRiskType\",\"riskLevel\":\"1\",\"guard\":\"False\",\"content\":{\"tca_policy\":{\"domain\":\"measurementsForVfScaling\",\"metricsPerEventName\":[{\"eventName\":\"vFirewallBroadcastPackets\",\"controlLoopSchemaType\":\"VNF\",\"policyScope\":\"DCAE\",\"policyName\":\"DCAE.Config_tca-hi-lo\",\"policyVersion\":\"v0.0.1\",\"thresholds\":[{\"closedLoopControlName\":\"ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a\",\"version\":\"1.0.2\",\"fieldPath\":\"$.event.measurementsForVfScalingFields.vNicUsageArray[*].receivedTotalPacketsDelta\",\"thresholdValue\":300,\"direction\":\"LESS_OR_EQUAL\",\"severity\":\"MAJOR\",\"closedLoopEventStatus\":\"ONSET\"},{\"closedLoopControlName\":\"ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a\",\"version\":\"1.0.2\",\"fieldPath\":\"$.event.measurementsForVfScalingFields.vNicUsageArray[*].receivedTotalPacketsDelta\",\"thresholdValue\":700,\"direction\":\"GREATER_OR_EQUAL\",\"severity\":\"CRITICAL\",\"closedLoopEventStatus\":\"ONSET\"}]}]}}}",
237 "matchingConditions": {
238 "ConfigName": "SampleConfigName",
239 "Location": "SampleServiceLocation",
241 "service": "tca_policy",
244 "policyConfigMessage": "Config Retrieved! ",
245 "policyConfigStatus": "CONFIG_RETRIEVED",
246 "policyName": "com.Config_MS_MicroServicevFirewall.1.xml",
247 "policyType": "MicroService",
248 "policyVersion": "1",
250 "responseAttributes": {},
255 "matchingConditions": {
256 "ConfigName": "BRMS_PARAM_RULE",
259 "policyConfigMessage": "Config Retrieved! ",
260 "policyConfigStatus": "CONFIG_RETRIEVED",
261 "policyName": "com.Config_BRMS_Param_BRMSParamvFirewall.1.xml",
262 "policyType": "BRMS_PARAM",
263 "policyVersion": "1",
265 "responseAttributes": {
266 "controller": "amsterdam"
273 while the following commands could be used to query an specific PDP-X replica:
277 :caption: Querying PDP-X 0 for the vCPE policy
279 curl --silent -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'ClientAuth: cHl0aG9uOnRlc3Q=' --header 'Authorization: Basic dGVzdHBkcDphbHBoYTEyMw==' --header 'Environment: TEST' -d '{"policyName": ".*vCPE.*"}' http://10.42.144.218:8081/pdp/api/getConfig | python -m json.tool
283 "matchingConditions": {
284 "ConfigName": "BRMS_PARAM_RULE",
287 "policyConfigMessage": "Config Retrieved! ",
288 "policyConfigStatus": "CONFIG_RETRIEVED",
289 "policyName": "com.Config_BRMS_Param_BRMSParamvCPE.1.xml",
290 "policyType": "BRMS_PARAM",
291 "policyVersion": "1",
293 "responseAttributes": {
294 "controller": "amsterdam"
299 "config": "{\"service\":\"tca_policy\",\"location\":\"SampleServiceLocation\",\"uuid\":\"test\",\"policyName\":\"MicroServicevCPE\",\"description\":\"MicroService vCPE Policy\",\"configName\":\"SampleConfigName\",\"templateVersion\":\"OpenSource.version.1\",\"version\":\"1.1.0\",\"priority\":\"1\",\"policyScope\":\"resource=SampleResource,service=SampleService,type=SampleType,closedLoopControlName=ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e\",\"riskType\":\"SampleRiskType\",\"riskLevel\":\"1\",\"guard\":\"False\",\"content\":{\"tca_policy\":{\"domain\":\"measurementsForVfScaling\",\"metricsPerEventName\":[{\"eventName\":\"Measurement_vGMUX\",\"controlLoopSchemaType\":\"VNF\",\"policyScope\":\"DCAE\",\"policyName\":\"DCAE.Config_tca-hi-lo\",\"policyVersion\":\"v0.0.1\",\"thresholds\":[{\"closedLoopControlName\":\"ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e\",\"version\":\"1.0.2\",\"fieldPath\":\"$.event.measurementsForVfScalingFields.additionalMeasurements[*].arrayOfFields[0].value\",\"thresholdValue\":0,\"direction\":\"EQUAL\",\"severity\":\"MAJOR\",\"closedLoopEventStatus\":\"ABATED\"},{\"closedLoopControlName\":\"ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e\",\"version\":\"1.0.2\",\"fieldPath\":\"$.event.measurementsForVfScalingFields.additionalMeasurements[*].arrayOfFields[0].value\",\"thresholdValue\":0,\"direction\":\"GREATER\",\"severity\":\"CRITICAL\",\"closedLoopEventStatus\":\"ONSET\"}]}]}}}",
300 "matchingConditions": {
301 "ConfigName": "SampleConfigName",
302 "Location": "SampleServiceLocation",
304 "service": "tca_policy",
307 "policyConfigMessage": "Config Retrieved! ",
308 "policyConfigStatus": "CONFIG_RETRIEVED",
309 "policyName": "com.Config_MS_MicroServicevCPE.1.xml",
310 "policyType": "MicroService",
311 "policyVersion": "1",
313 "responseAttributes": {},
321 A PDP-X container failure can be simulated by performing a"policy.sh stop" operation within the PDP-X container, this in fact will shutdown the PDP-X service. The kubernetes liveness operation will detect that the ports are down, inferring there's a problem with the service, and in turn, will restart the container. In the following example will cause PDP-X 1 to fail.
324 :caption: Causing PDP-X 1 service to fail
326 ubuntu@k8sx:~$ kubectl exec -it dev-pdp-1 --container pdp -n onap -- bash -c "source /opt/app/policy/etc/profile.d/env.sh; policy.sh stop;"
330 Upon detection of the service being down through the liveness check, the container will be restarted. Note the restart count when querying the status of the pods:
333 :caption: Checking PDP-X 1 restart count
335 ubuntu@k8sx:~$ kubectl get pods --all-namespaces -o=wide
337 NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE
339 onap dev-brmsgw-5dbc4c8dc4-llk5s 1/1 Running 0 3d 10.42.120.43 k8sx
340 onap dev-drools-0 1/1 Running 0 3d 10.42.60.27 k8sx
341 onap dev-drools-1 1/1 Running 0 3d 10.42.105.190 k8sx
342 onap dev-drools-2 1/1 Running 0 3d 10.42.139.82 k8sx
343 onap dev-drools-3 1/1 Running 0 3d 10.42.128.4 k8sx
344 onap dev-nexus-7d96568f5f-qp5td 1/1 Running 0 3d 10.42.172.8 k8sx
345 onap dev-pap-8587696769-vwj6k 2/2 Running 0 3d 10.42.19.137 k8sx
346 onap dev-pdp-0 2/2 Running 0 3d 10.42.144.218 k8sx
347 onap dev-pdp-1 2/2 Running 1 3d 10.42.233.111 k8sx <--- **
348 onap dev-policydb-587d55bdff-4f5dz 1/1 Running 0 3d 10.42.12.242 k8sx
351 During the restart process, the PAP component, will detect that PDP-X 1 is down and therefore its state being reflected in the PDP-X screen:
353 .. image:: srmPdpxResiliencyPdpMgmt1.png
355 This screen will be updated to reflect PDP-X 1 is back alive, after PDP-X 1 synchronizes itself with the PAP.
357 .. image:: srmPdpxResiliencyPdpMgmt2.png
359 At that point, PDP-X is usable either directly or through the service to query for policies.
363 :caption: Query PDP-X 1 for vCPE policy
365 ubuntu@k8sx:~$ curl --silent -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'ClientAuth: cHl0aG9uOnRlc3Q=' --header 'Authorization: Basic dGVzdHBkcDphbHBoYTEyMw==' --header 'Environment: TEST' -d '{"policyName": ".*vCPE.*"}' http://10.42.233.111:8081/pdp/api/getConfig | python -m json.tool
369 "matchingConditions": {
370 "ConfigName": "BRMS_PARAM_RULE",
373 "policyConfigMessage": "Config Retrieved! ",
374 "policyConfigStatus": "CONFIG_RETRIEVED",
375 "policyName": "com.Config_BRMS_Param_BRMSParamvCPE.1.xml",
376 "policyType": "BRMS_PARAM",
377 "policyVersion": "1",
379 "responseAttributes": {
380 "controller": "amsterdam"
385 "config": "{\"service\":\"tca_policy\",\"location\":\"SampleServiceLocation\",\"uuid\":\"test\",\"policyName\":\"MicroServicevCPE\",\"description\":\"MicroService vCPE Policy\",\"configName\":\"SampleConfigName\",\"templateVersion\":\"OpenSource.version.1\",\"version\":\"1.1.0\",\"priority\":\"1\",\"policyScope\":\"resource=SampleResource,service=SampleService,type=SampleType,closedLoopControlName=ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e\",\"riskType\":\"SampleRiskType\",\"riskLevel\":\"1\",\"guard\":\"False\",\"content\":{\"tca_policy\":{\"domain\":\"measurementsForVfScaling\",\"metricsPerEventName\":[{\"eventName\":\"Measurement_vGMUX\",\"controlLoopSchemaType\":\"VNF\",\"policyScope\":\"DCAE\",\"policyName\":\"DCAE.Config_tca-hi-lo\",\"policyVersion\":\"v0.0.1\",\"thresholds\":[{\"closedLoopControlName\":\"ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e\",\"version\":\"1.0.2\",\"fieldPath\":\"$.event.measurementsForVfScalingFields.additionalMeasurements[*].arrayOfFields[0].value\",\"thresholdValue\":0,\"direction\":\"EQUAL\",\"severity\":\"MAJOR\",\"closedLoopEventStatus\":\"ABATED\"},{\"closedLoopControlName\":\"ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e\",\"version\":\"1.0.2\",\"fieldPath\":\"$.event.measurementsForVfScalingFields.additionalMeasurements[*].arrayOfFields[0].value\",\"thresholdValue\":0,\"direction\":\"GREATER\",\"severity\":\"CRITICAL\",\"closedLoopEventStatus\":\"ONSET\"}]}]}}}",
386 "matchingConditions": {
387 "ConfigName": "SampleConfigName",
388 "Location": "SampleServiceLocation",
390 "service": "tca_policy",
393 "policyConfigMessage": "Config Retrieved! ",
394 "policyConfigStatus": "CONFIG_RETRIEVED",
395 "policyName": "com.Config_MS_MicroServicevCPE.1.xml",
396 "policyType": "MicroService",
397 "policyVersion": "1",
399 "responseAttributes": {},
407 A PDP-D container failure can be simulated by performing a"policy stop" operation within the PDP-D container, this in fact will shutdown the PDP-D service. The kubernetes liveness operation will detect that the ports are down, inferring there's a problem with the service, and in turn, will restart the container. In the following example will cause PDP-D 3 to fail.
410 :caption: Causing PDP-D 3 to fail
412 ubuntu@k8sx:~/oom/kubernetes$ kubectl exec -it dev-drools-3 --container drools -n onap -- bash -c "source /opt/app/policy/etc/profile.d/env.sh; policy stop"
413 [drools-pdp-controllers]
414 L []: Stopping Policy Management... Policy Management (pid=3284) is stopping... Policy Management has stopped.
417 Upon detection of the service being down through the liveness check, the container will be restarted. Note the restart count when querying the status of the pods:
420 :caption: Checking PDP-D 3 restart count
422 ubuntu@k8sx:~/oom/kubernetes$ kubectl get pods --all-namespaces -o=wide
424 NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE
426 onap dev-brmsgw-5549d99466-7989k 1/1 Running 0 1h 10.42.252.245 k8sx
427 onap dev-drools-0 1/1 Running 0 1h 10.42.30.52 k8sx
428 onap dev-drools-1 1/1 Running 0 1h 10.42.9.245 k8sx
429 onap dev-drools-2 1/1 Running 0 1h 10.42.95.0 k8sx
430 onap dev-drools-3 1/1 Running 1 1h 10.42.224.52 k8sx
431 onap dev-nexus-6558979c95-xlxcc 1/1 Running 0 1h 10.42.142.36 k8sx
432 onap dev-pap-64b67f66b9-lc8vl 2/2 Running 0 1h 10.42.187.255 k8sx
433 onap dev-pdp-0 2/2 Running 0 1h 10.42.164.57 k8sx
434 onap dev-pdp-1 2/2 Running 0 1h 10.42.155.145 k8sx
435 onap dev-policydb-7d4b75869-qd8n5 1/1 Running 0 1h 10.42.148.37 k8sx
441 To scale a new PDP-X, set the replica count appropriately. In our scenario below, we are going to scale the PDP-X with a new replica, PDP-X 2, to have a pool of 3 PDP-X.
444 :caption: Scaling a PDP-X
446 helm upgrade -i dev local/onap --namespace onap --set policy.pdp.replicaCount=3
448 Release "dev" has been upgraded. Happy Helming!
449 LAST DEPLOYED: Mon May 14 01:37:03 2018
454 kubectl get pods --all-namespaces -o=wide
456 NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE
458 onap dev-pdp-0 2/2 Running 0 1h 10.42.164.57 k8sx
459 onap dev-pdp-1 2/2 Running 0 1h 10.42.155.145 k8sx
460 onap dev-pdp-2 2/2 Running 0 1m 10.42.47.58 k8sx
467 To scale a new PDP-D, set the replica count appropriately. In our scenario below, we are going to scale the PDP-D service with a new replica, PDP-D 4, to have a pool of 5 PDP-D.
470 :caption: Scaling a PDP-D
472 helm upgrade -i dev local/onap --namespace onap --set policy.drools.replicaCount=5
473 Release "dev" has been upgraded. Happy Helming!
474 LAST DEPLOYED: Mon May 14 01:45:19 2018
478 ubuntu@k8sx:~/oom/kubernetes$ kubectl get pods --all-namespaces -o=wide
479 NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE
481 onap dev-drools-0 1/1 Running 0 1h 10.42.30.52 k8sx
482 onap dev-drools-1 1/1 Running 0 1h 10.42.9.245 k8sx
483 onap dev-drools-2 1/1 Running 0 1h 10.42.95.0 k8sx
484 onap dev-drools-3 1/1 Running 1 1h 10.42.224.52 k8sx
485 onap dev-drools-4 1/1 Running 0 1m 10.42.237.251 k8sx
493 .. SSNote: Wiki page ref. https://wiki.onap.org/display/DW/Scalability%2C+Resiliency+and+Manageability