1 .. This work is licensed under a
2 .. Creative Commons Attribution 4.0 International License.
3 .. http://creativecommons.org/licenses/by/4.0
10 Policy Drools PDP component
11 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
13 Both the Performance and the Stability tests were executed against an ONAP installation in the policy-k8s tenant
14 in the windriver lab, from an independent VM running the jmeter tool to inject the load.
19 The installation runs the following components in a single VM:
26 The VM has the following hardware spec:
30 - 155GB Ephemeral Disk
32 Jmeter is run from a different VM with the following configuration:
36 - 155GB Ephemeral Disk
38 The drools-pdp container uses the JVM memory settings from a default OOM installation.
40 Other ONAP components exercised during the stability tests were:
42 - Policy XACML PDP to process guard queries for each transaction.
43 - DMaaP to carry PDP-D and jmeter initiated traffic to complete transactions.
44 - Policy API to create (and delete at the end of the tests) policies for each
46 - Policy PAP to deploy (and undeploy at the end of the tests) policies for each scenario under test.
48 The following components are simulated during the tests.
50 - SO actor for the vDNS use case.
51 - APPC responses for the vCPE and vFW use cases.
52 - AAI to answer queries for the use cases under test.
54 SO, and AAI actors were simulated within the PDP-D JVM by enabling the
55 feature-controlloop-utils before running the tests.
60 The kubernetes charts were modified previous to the installation
61 to add the following script that enables the controlloop-utils feature:
65 oom/kubernetes/policy/charts/drools/resources/configmaps/features.pre.sh:
68 sh -c "features enable controlloop-utils"
70 Stability Test of Policy PDP-D
71 ******************************
76 The tests focused on the following use cases:
82 For 72 hours the following 5 scenarios ran in parallel:
84 - vCPE success scenario
85 - vCPE failure scenario (failure returned by simulated APPC recipient through DMaaP).
86 - vDNS success scenario.
87 - vDNS failure scenario (failure by introducing in the DCAE ONSET a non-existent vserver-name reference).
88 - vFirewall success scenario.
90 Five threads ran in parallel, one for each scenario, back to back with no pauses. The transactions were initiated
91 by each jmeter thread group. Each thread initiated a transaction, monitored the transaction, and
92 as soon as the transaction ending was detected, it initiated the next one.
94 JMeter was run in a docker container with the following command:
98 docker run --interactive --tty --name jmeter --rm --volume $PWD:/jmeter -e VERBOSE_GC="" egaillardon/jmeter-plugins --nongui --testfile s3p.jmx --loglevel WARN
100 The results were accessed by using the telemetry API to gather statistics:
103 vCPE Success scenario
104 =====================
106 ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e:
110 # Times are in milliseconds
112 Control Loop Name: ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e
113 Number of Transactions Executed: 114007
114 Number of Successful Transactions: 112727
115 Number of Failure Transactions: 1280
116 Average Execution Time: 434.9942021103967 ms.
119 vCPE Failure scenario
120 =====================
122 ControlLoop-vCPE-Fail:
126 # Times are in milliseconds
128 Control Loop Name: ControlLoop-vCPE-Fail
129 Number of Transactions Executed: 114367
130 Number of Successful Transactions: 114367 (failure transactions are expected)
131 Number of Failure Transactions: 0 (success transactions are not expected)
132 Average Execution Time: 433.61750330077734 ms.
135 vDNS Success scenario
136 =====================
138 ControlLoop-vDNS-6f37f56d-a87d-4b85-b6a9-cc953cf779b3:
142 # Times are in milliseconds
144 Control Loop Name: ControlLoop-vDNS-6f37f56d-a87d-4b85-b6a9-cc953cf779b3
145 Number of Transactions Executed: 237512
146 Number of Successful Transactions: 229532
147 Number of Failure Transactions: 7980
148 Average Execution Time: 268.028794334602 ms.
151 vDNS Failure scenario
152 =====================
154 ControlLoop-vDNS-Fail:
158 # Times are in milliseconds
160 Control Loop Name: ControlLoop-vDNS-Fail
161 Number of Transactions Executed: 1957987
162 Number of Successful Transactions: 1957987 (failure transactions are expected)
163 Number of Failure Transactions: 0 (success transactions are not expected)
164 Average Execution Time: 39.369322166081794
167 vFirewall Success scenario
168 ==========================
170 ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a:
174 # Times are in milliseconds
176 Control Loop Name: ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a
177 Number of Transactions Executed: 120308
178 Number of Successful Transactions: 118895
179 Number of Failure Transactions: 1413
180 Average Execution Time: 394.8609236293513 ms.
186 There has been a degradation of performance observed in this release
187 when compared with the previous one.
188 Approximately 1% of transactions were not completed as expected for
189 some use cases. Average Execution Times are extended as well.
190 The unexpected results seem to point in the direction of the
191 interactions of the distributed locking feature with the database.
192 These areas as well as the conditions for the test need to be investigated
197 # Common pattern in the audit.log for unexpected transaction completions
199 a8d637fc-a2d5-49f9-868b-5b39f7befe25||ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a|
200 policy:usecases:[org.onap.policy.drools-applications.controlloop.common:controller-usecases:1.9.0:usecases]|
201 2021-10-12T19:48:02.052+00:00|2021-10-12T19:48:02.052+00:00|0|
202 null:operational.modifyconfig.EVENT.MANAGER.FINAL:1.0.0|dev-policy-drools-pdp-0|
203 ERROR|400|Target Lock was lost|||VNF.generic-vnf.vnf-name||dev-policy-drools-pdp-0||
204 dev-policy-drools-pdp-0|microservice.stringmatcher|
205 {vserver.prov-status=ACTIVE, vserver.is-closed-loop-disabled=false,
206 generic-vnf.vnf-name=fw0002vm002fw002, vserver.vserver-name=OzVServer}||||
207 INFO|Session org.onap.policy.drools-applications.controlloop.common:controller-usecases:1.9.0:usecases|
209 # The "Target Lock was lost" is a common message error in the unexpected results.