.. toctree::
:maxdepth: 2
-Policy XACML PDP component
##########################
-Both the Performance and the Stability tests were executed by performing requests
-against the Policy RESTful APIs residing on the XACML PDP installed in the windriver
-lab. This was running on a kubernetes pod having the following configuration:
+Performance Test of Policy XACML PDP
+************************************
+
+The Performance test was executed by performing requests
+against the Policy RESTful APIs residing on the XACML PDP installed on a Cloud based Virtual Machine.
+VM Configuration:
- 16GB RAM
- 8 VCPU
-- 160GB Disk
-
-Both tests were run via jmeter, which was installed on a separate VM so-as not
-to impact the performance of the XACML-PDP being tested.
+- 1TB Disk
-Performance Test of Policy XACML PDP
-************************************
+ONAP was deployed using a K8s Configuration on a separate VM.
Summary
=======
jmeter -Jduration=1200 -Jusers=10 \
-Jxacml_ip=$ip -Jpap_ip=$ip -Japi_ip=$ip \
-Jxacml_port=31104 -Jpap_port=32425 -Japi_port=30709 \
- -n -t perf.jmx
-
- ./result.sh
+ -n -t perf.jmx -l testresults.jtl
Note: the ports listed above correspond to port 6969 of the respective components.
.. csv-table::
:header: "Number of Users", "Throughput (requests/second)", "Average Latency (ms)"
- 10, 6064, 4.1
- 20, 6495, 7.2
- 40, 6457, 12.2
- 80, 5803, 21.3
+ 10, 8929, 3.10
+ 20, 10827, 5.05
+ 40, 11800, 9.35
+ 80, 11750, 18.62
Stability Test of Policy XACML PDP
************************************
+The stability test was executed by performing requests
+against the Policy RESTful APIs residing on the XACML PDP installed in the windriver
+lab. This was running on a kubernetes pod having the following configuration:
+
+- 16GB RAM
+- 8 VCPU
+- 160GB Disk
+
+The test was run via jmeter, which was installed on a separate VM so-as not
+to impact the performance of the XACML-PDP being tested.
+
Summary
=======
-The Stability test was run with the same pods/VMs and uses the same jmeter script as the
-performance test, except that it was run for 72 hours instead of 20 minutes. In
-addition, it was run in the background via "nohup", to prevent it from being interrupted:
+The stability test was performed on a default ONAP OOM installation in the Intel Wind River Lab environment.
+JMeter was installed on a separate VM to inject the traffic defined in the
+`XACML PDP stability script
+<https://git.onap.org/policy/xacml-pdp/tree/testsuites/stability/src/main/resources/testplans/stability.jmx>`_
+with the following command:
.. code-block:: bash
- nohup jmeter -Jduration=259200 \
- -Jxacml_ip=$ip -Jpap_ip=$ip -Japi_ip=$ip \
- -Jxacml_port=31104 -Jpap_port=32425 -Japi_port=30709 \
- -n -t perf.jmx &
-
-The memory and CPU usage can be monitored by running "top" on the xacml pod. By taking
-a snapshot before the test is started, and again when it completes, the total CPU used
-by all of the requests can be computed.
+ jmeter.sh -Jduration=259200 -Jusers=2 -Jxacml_ip=$ip -Jpap_ip=$ip -Japi_ip=$ip \
+ -Jxacml_port=31104 -Jpap_port=32425 -Japi_port=30709 --nongui --testfile stability.jmx
-Results
-=======
+Note: the ports listed above correspond to port 6969 of the respective components.
-The final output of the jmeter script is found in the nohup.out file:
+The default log level of the root and org.eclipse.jetty.server.RequestLog loggers in the logback.xml
+of the XACML PDP
+(om/kubernetes/policy/components/policy-xacml-pdp/resources/config/logback.xml)
+was set to ERROR since the OOM installation did not have log rotation enabled of the
+container logs in the kubernetes worker nodes.
-.. image:: images/xacml-s3p-jmeter.png
+The stability test, stability.jmx, runs the following, all in parallel:
-The final memory and CPU from "top":
+- Healthcheck, 2 simultaneous threads
+- Statistics, 2 simultaneous threads
+- Decisions, 2 simultaneous threads, each running the following tasks in sequence:
+ - Monitoring Decision
+ - Monitoring Decision, abbreviated
+ - Naming Decision
+ - Optimization Decision
+ - Default Guard Decision (always "Permit")
+ - Frequency Limiter Guard Decision
+ - Min/Max Guard Decision
-.. image:: images/xacml-s3p-top.png
+When the script starts up, it uses policy-api to create, and policy-pap to deploy
+the policies that are needed by the test. It assumes that the "naming" policy has
+already been created and deployed. Once the test completes, it undeploys and deletes
+the policies that it previously created.
-The through-put reported by jmeter was 4849 requests/second, with 0 errors. In addition,
-the memory usage observed via "top" indicated that the virtual memory and resident set
-sizes remained virtually unchanged through-out the test.
+Results
+=======
-Unfortunately, the initial CPU usage was not recorded, so the CPU time reported in
-the "top" screen-shot includes XACML-PDP start-up time as well as requests that were
-executed before the stability test was started. Nevertheless, even including that, we find:
+The stability summary results were reported by JMeter with the following summary line:
.. code-block:: bash
- 13,166 CPU minutes * 60sec/min * 1000ms/sec / 1,256,834,239 requests = 0.63ms/request
+ summary = 207771010 in 72:00:01 = 801.6/s Avg: 6 Min: 0 Max: 411 Err: 0 (0.00%)
+
+The XACML PDP offered good performance with JMeter for the traffic mix described above, using 801 threads per second
+to inject the traffic load. No errors were encountered, and no significant CPU spikes were noted.
+The average transaction time was 6ms. with a maximum of 411ms.
+
Code Review / policy / parent.git / blobdiff